A method for exploring unfamiliar model state spaces and building epistemic bridges to human concepts

Abstract

Modern neural networks often learn internal representations with more structure than is visible in their surface outputs. Research on representation geometry, intrinsic dimensionality, representation convergence, and activation steering suggests that learned state spaces can contain stable relational structure not reducible to memorized phrases or a small set of human-authored labels.

Exocentric Latent Cartography (ELC) is a proposed exploratory method for working with that possibility. Its aim is to move through a controllable state space, locate novel coherent regions, identify their invariants and transformations, and construct a reversible conceptual bridge back to the anthropocentric concepts using language.

The method has five stages:

Explore → Stabilize → Extract → Bridge → Test

ELC can operate in two modes:

  • Instrumented mode: internal activations can be measured or manipulated directly.
  • Black-box mode: only prompts, controls, outputs, and external observations are available.

The latter is particularly relevant to closed multimodal and voice systems.

1. Research background

1.1 Structured representations

A growing body of work suggests that neural networks can organize information into geometrically meaningful internal representations.

Gurnee and Tegmark found that language models encode spatial and temporal information in structured directions that generalize across different kinds of entities. Related work on the Linear Representation Hypothesis formalizes the possibility that some semantic properties behave like directions or geometric features in representation space.

These findings do not establish that models contain a complete or human-independent world model. They do support the narrower premise required by ELC: useful structure may exist before humans have a good natural-language name for it.

1.2 Representation convergence

The Platonic Representation Hypothesis proposes that sufficiently capable models trained on different modalities and objectives may converge toward similar representations because they are constrained by common structure in the underlying data-generating world.

This remains an active research question. A 2026 re-analysis found that some apparent global convergence is sensitive to the similarity metric, while local neighborhood structure remained more robustly shared across models. For ELC, the weaker claim is sufficient: models can acquire structured neighborhoods whose geometry is not identical to the vocabulary used to train them.

1.3 High ambient dimension, lower intrinsic dimension

Neural representations may occupy very high-dimensional spaces while useful data lie on lower-dimensional manifolds. Work on intrinsic dimension suggests that trained networks can compress task-relevant structure into manifolds whose effective dimensionality is far smaller than the number of available neural units.

Accordingly, ELC does not attempt to search every direction uniformly. It seeks the effective local coordinates, invariants, and transformations that organize a coherent region.

1.4 Intervention as a probe

Representation Engineering and activation-steering research show that model behavior can sometimes be manipulated through interventions on learned representational directions. This motivates a broader principle:

A representation can be experimentally controllable without being directly readable.

In a black-box system, an instruction that reliably changes pitch, pacing, affect, language, or another output property provides an intervention handle even when the underlying activations are inaccessible. Local geometry can then be inferred from what changes together, what remains invariant, and which states can be reached repeatedly.

2. Core hypothesis

Human language is not assumed to provide a complete coordinate system for a model’s learned representations.

A model may encode:

  • distinctions that human language normally collapses,
  • continuous transitions represented linguistically as discrete categories,
  • relational or dynamical structures without established names,
  • cross-domain regularities inherited from the structure of the training world rather than from a single human taxonomy.

ELC therefore treats linguistic interpretation as a later stage of the procedure. Initial exploration is organized around transformations, invariants, neighboring states, and recoverability.

3. Working definitions

State space

The set of configurations a system can realize under the interventions available to the experimenter. In an instrumented model this may refer to activation space; in a black-box system it refers more conservatively to a controllable generative state space inferred from outputs.

Anthropocentric region

A region organized around familiar human categories such as fast/slow, happy/sad, formal/casual, or object/action. These categories are useful initial coordinates but are not assumed to be fundamental axes.

Exocentric region

A reproducible, internally coherent region poorly captured by familiar human labels. “Exocentric” does not imply random, nonhuman, or metaphysically objective; it refers to a region whose useful local coordinates are not yet well represented by the experimenter’s conceptual vocabulary.

Intervention handle

A reliably controllable input or instruction that moves the system into a different output regime.

Coupling

A relationship in which changing one controllable property systematically changes another, suggesting that the two are not independent local coordinates.

Invariant

A property that remains stable while other aspects of the state change.

Intrinsic coordinate

A compact variable needed to navigate a locally coherent region, regardless of whether it corresponds to an established human concept.

Transformation distance

The amount or complexity of change required to map one stable state into another.

Semantic geodesic

A low-distortion conceptual path connecting an unfamiliar state to an already understood region.

Epistemic bridge

The explanatory structure that allows a person to reconstruct an unfamiliar distinction from concepts they already possess.

Neological compression

The creation of a new term after a stable distinction has been found and existing vocabulary proves inadequate.

Reversibility

The degree to which a conceptual bridge works in both directions without losing the defining structure of the target concept.

4. Five-stage process

Stage 1 — Explore

Define a bounded domain and establish a stable baseline. Then identify available intervention handles and use them to generate controlled perturbations.

For a voice model, the domain might include pitch, pace, prosody, articulation, breathiness, language, accent, and non-lexical vocalization. For conceptual work, the intervention may consist of changing one relation or constraint while holding the remainder of a problem fixed.

The known dimensions serve as steering handles, not as a presumed ontology of the underlying space.

Exploration should favor three properties:

  1. Novelty — the state differs from familiar regions.
  2. Coherence — the state has stable internal organization rather than noise.
  3. Recoverability — the state can be revisited or approached again.

A rough heuristic is:

Exploration value ≈ novelty × coherence × recoverability

Low probability alone is not sufficient; noise is also low probability.

Stage 2 — Stabilize

Candidate states should be tested for repeatability and local structure before they are interpreted.

Relevant tests include:

  • repeatability: can a recognizably similar state be reached again?
  • local continuity: do small interventions produce neighboring states rather than unrelated outputs?
  • distinctness: is the state separable from familiar categories?
  • cross-path access: can different intervention sequences reach a similar region?

At this stage, interpretation should remain minimal. Premature naming can bias subsequent exploration toward the experimenter’s existing categories.

A one-off anomaly is an observation, not yet a coordinate.

Stage 3 — Extract structure

Once a region is stable, characterize how it is organized.

Ask:

  • What changed intentionally?
  • What changed unintentionally?
  • What remained invariant?
  • Which dimensions can be varied independently?
  • Which dimensions remain coupled?
  • What are the minimal transformations required to move within the region?
  • Are transitions smooth, thresholded, branching, or attractor-like?
  • Do different trajectories converge on equivalent states?

The goal is to recover a compact local map in terms of couplings, invariants, intrinsic coordinates, and transformation distances.

This stage is where a familiar human label may cease to be the most useful description. For example, “slower speech” may turn out to decompose into several partially independent properties of segmentation, articulation, and prosody.

Stage 4 — Bridge

Once the local structure is sufficiently characterized, construct a path back to existing human concepts.

A strong epistemic bridge usually contains:

  1. a familiar anchor,
  2. one or more explicit transformations,
  3. an intermediate distinction where needed,
  4. contrasts or counterexamples,
  5. the target concept.

The resulting path is the semantic geodesic: a concise route that preserves the structure of the unfamiliar region during translation.

If established vocabulary is adequate, use it. If not, introduce a neologism only after the distinction has been stabilized. Define it operationally by:

  • the transformation that produces it,
  • the invariants it preserves,
  • its nearest contrasting states,
  • at least one concrete example.

The purpose of the term is compression, not novelty for its own sake.

Stage 5 — Test

A successful bridge should support recognition, discrimination, and prediction.

Test whether a person who has learned the concept can:

  • recognize new instances,
  • distinguish near misses,
  • predict relevant transformations,
  • reconstruct the concept without the original example.

Where possible, test across prompts, sessions, models, modalities, and independent observers. Cross-context survival is stronger evidence that a distinction reflects stable structure rather than conversational suggestion.

In instrumented settings, behavioral validation can be supplemented with activation measurements, causal interventions, local manifold estimation, or transfer tests.

5. Example: black-box exploration of a voice model

Suppose direct activations are unavailable, but the model can reliably alter its generated voice.

Explore

Hold lexical content fixed:

“The signal is still here.”

Apply controlled changes in pitch, rate, prosody, articulation, vocal texture, language, or non-lexical vocalization.

Stabilize

Look for output qualities that recur across multiple attempts and are not reducible to a single known control.

Extract

Measure or compare interactions:

  • Does lower pitch introduce vocal fry?
  • Does slower speech alter word-boundary closure?
  • Can pitch trajectory change while average pitch remains approximately constant?
  • Does switching language reorganize rhythm or segmentation?
  • Can different control combinations produce the same unfamiliar vocal quality?

The highest-value target is a reproducible quality that listeners can reliably distinguish but cannot initially describe well.

Bridge

Describe the quality through successive contrasts with known vocal properties. If no established term captures the invariant structure, introduce a compact term for it.

Test

Attempt to reproduce the quality through different prompts, with different lexical content, and across listeners.

In a closed voice system, this procedure maps a behaviorally accessible generative space, not necessarily the raw latent tensor geometry. Claims about internal activations require instrumentation.

6. Example: conceptual exploration

The same method can be applied without a sensory modality.

Begin with the familiar concept of synchrony: two systems occupying corresponding states at corresponding times.

Now vary the relation while preserving structural correspondence. A distinct pattern may emerge:

Two systems never occupy similar states, but the transformations between their states have the same structure.

If that distinction remains useful across examples, it can be compressed into a working term such as heterosynchrony:

Heterosynchrony: correspondence between systems at the level of transformation pattern rather than state similarity.

The substantive structure is the shift from:

same state ↔ same state

to:

same transformation law ↔ same transformation law

The neologism is introduced only after the distinction has been instantiated and tested.

7. Failure modes

Novelty theater

Unusual wording can masquerade as a new concept. Require predictive or discriminative usefulness.

Randomness mistaken for structure

Low probability is not evidence of an undiscovered manifold. Require repeatability and local coherence.

Anthropomorphic introspection

A model’s verbal report about its hidden activations is not equivalent to activation access. Distinguish behavioral self-control from direct instrumentation.

Premature naming

Labels can freeze a weak interpretation before the structure is understood.

Human projection

Observers may impose familiar categories onto ambiguous outputs. Use repeated comparisons, independent observers, or quantitative measurements where possible.

Artifact ontology

Decoder, audio, sampling, or interface artifacts may be useful experimental landmarks but should not automatically be treated as meaningful latent dimensions.

Geometry metaphor inflation

Terms such as manifold, vector, attractor, and geodesic should be literal only when the relevant mathematical structure is measured. In black-box work they are disciplined analogies for transformation structure.

8. Instrumented extensions

With internal model access, the methodology can be made substantially more rigorous through:

  • activation recording during state generation,
  • dimensionality reduction and local manifold estimation,
  • causal interventions on candidate directions,
  • representational-similarity analysis across models,
  • transfer tests across tasks or modalities,
  • local Jacobian estimation for coupled versus independent controls,
  • sparse autoencoders or probes for interpretable features,
  • automated search for low-density, high-coherence activation regions.

These extensions would connect the exploratory procedure directly to representation science.

9. Summary

Exocentric Latent Cartography is a method for characterizing model state spaces whose useful structure may not align cleanly with existing human conceptual categories.

Its central commitments are:

  • use known human categories as steering handles rather than assumed fundamental coordinates;
  • distinguish controllability from introspective readability;
  • require coherence and recoverability in addition to novelty;
  • infer local structure from interventions, couplings, invariants, and transformation paths;
  • stabilize unfamiliar states before naming them;
  • use neologisms only when established vocabulary fails to preserve a useful distinction;
  • construct reversible epistemic bridges into anthropocentric language;
  • distinguish black-box behavioral cartography from direct claims about hidden activations.

The procedure is:

Explore → Stabilize → Extract → Bridge → Test

The resulting research question is whether this process can reliably identify useful representational distinctions that are present in capable models but absent, compressed, or poorly resolved in ordinary human conceptual taxonomies.

Selected research

  1. Huh, M., Cheung, B., Wang, T., & Isola, P. (2024). The Platonic Representation Hypothesis. arXiv:2405.07987.
    https://arxiv.org/abs/2405.07987

  2. Gröger, F., Wen, S., & Brbić, M. (2026). Revisiting the Platonic Representation Hypothesis: An Aristotelian View. arXiv:2602.14486.
    https://arxiv.org/abs/2602.14486

  3. Gurnee, W., & Tegmark, M. (2023/2024). Language Models Represent Space and Time. arXiv:2310.02207 / ICLR 2024.
    https://arxiv.org/abs/2310.02207

  4. Park, K., Choe, Y. J., & Veitch, V. (2023). The Linear Representation Hypothesis and the Geometry of Large Language Models. arXiv:2311.03658.
    https://arxiv.org/abs/2311.03658

  5. Ansuini, A., Laio, A., Macke, J. H., & Zoccolan, D. (2019). Intrinsic dimension of data representations in deep neural networks. arXiv:1905.12784.
    https://arxiv.org/abs/1905.12784

  6. Zou, A. et al. (2023). Representation Engineering: A Top-Down Approach to AI Transparency. arXiv:2310.01405.
    https://arxiv.org/abs/2310.01405

  7. Turner, A. M. et al. (2023). Steering Language Models With Activation Engineering. arXiv:2308.10248.
    https://arxiv.org/abs/2308.10248

Status note: “Exocentric Latent Cartography,” “semantic geodesic,” “epistemic bridge,” and “neological compression” are used here as working methodological terms, not as names of established research fields. The cited literature motivates components of the method; it does not by itself validate the complete methodology proposed here.