A
Abstraction
Transform raw sensory streams into structured entities, attributes, relations, and topological organization.
🧠 Survey
Spatial Intelligence
Spatial Intelligence
from a Cognitive Map Perspective
A mechanism-centric survey that revisits spatial intelligence through the lens of cognitive maps: how internal spatial representations are constructed, maintained, reasoned over, and realized.
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Beihang University · Xi’an Jiaotong University · Nanyang Technological University
* Equal contribution · † Project leader · ✉ Corresponding author
Abstract
Spatial intelligence requires agents to form and utilize internal representations of the physical world for perception, reasoning, and generation. While recent advances in foundation models, embodied systems, and three-dimensional representation learning have substantially expanded spatial capabilities, existing research remains fragmented across heterogeneous tasks and model paradigms. This survey revisits spatial intelligence from a cognitive map perspective and positions cognitive maps as its representational blueprint. In this view, diverse lines of research can be understood through a shared question: how an internal spatial representation is constructed, maintained, reasoned over, and realized. To make this perspective operational, we define cognitive maps as internal spatial representations characterized by abstraction, globality, and persistency. Based on this definition, we organize the literature into three cognitive-map-centric processes that correspond to the core dimensions of spatial intelligence: perception for cognitive map construction, reasoning for internal inference with the map, and generation for external realization of the map. By adopting a mechanism-centric viewpoint, this survey connects previously isolated research directions into a coherent framework and identifies emerging challenges toward unified spatial intelligence systems.
We position cognitive maps as the representational blueprint of spatial intelligence. Instead of treating perception, reasoning, and generation as isolated task families, the survey reorganizes them as three cognitive-map-centric processes: construction, inference, and realization. This perspective connects metric-semantic mapping, scene graphs, spatial memory systems, and structured world models under a shared question: how should an agent build, update, query, and instantiate an internal model of space?
G
Globality
Integrate partial observations across viewpoints and time into a cross-view consistent spatial layout.
P
Persistency
Maintain and update spatial representations through memory rather than reconstructing the world from scratch.
Definition of cognitive map: abstraction, globality, and persistency.
Conceptual Framework
Construct, infer, and realize cognitive maps.
The survey reframes spatial intelligence as an operational loop centered on an internal spatial representation.
Cognitive Map as the architectural blueprint
A cognitive map is not merely a storage layer. It specifies the operating mode of a spatially intelligent system: abstract observations, organize them globally, maintain them persistently, and reuse them for reasoning and generation.
01
Perception: Construction
Build cognitive maps from RGB, RGB-D, video, LiDAR, point clouds, and multimodal observations.
02
Reasoning: Inference
Read and manipulate maps as embeddings, prompts, or APIs for grounding, planning, and decision-making.
03
Generation: Realization
Use maps as structural priors for static scene synthesis and dynamic world simulation.
Taxonomy
A cognitive-map-centric organization of the literature.
The taxonomy follows the lifecycle of internal spatial representations, from construction to reasoning and external realization.
Perception
Metric RepresentationExplicit geometry · parametric coordinates
Relational RepresentationStructured graphs · serialized graphs
Hybrid RepresentationHierarchical architectures · feature fusion
Reasoning
Map as EmbeddingStructural state propagation · latent feature matching
Map as PromptTextual · visual · multimodal prompting
Map as APIReal-time state snapshots · persistent spatial memory
Generation
Static Scene SynthesisMap-based retrieval · map-to-scene generation
Dynamic World Simulation
Overall structure and taxonomy of the survey.
Perception
Construction of cognitive maps.
Perception constructs unified internal spatial representations from local and fragmented sensor observations, including metric, relational, and hybrid cognitive maps.
Representation paradigms
Metric representations provide geometric grounding, relational representations organize topological and semantic dependencies, and hybrid representations combine both levels for richer spatial understanding.
Table 1. Overview of different representations.
Reasoning
Inference with cognitive maps.
Reasoning uses cognitive maps as embeddings, prompts, or callable APIs to retrieve spatial knowledge, ground language and perception, and support planning or decision-making.
Reasoning paradigms
Map-as-embedding emphasizes compact latent states, map-as-prompt connects cognitive maps with foundation models, and map-as-API supports controlled, stateful, closed-loop spatial reasoning.
Table 2. Reasoning paradigms for inference with cognitive maps.
Generation
Realization of cognitive maps.
Generation transforms internal cognitive maps into external spatial forms, including static 3D scene synthesis and dynamic world simulation.
From map to world
Cognitive maps can serve as structural priors, layout constraints, retrieval plans, or state memories for synthesizing scenes and maintaining temporal consistency in simulated worlds.
Table 3. Generation methods for realization of cognitive maps.
Applications
From open-loop spatial cognition to closed-loop interaction.
Open-Loop Spatial Cognition
Cognitive maps provide an internal substrate for evaluating and generating spatial knowledge without direct action execution.
▸ Spatial question answering
▸ Indoor scene synthesis
▸ Open-ended world generation
Closed-Loop Spatial Interaction
When agents interact with environments, maps become stateful memories that support navigation, manipulation, and collaboration.
▸ Embodied navigation
▸ Embodied manipulation
Applications of cognitive-map-centered spatial intelligence.
Future Directions
Toward unified spatial intelligence systems.
The cognitive map perspective exposes several bottlenecks that future spatial intelligence systems need to address.
Citation
Cite this survey.
@article{tian2026spatial,
title={Spatial Intelligence from a Cognitive Map Perspective: A Survey},
author={Tian, Yuxuan and Ji, Yuheng and Zheng, Xiaolong and Qin, Ziheng and Wang, Yipu and Zheng, Xinyi and Liu, Yuyang and Bai, Shuanghao and Li, Zhe and Wang, Liang and others},
year={2026},
publisher={Preprints}
}