Yuhong Liu*
·
Shenxi Wu*
·
Junyi Chi
.
Beichen Zhang
Xuanlang Dai
·
Jiaqi Wang†
.
Yuhang Zang†
·
Dahua Lin†
*Equal contribution. †Corresponding authors.
📖Paper | 🏠Homepage | 🤗SpatialASM Collection | 🤗SpatialASM-128k Dataset | 🤗SpatialASM-NVG Model | 📰Daily Paper
- 🚀 [2026/10/12] We have released the SpatialASM 📖Arxiv paper and its repository.
- 🚀 [2025/10/12] We have released our allocentric novel-view generator 🤗SpatialASM-NVG.
- 🚀 [2025/10/12] We have released 🤗SpatialASM-edit140k, an image-text-to-image dataset for tuning novel-view generators, and two spatial-VQA datasets with high-quality CoT responses: 🤗SpatialASM-128k and its subset 🤗SpatialASM-66k.
Humans rarely solve difficult spatial problems by passively inspecting the views in front of them. Instead, we revisit a scene from task-relevant viewpoints and mentally reconstruct what should be visible there. We call this capability Active Spatial Mentalization (ASM).
We are thrilled to introduce SpatialASM, a scalable framework that internalizes ASM into LVLMs. Through the two-stage training on LVLMs, SpatialASM elicits active spatial exploration and imagination and enhances the overall spatial reasoning performance.
SpatialASM consists of three complementary components:
- Allocentric Novel-View Generator. SpatialASM-NVG renders the scene from a target viewpoint specified by an anchor image, a pixel location, and a viewing direction. This allocentric formulation avoids lengthy sequences of camera-centric movements.
- ASM Trajectory Synthesis. Given a spatial QA sample, an LVLM plans informative viewpoints and the NVG renders the corresponding observations. After GeoP and answer-consistency filtering, rendered views are replaced by textual captions to form explicit plan-imagine-reason trajectories for SFT.
- View-Imagination Reinforcement Learning. Geometry-filtered candidate views are used to construct automatically verifiable perspective-taking tasks. GRPO further strengthens the association between a viewpoint description and its expected observation.
- 🔥 Active spatial reasoning: SpatialASM jointly internalizes task-conditioned view planning and viewpoint-conditioned view imagination, enabling LVLMs to spontaneously seek and mentally construct missing spatial evidence.
- 🔥 Scalable spatial CoT supervision: SpatialASM converts existing answer-only spatial QA data into explicit plan-imagine-reason trajectories and automatically verifiable perspective-taking tasks, alleviating the scarcity of spatial Chain-of-Thought supervision.
- 🔥 Training-time spatial simulation: A novel-view generator acts as temporary training scaffolding. The final model does not depend on a world model, renderer, or additional visual observations at inference time.
- 🔥 Grounded and reliable supervision: An allocentric viewpoint interface reaches a target perspective in one query, while the Geometry Perturbation (GeoP) metric and answer-consistency checks filter unreliable synthesized views and trajectories.
We train four LVLM backbones of Qwen3-VL and Qwen3.5 with our SpatialASM framework, and the experimental results on both spatial reasoning and general-VQA tasks are shown below.
Here we provide some simple inference code for you to try our models. The models include the LVLMs such as 🤗SpatialASM-Qwen3-VL-4B (see 📢News above), and the novel-view generator 🤗SpatialASM-NVG.
To use the LVLMs, you can either start a vLLM server or load it in your device with Transformers.
# Start a vLLM server for fine-tuned LVLMs.
git clone https://github.com/InternLM/SpatialASM.git
cd SpatialASM
conda create -n spatialasm_vllm python==3.11
conda activate spatialasm_vllm
pip install -r Inference/vllm_server_requirements.txt # [spatialasm_vllm] Environment Preparation
bash Inference/vllm_server.sh \
"vllm--spatialasm-qwen3.5-9b" \
"YuhongLiu/SpatialASM-Qwen3.5-9B" \ # Your local path
"23333" \
"0"
# SpatialASM-NVG Inference
conda create -n NVG python==3.11
conda activate NVG
cd DiffSynth-Studio && pip install -e . # [NVG] Environment Preparation
cd ..
bash Inference/edit_server.sh
python3 Inference/check_NVG.py # Change the input image paths in check_NVG.py first.First download FLUX.2-klein-9B and SpatialASM-edit140k. Then convert the dataset to the format in training_samples.jsonl, and change paths in NVG-training.sh to your local path. And you can start training!
conda activate NVG # First prepare the environment NVG. (see `⭐️Quick Start')
cd DiffSynth-Studio
bash training-scripts/NVG-training.sh- 🔥 The training code for LVLMs will be coming soon.
Prepare the environment:
conda create -n evaluation python==3.10
conda activate evaluation
cd Evaluation/VLMEvalKit
pip install -e .
# Recommended
pip install flash-attn --no-build-isolationAll benchmarks except ESI-Bench are supported by VLMEvalkit. We use its official repository.
# Single GPU
python3 run.py --data <dataset_name> --model SpatialASM-Qwen3-VL-8B --verbose
# Multi-GPU (Distributed)
torchrun --nproc-per-node=8 run.py --data <dataset_name> --model SpatialASM-Qwen3-VL-8B --verbose
# For SpatialASM-Qwen3.5-9B, it's recommended to first start a vLLM server for evaluation
bash qwen3_5_api.sh- 🔥 The evaluation code for ESI-passive will be released soon.
If you find this project useful, please kindly cite:
@article{liu2026spatialasm,
title={SpatialASM: Internalizing Active Mentalization for Spatial Reasoning},
author={Yuhong Liu, Shenxi Wu, Junyi Chi, Beichen Zhang, Xuanlang Dai, Jiaqi Wang, Yuhang Zang, Dahua Lin},
journal={arXiv preprint arXiv:2610.13832},
year={2026}
}
Use of SpatialASM-NVG: The model SpatialASM-NVG is trained on FLUX.2-klein-9B, so please follow FLUX-NON-COMMERCIAL LICENSE.
Usage and License Notices: The data and code are intended and licensed for research use only.
The evaluation part is built on VLMEvalkit, a powerful toolkit to evaluate a vast range of LMMs!
The NVG training part is built on DiffSynth-Studio, an excellent codebase for Diffusion model training!