S-Agent 2.0 is an open spatial reasoning agent assembled from the ready-made OpenHands coding agent and a console-based set of perception tools, without training new models: the entire perception stack inside is frozen. The key idea is to "close the loop of evidence": the agent measures the 3D scene using tools and relies on measured numbers rather than visual intuition. With the Gemini-3.8-Flash backend, the system scores 74.1% on MMSI-Bench and 90.5% on ReVSI, outperforming the specialized SpatialClaw and GPT-6, and the code is published under the Apache 2.0 license.

image
image

What happened

The project was publicly released in three formats simultaneously: an article on arXiv (2610.28354), a project page, and the github.com/Ropedia/S-Agent-2.0 repository under the Apache 2.0 license. The agent is built on top of OpenHands: spatial operations are performed via console commands recon, segment, distance, compass, and bev, while vision is provided by frozen models Depth Anything 3, SAM 3, GeoCalib, and Pi3-MOS — no new weights are trained. The result of each tool call is saved in persistent scene memory with the object's position, size, pose, and name, and the agent must interpret the measurement in spatial language, specifying the coordinate system and units of measurement. In a systematic comparison with the Gemini-3.8-Flash backend, the results were 74.1% on MMSI-Bench and 90.5% on ReVSI, compared to 67.0% and 79.7% for the specialized SpatialClaw and 71.0% and 84.2% for GPT-6.

Context

The problem the project addresses is familiar to anyone working with VLM agents: the model sees a correct measurement but ignores it in the response, confuses coordinate systems, and "guesses" based on the image. S-Agent 2.0 views this as an interface error rather than a model error and tests the hypothesis in a pure form: no component of the stack was trained for the task, the coding agent is taken ready-made, so the entire quality improvement is explained by the protocol of "tool measurement, scene memory, mandatory spatial interpretation." This framing turns spatial reasoning into an engineering rather than a research task: superiority is achieved by assembling publicly available parts, not by a new architecture.

Why this matters for the industry

For the industry, this work implies that a separate specialized spatial controller is not mandatory: spatial intelligence can be embedded into an existing agentic pipeline through engineering assembly, without a proprietary model or ML team. The barrier to entry is lowered to the level of CLI perception tools and persistent scene memory on top of a familiar coding agent. The stack is modular and updated in parts: frozen perception models, the command set, and the LLM backend are independent of each other, so products based on OpenHands or similar frameworks can treat it as a standard library alongside search and vector databases. If independent reproduction confirms the gap, it is expected that the "mandatory reference to measurement" protocol will be copied into agentic frameworks, leading to the emergence of ready-made spatial perception toolkits, and subsequently vertical products such as construction site photo documentation with metric reports, inventory with dimensions, and inspections with natural language responses.

Why this matters for users

For engineers and researchers, the project is valuable because it can be reproduced immediately: the Apache 2.0 repository can be cloned, no fine-tuning is required, and only a CLI wrapper and access to the backend API are needed, while the MMSI-Bench and ReVSI benchmarks are reproducible. A realistic first step is to run the system on your own scenes and collect your own quality and tool call count metrics, rather than relying on tabular percentages. A minimal experiment fits into one evening: connect the recon, segment, distance, compass, and bev commands to your agent and include in the prompt a requirement to reference measured numbers instead of guessing from the image. A separate value is the ability to dissect how the mandatory spatial language for interpreting measurements is structured and adapt this protocol for your own tasks in robotics or when working with VLMs.

What is currently unknown / limitations

The claimed results are confirmed only for the Gemini-3.8-Flash backend; there is no data for other backends. The comparison is systematic: the contribution of the measurement loop and the contribution of the LLM itself to the gap over competitors are not separated, and ablations regarding the number of calls and latency have not been published. Winning on two benchmarks without confidence intervals and a breakdown by task type does not prove a systematic loss for specialization. There is no data on the latency and cost of each agent step, so planning production use based on these figures is premature. The authors themselves name a limitation: reconstruction errors and a mispositioned observer are carried over into the response, meaning the system's accuracy is limited from below by the geometry of the perception pipeline, and in complex scenes, perception rather than reasoning will be the main source of errors.

Sources

Author

Look at AI, editorial team