🤖 Google Research has published AgentHands: an XR agent with hands that speak in rhythm
This is a prototype for Android XR where a conversational AI agent accompanies its speech with synchronized animated hands: it points to real objects in the room, “shows” movements, and signals danger (a red glow near the hot nozzle of a 3D printer).
🌍 This is a step from “flat” hints on a phone screen to spatial agents in AR glasses: the LLM generates GestureEvents — gestures tied to specific words — and the device synchronizes them with TTS by word timestamps. The work is presented at CHI 2026.
👤 The agent no longer just says “over there” — it shows. In a test with 12 participants, gestures significantly (p < 0.05) facilitated object search and understanding of instructions better than voice alone.
Source 1: https://research.google/blog/agenthands-generating-interactive-hand-gestures-for-spatially-grounded-agent-conversations-in-xr/