Google DeepMind introduced agentic video understanding mode in the Gemini API: instead of the previous static slicing at a fixed rate of 1 frame per second and audio, the model in an agentic loop decides for itself which video segments to watch, at what frame rate, and in which modality. According to Google, on long recordings this reduces token usage by up to 88%, lowers analysis cost by up to 66%, and improves accuracy by up to 7%. The mode is enabled with a single processing: "agentic" flag, is already available for three Flash models, and is expected to appear later in the Gemini app and in Ask YouTube.

image
image

What happened

Google DeepMind introduced agentic video understanding mode in the Gemini API. Instead of the previous static sampling at a fixed rate of 1 FPS and audio, the model in an agentic loop decides for itself which video segments to watch, at what frame rate, and in which modality — frames, audio, or transcript — and can repeat multiple analysis cycles, requesting individual timecodes and frames through an internal tool for loading file segments. The mode is enabled with the processing: "agentic" flag and is already available for Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite in Google AI Studio and Gemini Enterprise Agent Platform at no additional cost on top of standard token rates. Early partners — Ponder, Revyl, Mosaic, and Resemble.AI — are building their first products on this mode.

Context

Previously, long videos in Gemini were processed in their entirety: the model received a uniform slice of 1 frame per second plus an audio track regardless of content, so tokens were spent on frames without useful information. The new mode transfers to video the principle that Google has already applied to images under the name agentic vision: the model itself chooses where to look. Essentially, this is a change in the strategy for distributing computation at the inference stage, not a new architecture or training method, so the scientific novelty here is moderate, and the value is primarily engineering. The claimed token savings are a structural consequence of abandoning uniform sampling: selective access to the necessary segments reduces consumption almost by design, so this part of Google's figures should be accepted without a discount.

Why this matters for the industry

For teams building products on video, the unit economics change: according to Google, savings of up to 88% of tokens and up to 66% of cost make scenarios that previously did not fit within the budget profitable — automatic analysis of multi-hour lectures, calls, and camera archives. Gemini 3.7 Flash with this mode is claimed to be on the Pareto frontier in terms of the accuracy-to-cost ratio among the tested models. If Google integrates the mode into Ask YouTube, agentic video understanding will become the default way to answer questions about videos on the largest video hosting platform, competitors will have to respond with their own selective perception mechanisms, and the cheap basic layer of “understanding long videos” will be commoditized.

Why this matters for users

If you work with the Gemini API, you just need to specify the processing: "agentic" flag in the request configuration: more accurate and cheaper right now, without rewriting the pipeline, and the documentation with a code example is already available in the Google AI Studio developer guide. Users of the Gemini app and YouTube should wait a few months: Google promises to bring the mode to the app for everyone and integrate it into Ask YouTube on the viewing page, so the quality of “ask about this video” answers should noticeably improve — up to finding moments with accuracy to fractions of a second and counting actions in the frame.

What is still unknown / limitations

The key figures — savings of up to 88% of tokens, up to 66% of cost, and an accuracy increase of “up to 7%” — are claimed by Google itself and have not yet undergone independent verification; the wording “up to” indicates the best case, and the names of benchmarks, baselines, and sample sizes have not been disclosed. Latency metrics have not been published, and an agentic loop with multiple iterations may take more time than a single pass through the file. The comparative set of models for the Pareto evaluation was chosen by Google itself, and there is no independent comparison. The token savings look the most reliable, and the claimed accuracy increase is better to check independently — with an A/B test on your own recordings against the previous static sampling.

Sources

Author

Look at AI, editorial team