The NeatContext platform engineering team has revised its approach to automation in operations, moving away from the idea of creating a fully autonomous AI-based SRE agent in favor of a managed context model.

What Happened

NeatContext engineers encountered several critical issues when attempting to implement a fully autonomous agent: information noise due to excessive telemetry, model hallucinations caused by using outdated documentation via RAG, and serious security risks such as the "confused deputy" problem when granting the agent write permissions to infrastructure. As a result, the company transitioned to a "targeted context assembly" method, which involves the manual or semi-automated assembly of isolated datasets in the form of Markdown profiles to be passed to the LLM.

Context

In modern cloud-native environments, attempts to use AI agents through uncontrolled scraping of logs and metrics often lead to overloading the model's context window. Using standard RAG systems with outdated documentation in critical infrastructure tasks triggers errors that are unacceptable for SRE/Ops processes.

Why It Matters for the Industry

This case marks a shift from the concept of full AI autonomy toward a human-in-the-loop model and a context-support approach. The industry is receiving a signal that uncontrolled data collection strategies are losing to methods of targeted delivery of verified data (context bundling), which could lead to the emergence of new product categories, such as Context Assemblers.

Why It Matters for Users

Developers and engineers implementing AI in operations should move away from the idea of granting agents full CLI access or permissions to execute critical commands (e.g., restarting pods). Instead, it is more effective to use AI as an intelligent assistant for analyzing cleaned data dumps, leaving the final decision-making authority to humans.

What Remains Unknown / Limitations

There is a noticeable difference in emphasis between technical roles, which focus on the degradation of reasoning quality, and product roles, which emphasize UX patterns and operational risks.

Sources

Author

Look at AI, Editorial Team