OpenSmith has been introduced—a new privacy-focused open-source tool that allows for local tracing of LLM pipeline execution, eliminating the need to send data to the cloud.
What Happened
Developers have introduced OpenSmith, a tool for local monitoring and tracing of model execution steps. The project supports the use of local SQLite or Postgres databases, allowing users to log data and analyze query costs without using external cloud services. For integration, it provides Python @trace decorators, context managers, and an autopatch mechanism for automatic support of popular backends, including OpenAI, Anthropic, and LiteLLM.
Context
In modern AI application development, standard monitoring tools like LangSmith often require sending data to the cloud, which creates security and privacy risks. OpenSmith positions itself as a local-first observability solution, offering a decentralized approach to debugging complex call chains and RAG systems.
Why It Matters for the Industry
The emergence of local-first observability tools reduces the risk of sensitive data leaks when debugging agents and RAG systems, allowing for compliance with strict regulatory requirements. This could lead to a decentralization of the observability market and force major cloud players to rethink their monetization models in favor of more flexible hybrid solutions.
Why It Matters for Users
AI application developers gain a way to visualize the entire model execution process directly on their own machines, helping to save on cloud monitoring platform subscriptions and ensuring the secure debugging of sensitive prompts within closed environments.
Sources
Author
Look at AI, Editorial Team