ToTra has been introduced—a high-performance open-source AI gateway written in Go, developed to ensure security and control when using Large Language Models within companies. The platform allows for quota management, expense tracking, and personal data protection, ensuring compliance with strict regulatory standards such as GDPR and the EU AI Act.
What Happened
The ToTra project has been released, serving as an infrastructure layer (gateway) between corporate applications and external LLMs. The Go-based solution supports an OpenAI-compatible API for easy integration and includes mechanisms for automatic PII (Personally Identifiable Information) blocking across 18 language groups at the edge level. The system also provides immutable audit logs for access control and operational transparency.
Context
With the growing use of cloud-based neural networks, companies face risks of sensitive information leaks and uncontrolled API expenditures. To operate in regulated sectors, such as fintech or medicine, a management intermediary layer is required to guarantee legal compliance (e.g., the EU AI Act) and prevent the transfer of sensitive data to third-party providers.
Why It Matters for the Industry
The emergence of specialized open-source solutions for managing LLM traffic simplifies AI adoption in the corporate sector, allowing companies to avoid vendor lock-in with proprietary tools. This contributes to the formation of a new architectural standard where an AI Gateway becomes a mandatory element of the stack to ensure security and compliance.
Why It Matters for Users
Developers gain a ready-made tool for creating a secure perimeter for LLM usage, which minimizes data leak risks and allows for easy scaling of neural network usage within an organization. Companies gain an efficient way to control API costs per individual user and team, making AI usage more predictable and economically justifiable.
Sources
Author
Look at AI, Editorial Staff