🛠 FlexInference: A free LLM router supporting OpenAI, Gemini, and Anthropic
FlexInference has been released—a system implementing the concept of deadline-aware routing via the start_within parameter. This allows for automatic switching between standard and cheaper (flex/batch) provider tariffs without changing the request structure.
🌍 The tool optimizes inference costs, working as a drop-in replacement without requiring changes to application logic or the models themselves.
👤 Reduces API costs by up to 50% for background tasks.
Source 1: https://docs.flexinference.com/ Source 2: https://pypi.org/project/flexinference/