llmproxy Released — A Python-based Proxy Server for Working with LLMs

🛠 llmproxy has been released — a lightweight, high-performance proxy server built on Python (Flask) for working with LLMs.

The tool emulates the APIs of popular local environments (Ollama, OpenAI v1, llama.cpp) by redirecting requests to NVIDIA's cloud API. It supports chat, embeddings, and streaming.

🌍 Such proxies simplify hybrid architectures, allowing developers to use powerful cloud GPUs via NVIDIA while maintaining the familiar interface of local tools.

👤 If you use Open WebUI, Ollama, or the OpenAI SDK, you can switch to high-performance NVIDIA models simply by changing the server address in your settings.

Source 1: https://github.com/lordraw77/llmproxy

Sources