SGLang Teaches Chat Models to Answer with Probabilities

The open-source inference engine SGLang has introduced /v1/decisions and /v1/systemone endpoints (PR #40992 and #41208, September 24–25, 2026): a standard chat model now responds with probabilities—choice from options, scale rating, and “yes/no”—calculated from next-token log-probabilities.

Compiled by Sergey KostenchukPublished 2026-10-02Updated 2026-10-02

2026-10-02 Coding

🛠 🌟 SGLang Teaches Chat Models to Answer with Probabilities

A chat model in the open-source SGLang engine responds not with text, but with probabilities: the /v1/decisions and /v1/systemone endpoints provide choice from options, scale rating, and “yes/no.” Before release, the feature is available in nightly builds; in the demo, Qwen3.8-27B chose actions in Pokémon FireRed (less than 100 ms).

🌍 SGLang makes probabilistic classification a native server endpoint alongside /v1/score and /v1/rerank; System One API compatibility—an open alternative to proprietary Jev.

👤 Any chat model on SGLang can be used as a classifier (triage, routing, guardrails) without a separate reward checkpoint; TypeSafe SDK clients connect by changing base_url. Probabilities are not calibrated: drift up to 0.07, thresholds need to be validated on your own data.

Source 1: https://x.com/sgl_project/status/2105095379272515603 Source 2: https://docs.sglang.io/docs/supported-models/decision_models

Sources