Qwen3.5-4B Fine-Tuned as an NLI Verifier

Qwen3.5-4B Fine-Tuned as an NLI Verifier — openjev

Alexander Wortega (AlexWortega) has released openjev on Hugging Face: Qwen3.5-4B fine-tuned as an NLI cross-encoder that verifies claims instead of generating answers, achieving 0.974 on MMLU and 0.996 on GSM8K with a reference.

Compiled by Sergey KostenchukPublished 2026-09-16Updated 2026-09-18

2026-09-16 Research HuggingFace

🤖 Qwen3.5-4B Fine-Tuned as an NLI Verifier — openjev

Alexander Wortega (AlexWortega) has released openjev on Hugging Face: Qwen3.5-4B as an NLI cross-encoder. Input — a "premise — hypothesis" pair, output — probabilities for entailment, contradiction, and neutral. Grading with a reference: 0.974 MMLU, 0.996 GSM8K; reranking without a reference — 0.472; in Flappy Bird — 28 out of 28 pipes.

🌍 In reranker and moderation pipelines, expensive LLM calls can be replaced with a 4B cross-encoder: one forward pass per pair, ~57 ms. But grading is easier than generation — high numbers do not mean the model "knows" the answers; guardrails are currently a hypothesis without measurements.

👤 The weights are open under the MIT license (inheriting Qwen3.5 terms — check separately for commercial use). The repo includes modeling_openjev.py with predict/rerank/grade: reranking and grading can be verified on your own data with a couple of lines.

Source 1: https://huggingface.co/AlexWortega/openjev Source 2: https://huggingface.co/AlexWortega/openjev/raw/main/results/full_report.md

Sources