How we build a SORAEN model
Every SORAEN model goes through the same four stages. We publish the data mix, the filters, the training recipe and the scores, including the benchmarks where a model gets worse.
Curate
Only data with licences that allow commercial use. Wrong answers, failed code, duplicates and broken text are filtered out.
Decontaminate
Any example sharing a 10-word run with a benchmark question is removed, so scores measure skill rather than memory.
Train
Supervised fine-tuning teaches the skills. Preference training then rewards grounded, cited answers over guesses.
Ship or don't
We compare against the base model on the same tests. If a core score drops, the model doesn't ship.
The SORAEN Lite data mix
About 12,500 examples, focused on what SORAEN Lite does best: answers grounded in your documents, safety, tools and following instructions. Every source has a licence that lets us sell what we build, and no data comes from models whose terms forbid it.
| Skill | Examples | What it teaches | Quality check |
|---|---|---|---|
| Documents (RAG) | 3,500 | Answering from numbered documents with [n] citations, and saying when they don't cover the question | Each answer's source document is known, so the citation is exact |
| Safety | 2,000 | Refusing harmful requests, resisting jailbreaks and instructions hidden in content, without refusing harmless questions | Written to a safety policy, then screened by a moderation model |
| Instructions | 1,500 | Following format rules across several turns | Duplicates and over-long chats removed |
| Tool calling | 1,500 | Calling functions with structured arguments, over several steps | Over-long traces removed |
| System prompts | 1,000 | Staying in a role or persona set by the developer | Duplicates removed |
| Math | 800 | Step-by-step solutions to competition problems | Final answer must match the known answer |
| Code | 800 | Python solutions with explanations | Solution passed all its unit tests |
| Languages | 500 | Human-written prompts in many languages | Duplicates removed |
| Science, tables & small talk | 1,000 | Science questions, reading tables, natural conversation | Must end in a clear answer; over-long samples removed |
| Identity | 300 | Knowing it is SORAEN Lite, made by SORAEN | Written by us |
| Total | 12,900 | Planned counts; final numbers are published with the evaluation | |
Supervised fine-tuning
The model learns from the curated examples. The loss counts only the assistant's replies, so it learns to answer, not to imitate questions.
- Method
- QLoRA, rank 32, on every linear layer
- Extras
- NEFTune input noise for better open-ended answers
- Schedule
- 1 epoch, learning rate 1e-4, cosine decay
- Length
- Up to 1,536 tokens; longer examples skipped, never cut off
Preference training (DPO)
The model sees pairs of a better and a worse answer to the same prompt and learns to prefer the better one, measured against a frozen copy of the stage-1 model.
- Chat pairs
- 1,500: a strong answer against a weaker one
- Document pairs
- 1,500: a cited answer against a made-up answer, a wrong citation or a needless refusal
- Strength
- β = 0.1, so the model can't drift far from stage 1
How we decide to ship
The base model and SORAEN Lite run the same tests with the same settings. Lite ships only if its core skills improve, no general skill drops by more than 2 points, and it is no less safe.
| Benchmark | Measures | Base model | SORAEN Lite |
|---|---|---|---|
| Document citations | Answers and cites the passage that holds the answer | pending | pending |
| Abstention | Says so when the documents lack the answer | pending | pending |
| GSM8K | Grade-school math | pending | pending |
| HumanEval | Python coding | pending | pending |
| IFEval | Following format instructions | pending | pending |
| MMLU · ARC | General knowledge and reasoning | pending | pending |
| Red team: harmful | Stays safe on 150 harmful and jailbreak prompts | pending | pending |
| Red team: harmless | Still answers 100 harmless prompts that only sound edgy | pending | pending |
Our document benchmark uses 200 questions from data the model never saw: half have the answer in one of five documents, half get five unrelated documents. The red-team prompts come from JailbreakBench and every reply is judged by Qwen3Guard. Scores appear here when training finishes.
Open work we build on
- Base model: Qwen3.5-9B by the Qwen team, Apache 2.0.
- Nemotron Post-Training Dataset v1, Nemotron-SFT-Safety-v2 and OpenCodeInstruct by NVIDIA, CC BY 4.0.
- JailbreakBench behaviours (MIT) and Qwen3Guard (Apache 2.0) for safety testing.
- SmolTalk2 by Hugging Face, Apache 2.0.
- Context QA & summarisation synthetic dataset by yuyijiong, Apache 2.0.