--- title: KnowLine-31B-Gen1 canonical_url: "https://www.modelscope.ai/models/EMLab/KnowLine-31B-Gen1" md_url: "https://www.modelscope.ai/models/EMLab/KnowLine-31B-Gen1.md" repository: EMLab/KnowLine-31B-Gen1 last_updated: 2026-11-13 license: apache-2.0 pipeline_tag: text-classification tasks: - text-classification model_type: - gemma4 architectures: - Gemma4ForConditionalGeneration base_model: - google/gemma-4-31B-it base_model_relation: finetune parameters: 31.3B tensor_type: - BF16 library_name: - safetensors - pytorch frameworks: - pytorch language: - en - zh downloads: 15 stars: 0 tags: - decision-model - system-one - decision-index - lora-merged --- # KnowLine-31B-Gen1 > KnowLine-31B-Gen1 - An open-source model by EMLab on ModelScope. The first release of PelaAI's System One decision model on a dense Gemma 4 base: 31B parameters, trained in the same loop as the KnowLine-26B-A4B, KnowLine-35B-A3B and KnowLine-4B lines, and… EMLab/KnowLine-31B-Gen1 is a 31.3B-parameter text-classification model on ModelScope. licensed under apache-2.0. derived from google/gemma-4-31B-it. - **Repository**: EMLab/KnowLine-31B-Gen1 - **License**: apache-2.0 - **Tasks**: text-classification - **Parameters**: 31.3B - **Base model**: google/gemma-4-31B-it - **Tags**: decision-model, system-one, decision-index, lora-merged - **Downloads**: 15 - **Stars**: 0 - **Last updated**: 2026-11-13 Source: https://www.modelscope.ai/models/EMLab/KnowLine-31B-Gen1 --- # KnowLine-31B-Gen1 **The first release of PelaAI's System One decision model on a dense Gemma 4 base: 31B parameters, trained in the same loop as the KnowLine-26B-A4B, KnowLine-35B-A3B and KnowLine-4B lines, and served with read twice.** [Inference guide](INFERENCE.md) · [中文说明](README.zh.md) · weights Apache-2.0 · siblings: [KnowLine-26B-A4B-Gen2.5](https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen2.5), [KnowLine-35B-A3B-Gen1.5](https://huggingface.co/PelaAI/KnowLine-35B-A3B-Gen1.5), [KnowLine-4B-Gen5.5](https://huggingface.co/PelaAI/KnowLine-4B-Gen5.5) Highlights: - **Decision Index 0.3 public suite: 70.13** (self-run, fresh full run, read twice). KnowLine-35B-A3B-Gen1.5 scores 70.72 and KnowLine-26B-A4B-Gen2.5 68.21 on the same suite with the same serving; see [Comparison](#comparison). - **Other public benchmarks:** C-Eval 82.5 (KnowLine-26B-A4B-Gen2.5: 77.9), JevBench / JevBench-hard 90.5 / 80.2 (90.5 / 81.1); see [Evaluation](#evaluation). - **Calibrated probabilities:** a temperature, fitted on held-out data, is folded into the weights. Computed with the board's method on the public suite, ECE is about 0.012-0.021 and the Brier score about 0.23-0.26 (KnowLine-26B-A4B-Gen2.5: 0.013-0.018 and 0.24-0.27); see [Calibration](#calibration). - **Images without any image training:** trained on text only, it keeps the base model's vision tower and scores 69.3 on the official vision public suite (self-run); see [Images](#images-zero-shot). KnowLine is an independent model. It is not affiliated with or endorsed by TypeSafe, Jev or Google, and it is not derived from TypeSafe or Jev. ## What it does You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model answers every question with a probability distribution over its options: - one forward pass per question, with no text generation; - the output is the probability of each option's label token; - existing Jev clients only need to change the base URL, and users of other KnowLine models only need to change the model name, because the interface is the same. | | | |---|---| | Base model | [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) (Apache-2.0): dense, about 31B parameters | | Training | LoRA SFT (rank 32, alpha 64, language model only) for one epoch, merged into bf16 weights (see [How Gen1 was made](#how-gen1-was-made)) | | Release format | bf16 weights (about 63 GB), with a readout temperature T = 1.44 folded into the final norm weight and the final logit soft-cap (see [Calibration](#calibration)); the front end reads score questions at an effective temperature of 1.0 (`temperatures.json`). The vision tower is unchanged from the base model; the tokenizer is Gemma 4's, not the one of the Qwen-based KnowLine models. | | Serving | `serve_knowline.sh`: SGLang in bf16, as for the 26B-A4B (we did not test FP8 at load for this model); `knowline_server.py` reads short requests twice, as in KnowLine-26B-A4B-Gen2.5 and KnowLine-35B-A3B-Gen1.5 (on by default; `--no-read-twice` reads each request once). SGLang reserves 85% of the GPU's memory by default (MEM); the weights take about 63 GB. We ran every evaluation with one server on one 96 GB H20. | | Languages | English, Simplified Chinese, Traditional Chinese | ## Quickstart ```bash pip install "sglang==0.5.21" "transformers==5.12.1" requests bash serve_knowline.sh PelaAI/KnowLine-31B-Gen1 0 8080 # SGLang (bf16) on :9080 + /v1/systemone on :8080 curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{ "model": "m", "state": "Customer: my order arrived broken, I want my money back.", "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"}, "tone": {"type": "choice", "instructions": "Customer tone?", "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}' ``` - **Front end:** `knowline_server.py` with `knowline_layout.py` next to it; together they depend only on transformers and requests. Text requests are rendered exactly as by the KnowLine-26B-A4B-Gen2.5 front end, prompt for prompt. - **Full settings:** the exact settings used for our Decision Index run are in [INFERENCE.md](INFERENCE.md). ## How Gen1 was made - **Base model:** [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) (Apache-2.0), a dense model. - **Training:** one epoch with a LoRA adapter (rank 32, alpha 64, language model only), merged into bf16 weights. Training examples include decision tasks written and labelled by LLM teachers, rendered with Gemma's chat template. - **Serving and temperature:** it is served with read twice, as KnowLine-26B-A4B-Gen2.5 and KnowLine-35B-A3B-Gen1.5. A single readout temperature T = 1.44, fitted on held-out data with this serving, is folded into the weights. It changes the size of the probabilities, but not which option ranks first. The training data is not included in this repository. ## Comparison Scores on the Decision Index. Since Decision Index 0.4, the board's full score is 60% private tests of the same skills and 40% private tasks from new domains; the public suite is shown but no longer counts. This model has no board entry yet. | model | size | DI 0.3 public | DI 0.4.1 full | source | |---|---|---|---|---| | **KnowLine-31B-Gen1** | 31B | **70.13** | not yet scored | self-run, official kit | | KnowLine-35B-A3B-Gen1.5 | 35B-A3B | 70.72 | 65.77 | board | | KnowLine-26B-A4B-Gen2.5 | 26B-A4B | 68.21 | 64.57 | board | | Fastino GLiDE V1.2 | – | 71.99 | 71.52 | board | | Jeffan-31b-1710 | 31B | 73.13 | 69.75 | board | | mammoth-1 | 27B | 75.34 | 68.44 | board | | Perplexity Decider v1.2 | 27B | 67.74 | 67.98 | board | | StartLux-Decision v1.1 27B | 27B | 72.88 | 67.92 | board | | FlexDM · Gemma-4-31B | 31B | 63.63 | 67.56 | board | | Jev 1.14 | (API) | 60.70 | 60.24 | board | Board rows from the board's data of 2026-10-22 (20:10 UTC, Decision Index 0.4.1). Public-suite skill by area (DI 0.3, all three served read twice): | area | KnowLine-31B-Gen1 | KnowLine-26B-A4B-Gen2.5 | KnowLine-35B-A3B-Gen1.5 | |---|---|---|---| | Knowledge & reasoning | 59.4 | 55.1 | 55.0 | | Language | 72.0 | 71.2 | 72.9 | | Retrieval & routing | 73.0 | 70.9 | 75.4 | | Tools & agents | 84.7 | 83.3 | 88.2 | | Arts & taste | 60.5 | 61.4 | 64.2 | ## Releases The model is trained in a self-evolving loop, and new versions will follow. | model | date | DI 0.3 public | notes | |---|---|---|---| | KnowLine-31B-Gen1 (this model) | 2026-10-24 | 70.13 | first release on a dense Gemma 4 base; read-twice serving | | [KnowLine-26B-A4B-Gen2.5](https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen2.5) | 2026-10-22 | 68.21 | Gen2 weights, read-twice serving | | [KnowLine-35B-A3B-Gen1.5](https://huggingface.co/PelaAI/KnowLine-35B-A3B-Gen1.5) | 2026-10-22 | 70.72 | 35B-A3B Gen1 weights, read-twice serving | | [KnowLine-4B-Gen5.5](https://huggingface.co/PelaAI/KnowLine-4B-Gen5.5) | 2026-10-22 | 66.75 | Gen5 weights, read-twice serving | | [KnowLine-4B-Gen5](https://huggingface.co/PelaAI/KnowLine-4B-Gen5) | 2026-10-21 | 65.67 | 4B, fifth release | | [KnowLine-26B-A4B-Gen2](https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen2) | 2026-10-19 | 66.94 | second 26B-A4B release | | [KnowLine-26B-A4B-Gen1](https://huggingface.co/PelaAI/KnowLine-26B-A4B-Gen1) | 2026-10-19 | 65.76 | first release on a Gemma 4 base | | [KnowLine-35B-A3B-Gen1](https://huggingface.co/PelaAI/KnowLine-35B-A3B-Gen1) | 2026-10-12 | 69.49 | first 35B-A3B release | ## Evaluation All results are self-run and have not been verified by a third party. Every number for this model was measured through its release front end (read twice on). The 26B-A4B-Gen2.5 and 35B-A3B-Gen1.5 numbers are from their release evaluations with the same front end. ### Other public benchmarks | suite | 31B-Gen1 | 26B-A4B-Gen2.5 | 35B-A3B-Gen1.5 | |---|---|---|---| | C-Eval (4 categories, macro) | 82.5 | 77.9 | 88.1 | | JevBench / JevBench-hard | 90.5 / 80.2 | 90.5 / 81.1 | 87.0 / 73.0 | - In a paired sign test against KnowLine-26B-A4B-Gen2.5, it is higher on C-Eval (p < 0.001); JevBench and JevBench-hard (p = 1.0) do not differ. - Against KnowLine-35B-A3B-Gen1.5, it is lower on C-Eval (p < 0.001); JevBench and JevBench-hard (p = 0.06) do not differ at p < 0.05. ### Web operation (Mind2Web official test splits) | split | 31B-Gen1 element selection | 31B-Gen1 operation (balanced) | 26B-A4B-Gen2.5 element selection | 26B-A4B-Gen2.5 operation (balanced) | |---|---|---|---|---| | test_task (Cross-Task) | 91.5 | 92.8 | 89.8 | 90.3 | | test_website (Cross-Website) | 91.5 | 95.9 | 89.2 | 94.0 | | test_domain (Cross-Domain) | 90.4 | 96.6 | 89.6 | 92.9 | - Element selection is higher than with KnowLine-26B-A4B-Gen2.5 on all three splits (paired test p = 0.05, 0.01 and 0.11); operation accuracy is within noise (p ≥ 0.09). No request failed. - The task is to pick the target element from a candidate set: the target plus up to 5 other elements sampled from the page. This is easier than the original Mind2Web protocol, so do not compare these results with the Mind2Web leaderboard. ### Images (zero-shot) - **No image training:** the vision tower is the base model's, unchanged, and no images were seen in training. `knowline_server.py` accepts OpenAI-style `image` / `image_url` content parts in a chat state and passes them to SGLang; each question is still one prefill. Requests with images are read once (read twice is text-only). - **Result:** 69.3 on the official vision public suite (12,210 rows, complete, 0 failed requests; self-run with the official kit, through the release server). ### Calibration Computed on the 0.3 public suite using the method described by the Decision Index board: each field is scored right or wrong, confidence is the probability of the chosen option, benchmarks are weighted equally, and 10 equal-width bins are used. | metric | 31B-Gen1 | 26B-A4B-Gen2.5 | 35B-A3B-Gen1.5 | |---|---|---|---| | ECE (lower is better) | 0.012-0.021 | 0.013-0.018 | 0.03-0.04 | | Brier score (lower is better) | 0.23-0.26 | 0.24-0.27 | 0.24-0.26 | | Confidence ≥95% but wrong | 0.9-1.0% | 0.9% | 1.5% | | Mean confidence / accuracy | 0.83-0.84 / 0.81-0.83 | 0.82-0.83 / 0.80-0.82 | 0.85-0.86 / 0.81-0.83 | - **Temperature in the weights:** the final norm weight and the final logit soft-cap include a readout temperature T = 1.44, fitted on held-out data with the read-twice serving, so every final logit is divided by 1.44 before the softmax over the option labels. The top-ranked option does not change. - **Score questions at temperature 1.0 (new in this release):** `serve_knowline.sh` passes `temperatures.json` (`{"score": 0.6944}`), so the front end divides the logits of score (rubric) questions by a further 0.6944 and reads them at an effective temperature of 1.44 × 0.6944 = 1.0; yes/no and choice questions stay at 1.44. Score questions are often read as an expected value over the scale, which a higher temperature pulls toward the middle. This is a serving setting, not a weight change, and the earlier KnowLine releases did not use it; starting `knowline_server.py` without `--temperatures` (or `knowline_engine.py` with `--option temperatures=none`) reads every question at 1.44. - **Why ranges:** the board does not publish which benchmarks it uses for calibration, so we report ranges over the plausible benchmark sets, as for the earlier releases. - **Recommendation:** if your use depends on probability thresholds, check calibration on your own data. ## Disclosures - **Decontamination:** all training data was checked with standard and strict decontamination against the Decision Index public suite and the public benchmarks we report (Intern-Decision, S1MB, JevBench, MMDM). 491 rows that nearly duplicated an evaluation item, repeated its question or claim, or contained its whole state were removed. - **Teacher-labelled synthetic data:** LLM teachers wrote and labelled our synthetic tasks. The labels were filtered, but not all of them were checked by a human. - **Public suite:** the Decision Index public suite is public; the board's private tests may show a lower score. ## Limitations - **Size and speed:** about 63 GB of bf16 weights and dense, so every token uses all 31B parameters; we have only run it on 96 GB GPUs. One request at a time on one H20, requests take 340 ms median and 910 ms mean (3,977 ms at the 95th percentile), against 66.3 ms and 151 ms for KnowLine-26B-A4B-Gen2.5; see [INFERENCE.md](INFERENCE.md). - **Read twice:** text-only requests up to 3,000 tokens are written twice, about 1.54 times the prompt tokens on the Decision Index suite. Use `--no-read-twice` if latency matters more; we did not evaluate this model without read twice for this card. - **Planted instructions:** this model can be steered by instructions planted in the input; put untrusted content in data fields of the state, not in the instructions, and keep it clearly delimited. - **C-Eval:** below KnowLine-35B-A3B-Gen1.5 (82.5 against 88.1). - **Calibration:** probabilities are calibrated on average, not for every domain; check calibration on your own data. - **Images:** there was no image training, so check image use on your own data. - **Other attacks:** other prompt-injection attacks may still work. - **Private tests:** the board's private tests may show a lower score than the public suite. ## Citation ```bibtex @misc{knowline31bgen1, title = {KnowLine-31B-Gen1: a 31B decision model}, author = {PelaAI}, year = {2026}, url = {https://huggingface.co/PelaAI/KnowLine-31B-Gen1} } ``` ## License - **Weights:** Apache-2.0, the same as the base model. - **Base model licence:** google/gemma-4-31B-it is released under Apache-2.0; see Google's licence page, [ai.google.dev/gemma/docs/gemma_4_license](https://ai.google.dev/gemma/docs/gemma_4_license). - **Code:** `knowline_server.py`, `knowline_layout.py` and `knowline_engine.py` are MIT (see the file headers).