--- title: jebadiah-9b-v2 canonical_url: "https://www.modelscope.ai/models/JasonBrashear/jebadiah-9b-v2" md_url: "https://www.modelscope.ai/models/JasonBrashear/jebadiah-9b-v2.md" repository: JasonBrashear/jebadiah-9b-v2 last_updated: 2026-09-26 license: apache-2.0 pipeline_tag: text-classification tasks: - text-classification model_type: - qwen3_5 architectures: - Qwen3_5ForConditionalGeneration base_model: - Qwen/Qwen3.5-9B base_model_relation: finetune parameters: 9.7B tensor_type: - BF16 - F32 library_name: - safetensors - pytorch frameworks: - pytorch language: - en downloads: 21 stars: 0 tags: - decision-model - system-one - calibrated-probabilities - typed-decisions - ainode - merged-lora --- # jebadiah-9b-v2 > jebadiah-9b-v2 - An open-source model by JasonBrashear on ModelScope. Mirror of huggingface.co/frontier-infra/jebadiah-9b-v2. The Hugging Face repo is the source of truth: https://huggingface.co/frontier-infra/jebadiah-9b-v2. Code and the standalone server:… JasonBrashear/jebadiah-9b-v2 is a 9.7B-parameter text-classification model on ModelScope. licensed under apache-2.0. derived from Qwen/Qwen3.5-9B. - **Repository**: JasonBrashear/jebadiah-9b-v2 - **License**: apache-2.0 - **Tasks**: text-classification - **Parameters**: 9.7B - **Base model**: Qwen/Qwen3.5-9B - **Tags**: decision-model, system-one, calibrated-probabilities, typed-decisions, ainode, merged-lora - **Downloads**: 21 - **Stars**: 0 - **Last updated**: 2026-09-26 Source: https://www.modelscope.ai/models/JasonBrashear/jebadiah-9b-v2 --- # Jebadiah 9B v2 > Mirror of huggingface.co/frontier-infra/jebadiah-9b-v2. The Hugging Face repo is the source of truth: [https://huggingface.co/frontier-infra/jebadiah-9b-v2](https://huggingface.co/frontier-infra/jebadiah-9b-v2). Code and the standalone server: [github.com/getainode/jebadiah](https://github.com/getainode/jebadiah). [Code](https://github.com/getainode/jebadiah) · Sizes: [27B](https://huggingface.co/frontier-infra/jebadiah-27b) · **9B v2** · [4B v2](https://huggingface.co/frontier-infra/jebadiah-4b-v2) · Previous: [9B v1](https://huggingface.co/frontier-infra/jebadiah-9b-v1) Jebadiah (Jeb for short) is Frontier Infra's open System One style decision model. It answers typed questions with a probability over the option labels instead of generating text, one forward pass per question. Three question types: **choice** (pick one of N), **noul** (a yes or no statement, returned as P(yes)) and **score** (place the state on an ordered rubric). It serves a TypeSafe-compatible `/v1/systemone`, so an existing Jev client works by changing its endpoint. It is a standard transformers model: run it anywhere. Trained on public data only. They call me Jeb. He does not talk much. He just decides. This repository holds full bf16 weights: the v2 LoRA merged into `Qwen/Qwen3.5-9B` (revision `c2022362`, the chat checkpoint, thinking off). What changed from v1 is one thing, the base: v1 was trained on `Qwen/Qwen3.5-9B-Base`. The data, LoRA, objective and temperature fit are v1's. Made in Texas. ## Results Measured by us with AINode's bench, one logit read per question, the same rendered prompt for every model. Accuracy is the share of questions whose top label is the human label. Headline is the macro over the zero-shot public sets. These are our numbers on the public suites, not rows on the Jevals board. | Accuracy | 27B | **9B v2** | 4B v2 | 9B v1 | |---|---:|---:|---:|---:| | **Headline** | 78.9 | **73.9** | 72.5 | 73.3 | | Jevals PubMedQA (noul, 300) | 90.0 | 90.3 | 88.7 | 89.7 | | Jevals Banking77 (choice, 77 options, 300) | 77.0 | 70.7 | 70.0 | 70.0 | | Jevals HelpSteer2 helpfulness (score, 300) | 48.7 | 40.7 | 40.0 | 40.3 | | Nimble held-out eval (mixed, 324) | 93.5 | 81.2 | 77.2 | 78.7 | | Kev transfer-v4 test (mixed, 764) | 85.9 | 83.8 | 83.2 | 84.0 | | Nimble public, 13 subsets (macro, 3,880) | 78.6 | 77.0 | 75.9 | 77.0 | The 27B is the same recipe on `Qwen/Qwen3.8-27B`; see [its card](https://huggingface.co/frontier-infra/jebadiah-27b). For scale, Bespoke's published table puts Nimble-9B at 74.8 and Jev at 76.0 on the same 13 Nimble public subsets, with their scorer. Where v2 is not better than v1: the Jevals HelpSteer2 Decision Score is 10.4 against 11.9, and its top level flips over identical repeats on 2.3% of questions against 0.3%. The Nimble public macro is flat (77.0): MultiNLI and PAWS give back what PubMedQA and HelpSteer2 gain. ECE on the Nimble 324 set rises from 0.063 to 0.085. The headline gain is 0.6 points, most of it from the Nimble 324 set. HelpSteer2 and SummEval are not zero-shot for Jeb: the pool trains on their train split and unscored articles (no evaluation item overlaps), so those rows are held-out items of a seen rubric. Every calibrated number applies the per-type temperatures in `temperatures.json`. Decision Scores, ECE, repeat flips, the in-distribution sets, every Nimble subset and the merge check are in [`eval/RESULTS.md`](eval/RESULTS.md); the per-question records are beside it in `eval/`. The nonce robustness pass is pending for this model, so no robustness number is claimed. ## Run it anywhere **Standalone server.** [`server/`](https://github.com/getainode/jebadiah/tree/main/server) in the Jebadiah repository serves one model on one CUDA GPU or an Apple Silicon Mac: `POST /v1/systemone` (Jev's wire format), AINode's `POST /v1/decide`, and a browser playground. Python 3.12 and [uv](https://docs.astral.sh/uv/). The 9B needs about 19 GB for its bf16 weights plus runtime overhead. ```bash git clone https://github.com/getainode/jebadiah.git cd jebadiah/server uv sync # add --extra cuda on a CUDA box for the fast linear-attention kernels uv run hf download frontier-infra/jebadiah-9b-v2 uv run jebadiah-serve --model frontier-infra/jebadiah-9b-v2 ``` Open **http://localhost:8000** for the playground, or `/docs` for the API. `--host 0.0.0.0` serves other machines; `JEBADIAH_API_KEY` requires a bearer key. It is the server's default model, so `uv run jebadiah-serve` alone serves it. ```bash curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{ "state": {"ticket": "Customer says the invoice total does not match the quote."}, "questions": { "route": {"type": "choice", "instructions": "Which team should take this ticket?", "criteria": {"billing": "an invoice, a charge or a refund", "support": "a product question", "sales": "a quote or a renewal"}}, "urgent": {"type": "noul", "instructions": "The customer is blocked from working.", "criteria": {"true": "work has stopped", "false": "it can wait"}}}}' ``` A choice comes back as `choice`, `confidence` and `probabilities` per key; a noul as `noul` = P(true); a score as `score`, `legend` and `probabilities`. The response carries a `calibration` block with the temperatures applied. **transformers only.** `scripts/` in this repository is the renderer and logit read the model was trained with, and `decide_standalone.py` runs one request end to end: ```bash pip install torch transformers==5.17.0 huggingface_hub hf download frontier-infra/jebadiah-9b-v2 --local-dir jeb python jeb/scripts/decide_standalone.py --model jeb --request jeb/scripts/example-request.json # --no-temperatures for the raw distribution ``` Runs on CPU too (slow without the DeltaNet kernels), and `--device mps` on Apple silicon, which is how it was checked before publishing. **LM Studio.** Load a GGUF build in LM Studio's server and run `scripts/decide_lmstudio.py` from [jebadiah-9b-v2-GGUF](https://huggingface.co/frontier-infra/jebadiah-9b-v2-GGUF#use-it-in-lm-studio); questions of up to 20 options. ## Run it on AINode The AINode model id is the Hugging Face repo id. Load it on a node (`--language-model-only` keeps vLLM from profiling the unused vision tower): ```bash curl -sS https:///api/models/load -H "Authorization: Bearer " -H "Content-Type: application/json" \ -d '{"model": "frontier-infra/jebadiah-9b-v2", "gpu_memory_utilization": 0.30, "max_model_len": 4096, "kv_cache_dtype": "auto", "extra_vllm_args": ["--language-model-only", "--enforce-eager"]}' ``` Then send the same body to `https:///v1/systemone` with `"model": "frontier-infra/jebadiah-9b-v2"` added, or ask AINode's own shape at `/v1/decide`. From AINode 0.5.32 both routes apply `temperatures.json` (send `"calibration": "raw"` for the untempered distribution), and from 0.5.33 also when the request enters through a master that does not serve the model. An older node returns raw probabilities, and on 0.5.31 the first request of each question type compiles its grammar for a minute or more, so warm each type with a one-question request there. ## Other formats - **GGUF** for llama.cpp: [jebadiah-9b-v2-GGUF](https://huggingface.co/frontier-infra/jebadiah-9b-v2-GGUF). Same answer as these weights on 257 of 260 held-out questions (Q8_0) and 240 of 260 (Q4_K_M). - **MLX** for Apple silicon: [jebadiah-9b-v2-MLX](https://huggingface.co/frontier-infra/jebadiah-9b-v2-MLX). Same answer on 256 of 260 (8-bit; the 4-bit build fell under 90% and was not published). ## How it decides The prompt is AINode's own decide rendering (source commit `e5c08938`, hash in `prompt_contract.json`), through the chat template with thinking off. The option labels are single tokens; the answer is the distribution over those label tokens at the last prompt position, read in fp32 and temperature scaled per type (`temperatures.json`: choice 1.19, noul 1.09, score 1.22). Nothing is generated. Those are the `train` fit, which minimises NLL against the soft or ordinal target the model was taught; it softens, and it costs some ECE against hard labels on the calibration split. A sharper `hard` fit (0.79 / 0.67 / 0.83) is in the same file for a consumer who gates on the argmax. Thresholds belong to the caller: act on a high probability, confirm or escalate on a middle one, hand a low one to a person or a bigger model. The model never refuses. ## Training - **Base:** `Qwen/Qwen3.5-9B` at `c2022362`, the chat checkpoint, thinking off (v1: `Qwen/Qwen3.5-9B-Base`). - **Objective:** cross-entropy over the option-label logits at the answer position; score questions use an ordinal target around the human label. - **Adapter:** LoRA r=16, alpha=32 on every linear projection (43.3M trainable), learning rate 1e-4, one epoch, 1,862 steps, seed 17. Merged into the bf16 weights: 248 of 248 pairs, and the merged weights give the run's pick on 257 of 260 re-scored questions. - **Data, public only:** 11,013 records and 15,813 questions from `LocalLLaMA/typed-decisions`, the Kev v7 sources whose licenses permit derived weights, the `nvidia/HelpSteer2` train split and SummEval (Nimble's 15 evaluation articles excluded). A 95/5 split into training and calibration. Nimble's train set and the Jevals test items are excluded. - **Compute:** one NVIDIA A100 80GB PCIe, 90 minutes of training. ## Limitations - Single-hop judgments only. Split a chain of inference into hops. - A choice question is capped at 20 options on `/v1/systemone`; Banking77's 77 options were scored with an extended single-token alphabet for the benchmark only. - English data. Training cut states past 2,048 prompt tokens. - Calibration was fitted on the training distribution. Refit before trusting a threshold. ## Versioning and license v2 is frozen; later versions land as new `jebadiah-*` repositories, and v1 stays as it is. Weights Apache-2.0, base model `Qwen/Qwen3.5-9B` Apache-2.0. Evaluation data: Jevals suite 0.1.0 (CC-BY-4.0, "Jevals (jevals.com), release 2026-09-18"), Nimble public subsets (Bespoke Labs), Kev test sets and typed-decisions test, each under its own license. Not affiliated with TypeSafe AI or Jev.