--- title: jebadiah-9b-v2-GGUF canonical_url: "https://www.modelscope.ai/models/JasonBrashear/jebadiah-9b-v2-GGUF" md_url: "https://www.modelscope.ai/models/JasonBrashear/jebadiah-9b-v2-GGUF.md" repository: JasonBrashear/jebadiah-9b-v2-GGUF last_updated: 2026-09-29 license: apache-2.0 pipeline_tag: text-classification tasks: - text-classification base_model: - JasonBrashear/jebadiah-9b-v2 base_model_relation: quantized library_name: - gguf language: - en downloads: 7 stars: 0 tags: - gguf - llama.cpp - decision-model - system-one - typed-decisions - calibrated-probabilities - ainode --- # jebadiah-9b-v2-GGUF > jebadiah-9b-v2-GGUF - An open-source model by JasonBrashear on ModelScope. Mirror of huggingface.co/frontier-infra/jebadiah-9b-v2-GGUF. The Hugging Face repo is the source of truth: https://huggingface.co/frontier-infra/jebadiah-9b-v2-GGUF. Code and the… JasonBrashear/jebadiah-9b-v2-GGUF is a text-classification model on ModelScope. licensed under apache-2.0. derived from JasonBrashear/jebadiah-9b-v2. - **Repository**: JasonBrashear/jebadiah-9b-v2-GGUF - **License**: apache-2.0 - **Tasks**: text-classification - **Base model**: JasonBrashear/jebadiah-9b-v2 - **Tags**: gguf, llama.cpp, decision-model, system-one, typed-decisions, calibrated-probabilities, ainode - **Downloads**: 7 - **Stars**: 0 - **Last updated**: 2026-09-29 Source: https://www.modelscope.ai/models/JasonBrashear/jebadiah-9b-v2-GGUF --- # Jebadiah 9B v2 GGUF > Mirror of huggingface.co/frontier-infra/jebadiah-9b-v2-GGUF. The Hugging Face repo is the source of truth: [https://huggingface.co/frontier-infra/jebadiah-9b-v2-GGUF](https://huggingface.co/frontier-infra/jebadiah-9b-v2-GGUF). Code and the standalone server: [github.com/getainode/jebadiah](https://github.com/getainode/jebadiah). GGUF builds of [Jebadiah 9B v2](https://huggingface.co/frontier-infra/jebadiah-9b-v2) for [llama.cpp](https://github.com/ggml-org/llama.cpp), which runs on NVIDIA, AMD and Apple GPUs and on plain CPUs. Jebadiah answers a typed question (choice, noul or score) with a probability for every option, read from one forward pass. Nothing is generated. Code, trainer and evals: [getainode/jebadiah](https://github.com/getainode/jebadiah). **Results and docs** - Project site, with every result and how to run the models: [jebadiah.ai](https://jebadiah.ai). - JevBench v1.4.2: on its 231 public items, run through its own harness, Jebadiah 9B v2 scores 0.818 (Jebadiah 27B: 0.866, the same as Jev 1.13.0). This is my own run on the public items, not the official board, which also uses sealed items. [Details and caveats](https://github.com/getainode/jebadiah#where-we-stand-on-jevbench). - [JDE](https://github.com/Titanium-Devops/jde) blind test (as of 2026-09-26): on 290 real decisions from Titanium Computing's production decision engine, Jebadiah 9B v2 got 277 against Jev's 282, with 0 flips across 5,800 repeat calls. Jev still leads on coverage checks. - For the family: [Decision Index 0.2.1](https://huggingface.co/spaces/multimodalart/jev-decision-index): the 27B scores 54.67, #5 of 67 open models (as of 2026-09-26). This is the board's own number; the maintainer validated my run and put it on the leaderboard. [Run record](https://huggingface.co/datasets/frontier-infra/jebadiah-decision-index-results/tree/main/runs/jebadiah-27b-1c0d794f). - All sizes: the [Hugging Face collection](https://huggingface.co/collections/frontier-infra/jebadiah-open-system-one-decision-models-6ab80765ddd3fa0b3eba5213), mirrored on [ModelScope](https://www.modelscope.ai/profile/JasonBrashear). **Which one should I use?** For local use, start with [Jebadiah 9B v2 GGUF](https://huggingface.co/frontier-infra/jebadiah-9b-v2-GGUF). On Apple silicon, use an MLX build: [27B](https://huggingface.co/frontier-infra/jebadiah-27b-MLX), [9B v2](https://huggingface.co/frontier-infra/jebadiah-9b-v2-MLX) or [4B v2](https://huggingface.co/frontier-infra/jebadiah-4b-v2-MLX). For vLLM or fine-tuning, use the full weights: [27B](https://huggingface.co/frontier-infra/jebadiah-27b), [9B v2](https://huggingface.co/frontier-infra/jebadiah-9b-v2) or [4B v2](https://huggingface.co/frontier-infra/jebadiah-4b-v2). Setup for every runtime, and for JDE: [Run Jeb locally](https://github.com/getainode/jebadiah/blob/main/docs/run-locally.md). ## Files Every file was checked on the 260 held-out questions the merged weights were checked on, and compared with the merged bf16 weights and with the training run's own eval records. | File | Size | Same answer as bf16 | Same as the run | choice + noul | score | Prob. diff median / max | |---|---:|---:|---:|---:|---:|---:| | `jebadiah-9b-v2-Q8_0.gguf` | 9.8 GB | **257 / 260** | 256 / 260 | 171 / 173 | 85 / 87 | 0.003 / 0.051 | | `jebadiah-9b-v2-Q5_K_M.gguf` | 6.6 GB | **250 / 260** | 251 / 260 | 169 / 173 | 82 / 87 | 0.012 / 0.252 | | `jebadiah-9b-v2-Q4_K_M.gguf` | 5.8 GB | **240 / 260** | 239 / 260 | 164 / 173 | 75 / 87 | 0.022 / 0.415 | | *bf16 weights* | | | 257 / 260 | 172 / 173 | 85 / 87 | 0.002 / 0.029 | Which one: `Q8_0` if it fits (it changed 3 answers here); `Q4_K_M` when memory is short. A file needs about its own size in GPU or unified memory, plus about 1 GB for a 4k context. **`Q5_K_M` changes 10 of 260 answers** against bf16 (7 on score questions) and moves probabilities more (median 0.012, max 0.25). Use it only when a larger build does not fit. **`Q4_K_M` changes 20 of 260 answers** against bf16 (10 on score questions) and moves probabilities more (median 0.022, max 0.41). Use it only when a larger build does not fit. ## Run it The answer is the log probability of each option label ("A", "B", ...) at the answer position, which llama-server's `/completion` returns. The script renders the prompt exactly as AINode does, sends the raw text (so the server's own chat template is never used), renormalises over the labels and applies `temperatures.json` (choice 1.1863, noul 1.0903, score 0.8329). You need a llama.cpp that knows the `qwen35` architecture: we checked v0.5.0 (older builds refuse the file). ```bash hf download frontier-infra/jebadiah-9b-v2-GGUF --include "*Q8_0.gguf" --include "scripts/*" --include "*.json" --include "*.jinja" --include "*.txt" --local-dir jebadiah-9b-v2-GGUF cd jebadiah-9b-v2-GGUF llama-server -m jebadiah-9b-v2-Q8_0.gguf -c 4096 -np 1 --port 8080 pip install transformers # the tokenizer only, no torch python scripts/decide_gguf.py --server http://127.0.0.1:8080 --request scripts/example-request.json ``` `--no-temperatures` returns the raw probabilities. For Ollama and LM Studio, see the next sections. **HelpSteer2-like traffic.** A held-out HelpSteer2 check (2026-09-29): on the 418 `nvidia/HelpSteer2` validation rows that no reported evaluation set uses (natural label mix, never in the training pool), this model's score answers want a temperature of 1.29, not 0.83, and still 1.11 after reweighting to a flat label mix. If your traffic looks like HelpSteer2, keep the older score temperature, 1.22 (the `train` fit in `temperatures.json`; every script here takes `--temperatures` with a file of your own), or refit on your own labels. v3's calibration split will draw HelpSteer2 from held-out data. On `example-request.json` (jebadiah-9b-v2-Q8_0.gguf): ```json { "route": {"type": "choice", "choice": "billing", "confidence": 0.461713, "probabilities": {"billing": 0.641142, "support": 0.036095, "sales": 0.322763}}, "urgent": {"type": "noul", "noul": 0.167016} } ``` ## Use it in Ollama ```bash pip install jebadiah-decide jeb doctor # pulls hf.co/frontier-infra/jebadiah-9b-v2-GGUF:Q8_0 into Ollama the first time, checks it, answers the example jeb serve # POST http://localhost:8100/v1/systemone ``` `jeb serve` renders the prompt, calls Ollama raw with thinking off, reads the label probabilities and applies `temperatures.json`; it serves AINode's `/v1/systemone` and `/v1/decide` on localhost for JDE or your code. Don't use `ollama run` or the chat window: Jeb isn't a chat model, and what it writes there is nonsense by design. Tested with Ollama 0.34.4: 257 of 260 answers the same as bf16, and the same answer as llama-server on all 260. At most 20 options per question. `scripts/decide_ollama.py` does the same for one request file. Every runtime, step by step: [Run Jeb locally](https://github.com/getainode/jebadiah/blob/main/docs/run-locally.md). ## Use it in LM Studio Jeb works in LM Studio through its local server, not the chat window: chat runs with thinking on and shows text, while a decision needs the probability of every option label. `scripts/decide_lmstudio.py` takes the same arguments and prints the same output as `decide_gguf.py`. It sends AINode's messages to LM Studio's `/v1/chat/completions` with thinking off (`"reasoning_effort": "none"`), where LM Studio renders the same prompt text the llama-server path sends, and reads the option labels from the top log probabilities that come back. It stops with an error if LM Studio's prompt length differs from the local tokenizer's. 1. In LM Studio, search for `jebadiah-9b-v2` and download `jebadiah-9b-v2-Q8_0.gguf` from this repository. 2. Open the **Developer** tab, start the server and load the model. Note the identifier LM Studio shows for it (for example `jebadiah-9b-v2`). 3. In a terminal: ```bash hf download frontier-infra/jebadiah-9b-v2-GGUF --include "scripts/*" --include "*.json" --include "*.jinja" --include "*.txt" --local-dir jebadiah-9b-v2-GGUF pip install transformers # the tokenizer only, no torch python jebadiah-9b-v2-GGUF/scripts/decide_lmstudio.py --model jebadiah-9b-v2 --request jebadiah-9b-v2-GGUF/scripts/example-request.json ``` If **Require Authentication** is on in LM Studio's server settings, create a token there and `export LM_API_TOKEN=...` first. Tested on the 9B only: LM Studio 0.4.21 with `jebadiah-9b-v2-Q8_0`, 257 of 260 answers the same as bf16, and the same answer as llama-server on the same file on 260 of 260. The 27B and 4B builds use the same script but have not been run in LM Studio. **At most 20 options per question.** LM Studio returns only the top 20 log probabilities, the same cap AINode's own route has. On the 77-option Banking77 questions the pick was still right, but the probabilities moved by up to 0.16, so do not rely on them past 20 options. **If LM Studio shows a "Vision" tag, ignore it.** Some GGUF repositories, including third-party quants of Jeb, ship a vision file (mmproj) from the Qwen base, and LM Studio labels the model Vision because of it. Jeb was not trained on images, and it does not make decisions in the chat window. It decides only through `decide_gguf.py` or `decide_lmstudio.py`, which send the typed question with thinking off and read the probabilities. The mmproj is not needed. ## How it was measured Jevals PubMedQA, Banking77 (77 options) and HelpSteer2, plus Nimble: the merge check's fixed sample (seed 20260925), the run's option order and the run's temperatures, so the probability differences compare like with like (the shipped temperatures.json since 2026-09-29 uses 0.8329 for score; top picks do not depend on it). "Same answer" is the top option; "prob. diff" is the largest change on any option against the run's CUDA record. llama-server ran on Metal (M3 Ultra) with the same tokens as the Python renderer on every prompt. Records: `eval/agreement-*.json`. ## License Apache-2.0, as the base model. Made in Texas.