cross-posted from: https://reddthat.com/post/74093908

Ollama quietly shipped v0.35 with a new class of models they’re calling “decision models,” and the first one is worth a look. It’s called Nimble, it’s a 9B model from Bespoke Labs, and it works completely differently from the chat models you’re used to.

What it actually does

Instead of generating text, you give Nimble some input text (a support ticket, a document, whatever) and a set of questions. It picks answers and returns probabilities for each option. No reasoning step, no generated prose — just typed classifications and scores.

Three question types are supported:

• Choice: pick from a list of 2–26 options
• Yes/No: true/false with a probability score
• Score/Rubric: place something on an ordered scale (lowest to highest)

You can ask up to 64 questions per request about the same input, and it supports a 256K context window. According to Ollama’s benchmarks, it averages around 91ms per decision on an M5 Max MacBook Pro.

How it compares to other decision models

Ollama’s comparison table:

Model	From	Accuracy
Tev1 4B	Together AI	73.3%
Tev1 0.8B	Together AI	63.5%
Nimble 9B	Bespoke Labs	75.7%
Jev 1.13	TypeSafe	76.0%

What you’d use it for

This isn’t a replacement for your chat model. It’s for the stuff you’d otherwise hack together with brittle JSON-mode prompting:

• Ticket triage and routing (e.g., “which team should handle this?”)
• Content moderation decisions
• Model routing (deciding which model gets the expensive work)
• Policy checks and rubric-based scoring

How to try it

ollama pull nimble

Then you hit the /v1/systemone endpoint instead of /v1/chat/completions. Example:

curl http://localhost:11434/v1/systemone \
  -d '{
    "model": "nimble",
    "state": "I was charged twice. Please refund the extra payment.",
    "questions": {
      "team": {
        "type": "choice",
        "instructions": "Which team should handle this ticket?",
        "criteria": {
          "billing": "Payments and refunds",
          "technical": "Bugs and integrations",
          "other": "None of the above"
        }
      },
      "refund": {
        "type": "yes_no",
        "instructions": "Does the customer explicitly ask for a refund?"
      }
    }
  }'

You get back probabilities, not vibes. Apache 2.0 license, fine-tuned from Qwen3.5.

Things worth knowing

One thing I noticed digging into the docs: Nimble reads the prompt once per question. So a single request with 64 questions on a 4K-token prompt is doing 256K token positions of work internally. If you’re budgeting by HTTP request count, that’s a monitoring blind spot. Not necessarily a dealbreaker, but worth instrumenting if you plan to scale it.

Also, it’s a decision model — no streaming, no tools, no images. Text in, structured answers out. Requests cap at 64 KiB.

Has anyone here tried it for routing or triage yet? Curious how the probabilities look in practice vs. just fine-tuning a small classifier.