Before You Upgrade the Model, Try Thinking Mode

Table of Contents

• Introduction
• What Is Thinking Mode?
• Setup
• Multi-Step Problems: Where Thinking Helps
• Single-Step Problems: Where It Does Not
• Final Thoughts

Introduction

When a local model gets a simple task wrong, the natural reaction is to reach for a bigger one. But some failures are not model-size failures. They are reasoning failures.

This showed up while I was testing a local model on imported finance data. The model’s job was to review transactions before reporting: catch bad splits, duplicate card rows, refunds, and fee calculations.

One test looked like this:

A parent transaction of $1,000.00 was split into these children:
  8.84, 86.18, 74.53, 74.53, 20.64, 12.61, 1.49, 1.49, 719.59
What is the difference between the parent amount and the total of the children?
Answer with the absolute difference, to 2 decimal places, and conclude whether
the split balances.

The model answered:

{"difference": 0.0, "conclusion": "The split balances."}

But the children total $999.90, so the correct answer is 0.10.

That is the dangerous failure mode. The model did not merely miss by ten cents; it reported that there was no discrepancy at all. In a transaction-review workflow, that means the bad split moves forward as if it had been checked.

At that point, the obvious fix was to try a bigger local model or move the task to a stronger cloud model. Before doing that, I wanted to test the cheaper option: could the same model improve if I turned on thinking mode? The answer was yes.

This article explores thinking mode across several finance prompts, so that you can recognize the kinds of tasks in your own workflow where it is likely to help.

💻 Get the Code: Grab the runnable benchmark script, prompt list, and
recorded results from
GitHub.

What Is Thinking Mode?

Thinking mode asks the model to work through the problem before returning the final answer. The reasoning trace is kept separate, so your application can still receive the same clean response shape.

That extra reasoning step is what improves the answer, but it is also what makes the response slower.

Setup

Pull the model with Ollama. qwen3:30b-a3b is a mixture-of-experts model: it has about 30 billion total parameters, but activates only about 3 billion for each token. That makes it practical to run locally while still supporting thinking mode:

ollama pull qwen3:30b-a3b

This article uses ollama v0.32.8 and qwen3:30b-a3b.

Point at the local endpoint and the model:

import json
import time
import urllib.request

MODEL = "qwen3:30b-a3b"
URL = "http://localhost:11434/api/chat"

The code below defines the response contract. The model should answer with exactly one numeric field, answer, so each result can be compared directly with the known correct value:

SYSTEM = (
    "You are a careful financial assistant. Answer the question with a "
    "single number only. Do not include currency symbols, thousands "
    "separators, units, or any explanation in the answer field."
)

SCHEMA = {
    "type": "object",
    "required": ["answer"],
    "properties": {"answer": {"type": "number"}},
}

For a typed Python approach to the same problem, the PydanticAI structured-output guide shows how to validate LLM responses with models instead of raw schemas.

Next, the sampling settings. The benchmark uses two decoding setups: greedy decoding for the fast non-thinking baseline, and non-greedy decoding for thinking mode.

Greedy decoding means the model always takes the most likely next token:

yes    70%  <- chosen
no     20%
maybe  10%

That is useful for the non-thinking baseline because it is fast and repeatable.

Thinking mode uses Qwen’s recommended non-greedy settings. Non-greedy decoding can choose from a controlled set of likely tokens instead of locking every step to the top one:

yes    70%  <- likely
no     20%  <- still allowed
maybe  10%  <- unlikely, but possible depending on settings

That extra flexibility is useful for reasoning because the model has to explore intermediate steps before settling on the final answer:

GREEDY = {"temperature": 0}
THINKING = {"temperature": 0.6, "top_p": 0.95, "top_k": 20, "min_p": 0.0}  # settings recommended by Qwen

Next, define the helpers that send the request to Ollama and get the answer:

def _get_ollama_message(question, think):
    options = dict(THINKING if think else GREEDY, seed=0)
    body = {
        "model": MODEL,
        "stream": False,
        "think": think,  # the only variable
        "format": SCHEMA,
        "options": options,
        "messages": [
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": question},
        ],
    }
    request = urllib.request.Request(
        URL, json.dumps(body).encode(), {"Content-Type": "application/json"}
    )
    reply = json.loads(urllib.request.urlopen(request).read())
    return reply["message"]


def _print_thinking_preview(message, max_lines):
    if max_lines == 0 or not message.get("thinking"):
        return
    lines = [line for line in message["thinking"].splitlines() if line.strip()]
    if len(lines) > max_lines:
        head_count = max_lines // 2
        tail_count = max_lines - head_count
        lines = lines[:head_count] + ["..."] + lines[-tail_count:]
    print("\n".join(lines))


def _extract_answer(message):
    return json.loads(message["content"])["answer"]

Then wrap them in ask(). The decorator records each call’s latency in ask.last_seconds, which makes plain and thinking runs easy to compare:

def _timed(func):
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        wrapper.last_seconds = time.perf_counter() - start
        return result

    wrapper.last_seconds = 0.0
    return wrapper


@_timed
def ask(question, think=False, trace_lines=0):
    message = _get_ollama_message(question, think)
    _print_thinking_preview(message, trace_lines)
    return _extract_answer(message)

Multi-Step Problems: Where Thinking Helps

Multi-step problems are hard because the answer is not available from one operation. It has to be built through several intermediate steps. For example:

• Subtract charges in order
• Add split transactions before comparing totals
• Divide an invoice before applying a fee

These are the questions that thinking mode is good for: keeping track of the steps before giving the answer.

To test that, I ran eight multi-step finance questions with thinking mode off and on. Plain mode missed five, while thinking mode answered every one correctly:

Let’s look at the two multi-step problems that plain mode got wrong.

The first failure is a two-step reconciliation problem. The model has to total the nine child transactions first, then subtract that total from the $1,000.00 parent:

split_check = """A parent transaction of $1,000.00 was split into these children:
  8.84, 86.18, 74.53, 74.53, 20.64, 12.61, 1.49, 1.49, 719.59
What is the difference between the parent amount and the total of the children?
Answer the absolute difference, to 2 decimal places."""

print(ask(split_check))

0.0

The answer is wrong. The children total $999.90, so the correct difference is 0.10.

Let’s turn the flag on and see if the answer improves:

print(ask(split_check, think=True, trace_lines=4))

Okay, let's see. I need to find the difference between the parent transaction amount of $1,000.00 and the total of all the children amounts. The children amounts are: 8.84, 86.18, 74.53, 74.53, 20.64, 12.61, 1.49, 1.49, 719.59.
First, I should add up all the children's amounts. Let me list them out again to make sure I have them all:
...
So the absolute difference is $0.10, which is 0.10 when rounded to two decimal places.
I think that's right. My initial mistake was in the first addition step where I added 169.55 +74.53 as 244.63 instead of 244.08. That was the error. But after correcting that, it's 999.90, so difference is 0.10.
0.1

The answer is now correct! The trace shows the extra work that made the difference:

• It restates the task: compare the $1,000.00 parent with the total of the child transactions.
• It starts by adding the child amounts instead of jumping straight to an answer.
• It notices that an intermediate sum is wrong.
• It recomputes the total another way to check itself.
• It lands on $999.90 for the children.
• It subtracts that from $1,000.00 and returns the correct difference: 0.1.

Instead of jumping to 0.0, thinking mode gives the model room to work through the sum and find the missing ten cents.

The second failure is about operation order. The model has to divide the invoice into installments before applying the processing fee.

installment_fees = """An invoice of $4,800.00 was paid in 3 equal installments.
Each installment was charged a 2.5% processing fee.
What were the total processing fees, to 2 decimal places?"""

print(ask(installment_fees))

360.0

The answer is wrong. The invoice is paid in three installments, so the fee should be calculated after the split:

• $4,800 / 3 = $1,600 per installment
• $1,600 × 2.5% = $40 fee per installment
• $40 × 3 = $120 total fees

The model probably came up with the wrong answer because it applied the fee to the full invoice three times: $4,800 × 2.5% × 3 = $360.

The speed is the problem here. It gets to a plausible formula quickly, but skips the slower intermediate step: calculate the installment amount first.

Now, let’s turn on thinking mode and see if the answer improves:

print(ask(installment_fees, think=True, trace_lines=4))

Okay, let's see. The problem is about an invoice of $4,800.00 paid in 3 equal installments, each with a 2.5% processing fee. I need to find the total processing fees to two decimal places.
First, I need to figure out what each installment amount is before the processing fee. Since it's 3 equal installments, I should divide the total invoice by 3. So, $4,800 divided by 3. Let me calculate that: 4800 / 3 = 1600. So each installment is $1,600 before the fee.
...
Another way: Total fees = 3 * (4800/3 * 0.025) = 3 * (1600 * 0.025) = 3 * 40 = 120. Yep, same result.
So the total processing fees are $120.00. To two decimal places, that's 120.00.
120.0

This time the model gets the operation order right:

• It divides $4,800 by 3 before applying the fee.
• It calculates the fee on one $1,600 installment.
• It multiplies the $40 fee by 3 installments.
• It checks the same calculation another way before returning 120.0.

Thinking mode gets it right because it takes time to work through the problem instead of jumping to the most obvious formula.

Single-Step Problems: Where It Does Not

Single-step problems are different: the answer comes from one direct calculation, with no intermediate value to carry forward. Here is a percentage example:

percent_of_deposit = """A deposit of $1,250.00 funded a single charge of $100.00.
What percentage of the deposit was that charge?
Answer to 2 decimal places."""

print(f"answer: {ask(percent_of_deposit)}")
print(f"time: {ask.last_seconds:.2f}s")

answer: 8.0
time: 0.40s

Plain mode gets the right answer in 0.40s. Thinking mode gives the same answer in 6.90s, about 17x slower:

print(f"answer: {ask(percent_of_deposit, think=True)}")
print(f"time: {ask.last_seconds:.2f}s")

answer: 8.0
time: 6.90s

The other four single-step prompts also show the same pattern: both models get the right answer, but plain mode is faster, and thinking mode is slower.

This shows that for one-step tasks, plain mode is the better default: it gives the same answer but with much less latency.

To view the full results, with the exact prompt and both models’ answers for each, see the full results file.

Final Thoughts

Here is the summary of the benchmark results:

The decision whether to use thinking mode comes down to one question: is this a one-step answer, or does the model need to carry state across steps?

Here is the rule of thumb in more concrete terms:

In short, start with plain mode for obvious one-step tasks and thinking mode for obvious multi-step tasks. When the boundary is unclear, test both on a small set of real failures before deciding which default to use.

When thinking mode does not help, prompt optimization is the next lever. The DSPy guide covers that path for classifier-style LLM tasks.

Related Tutorials

• Structured Output Tools for LLMs: enforcing the JSON schema this article relies on for gradeable answers.
• Run Private AI Workflows with LangChain and Ollama: the local setup this runs on.
• Build a Private Email Q&A System with Local and Cloud LLMs: a hybrid architecture for deciding what stays local and what gets escalated.

📚 Want to go deeper? My book shows you how to build data science projects that actually make it to production. Get the book →