neoned71/qwen3-0.6b-math-tutor-4bit

Qwen3-0.6B — math tutor (4-bit)

A tool-calling maths tutor built on Qwen/Qwen3-0.6B, fine-tuned locally with MLX on Apple Silicon.

The model does not do arithmetic. It turns a question into a single expression and hands it to a calculator tool:

<tool_call>
{"name": "calculate", "arguments": {"expression": "40*0.75*1.2"}}
</tool_call>

It also abstains when a question needs no calculation, so it does not call the calculator for "Who wrote Hamlet?" or "What was the population of Paris in 2019?" — including questions that contain numbers.

Training

• Started from the tool-calling fine-tune of Qwen3-0.6B, continued (not restarted)
• Data: generated, not scraped — every answer is computed by the calculator that the model is trained to call, so no arithmetic errors enter the labels
• 2 epochs, lr 1e-05, effective batch 16
• LoRA rank 16, scale 32, on the last 16 of 28 blocks
• Trainable params: 5,767,168 / 596,050,000 (0.97%)

How this model was evaluated

Three separate held-out sets, because a single number hides too much:

set	what it measures	rows
valid	in-distribution: held-out questions, training templates	230
valid_phrasing	generalisation: the same reasoning patterns, paraphrases never trained on	120
valid_novel	the model's reach: problem families absent from training	90

The valid_phrasing set is the one that matters. An earlier version of this model's training data used a single phrasing and a single expression per reasoning pattern; that model scored 100% on valid and was worse than its own starting point on unseen problem shapes. The training data here carries several paraphrases and several equivalent expressions per pattern, so the surface form of a question carries no information about the answer.

Results

Generalisation set (valid_phrasing, n=120)

The same reasoning patterns as training, worded in ways that were never trained on. This is the number to look at.

slice	n	answer correct
direct arithmetic	43	88.4%
word problems	37	97.3%
multi-step	40	80.0%

Overall on this set: format valid 98.3%, tool name correct 98.3%, invalid expression 0.0%, refused a calculation 1.7%, hallucinated numbers 0.0% — the rate at which the model stated a number in prose that the calculator never returned.

"answer correct" evaluates the model's own expression with the calculator and compares the result, so a mathematically equivalent expression counts as correct. "expression exact" (not shown) compares the string, which is a consistency measure rather than a correctness one.

In-distribution set (valid, n=230)

slice	n	format valid	answer correct
direct arithmetic	60	96.7%	96.7%
word problems	60	100.0%	96.7%
multi-step	60	100.0%	93.3%
non-maths (no call is correct)	50	—	100.0% correct

Against the un-tuned tool-calling model

The starting point — the tool-calling fine-tune, before any maths training — scored on the same sets. This is the comparison that matters: a maths fine-tune that only improves its own training distribution is not a maths model.

set	tool-calling model	this tutor
in-distribution	43.3%	95.6%
generalisation	37.5%	88.3%
reach (out-of-training families)	25.0%	53.3%

Limitations

• One expression per question. A multi-step question is decomposed into a single expression, not a chain of tool calls. The model never receives an intermediate result.
• Arithmetic only. No symbolic algebra, no equation solving, no calculus. (Fractions, rectangle area and unit conversion do work, but they are extrapolations, not trained capabilities.)
• The training data is generated from templates. The model is tuned to that style of problem rather than to arbitrary real-world maths. The valid_novel set (fractions, geometry, unit conversion, algebra, simple interest) exists to bound that.
• The evaluation sets are small. A few dozen examples per slice; one or two examples of difference is noise.

Where it stops

On problem families that were never in the training data (valid_novel, n=90):

family	n	answer correct
algebra	16	0.0%
compound	11	9.1%
fraction of	8	100.0%
geometry	11	100.0%
two leg	9	77.8%
unit conversion	5	100.0%

Fractions, rectangle areas, unit conversions and two-leg journeys generalise better than expected. Solving for x does not work at all — the model emits the equation itself (5x+11=21) rather than an expression, which the calculator cannot evaluate. Simple interest is largely missed.

• 1.7% of calculations are refused and 0.0% produce an expression the calculator cannot evaluate — the two dominant failure modes.

It is not a drop-in replacement for a general tool-calling model

This model was fine-tuned from a general tool-calling model, and the maths training degrades that model's general tool-calling skill. Measured on the tool-calling model's own held-out set (239 rows, arbitrary tool schemas — a different task from anything this model is for):

metric	tool-calling model	this tutor
correct function name	97.5%	86.2%
correct argument values	78.8%	58.1%
abstention correct	77.2%	49.4%
false calls	22.8%	50.6%

The cause is the training mix: only 5.9% of this model's training rows are abstentions, against 15.8% in the tool-calling model's data, so the model is pushed hard towards always emitting a call.

Use this model for maths with a calculator. Do not use it as a replacement for a general tool-calling model.

Usage (MLX)

The model only emits the call. You execute it and feed the result back — that is the point of the model, since it never does arithmetic itself.

from mlx_lm import load, generate

model, tokenizer = load("neoned71/qwen3-0.6b-math-tutor-4bit")

TOOLS = [{
    "type": "function",
    "function": {
        "name": "calculate",
        "description": "Evaluate a mathematical expression and return the numeric result.",
        "parameters": {
            "type": "object",
            "properties": {
                "expression": {
                    "type": "string",
                    "description": "The expression to evaluate, e.g. '40*0.75*1.2'.",
                }
            },
            "required": ["expression"],
        },
    },
}]

def calculate(expression: str):
    # your arithmetic here — keep it safe, do not eval() untrusted input
    ...

messages = [{"role": "user", "content": "A 40 pound jacket is 20% off. Final price?"}]
prompt = tokenizer.apply_chat_template(
    messages, tools=TOOLS, add_generation_prompt=True, tokenize=False,
    enable_thinking=False)
out = generate(model, tokenizer, prompt=prompt, max_tokens=128)
print(out)
# <tool_call>
# {"name": "calculate", "arguments": {"expression": "40*(100-20)/100"}}
# </tool_call>