Instructions to use masterofaudio2077/sid1-qwen3-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use masterofaudio2077/sid1-qwen3-4b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="masterofaudio2077/sid1-qwen3-4b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("masterofaudio2077/sid1-qwen3-4b") model = AutoModelForCausalLM.from_pretrained("masterofaudio2077/sid1-qwen3-4b", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use masterofaudio2077/sid1-qwen3-4b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "masterofaudio2077/sid1-qwen3-4b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "masterofaudio2077/sid1-qwen3-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/masterofaudio2077/sid1-qwen3-4b
- SGLang
How to use masterofaudio2077/sid1-qwen3-4b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "masterofaudio2077/sid1-qwen3-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "masterofaudio2077/sid1-qwen3-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "masterofaudio2077/sid1-qwen3-4b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "masterofaudio2077/sid1-qwen3-4b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use masterofaudio2077/sid1-qwen3-4b with Docker Model Runner:
docker model run hf.co/masterofaudio2077/sid1-qwen3-4b
sid1-qwen3-4b: a search agent trained with reinforcement learning
Qwen/Qwen3-4B fine-tuned with reinforcement learning (GRPO) to find the documents that answer a multi-hop question. It does not write an answer.
It searches a collection with dense and keyword search, reads, searches again hop by hop, and finally reports a ranked list of document ids.
The recipe follows the SID-1 technical report, scaled down to a 4B model, three public datasets and one GPU.
This is an early stop: 175 of a planned 400 training steps (see Training). It is a research artifact, not a finished replication of SID-1.
Model details
| Base model | Qwen/Qwen3-4B, 4,022,468,096 parameters, all of them trained |
| Training | RL only, no supervised fine-tuning (there is no target text to imitate): the reward is the NDCG of the reported list against the datasets' annotated supporting paragraphs |
| Format | bf16 safetensors, same architecture and tensor names as the base model (loads with AutoModelForCausalLM) |
| Language | English |
| License | Apache 2.0, as the base model |
| Developed by | masterofaudio2077 |
| Demo | Agentic Search, Trained with RL: replay recorded runs or run the model live, next to plain vector search |
| Code | The training notebook and the reference agent loop (agent.py, agent_core.py) are in the demo Space |
Uses
Direct use. Give it a question and four tools over your own collection: search_dense, search_bm25, read, report. It returns the ids of the documents needed to answer the question.
Use it as the retrieval step of a question-answering system, or to study agentic search.
Out-of-scope. It is not a chat model and does not answer questions. It was trained on one tool schema and one system prompt (below) and may not follow different ones. It was never trained on the open web, other languages, or other kinds of collections.
Bias, risks and limitations
- In-domain evaluation. Training and evaluation questions come from the same three benchmarks (MuSiQue, HotpotQA, 2WikiMultiHopQA) over Wikipedia paragraphs. Train and dev questions do not overlap, but the results do not show how well it works on other question styles or collections.
- Early stop. Stopped at step 175 of 400. The learning-rate decay phase of the schedule never started. On the 96-question checkpoints the score had flattened at about 0.66 since step 100, so more steps may or may not have helped.
- It never uses
read. During training thereadtool was called 0.0 times per trajectory; it relies on the 160-character excerpts that the searches return. Do not expect it to open documents. - Lower precision. It reports more documents than the base model (up to 20), so recall and NDCG rise while precision falls (table below).
- One run, one seed. There are no confidence intervals. The evaluation has 600 questions (200 per dataset), and the 96-question checkpoints during training are noisy.
- Cost of 4 samples. The best number (4 samples fused) needs four full runs per question.
- Inherits the biases and failure modes of the Qwen3 base model and of the Wikipedia text it searches.
How to get started
One turn of the agent with transformers. The prompt and tool schema must be exactly these, because the model was trained on them.
import json, re, torch
from transformers import AutoTokenizer, AutoModelForCausalLM
repo = "masterofaudio2077/sid1-qwen3-4b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto").eval()
TOOLS = [
{"type": "function", "function": {"name": "search_dense",
"description": "Semantic (embedding) search over the document collection. Returns the top matches as '[doc_id] title: excerpt'.",
"parameters": {"type": "object", "properties": {"query": {"type": "string", "description": "a natural-language query or a short drafted answer passage"}}, "required": ["query"]}}},
{"type": "function", "function": {"name": "search_bm25",
"description": "Keyword (BM25) search over the document collection. Best for names, titles and rare terms. Returns the top matches as '[doc_id] title: excerpt'.",
"parameters": {"type": "object", "properties": {"query": {"type": "string", "description": "keywords"}}, "required": ["query"]}}},
{"type": "function", "function": {"name": "read",
"description": "Read the full text of one document.",
"parameters": {"type": "object", "properties": {"doc_id": {"type": "string"}}, "required": ["doc_id"]}}},
{"type": "function", "function": {"name": "report",
"description": "Finish. Report every document needed to answer the question, most relevant first. Must be the only call in its turn.",
"parameters": {"type": "object", "properties": {"doc_ids": {"type": "array", "items": {"type": "string"}}}, "required": ["doc_ids"]}}},
]
SYSTEM_PROMPT = (
"You are a retrieval agent. You are given a question. Do not answer it. Find the documents in the collection that are needed to answer it "
"and report their ids, most relevant first.\n"
"Search with several different queries in the same turn (semantic and keyword), read documents when an excerpt is not enough, and follow "
"multi-step questions hop by hop. When you are done, call report once with the ids of all the documents needed. "
"You have at most 6 turns, and report must be called before they run out."
)
msgs = [{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": "When did the luxury division of the maker of the Toyopet Master change the RX 350 body style?"}]
prompt = tok.apply_chat_template(msgs, tools=TOOLS, add_generation_prompt=True, enable_thinking=True, tokenize=False)
ids = tok(prompt, return_tensors="pt", add_special_tokens=False).to(model.device)
out = model.generate(**ids, max_new_tokens=1536, do_sample=True, temperature=0.7, top_k=0, top_p=1.0,
eos_token_id=tok.convert_tokens_to_ids("<|im_end|>"))
text = tok.decode(out[0, ids.input_ids.shape[1]:], skip_special_tokens=True)
calls = [json.loads(c) for c in re.findall(r"<tool_call>\s*(.*?)\s*</tool_call>", text.rsplit("</think>", 1)[-1], re.S)]
print(calls) # a list of {"name": ..., "arguments": {...}}: several searches per turn, or a single report
To run the whole loop (up to 6 turns), run each call against your collection and feed the results back as the model's next input, formatted as [doc_id] title: excerpt lines inside <tool_response> blocks. Stop at report.
agent.py and agent_core.py in the demo Space do this, including the parsing and the exact result format.
Sampling. The model was trained and evaluated with temperature only (1.0 in training, 0.7 in evaluation), no top-k and no top-p, and generation stops at <|im_end|>.
This repository's generation_config.json is the Qwen3 default (temperature 0.6, top-k 20, top-p 0.95), so pass top_k=0, top_p=1.0 as above.
Training details
Data
Questions from the train splits of three public multi-hop QA sets, with the supporting paragraphs as gold documents:
| Dataset | Question pool |
|---|---|
| MuSiQue | all 19,938 train questions |
| HotpotQA (distractor) | 50,000 sampled |
| 2WikiMultiHopQA | 50,000 sampled |
119,938 questions in the pool; 175 steps Γ 16 questions = 2,800 of them were actually used. The search collection is the pooled, de-duplicated paragraphs of all three sets (train and dev): 578,747 paragraphs with random ids. 200 dev questions per dataset (600 in total) are held out for evaluation; none of them is a training question.
Procedure
- Algorithm: GRPO in a custom loop (Magistral style, as in SID-1): 16 questions Γ 8 attempts per step, no KL term, no importance ratio, one update per batch. A group whose attempts all get the same reward gives zero signal and is skipped. The advantage is the reward minus the group mean, divided by the group's total generated tokens (SID-1's length-bias fix).
- Rollouts: vLLM, tool results appended as tokens (the chat template is never re-applied). At most 6 turns, 1,536 tokens per turn, up to 6 parallel tool calls per turn, context cap 4,608 β 6,144 tokens (from step 100).
- Tools:
search_dense(Qwen3-Embedding-0.6B, top 4),search_bm25(bm25s, top 4),read(first 1,800 characters),report(at most 20 ids, must be the only call in its turn). - Reward: NDCG of the reported list against the gold paragraphs, (0 if it never reported), minus 0.1 for a malformed turn, minus a length penalty that grows linearly from 0 at 3,000 generated tokens to 0.1 at 7,000.
- Optimizer: Lion with 8-bit momentum and stochastic rounding, peak learning rate 5e-7 on a warmup-stable-decay schedule (40 warmup steps, constant until step 320, then linear decay to 10%), weight decay 0.1, gradient clip 1.0. Linear layers trained with FP8 matmuls (torchao) on bf16 weights; gradient checkpointing.
- Stopped at step 175 of 400, in the constant phase of the schedule, so the learning rate was still 5e-7.
| Hyperparameter | Value |
|---|---|
| Questions Γ attempts per step | 16 Γ 8 |
| Steps run / planned | 175 / 400 |
| Training temperature | 1.0 |
| Seed | 0 (one run) |
| Training-time peak GPU memory | 32.6 GiB |
Training progress
Quick checkpoints on 96 held-out dev questions (32 per dataset, 1 sample, temperature 0.7, so noisy), then the full evaluation:
| Step | 0 (base) | 25 | 50 | 75 | 100 | 125 | 150 | 175 | final (600 questions) |
|---|---|---|---|---|---|---|---|---|---|
| NDCG | 0.455 | 0.504 | 0.618 | 0.586 | 0.658 | 0.660 | 0.672 | 0.662 | 0.684 |
Evaluation
Setup. 600 held-out dev questions (200 per dataset) searched over the full 578,747-paragraph collection, temperature 0.7, context cap 8,192 tokens, at most 6 turns. "1 sample" is the first attempt; "4 samples fused" combines the four attempts' lists with reciprocal rank fusion and keeps the top 20. Metrics are computed on the reported list of at most 20 ids. The two baselines use the question itself as the query and take the top 10.
| System | NDCG | Recall | Precision | F1 |
|---|---|---|---|---|
| Qwen3-4B (base), 1 sample | 0.455 | 0.413 | 0.511 | 0.436 |
| Qwen3-4B (base), 4 samples fused | 0.604 | 0.598 | 0.513 | 0.517 |
| sid1-qwen3-4b, 1 sample | 0.684 | 0.699 | 0.356 | 0.461 |
| sid1-qwen3-4b, 4 samples fused | 0.729 | 0.781 | 0.237 | 0.355 |
| Vector search only, top 10 | 0.579 | 0.602 | n/a | n/a |
| BM25 only, top 10 | 0.508 | n/a | n/a | n/a |
Behaviour on the same questions:
| Base | sid1-qwen3-4b | |
|---|---|---|
| Well-formed tool calls | 94.9% | 99.9% |
| Ends with a report | 86.3% | 99.9% |
| Turns per question | 2.6 | 4.1 |
| Tool calls per turn | 0.96 | 1.52 |
| Generated tokens per question | 1,437 | 1,510 |
The vector-search rows retrieve 10 documents and the agent reports up to 20, so precision is not comparable across those rows; NDCG and recall are the fairer comparison. Per-dataset numbers are in the MLflow run linked below.
Environmental impact
| Hardware | 1 Γ NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GB) |
| Wall-clock | about 9.0 hours for the whole run, including the base, 25-step and final evaluations |
| Platform | Google Colab |
| Carbon emitted | not measured |
Technical specifications
- Architecture: Qwen3 dense decoder, 36 layers, tied input and output embeddings, 8 key-value heads, head dimension 128.
- Software: torch 2.14.0, vLLM 0.34.0 (rollouts), torchao 0.18.0 (FP8), transformers 5.22, bm25s 0.3.16, MLflow 3.18.0.
- Metrics and curves: the run
rl-20261120-0500in thesid1-qwen3-4b-agentic-searchexperiment on DagsHub MLflow.
Citation
@misc{sid1_qwen3_4b_2026,
title = {sid1-qwen3-4b: a search agent trained with reinforcement learning},
author = {masterofaudio2077},
year = {2026},
howpublished = {\url{https://huggingface.co/masterofaudio2077/sid1-qwen3-4b}}
}
Method: the SID-1 technical report. Base model: Qwen3.
Model card contact
Open a discussion on this repository.
- Downloads last month
- -
Model tree for masterofaudio2077/sid1-qwen3-4b
Datasets used to train masterofaudio2077/sid1-qwen3-4b
dgslibisey/MuSiQue
framolfese/2WikiMultihopQA
Space using masterofaudio2077/sid1-qwen3-4b 1
Evaluation results
- NDCG, 1 sample on MuSiQue + HotpotQA + 2WikiMultiHopQA (pooled dev, 600 questions)self-reported0.684
- NDCG, 4 samples fused (RRF) on MuSiQue + HotpotQA + 2WikiMultiHopQA (pooled dev, 600 questions)self-reported0.729
- Recall, 1 sample on MuSiQue + HotpotQA + 2WikiMultiHopQA (pooled dev, 600 questions)self-reported0.699