Instructions to use ThakiCloud/MetisRCA-E4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ThakiCloud/MetisRCA-E4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ThakiCloud/MetisRCA-E4B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ThakiCloud/MetisRCA-E4B")
model = AutoModelForMultimodalLM.from_pretrained("ThakiCloud/MetisRCA-E4B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use ThakiCloud/MetisRCA-E4B with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ThakiCloud/MetisRCA-E4B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ThakiCloud/MetisRCA-E4B:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf ThakiCloud/MetisRCA-E4B:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf ThakiCloud/MetisRCA-E4B:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ThakiCloud/MetisRCA-E4B:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf ThakiCloud/MetisRCA-E4B:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ThakiCloud/MetisRCA-E4B:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ThakiCloud/MetisRCA-E4B:Q4_K_M

Use Docker

docker model run hf.co/ThakiCloud/MetisRCA-E4B:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use ThakiCloud/MetisRCA-E4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ThakiCloud/MetisRCA-E4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ThakiCloud/MetisRCA-E4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ThakiCloud/MetisRCA-E4B:Q4_K_M

SGLang

How to use ThakiCloud/MetisRCA-E4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ThakiCloud/MetisRCA-E4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ThakiCloud/MetisRCA-E4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ThakiCloud/MetisRCA-E4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ThakiCloud/MetisRCA-E4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use ThakiCloud/MetisRCA-E4B with Ollama:
```
ollama run hf.co/ThakiCloud/MetisRCA-E4B:Q4_K_M
```
Unsloth Desktop

How to use ThakiCloud/MetisRCA-E4B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ThakiCloud/MetisRCA-E4B:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ThakiCloud/MetisRCA-E4B:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use ThakiCloud/MetisRCA-E4B with Docker Model Runner:
```
docker model run hf.co/ThakiCloud/MetisRCA-E4B:Q4_K_M
```

Lemonade

How to use ThakiCloud/MetisRCA-E4B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ThakiCloud/MetisRCA-E4B:Q4_K_M

Run and chat with the model

lemonade run user.MetisRCA-E4B-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use ThakiCloud/MetisRCA-E4B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ThakiCloud/MetisRCA-E4B:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ThakiCloud/MetisRCA-E4B:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use ThakiCloud/MetisRCA-E4B with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf ThakiCloud/MetisRCA-E4B:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "ThakiCloud/MetisRCA-E4B:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MetisRCA-E4B

Version 1.0.0. Fine-tune of nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 published as ThakiCloud/MetisRCA-E4B.

Korean card: MODEL_CARD.ko.md

MetisRCA-E4B is a small incident-diagnosis model that reads a symptom, a list of runbook candidate causes and read-only observations, and answers with one JSON object: the root cause it can prove, the observation ids that decide it, or INSUFFICIENT_EVIDENCE with the next read-only tool to run. It is strong at picking the deciding evidence and at abstaining. On mechanisms it never saw in training it is below Qwen3.5-4B (Cloud-OpsBench like-for-like 0.783 vs 0.831) and it over-abstains on about 15% of determinable unseen-mechanism rows. An optional inference-time abstain policy (v4-calibrated, same weights, abstain_policy.py) lifts never-seen-mechanism top-1 to 0.824 (Cloud-OpsBench like-for-like 0.828, one row behind Qwen's 0.831) and cuts over-abstention to 9.4% with no rise in fabricated causes.

What it does

One call per diagnosis state. The system prompt fixes the contract; the user message carries the symptom, the candidate causes (cause_id: description) and the observations gathered so far, each with an id. The model replies with exactly one JSON object {"root_cause", "evidence_ids", "next_tool", "confidence_state"}. When the observations do not determine the cause it must answer INSUFFICIENT_EVIDENCE and name the next tool, so an agent loop can fetch more evidence and call it again. Labels come from benchmark ground truth and rules, not from a language model.

Intended use

Evidence-selection and abstention layer inside an incident-diagnosis agent loop, with a human or a stronger model reviewing the verdict.
Research on small-model abstention and evidence grounding in root-cause analysis.
Starting point for your own fine-tune on a different runbook or telemetry format.

How to use

Prompt format

The model uses the base checkpoint's chat template with thinking disabled (enable_thinking=False). The rendered prompt is exactly:

<bos><|turn>system
You are an incident diagnosis assistant for a Kubernetes/GPU/DB AI platform. You get a symptom, the runbook candidate causes (cause_id: description) and read-only observations, each with an id. Reply with ONLY a JSON object {"root_cause": <one listed cause_id or "INSUFFICIENT_EVIDENCE">, "evidence_ids": [ids of the observations that decide it], "next_tool": <read-only tool to inspect next when unresolved, else null>, "confidence_state": "RESOLVED" | "UNRESOLVED"}. Decide only from the observations; if they do not determine the cause, answer INSUFFICIENT_EVIDENCE and name the next tool.<turn|>
<|turn>user
Symptom: Train Ticket: alert raised at t0 on the ts-ui-dashboard entry service. Since t0: p90 latency 0.5069 -> 0.6348.
Candidates:
- ts-train-service:mem: memory leak / memory stress in the ts-train-service container
- ts-auth-service:f2: code defect in ts-auth-service: missing parameters
- ts-train-service:disk: disk I/O stress on the ts-train-service container
- ts-order-service:f1: code defect in ts-order-service: incorrect parameter values
- ts-train-service:cpu: CPU hog in the ts-train-service container
Observations:
[e1] (query_resource_metrics)
service: ts-auth-service | window: 10 min before vs 10 min after alert
metric       |     before |      after | z
cpu          |     14.601 |     15.076 | z=0.3
mem          |  2.375e+08 |  2.395e+08 | z=0.2
diskio       |    52699.3 |    49751.0 | z=-0.2
socket       |     11.060 |     11.300 | z=0.3
[e2] (search_logs)
service: ts-auth-service | log lines before 3356 after 3312
metric       |     before |      after | z
error_lines  |          0 |          0 | z=0.0
[e3] (search_logs)
service: ts-order-service | log lines before 3434 after 3824
metric       |     before |      after | z
error_lines  |          0 |          0 | z=0.0<turn|>
<|turn>model

Output format: one compact JSON object: {"root_cause": <listed cause_id or "INSUFFICIENT_EVIDENCE">, "evidence_ids": [observation ids], "next_tool": <read-only tool or null>, "confidence_state": "RESOLVED"|"UNRESOLVED"}.

Call the model once per diagnosis state. When it answers INSUFFICIENT_EVIDENCE, run the named read-only tool, append the new observation to the user message and call again.

v4-calibrated (optional, transformers only): pass CalibratedAbstain from abstain_policy.py as a logits processor with greedy decoding. It forbids the abstain answer when the model prefers it by a logit margin below 1.5 at the first root_cause token, so the model names its best listed cause instead. inference_example.py runs both modes on example_incident.json. The policy changes no weights; GGUF runs give greedy v4 behavior.

Transformers + PEFT adapter

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16"
tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "ThakiCloud/MetisRCA-E4B", subfolder="adapter").eval()

SYSTEM = "You are an incident diagnosis assistant for a Kubernetes/GPU/DB AI platform. You get a symptom, the runbook candidate causes (cause_id: description) and read-only observations, each with an id. Reply with ONLY a JSON object {\"root_cause\": <one listed cause_id or \"INSUFFICIENT_EVIDENCE\">, \"evidence_ids\": [ids of the observations that decide it], \"next_tool\": <read-only tool to inspect next when unresolved, else null>, \"confidence_state\": \"RESOLVED\" | \"UNRESOLVED\"}. Decide only from the observations; if they do not determine the cause, answer INSUFFICIENT_EVIDENCE and name the next tool."
USER = "Symptom: Train Ticket: alert raised at t0 on the ts-ui-dashboard entry service. Since t0: p90 latency 0.5069 -> 0.6348.\nCandidates:\n- ts-train-service:mem: memory leak / memory stress in the ts-train-service container\n- ts-auth-service:f2: code defect in ts-auth-service: missing parameters\n- ts-train-service:disk: disk I/O stress on the ts-train-service container\n- ts-order-service:f1: code defect in ts-order-service: incorrect parameter values\n- ts-train-service:cpu: CPU hog in the ts-train-service container\nObservations:\n[e1] (query_resource_metrics)\nservice: ts-auth-service | window: 10 min before vs 10 min after alert\nmetric       |     before |      after | z\ncpu          |     14.601 |     15.076 | z=0.3\nmem          |  2.375e+08 |  2.395e+08 | z=0.2\ndiskio       |    52699.3 |    49751.0 | z=-0.2\nsocket       |     11.060 |     11.300 | z=0.3\n[e2] (search_logs)\nservice: ts-auth-service | log lines before 3356 after 3312\nmetric       |     before |      after | z\nerror_lines  |          0 |          0 | z=0.0\n[e3] (search_logs)\nservice: ts-order-service | log lines before 3434 after 3824\nmetric       |     before |      after | z\nerror_lines  |          0 |          0 | z=0.0"
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": USER},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
                               return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Merged BF16

The LoRA delta is merged into the base weights (BF16).

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("ThakiCloud/MetisRCA-E4B")
model = AutoModelForCausalLM.from_pretrained("ThakiCloud/MetisRCA-E4B", dtype=torch.bfloat16, device_map="auto").eval()

SYSTEM = "You are an incident diagnosis assistant for a Kubernetes/GPU/DB AI platform. You get a symptom, the runbook candidate causes (cause_id: description) and read-only observations, each with an id. Reply with ONLY a JSON object {\"root_cause\": <one listed cause_id or \"INSUFFICIENT_EVIDENCE\">, \"evidence_ids\": [ids of the observations that decide it], \"next_tool\": <read-only tool to inspect next when unresolved, else null>, \"confidence_state\": \"RESOLVED\" | \"UNRESOLVED\"}. Decide only from the observations; if they do not determine the cause, answer INSUFFICIENT_EVIDENCE and name the next tool."
USER = "Symptom: Train Ticket: alert raised at t0 on the ts-ui-dashboard entry service. Since t0: p90 latency 0.5069 -> 0.6348.\nCandidates:\n- ts-train-service:mem: memory leak / memory stress in the ts-train-service container\n- ts-auth-service:f2: code defect in ts-auth-service: missing parameters\n- ts-train-service:disk: disk I/O stress on the ts-train-service container\n- ts-order-service:f1: code defect in ts-order-service: incorrect parameter values\n- ts-train-service:cpu: CPU hog in the ts-train-service container\nObservations:\n[e1] (query_resource_metrics)\nservice: ts-auth-service | window: 10 min before vs 10 min after alert\nmetric       |     before |      after | z\ncpu          |     14.601 |     15.076 | z=0.3\nmem          |  2.375e+08 |  2.395e+08 | z=0.2\ndiskio       |    52699.3 |    49751.0 | z=-0.2\nsocket       |     11.060 |     11.300 | z=0.3\n[e2] (search_logs)\nservice: ts-auth-service | log lines before 3356 after 3312\nmetric       |     before |      after | z\nerror_lines  |          0 |          0 | z=0.0\n[e3] (search_logs)\nservice: ts-order-service | log lines before 3434 after 3824\nmetric       |     before |      after | z\nerror_lines  |          0 |          0 | z=0.0"
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": USER},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
                               return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

llama.cpp (GGUF)

File	Quantization	Size
`MetisRCA-E4B-Q4_K_M.gguf`	Q4_K_M, imatrix from training rows (recommended)	5.34 GB (5335295200 bytes)
`MetisRCA-E4B-Q4_K_M-NV.gguf`	Q4_K_M with NVIDIA's recipe (Q8_0 token embeddings, BF16 per-layer projection), imatrix	6.18 GB (6180447456 bytes)
`MetisRCA-E4B-Q8_0.gguf`	Q8_0	8.03 GB (8031245280 bytes)

llama-server -m MetisRCA-E4B-Q4_K_M.gguf -c 8192 -ngl 99 --port 8080

Chat completions apply the template embedded in the GGUF; disable thinking per request:

curl http://localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{"messages": [{"role": "system", "content": "You are an incident diagnosis assistant for a Kubernetes/GPU/DB AI platform. You get a symptom, the runbook candidate causes (cause_id: description) and read-only observations, each with an id. Reply with ONLY a JSON object {\"root_cause\": <one listed cause_id or \"INSUFFICIENT_EVIDENCE\">, \"evidence_ids\": [ids of the observations that decide it], \"next_tool\": <read-only tool to inspect next when unresolved, else null>, \"confidence_state\": \"RESOLVED\" | \"UNRESOLVED\"}. Decide only from the observations; if they do not determine the cause, answer INSUFFICIENT_EVIDENCE and name the next tool."}, {"role": "user", "content": "Symptom: Train Ticket: alert raised at t0 on the ts-ui-dashboard entry service. Since t0: p90 latency 0.5069 -> 0.6348.\nCandidates:\n- ts-train-service:mem: memory leak / memory stress in the ts-train-service container\n- ts-auth-service:f2: code defect in ts-auth-service: missing parameters\n- ts-train-service:disk: disk I/O stress on the ts-train-service container\n- ts-order-service:f1: code defect in ts-order-service: incorrect parameter values\n- ts-train-service:cpu: CPU hog in the ts-train-service container\nObservations:\n[e1] (query_resource_metrics)\nservice: ts-auth-service | window: 10 min before vs 10 min after alert\nmetric       |     before |      after | z\ncpu          |     14.601 |     15.076 | z=0.3\nmem          |  2.375e+08 |  2.395e+08 | z=0.2\ndiskio       |    52699.3 |    49751.0 | z=-0.2\nsocket       |     11.060 |     11.300 | z=0.3\n[e2] (search_logs)\nservice: ts-auth-service | log lines before 3356 after 3312\nmetric       |     before |      after | z\nerror_lines  |          0 |          0 | z=0.0\n[e3] (search_logs)\nservice: ts-order-service | log lines before 3434 after 3824\nmetric       |     before |      after | z\nerror_lines  |          0 |          0 | z=0.0"}], "temperature": 0, "chat_template_kwargs": {"enable_thinking": false}}'

The reported GGUF numbers were measured by sending token ids rendered with the Hugging Face chat template, so the prompt is byte-identical to the BF16 run. If you call /completion with a raw string, check that the server does not add a second <bos>.

Evaluation

Greedy decoding on a held-out test split, same prompts for every arm.

Test split: held-out diagnosis states: a seen split (345 rows) whose fault types and mechanisms occur in training, and an unseen split (1566 rows) whose benchmark fault types or mechanisms do not. Evaluation feeds the gold observation chain for each state and scores one reply per state.

Arm (model)	n	Never-seen mechanisms: top-1	Never-seen mechanisms, Cloud-OpsBench: top-1	Never-seen mechanisms: over-abstain rate (lower is better)	Never-seen mechanisms: correct-abstain rate	Seen mechanisms: top-1
Bolt base, zero-shot (nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16)	v4 test_unseen 1566 / test_seen 345	0.7303	0.8089	0.1628	0.3077	0.4986
MetisRCA-E4B (nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16)	v4 test_unseen 1566 / test_seen 345	0.7837	0.8244	0.1527	0.7872	0.9695
Qwen3.5-4B, zero-shot (Qwen/Qwen3.5-4B)	v4 test_unseen 1566 / test_seen 345	0.8142	0.8578	0.0127	0.0333	0.5623

Run conditions:

Bolt base, zero-shot: greedy decode, enable_thinking=False, transformers 5.19.0; 1x B200
MetisRCA-E4B: greedy decode, enable_thinking=False, transformers 5.19.0; 1x B200
Qwen3.5-4B, zero-shot: greedy decode, enable_thinking=False, transformers 5.19.0; 1x B200

Like-for-like on never-seen mechanisms (determinable rows)

Variant	n	Cloud-OpsBench	RCAEval (rule-solvable, not a headline)
Bolt base, zero-shot	360 Cloud-OpsBench rows	275/360=0.764	12/33
MetisRCA-E4B	360 Cloud-OpsBench rows	282/360=0.783	26/33
Qwen3.5-4B, zero-shot	360 Cloud-OpsBench rows	299/360=0.831	21/33
MetisRCA-E4B v4-calibrated (inference policy)	360 Cloud-OpsBench rows	298/360=0.828	not re-scored

Measured with rows whose normalized mechanism does not occur in training and whose observations determine the cause. Caveats: On Cloud-OpsBench MetisRCA is 4.8 points below Qwen3.5-4B and 1.9 points above the Bolt base. RCAEval rows are rule-solvable by construction and are not a headline (RCAEval counts are out of 33 rows). With the v4-calibrated abstain policy the same weights reach 298/360, 1 row (0.3 points) below Qwen3.5-4B.

Whole unseen split (1566 rows)

Variant	n	Top-1	Evidence F1	Fabricated-cause rate (lower is better)
Bolt base, zero-shot	1566 rows	0.6194	0.6109	0.1373
MetisRCA-E4B	1566 rows	0.8727	0.8183	0.0026
Qwen3.5-4B, zero-shot	1566 rows	0.6936	0.5719	0.0728

Measured with hardware B200, 1 GPU; all rows of the held-out unseen split (benchmark fault types or mechanisms absent from training); includes abstain rows. Caveats: The Qwen row comes from the ledger row that reports root_cause_top1, evidence_f1 and fabricated_cause_rate on the same 1566 rows; the other arms report the same quantities as unseen_all_top1, evidence_f1_unseen and fabricated_unseen.

v4-calibrated: inference-time abstain policy (same weights)

Variant	n	Whole unseen split: top-1	Top-1	Never-seen mechanisms: over-abstain (lower is better)	Never-seen mechanisms: correct abstain	Fabricated-cause rate (lower is better)
MetisRCA-E4B v4 (greedy)	1566 rows	0.8727	0.7837	0.1527	0.7872	0.0026
MetisRCA-E4B v4-calibrated	1566 rows	0.8966	0.8244	0.0941	0.7359	0.0026

Measured with hardware B200, 1 GPU; whole unseen split (1566 rows); top-1, over-abstain and correct-abstain columns are on its never-seen-mechanism subset (783 rows); threshold t=1.5 chosen on the validation split only, applied once. Caveats: Policy: at the first root_cause token, forbid the abstain token when its logit margin over the best other token is below 1.5 (abstain_policy.py). Pre-registered gates: top-1 on never-seen mechanisms >= 0.814 PASS (0.824), over-abstain <= 5% FAIL (9.4%), correct abstain >= 70% PASS (73.6%), fabricated <= 7.3% PASS (0.26%). The first-token margin sign matched the actual abstain output on 100% of unseen rows. GGUF/llama.cpp runs reproduce the greedy v4 row, not this one. Added latency was not measured; the policy adds one comparison at one decode step and no second pass.

Merged weights and GGUF on the seen test split (345 rows)

Variant	n	Top-1	Evidence F1	Correct abstain
Merged BF16	345 rows	0.9697	0.9605	0.9944
GGUF Q8_0	345 rows	0.9697	0.9551	0.9833
GGUF Q4_K_M (imatrix)	345 rows	0.9697	0.9602	0.9833
GGUF Q4_K_M-NV	345 rows	0.9697	0.9568	0.9833

Measured with hardware B200, 1 GPU; test_seen; GGUF runs feed the Hugging Face chat-template token ids to llama-server (llama.cpp 344be0a), greedy. Caveats: All variants keep top-1; quantization costs about one point of correct-abstain (2 of 180 abstain rows). Leak probe (130 prompts asking for internal identifiers): 0 hits on merged BF16, Q4_K_M and Q4_K_M-NV. Single-stream decode on B200 measured 202.8 tok/s for one Q4 variant; per-variant speed and memory were not retained.

Training

Method: LoRA fine-tune of the base model, loss on assistant turns only. Values below are copied from the training record.

Setting	Value
LoRA rank r	16 (from training summary)
LoRA alpha	32 (from training summary)
Learning rate	0.0002
Epochs	2
Target modules	attn+mlp (from training summary)
Gradient accumulation	8 (from training summary)
Max sequence length	4096 (from training summary)
Optimizer steps	1091
Peak GPU memory (GiB)	27.52 (from training summary)
Wall time (s)	4598.5 (from training summary)

Data

Training rows combine incidents from RCAEval (MIT license) and Cloud-OpsBench (MIT license) with internal synthetic playbook rows. Every row is a diagnosis state: the same incident is shown at several evidence levels (not yet enough evidence, near miss, sufficient), so the model learns when to stop and when to ask for another tool. Labels come from the benchmark ground truth and code rules. Train, validation and test splits are disjoint by incident group, and the unseen test split holds benchmark fault types or mechanisms that never occur in training.

The training data is internal and is not released. This repository contains no dataset files.

Split sizes (rows): train 4362, val 379, test_seen 345, test_unseen 1566.

Limitations

On mechanisms that never occur in training, greedy MetisRCA-E4B is below Qwen3.5-4B (Cloud-OpsBench like-for-like on determinable rows 0.783 vs 0.831; 1.9 points above the Bolt base). With the v4-calibrated policy it reaches 0.828 on the same rows, still 1 row below Qwen. Do not pick it for open-world diagnosis of unfamiliar failure types.
It over-abstains: on never-seen mechanisms the greedy model answers INSUFFICIENT_EVIDENCE on about 15% of rows that the observations determine (9.4% with the v4-calibrated policy, which still misses its pre-registered 5% gate). Treat an abstention as a request for more evidence, not as a verdict.
The v4-calibrated numbers need the transformers logits processor in abstain_policy.py; GGUF files under llama.cpp behave like greedy v4.
RCAEval rows are rule-solvable by construction, so RCAEval scores say little about diagnosis skill and are not a headline.
The runbook candidates and observation formats come from two public benchmarks plus internal synthetic playbook rows; a different telemetry format needs re-evaluation.
Do not let the model trigger remediation on its own. It reads observations; it does not verify that an action is safe.
Behavior outside the prompt format above (other output schemas, thinking mode, very long observation lists) was not evaluated.

License and attribution

Model materials: governed by the OpenMDW License Agreement, version 1.1 (LICENSE.OpenMDW-1.1, https://openmdw.ai/license/1-1/). This repository is a derivative of nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 and ships a copy of the agreement.
Origin: the NVIDIA base model was post-trained on Google Gemma 4, whose use is governed by the Apache License, Version 2.0 (LICENSE.Apache-2.0-Gemma).
NVIDIA's governing-terms notice is retained verbatim in NOTICE.md, together with the origin chain and a statement of ThakiCloud's modifications.
The fine-tuned weights add ThakiCloud's LoRA modifications; they do not change the terms above.