Instructions to use wesjos/mario-html-v2-9B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use wesjos/mario-html-v2-9B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf wesjos/mario-html-v2-9B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf wesjos/mario-html-v2-9B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf wesjos/mario-html-v2-9B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use wesjos/mario-html-v2-9B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "wesjos/mario-html-v2-9B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "wesjos/mario-html-v2-9B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Ollama
How to use wesjos/mario-html-v2-9B-GGUF with Ollama:
```
ollama run hf.co/wesjos/mario-html-v2-9B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use wesjos/mario-html-v2-9B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "wesjos/mario-html-v2-9B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use wesjos/mario-html-v2-9B-GGUF with Docker Model Runner:
```
docker model run hf.co/wesjos/mario-html-v2-9B-GGUF:Q4_K_M
```

Lemonade

How to use wesjos/mario-html-v2-9B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.mario-html-v2-9B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use wesjos/mario-html-v2-9B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use wesjos/mario-html-v2-9B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf wesjos/mario-html-v2-9B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "wesjos/mario-html-v2-9B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

🍄 Mario-HTML-v2-9B

V2: less html tag in Cot than v1.

This is the GGUF build of wesjos/mario-html-v2-9B. The weights here are quantized; the card below and the demos are that model's, unchanged.

📦 GGUF files

File	Quant	Size	Notes
`Qwen3.5-9B-HTML-v5-Q8_0.gguf`	Q8_0	8.87 GiB	near-lossless
`Qwen3.5-9B-HTML-v5-Q4_K_M.gguf`	Q4_K_M	5.24 GiB	smaller, faster
`mmproj-Qwen3.5-9B-BF16.gguf`	BF16	0.86 GiB	vision projector — required for image input

Text → HTML: one quant is enough, e.g. llama serve -hf wesjos/mario-html-v2-9B-GGUF:Q8_0 (the tag form the Hub generates for this repo).
Text + image → HTML: add the projector explicitly — --mmproj mmproj-Qwen3.5-9B-BF16.gguf (it does follow the mmproj-* naming, but keeping it explicit costs nothing).
Manual form, equivalent: llama-server -m Qwen3.5-9B-HTML-v5-Q8_0.gguf --mmproj mmproj-Qwen3.5-9B-BF16.gguf -ngl 99 -c 65536.
Chat template default system: none (zero-prompt native).
If you reuse the 4B repo's WebUI config, its model_path points at the 4B quant — set it to ./Qwen3.5-9B-HTML-v5-Q8_0.gguf here.


Base	`Qwen/Qwen3.5-9B` · 9.65 B (bf16) · 17.98 GiB
Architecture	`Qwen3_5ForConditionalGeneration` · 32 layers (24 linear + 8 full attention, interval 4) · hidden 4096
Context	262,144 native
Modalities	text → HTML, text + image → HTML

🖼️ Demo

All shots are the WebUI (shipped with the mario-html-v2-4B repo, theme_preset: editorial) with the CoT toggle switched on — its webui/config.json ships enable_thinking: false, which hides the reasoning panel entirely, so flip it if you want to see the traces. The panel above each answer is the raw <think> stream; the page below it is the unmodified render.

Microservices vs. Monolith — comparison grid

Compare Microservices vs Monolith architecture in a modern comparison grid (two-card grid, callouts, metric tiles and a head-to-head table; reasoning panel collapsed)

Architecture comparison demo: a verdict callout, a two-card Monolith vs Microservices grid with BEST FOR / RISK / OPS / TRADE-OFF bullets, metric tiles, and a head-to-head comparison table

Step-by-step math, CoT expanded

Solve the linear equation: 2x + 5 = 17 step by step (the <think> stream is pure plain-text planning — zero HTML tags — while the answer renders as cards, code blocks and a stepper)

Math demo: reasoning panel expanded above the answer, showing plain-text planning followed by a rendered step-by-step solution with code blocks and a golden-rule callout

Vision input

llama photo + what is this (image paste via Ctrl+V, answered as a llama-vs-alpaca breakdown)

Vision demo: an uploaded llama photo with the question 'what is this', answered with a two-card llama-vs-alpaca comparison, a feature table and characteristic tiles

📤 Output Contract

One root element; everything else is an optional child. Class list verified against webui/static/theme.css (shipped with the mario-html-v2-4B repo):

Component	Markup
Root	`<div class="ai-view">`
Summary	`<div class="summary">` · `<div class="summary-card">` · `<p class="lead">`
Card	`<div class="card">`
Grid	`<div class="grid grid-2">` · `<div class="grid grid-3">`
Metrics	`<div class="metric-group"><div class="metric-card"><span class="metric-label">…</span><span class="metric-value">…</span><span class="metric-sub">…</span></div></div>`
Stepper	`<div class="stepper"><div class="step-item">…</div></div>`
Callout	`<div class="callout callout-info\|warning\|success\|danger"><div class="callout-header">…</div></div>`
Badge	`<span class="badge badge-info\|success\|warning\|danger\|accent">`
Table	`<div class="table-container"><table class="modern-table">…</table></div>`
Feature list	`<div class="feature-list"><div class="feature-item">…</div></div>`
Code block	`<div class="code-box"><div class="code-header"><span class="code-title">…</span></div>…</div>`

Raw sample (message.content verbatim, v2 zero-system-prompt greedy)

<div class="ai-view">
<h3>为什么天空是蓝色的？</h3>
<div class="summary">
<p class="lead">阳光中的蓝光被大气中的分子"散射"到四面八方，而我们的眼睛对蓝光更敏感，所以我们看到天空是蓝色的。</p>
</div>
<div class="card">
<h4>三句话解释</h4>
<div class="step-list"><div class="step-item">…</div></div>
</div>
</div>

The <think> trace that precedes this sample contains zero HTML tags — pure plain-text planning ("a summary lead, then a three-item stepper..."). Two classes in the sample are content-only: step-list and ai-view-header are not in theme.css, so they render with default block styling — define your own if you want tighter control. (step-item itself is styled.)

🎨 WebUI

The WebUI ships with the sibling 4B repo — clone wesjos/mario-html-v2-4B, point webui/config.json at your serving endpoint for this model, and:

cd webui
pip install -r requirements.txt
python server.py --port 8791

Open http://localhost:8791. Dual-stream rendering (collapsible <think> + live HTML), clipboard image paste (Ctrl+V), nine theme presets (default · apple · notion · cyber · editorial · cards_rounded · glassmorphism · soft_shadows · compact) plus live custom CSS, and a raw-source / copy-HTML view per message.

⚠️ Notes

The output is HTML, not Markdown. Feed it a Markdown renderer and you get raw tags.
Serve with a clear/absent system prompt — the zero-system-prompt path is the trained one.
The model picks from the class vocabulary above; arbitrary new CSS classes are not a contract. Custom theme.css is the supported way to restyle.
Arithmetic is reliable but not bank-grade — check long decimal chains.
--reasoning-parser qwen3 needs a recent vLLM.