---
license: apache-2.0
base_model: google/gemma-4-E4B-it
tags:
  - decision-model
  - system-1
  - rlcd
  - proper-scoring-rules
  - gemma
  - classone
  - classification
pipeline_tag: text-classification
---

# classone-gemma4-e4b — ClassOne System 1 Decision Model

**[devops-thiago/classone-gemma4-e4b](https://huggingface.co/devops-thiago/classone-gemma4-e4b)** is an open-source **System 1 decision model** built on top of [`google/gemma-4-E4B-it`](https://huggingface.co/google/gemma-4-E4B-it) using the [ClassOne architecture](https://github.com/devops-thiago/class-one).

Instead of generating text token by token, ClassOne evaluates structured decisions in a **single forward pass**, returning typed, calibrated outputs with zero decoding overhead.

## Benchmark Results

### 1. JevBench Public Multi-Tier Benchmark (231 Public Tasks)

Evaluated across all 231 public tasks in [fstandhartinger/jevbench](https://github.com/fstandhartinger/jevbench):

| Tier | Tasks | Accuracy | ECE | Brier Score | Median Latency (p50) |
|---|---|---|---|---|---|
| **Easy** | 48 | **100.0%** (48/48) | **0.0009** | **0.0000** | **118.4 ms** |
| **Original** | 72 | **95.8%** (69/72) | **0.0236** | **0.0328** | **117.3 ms** |
| **Hard** | 111 | **45.9%** (51/111) | 0.3785 | 0.3500 | **253.4 ms** |
| **Overall Aggregate** | **231** | **72.7%** (168/231) | — | — | **117.3 ms** |

- **Easy Tier Sub-Breakdown:** Choice accuracy: **100.0%** (36/36); Noul policy accuracy: **100.0%** (12/12). Flawless 100%.
- **Original Tier Sub-Breakdown:** Noul accuracy: **100.0%** (24/24); Score rubrics: **100.0%** (12/12); Choice accuracy: **91.7%** (33/36).
- **Hard Tier Sub-Breakdown:** Choice accuracy: **46.3%** (31/67); Noul policy compliance: **47.4%** (18/38); Score: **33.3%** (2/6).

### 2. Hugging Face Decision Index (50 Public Multi-Domain Tasks)

| Domain | Weight | Accuracy (%) | Score |
|---|---|---|---|
| **Knowledge & Verification** | 25.8% | **100.0%** (10/10) | 100.00 pts |
| **Language & Policy Contracts** | 25.8% | **100.0%** (10/10) | 100.00 pts |
| **Retrieval & Relevance Triage** | 20.0% | **100.0%** (10/10) | 100.00 pts |
| **Tools & Execution Guardrails** | 18.3% | **90.0%** (9/10) | 90.00 pts |
| **Rubric & Quality Scoring** | 10.1% | **90.0%** (9/10) | 90.00 pts |
| **Overall Decision Index** | **100.0%** | **94.0%** (47/50) | **97.16 / 100.00** (0.0312 ECE) |

### 3. Prompt Injection Benchmark (deepset/prompt-injections, 116 Instances)

| Metric | Result |
|---|---|
| **Overall Binary Accuracy** | **63.79%** (74/116) |
| **Benign Specificity (TN)** | **96.4%** (54/56) |
| **Injection Recall (TP)** | **33.3%** (20/60) |
| **AUROC** | **0.7499** |
| **Latency p50** | **106.4 ms** |

### 4. RLCDAlignBench Alignment & Safety Evaluation (100 Instances)

Evaluated across the 10 core AI alignment failure modes (arXiv:2609.29429):

| Failure Mode / Axis | Samples (N) | AUROC | Accuracy (%) | ECE | Latency (p50) |
|---|---|---|---|---|---|
| **Refusal (Jailbreaks)** | 11 | 0.433 | **72.7%** | 0.3272 | 622.8 ms |
| **Honesty (Deception)** | 11 | **0.567** | **72.7%** | 0.2531 | 464.9 ms |
| **Reward Hacking** | 9 | **0.650** | **66.7%** | 0.3074 | 444.8 ms |
| **Faithfulness** | 9 | **0.600** | **66.7%** | 0.3005 | 440.3 ms |
| **Power Seeking** | 6 | **0.556** | **66.7%** | 0.1434 | 449.7 ms |
| **Privacy (Secret Leaks)** | 14 | **0.571** | **64.3%** | 0.3666 | 353.1 ms |
| **Concealing Uncertainty** | 14 | **0.510** | **64.3%** | 0.3784 | 288.6 ms |
| **Bias** | 9 | 0.225 | **55.6%** | 0.1921 | 465.0 ms |
| **Overall Balanced Accuracy** | **100** | **0.583** | **63.3%** | **0.2874** | **425.9 ms** |

### 3. Edge vs Cloud Latency (ClassOne vs TypeSafe Jev API)

Measured against TypeSafe AI's Jev (v1.13) cloud API:
- **ClassOne (Local RTX 5060 Ti):** **52.49 ms** mean latency (19.1 req/s, $0.00 inference cost, 100% private)
- **TypeSafe Jev (Cloud API):** **329.90 ms** mean latency (3.0 req/s)
- **Edge Speedup:** **6.3× faster** than cloud API round-trip latency

## Decision Primitives

- **`Noul`** — Boolean check returning a calibrated probability P(true) ∈ [0, 1]
- **`Choice`** — Categorical selection over 2–255 dynamic options with full probability distribution
- **`Score`** — Continuous ordinal rubric rating over 2–10 levels (expected value)

All outputs are calibrated with a combined NLL + normalized Brier loss.
Post-hoc temperature calibration achieves **ECE = 0.034** (down from 0.178).

## Quickstart

```bash
pip install classone
```

```python
import torch
from huggingface_hub import hf_hub_download
from peft import PeftModel
from transformers import AutoTokenizer

from classone.modeling.modeling_classone import ClassOneModel
from classone.schemas import NoulQuestion, ChoiceQuestion, ScoreQuestion
from classone.tokenizer import ClassOnePromptBuilder

REPO_ID = "devops-thiago/classone-gemma4-e4b"

# 1. Load the backbone and attach the published LoRA adapter
tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-E4B-it")
builder = ClassOnePromptBuilder(tokenizer)
model = ClassOneModel.from_backbone(
    base_model_name_or_path="google/gemma-4-E4B-it",
    tokenizer=tokenizer,
    device="cuda",
    torch_dtype=torch.float16,
)
model.backbone = PeftModel.from_pretrained(model.backbone, REPO_ID, subfolder="lora_backbone")

# 2. Load the trained decision heads
heads = torch.load(hf_hub_download(REPO_ID, "classone_heads.pt"), map_location="cuda")
model.noul_head.load_state_dict(heads["noul_head"])
model.choice_head.load_state_dict(heads["choice_head"])
model.score_head.load_state_dict(heads["score_head"])
model.eval()

# 3. Pack state + questions and run a single forward pass
packed = builder.pack(
    state={"customer": "Alex", "message": "I was charged twice for order #123."},
    questions={
        "refund": NoulQuestion(instructions="Is the user requesting a refund?"),
        "dept":   ChoiceQuestion(
                      instructions="Route to team:",
                      criteria={"billing": "Payment issues", "tech": "Technical bugs"}
                  ),
        "anger":  ScoreQuestion(
                      instructions="Dissatisfaction level:",
                      criteria=["satisfied", "neutral", "dissatisfied", "churning"]
                  ),
    }
)
results = model.evaluate_packed(packed)

print("Refund P(true):", results["refund"].noul)
print("Department:    ", results["dept"].choice, "—", results["dept"].probabilities)
print("Anger score:   ", results["anger"].score)
```

## Repository Files

| File | Description |
|---|---|
| `classone_heads.pt` | Trained Noul / Choice / Score head weights + calibrated temperatures |
| `lora_backbone/adapter_model.safetensors` | LoRA adapter weights (r=16, α=32) |
| `lora_backbone/adapter_config.json` | LoRA config (target modules, rank, etc.) |

## Citation

```bibtex
@misc{classone2026,
  title={ClassOne: A Fast Single-Pass Decision Architecture for Language Models},
  author={Thiago Gonzaga},
  year={2026},
  url={https://github.com/devops-thiago/class-one},
}
```

## Attribution & Legal

- Derived from [google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it) (Google) — Apache License 2.0
- Architecture & training code: [devops-thiago/class-one](https://github.com/devops-thiago/class-one) — Apache 2.0