Text Classification
Transformers
English
Indonesian
code
security
code-review
static-analysis
vulnerability-detection
sarif
sliding-window-attention
spark
cpp
avx2
awq
experimental
Instructions to use wxsys/spark-servitor with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use wxsys/spark-servitor with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="wxsys/spark-servitor")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("wxsys/spark-servitor", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from wxsys/spark-servitor: direct link, hf CLI and curl.
- Browser
- Download file 12.5 kB
-
https://huggingface.co/wxsys/spark-servitor/resolve/main/README.md
- Command line
-
hf download hf://wxsys/spark-servitor/README.md
-
curl -L -o README.md https://huggingface.co/wxsys/spark-servitor/resolve/main/README.md
12.5 kB
| language: | |
| - en | |
| - id | |
| license: apache-2.0 | |
| library_name: transformers | |
| tags: | |
| - code | |
| - security | |
| - code-review | |
| - static-analysis | |
| - vulnerability-detection | |
| - sarif | |
| - sliding-window-attention | |
| - spark | |
| - cpp | |
| - avx2 | |
| - awq | |
| - experimental | |
| base_model: XHToken/Spark-X2.5-1.7B | |
| pipeline_tag: text-classification | |
| inference: false | |
| # β‘ Spark-Servitor | |
| <p align="center"> | |
| <strong>Experimental rule-assisted code diff gate with an unvalidated 1.7B neural head & OASIS SARIF v2.1.0 output</strong> | |
| </p> | |
| <p align="center"> | |
| <a href="https://huggingface.co/wxsys/spark-servitor"><img src="https://img.shields.io/badge/π€%20HuggingFace-wxsys%2Fspark--servitor-ffd21e.svg" alt="Hugging Face Model"></a> | |
| <img src="https://img.shields.io/badge/License-Apache_2.0-red.svg" alt="License"> | |
| <img src="https://img.shields.io/badge/Status-Experimental-orange.svg" alt="Status: experimental"> | |
| <img src="https://img.shields.io/badge/Held--out_accuracy-~50%25_(chance)-critical.svg" alt="Held-out accuracy"> | |
| </p> | |
| --- | |
| > [!WARNING] | |
| > **Status: experimental. Do not use as a security gate.** | |
| > | |
| > - On an independent held-out benchmark (PrimeVul paired CVE fixes, 433 pairs) the shipped pipeline scores **50.0% accuracy** with **1.8% recall** on vulnerability-introducing diffs. That is chance level. | |
| > - Earlier versions of this card reported "93.5% Real-World PR Accuracy" and "100% Critical Vulnerability Recall". Those numbers were **withdrawn**: 93.5% is the *training-set* accuracy of the last training epoch (see [What went wrong](#known-issues--what-was-wrong-with-earlier-claims)), and no run produced the 100% recall figure. | |
| > - Most verdicts produced by the `--weights` / native path come from **string-matching rules and a taint extractor**, not from neural inference. | |
| > - Full evaluation details are in [Evaluation](#evaluation). | |
| --- | |
| ## Overview | |
| Spark-Servitor is a code diff screening tool. It canonicalizes a unified diff, runs it through rule-based and taint-flow checks, and (optionally) a 1.7B-parameter neural verdict/risk/pathology head, then emits a verdict (`APPROVE` / `QUARANTINE` / `REJECT`), a risk score, pathology labels and an OASIS SARIF v2.1.0 report with exploit flows. | |
| It is a sibling of [`wxsys/qwen-servitor`](https://huggingface.co/wxsys/qwen-servitor). | |
| What works today: | |
| - Diff canonicalization, multi-file parsing, SARIF v2.1.0 emission, CLI / daemon / FFI plumbing. The repo test suite passes (149 passed, locally re-run). | |
| - A rule/taint layer that catches classic web/shell patterns (e.g. `os.system(user_input)`, string-concatenated SQL, `eval` on request data, double-lock deadlocks) and passes the safe forms of those patterns. | |
| What does **not** work yet: | |
| - Detection of vulnerabilities outside those patterns (memory safety, auth/logic flaws, IDOR, hardcoded secrets, `verify=False`, XSS sinks, β¦). | |
| - A neural head that adds measurable value over the rules (see [Evaluation](#evaluation)). | |
| --- | |
| ## Architecture (as implemented) | |
| ```text | |
| [ unified diff ] | |
| β | |
| βΌ | |
| DiffCanonicalizer (parse, tokenize) | |
| β | |
| ββββββββββββββΌβββββββββββββββββββββββββββ | |
| βΌ βΌ βΌ | |
| Rule / taint Native C++ engine Neural cortex (PyTorch) | |
| extractor (string-match heuristics) SparkServitorModel | |
| (Python) used by --weights / 28-layer SWA/full-attn backbone | |
| daemon / FFI path + DiffCAMQP cortex head | |
| β β β (not used by the native path) | |
| ββββββββββββββ΄βββββββββββ¬ββββββββββββββββ | |
| βΌ | |
| Hierarchical veto + (QUARANTINE) Z3 SMT gate | |
| βΌ | |
| verdict Β· risk Β· pathologies Β· SARIF | |
| ``` | |
| Important implementation facts: | |
| - The native C++ engine (`src/spark_simd_engine.cpp`) decides verdicts with substring checks such as `system(`, `shlex.quote`, `Lock()`/`Unlock()`. Weights are loaded into memory, but the verdict path does not run a forward pass over them. | |
| - In `spark_servitor/engine.py`, an unmitigated taint flow forces `REJECT` with risk β₯ 0.95, overriding the neural output. | |
| - The CLI defaults to `--mock-mode` (rule-based emulation) unless `--weights` is given. | |
| - The PyTorch `SparkServitorModel` neural path exists and can be run by dequantizing a `.servitor` file; it was evaluated separately (below). | |
| --- | |
| ## Specifications | |
| | Parameter | Value | | |
| |---|---| | |
| | **Architecture** | Spark-X2.5-inspired decoder: d_model 2048, 28 layers (21 SWA window 512 + 7 full attention), 16 Q heads Γ head_dim 128 / 2 KV heads, SiLU MLP, vocab 151,936, headwise sigmoid output gate. The upstream `XHToken/Spark-X2.5-1.7B` config differs (8 heads Γ head_dim 256, GELU, vocab 131,072, partial rotary on full-attention layers), so its checkpoint is **not drop-in compatible** with this implementation. | | |
| | **Heads** | 3-way verdict, continuous risk, 10-label pathology | | |
| | **Weights provenance** | Per `scripts/train_spark_l4.py` and `models/training.log`: model is instantiated from a fresh `SparkConfig` (no pretrained backbone load found in the script), then DoRA (r=16, ~1% of parameters) is trained for 3 epochs on ~250 samples. The repo does **not** demonstrate that Spark-X2.5 pretrained weights were used. | | |
| | **Training data** | ~250 samples (62 steps Γ batch 4 per epoch), generated by `data/generate_dataset.py` (synthetic templates). No separate validation split in the log. | | |
| | **Formats** | Outlier-preserving AWQ INT4 / W8A8 packed `.servitor` files | | |
| | **Context window** | Architecture supports a long circular SWA buffer; long-context quality was **not** evaluated | | |
| > The previously listed figures "4.08 MB resident memory" and "0.044 ms per diff" describe the C++ rule path only, were not independently re-measured, and are not neural-inference numbers. Loading the weights alone takes ~1.3β1.6 GB on disk. | |
| --- | |
| ## Model Files | |
| | Variant | Path in Repository | Size on Disk | Notes | | |
| |---|---|---|---| | |
| | Slim | `slim/spark-servitor-slim.servitor` | ~1.3 GB (1,312 MB) | INT8-quantized embeddings | | |
| | Baseline | `int4/spark-servitor-awq_int4.servitor` | ~1.6 GB (1,608 MB) | FP16 embeddings. (An earlier target of ~720 MB was not met.) | | |
| The repository does **not** contain `model.safetensors` or `cortex_head.pt`. Older docs that mention them are wrong. | |
| --- | |
| ## Evaluation | |
| All numbers below were produced on 2026-10-16 on a CPU-only machine with data **not** used for training. | |
| ### 1. PrimeVul paired test (real CVEs, C/C++) β shipped pipeline | |
| Source: [`colin/PrimeVul`](https://huggingface.co/datasets/colin/PrimeVul), `paired` config, `test` split. 433 usable (vulnerable, fixed) function pairs. For each pair two diffs were built with `difflib`: | |
| - fixed β vulnerable ("a PR that introduces the bug", label 1) | |
| - vulnerable β fixed ("a PR that fixes the bug", label 0) | |
| | Configuration | Recall on bug-introducing diffs | False-positive rate on fix diffs | Accuracy | | |
| |---|---|---|---| | |
| | `SparkServitorEngine` + `--weights` (final verdict) | 1.8% | 1.8% | 50.0% | | |
| | Default CLI (mock / rule-based) | 11.1% | 6.5% | 52.3% | | |
| | Raw native engine output | 1.8% | 1.8% | 50.0% | | |
| 97.5% of pairs were approved on both sides. The rules do not target C/C++ memory-safety bugs. PrimeVul is a hard benchmark; this result does not claim the task is easy, only that this tool does not solve it. | |
| ### 2. Hand-written multi-language set (38 diffs) β shipped pipeline | |
| 8 obvious vulnerabilities, 10 subtle vulnerabilities, 10 "looks dangerous but safe" cases, 10 clean diffs (Python, Go, JS, C). Labels written by the evaluator. | |
| | Configuration | Accuracy | Recall | False-positive rate | | |
| |---|---|---|---| | |
| | Final (with weights) | 0.68 | 0.50 | 0.15 | | |
| | Default CLI (mock) | 0.61 | 0.44 | 0.25 | | |
| | Raw native only | 0.63 | 0.39 | 0.15 | | |
| | Taint rules only | 0.68 | 0.44 | 0.10 | | |
| The final pipeline equals the taint rules alone (0.68). Missed examples: `strcpy` overflow, hardcoded AWS key, `innerHTML = location.hash`, `or True` auth bypass, IDOR, `verify=False`, path traversal on upload. Wrongly rejected: `ast.literal_eval`, a `realpath`-checked file open. | |
| ### 3. Neural head run directly (PyTorch, dequantized INT4 weights, fp32, CPU) | |
| > [!NOTE] | |
| > **Tokenizer caveat.** When these `.servitor` weights were trained, `DiffCanonicalizer.tokenize_canonical_text` mapped words to vocabulary ids using Python's process-salted `hash()` instead of the Spark-X2.5 tokenizer. `DiffCanonicalizer` now uses deterministic `zlib.crc32` across processes, but the existing `.servitor` checkpoints remain mismatched with any inference token mapping until retrained in Phase 3. | |
| | Test | Result | | |
| |---|---| | |
| | 38 hand-written diffs, AUC of risk score | 0.59 | | |
| | 38 hand-written diffs, argmax verdict: recall / false-positive rate | 0.67 / 0.40 | | |
| | PrimeVul, 50 random pairs, AUC (bug-introducing vs fix) | 0.67 | | |
| | PrimeVul, 50 random pairs, argmax flags bug-introducing / fix diffs | 94% / 90% | | |
| | PrimeVul, 50 random pairs, risk(intro) > risk(fix) | 80% | | |
| As a gate the neural head flags almost everything. The 80% pairwise figure is not evidence of skill: a trivial "more deleted lines β riskier" rule reaches ~73% on the same pairs, and 50 pairs gives a wide confidence interval. Verdict probabilities for a plain `strcpy` overflow were near-uniform (0.37 / 0.36 / 0.28). | |
| ### Limits of this evaluation | |
| - Small samples (38 hand-written diffs, 50 neural pairs, tokens truncated to 512). Treat results as indicative, not precise. | |
| - PrimeVul is C/C++ only; Python/JS/Go were covered only by the 38 hand-written diffs. | |
| - The hand-written labels were created by the evaluator, not an independent annotator. | |
| - Latency and memory were not re-measured for the neural path. | |
| --- | |
| ## Known issues & what was wrong with earlier claims | |
| | Earlier claim | Reality | | |
| |---|---| | |
| | "Real-World PR Accuracy 93.5%" | Training-set accuracy of epoch 3 (`models/training.log`: `Acc: 93.50% \| Steps: 62`), on ~250 synthetic samples. Not a held-out measure. | | |
| | "Critical Vulnerability Recall 100%" | No run supports it. Measured recall on real CVE diffs is 1.8% (shipped pipeline). | | |
| | "Neural code decision gate" | Verdicts on the weights/native path come from string-matching rules; taint flows override model output. | | |
| | "Zero-shot" tag | Removed. | | |
| | Spark-X2.5 vocabulary / tokenizer | Checkpoints were trained with a per-process `hash()` tokenizer (not Spark-X2.5 BPE). Code now uses deterministic `crc32` pending Phase 3 real tokenizer integration. | | |
| | INT4 ~720 MB | File is 1,608 MB. | | |
| | `model.safetensors`, `cortex_head.pt` in the repo | Not present. | | |
| | `SparkServitor.summon(...)`, `spark-servitor judge`, `--model` flag (older USAGE) | Not implemented in the code. | | |
| --- | |
| ## Quickstart (what actually exists) | |
| ### CLI | |
| ```bash | |
| # Rule-based emulation (default, no weights) | |
| git diff HEAD~1..HEAD | spark-servitor --format tree | |
| # With the packed weights file (still rule/taint driven, see warnings above) | |
| spark-servitor --file patch.diff --weights models/spark-servitor-awq_int4.servitor --format json | |
| ``` | |
| ### Resident daemon | |
| ```bash | |
| spark-servitord --socket /run/user/1000/spark-servitor.sock --weights models/spark-servitor-awq_int4.servitor --daemon | |
| spark-servitor --socket /run/user/1000/spark-servitor.sock --file patch.diff | |
| ``` | |
| ### Python | |
| ```python | |
| from spark_servitor.engine import SparkServitorEngine | |
| engine = SparkServitorEngine(mock_mode=False, weights_path="models/spark-servitor-awq_int4.servitor") | |
| verdict = engine.evaluate_diff(diff_text) | |
| print(verdict.status, verdict.risk, verdict.pathology) | |
| ``` | |
| --- | |
| ## Intended use | |
| - Research and development of diff-screening pipelines, SARIF plumbing, and as a lightweight rule-based pre-filter for a small set of web/shell patterns. | |
| - **Not** suitable for gating merges, replacing SAST tools, or any decision where a missed vulnerability matters. | |
| ## Roadmap | |
| A plan to rebuild the neural component properly (pretrained backbone, real training data, held-out evaluation, honest reporting) is tracked in the project's improvement plan. | |
| --- | |
| ## Related Repositories | |
| - [**wxsys/qwen-servitor**](https://huggingface.co/wxsys/qwen-servitor): Sister model based on Qwen3.5-0.8B Linear Attention (DeltaNet), specialized for sub-15ms local pre-commit hooks. | |