| --- |
| license: mit |
| pipeline_tag: text-to-image |
| language: |
| - en |
| tags: |
| - image |
| - t2i |
| - text-to-image |
| - custom-code |
| - tiny |
| model-index: |
| - name: PixelModel-v1 |
| results: |
| - task: |
| type: text-to-image |
| dataset: |
| name: fid |
| type: fid |
| metrics: |
| - name: fid |
| type: fid |
| value: 439.46 |
| - task: |
| type: text-to-image |
| dataset: |
| name: clip |
| type: clip |
| metrics: |
| - name: clip |
| type: clip |
| value: 0.2002 |
| new_version: bench-labs/PixelModel-v2 |
| --- |
| |
| <table align="center"> |
| <tr> |
| <td align="center"> |
|
|
| <a href="https://huggingface.co/spaces/bench-labs/BenchLabs-Leaderboard">🏆 BenchLabs Leaderboard</a> • |
| <a href="https://huggingface.co/spaces/FlameF0X/Tiny-T2I-Leaderboard">🌍 Tiny-T2I Leaderboard</a> • |
| <a href="https://huggingface.co/spaces/bench-labs/blog?post=pixelmodel-v1.html">📝 Blog Post</a> |
|
|
| </td> |
| </tr> |
| </table> |
|
|
| # PixelModel v1 🖼️ |
|
|
| <img src="model.png" alt="The entire PixelModel v1: a 160x149 PNG containing all 23,747 weights" style="image-rendering: pixelated; width: 320px; max-width: 100%;"> |
|
|
|
|
| A neural network where the weights **are** the image. Now the model is a thumbnail. |
|
|
| ## 📌 What is this? |
|
|
| `model.png` is not a picture — it *is* the model. |
|
|
| Every pixel encodes neural network weights. At inference, the PNG is decoded |
| into weight matrices, the prompt is hashed into an embedding, and a |
| coordinate-conditioned decoder paints an image — at **any resolution**. |
|
|
| [v0](https://huggingface.co/bench-labs/pixelmodel) was a 202,752-parameter MLP |
| welded to 32×32 output, trained on 6 color swatches. v1 is **8.5× smaller |
| (23,747 parameters)**, renders at any resolution (native 64×64), and is trained |
| on ~20K real MS-COCO caption/image pairs. |
|
|
| # cherry on top 🍒 |
| The model generates 600 images (cpu) in 5 (five) seconds. |
| Thats 5000 images in 24 seconds on cpu. |
| The model trained on cpu for just 30 minutes. |
|
|
| ## 🆕 v0 → v1 |
|
|
| | | v0 | v1 | |
| |---|---|---| |
| | Parameters | 202,752 | **23,747** | |
| | `model.png` | 64×3200 px | **160×149 px** (a thumbnail) | |
| | Output head | 1 weight row per pixel (196K params, 97% of model) | CPPN decoder on (x, y) — resolution-free | |
| | Resolution | fixed 32×32 | any; native 64×64 | |
| | Prompt embedding | char-sum (order-blind, collision-heavy) | hashed trigrams + words (FNV-1a), still 0 params | |
| | Biases | none | yes | |
| | Weight precision | 8-bit (G channel wasted) | **16-bit** (R=high byte, G=low byte) | |
| | Training data | 6 solid-color swatches | ~20K MS-COCO caption/image pairs | |
|
|
| <img src="pixelmodel-v1-loss.png" alt="Training loss curve crossing below both naive-predictor baselines" style="width: 100%; max-width: 720px;"> |
|
|
| ## 🎨 Weight Encoding |
|
|
| Each pixel stores one weight at 16-bit precision, mapped from [-2, 2]: |
|
|
| - **R channel** → high byte |
| - **G channel** → low byte |
| - **B channel** → reserved |
|
|
| Round-trip quantization error ≈ 3×10⁻⁵ per weight. |
|
|
| ## 🧠 Architecture |
|
|
| ```text |
| prompt string |
| → hashed char-trigram + word embedding (64-dim, deterministic, 0 params) |
| → T1 (80×64)+b → tanh |
| → T2 (64×80)+b → tanh = latent z (64) |
| |
| for every pixel (x, y): |
| concat(z, fourier features of (x, y)) ← 18 coord dims, freqs 1/2/4/8 |
| → D1 (80×82)+b → tanh |
| → D2 (80×80)+b → tanh |
| → D3 (3×80)+b → sigmoid = RGB |
| ``` |
|
|
| Because pixels are decoded from coordinates, parameter count is independent of |
| resolution — `--res 256` works with the same 23,747 weights. |
|
|
| All weights live inside `model.png` (160×149 px). |
|
|
| ## 📦 Standard weights (safetensors) |
|
|
| `model.png` is the canonical model — training writes to it directly. For |
| tooling that expects standard weight files, the same 10 tensors are exported as |
| `model.safetensors` (23,747 parameters total): |
|
|
| ```bash |
| python convert_to_safetensors.py # model.png -> model.safetensors |
| ``` |
|
|
| Parameter count is verifiable without running code: `config.json` |
| (`total_parameters: 23747`, full per-layer breakdown) and the safetensors |
| header metadata (`total_parameters`, `param_breakdown`, `has_bias`, |
| `text_encoder_parameters: 0`, `vae_parameters: 0`). |
|
|
| ## 📊 Benchmark results |
|
|
| Measured per the [Tiny-T2I-Leaderboard](https://huggingface.co/spaces/FlameF0X/Tiny-T2I-Leaderboard) protocol — see `eval.md` for full method and `EVAL_REPRODUCTION.md` to re-run: |
|
|
| | Metric | v0 | v1 | Tooling | |
| |---|---|---|---| |
| | FID ↓ | 566.84 (n=40) | **439.46** (n=5000) | `torchmetrics.image.fid.FrechetInceptionDistance` | |
| | CLIP Score ↑ | 18.60 (n=40) | **20.02** (n=5000) | `torchmetrics.multimodal.CLIPScore`, `openai/clip-vit-base-patch32` | |
| | Native resolution | 32×32 | 64×64 | | |
|
|
| Real set: MS-COCO val2014 pairs from `sayakpaul/coco-30-val-2014` (256×256 |
| center-crop). Training images are hash-checked to be disjoint from the eval |
| images. |
|
|
| Expectation management: a 23K-parameter model does not draw recognizable |
| objects. It learns caption-conditioned color, layout, and texture statistics of |
| COCO photos — the outputs are blurry impressions, not pictures. The point is |
| the size:quality ratio and the PNG-as-model gimmick, executed honestly. |
|
|
| <img src="pixelmodel-v1-probe.png" alt="Target photos vs model outputs: the model produces caption-conditioned color washes" style="width: 100%; max-width: 640px;"> |
|
|
| ## ⚙️ Usage |
|
|
| ```bash |
| # training data (COCO subset) |
| python fetch_coco_subset.py --out ../pm-work |
| |
| # train (writes model.png every epoch) |
| python train.py --data ../pm-work/coco_train.npz --epochs 50 |
| |
| # inference from model.png (canonical) |
| python main.py "a red double decker bus" --out bus.png |
| |
| # inference from model.safetensors (standalone, needs only this one file + torch) |
| python convert_to_safetensors.py |
| python INFERENCE.py "a red double decker bus" --out bus.png |
| |
| # any resolution from the same 23,747 weights |
| python main.py "a beach with palm trees" --res 256 --scale 1 |
| |
| # benchmark eval |
| python eval/run_eval.py --work ../pm-work --model model.png --n 5000 |
| ``` |
|
|
| `main.py` and `INFERENCE.py` produce identical output for the same prompt and |
| resolution. |
|
|
| ## 📁 Files |
|
|
| ```text |
| model.png ← THE MODEL (160×149 px) |
| model.safetensors ← same weights, standard format + param metadata |
| config.json ← architecture + parameter-count metadata |
| main.py ← inference, loads model.png |
| INFERENCE.py ← inference, loads model.safetensors (standalone) |
| convert_to_safetensors.py ← model.png -> model.safetensors |
| train.py ← training |
| model.py ← architecture + PNG weight codec |
| fetch_coco_subset.py ← builds train/eval data from COCO |
| eval/run_eval.py ← FID + CLIP Score via torchmetrics |
| eval.md ← benchmark results + method |
| EVAL_REPRODUCTION.md ← step-by-step reproduction guide |
| ``` |
|
|
| --- |
|
|
| *Still a toy. Slightly less useless. The model is literally a thumbnail.* |
|
|
| Bench Labs · Simple, Reliable, Open sourced |
