Instructions to use josh-a/rivet-1b-pt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use josh-a/rivet-1b-pt with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="josh-a/rivet-1b-pt")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("josh-a/rivet-1b-pt")
model = AutoModelForCausalLM.from_pretrained("josh-a/rivet-1b-pt", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use josh-a/rivet-1b-pt with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "josh-a/rivet-1b-pt"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "josh-a/rivet-1b-pt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/josh-a/rivet-1b-pt

SGLang

How to use josh-a/rivet-1b-pt with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "josh-a/rivet-1b-pt" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "josh-a/rivet-1b-pt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "josh-a/rivet-1b-pt" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "josh-a/rivet-1b-pt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use josh-a/rivet-1b-pt with Docker Model Runner:
```
docker model run hf.co/josh-a/rivet-1b-pt
```

josh-a commited on Aug 3

Commit

4f11ef6

verified ·

1 Parent(s): 3f9e439

Base model card (v9 pretrained checkpoint)

Browse files

Files changed (1) hide show

README.md +74 -0

README.md ADDED Viewed

	@@ -0,0 +1,74 @@

+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+tags:
+- text-generation
+- pretrained
+- base-model
+- from-scratch
+- qwen3
+library_name: transformers
+---
+# Rivet-1B-PT (base model)
+The **pretrained base** behind [josh-a/rivet-1b-it](https://huggingface.co/josh-a/rivet-1b-it): a 1.09B parameter language model trained **entirely from scratch** — custom tokenizer, custom corpus, no inherited weights. This repo holds the raw pretrained checkpoint (v9 lineage), before any supervised fine-tuning.
+> Uses the Qwen3 *architecture* for serving compatibility. Shares **no weights** with Qwen — the tokenizer (custom 32k SentencePiece) and every parameter were trained from zero.
+**Want the usable model?** You almost certainly want [rivet-1b-it](https://huggingface.co/josh-a/rivet-1b-it), the post-trained version that chats, reasons in `<think>` blocks, and does tasks. This base model completes text; it does not follow instructions.
+## What this is for
+- Fine-tuning your own model on top of a from-scratch 1B with a 16K context
+- Research into small-model pretraining dynamics (full cost/data disclosure in the [technical report](https://huggingface.co/josh-a/rivet-1b-it/blob/main/TECHNICAL_REPORT.md))
+- A clean, fully-documented provenance chain: every pretraining token is public data or documented
+## Training lineage
+1. **v6 pilot** — 6.3B tokens, deliberately undertrained (2x RTX 5090)
+2. **v7 main pretrain** — 283.4B tokens over 7 weeks, eval loss 1.598 (2x RTX 5090)
+3. **v8 context extension** — RoPE ABF (theta 100K → 1e6) to 16K context, +3B long-doc tokens (2x H100)
+4. **v9 "intelligence run"** — +20B tokens reweighted toward reasoning/math/code, eval loss 1.477. **This checkpoint.**
+Total: ~306B tokens trained from a 387.5B-token pool (composition in [josh-a/rivet-cpt](https://huggingface.co/datasets/josh-a/rivet-cpt)).
+## Model details
+| | |
+|---|---|
+| Params | 1,086,422,528 |
+| Architecture | Qwen3ForCausalLM (from-scratch weights) |
+| Hidden / layers | 2048 / 22 |
+| Heads | 16 query / 8 KV (GQA), head_dim 128 |
+| FFN | 5504 (SwiGLU) |
+| Vocab | 32000 (custom SentencePiece) |
+| Context | 16384 (RoPE θ=1e6; YaRN-extrapolable to ~64K) |
+| Tokenizer | custom 32k; `<think>`=4, `</think>`=5, `<|im_start|>`=6, `<|im_end|>`=7 |
+## Usage
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+m = "josh-a/rivet-1b-pt"
+tok = AutoTokenizer.from_pretrained(m)
+model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.bfloat16).to("cuda").eval()
+ids = tok("The capital of New Zealand is", return_tensors="pt").to("cuda")
+out = model.generate(**ids, max_new_tokens=20, do_sample=False)
+print(tok.decode(out[0][ids["input_ids"].shape[1]:]))
+# -> " Wellington" (as of the v9 intelligence run)
+```
+## Limitations
+- This is a **base model**: no instruction following, no chat template behaviour, no refusals, no alignment of any kind.
+- Factually modest (1B parameters); see the technical report for the honest eval picture.
+## License
+Apache-2.0. From-scratch weights; no upstream model license applies.