Image-Text-to-Text
Transformers
Safetensors
nemotron_parse
text-generation
VLM
OCR
Parse
conversational
custom_code
Eval Results
Instructions to use nvidia/Nemotron-3.5-Parse with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nvidia/Nemotron-3.5-Parse with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="nvidia/Nemotron-3.5-Parse", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("nvidia/Nemotron-3.5-Parse", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use nvidia/Nemotron-3.5-Parse with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nvidia/Nemotron-3.5-Parse" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/Nemotron-3.5-Parse", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/nvidia/Nemotron-3.5-Parse
- SGLang
How to use nvidia/Nemotron-3.5-Parse with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "nvidia/Nemotron-3.5-Parse" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/Nemotron-3.5-Parse", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "nvidia/Nemotron-3.5-Parse" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/Nemotron-3.5-Parse", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use nvidia/Nemotron-3.5-Parse with Docker Model Runner:
docker model run hf.co/nvidia/Nemotron-3.5-Parse
Download dflash2/performance/results.csv from nvidia/Nemotron-3.5-Parse: direct link, hf CLI and curl.
- Browser
- Download file 3.73 kB
-
https://huggingface.co/nvidia/Nemotron-3.5-Parse/resolve/main/dflash2/performance/results.csv
- Command line
-
hf download hf://nvidia/Nemotron-3.5-Parse/dflash2/performance/results.csv
-
curl -L -o results.csv https://huggingface.co/nvidia/Nemotron-3.5-Parse/resolve/main/dflash2/performance/results.csv
3.73 kB
| method,concurrency,draft_tokens,repeats,pages_per_second,pages_per_second_min,pages_per_second_max,output_tokens_per_second,length_capped_pages_median,speedup_vs_base,speedup_vs_mtp | |
| base,1,0,3,0.5384719450403836,0.5284642021654238,0.5393986613144893,674.0471571739056,12,1.0,0.6847192718442942 | |
| mtp,1,1,3,0.7864127200480384,0.7445693401579543,0.7906644588545633,987.2193101089284,13,1.4604525403622652,1.0 | |
| dflash2,1,32,3,4.104182594835197,3.956050918009935,4.241881777143353,5138.130473843292,12,7.621906085613052,5.2188659850065635 | |
| base,2,0,3,1.024702789106118,1.0238376975487864,1.0354294746680688,1288.362564453034,12,1.0,0.6769006347837822 | |
| mtp,2,1,3,1.513815671680426,1.499750960186973,1.524454101096625,1898.9950690427625,12,1.4773217051560652,1.0 | |
| dflash2,2,32,3,6.706409177199086,6.704940206261365,7.109290871308974,8415.968025027918,13,6.5447359453850105,4.43013591592335 | |
| base,4,0,3,1.8763660023655073,1.7338043018685754,1.95347959426876,2349.9723393437116,11,1.0,0.6982578451035966 | |
| mtp,4,1,3,2.6872107710972033,2.561578807675198,2.88346678766922,3354.0774275321014,11,1.4321357175036618,1.0 | |
| dflash2,4,32,3,10.66076212028022,10.639304234494402,10.956379118639115,13333.7814872105,10,5.681600554923907,3.9672221602205657 | |
| base,8,0,3,3.373600692857952,3.1939594282931987,3.41474243865981,4224.731899036535,12,1.0,0.734910788519373 | |
| mtp,8,1,3,4.59049009153173,4.585574875763909,4.601685455047932,5746.005573427087,12,1.3607093753715374,1.0 | |
| dflash2,8,24,3,14.539573757984611,14.511899519823368,14.825853667690335,18306.64577681767,13,4.3098087419668465,3.167324940926542 | |
| base,16,0,3,5.33456540840746,5.316026086691856,5.590496141598985,6687.529798925631,12,1.0,0.7268727061783999 | |
| mtp,16,1,3,7.339064134701698,6.417202624856532,7.351978086174866,9138.420420054448,13,1.3757567060917613,1.0 | |
| dflash2,16,24,3,17.35748501805556,16.38894039673584,17.653746495165027,21686.8744657376,12,3.2537767726494757,2.3650815280361446 | |
| base,32,0,3,8.170436988514949,8.07828588280311,8.24219190673298,10211.638624266454,11,1.0,0.7643417542036015 | |
| mtp,32,1,3,10.689507597328706,10.322072937437365,10.802278683302331,13346.709305413473,11,1.3083152850153272,1.0 | |
| dflash2,32,24,3,19.085194696195384,18.980576688875544,19.0965556714819,23914.29083425922,12,2.335884203381427,1.7854138296290418 | |
| base,64,0,3,10.46874011628935,10.44204190527327,10.546447930372134,13120.077016992258,12,1.0,0.7806177526816234 | |
| mtp,64,1,3,13.410840427759332,13.19625678650178,13.451364217462004,16812.66464397199,12,1.2810367129939615,1.0 | |
| dflash2,64,16,3,19.71185413825793,19.533445192290596,20.055535193580372,24643.716408169297,12,1.8829251580700053,1.469844805360298 | |
| base,128,0,3,12.46436352707653,12.414395280518091,12.506557450860988,15688.206406983776,13,1.0,0.8247032815599422 | |
| mtp,128,1,3,15.113755220544224,15.067313967101043,15.572014327094488,18953.325456204184,11,1.2125573189286705,1.0 | |
| dflash2,128,16,3,20.1501473287165,19.830843499673602,20.19282688562108,25157.745484198313,11,1.6166206389073956,1.3332323459444597 | |
| base,256,0,3,13.338161999247934,13.262679605500876,13.398167622421258,16735.201132844908,11,1.0,0.8297950676383604 | |
| mtp,256,1,3,16.074043483060258,16.030676367357607,16.494192583779043,20129.142527874832,12,1.2051168282381475,1.0 | |
| dflash2,256,16,3,20.360581930883804,20.16513012520168,20.504047982145792,25499.855400997174,12,1.5264908262496606,1.266674558417174 | |
| base,512,0,3,13.480162486000307,13.466388380683208,13.623075198319366,16900.646679921527,13,1.0,0.8102820133634046 | |
| mtp,512,1,3,16.63638370799497,16.535603320791115,16.67666627356759,20765.921632756024,12,1.234138217938584,1.0 | |
| dflash2,512,16,3,20.440898294322118,20.14985977306065,20.66004180339928,25686.01509038101,13,1.5163688357281164,1.2286863932153 | |