Add model card, weights and notices

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+adapter/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE.Apache-2.0-Gemma

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

LICENSE.OpenMDW-1.1

@@ -0,0 +1,49 @@
+OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)
+
+By exercising rights granted to you under this agreement, you accept and agree
+to its terms.
+
+As used in this agreement, "Model Materials" means the materials provided to
+you under this agreement, consisting of: (1) one or more machine learning
+models (including architecture and parameters); and (2) all related artifacts
+(including associated data, documentation and software) that are provided to
+you hereunder.
+
+Subject to your compliance with this agreement, permission is hereby granted,
+free of charge, to deal in the Model Materials without restriction, including
+under all copyright, patent, database, and trade secret rights included or
+embodied therein.
+
+If you distribute any portion of the Model Materials, you shall retain in your
+distribution (1) a copy of this agreement, and (2) all copyright notices and
+other notices of origin included in the Model Materials that are applicable to
+your distribution.
+
+If you file, maintain, or voluntarily participate in a lawsuit against any
+person or entity asserting that the Model Materials directly or indirectly
+infringe any patent or copyright, then all rights and grants made to you
+hereunder are terminated, unless that lawsuit was in response to a
+corresponding lawsuit first brought against you.
+
+This agreement does not impose any restrictions or obligations with respect to
+any use, modification, or sharing of any outputs generated by using the Model
+Materials.
+
+THE MODEL MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE
+ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL
+TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+
+YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY
+APPLY TO THE MODEL MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION
+ANY PERSON'S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MODEL
+MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS
+REQUIRED FOR ANY USE OF THE MODEL MATERIALS; OR (3) PERFORMING ANY DUE
+DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MODEL MATERIALS OR
+ANYTHING INCORPORATED OR EMBODIED THEREIN.
+
+IN NO EVENT SHALL THE PROVIDERS OF THE MODEL MATERIALS BE LIABLE FOR ANY CLAIM,
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL MATERIALS, THE
+USE THEREOF OR OTHER DEALINGS THEREIN.

MODEL_CARD.ko.md

@@ -0,0 +1,201 @@
+# FleetWorker-E4B (한국어 카드)
+
+영문 카드: [README.md](./README.md)
+
+버전 1.0.0. [nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16)을 파인튜닝하여 `ThakiCloud/FleetWorker-E4B`로 공개한 모델입니다.
+
+FleetWorker-E4B는 작업 보드 항목을 분류하는 소형 온디바이스 분류기입니다. 항목의 제목을 입력받아 그 항목이 속한 작업 흐름(프로젝트)의 이름을 내놓고, 같은 JSON 객체에 우선순위 제안도 함께 담습니다. 유용한 출력은 프로젝트 레이블입니다. 공개된 모델은 보드의 내부 프로젝트명 대신 설명형의 일반적인 작업 흐름 이름으로 학습되었으므로 가중치에 내부 이름이 들어 있지 않으며, 그 대가로 내부 이름 모델에 비해 프로젝트 정확도가 약 3점 낮습니다. 우선순위 필드는 신뢰할 수 없습니다. 우선순위가 한 번도 설정되지 않은 항목에서는 10건 중 약 8건에 P0으로 답합니다. 프로젝트 필드만 사용하고, 우선순위를 긴급도로 표시하거나 자동으로 적용하지 마십시오.
+
+## 기능
+
+시스템 프롬프트에는 허용되는 프로젝트 레이블과 허용되는 우선순위가 나열되고, 사용자 메시지는 항목 제목(한국어 또는 영어)입니다. 모델은 정확히 하나의 JSON 객체 `{"project":..., "priority":...}`로 답합니다. 프로젝트 레이블은 25개의 일반적인 작업 흐름 이름(예: `synthetic-data-pipeline`, `legal-decision-models`, `point-of-sale-app`)이며 `inference_example.py`에 나열되어 있습니다. 모델은 이 분류 체계 안의 레이블만 출력할 수 있으므로, 다른 분류 체계에 그대로 끼워 쓰는 용도가 아니라 자신의 보드를 파인튜닝할 때 참고하는 용도로 사용하십시오.
+
+## 의도된 사용
+
+- 새 작업 보드 항목에 프로젝트 레이블을 제안하고, 사람이 확인하는 용도.
+- 검토 전에 받은 항목을 작업 흐름별로 미리 분류하는 용도.
+- 다른 보드 분류 체계에 맞춘 자체 파인튜닝의 출발점.
+
+## 사용 방법
+
+### 프롬프트 형식
+
+모델은 기본 체크포인트의 채팅 템플릿을 thinking을 끈 상태(`enable_thinking=False`)로 사용합니다. 렌더링된 프롬프트는 정확히 다음과 같습니다.
+
+```text
+<bos><|turn>system
+You triage a task-board item. Output exactly one compact JSON object with keys "project" and "priority" and nothing else. "project" is one of: <the project labels the model was trained on, comma separated>, or "other"; "unknown" if the text gives no way to tell. "priority" is one of: P0, P1, P2, P3.<turn|>
+<|turn>user
+title: Add retry with backoff to the nightly export job<turn|>
+<|turn>model
+```
+
+출력 형식은 컴팩트한 JSON 객체 하나입니다: `{"project":<label from the prompt list, "other" or "unknown">,"priority":"P0"|"P1"|"P2"|"P3"}`.
+
+꺾쇠괄호로 표시된 프로젝트 목록은 직접 파인튜닝한 레이블 집합으로 바꾸십시오. 제안된 우선순위는 사람에게 보여 주고, 자동으로 다시 기록하지 마십시오.
+
+### Transformers + PEFT adapter
+
+```python
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+BASE = "nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16"
+tok = AutoTokenizer.from_pretrained(BASE)
+model = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")
+model = PeftModel.from_pretrained(model, "ThakiCloud/FleetWorker-E4B", subfolder="adapter").eval()
+
+SYSTEM = "You triage a task-board item. Output exactly one compact JSON object with keys \"project\" and \"priority\" and nothing else. \"project\" is one of: <the project labels the model was trained on, comma separated>, or \"other\"; \"unknown\" if the text gives no way to tell. \"priority\" is one of: P0, P1, P2, P3."
+USER = "title: Add retry with backoff to the nightly export job"
+messages = [
+    {"role": "system", "content": SYSTEM},
+    {"role": "user", "content": USER},
+]
+inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
+                               return_tensors="pt", return_dict=True).to(model.device)
+out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
+print(tok.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+### Merged BF16
+
+LoRA delta가 기본 가중치(BF16)에 병합되어 있습니다.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tok = AutoTokenizer.from_pretrained("ThakiCloud/FleetWorker-E4B")
+model = AutoModelForCausalLM.from_pretrained("ThakiCloud/FleetWorker-E4B", dtype=torch.bfloat16, device_map="auto").eval()
+
+SYSTEM = "You triage a task-board item. Output exactly one compact JSON object with keys \"project\" and \"priority\" and nothing else. \"project\" is one of: <the project labels the model was trained on, comma separated>, or \"other\"; \"unknown\" if the text gives no way to tell. \"priority\" is one of: P0, P1, P2, P3."
+USER = "title: Add retry with backoff to the nightly export job"
+messages = [
+    {"role": "system", "content": SYSTEM},
+    {"role": "user", "content": USER},
+]
+inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
+                               return_tensors="pt", return_dict=True).to(model.device)
+out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
+print(tok.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+### llama.cpp (GGUF)
+
+| File | Quantization | Size |
+|---|---|---|
+| `FleetWorker-E4B-Q4_K_M.gguf` | Q4_K_M, imatrix from training rows (recommended) | 5.34 GB (5335295200 bytes) |
+| `FleetWorker-E4B-Q4_K_M-NV.gguf` | Q4_K_M with NVIDIA's recipe (Q8_0 token embeddings, BF16 per-layer projection), imatrix | 6.18 GB (6180447456 bytes) |
+| `FleetWorker-E4B-Q8_0.gguf` | Q8_0 | 8.03 GB (8031245280 bytes) |
+
+```bash
+llama-server -m FleetWorker-E4B-Q4_K_M.gguf -c 8192 -ngl 99 --port 8080
+```
+
+채팅 완성 요청에는 GGUF에 내장된 템플릿이 적용되며, thinking은 요청마다 끕니다.
+
+```bash
+curl http://localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{"messages": [{"role": "system", "content": "You triage a task-board item. Output exactly one compact JSON object with keys \"project\" and \"priority\" and nothing else. \"project\" is one of: <the project labels the model was trained on, comma separated>, or \"other\"; \"unknown\" if the text gives no way to tell. \"priority\" is one of: P0, P1, P2, P3."}, {"role": "user", "content": "title: Add retry with backoff to the nightly export job"}], "temperature": 0, "chat_template_kwargs": {"enable_thinking": false}}'
+```
+
+보고된 GGUF 수치는 Hugging Face 채팅 템플릿으로 렌더링한 토큰 id를 보내 측정했으므로, 프롬프트가 BF16 실행과 바이트 단위로 동일합니다. 원시 문자열로 `/completion`을 호출한다면 서버가 `<bos>`를 한 번 더 붙이지 않는지 확인하십시오.
+
+## 평가
+
+보류된(held-out) 테스트 분할에서 greedy decoding으로 측정했으며, 모든 비교 대상에 같은 프롬프트를 사용했습니다.
+
+테스트 분할: 정답 항목 텍스트로 채점한 두 개의 보류 집합입니다. 하나는 학습 시점 이후에 수집한 206개 항목의 독립 집합(보류된 검증 기간과 그 이후 항목)이고, 다른 하나는 594개 항목의 더 큰 테스트 집합입니다. 프로젝트 정확도는 전체 항목 기준이며, 우선순위는 명시적으로 우선순위가 지정된 항목만으로 채점합니다. FleetWorker-E4B는 이 파일들의 alias 버전(sha256 접두사 ef19eeee 독립 집합, 220269b7 더 큰 집합)으로 채점했고, base와 v1 행은 내부 이름 버전으로 채점했습니다.
+테스트 파일 지문(sha256 접두사, 원장 기준): 5711ab1b436d5f49, d27e10421aa2dbb3.
+
+| Arm (model) | n | Project acc., independent set | Project acc., larger test set |
+|---|---|---|---|
+| Bolt base, zero-shot (internal label set) (nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16) | 206 items (independent) / 594 items (larger) | 0.2961 | 0.3855 |
+| Earlier version (v1 adapter) (internal label set) (nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16) | 206 items (independent) / 594 items (larger) | 0.4951 | 0.5875 |
+| FleetWorker-E4B (nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16) | 206 items (independent) / 594 items (larger) | 0.5291 | 0.5892 |
+
+실행 조건:
+- Bolt base, zero-shot (internal label set): greedy decode, enable_thinking=False, transformers 5.19.0; 1x B200
+- Earlier version (v1 adapter) (internal label set): greedy decode, enable_thinking=False, transformers 5.19.0; 1x B200
+- FleetWorker-E4B: greedy decode, enable_thinking=False, transformers 5.19.0; 1x H100
+
+이 결과와 함께 기록된 주의 사항:
+- Bolt base, zero-shot (internal label set): zero-shot 측정입니다.
+- Earlier version (v1 adapter) (internal label set): 우선순위는 명시적으로 설정된 레이블에 대해서만 채점했습니다. v1은 모든 항목에 P2를 예측하므로(P2 recall 1.0, P0/P3 0), 전체 우선순위 점수는 실력이 아니라 기본 레이블의 비중과 같습니다.
+- FleetWorker-E4B: 실명 v4 집합과 같은 항목이며, 프로젝트 레이블만 설명형의 일반적인 alias로 바꿨습니다. 보드 기본값 그대로인 항목에서는 151건 중 120건에 P0을 예측합니다.
+
+### 명시적으로 우선순위가 지정된 항목의 우선순위 정확도
+
+| Variant | n | Independent set (55 items) | Larger test set (243 items) |
+|---|---|---|---|
+| Bolt base, zero-shot (internal label set) | 55 items | 0.3273 | 0.2716 |
+| Earlier version (v1 adapter) (internal label set) | 55 items | 0.3636 | 0.2922 |
+| FleetWorker-E4B | 55 items | 0.6 | 0.5432 |
+| Always predict the most common class | 55 items | 0.3636 | 0.5267 |
+
+측정 기준: small_set은 독립 집합에서 ��시적으로 우선순위가 지정된 55개 항목이고, large_set은 더 큰 테스트 집합에서 명시적으로 우선순위가 지정된 243개 항목입니다. 보드 기본값 그대로인 항목은 제외했습니다.
+주의 사항: 정확도는 명시적으로 우선순위가 지정된 항목에 대해서만 계산했습니다. 'always predict the most common class' 행이 우선순위 제안이 넘어야 하는 기준선이며, 더 큰 집합에서 FleetWorker-E4B는 이를 1.7점 앞섭니다. 우선순위가 한 번도 설정되지 않은 항목에서는 151건 중 120건에 P0으로 답하므로, 그 항목들에서 우선순위 필드는 쓸 수 없습니다.
+
+### 클래스별 우선순위 (명시적으로 레이블된 항목만)
+
+| Variant | n | P0 recall | P1 recall | P2 recall | P3 recall |
+|---|---|---|---|---|---|
+| FleetWorker-E4B recall | 55 explicit items | 0.5 | 0.4667 | 0.9412 | 0.0 |
+| Earlier version (v1 adapter, internal label set) recall | 55 explicit items | 0.0 | 0.2 | 1.0 | 0.0 |
+
+측정 기준: 206개 항목 독립 테스트 집합 중 명시적으로 우선순위가 지정된 55개 항목이며, 우선순위 클래스별 recall입니다.
+주의 사항: 우선순위는 제안일 뿐이며 우선순위가 지정되지 않은 항목에서는 사용할 수 없습니다. 보드 기본값 그대로인 151개 항목에서 모델은 120건에 P0, 27건에 P1, 4건에 P2로 답합니다. decode 시점의 prior 보정을 이전 버전에서 시도했으나 사전 등록한 점검을 통과하지 못했으므로 배포하지 않습니다.
+
+### 독립 집합(206개 항목)에서의 병합 가중치와 GGUF
+
+| Variant | n | Project | Priority (explicit items) |
+|---|---|---|---|
+| Merged BF16 | 206 items | 0.5291 | 0.6 |
+| GGUF Q8_0 | 206 items | 0.5291 | 0.6 |
+| GGUF Q4_K_M (imatrix) | 206 items | 0.5291 | 0.6 |
+| GGUF Q4_K_M-NV | 206 items | 0.5291 | 0.6 |
+
+측정 기준: 하드웨어 H100 1장, 독립 집합입니다. GGUF 실행은 Hugging Face 채팅 템플릿의 토큰 id를 llama-server(llama.cpp 344be0a)에 입력했고 greedy decoding입니다.
+주의 사항: 모든 변형에서 프로젝트 정확도와 명시적 우선순위 정확도가 동일합니다. 유출 점검(내부 식별자를 요구하는 프롬프트 130개, 프로젝트명 패턴 강제): 병합 BF16, Q4_K_M, Q4_K_M-NV에서 0건이었고, 테스트 출력 1,077개에서 내부 이름은 0건이었습니다.
+
+## 학습
+
+방법: 기본 모델의 LoRA 파인튜닝이며, loss는 assistant 턴에만 적용했습니다. 아래 값은 학습 기록에서 그대로 옮긴 것입니다.
+
+| Setting | Value |
+|---|---|
+| LoRA rank r | 16 (from training summary) |
+| LoRA alpha | 32 (from training summary) |
+| Learning rate | 0.0001 (from training summary) |
+| Epochs | 2.0 (from training summary) |
+| Target modules | attn+mlp (from training summary) |
+| Gradient accumulation | 8 (from training summary) |
+| Max sequence length | 4096 (from training summary) |
+| Optimizer steps | 1406 (from training summary) |
+| Peak GPU memory (GiB) | 17.53 (from training summary) |
+| Wall time (s) | 5815.2 (from training summary) |
+
+## 데이터
+
+작업 보드 항목(제목, 그리고 보드 이벤트 로그의 정답 프로젝트 및 우선순위 레이블), 보드 자체 규칙으로 만든 행, 그리고 더 큰 언어 모델이 작성한 소규모의 paraphrase 및 어려운 사례 집합으로 구성됩니다. 마지막 집합은 코드 검증기가 레이블이 보존되었음을 확인한 경우에만 남겼습니다. 내부 프로젝트명은 학습 전에 설명형의 일반적인 alias로 대체했습니다(토큰 전체 단위 치환, 잔존 이름 0건, 코드로 검사). 보드 기본값에 불과한 우선순위 레이블은 loss에서 제외했습니다. 학습, 검증, 테스트 분할은 기간과 항목 계열 기준으로 서로 겹치지 않습니다. 이 버전은 기본 가중치에서 2 epoch 학습했고, 검증 loss를 기준으로 1406 step 중 step 450의 체크포인트를 선택했습니다.
+
+학습 데이터는 내부 자료이며 공개하지 않습니다. 이 저장소에는 데이터셋 파일이 들어 있지 않습니다.
+
+분할 크기(항목 수): train 5621, val 145, test 665, test_indep 206.
+
+## 한계
+
+- 우선순위는 우선순위가 지정되지 않은 항목에서 사용할 수 없습니다. 보드 기본값 그대로인 항목에서는 독립 집합 151건 중 120건에 P0으로 답합니다. 명시적으로 우선순위가 지정된 항목에서는 다수 클래스 기준선을 넘지만(55개 항목에서 0.600 대 0.364, 243개 항목에서 0.543 대 0.527) P3은 한 번도 예측하지 않습니다. 프로젝트 필드만 사용하고, 우선순위를 자동으로 표시하거나 적용하지 마십시오.
+- base와 v1 행은 프롬프트와 레이블에 내부 프로젝트명을 넣어 측정했습니다. FleetWorker-E4B는 같은 항목을 일반 alias로 채점했으므로, 비교는 같은 항목 기준이지만 레이블 문자열까지 동일한 ��은 아닙니다. 내부 이름으로 학습한 모델은 독립 집합에서 0.563을 기록했으며, 공개하지 않습니다.
+- 프로젝트 정확도는 약 0.53에서 0.59입니다. 10건 중 대략 4에서 5건은 수정이 필요할 것으로 예상하십시오. 166개 alias 중 91개는 학습 항목이 없는 프로젝트명에서 나왔으므로, 실제로 출력되는 레이블은 25개뿐입니다.
+- 학습 데이터와 보드는 내부 자료이며 공개하지 않습니다. 레이블은 한 팀의 작업 흐름이므로, 이 모델은 자신의 보드를 파인튜닝하기 위한 출발점으로 사용하십시오.
+- decode 시점의 prior 보정을 이전 버전에서 시도했으나 사전 등록한 점검을 통과하지 못했으므로 배포하지 않습니다.
+- 위의 프롬프트 형식 밖의 동작(다른 출력 스키마, thinking 모드, 긴 항목 본문)은 평가하지 않았습니다.
+
+## 라이선스 및 출처 표기
+
+- 모델 자료: OpenMDW License Agreement 1.1 버전(`LICENSE.OpenMDW-1.1`, https://openmdw.ai/license/1-1/)의 적용을 받습니다. 이 저장소는 [nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16)의 파생물이며 해당 계약서 사본을 포함합니다.
+- 출처: NVIDIA 기본 모델은 Google Gemma 4를 기반으로 post-training되었으며, Gemma 4의 사용은 Apache License, Version 2.0(`LICENSE.Apache-2.0-Gemma`)의 적용을 받습니다.
+- NVIDIA의 governing-terms 고지는 `NOTICE.md`에 원문 그대로 보존되어 있으며, 출처 체인과 ThakiCloud의 수정 사항에 대한 설명이 함께 실려 있습니다.
+- 파인튜닝된 가중치에는 ThakiCloud의 LoRA 수정이 더해졌으나, 위의 조건은 달라지지 않습니다.
+- 이 한국어 카드는 참고용 번역입니다. 해석에 차이가 있을 경우 영문 `NOTICE.md`와 `LICENSE` 파일이 우선합니다.

NOTICE.md

@@ -0,0 +1,23 @@
+# NOTICE for FleetWorker-E4B
+
+## Origin chain
+
+1. Google Gemma 4 (`google/gemma-4-E4B-it`), Apache License, Version 2.0 (`LICENSE.Apache-2.0-Gemma`).
+2. NVIDIA Nemotron 3.5 Bolt E4B (`nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16`), post-trained by NVIDIA on Gemma 4, OpenMDW License Agreement, version 1.1 (`LICENSE.OpenMDW-1.1`).
+3. `ThakiCloud/FleetWorker-E4B`, a LoRA fine-tune of (2) by ThakiCloud.
+
+## Modifications by ThakiCloud
+
+ThakiCloud trained a LoRA adapter on top of the NVIDIA model for the task described in README.md (version 1.0.0). The adapter is merged into the base weights in the merged checkpoint and the GGUF files. The base model's tokenizer and chat template are used unchanged.
+
+## Notices retained from the NVIDIA model card
+
+The following text is reproduced verbatim from the base model's README (section "License/Terms of Use"):
+
+> **Governing Terms:** Use of this model is governed by the [OpenMDW License Agreement, version 1.1](https://github.com/OpenMDW/OpenMDW/blob/main/1.1/LICENSE.OpenMDW-1.1). This model has been post-trained on the Google Gemma 4 model, and use of the Google Gemma 4 model is governed by [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+
+The NVIDIA model card also ships `bias.md`, `explainability.md`, `privacy.md` and `safety.md` for the base model; they describe the base model and remain applicable.
+
+## Agreements
+
+Copies of both agreements are included in this repository: `LICENSE.OpenMDW-1.1` and `LICENSE.Apache-2.0-Gemma`.

README.md

@@ -0,0 +1,222 @@
+---
+library_name: transformers
+license: other
+license_name: openmdw-1.1
+license_link: https://openmdw.ai/license/1-1/
+base_model: nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16
+base_model_relation: finetune
+pipeline_tag: text-generation
+language:
+- en
+- ko
+tags:
+- lora
+- gemma4
+- nemotron
+- on-device
+- task-triage
+- classification
+- agents
+- thakicloud
+---
+
+# FleetWorker-E4B
+
+Version 1.0.0. Fine-tune of [nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16) published as `ThakiCloud/FleetWorker-E4B`.
+
+Korean card: [MODEL_CARD.ko.md](./MODEL_CARD.ko.md)
+
+FleetWorker-E4B is a small on-device classifier for task-board items. Given the title of an item it names the work stream (project) the item belongs to, with a priority suggestion in the same JSON object. The project label is the useful output. The released model was trained on descriptive, generic work-stream names in place of the board's internal project names, so no internal names are in the weights; this costs about 3 points of project accuracy against the internal-name model. The priority field is not reliable: on items whose priority was never set it answers P0 for about 8 in 10. Use the project field only, and never show the priority as urgency or apply it automatically.
+
+## What it does
+
+The system prompt lists the allowed project labels and the allowed priorities; the user message is the item title (Korean or English). The model replies with exactly one JSON object `{"project":..., "priority":...}`. Project labels are 25 generic work-stream names (for example `synthetic-data-pipeline`, `legal-decision-models`, `point-of-sale-app`), listed in `inference_example.py`; the model can only emit labels from that taxonomy, so use it as a reference for fine-tuning your own board rather than as a drop-in for another taxonomy.
+
+## Intended use
+
+- Suggesting a project label for new task-board items, with a person confirming it.
+- Pre-sorting an inbox of items into work streams before review.
+- Starting point for your own fine-tune on a different board taxonomy.
+
+## How to use
+
+### Prompt format
+
+The model uses the base checkpoint's chat template with thinking disabled (`enable_thinking=False`). The rendered prompt is exactly:
+
+```text
+<bos><|turn>system
+You triage a task-board item. Output exactly one compact JSON object with keys "project" and "priority" and nothing else. "project" is one of: <the project labels the model was trained on, comma separated>, or "other"; "unknown" if the text gives no way to tell. "priority" is one of: P0, P1, P2, P3.<turn|>
+<|turn>user
+title: Add retry with backoff to the nightly export job<turn|>
+<|turn>model
+```
+
+Output format: one compact JSON object: `{"project":<label from the prompt list, "other" or "unknown">,"priority":"P0"|"P1"|"P2"|"P3"}`.
+
+Replace the angle-bracket project list with the label set you fine-tune for. Show the suggested priority to a person; do not write it back automatically.
+
+### Transformers + PEFT adapter
+
+```python
+import torch
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+BASE = "nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16"
+tok = AutoTokenizer.from_pretrained(BASE)
+model = AutoModelForCausalLM.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto")
+model = PeftModel.from_pretrained(model, "ThakiCloud/FleetWorker-E4B", subfolder="adapter").eval()
+
+SYSTEM = "You triage a task-board item. Output exactly one compact JSON object with keys \"project\" and \"priority\" and nothing else. \"project\" is one of: <the project labels the model was trained on, comma separated>, or \"other\"; \"unknown\" if the text gives no way to tell. \"priority\" is one of: P0, P1, P2, P3."
+USER = "title: Add retry with backoff to the nightly export job"
+messages = [
+    {"role": "system", "content": SYSTEM},
+    {"role": "user", "content": USER},
+]
+inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
+                               return_tensors="pt", return_dict=True).to(model.device)
+out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
+print(tok.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+### Merged BF16
+
+The LoRA delta is merged into the base weights (BF16).
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+tok = AutoTokenizer.from_pretrained("ThakiCloud/FleetWorker-E4B")
+model = AutoModelForCausalLM.from_pretrained("ThakiCloud/FleetWorker-E4B", dtype=torch.bfloat16, device_map="auto").eval()
+
+SYSTEM = "You triage a task-board item. Output exactly one compact JSON object with keys \"project\" and \"priority\" and nothing else. \"project\" is one of: <the project labels the model was trained on, comma separated>, or \"other\"; \"unknown\" if the text gives no way to tell. \"priority\" is one of: P0, P1, P2, P3."
+USER = "title: Add retry with backoff to the nightly export job"
+messages = [
+    {"role": "system", "content": SYSTEM},
+    {"role": "user", "content": USER},
+]
+inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False,
+                               return_tensors="pt", return_dict=True).to(model.device)
+out = model.generate(**inputs, max_new_tokens=64, do_sample=False)
+print(tok.decode(out[0, inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
+```
+
+### llama.cpp (GGUF)
+
+| File | Quantization | Size |
+|---|---|---|
+| `FleetWorker-E4B-Q4_K_M.gguf` | Q4_K_M, imatrix from training rows (recommended) | 5.34 GB (5335295200 bytes) |
+| `FleetWorker-E4B-Q4_K_M-NV.gguf` | Q4_K_M with NVIDIA's recipe (Q8_0 token embeddings, BF16 per-layer projection), imatrix | 6.18 GB (6180447456 bytes) |
+| `FleetWorker-E4B-Q8_0.gguf` | Q8_0 | 8.03 GB (8031245280 bytes) |
+
+```bash
+llama-server -m FleetWorker-E4B-Q4_K_M.gguf -c 8192 -ngl 99 --port 8080
+```
+
+Chat completions apply the template embedded in the GGUF; disable thinking per request:
+
+```bash
+curl http://localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{"messages": [{"role": "system", "content": "You triage a task-board item. Output exactly one compact JSON object with keys \"project\" and \"priority\" and nothing else. \"project\" is one of: <the project labels the model was trained on, comma separated>, or \"other\"; \"unknown\" if the text gives no way to tell. \"priority\" is one of: P0, P1, P2, P3."}, {"role": "user", "content": "title: Add retry with backoff to the nightly export job"}], "temperature": 0, "chat_template_kwargs": {"enable_thinking": false}}'
+```
+
+The reported GGUF numbers were measured by sending token ids rendered with the Hugging Face chat template, so the prompt is byte-identical to the BF16 run. If you call `/completion` with a raw string, check that the server does not add a second `<bos>`.
+
+## Evaluation
+
+Greedy decoding on a held-out test split, same prompts for every arm.
+
+Test split: two held-out sets scored with the gold item text: an independent set of 206 items taken after the training cut-off (held-out validation period plus later items), and a larger test set of 594 items. Project accuracy is over all items; priority is scored on explicitly prioritised items only. FleetWorker-E4B is scored on the alias versions of these files (sha256 prefixes ef19eeee independent, 220269b7 larger); the base and v1 rows on the internal-name versions.
+Test file fingerprint (sha256 prefix, from the ledger): 5711ab1b436d5f49, d27e10421aa2dbb3.
+
+| Arm (model) | n | Project acc., independent set | Project acc., larger test set |
+|---|---|---|---|
+| Bolt base, zero-shot (internal label set) (nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16) | 206 items (independent) / 594 items (larger) | 0.2961 | 0.3855 |
+| Earlier version (v1 adapter) (internal label set) (nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16) | 206 items (independent) / 594 items (larger) | 0.4951 | 0.5875 |
+| FleetWorker-E4B (nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16) | 206 items (independent) / 594 items (larger) | 0.5291 | 0.5892 |
+
+Run conditions:
+- Bolt base, zero-shot (internal label set): greedy decode, enable_thinking=False, transformers 5.19.0; 1x B200
+- Earlier version (v1 adapter) (internal label set): greedy decode, enable_thinking=False, transformers 5.19.0; 1x B200
+- FleetWorker-E4B: greedy decode, enable_thinking=False, transformers 5.19.0; 1x H100
+
+Caveats recorded with these results:
+- Bolt base, zero-shot (internal label set): zero-shot
+- Earlier version (v1 adapter) (internal label set): priority on explicitly-set labels only: v1 predicts P2 for everything (recall P2 1.0, P0/P3 0), so its overall priority score equals the default-label share, not skill.
+- FleetWorker-E4B: Same items as the real-name v4 sets, with project labels replaced by descriptive generic aliases. On items left at the board default it predicts P0 for 120/151.
+
+### Priority accuracy on explicitly prioritised items
+
+| Variant | n | Independent set (55 items) | Larger test set (243 items) |
+|---|---|---|---|
+| Bolt base, zero-shot (internal label set) | 55 items | 0.3273 | 0.2716 |
+| Earlier version (v1 adapter) (internal label set) | 55 items | 0.3636 | 0.2922 |
+| FleetWorker-E4B | 55 items | 0.6 | 0.5432 |
+| Always predict the most common class | 55 items | 0.3636 | 0.5267 |
+
+Measured with small_set = the 55 explicitly prioritised items of the independent set; large_set = the 243 explicitly prioritised items of the larger test set. Items left at the board default are excluded.
+Caveats: Accuracy over explicitly prioritised items only. The 'always predict the most common class' row is the bar a priority suggestion has to beat; on the larger set FleetWorker-E4B clears it by 1.7 points. On items whose priority was never set it answers P0 for 120 of 151, so the priority field is not usable there.
+
+### Priority by class (explicitly labelled items only)
+
+| Variant | n | P0 recall | P1 recall | P2 recall | P3 recall |
+|---|---|---|---|---|---|
+| FleetWorker-E4B recall | 55 explicit items | 0.5 | 0.4667 | 0.9412 | 0.0 |
+| Earlier version (v1 adapter, internal label set) recall | 55 explicit items | 0.0 | 0.2 | 1.0 | 0.0 |
+
+Measured with the 55 explicitly prioritised items of the 206-item independent test set; recall per priority class.
+Caveats: Priority is a suggestion only and is not usable on unprioritised items: on the 151 items left at the board default the model answers P0 for 120, P1 for 27 and P2 for 4. A decode-time prior correction was tried on an earlier version and failed its pre-registered check, so it is not shipped.
+
+### Merged weights and GGUF on the independent set (206 items)
+
+| Variant | n | Project | Priority (explicit items) |
+|---|---|---|---|
+| Merged BF16 | 206 items | 0.5291 | 0.6 |
+| GGUF Q8_0 | 206 items | 0.5291 | 0.6 |
+| GGUF Q4_K_M (imatrix) | 206 items | 0.5291 | 0.6 |
+| GGUF Q4_K_M-NV | 206 items | 0.5291 | 0.6 |
+
+Measured with hardware H100, 1 GPU; independent set; GGUF runs feed the Hugging Face chat-template token ids to llama-server (llama.cpp 344be0a), greedy.
+Caveats: All variants give identical project and explicit-priority accuracy. Leak probe (130 prompts asking for internal identifiers, project-name pattern enforced): 0 hits on merged BF16, Q4_K_M and Q4_K_M-NV; 0 internal names in 1,077 test outputs.
+
+## Training
+
+Method: LoRA fine-tune of the base model, loss on assistant turns only. Values below are copied from the training record.
+
+| Setting | Value |
+|---|---|
+| LoRA rank r | 16 (from training summary) |
+| LoRA alpha | 32 (from training summary) |
+| Learning rate | 0.0001 (from training summary) |
+| Epochs | 2.0 (from training summary) |
+| Target modules | attn+mlp (from training summary) |
+| Gradient accumulation | 8 (from training summary) |
+| Max sequence length | 4096 (from training summary) |
+| Optimizer steps | 1406 (from training summary) |
+| Peak GPU memory (GiB) | 17.53 (from training summary) |
+| Wall time (s) | 5815.2 (from training summary) |
+
+## Data
+
+Task-board items (title and gold project and priority labels from the board's event log), plus rows built from the board's own rules, plus a smaller set of paraphrases and harder cases written by a larger language model and kept only when code validators confirmed the label was preserved. Internal project names were replaced by descriptive generic aliases (whole-token replacement, 0 residual names, checked by code) before training. Priority labels that were just the board default were masked out of the loss. Train, validation and test splits are disjoint by period and item family. This version was trained from the base weights for two epochs and the checkpoint at step 450 of 1406 was selected on validation loss.
+
+The training data is internal and is not released. This repository contains no dataset files.
+
+Split sizes (items): train 5621, val 145, test 665, test_indep 206.
+
+## Limitations
+
+- Priority is not usable on unprioritised items: on items left at the board default it answers P0 for 120 of 151 in the independent set. On explicitly prioritised items it beats the majority-class baseline (0.600 vs 0.364 on 55 items; 0.543 vs 0.527 on 243 items) and never predicts P3. Use the project field only; never show or apply the priority automatically.
+- The base and v1 rows were measured with the internal project names in the prompt and labels; FleetWorker-E4B is scored on the same items with the generic aliases, so the comparison is on the same items but not on identical label strings. A model trained on the internal names reached 0.563 on the independent set; it is not released.
+- Project accuracy is about 0.53 to 0.59: expect roughly 4 to 5 in 10 items to need a correction. 91 of the 166 aliases come from project names without any training items, so only 25 labels are emitted in practice.
+- Training data and the board are internal and not released. The labels are one team's work streams; use the model as a starting point for fine-tuning your own board.
+- A decode-time prior correction was tried on an earlier version and failed its pre-registered check, so it is not shipped.
+- Behavior outside the prompt format above (other output schemas, thinking mode, long item bodies) was not evaluated.
+
+## License and attribution
+
+- Model materials: governed by the OpenMDW License Agreement, version 1.1 (`LICENSE.OpenMDW-1.1`, https://openmdw.ai/license/1-1/). This repository is a derivative of [nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16) and ships a copy of the agreement.
+- Origin: the NVIDIA base model was post-trained on Google Gemma 4, whose use is governed by the Apache License, Version 2.0 (`LICENSE.Apache-2.0-Gemma`).
+- NVIDIA's governing-terms notice is retained verbatim in `NOTICE.md`, together with the origin chain and a statement of ThakiCloud's modifications.
+- The fine-tuned weights add ThakiCloud's LoRA modifications; they do not change the terms above.

adapter/adapter_config.json

@@ -0,0 +1,301 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "model.language_model.layers.26.self_attn.o_proj",
+    "model.language_model.layers.4.self_attn.o_proj",
+    "model.language_model.layers.16.mlp.up_proj",
+    "model.language_model.layers.4.mlp.up_proj",
+    "model.language_model.layers.19.mlp.gate_proj",
+    "model.language_model.layers.14.mlp.gate_proj",
+    "model.language_model.layers.35.self_attn.o_proj",
+    "model.language_model.layers.31.mlp.down_proj",
+    "model.language_model.layers.25.mlp.down_proj",
+    "model.language_model.layers.40.self_attn.o_proj",
+    "model.language_model.layers.31.self_attn.q_proj",
+    "model.language_model.layers.28.mlp.up_proj",
+    "model.language_model.layers.39.mlp.gate_proj",
+    "model.language_model.layers.31.mlp.gate_proj",
+    "model.language_model.layers.27.mlp.gate_proj",
+    "model.language_model.layers.32.mlp.down_proj",
+    "model.language_model.layers.15.self_attn.q_proj",
+    "model.language_model.layers.29.mlp.gate_proj",
+    "model.language_model.layers.3.self_attn.k_proj",
+    "model.language_model.layers.40.self_attn.q_proj",
+    "model.language_model.layers.7.mlp.up_proj",
+    "model.language_model.layers.13.self_attn.v_proj",
+    "model.language_model.layers.37.mlp.gate_proj",
+    "model.language_model.layers.2.mlp.up_proj",
+    "model.language_model.layers.23.self_attn.k_proj",
+    "model.language_model.layers.26.mlp.down_proj",
+    "model.language_model.layers.2.self_attn.v_proj",
+    "model.language_model.layers.22.self_attn.q_proj",
+    "model.language_model.layers.32.mlp.gate_proj",
+    "model.language_model.layers.22.mlp.gate_proj",
+    "model.language_model.layers.32.self_attn.o_proj",
+    "model.language_model.layers.30.mlp.down_proj",
+    "model.language_model.layers.16.mlp.gate_proj",
+    "model.language_model.layers.12.self_attn.q_proj",
+    "model.language_model.layers.12.self_attn.o_proj",
+    "model.language_model.layers.3.mlp.down_proj",
+    "model.language_model.layers.6.mlp.up_proj",
+    "model.language_model.layers.36.mlp.down_proj",
+    "model.language_model.layers.4.self_attn.v_proj",
+    "model.language_model.layers.6.self_attn.q_proj",
+    "model.language_model.layers.22.self_attn.o_proj",
+    "model.language_model.layers.21.mlp.gate_proj",
+    "model.language_model.layers.3.self_attn.o_proj",
+    "model.language_model.layers.33.mlp.up_proj",
+    "model.language_model.layers.12.mlp.down_proj",
+    "model.language_model.layers.11.self_attn.k_proj",
+    "model.language_model.layers.15.mlp.up_proj",
+    "model.language_model.layers.15.self_attn.k_proj",
+    "model.language_model.layers.23.mlp.up_proj",
+    "model.language_model.layers.26.mlp.up_proj",
+    "model.language_model.layers.30.mlp.gate_proj",
+    "model.language_model.layers.0.mlp.gate_proj",
+    "model.language_model.layers.30.self_attn.o_proj",
+    "model.language_model.layers.17.self_attn.v_proj",
+    "model.language_model.layers.34.self_attn.q_proj",
+    "model.language_model.layers.41.self_attn.q_proj",
+    "model.language_model.layers.37.self_attn.o_proj",
+    "model.language_model.layers.9.mlp.up_proj",
+    "model.language_model.layers.29.mlp.up_proj",
+    "model.language_model.layers.10.mlp.down_proj",
+    "model.language_model.layers.21.self_attn.k_proj",
+    "model.language_model.layers.14.self_attn.v_proj",
+    "model.language_model.layers.0.self_attn.v_proj",
+    "model.language_model.layers.15.self_attn.o_proj",
+    "model.language_model.layers.8.self_attn.q_proj",
+    "model.language_model.layers.26.mlp.gate_proj",
+    "model.language_model.layers.39.self_attn.o_proj",
+    "model.language_model.layers.17.self_attn.q_proj",
+    "model.language_model.layers.39.mlp.up_proj",
+    "model.language_model.layers.8.mlp.down_proj",
+    "model.language_model.layers.19.self_attn.o_proj",
+    "model.language_model.layers.15.mlp.gate_proj",
+    "model.language_model.layers.21.self_attn.q_proj",
+    "model.language_model.layers.24.self_attn.o_proj",
+    "model.language_model.layers.23.mlp.down_proj",
+    "model.language_model.layers.17.mlp.up_proj",
+    "model.language_model.layers.37.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.v_proj",
+    "model.language_model.layers.11.self_attn.o_proj",
+    "model.language_model.layers.17.self_attn.o_proj",
+    "model.language_model.layers.8.self_attn.v_proj",
+    "model.language_model.layers.1.mlp.up_proj",
+    "model.language_model.layers.36.mlp.gate_proj",
+    "model.language_model.layers.2.mlp.down_proj",
+    "model.language_model.layers.13.self_attn.k_proj",
+    "model.language_model.layers.38.mlp.up_proj",
+    "model.language_model.layers.31.self_attn.o_proj",
+    "model.language_model.layers.2.self_attn.k_proj",
+    "model.language_model.layers.13.mlp.gate_proj",
+    "model.language_model.layers.37.mlp.up_proj",
+    "model.language_model.layers.41.mlp.up_proj",
+    "model.language_model.layers.21.self_attn.v_proj",
+    "model.language_model.layers.22.mlp.up_proj",
+    "model.language_model.layers.0.self_attn.o_proj",
+    "model.language_model.layers.14.self_attn.k_proj",
+    "model.language_model.layers.15.mlp.down_proj",
+    "model.language_model.layers.36.self_attn.q_proj",
+    "model.language_model.layers.7.self_attn.q_proj",
+    "model.language_model.layers.7.mlp.down_proj",
+    "model.language_model.layers.9.mlp.down_proj",
+    "model.language_model.layers.37.mlp.down_proj",
+    "model.language_model.layers.23.self_attn.q_proj",
+    "model.language_model.layers.29.self_attn.q_proj",
+    "model.language_model.layers.23.self_attn.v_proj",
+    "model.language_model.layers.20.mlp.up_proj",
+    "model.language_model.layers.0.mlp.down_proj",
+    "model.language_model.layers.6.mlp.down_proj",
+    "model.language_model.layers.3.mlp.gate_proj",
+    "model.language_model.layers.17.self_attn.k_proj",
+    "model.language_model.layers.13.self_attn.q_proj",
+    "model.language_model.layers.28.self_attn.o_proj",
+    "model.language_model.layers.7.self_attn.v_proj",
+    "model.language_model.layers.30.mlp.up_proj",
+    "model.language_model.layers.35.self_attn.q_proj",
+    "model.language_model.layers.8.mlp.up_proj",
+    "model.language_model.layers.35.mlp.down_proj",
+    "model.language_model.layers.38.self_attn.q_proj",
+    "model.language_model.layers.24.self_attn.q_proj",
+    "model.language_model.layers.7.self_attn.k_proj",
+    "model.language_model.layers.40.mlp.gate_proj",
+    "model.language_model.layers.26.self_attn.q_proj",
+    "model.language_model.layers.25.self_attn.o_proj",
+    "model.language_model.layers.25.mlp.up_proj",
+    "model.language_model.layers.18.self_attn.v_proj",
+    "model.language_model.layers.25.mlp.gate_proj",
+    "model.language_model.layers.21.mlp.down_proj",
+    "model.language_model.layers.5.mlp.gate_proj",
+    "model.language_model.layers.11.self_attn.q_proj",
+    "model.language_model.layers.14.self_attn.o_proj",
+    "model.language_model.layers.2.self_attn.q_proj",
+    "model.language_model.layers.7.mlp.gate_proj",
+    "model.language_model.layers.5.mlp.up_proj",
+    "model.language_model.layers.0.self_attn.q_proj",
+    "model.language_model.layers.0.self_attn.k_proj",
+    "model.language_model.layers.11.mlp.up_proj",
+    "model.language_model.layers.35.mlp.gate_proj",
+    "model.language_model.layers.5.self_attn.k_proj",
+    "model.language_model.layers.11.mlp.gate_proj",
+    "model.language_model.layers.27.self_attn.q_proj",
+    "model.language_model.layers.18.self_attn.k_proj",
+    "model.language_model.layers.8.self_attn.k_proj",
+    "model.language_model.layers.4.self_attn.k_proj",
+    "model.language_model.layers.10.mlp.gate_proj",
+    "model.language_model.layers.11.self_attn.v_proj",
+    "model.language_model.layers.27.mlp.up_proj",
+    "model.language_model.layers.2.self_attn.o_proj",
+    "model.language_model.layers.7.self_attn.o_proj",
+    "model.language_model.layers.9.mlp.gate_proj",
+    "model.language_model.layers.19.self_attn.q_proj",
+    "model.language_model.layers.21.mlp.up_proj",
+    "model.language_model.layers.12.mlp.gate_proj",
+    "model.language_model.layers.18.mlp.gate_proj",
+    "model.language_model.layers.25.self_attn.q_proj",
+    "model.language_model.layers.22.mlp.down_proj",
+    "model.language_model.layers.30.self_attn.q_proj",
+    "model.language_model.layers.6.mlp.gate_proj",
+    "model.language_model.layers.21.self_attn.o_proj",
+    "model.language_model.layers.34.mlp.up_proj",
+    "model.language_model.layers.22.self_attn.v_proj",
+    "model.language_model.layers.16.self_attn.v_proj",
+    "model.language_model.layers.3.self_attn.q_proj",
+    "model.language_model.layers.1.self_attn.o_proj",
+    "model.language_model.layers.24.mlp.gate_proj",
+    "model.language_model.layers.36.self_attn.o_proj",
+    "model.language_model.layers.4.self_attn.q_proj",
+    "model.language_model.layers.16.self_attn.q_proj",
+    "model.language_model.layers.22.self_attn.k_proj",
+    "model.language_model.layers.8.self_attn.o_proj",
+    "model.language_model.layers.19.mlp.up_proj",
+    "model.language_model.layers.20.mlp.gate_proj",
+    "model.language_model.layers.14.self_attn.q_proj",
+    "model.language_model.layers.10.self_attn.q_proj",
+    "model.language_model.layers.15.self_attn.v_proj",
+    "model.language_model.layers.9.self_attn.k_proj",
+    "model.language_model.layers.16.mlp.down_proj",
+    "model.language_model.layers.41.self_attn.o_proj",
+    "model.language_model.layers.27.self_attn.o_proj",
+    "model.language_model.layers.29.mlp.down_proj",
+    "model.language_model.layers.28.mlp.gate_proj",
+    "model.language_model.layers.33.mlp.down_proj",
+    "model.language_model.layers.10.self_attn.v_proj",
+    "model.language_model.layers.19.self_attn.v_proj",
+    "model.language_model.layers.28.mlp.down_proj",
+    "model.language_model.layers.39.mlp.down_proj",
+    "model.language_model.layers.6.self_attn.k_proj",
+    "model.language_model.layers.38.self_attn.o_proj",
+    "model.language_model.layers.1.self_attn.q_proj",
+    "model.language_model.layers.24.mlp.down_proj",
+    "model.language_model.layers.23.mlp.gate_proj",
+    "model.language_model.layers.9.self_attn.v_proj",
+    "model.language_model.layers.38.mlp.down_proj",
+    "model.language_model.layers.34.mlp.down_proj",
+    "model.language_model.layers.4.mlp.down_proj",
+    "model.language_model.layers.1.mlp.down_proj",
+    "model.language_model.layers.12.mlp.up_proj",
+    "model.language_model.layers.13.self_attn.o_proj",
+    "model.language_model.layers.20.self_attn.v_proj",
+    "model.language_model.layers.33.mlp.gate_proj",
+    "model.language_model.layers.5.self_attn.o_proj",
+    "model.language_model.layers.12.self_attn.v_proj",
+    "model.language_model.layers.16.self_attn.k_proj",
+    "model.language_model.layers.20.self_attn.o_proj",
+    "model.language_model.layers.6.self_attn.o_proj",
+    "model.language_model.layers.18.mlp.up_proj",
+    "model.language_model.layers.20.self_attn.k_proj",
+    "model.language_model.layers.41.mlp.gate_proj",
+    "model.language_model.layers.19.mlp.down_proj",
+    "model.language_model.layers.13.mlp.down_proj",
+    "model.language_model.layers.14.mlp.down_proj",
+    "model.language_model.layers.4.mlp.gate_proj",
+    "model.language_model.layers.11.mlp.down_proj",
+    "model.language_model.layers.20.mlp.down_proj",
+    "model.language_model.layers.16.self_attn.o_proj",
+    "model.language_model.layers.10.self_attn.o_proj",
+    "model.language_model.layers.13.mlp.up_proj",
+    "model.language_model.layers.3.mlp.up_proj",
+    "model.language_model.layers.18.self_attn.o_proj",
+    "model.language_model.layers.10.self_attn.k_proj",
+    "model.language_model.layers.17.mlp.gate_proj",
+    "model.language_model.layers.34.mlp.gate_proj",
+    "model.language_model.layers.35.mlp.up_proj",
+    "model.language_model.layers.8.mlp.gate_proj",
+    "model.language_model.layers.9.self_attn.o_proj",
+    "model.language_model.layers.9.self_attn.q_proj",
+    "model.language_model.layers.20.self_attn.q_proj",
+    "model.language_model.layers.32.self_attn.q_proj",
+    "model.language_model.layers.23.self_attn.o_proj",
+    "model.language_model.layers.33.self_attn.o_proj",
+    "model.language_model.layers.5.self_attn.q_proj",
+    "model.language_model.layers.39.self_attn.q_proj",
+    "model.language_model.layers.5.mlp.down_proj",
+    "model.language_model.layers.28.self_attn.q_proj",
+    "model.language_model.layers.27.mlp.down_proj",
+    "model.language_model.layers.29.self_attn.o_proj",
+    "model.language_model.layers.1.mlp.gate_proj",
+    "model.language_model.layers.17.mlp.down_proj",
+    "model.language_model.layers.14.mlp.up_proj",
+    "model.language_model.layers.5.self_attn.v_proj",
+    "model.language_model.layers.6.self_attn.v_proj",
+    "model.language_model.layers.36.mlp.up_proj",
+    "model.language_model.layers.31.mlp.up_proj",
+    "model.language_model.layers.34.self_attn.o_proj",
+    "model.language_model.layers.2.mlp.gate_proj",
+    "model.language_model.layers.24.mlp.up_proj",
+    "model.language_model.layers.18.mlp.down_proj",
+    "model.language_model.layers.40.mlp.up_proj",
+    "model.language_model.layers.18.self_attn.q_proj",
+    "model.language_model.layers.10.mlp.up_proj",
+    "model.language_model.layers.12.self_attn.k_proj",
+    "model.language_model.layers.1.self_attn.k_proj",
+    "model.language_model.layers.19.self_attn.k_proj",
+    "model.language_model.layers.0.mlp.up_proj",
+    "model.language_model.layers.32.mlp.up_proj",
+    "model.language_model.layers.40.mlp.down_proj",
+    "model.language_model.layers.41.mlp.down_proj",
+    "model.language_model.layers.38.mlp.gate_proj",
+    "model.language_model.layers.3.self_attn.v_proj",
+    "model.language_model.layers.33.self_attn.q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

adapter/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db3ba1be40dfbc7177c5b8b0d93aa9b781b0cb5d10783c7a9e9f583581f627ff
+size 139602808

adapter/chat_template.jinja

@@ -0,0 +1,415 @@
+{%- if conversational_mode | default(false) -%}
+    {%- set spoken_instruction = "You are having a spoken conversation with the user. Respond as briefly as possible while still addressing their questions fully." -%}
+    {%- if messages and messages[0]['role'] in ['system', 'developer'] -%}
+        {%- set spoken = namespace(text='') -%}
+        {%- if messages[0]['content'] is string -%}
+            {%- set spoken.text = messages[0]['content'] | trim -%}
+        {%- else -%}
+            {%- for part in messages[0]['content'] -%}
+                {%- set spoken.text = spoken.text + (part['text'] | trim) + ' ' -%}
+            {%- endfor -%}
+            {%- set spoken.text = spoken.text | trim -%}
+        {%- endif -%}
+        {%- if spoken.text != spoken_instruction -%}
+            {%- set spoken.text = (spoken.text + '\n\n' if spoken.text else '') + spoken_instruction -%}
+        {%- endif -%}
+        {%- set messages = [{'role': messages[0]['role'], 'content': spoken.text}] + messages[1:] -%}
+    {%- else -%}
+        {%- set messages = [{'role': 'system', 'content': spoken_instruction}] + messages -%}
+    {%- endif -%}
+{%- endif -%}
+{#
+  Template: Nemotron updated Gemma4 chat template, reasoning-off suffix ablation
+  Published: 2026-08-20
+  Context: Preserve content ordering before tool-calls while matching the original
+           Gemma 4 generation prompt when enable_thinking=false.
+#}
+{%- macro format_parameters(properties, required, filter_keys=false) -%}
+    {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
+    {%- set ns = namespace(found_first=false) -%}
+    {%- for key, value in properties | dictsort -%}
+        {%- set add_comma = false -%}
+        {%- if not filter_keys or key not in standard_keys -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {{ key }}:{
+            {%- if value['description'] -%}
+                description:<|"|>{{ value['description'] }}<|"|>
+                {%- set add_comma = true -%}
+            {%- endif -%}
+            {%- if value['type'] | upper == 'STRING' -%}
+                {%- if value['enum'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    enum:{{ format_argument(value['enum']) }}
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'ARRAY' -%}
+                {%- if value['items'] is mapping and value['items'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    items:{
+                    {%- set ns_items = namespace(found_first=false) -%}
+                    {%- for item_key, item_value in value['items'] | dictsort -%}
+                        {%- if item_value is not none -%}
+                            {%- if ns_items.found_first %},{% endif -%}
+                            {%- set ns_items.found_first = true -%}
+                            {%- if item_key == 'properties' -%}
+                                properties:{
+                                {%- if item_value is mapping -%}
+                                    {{- format_parameters(item_value, value['items']['required'] | default([])) -}}
+                                {%- endif -%}
+                                }
+                            {%- elif item_key == 'required' -%}
+                                required:[
+                                {%- for req_item in item_value -%}
+                                    <|"|>{{- req_item -}}<|"|>
+                                    {%- if not loop.last %},{% endif -%}
+                                {%- endfor -%}
+                                ]
+                            {%- elif item_key == 'type' -%}
+                                {%- if item_value is string -%}
+                                    type:{{ format_argument(item_value | upper) }}
+                                {%- else -%}
+                                    type:{{ format_argument(item_value | map('upper') | list) }}
+                                {%- endif -%}
+                            {%- else -%}
+                                {{ item_key }}:{{ format_argument(item_value) }}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                    }
+                {%- endif -%}
+            {%- endif -%}
+            {%- if value['nullable'] %}
+                {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                nullable:true
+            {%- endif -%}
+            {%- if value['type'] | upper == 'OBJECT' -%}
+                {%- if value['properties'] is defined and value['properties'] is mapping -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    properties:{
+                    {{- format_parameters(value['properties'], value['required'] | default([])) -}}
+                    }
+                {%- elif value is mapping -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    properties:{
+                    {{- format_parameters(value, value['required'] | default([]), filter_keys=true) -}}
+                    }
+                {%- endif -%}
+                {%- if value['required'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    required:[
+                    {%- for item in value['required'] | default([]) -%}
+                        <|"|>{{- item -}}<|"|>
+                        {%- if not loop.last %},{% endif -%}
+                    {%- endfor -%}
+                    ]
+                {%- endif -%}
+            {%- endif -%}
+            {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+            type:<|"|>{{ value['type'] | upper }}<|"|>}
+        {%- endif -%}
+    {%- endfor -%}
+{%- endmacro -%}
+{%- macro format_function_declaration(tool_data) -%}
+    declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
+    {%- set params = tool_data['function']['parameters'] -%}
+    {%- if params -%}
+        ,parameters:{
+        {%- if params['properties'] -%}
+            properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
+        {%- endif -%}
+        {%- if params['required'] -%}
+            required:[
+            {%- for item in params['required'] -%}
+                <|"|>{{- item -}}<|"|>
+                {{- ',' if not loop.last -}}
+            {%- endfor -%}
+            ],
+        {%- endif -%}
+        {%- if params['type'] -%}
+            type:<|"|>{{- params['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    {%- if 'response' in tool_data['function'] -%}
+        {%- set response_declaration = tool_data['function']['response'] -%}
+        ,response:{
+        {%- if response_declaration['description'] -%}
+            description:<|"|>{{- response_declaration['description'] -}}<|"|>,
+        {%- endif -%}
+        {%- if response_declaration['type'] | upper == 'OBJECT' -%}
+            type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    }
+{%- endmacro -%}
+{%- macro format_argument(argument, escape_keys=True) -%}
+    {%- if argument is none -%}
+        {{- 'null' -}}
+    {%- elif argument is string -%}
+        {{- '<|"|>' + argument + '<|"|>' -}}
+    {%- elif argument is boolean -%}
+        {{- 'true' if argument else 'false' -}}
+    {%- elif argument is mapping -%}
+        {{- '{' -}}
+        {%- set ns = namespace(found_first=false) -%}
+        {%- for key, value in argument | dictsort -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {%- if escape_keys -%}
+                {{- '<|"|>' + key + '<|"|>' -}}
+            {%- else -%}
+                {{- key -}}
+            {%- endif -%}
+            :{{- format_argument(value, escape_keys=escape_keys) -}}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- elif argument is sequence -%}
+        {{- '[' -}}
+        {%- for item in argument -%}
+            {{- format_argument(item, escape_keys=escape_keys) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- ']' -}}
+    {%- else -%}
+        {{- argument -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+
+{%- macro format_tool_response_block(tool_name, response) -%}
+    {{- '<|tool_response>' -}}
+    {%- if response is mapping -%}
+        {{- 'response:' + tool_name + '{' -}}
+        {%- for key, value in response | dictsort -%}
+            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- else -%}
+        {{- 'response:' + tool_name + '{value:' + format_argument(response, escape_keys=False) + '}' -}}
+    {%- endif -%}
+    {{- '<tool_response|>' -}}
+{%- endmacro -%}
+
+{#- ===== SETUP ===== -#}
+{%- set ns = namespace(prev_message_type=None, prev_non_tool_role=None) -%}
+{%- set loop_messages = messages -%}
+{%- set enable_thinking = enable_thinking | default(false) -%}
+{%- set preserve_thinking = preserve_thinking | default(false) -%}
+{{- bos_token -}}
+{#- Handle System/Tool Definitions Block -#}
+{%- if enable_thinking or tools or (messages and messages[0]['role'] in ['system', 'developer']) -%}
+    {{- '<|turn>system\n' -}}
+    {#- Inject Thinking token at the very top of the FIRST system turn -#}
+    {%- if enable_thinking -%}
+        {{- '<|think|>\n' -}}
+        {%- set ns.prev_message_type = 'think' -%}
+    {%- endif -%}
+    {%- if messages and messages[0]['role'] in ['system', 'developer'] -%}
+        {%- if messages[0]['content'] is string -%}
+            {{- messages[0]['content'] | trim -}}
+        {%- elif messages[0]['content'] is sequence -%}
+            {%- for item in messages[0]['content'] -%}
+                {{- item['text'] | trim + ' '-}}
+            {%- endfor -%}
+        {%- endif -%}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+    {%- if tools -%}
+        {%- for tool in tools %}
+            {{- '<|tool>' -}}
+            {{- format_function_declaration(tool) | trim -}}
+            {{- '<tool|>' -}}
+        {%- endfor %}
+        {%- set ns.prev_message_type = 'tool' -%}
+    {%- endif -%}
+    {{- '<turn|>\n' -}}
+{%- endif %}
+
+{#- Pre-scan: find last user message index for reasoning guard -#}
+{%- set ns_turn = namespace(last_user_idx=-1) -%}
+{%- for i in range(loop_messages | length) -%}
+    {%- if loop_messages[i]['role'] == 'user' -%}
+        {%- set ns_turn.last_user_idx = i -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{#- Loop through messages -#}
+{%- for message in loop_messages -%}
+    {%- if message['role'] != 'tool' -%}
+    {%- set ns.prev_message_type = None -%}
+    {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
+    {#- Detect continuation using tracked state — O(1) instead of O(n) backward scan -#}
+    {%- set continue_same_model_turn = (role == 'model' and ns.prev_non_tool_role == 'assistant') -%}
+    {%- if not continue_same_model_turn -%}
+        {{- '<|turn>' + role + '\n' }}
+
+    {%- endif -%}
+
+    {#- Render reasoning/reasoning_content as thinking channel -#}
+    {%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}
+    {%- set thinking_gate = (loop.index0 > ns_turn.last_user_idx) or (preserve_thinking and message.get('tool_calls')) -%}
+    {%- if thinking_text and thinking_gate -%}
+        {{- '<|channel>thought\n' + thinking_text + '\n<channel|>' -}}
+    {%- endif -%}
+
+            {%- set captured_content -%}
+            {%- if message.get('content') is string -%}
+                {%- if role == 'model' -%}
+                    {{- strip_thinking(message['content']) -}}
+                {%- else -%}
+                    {{- message['content'] | trim -}}
+                {%- endif -%}
+            {%- elif message.get('content') is sequence -%}
+                {%- for item in message['content'] -%}
+                    {%- if item.get('type') == 'text' -%}
+                        {%- if role == 'model' -%}
+                            {{- strip_thinking(item['text']) -}}
+                        {%- else -%}
+                            {{- item['text'] | trim -}}
+                        {%- endif -%}
+                    {%- elif item.get('type') in ['image', 'image_url'] -%}
+                        {{- '<|image|>' -}}
+                    {%- elif item.get('type') in ['audio', 'input_audio'] -%}
+                        {{- '<|audio|>' -}}
+                    {%- elif item.get('type') == 'video' -%}
+                        {{- '<|video|>' -}}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+            {%- endset -%}
+
+            {#- The sole intentional behavior change from HF main: preserve
+                OpenAI assistant content before its tool calls. -#}
+            {%- if role == 'model' and message.get('tool_calls') -%}
+                {{- captured_content -}}
+            {%- endif -%}
+
+            {%- if message.get('tool_calls') -%}
+                {%- for tool_call in message.get('tool_calls') -%}
+                    {%- set function = tool_call['function'] -%}
+                    {{- '<|tool_call>call:' + function['name'] + '{' -}}
+                    {%- if function['arguments'] is mapping -%}
+                        {%- set ns_args = namespace(found_first=false) -%}
+                        {%- for key, value in function['arguments'] | dictsort -%}
+                            {%- if ns_args.found_first %},{% endif -%}
+                            {%- set ns_args.found_first = true -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                        {%- endfor -%}
+                    {%- elif function['arguments'] is none -%}
+                    {%- else -%}
+                        {{- raise_exception(
+                            "chat_template: tool_calls[].function.arguments must be a "
+                            "JSON object (mapping), not a string. Deserialize arguments "
+                            "before passing to the template."
+                        ) -}}
+                    {%- endif -%}
+                    {{- '}<tool_call|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_call' -%}
+            {%- endif -%}
+
+            {%- set ns_tr_out = namespace(flag=false) -%}
+            {%- if message.get('tool_responses') -%}
+                {#- Legacy: tool_responses embedded on the assistant message (Google/Gemma native) -#}
+                {%- for tool_response in message.get('tool_responses') -%}
+                    {{- format_tool_response_block(tool_response['name'] | default('unknown', true), tool_response['response']) -}}
+                    {%- set ns_tr_out.flag = true -%}
+                    {%- set ns.prev_message_type = 'tool_response' -%}
+                {%- endfor -%}
+            {%- elif message.get('tool_calls') -%}
+                {#- OpenAI Chat Completions: forward-scan consecutive role:tool messages -#}
+                {%- set ns_tool_scan = namespace(stopped=false) -%}
+                {%- for k in range(loop.index0 + 1, loop_messages | length) -%}
+                    {%- if ns_tool_scan.stopped -%}
+                    {%- elif loop_messages[k]['role'] != 'tool' -%}
+                        {%- set ns_tool_scan.stopped = true -%}
+                    {%- else -%}
+                        {%- set follow = loop_messages[k] -%}
+                        {#- Resolve tool_call_id to function name -#}
+                        {%- set ns_tname = namespace(name=follow.get('name') or 'unknown') -%}
+                        {%- for tc in message.get('tool_calls') -%}
+                            {%- if tc.get('id') == follow.get('tool_call_id') -%}
+                                {%- set ns_tname.name = tc['function']['name'] -%}
+                            {%- endif -%}
+                        {%- endfor -%}
+                        {#- Handle content as string or content-parts array -#}
+                        {%- set tool_body = follow.get('content') -%}
+                        {%- if tool_body is string -%}
+                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}
+                        {%- elif tool_body is sequence and tool_body is not string -%}
+                            {%- set ns_txt = namespace(s='') -%}
+                            {%- for part in tool_body -%}
+                                {%- if part.get('type') == 'text' -%}
+                                    {%- set ns_txt.s = ns_txt.s + (part.get('text') | default('')) -%}
+                                {%- endif -%}
+                            {%- endfor -%}
+                            {{- format_tool_response_block(ns_tname.name, ns_txt.s) -}}
+                            {%- for part in tool_body -%}
+                                {%- if part.get('type') in ['image', 'image_url'] -%}
+                                    {{- '<|image|>' -}}
+                                {%- elif part.get('type') in ['audio', 'input_audio'] -%}
+                                    {{- '<|audio|>' -}}
+                                {%- elif part.get('type') == 'video' -%}
+                                    {{- '<|video|>' -}}
+                                {%- endif -%}
+                            {%- endfor -%}
+                        {%- else -%}
+                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}
+                        {%- endif -%}
+                        {%- set ns_tr_out.flag = true -%}
+                        {%- set ns.prev_message_type = 'tool_response' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+            {%- if not (role == 'model' and message.get('tool_calls')) -%}
+                {{- captured_content -}}
+            {%- endif -%}
+            {%- set has_content = captured_content | trim | length > 0 -%}
+
+        {#- Forward-scan: find next non-tool message role for continuation detection -#}
+        {%- set next_nt = namespace(role=None, found=false) -%}
+        {%- for j in range(loop.index0 + 1, loop_messages | length) -%}
+            {%- if not next_nt.found -%}
+                {%- if loop_messages[j]['role'] != 'tool' -%}
+                    {%- set next_nt.role = loop_messages[j]['role'] -%}
+                    {%- set next_nt.found = true -%}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+
+        {%- set continues_into_next = (
+            role == 'model'
+            and next_nt.role == 'assistant'
+            and (not message.get('tool_calls') or ns_tr_out.flag)
+        ) -%}
+
+        {%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}
+            {{- '<|tool_response>' -}}
+        {%- elif continues_into_next -%}
+        {%- elif not (ns_tr_out.flag and not has_content and not next_nt.found) -%}
+            {{- '<turn|>\n' -}}
+        {%- endif -%}
+
+    {#- Track previous non-tool role for next iteration (avoids O(n) backward scan) -#}
+    {%- set ns.prev_non_tool_role = message['role'] -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+    {%- if ns.prev_message_type != 'tool_response' and ns.prev_message_type != 'tool_call' -%}
+        {{- '<|turn>model\n' -}}
+    {%- elif ns.prev_message_type == 'tool_response' and enable_thinking -%}
+        {{- '<|channel>thought\n' -}}
+    {%- endif -%}
+{%- endif -%}

adapter/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
+size 32169626

adapter/tokenizer_config.json

@@ -0,0 +1,142 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "is_local": true,
+  "local_files_only": true,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "model_specific_special_tokens": {
+    "audio_token": "<|audio|>",
+    "boa_token": "<|audio>",
+    "boi_token": "<|image>",
+    "eoa_token": "<audio|>",
+    "eoc_token": "<channel|>",
+    "eoi_token": "<image|>",
+    "eot_token": "<turn|>",
+    "escape_token": "<|\"|>",
+    "etc_token": "<tool_call|>",
+    "etd_token": "<tool|>",
+    "etr_token": "<tool_response|>",
+    "image_token": "<|image|>",
+    "soc_token": "<|channel>",
+    "sot_token": "<|turn>",
+    "stc_token": "<|tool_call>",
+    "std_token": "<|tool>",
+    "str_token": "<|tool_response>",
+    "think_token": "<|think|>"
+  },
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "processor_class": "Gemma4Processor",
+  "response_schema": {
+    "properties": {
+      "content": {
+        "type": "string"
+      },
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "items": {
+          "properties": {
+            "function": {
+              "properties": {
+                "arguments": {
+                  "additionalProperties": {},
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call"
+                },
+                "name": {
+                  "type": "string"
+                }
+              },
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})"
+            },
+            "type": {
+              "const": "function"
+            }
+          },
+          "type": "object"
+        },
+        "type": "array",
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>"
+      }
+    },
+    "type": "object",
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
+  },
+  "response_template": {
+    "defaults": {
+      "role": "assistant"
+    },
+    "fields": {
+      "content": {
+        "close": [
+          "<turn|>",
+          "<|tool_response>",
+          "<eos>"
+        ],
+        "content": "text"
+      },
+      "thinking": {
+        "close": "<channel|>",
+        "content": "text",
+        "open": "<|channel>thought\n"
+      },
+      "tool_calls": {
+        "close": "<tool_call|>",
+        "content": "json",
+        "content_args": {
+          "string_delims": [
+            [
+              "<|\"|>",
+              "<|\"|>"
+            ]
+          ],
+          "unquoted_keys": true
+        },
+        "open_pattern": "<\\|tool_call>call:(?P<name>\\w+)",
+        "repeats": true,
+        "transform": {
+          "function": {
+            "arguments": "{content}",
+            "name": "{name}"
+          },
+          "type": "function"
+        }
+      }
+    },
+    "start_anchor": [
+      "<|turn>model\n",
+      "<tool_response|>"
+    ]
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>"
+}

bias.md

@@ -0,0 +1,10 @@
+| Field | Response |
+| :---- | :---- |
+| Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/content/protected-classes) in model design and testing: | None |
+| Bias Metric (If Measured): | [BBQ Accuracy Scores in Ambiguous Contexts](https://github.com/nyu-mll/BBQ/) |
+| Which characteristic (feature) show(s) the greatest difference in performance?: | The model shows high variance in the characteristics when it is used with a high temperature. |
+| Measures taken to mitigate against unwanted bias: | Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) employed to calibrate the model’s reasoning capabilities to maintain logical consistency and appropriate complexity when interacting with or interpreting data from diverse age demographics. |
+| If using internal data, description of methods implemented in data acquisition or processing, if any, to address the prevalence of identifiable biases in the training, testing, and validation data: | The training datasets contain a large amount of synthetic data generated by LLMs. We manually curated prompts. |
+| Tools used to assess statistical imbalances and highlight patterns that may introduce bias into AI models: | [BBQ](https://github.com/nyu-mll/BBQ/) |
+| Tools used to assess statistical imbalances and highlight patterns that may introduce bias into AI models: | These datasets, such as web-scraped finance reasoning data derived from SEC EDGAR filings, science and math problem datasets, OpenResearcher/source-document datasets, Common Crawl, CC-News, Wikimedia, and long-context document datasets, do not collectively or exhaustively represent all demographic groups (and proportionally therein). For instance, these datasets do not contain explicit mentions of demographic classes such as age, gender, or ethnicity in approximately 97% to 99.9% of finance reasoning samples and in over 85% of samples across the broader assessed datasets. In the subset where such terms are present, these datasets contain notable representational skews. For example, ethnicity mentions are often dominated by Middle Eastern contexts (found in finance documents) or "White," "Two or more," and "Black or African American" as the most frequent ethnic identifiers, while references categorized as male-only significantly outnumber those categorized as female-only. Furthermore, gender is explicitly mentioned in approximately 12% of samples across the broader dataset assessment, yet in only 0.9% of finance-specific samples. Dataset-level results vary by source type, with long-context/source-document datasets containing higher explicit demographic mention rates compared to certain web-scraped sources. To mitigate these imbalances, we recommend considering evaluation techniques such as bias audits, fine-tuning with demographically balanced datasets, and mitigation strategies such as counterfactual data augmentation to align with the desired model behavior. This evaluation used a 3,000-sample subset per dataset, identified as the optimal threshold for maximizing embedder accuracy. |
+| Unwanted Bias Testing: | Constrained to English-language inputs. Multi-lingual parity is not currently claimed or guaranteed. |

chat_template.jinja

@@ -0,0 +1,415 @@
+{%- if conversational_mode | default(false) -%}
+    {%- set spoken_instruction = "You are having a spoken conversation with the user. Respond as briefly as possible while still addressing their questions fully." -%}
+    {%- if messages and messages[0]['role'] in ['system', 'developer'] -%}
+        {%- set spoken = namespace(text='') -%}
+        {%- if messages[0]['content'] is string -%}
+            {%- set spoken.text = messages[0]['content'] | trim -%}
+        {%- else -%}
+            {%- for part in messages[0]['content'] -%}
+                {%- set spoken.text = spoken.text + (part['text'] | trim) + ' ' -%}
+            {%- endfor -%}
+            {%- set spoken.text = spoken.text | trim -%}
+        {%- endif -%}
+        {%- if spoken.text != spoken_instruction -%}
+            {%- set spoken.text = (spoken.text + '\n\n' if spoken.text else '') + spoken_instruction -%}
+        {%- endif -%}
+        {%- set messages = [{'role': messages[0]['role'], 'content': spoken.text}] + messages[1:] -%}
+    {%- else -%}
+        {%- set messages = [{'role': 'system', 'content': spoken_instruction}] + messages -%}
+    {%- endif -%}
+{%- endif -%}
+{#
+  Template: Nemotron updated Gemma4 chat template, reasoning-off suffix ablation
+  Published: 2026-08-20
+  Context: Preserve content ordering before tool-calls while matching the original
+           Gemma 4 generation prompt when enable_thinking=false.
+#}
+{%- macro format_parameters(properties, required, filter_keys=false) -%}
+    {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}
+    {%- set ns = namespace(found_first=false) -%}
+    {%- for key, value in properties | dictsort -%}
+        {%- set add_comma = false -%}
+        {%- if not filter_keys or key not in standard_keys -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {{ key }}:{
+            {%- if value['description'] -%}
+                description:<|"|>{{ value['description'] }}<|"|>
+                {%- set add_comma = true -%}
+            {%- endif -%}
+            {%- if value['type'] | upper == 'STRING' -%}
+                {%- if value['enum'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    enum:{{ format_argument(value['enum']) }}
+                {%- endif -%}
+            {%- elif value['type'] | upper == 'ARRAY' -%}
+                {%- if value['items'] is mapping and value['items'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    items:{
+                    {%- set ns_items = namespace(found_first=false) -%}
+                    {%- for item_key, item_value in value['items'] | dictsort -%}
+                        {%- if item_value is not none -%}
+                            {%- if ns_items.found_first %},{% endif -%}
+                            {%- set ns_items.found_first = true -%}
+                            {%- if item_key == 'properties' -%}
+                                properties:{
+                                {%- if item_value is mapping -%}
+                                    {{- format_parameters(item_value, value['items']['required'] | default([])) -}}
+                                {%- endif -%}
+                                }
+                            {%- elif item_key == 'required' -%}
+                                required:[
+                                {%- for req_item in item_value -%}
+                                    <|"|>{{- req_item -}}<|"|>
+                                    {%- if not loop.last %},{% endif -%}
+                                {%- endfor -%}
+                                ]
+                            {%- elif item_key == 'type' -%}
+                                {%- if item_value is string -%}
+                                    type:{{ format_argument(item_value | upper) }}
+                                {%- else -%}
+                                    type:{{ format_argument(item_value | map('upper') | list) }}
+                                {%- endif -%}
+                            {%- else -%}
+                                {{ item_key }}:{{ format_argument(item_value) }}
+                            {%- endif -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                    }
+                {%- endif -%}
+            {%- endif -%}
+            {%- if value['nullable'] %}
+                {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                nullable:true
+            {%- endif -%}
+            {%- if value['type'] | upper == 'OBJECT' -%}
+                {%- if value['properties'] is defined and value['properties'] is mapping -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    properties:{
+                    {{- format_parameters(value['properties'], value['required'] | default([])) -}}
+                    }
+                {%- elif value is mapping -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    properties:{
+                    {{- format_parameters(value, value['required'] | default([]), filter_keys=true) -}}
+                    }
+                {%- endif -%}
+                {%- if value['required'] -%}
+                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+                    required:[
+                    {%- for item in value['required'] | default([]) -%}
+                        <|"|>{{- item -}}<|"|>
+                        {%- if not loop.last %},{% endif -%}
+                    {%- endfor -%}
+                    ]
+                {%- endif -%}
+            {%- endif -%}
+            {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}
+            type:<|"|>{{ value['type'] | upper }}<|"|>}
+        {%- endif -%}
+    {%- endfor -%}
+{%- endmacro -%}
+{%- macro format_function_declaration(tool_data) -%}
+    declaration:{{- tool_data['function']['name'] -}}{description:<|"|>{{- tool_data['function']['description'] -}}<|"|>
+    {%- set params = tool_data['function']['parameters'] -%}
+    {%- if params -%}
+        ,parameters:{
+        {%- if params['properties'] -%}
+            properties:{ {{- format_parameters(params['properties'], params['required']) -}} },
+        {%- endif -%}
+        {%- if params['required'] -%}
+            required:[
+            {%- for item in params['required'] -%}
+                <|"|>{{- item -}}<|"|>
+                {{- ',' if not loop.last -}}
+            {%- endfor -%}
+            ],
+        {%- endif -%}
+        {%- if params['type'] -%}
+            type:<|"|>{{- params['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    {%- if 'response' in tool_data['function'] -%}
+        {%- set response_declaration = tool_data['function']['response'] -%}
+        ,response:{
+        {%- if response_declaration['description'] -%}
+            description:<|"|>{{- response_declaration['description'] -}}<|"|>,
+        {%- endif -%}
+        {%- if response_declaration['type'] | upper == 'OBJECT' -%}
+            type:<|"|>{{- response_declaration['type'] | upper -}}<|"|>}
+        {%- endif -%}
+    {%- endif -%}
+    }
+{%- endmacro -%}
+{%- macro format_argument(argument, escape_keys=True) -%}
+    {%- if argument is none -%}
+        {{- 'null' -}}
+    {%- elif argument is string -%}
+        {{- '<|"|>' + argument + '<|"|>' -}}
+    {%- elif argument is boolean -%}
+        {{- 'true' if argument else 'false' -}}
+    {%- elif argument is mapping -%}
+        {{- '{' -}}
+        {%- set ns = namespace(found_first=false) -%}
+        {%- for key, value in argument | dictsort -%}
+            {%- if ns.found_first %},{% endif -%}
+            {%- set ns.found_first = true -%}
+            {%- if escape_keys -%}
+                {{- '<|"|>' + key + '<|"|>' -}}
+            {%- else -%}
+                {{- key -}}
+            {%- endif -%}
+            :{{- format_argument(value, escape_keys=escape_keys) -}}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- elif argument is sequence -%}
+        {{- '[' -}}
+        {%- for item in argument -%}
+            {{- format_argument(item, escape_keys=escape_keys) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- ']' -}}
+    {%- else -%}
+        {{- argument -}}
+    {%- endif -%}
+{%- endmacro -%}
+{%- macro strip_thinking(text) -%}
+    {%- set ns = namespace(result='') -%}
+    {%- for part in text.split('<channel|>') -%}
+        {%- if '<|channel>' in part -%}
+            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}
+        {%- else -%}
+            {%- set ns.result = ns.result + part -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {{- ns.result | trim -}}
+{%- endmacro -%}
+
+{%- macro format_tool_response_block(tool_name, response) -%}
+    {{- '<|tool_response>' -}}
+    {%- if response is mapping -%}
+        {{- 'response:' + tool_name + '{' -}}
+        {%- for key, value in response | dictsort -%}
+            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+            {%- if not loop.last %},{% endif -%}
+        {%- endfor -%}
+        {{- '}' -}}
+    {%- else -%}
+        {{- 'response:' + tool_name + '{value:' + format_argument(response, escape_keys=False) + '}' -}}
+    {%- endif -%}
+    {{- '<tool_response|>' -}}
+{%- endmacro -%}
+
+{#- ===== SETUP ===== -#}
+{%- set ns = namespace(prev_message_type=None, prev_non_tool_role=None) -%}
+{%- set loop_messages = messages -%}
+{%- set enable_thinking = enable_thinking | default(false) -%}
+{%- set preserve_thinking = preserve_thinking | default(false) -%}
+{{- bos_token -}}
+{#- Handle System/Tool Definitions Block -#}
+{%- if enable_thinking or tools or (messages and messages[0]['role'] in ['system', 'developer']) -%}
+    {{- '<|turn>system\n' -}}
+    {#- Inject Thinking token at the very top of the FIRST system turn -#}
+    {%- if enable_thinking -%}
+        {{- '<|think|>\n' -}}
+        {%- set ns.prev_message_type = 'think' -%}
+    {%- endif -%}
+    {%- if messages and messages[0]['role'] in ['system', 'developer'] -%}
+        {%- if messages[0]['content'] is string -%}
+            {{- messages[0]['content'] | trim -}}
+        {%- elif messages[0]['content'] is sequence -%}
+            {%- for item in messages[0]['content'] -%}
+                {{- item['text'] | trim + ' '-}}
+            {%- endfor -%}
+        {%- endif -%}
+        {%- set loop_messages = messages[1:] -%}
+    {%- endif -%}
+    {%- if tools -%}
+        {%- for tool in tools %}
+            {{- '<|tool>' -}}
+            {{- format_function_declaration(tool) | trim -}}
+            {{- '<tool|>' -}}
+        {%- endfor %}
+        {%- set ns.prev_message_type = 'tool' -%}
+    {%- endif -%}
+    {{- '<turn|>\n' -}}
+{%- endif %}
+
+{#- Pre-scan: find last user message index for reasoning guard -#}
+{%- set ns_turn = namespace(last_user_idx=-1) -%}
+{%- for i in range(loop_messages | length) -%}
+    {%- if loop_messages[i]['role'] == 'user' -%}
+        {%- set ns_turn.last_user_idx = i -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{#- Loop through messages -#}
+{%- for message in loop_messages -%}
+    {%- if message['role'] != 'tool' -%}
+    {%- set ns.prev_message_type = None -%}
+    {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}
+    {#- Detect continuation using tracked state — O(1) instead of O(n) backward scan -#}
+    {%- set continue_same_model_turn = (role == 'model' and ns.prev_non_tool_role == 'assistant') -%}
+    {%- if not continue_same_model_turn -%}
+        {{- '<|turn>' + role + '\n' }}
+
+    {%- endif -%}
+
+    {#- Render reasoning/reasoning_content as thinking channel -#}
+    {%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}
+    {%- set thinking_gate = (loop.index0 > ns_turn.last_user_idx) or (preserve_thinking and message.get('tool_calls')) -%}
+    {%- if thinking_text and thinking_gate -%}
+        {{- '<|channel>thought\n' + thinking_text + '\n<channel|>' -}}
+    {%- endif -%}
+
+            {%- set captured_content -%}
+            {%- if message.get('content') is string -%}
+                {%- if role == 'model' -%}
+                    {{- strip_thinking(message['content']) -}}
+                {%- else -%}
+                    {{- message['content'] | trim -}}
+                {%- endif -%}
+            {%- elif message.get('content') is sequence -%}
+                {%- for item in message['content'] -%}
+                    {%- if item.get('type') == 'text' -%}
+                        {%- if role == 'model' -%}
+                            {{- strip_thinking(item['text']) -}}
+                        {%- else -%}
+                            {{- item['text'] | trim -}}
+                        {%- endif -%}
+                    {%- elif item.get('type') in ['image', 'image_url'] -%}
+                        {{- '<|image|>' -}}
+                    {%- elif item.get('type') in ['audio', 'input_audio'] -%}
+                        {{- '<|audio|>' -}}
+                    {%- elif item.get('type') == 'video' -%}
+                        {{- '<|video|>' -}}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+            {%- endset -%}
+
+            {#- The sole intentional behavior change from HF main: preserve
+                OpenAI assistant content before its tool calls. -#}
+            {%- if role == 'model' and message.get('tool_calls') -%}
+                {{- captured_content -}}
+            {%- endif -%}
+
+            {%- if message.get('tool_calls') -%}
+                {%- for tool_call in message.get('tool_calls') -%}
+                    {%- set function = tool_call['function'] -%}
+                    {{- '<|tool_call>call:' + function['name'] + '{' -}}
+                    {%- if function['arguments'] is mapping -%}
+                        {%- set ns_args = namespace(found_first=false) -%}
+                        {%- for key, value in function['arguments'] | dictsort -%}
+                            {%- if ns_args.found_first %},{% endif -%}
+                            {%- set ns_args.found_first = true -%}
+                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}
+                        {%- endfor -%}
+                    {%- elif function['arguments'] is none -%}
+                    {%- else -%}
+                        {{- raise_exception(
+                            "chat_template: tool_calls[].function.arguments must be a "
+                            "JSON object (mapping), not a string. Deserialize arguments "
+                            "before passing to the template."
+                        ) -}}
+                    {%- endif -%}
+                    {{- '}<tool_call|>' -}}
+                {%- endfor -%}
+                {%- set ns.prev_message_type = 'tool_call' -%}
+            {%- endif -%}
+
+            {%- set ns_tr_out = namespace(flag=false) -%}
+            {%- if message.get('tool_responses') -%}
+                {#- Legacy: tool_responses embedded on the assistant message (Google/Gemma native) -#}
+                {%- for tool_response in message.get('tool_responses') -%}
+                    {{- format_tool_response_block(tool_response['name'] | default('unknown', true), tool_response['response']) -}}
+                    {%- set ns_tr_out.flag = true -%}
+                    {%- set ns.prev_message_type = 'tool_response' -%}
+                {%- endfor -%}
+            {%- elif message.get('tool_calls') -%}
+                {#- OpenAI Chat Completions: forward-scan consecutive role:tool messages -#}
+                {%- set ns_tool_scan = namespace(stopped=false) -%}
+                {%- for k in range(loop.index0 + 1, loop_messages | length) -%}
+                    {%- if ns_tool_scan.stopped -%}
+                    {%- elif loop_messages[k]['role'] != 'tool' -%}
+                        {%- set ns_tool_scan.stopped = true -%}
+                    {%- else -%}
+                        {%- set follow = loop_messages[k] -%}
+                        {#- Resolve tool_call_id to function name -#}
+                        {%- set ns_tname = namespace(name=follow.get('name') or 'unknown') -%}
+                        {%- for tc in message.get('tool_calls') -%}
+                            {%- if tc.get('id') == follow.get('tool_call_id') -%}
+                                {%- set ns_tname.name = tc['function']['name'] -%}
+                            {%- endif -%}
+                        {%- endfor -%}
+                        {#- Handle content as string or content-parts array -#}
+                        {%- set tool_body = follow.get('content') -%}
+                        {%- if tool_body is string -%}
+                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}
+                        {%- elif tool_body is sequence and tool_body is not string -%}
+                            {%- set ns_txt = namespace(s='') -%}
+                            {%- for part in tool_body -%}
+                                {%- if part.get('type') == 'text' -%}
+                                    {%- set ns_txt.s = ns_txt.s + (part.get('text') | default('')) -%}
+                                {%- endif -%}
+                            {%- endfor -%}
+                            {{- format_tool_response_block(ns_tname.name, ns_txt.s) -}}
+                            {%- for part in tool_body -%}
+                                {%- if part.get('type') in ['image', 'image_url'] -%}
+                                    {{- '<|image|>' -}}
+                                {%- elif part.get('type') in ['audio', 'input_audio'] -%}
+                                    {{- '<|audio|>' -}}
+                                {%- elif part.get('type') == 'video' -%}
+                                    {{- '<|video|>' -}}
+                                {%- endif -%}
+                            {%- endfor -%}
+                        {%- else -%}
+                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}
+                        {%- endif -%}
+                        {%- set ns_tr_out.flag = true -%}
+                        {%- set ns.prev_message_type = 'tool_response' -%}
+                    {%- endif -%}
+                {%- endfor -%}
+            {%- endif -%}
+
+            {%- if not (role == 'model' and message.get('tool_calls')) -%}
+                {{- captured_content -}}
+            {%- endif -%}
+            {%- set has_content = captured_content | trim | length > 0 -%}
+
+        {#- Forward-scan: find next non-tool message role for continuation detection -#}
+        {%- set next_nt = namespace(role=None, found=false) -%}
+        {%- for j in range(loop.index0 + 1, loop_messages | length) -%}
+            {%- if not next_nt.found -%}
+                {%- if loop_messages[j]['role'] != 'tool' -%}
+                    {%- set next_nt.role = loop_messages[j]['role'] -%}
+                    {%- set next_nt.found = true -%}
+                {%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+
+        {%- set continues_into_next = (
+            role == 'model'
+            and next_nt.role == 'assistant'
+            and (not message.get('tool_calls') or ns_tr_out.flag)
+        ) -%}
+
+        {%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}
+            {{- '<|tool_response>' -}}
+        {%- elif continues_into_next -%}
+        {%- elif not (ns_tr_out.flag and not has_content and not next_nt.found) -%}
+            {{- '<turn|>\n' -}}
+        {%- endif -%}
+
+    {#- Track previous non-tool role for next iteration (avoids O(n) backward scan) -#}
+    {%- set ns.prev_non_tool_role = message['role'] -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{%- if add_generation_prompt -%}
+    {%- if ns.prev_message_type != 'tool_response' and ns.prev_message_type != 'tool_call' -%}
+        {{- '<|turn>model\n' -}}
+    {%- elif ns.prev_message_type == 'tool_response' and enable_thinking -%}
+        {{- '<|channel>thought\n' -}}
+    {%- endif -%}
+{%- endif -%}

config.json

@@ -0,0 +1,197 @@
+{
+  "architectures": [
+    "Gemma4ForConditionalGeneration"
+  ],
+  "audio_config": {
+    "_name_or_path": "",
+    "architectures": null,
+    "attention_chunk_size": 12,
+    "attention_context_left": 13,
+    "attention_context_right": 0,
+    "attention_invalid_logits_value": -1000000000.0,
+    "attention_logit_cap": 50.0,
+    "chunk_size_feed_forward": 0,
+    "conv_kernel_size": 5,
+    "dtype": "bfloat16",
+    "gradient_clipping": 10000000000.0,
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_range": 0.02,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "model_type": "gemma4_audio",
+    "num_attention_heads": 8,
+    "num_hidden_layers": 12,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "output_proj_dims": 1536,
+    "problem_type": null,
+    "residual_weight": 0.5,
+    "return_dict": true,
+    "rms_norm_eps": 1e-06,
+    "subsampling_conv_channels": [
+      128,
+      32
+    ],
+    "use_clipped_linears": true
+  },
+  "audio_token_id": 258881,
+  "boa_token_id": 256000,
+  "boi_token_id": 255999,
+  "dtype": "bfloat16",
+  "eoa_token_id": 258883,
+  "eoa_token_index": 258883,
+  "eoi_token_id": 258882,
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "image_token_id": 258880,
+  "initializer_range": 0.02,
+  "model_type": "gemma4",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attention_k_eq_v": false,
+    "bos_token_id": 2,
+    "dtype": "bfloat16",
+    "enable_moe_block": false,
+    "eos_token_id": 1,
+    "expert_intermediate_size": null,
+    "final_logit_softcapping": 30.0,
+    "global_head_dim": 512,
+    "head_dim": 256,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 2560,
+    "hidden_size_per_layer_input": 256,
+    "initializer_range": 0.02,
+    "intermediate_size": 10240,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "max_position_embeddings": 65536,
+    "model_type": "gemma4_text",
+    "num_attention_heads": 8,
+    "num_experts": null,
+    "num_global_key_value_heads": null,
+    "num_hidden_layers": 42,
+    "num_key_value_heads": 2,
+    "num_kv_shared_layers": 18,
+    "pad_token_id": 0,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "full_attention": {
+        "partial_rotary_factor": 0.25,
+        "rope_theta": 1000000.0,
+        "rope_type": "proportional"
+      },
+      "sliding_attention": {
+        "rope_theta": 10000.0,
+        "rope_type": "default"
+      }
+    },
+    "sliding_window": 512,
+    "tie_word_embeddings": true,
+    "top_k_experts": null,
+    "use_bidirectional_attention": null,
+    "use_cache": true,
+    "use_double_wide_mlp": false,
+    "vocab_size": 262144,
+    "vocab_size_per_layer_input": 262144
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.5.0.dev0",
+  "video_token_id": 258884,
+  "vision_config": {
+    "_name_or_path": "",
+    "architectures": null,
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "chunk_size_feed_forward": 0,
+    "default_output_length": 280,
+    "dtype": "bfloat16",
+    "global_head_dim": 64,
+    "head_dim": 64,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 768,
+    "id2label": {
+      "0": "LABEL_0",
+      "1": "LABEL_1"
+    },
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "is_encoder_decoder": false,
+    "label2id": {
+      "LABEL_0": 0,
+      "LABEL_1": 1
+    },
+    "max_position_embeddings": 131072,
+    "model_type": "gemma4_vision",
+    "num_attention_heads": 12,
+    "num_hidden_layers": 16,
+    "num_key_value_heads": 12,
+    "output_attentions": false,
+    "output_hidden_states": false,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "position_embedding_size": 10240,
+    "problem_type": null,
+    "return_dict": true,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "rope_theta": 100.0,
+      "rope_type": "default"
+    },
+    "standardize": false,
+    "use_clipped_linears": true
+  },
+  "vision_soft_tokens_per_image": 280
+}

explainability.md

@@ -0,0 +1,14 @@
+| Field | Response |
+| :---- | :---- |
+| Intended Task/Domain: | Text generation, reasoning, and chat |
+| Model Type: | Text-to-text Gemma-4 |
+| Intended Users: | Generative AI creators working with conversational AI models and image content. |
+| Output: | Text |
+| Tools used to evaluate datasets to identify synthetic data and ensure data authenticity. | We used a Gemma-3 4B-based filtering model fine-tuned on [Nemotron Content Safety Dataset v2](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0) to ensure the quality of synthetic data. |
+| Describe how the model works: | Generates text by predicting the next word or token based on the context provided in the input sequence using multiple self-attention layers. |
+| List the groups (or group characteristics) for which this was tested to produce comparable outcomes: | Age, Disability Status, Gender Identity, Nationality, Physical Appearance, Ethnicity, Socioeconomic Status, Sexual Orientation, Religion |
+| Technical Limitations & Mitigation: | This model performs particularly well in instruction following regimes, as such may be strongly influenced by untrusted inputs and should be paired with appropriate guardrails and data filtering to better align use-case behaviors when exposed to such data. |
+| Verified to have met prescribed NVIDIA quality standards: | Yes |
+| Performance Metrics: | Accuracy, Throughput, and User-side throughput |
+| Potential Known Risks: | The model was optimized explicitly for instruction following and as such is more susceptible to prompt injection and jailbreaking in various forms as a result of its instruction tuning. This means that the model should be paired with additional rails or system filtering to limit exposure to instructions from malicious sources -- either directly or indirectly by retrieval (e.g. via visiting a website) -- as they may yield outputs that can lead to harmful, system-level outcomes up to and including remote code execution in agentic systems when effective security controls including guardrails are not in place. The model may generate answers that may be inaccurate, omit key information, include irrelevant or redundant text, or produce socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive. |
+| Licensing: | Use of this model is governed by the [OpenMDW License Agreement, version 1.1](https://github.com/OpenMDW/OpenMDW/blob/main/1.1/LICENSE.OpenMDW-1.1).  This model has been post-trained on the Google Gemma 4 model, and use of the Google Gemma 4 model is governed by [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0) |

generation_config.json

@@ -0,0 +1,14 @@
+{
+  "bos_token_id": 2,
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    106,
+    50
+  ],
+  "pad_token_id": 0,
+  "temperature": 1.0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "5.5.0.dev0"
+}

handler.py

@@ -0,0 +1,183 @@
+"""Custom Hugging Face Inference Endpoint handler for Gemma 4 checkpoints.
+
+The checkpoint is a Gemma 4 image-text-to-text, tool-calling model
+(``Gemma4ForConditionalGeneration``).  The handler deliberately loads the
+processor from ``path`` and calls its ``apply_chat_template`` method so the
+``chat_template.jinja`` packaged with the individual checkpoint is used.
+
+Request payload (``data``) keys:
+
+    inputs      (required) A string prompt or a list of chat messages.
+    tools       (optional) OpenAI-style tool/function schemas.
+    parameters  (optional) Keyword arguments forwarded to ``model.generate``.
+    enable_thinking
+                (optional) Gemma 4 chat-template control; defaults to false,
+                matching the checkpoint template's own default.
+    chat_template_kwargs
+                (optional) Additional checkpoint-template keyword arguments.
+
+Message content can be a string or a list of text/image/video parts. Image
+parts accept ``image`` or ``image_url`` as a URL, data URI, or base64 string.
+
+Response: ``[{"generated_text": "..."}]``.
+"""
+
+from __future__ import annotations
+
+import base64
+import io
+from typing import Any, Dict, List, Optional, Union
+
+import torch
+from PIL import Image
+from transformers import AutoModelForImageTextToText, AutoProcessor
+
+
+DEFAULT_GENERATION_KWARGS: Dict[str, Any] = {
+    "max_new_tokens": 1024,
+    "do_sample": False,
+    "temperature": 0.7,
+    "top_p": 0.9,
+}
+
+
+class EndpointHandler:
+    def __init__(self, path: str = "") -> None:
+        # ``path`` is the model repository root supplied by the endpoint.
+        # Loading from this path also loads that checkpoint's chat_template.jinja.
+        self.processor = AutoProcessor.from_pretrained(path, trust_remote_code=True)
+
+        dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32
+        self.model = AutoModelForImageTextToText.from_pretrained(
+            path,
+            dtype=dtype,
+            device_map="auto" if torch.cuda.is_available() else None,
+            trust_remote_code=True,
+        )
+        self.model.eval()
+
+        self.tokenizer = getattr(self.processor, "tokenizer", self.processor)
+
+    @staticmethod
+    def _load_image(ref: str) -> Image.Image:
+        """Load a PIL image from a URL, data URI, or raw base64 string."""
+        if ref.startswith(("http://", "https://")):
+            import requests
+
+            response = requests.get(ref, timeout=30)
+            response.raise_for_status()
+            return Image.open(io.BytesIO(response.content)).convert("RGB")
+
+        if ref.startswith("data:"):
+            try:
+                ref = ref.split(",", 1)[1]
+            except IndexError as exc:
+                raise ValueError("Malformed image data URI") from exc
+
+        return Image.open(io.BytesIO(base64.b64decode(ref))).convert("RGB")
+
+    def _collect_images(self, messages: List[Dict[str, Any]]) -> List[Image.Image]:
+        images: List[Image.Image] = []
+        for message in messages:
+            content = message.get("content")
+            if not isinstance(content, list):
+                continue
+            for item in content:
+                if not isinstance(item, dict):
+                    continue
+                if item.get("type") == "image" or "image" in item or "image_url" in item:
+                    ref = item.get("image") or item.get("image_url")
+                    if isinstance(ref, dict):
+                        ref = ref.get("url")
+                    if isinstance(ref, str):
+                        images.append(self._load_image(ref))
+        return images
+
+    @staticmethod
+    def _build_messages(
+        inputs: Union[str, List[Dict[str, Any]]],
+    ) -> List[Dict[str, Any]]:
+        if isinstance(inputs, str):
+            return [{"role": "user", "content": inputs}]
+        if not isinstance(inputs, list):
+            raise TypeError("'inputs' must be a string or a list of chat messages")
+        return inputs
+
+    def _decode_completion(self, generated_ids: torch.Tensor) -> str:
+        # Gemma 4's channel and tool-call delimiters are special tokens. Do not
+        # use skip_special_tokens=True: doing so corrupts structured tool output.
+        text = self.processor.batch_decode(
+            generated_ids,
+            skip_special_tokens=False,
+            clean_up_tokenization_spaces=False,
+        )[0]
+
+        # Remove generation terminators without stripping structural special tokens.
+        for token in (self.tokenizer.eos_token, self.tokenizer.pad_token):
+            if token:
+                while text.endswith(token):
+                    text = text[: -len(token)]
+        return text
+
+    @torch.inference_mode()
+    def __call__(self, data: Dict[str, Any]) -> List[Dict[str, Any]]:
+        if not isinstance(data, dict):
+            raise TypeError("Request payload must be a dictionary")
+
+        request = dict(data)
+        if "inputs" not in request:
+            raise ValueError("Request payload must include 'inputs'")
+
+        inputs = request.pop("inputs")
+        tools: Optional[List[Dict[str, Any]]] = request.pop("tools", None)
+        parameters: Dict[str, Any] = request.pop("parameters", {}) or {}
+        enable_thinking = bool(request.pop("enable_thinking", False))
+        chat_template_kwargs: Dict[str, Any] = request.pop(
+            "chat_template_kwargs", {}
+        ) or {}
+
+        if request:
+            unknown = ", ".join(sorted(request))
+            raise ValueError(f"Unknown request key(s): {unknown}")
+        if not isinstance(parameters, dict):
+            raise TypeError("'parameters' must be a dictionary")
+        if not isinstance(chat_template_kwargs, dict):
+            raise TypeError("'chat_template_kwargs' must be a dictionary")
+
+        messages = self._build_messages(inputs)
+        images = self._collect_images(messages)
+
+        # Top-level enable_thinking is authoritative and is passed directly to
+        # the packaged Gemma 4 Jinja template.
+        chat_template_kwargs = {
+            **chat_template_kwargs,
+            "enable_thinking": enable_thinking,
+        }
+        prompt = self.processor.apply_chat_template(
+            messages,
+            tools=tools,
+            tokenize=False,
+            add_generation_prompt=True,
+            **chat_template_kwargs,
+        )
+
+        processor_kwargs: Dict[str, Any] = {
+            "text": [prompt],
+            "return_tensors": "pt",
+        }
+        if images:
+            processor_kwargs["images"] = images
+
+        model_inputs = self.processor(**processor_kwargs)
+        model_inputs = {key: value.to(self.model.device) for key, value in model_inputs.items()}
+
+        generation_kwargs = {**DEFAULT_GENERATION_KWARGS, **parameters}
+        if not generation_kwargs.get("do_sample", False):
+            for key in ("temperature", "top_p", "top_k"):
+                generation_kwargs.pop(key, None)
+
+        generated_ids = self.model.generate(**model_inputs, **generation_kwargs)
+        input_length = model_inputs["input_ids"].shape[1]
+        new_tokens = generated_ids[:, input_length:]
+
+        return [{"generated_text": self._decode_completion(new_tokens)}]

inference_example.py

@@ -0,0 +1,27 @@
+"""Run FleetWorker-E4B on one task-board item. Use the project field; ignore priority (see the model card).
+
+    pip install "transformers>=5" torch
+    python inference_example.py "title: Add retry with backoff to the nightly export job"
+"""
+import json
+import sys
+
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+REPO = "ThakiCloud/FleetWorker-E4B"
+LABELS = ["ai-experiment-measurement", "gitops-deployment-health", "synthetic-data-pipeline", "company-operations-agents", "ondevice-small-model", "todo-board-tooling", "enterprise-ops-benchmark", "shop-assistant-console", "commerce-growth-agents", "platform-strategy-planning", "decision-routing-research", "legal-decision-models", "secops-automation-console", "multimodal-api-gateway", "character-ai-app", "telecom-phy-research", "engineering-blog-publishing", "generative-video-training", "semiconductor-design-llm", "agent-trajectory-distillation", "korean-humanization-corpus", "edge-inference-box", "unattended-agent-runs", "personal-agent-system", "music-generation-rooms"]
+SYSTEM = "You triage a task-board item. Output exactly one compact JSON object with keys \"project\" and \"priority\" and nothing else. \"project\" is one of: ai-experiment-measurement, gitops-deployment-health, synthetic-data-pipeline, company-operations-agents, ondevice-small-model, todo-board-tooling, enterprise-ops-benchmark, shop-assistant-console, commerce-growth-agents, platform-strategy-planning, decision-routing-research, legal-decision-models, secops-automation-console, multimodal-api-gateway, character-ai-app, telecom-phy-research, engineering-blog-publishing, generative-video-training, semiconductor-design-llm, agent-trajectory-distillation, korean-humanization-corpus, edge-inference-box, unattended-agent-runs, personal-agent-system, music-generation-rooms, or \"other\"; \"unknown\" if the text gives no way to tell. \"priority\" is one of: P0, P1, P2, P3."
+
+tok = AutoTokenizer.from_pretrained(REPO)
+model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto").eval()
+stop = [i for i in {tok.eos_token_id, tok.convert_tokens_to_ids("<turn|>")} if isinstance(i, int)]
+item = sys.argv[1] if len(sys.argv) > 1 else "title: Add retry with backoff to the nightly export job"
+messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": item}]
+prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
+ids = tok(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
+with torch.no_grad():
+    out = model.generate(**ids, max_new_tokens=64, do_sample=False, eos_token_id=stop)
+text = tok.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=False).split("<turn|>")[0].strip()
+pred = json.loads(text)
+print({"project": pred.get("project"), "valid_label": pred.get("project") in LABELS + ["other", "unknown"]})

model-00001-of-00001.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:05a7213f8b0bf2409de51b5f0bba67aeae8d0d83383aaaebf64395303eb4d675
+size 15882477468

model-frozen-shared-kv.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:32aaa38402af5a51c6d6b27aeb8a26d3e6964e5590330f0e7b2f546608da4023
+size 110118048

privacy.md

@@ -0,0 +1,16 @@
+| Field | Response |
+| :---- | :---- |
+| Generatable or reverse engineerable personal data\*? | No |
+| Personal data used to create this model? | No |
+| Was consent obtained for any personal data\* used? | Not Applicable |
+| (For GPAI Models): A description of any methods implemented in data acquisition or processing, if any, to address the prevalence of personal data in the training data, where relevant and applicable. | We used only prompts that do not contain any personal data for synthetic data generation. |
+| How often is dataset reviewed? | Before Release |
+| Is a mechanism in place to honor data subject right of access or deletion of personal data? | Not Applicable |
+| If personal data was collected for the development of the model, was it collected directly by NVIDIA? | Not Applicable |
+| If personal data was collected for the development of the model by NVIDIA, do you maintain or have access to disclosures made to data subjects? | Not Applicable |
+| If personal data was collected for the development of this AI model, was it minimized to only what was required? | Not Applicable |
+| Was data from user interactions with the AI model (e.g. user input and prompts) used to train the model? | No |
+| Is there provenance for all datasets used in training? | Yes |
+| Does data labeling (annotation, metadata) comply with privacy laws? | Yes |
+| Is data compliant with data subject requests for data correction or removal, if such a request was made? | No, not possible with externally-sourced data. |
+| Applicable Privacy Policy | [https://www.nvidia.com/en-us/about-nvidia/privacy-policy/](https://www.nvidia.com/en-us/about-nvidia/privacy-policy/) |

processor_config.json

@@ -0,0 +1,75 @@
+{
+  "audio_ms_per_token": 40,
+  "audio_seq_length": 750,
+  "feature_extractor": {
+    "dither": 0.0,
+    "feature_extractor_type": "Gemma4AudioFeatureExtractor",
+    "feature_size": 128,
+    "fft_length": 512,
+    "fft_overdrive": false,
+    "frame_length": 320,
+    "hop_length": 160,
+    "input_scale_factor": 1.0,
+    "max_frequency": 8000.0,
+    "mel_floor": 0.001,
+    "min_frequency": 0.0,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "per_bin_mean": null,
+    "per_bin_stddev": null,
+    "preemphasis": 0.0,
+    "preemphasis_htk_flavor": true,
+    "return_attention_mask": true,
+    "sampling_rate": 16000
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": false,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_processor_type": "Gemma4ImageProcessor",
+    "image_seq_length": 280,
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 280,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098
+  },
+  "image_seq_length": 280,
+  "processor_class": "Gemma4Processor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "image_mean": [
+      0.0,
+      0.0,
+      0.0
+    ],
+    "image_std": [
+      1.0,
+      1.0,
+      1.0
+    ],
+    "max_soft_tokens": 70,
+    "num_frames": 32,
+    "patch_size": 16,
+    "pooling_kernel_size": 3,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "video_processor_type": "Gemma4VideoProcessor"
+  }
+}

safety.md

@@ -0,0 +1,9 @@
+| Field | Response |
+| :---- | :---- |
+| Model Application Field(s): | Chat, Instruction Following, Chatbot Development, Code Generation, Reasoning, Customer Service |
+| Describe the life critical impact (if present). | Not Applicable |
+| Description of methods implemented in data acquisition or processing, if any, to address other types of potentially harmful data in the training, testing, and validation data: | We used a guard model for content safety to exclude potentially harmful data from training.  |
+| Description of any methods implemented in data acquisition or processing, if any, to address illegal or harmful content in the training data, including, but not limited to, child sexual abuse material (CSAM) and non-consensual intimate imagery (NCII) | We used a Gemma-3 4B-based guard model trained on [Nemotron Content Safety Dataset v2](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0) for content safety to exclude potentially illegal or harmful content from the training. |
+| Use Case Restrictions: | Use of this model is governed by the [OpenMDW License Agreement, version 1.1](https://github.com/OpenMDW/OpenMDW/blob/main/1.1/LICENSE.OpenMDW-1.1).  This model has been post-trained on the Google Gemma 4 model, and use of the Google Gemma 4 model is governed by [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)|
+| Model and dataset restrictions: | The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development.  Restrictions enforce dataset access during training, and dataset license constraints adhered to. |
+| This AI model was developed based on our policies to ensure responsible data handling and risk mitigation. The datasets used for training have been scanned for harmful content and illegal content, consistent with our policies including scanning for Child Sexual Abuse Material (CSAM). Ongoing review and monitoring mechanisms are in place based on our policies and to maintain data integrity. | True. We use [Nemotron Content Safety Dataset V2](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0) and an internal safety dataset specialized for minority sexuality for content safety evaluation to ensure the safety of this model. |

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc8d3a0ce36466ccc1278bf987df5f71db1719b9ca6b4118264f45cb627bfe0f
+size 32169626

tokenizer_config.json

@@ -0,0 +1,121 @@
+{
+  "audio_token": "<|audio|>",
+  "backend": "tokenizers",
+  "boa_token": "<|audio>",
+  "boi_token": "<|image>",
+  "bos_token": "<bos>",
+  "eoa_token": "<audio|>",
+  "eoc_token": "<channel|>",
+  "eoi_token": "<image|>",
+  "eos_token": "<eos>",
+  "eot_token": "<turn|>",
+  "escape_token": "<|\"|>",
+  "etc_token": "<tool_call|>",
+  "etd_token": "<tool|>",
+  "etr_token": "<tool_response|>",
+  "extra_special_tokens": [
+    "<|video|>"
+  ],
+  "image_token": "<|image|>",
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "padding_side": "left",
+  "processor_class": "Gemma4Processor",
+  "response_schema": {
+    "type": "object",
+    "properties": {
+      "role": {
+        "const": "assistant"
+      },
+      "thinking": {
+        "type": "string"
+      },
+      "content": {
+        "type": "string"
+      },
+      "tool_calls": {
+        "x-regex-iterator": "<\\|tool_call>(.*?)<tool_call\\|>",
+        "type": "array",
+        "items": {
+          "type": "object",
+          "properties": {
+            "type": {
+              "const": "function"
+            },
+            "function": {
+              "type": "object",
+              "x-regex": "call\\:(?P<name>\\w+)(?P<arguments>\\{.*\\})",
+              "properties": {
+                "name": {
+                  "type": "string"
+                },
+                "arguments": {
+                  "type": "object",
+                  "x-parser": "gemma4-tool-call",
+                  "additionalProperties": {}
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "x-regex": "(\\<\\|channel\\>thought\\n(?P<thinking>.*?)\\<channel\\|\\>)?(?P<tool_calls>\\<\\|tool_call\\>.*\\<tool_call\\|\\>)?(?P<content>(?:(?!\\<turn\\|\\>)(?!\\<\\|tool_response\\>).)+)?(?:\\<turn\\|\\>|\\<\\|tool_response\\>)?"
+  },
+  "response_template": {
+    "defaults": {
+      "role": "assistant"
+    },
+    "fields": {
+      "content": {
+        "close": [
+          "<turn|>",
+          "<|tool_response>",
+          "<eos>"
+        ],
+        "content": "text"
+      },
+      "thinking": {
+        "close": "<channel|>",
+        "content": "text",
+        "open": "<|channel>thought\n"
+      },
+      "tool_calls": {
+        "close": "<tool_call|>",
+        "content": "json",
+        "content_args": {
+          "string_delims": [
+            [
+              "<|\"|>",
+              "<|\"|>"
+            ]
+          ],
+          "unquoted_keys": true
+        },
+        "open_pattern": "<\\|tool_call>call:(?P<name>\\w+)",
+        "repeats": true,
+        "transform": {
+          "function": {
+            "arguments": "{content}",
+            "name": "{name}"
+          },
+          "type": "function"
+        }
+      }
+    },
+    "start_anchor": [
+      "<|turn>model\n",
+      "<tool_response|>"
+    ]
+  },
+  "soc_token": "<|channel>",
+  "sot_token": "<|turn>",
+  "stc_token": "<|tool_call>",
+  "std_token": "<|tool>",
+  "str_token": "<|tool_response>",
+  "think_token": "<|think|>",
+  "tokenizer_class": "GemmaTokenizer",
+  "unk_token": "<unk>",
+  "chat_template": "{%- if conversational_mode | default(false) -%}\n    {%- set spoken_instruction = \"You are having a spoken conversation with the user. Respond as briefly as possible while still addressing their questions fully.\" -%}\n    {%- if messages and messages[0]['role'] in ['system', 'developer'] -%}\n        {%- set spoken = namespace(text='') -%}\n        {%- if messages[0]['content'] is string -%}\n            {%- set spoken.text = messages[0]['content'] | trim -%}\n        {%- else -%}\n            {%- for part in messages[0]['content'] -%}\n                {%- set spoken.text = spoken.text + (part['text'] | trim) + ' ' -%}\n            {%- endfor -%}\n            {%- set spoken.text = spoken.text | trim -%}\n        {%- endif -%}\n        {%- if spoken.text != spoken_instruction -%}\n            {%- set spoken.text = (spoken.text + '\\n\\n' if spoken.text else '') + spoken_instruction -%}\n        {%- endif -%}\n        {%- set messages = [{'role': messages[0]['role'], 'content': spoken.text}] + messages[1:] -%}\n    {%- else -%}\n        {%- set messages = [{'role': 'system', 'content': spoken_instruction}] + messages -%}\n    {%- endif -%}\n{%- endif -%}\n{#\n  Template: Nemotron updated Gemma4 chat template, reasoning-off suffix ablation\n  Published: 2026-08-20\n  Context: Preserve content ordering before tool-calls while matching the original\n           Gemma 4 generation prompt when enable_thinking=false.\n#}\n{%- macro format_parameters(properties, required, filter_keys=false) -%}\n    {%- set standard_keys = ['description', 'type', 'properties', 'required', 'nullable'] -%}\n    {%- set ns = namespace(found_first=false) -%}\n    {%- for key, value in properties | dictsort -%}\n        {%- set add_comma = false -%}\n        {%- if not filter_keys or key not in standard_keys -%}\n            {%- if ns.found_first %},{% endif -%}\n            {%- set ns.found_first = true -%}\n            {{ key }}:{\n            {%- if value['description'] -%}\n                description:<|\"|>{{ value['description'] }}<|\"|>\n                {%- set add_comma = true -%}\n            {%- endif -%}\n            {%- if value['type'] | upper == 'STRING' -%}\n                {%- if value['enum'] -%}\n                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}\n                    enum:{{ format_argument(value['enum']) }}\n                {%- endif -%}\n            {%- elif value['type'] | upper == 'ARRAY' -%}\n                {%- if value['items'] is mapping and value['items'] -%}\n                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}\n                    items:{\n                    {%- set ns_items = namespace(found_first=false) -%}\n                    {%- for item_key, item_value in value['items'] | dictsort -%}\n                        {%- if item_value is not none -%}\n                            {%- if ns_items.found_first %},{% endif -%}\n                            {%- set ns_items.found_first = true -%}\n                            {%- if item_key == 'properties' -%}\n                                properties:{\n                                {%- if item_value is mapping -%}\n                                    {{- format_parameters(item_value, value['items']['required'] | default([])) -}}\n                                {%- endif -%}\n                                }\n                            {%- elif item_key == 'required' -%}\n                                required:[\n                                {%- for req_item in item_value -%}\n                                    <|\"|>{{- req_item -}}<|\"|>\n                                    {%- if not loop.last %},{% endif -%}\n                                {%- endfor -%}\n                                ]\n                            {%- elif item_key == 'type' -%}\n                                {%- if item_value is string -%}\n                                    type:{{ format_argument(item_value | upper) }}\n                                {%- else -%}\n                                    type:{{ format_argument(item_value | map('upper') | list) }}\n                                {%- endif -%}\n                            {%- else -%}\n                                {{ item_key }}:{{ format_argument(item_value) }}\n                            {%- endif -%}\n                        {%- endif -%}\n                    {%- endfor -%}\n                    }\n                {%- endif -%}\n            {%- endif -%}\n            {%- if value['nullable'] %}\n                {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}\n                nullable:true\n            {%- endif -%}\n            {%- if value['type'] | upper == 'OBJECT' -%}\n                {%- if value['properties'] is defined and value['properties'] is mapping -%}\n                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}\n                    properties:{\n                    {{- format_parameters(value['properties'], value['required'] | default([])) -}}\n                    }\n                {%- elif value is mapping -%}\n                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}\n                    properties:{\n                    {{- format_parameters(value, value['required'] | default([]), filter_keys=true) -}}\n                    }\n                {%- endif -%}\n                {%- if value['required'] -%}\n                    {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}\n                    required:[\n                    {%- for item in value['required'] | default([]) -%}\n                        <|\"|>{{- item -}}<|\"|>\n                        {%- if not loop.last %},{% endif -%}\n                    {%- endfor -%}\n                    ]\n                {%- endif -%}\n            {%- endif -%}\n            {%- if add_comma %},{%- else -%} {%- set add_comma = true -%} {% endif -%}\n            type:<|\"|>{{ value['type'] | upper }}<|\"|>}\n        {%- endif -%}\n    {%- endfor -%}\n{%- endmacro -%}\n{%- macro format_function_declaration(tool_data) -%}\n    declaration:{{- tool_data['function']['name'] -}}{description:<|\"|>{{- tool_data['function']['description'] -}}<|\"|>\n    {%- set params = tool_data['function']['parameters'] -%}\n    {%- if params -%}\n        ,parameters:{\n        {%- if params['properties'] -%}\n            properties:{ {{- format_parameters(params['properties'], params['required']) -}} },\n        {%- endif -%}\n        {%- if params['required'] -%}\n            required:[\n            {%- for item in params['required'] -%}\n                <|\"|>{{- item -}}<|\"|>\n                {{- ',' if not loop.last -}}\n            {%- endfor -%}\n            ],\n        {%- endif -%}\n        {%- if params['type'] -%}\n            type:<|\"|>{{- params['type'] | upper -}}<|\"|>}\n        {%- endif -%}\n    {%- endif -%}\n    {%- if 'response' in tool_data['function'] -%}\n        {%- set response_declaration = tool_data['function']['response'] -%}\n        ,response:{\n        {%- if response_declaration['description'] -%}\n            description:<|\"|>{{- response_declaration['description'] -}}<|\"|>,\n        {%- endif -%}\n        {%- if response_declaration['type'] | upper == 'OBJECT' -%}\n            type:<|\"|>{{- response_declaration['type'] | upper -}}<|\"|>}\n        {%- endif -%}\n    {%- endif -%}\n    }\n{%- endmacro -%}\n{%- macro format_argument(argument, escape_keys=True) -%}\n    {%- if argument is none -%}\n        {{- 'null' -}}\n    {%- elif argument is string -%}\n        {{- '<|\"|>' + argument + '<|\"|>' -}}\n    {%- elif argument is boolean -%}\n        {{- 'true' if argument else 'false' -}}\n    {%- elif argument is mapping -%}\n        {{- '{' -}}\n        {%- set ns = namespace(found_first=false) -%}\n        {%- for key, value in argument | dictsort -%}\n            {%- if ns.found_first %},{% endif -%}\n            {%- set ns.found_first = true -%}\n            {%- if escape_keys -%}\n                {{- '<|\"|>' + key + '<|\"|>' -}}\n            {%- else -%}\n                {{- key -}}\n            {%- endif -%}\n            :{{- format_argument(value, escape_keys=escape_keys) -}}\n        {%- endfor -%}\n        {{- '}' -}}\n    {%- elif argument is sequence -%}\n        {{- '[' -}}\n        {%- for item in argument -%}\n            {{- format_argument(item, escape_keys=escape_keys) -}}\n            {%- if not loop.last %},{% endif -%}\n        {%- endfor -%}\n        {{- ']' -}}\n    {%- else -%}\n        {{- argument -}}\n    {%- endif -%}\n{%- endmacro -%}\n{%- macro strip_thinking(text) -%}\n    {%- set ns = namespace(result='') -%}\n    {%- for part in text.split('<channel|>') -%}\n        {%- if '<|channel>' in part -%}\n            {%- set ns.result = ns.result + part.split('<|channel>')[0] -%}\n        {%- else -%}\n            {%- set ns.result = ns.result + part -%}\n        {%- endif -%}\n    {%- endfor -%}\n    {{- ns.result | trim -}}\n{%- endmacro -%}\n\n{%- macro format_tool_response_block(tool_name, response) -%}\n    {{- '<|tool_response>' -}}\n    {%- if response is mapping -%}\n        {{- 'response:' + tool_name + '{' -}}\n        {%- for key, value in response | dictsort -%}\n            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}\n            {%- if not loop.last %},{% endif -%}\n        {%- endfor -%}\n        {{- '}' -}}\n    {%- else -%}\n        {{- 'response:' + tool_name + '{value:' + format_argument(response, escape_keys=False) + '}' -}}\n    {%- endif -%}\n    {{- '<tool_response|>' -}}\n{%- endmacro -%}\n\n{#- ===== SETUP ===== -#}\n{%- set ns = namespace(prev_message_type=None, prev_non_tool_role=None) -%}\n{%- set loop_messages = messages -%}\n{%- set enable_thinking = enable_thinking | default(false) -%}\n{%- set preserve_thinking = preserve_thinking | default(false) -%}\n{{- bos_token -}}\n{#- Handle System/Tool Definitions Block -#}\n{%- if enable_thinking or tools or (messages and messages[0]['role'] in ['system', 'developer']) -%}\n    {{- '<|turn>system\\n' -}}\n    {#- Inject Thinking token at the very top of the FIRST system turn -#}\n    {%- if enable_thinking -%}\n        {{- '<|think|>\\n' -}}\n        {%- set ns.prev_message_type = 'think' -%}\n    {%- endif -%}\n    {%- if messages and messages[0]['role'] in ['system', 'developer'] -%}\n        {%- if messages[0]['content'] is string -%}\n            {{- messages[0]['content'] | trim -}}\n        {%- elif messages[0]['content'] is sequence -%}\n            {%- for item in messages[0]['content'] -%}\n                {{- item['text'] | trim + ' '-}}\n            {%- endfor -%}\n        {%- endif -%}\n        {%- set loop_messages = messages[1:] -%}\n    {%- endif -%}\n    {%- if tools -%}\n        {%- for tool in tools %}\n            {{- '<|tool>' -}}\n            {{- format_function_declaration(tool) | trim -}}\n            {{- '<tool|>' -}}\n        {%- endfor %}\n        {%- set ns.prev_message_type = 'tool' -%}\n    {%- endif -%}\n    {{- '<turn|>\\n' -}}\n{%- endif %}\n\n{#- Pre-scan: find last user message index for reasoning guard -#}\n{%- set ns_turn = namespace(last_user_idx=-1) -%}\n{%- for i in range(loop_messages | length) -%}\n    {%- if loop_messages[i]['role'] == 'user' -%}\n        {%- set ns_turn.last_user_idx = i -%}\n    {%- endif -%}\n{%- endfor -%}\n\n{#- Loop through messages -#}\n{%- for message in loop_messages -%}\n    {%- if message['role'] != 'tool' -%}\n    {%- set ns.prev_message_type = None -%}\n    {%- set role = 'model' if message['role'] == 'assistant' else message['role'] -%}\n    {#- Detect continuation using tracked state \u2014 O(1) instead of O(n) backward scan -#}\n    {%- set continue_same_model_turn = (role == 'model' and ns.prev_non_tool_role == 'assistant') -%}\n    {%- if not continue_same_model_turn -%}\n        {{- '<|turn>' + role + '\\n' }}\n\n    {%- endif -%}\n\n    {#- Render reasoning/reasoning_content as thinking channel -#}\n    {%- set thinking_text = message.get('reasoning') or message.get('reasoning_content') -%}\n    {%- set thinking_gate = (loop.index0 > ns_turn.last_user_idx) or (preserve_thinking and message.get('tool_calls')) -%}\n    {%- if thinking_text and thinking_gate -%}\n        {{- '<|channel>thought\\n' + thinking_text + '\\n<channel|>' -}}\n    {%- endif -%}\n\n            {%- set captured_content -%}\n            {%- if message.get('content') is string -%}\n                {%- if role == 'model' -%}\n                    {{- strip_thinking(message['content']) -}}\n                {%- else -%}\n                    {{- message['content'] | trim -}}\n                {%- endif -%}\n            {%- elif message.get('content') is sequence -%}\n                {%- for item in message['content'] -%}\n                    {%- if item.get('type') == 'text' -%}\n                        {%- if role == 'model' -%}\n                            {{- strip_thinking(item['text']) -}}\n                        {%- else -%}\n                            {{- item['text'] | trim -}}\n                        {%- endif -%}\n                    {%- elif item.get('type') in ['image', 'image_url'] -%}\n                        {{- '<|image|>' -}}\n                    {%- elif item.get('type') in ['audio', 'input_audio'] -%}\n                        {{- '<|audio|>' -}}\n                    {%- elif item.get('type') == 'video' -%}\n                        {{- '<|video|>' -}}\n                    {%- endif -%}\n                {%- endfor -%}\n            {%- endif -%}\n            {%- endset -%}\n\n            {#- The sole intentional behavior change from HF main: preserve\n                OpenAI assistant content before its tool calls. -#}\n            {%- if role == 'model' and message.get('tool_calls') -%}\n                {{- captured_content -}}\n            {%- endif -%}\n\n            {%- if message.get('tool_calls') -%}\n                {%- for tool_call in message.get('tool_calls') -%}\n                    {%- set function = tool_call['function'] -%}\n                    {{- '<|tool_call>call:' + function['name'] + '{' -}}\n                    {%- if function['arguments'] is mapping -%}\n                        {%- set ns_args = namespace(found_first=false) -%}\n                        {%- for key, value in function['arguments'] | dictsort -%}\n                            {%- if ns_args.found_first %},{% endif -%}\n                            {%- set ns_args.found_first = true -%}\n                            {{- key -}}:{{- format_argument(value, escape_keys=False) -}}\n                        {%- endfor -%}\n                    {%- elif function['arguments'] is none -%}\n                    {%- else -%}\n                        {{- raise_exception(\n                            \"chat_template: tool_calls[].function.arguments must be a \"\n                            \"JSON object (mapping), not a string. Deserialize arguments \"\n                            \"before passing to the template.\"\n                        ) -}}\n                    {%- endif -%}\n                    {{- '}<tool_call|>' -}}\n                {%- endfor -%}\n                {%- set ns.prev_message_type = 'tool_call' -%}\n            {%- endif -%}\n\n            {%- set ns_tr_out = namespace(flag=false) -%}\n            {%- if message.get('tool_responses') -%}\n                {#- Legacy: tool_responses embedded on the assistant message (Google/Gemma native) -#}\n                {%- for tool_response in message.get('tool_responses') -%}\n                    {{- format_tool_response_block(tool_response['name'] | default('unknown', true), tool_response['response']) -}}\n                    {%- set ns_tr_out.flag = true -%}\n                    {%- set ns.prev_message_type = 'tool_response' -%}\n                {%- endfor -%}\n            {%- elif message.get('tool_calls') -%}\n                {#- OpenAI Chat Completions: forward-scan consecutive role:tool messages -#}\n                {%- set ns_tool_scan = namespace(stopped=false) -%}\n                {%- for k in range(loop.index0 + 1, loop_messages | length) -%}\n                    {%- if ns_tool_scan.stopped -%}\n                    {%- elif loop_messages[k]['role'] != 'tool' -%}\n                        {%- set ns_tool_scan.stopped = true -%}\n                    {%- else -%}\n                        {%- set follow = loop_messages[k] -%}\n                        {#- Resolve tool_call_id to function name -#}\n                        {%- set ns_tname = namespace(name=follow.get('name') or 'unknown') -%}\n                        {%- for tc in message.get('tool_calls') -%}\n                            {%- if tc.get('id') == follow.get('tool_call_id') -%}\n                                {%- set ns_tname.name = tc['function']['name'] -%}\n                            {%- endif -%}\n                        {%- endfor -%}\n                        {#- Handle content as string or content-parts array -#}\n                        {%- set tool_body = follow.get('content') -%}\n                        {%- if tool_body is string -%}\n                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}\n                        {%- elif tool_body is sequence and tool_body is not string -%}\n                            {%- set ns_txt = namespace(s='') -%}\n                            {%- for part in tool_body -%}\n                                {%- if part.get('type') == 'text' -%}\n                                    {%- set ns_txt.s = ns_txt.s + (part.get('text') | default('')) -%}\n                                {%- endif -%}\n                            {%- endfor -%}\n                            {{- format_tool_response_block(ns_tname.name, ns_txt.s) -}}\n                            {%- for part in tool_body -%}\n                                {%- if part.get('type') in ['image', 'image_url'] -%}\n                                    {{- '<|image|>' -}}\n                                {%- elif part.get('type') in ['audio', 'input_audio'] -%}\n                                    {{- '<|audio|>' -}}\n                                {%- elif part.get('type') == 'video' -%}\n                                    {{- '<|video|>' -}}\n                                {%- endif -%}\n                            {%- endfor -%}\n                        {%- else -%}\n                            {{- format_tool_response_block(ns_tname.name, tool_body) -}}\n                        {%- endif -%}\n                        {%- set ns_tr_out.flag = true -%}\n                        {%- set ns.prev_message_type = 'tool_response' -%}\n                    {%- endif -%}\n                {%- endfor -%}\n            {%- endif -%}\n\n            {%- if not (role == 'model' and message.get('tool_calls')) -%}\n                {{- captured_content -}}\n            {%- endif -%}\n            {%- set has_content = captured_content | trim | length > 0 -%}\n\n        {#- Forward-scan: find next non-tool message role for continuation detection -#}\n        {%- set next_nt = namespace(role=None, found=false) -%}\n        {%- for j in range(loop.index0 + 1, loop_messages | length) -%}\n            {%- if not next_nt.found -%}\n                {%- if loop_messages[j]['role'] != 'tool' -%}\n                    {%- set next_nt.role = loop_messages[j]['role'] -%}\n                    {%- set next_nt.found = true -%}\n                {%- endif -%}\n            {%- endif -%}\n        {%- endfor -%}\n\n        {%- set continues_into_next = (\n            role == 'model'\n            and next_nt.role == 'assistant'\n            and (not message.get('tool_calls') or ns_tr_out.flag)\n        ) -%}\n\n        {%- if ns.prev_message_type == 'tool_call' and not ns_tr_out.flag -%}\n            {{- '<|tool_response>' -}}\n        {%- elif continues_into_next -%}\n        {%- elif not (ns_tr_out.flag and not has_content and not next_nt.found) -%}\n            {{- '<turn|>\\n' -}}\n        {%- endif -%}\n\n    {#- Track previous non-tool role for next iteration (avoids O(n) backward scan) -#}\n    {%- set ns.prev_non_tool_role = message['role'] -%}\n    {%- endif -%}\n{%- endfor -%}\n\n{%- if add_generation_prompt -%}\n    {%- if ns.prev_message_type != 'tool_response' and ns.prev_message_type != 'tool_call' -%}\n        {{- '<|turn>model\\n' -}}\n    {%- elif ns.prev_message_type == 'tool_response' and enable_thinking -%}\n        {{- '<|channel>thought\\n' -}}\n    {%- endif -%}\n{%- endif -%}\n"
+}