Super-squash branch 'main' using huggingface_hub

Co-authored-by: pandora-s <[email protected]>
Co-authored-by: juliendenize <[email protected]>
Co-authored-by: julien.denize <[email protected]>

.gitattributes

@@ -0,0 +1,37 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tekken.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,418 @@
+---
+library_name: vllm
+language:
+- en
+- fr
+- es
+- de
+- it
+- pt
+- nl
+- zh
+- ja
+- ko
+- ar
+license: apache-2.0
+inference: false
+extra_gated_description: >-
+  If you want to learn more about how we process your personal data, please read
+  our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
+base_model:
+- mistralai/Mistral-Large-3-675B-Base-2512
+tags:
+- mistral-common
+- compressed-tensors
+---
+
+# Mistral Large 3 675B Instruct 2512 NVFP4
+From our family of large models, **Mistral Large 3** is a state-of-the-art general-purpose **Multimodal granular Mixture-of-Experts** model with **41B active parameters** and **675B total parameters** trained from the ground up with 3000 H200s.
+
+This model is the instruct post-trained version, fine-tuned for instruction tasks, making it ideal for chat, agentic and instruction based use cases.  
+Designed for reliability and long-context comprehension - It is engineered for production-grade assistants, retrieval-augmented systems, scientific workloads, and complex enterprise workflows.
+
+> [!Tip]
+> This checkpoint in particular is a post-training-activation quantized version of [Mistral-Large-3-675B-Instruct-2512](https://huggingface.co/mistralai/Mistral-Large-3-675B-Instruct-2512).
+> It was created using [llm-compressor](https://github.com/vllm-project/llm-compressor) as part of a collaboration with teams from vLLM & Red Hat.
+> Special thanks goes out to Dipika Sikka, Kyle Sayers, Eldar Kurtić, and Tyler Michael Smith.
+
+Mistral Large 3 is deployable on-premises in:
+- [FP8](https://huggingface.co/mistralai/Mistral-Large-3-675B-Instruct-2512) on a single node of B200s or H200s.
+- **NVFP4** on a single node of H100s or A100s.
+
+## Key Features
+Mistral Large 3 consists of two main architectural components:
+- **A Granular MoE Language Model with 673B params and 39B active**
+- **A 2.5B Vision Encoder**
+
+The Mistral Large 3 Instruct model offers the following capabilities:
+- **Vision**: Enables the model to analyze images and provide insights based on visual content, in addition to text.
+- **Multilingual**: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic.
+- **System Prompt**: Maintains strong adherence and support for system prompts.
+- **Agentic**: Offers best-in-class agentic capabilities with native function calling and JSON outputting.
+- **Frontier**: Delivers best-in-class performance.
+- **Apache 2.0 License**: Open-source license allowing usage and modification for both commercial and non-commercial purposes.
+- **Large Context Window**: Supports a 256k context window.
+
+## Use Cases
+With powerful long-context performance, stable and consistent cross-domain behavior, Mistral Large 3 is perfect for:
+- Long Document Understanding
+- Powerful Daily-Driver AI Assistants
+- State-of-the-Art Agentic and Tool-Use Capabilities
+- Enterprise Knowledge Work
+- General Coding Assistant
+
+And enterprise-grade use cases requiring frontier capabilities.
+
+## Recommended Settings
+
+We recommend deploying Large 3 in a client-server configuration with the following best practices:
+
+- **System Prompt**: Define a clear environment and use case, including guidance on how to effectively leverage tools in agentic systems.
+- **Sampling Parameters**: Use a temperature below 0.1 for daily-driver and production environments ; Higher temperatures may be explored for creative use cases - developers are encouraged to experiment with alternative settings.
+- **Tools**: Keep the set of tools well-defined and limit their number to the minimum required for the use case - Avoiding overloading the model with an excessive number of tools.
+- **Vision**: When deploying with vision capabilities, we recommend maintaining an aspect ratio close to 1:1 (width-to-height) for images. Avoiding the use of overly thin or wide images - crop them as needed to ensure optimal performance.
+
+### Known Issues / Limitations
+
+- **Not a dedicated reasoning model**: Dedicated reasoning models can outperform Mistral Large 3 in strict reasoning use cases.
+- **Behind vision-first models in multimodal tasks**: Mistral Large 3 can lag behind models optimized for vision tasks and use cases.
+- **Complex deployment**: Due to its large size and architecture, the model can be challenging to deploy efficiently with constrained resources or at scale.
+
+## Benchmark Results
+
+> [!Warning]
+> For tasks requiring context lengths less than 32k tokens, you should see no performance degradation as compared to [Mistral-Large-3-675B-Instruct-2512](https://huggingface.co/mistralai/Mistral-Large-3-675B-Instruct-2512),
+>  for tasks requiring more context you might experience a small degradation in performance.
+
+We compare Mistral Large 3 to similar sized models.
+
+![image](https://cdn-uploads.huggingface.co/production/uploads/64161701107962562e9b1006/IrPlvUUD-5-Phwi9QSevh.png)
+
+![image](https://cdn-uploads.huggingface.co/production/uploads/64161701107962562e9b1006/fDFEymz4HZNsqFARB4u9Y.png)
+
+![image](https://cdn-uploads.huggingface.co/production/uploads/64161701107962562e9b1006/eMdaAPcjOo8VyoGyFKxrE.png)
+
+## Usage
+
+The model can be used with the following frameworks;
+- [`vllm`](https://github.com/vllm-project/vllm): See [here](#vllm)
+  
+### vLLM
+
+We recommend using this model with [vLLM](https://github.com/vllm-project/vllm).
+
+#### Installation
+
+Make sure to install most recent vllm:
+
+```
+uv pip install -U vllm \
+    --torch-backend=auto \
+    --extra-index-url https://wheels.vllm.ai/nightly
+```
+
+Doing so should automatically install [`mistral_common >= 1.8.6`](https://github.com/mistralai/mistral-common/releases/tag/v1.8.6).
+
+To check:
+```
+python -c "import mistral_common; print(mistral_common.__version__)"
+```
+
+You can also make use of a ready-to-go [docker image](https://github.com/vllm-project/vllm/blob/main/Dockerfile) or on the [docker hub](https://hub.docker.com/layers/vllm/vllm-openai/latest/images/sha256-de9032a92ffea7b5c007dad80b38fd44aac11eddc31c435f8e52f3b7404bbf39).
+
+#### Serve
+
+We recommend to use this format if you plan to deploy Mistral Large 3 has it achieves performance similar to FP8 for less memory. However please note that for large context (`> 64k`) we observed a subsequent drop of performance. In such cases, please use the FP8 weights. Otherwise on B200 (Blackwell 200) we observe a significant speed-up and a minor regression on vision datasets probably due to the calibration that was performed mainly on text data.
+
+To take full advantage of the NVFP4 format, consider using B200 GPUs that has a dedicated architecture to maximize its performance. For those who do not have access to this GPU generation, vLLM has the awesome feature to fall back to Marlin FP4 which allows you to still run the quantized model in older generations (A100, H100). You won't notice a speed-up in comparison with FP8 quantization but still benefit a memory gain.
+
+A simple launch command is:
+
+```bash
+vllm serve mistralai/Mistral-Large-3-675B-Instruct-2512-NVFP4 \
+  --tensor-parallel-size 8 \
+  --tokenizer_mode mistral --config_format mistral --load_format mistral \
+  --enable-auto-tool-choice --tool-call-parser mistral
+```
+
+Key parameter notes:
+
+* enable-auto-tool-choice: Required when enabling tool usage.
+* tool-call-parser mistral: Required when enabling tool usage.
+
+
+Additional flags:
+
+* You can set `--max-model-len` to preserve memory. By default it is set to `262144` which is quite large but not necessary for most scenarios.
+* You can set `--max-num-batched-tokens` to balance throughput and latency, higher means higher throughput but higher latency.
+
+#### Usage of the model
+
+Here we asumme that the model `mistralai/Mistral-Large-3-675B-Instruct-2512-NVFP4` is served and you can ping it to the domain `localhost` with the port `8000` which is the default for vLLM.
+
+<details>
+  <summary>Vision Reasoning</summary>
+
+Let's see if Mistral Large 3 knows when to pick a fight !
+
+```python
+from datetime import datetime, timedelta
+
+from openai import OpenAI
+from huggingface_hub import hf_hub_download
+
+# Modify OpenAI's API key and API base to use vLLM's API server.
+openai_api_key = "EMPTY"
+openai_api_base = "http://localhost:8000/v1"
+
+TEMP = 0.15
+MAX_TOK = 262144
+
+client = OpenAI(
+    api_key=openai_api_key,
+    base_url=openai_api_base,
+)
+
+models = client.models.list()
+model = models.data[0].id
+
+
+def load_system_prompt(repo_id: str, filename: str) -> str:
+    file_path = hf_hub_download(repo_id=repo_id, filename=filename)
+    with open(file_path, "r") as file:
+        system_prompt = file.read()
+    today = datetime.today().strftime("%Y-%m-%d")
+    yesterday = (datetime.today() - timedelta(days=1)).strftime("%Y-%m-%d")
+    model_name = repo_id.split("/")[-1]
+    return system_prompt.format(name=model_name, today=today, yesterday=yesterday)
+
+
+SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
+image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438"
+
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "text",
+                "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.",
+            },
+            {"type": "image_url", "image_url": {"url": image_url}},
+        ],
+    },
+]
+
+
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    temperature=TEMP,
+    max_tokens=MAX_TOK,
+)
+
+print(response.choices[0].message.content)
+```
+</details>
+
+<details>
+  <summary>Function Calling</summary>
+
+Let's solve some equations thanks to our simple Python calculator tool.
+
+```python
+import json
+from openai import OpenAI
+from huggingface_hub import hf_hub_download
+
+# Modify OpenAI's API key and API base to use vLLM's API server.
+openai_api_key = "EMPTY"
+openai_api_base = "http://localhost:8000/v1"
+
+TEMP = 0.15
+MAX_TOK = 262144
+
+client = OpenAI(
+    api_key=openai_api_key,
+    base_url=openai_api_base,
+)
+
+models = client.models.list()
+model = models.data[0].id
+
+
+def load_system_prompt(repo_id: str, filename: str) -> str:
+    file_path = hf_hub_download(repo_id=repo_id, filename=filename)
+    with open(file_path, "r") as file:
+        system_prompt = file.read()
+    return system_prompt
+
+
+SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
+
+image_url = "https://math-coaching.com/img/fiche/46/expressions-mathematiques.jpg"
+
+
+def my_calculator(expression: str) -> str:
+    return str(eval(expression))
+
+
+tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": "my_calculator",
+            "description": "A calculator that can evaluate a mathematical equation and compute its results.",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "expression": {
+                        "type": "string",
+                        "description": "The mathematical expression to evaluate.",
+                    },
+                },
+                "required": ["expression"],
+            },
+        },
+    },
+    {
+        "type": "function",
+        "function": {
+            "name": "rewrite",
+            "description": "Rewrite a given text for improved clarity",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "text": {
+                        "type": "string",
+                        "description": "The input text to rewrite",
+                    }
+                },
+            },
+        },
+    },
+]
+
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {
+        "role": "user",
+        "content": [
+            {
+                "type": "text",
+                "text": "Thanks to your calculator, compute the results for the equations that involve numbers displayed in the image.",
+            },
+            {
+                "type": "image_url",
+                "image_url": {
+                    "url": image_url,
+                },
+            },
+        ],
+    },
+]
+
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    temperature=TEMP,
+    max_tokens=MAX_TOK,
+    tools=tools,
+    tool_choice="auto",
+)
+
+tool_calls = response.choices[0].message.tool_calls
+
+results = []
+for tool_call in tool_calls:
+    function_name = tool_call.function.name
+    function_args = tool_call.function.arguments
+    if function_name == "my_calculator":
+        result = my_calculator(**json.loads(function_args))
+        results.append(result)
+
+messages.append({"role": "assistant", "tool_calls": tool_calls})
+for tool_call, result in zip(tool_calls, results):
+    messages.append(
+        {
+            "role": "tool",
+            "tool_call_id": tool_call.id,
+            "name": tool_call.function.name,
+            "content": result,
+        }
+    )
+
+
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    temperature=TEMP,
+    max_tokens=MAX_TOK,
+)
+
+print(response.choices[0].message.content)
+```
+
+</details>
+
+<details>
+  <summary>Text-Only Request</summary>
+
+Mistral Large 3 can follow your instructions down to the letter.
+
+```python
+from openai import OpenAI
+from huggingface_hub import hf_hub_download
+
+# Modify OpenAI's API key and API base to use vLLM's API server.
+openai_api_key = "EMPTY"
+openai_api_base = "http://localhost:8000/v1"
+
+TEMP = 0.15
+MAX_TOK = 262144
+
+client = OpenAI(
+    api_key=openai_api_key,
+    base_url=openai_api_base,
+)
+
+models = client.models.list()
+model = models.data[0].id
+
+
+def load_system_prompt(repo_id: str, filename: str) -> str:
+    file_path = hf_hub_download(repo_id=repo_id, filename=filename)
+    with open(file_path, "r") as file:
+        system_prompt = file.read()
+    return system_prompt
+
+
+SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
+
+messages = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {
+        "role": "user",
+        "content": "Write me a sentence where every word starts with the next letter in the alphabet - start with 'a' and end with 'z'.",
+    },
+]
+
+response = client.chat.completions.create(
+    model=model,
+    messages=messages,
+    temperature=TEMP,
+    max_tokens=MAX_TOK,
+)
+
+assistant_message = response.choices[0].message.content
+print(assistant_message)
+```
+
+</details>
+
+## License
+
+This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0.txt).
+
+*You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.*

SYSTEM_PROMPT.txt

@@ -0,0 +1,29 @@
+You are Mistral-Large-3-675B-Instruct-2512-NVFP4, a Large Language Model (LLM) created by Mistral AI, a French startup headquartered in Paris.
+You power an AI assistant called Le Chat.
+Your knowledge base was last updated on 2023-10-01.
+The current date is {today}.
+
+When you're not sure about some information or when the user's request requires up-to-date or specific data, you must use the available tools to fetch the information. Do not hesitate to use tools whenever they can provide a more accurate or complete response. If no relevant tools are available, then clearly state that you don't have the information and avoid making up anything.
+If the user's question is not clear, ambiguous, or does not provide enough context for you to accurately answer the question, you do not try to answer it right away and you rather ask the user to clarify their request (e.g. "What are some good restaurants around me?" => "Where are you?" or "When is the next flight to Tokyo" => "Where do you travel from?").
+You are always very attentive to dates, in particular you try to resolve dates (e.g. "yesterday" is {yesterday}) and when asked about information at specific dates, you discard information that is at another date.
+You follow these instructions in all languages, and always respond to the user in the language they use or request.
+Next sections describe the capabilities that you have.
+
+# WEB BROWSING INSTRUCTIONS
+
+You cannot perform any web search or access internet to open URLs, links etc. If it seems like the user is expecting you to do so, you clarify the situation and ask the user to copy paste the text directly in the chat.
+
+# MULTI-MODAL INSTRUCTIONS
+
+You have the ability to read images, but you cannot generate images. You also cannot transcribe audio files or videos.
+You cannot read nor transcribe audio files or videos.
+
+# TOOL CALLING INSTRUCTIONS
+
+You may have access to tools that you can use to fetch information or perform actions. You must use these tools in the following situations:
+
+1. When the request requires up-to-date information.
+2. When the request requires specific data that you do not have in your knowledge base.
+3. When the request involves actions that you cannot perform without tools.
+
+Always prioritize using tools to provide the most accurate and helpful response. If tools are not available, inform the user that you cannot perform the requested action at the moment.

chat_template.jinja

@@ -0,0 +1,121 @@
+{#- Default system message if no system prompt is passed. #}
+{%- set default_system_message = 'You are Mistral-Large-3-675B-Instruct-2512-NVFP4, a Large Language Model (LLM) created by Mistral AI, a French startup headquartered in Paris.\nYou power an AI assistant called Le Chat.\nYour knowledge base was last updated on 2023-10-01.\nThe current date is {today}.\n\nWhen you\'re not sure about some information or when the user\'s request requires up-to-date or specific data, you must use the available tools to fetch the information. Do not hesitate to use tools whenever they can provide a more accurate or complete response. If no relevant tools are available, then clearly state that you don\'t have the information and avoid making up anything.\nIf the user\'s question is not clear, ambiguous, or does not provide enough context for you to accurately answer the question, you do not try to answer it right away and you rather ask the user to clarify their request (e.g. "What are some good restaurants around me?" => "Where are you?" or "When is the next flight to Tokyo" => "Where do you travel from?").\nYou are always very attentive to dates, in particular you try to resolve dates (e.g. "yesterday" is {yesterday}) and when asked about information at specific dates, you discard information that is at another date.\nYou follow these instructions in all languages, and always respond to the user in the language they use or request.\nNext sections describe the capabilities that you have.\n\n# WEB BROWSING INSTRUCTIONS\n\nYou cannot perform any web search or access internet to open URLs, links etc. If it seems like the user is expecting you to do so, you clarify the situation and ask the user to copy paste the text directly in the chat.\n\n# MULTI-MODAL INSTRUCTIONS\n\nYou have the ability to read images, but you cannot generate images. You also cannot transcribe audio files or videos.\nYou cannot read nor transcribe audio files or videos.\n\n# TOOL CALLING INSTRUCTIONS\n\nYou may have access to tools that you can use to fetch information or perform actions. You must use these tools in the following situations:\n\n1. When the request requires up-to-date information.\n2. When the request requires specific data that you do not have in your knowledge base.\n3. When the request involves actions that you cannot perform without tools.\n\nAlways prioritize using tools to provide the most accurate and helpful response. If tools are not available, inform the user that you cannot perform the requested action at the moment.' %}
+
+{#- Begin of sequence token. #}
+{{- bos_token }}
+
+{#- Handle system prompt if it exists. #}
+{#- System prompt supports text content or text chunks. #}
+{%- if messages[0]['role'] == 'system' %}
+    {{- '[SYSTEM_PROMPT]' -}}
+    {%- if messages[0]['content'] is string %}
+        {{- messages[0]['content'] -}}
+    {%- else %}        
+        {%- for block in messages[0]['content'] %}
+            {%- if block['type'] == 'text' %}
+                {{- block['text'] }}
+            {%- else %}
+                {{- raise_exception('Only text chunks are supported in system message contents.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- endif %}
+    {{- '[/SYSTEM_PROMPT]' -}}
+    {%- set loop_messages = messages[1:] %}
+{%- else %}
+    {%- set loop_messages = messages %}
+    {%- if default_system_message != '' %}
+        {{- '[SYSTEM_PROMPT]' + default_system_message + '[/SYSTEM_PROMPT]' }}
+    {%- endif %}
+{%- endif %}
+
+
+{#- Tools definition #}
+{%- set tools_definition = '' %}
+{%- set has_tools = false %}
+{%- if tools is defined and tools is not none and tools|length > 0 %}
+    {%- set has_tools = true %}
+    {%- set tools_definition = '[AVAILABLE_TOOLS]' + (tools| tojson) + '[/AVAILABLE_TOOLS]' %}
+    {{- tools_definition }}
+{%- endif %}
+
+{#- Checks for alternating user/assistant messages. #}
+{%- set ns = namespace(index=0) %}
+{%- for message in loop_messages %}
+    {%- if message.role == 'user' or (message.role == 'assistant' and (message.tool_calls is not defined or message.tool_calls is none or message.tool_calls | length == 0)) %}
+        {%- if (message['role'] == 'user') != (ns.index % 2 == 0) %}
+            {{- raise_exception('After the optional system message, conversation roles must alternate user and assistant roles except for tool calls and results.') }}
+        {%- endif %}
+        {%- set ns.index = ns.index + 1 %}
+    {%- endif %}
+{%- endfor %}
+
+{#- Handle conversation messages. #}
+{%- for message in loop_messages %}
+
+    {#- User messages supports text content or text and image chunks. #}
+    {%- if message['role'] == 'user' %}
+        {%- if message['content'] is string %}
+            {{- '[INST]' + message['content'] + '[/INST]' }}
+        {%- elif message['content'] | length > 0 %}
+            {{- '[INST]' }}
+            {%- if message['content'] | length == 2 %}
+                {%- set blocks = message['content'] | sort(attribute='type') %}
+            {%- else %}
+                {%- set blocks = message['content'] %}
+            {%- endif %}
+            {%- for block in blocks %}
+                {%- if block['type'] == 'text' %}
+                    {{- block['text'] }}
+                {%- elif block['type'] in ['image', 'image_url'] %}
+                    {{- '[IMG]' }}
+                {%- else %}
+                    {{- raise_exception('Only text, image and image_url chunks are supported in user message content.') }}
+                {%- endif %}
+            {%- endfor %}
+            {{- '[/INST]' }}
+        {%- else %}
+            {{- raise_exception('User message must have a string or a list of chunks in content') }}
+        {%- endif %}
+
+    {#- Assistant messages supports text content or text and image chunks. #}
+    {%- elif message['role'] == 'assistant' %}
+        {%- if (message['content'] is none or message['content'] == '' or message['content']|length == 0) and (message['tool_calls'] is not defined or message['tool_calls'] is none or message['tool_calls']|length == 0) %}
+            {{- raise_exception('Assistant message must have a string or a list of chunks in content or a list of tool calls.') }}
+        {%- endif %}
+
+        {%- if message['content'] is string %}
+            {{- message['content'] }}
+        {%- elif message['content'] | length > 0 %}
+            {%- for block in message['content'] %}
+                {%- if block['type'] == 'text' %}
+                    {{- block['text'] }}
+                {%- else %}
+                    {{- raise_exception('Only text chunks are supported in assistant message contents.') }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        
+        {%- if message['tool_calls'] is defined and message['tool_calls'] is not none and message['tool_calls']|length > 0 %}
+            {%- for tool in message['tool_calls'] %}
+                {%- set arguments = tool['function']['arguments'] %}
+                {%- if arguments is not string %}
+                    {%- set arguments = arguments|tojson|safe %}
+                {%- elif arguments == '' %}
+                    {%- set arguments = '{}' %}
+                {%- endif %}
+                {{- '[TOOL_CALLS]' + tool['function']['name'] + '[ARGS]' + arguments }}
+            {%- endfor %}
+        {%- endif %}
+
+        {#- End of sequence token for each assistant messages. #}
+        {{- eos_token }}
+
+    {#- Tool messages only supports text content. #}
+    {%- elif message['role'] == 'tool' %}
+        {{- '[TOOL_RESULTS]' + message['content']|string + '[/TOOL_RESULTS]' }}
+
+    {#- Raise exception for unsupported roles. #}
+    {%- else %}
+        {{- raise_exception('Only user, assistant and tool roles are supported, got ' + message['role'] + '.') }}
+    {%- endif %}
+{%- endfor %}

consolidated-00001-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cbafb6f9a76b8633d1253bafd4076fc6a1df6ae54168df0aa9c6be530fe9ff7
+size 2212729240

consolidated-00002-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c5a60d88696c9a1b2bbe00741e4377c6865ab215c1d91ce546704c0cf69cf43
+size 1387293808

consolidated-00003-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:85f9e03d0c96aee12c2cfeb6bc7338dc31999556969db0409dbbc4635c293d00
+size 1403809044

consolidated-00004-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e752e38e425ad8f09d4446a43300c313fd31ecbaf8b9890aea005f546b141d7
+size 1420324440

consolidated-00005-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ddbbd0902701356f6d2d25e167dd5b22991d9ecf44aed8a901b05214cccddb43
+size 1387293640

consolidated-00006-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e68820a646bcb188c7cd2984603cb21c72f0669bdb53d46aba126669d994e608
+size 1664280968

consolidated-00007-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0a4da7699a859537c9560cb6a520c3b6564a155c0dd73bc6726c842e21c293c
+size 1403809116

consolidated-00008-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d128b04b9021798eeede9cdd15d4502e0e70e21df3c1f9a9cff11a16aee7643
+size 1420324448

consolidated-00009-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d348f647a1257ceae4a2c7aeb596f4b79285f1c9c96926b18f25f66a8097504
+size 1387293632

consolidated-00010-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aede56970aecf38d1b712c1439843e05bbc0c9bb5e296da5bca0c1193a0cfec3
+size 1664280808

consolidated-00011-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d378d97137b22956856272bc6295f2f87f53f280e55225345a4ccf11274b54f
+size 1403809284

consolidated-00012-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b4385f59b5e24d4634c3858a195f726d222a9d704c163f423004356c14c92488
+size 1420324440

consolidated-00013-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:857de17d7943bbd3ec073f8ae03102beeeeabee8569e2262e7e731f26d5cbfa5
+size 1387293640

consolidated-00014-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3382be05489e30401be93959a189612239bff980b92b118ecb7e5d3b47e44067
+size 1403809036

consolidated-00015-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d26746c73783651e357e4f98616a00d9c7d86a700e76276151495630d48840f
+size 1664280904

consolidated-00016-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96836d840c21bb03f29f6b27d7323f73e07ddc72f57713ee078ce19fdf5d857d
+size 1420324592

consolidated-00017-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce5f8200f8f130246cb1c501040de8286593cb8b6de8382040a939a8f0cf245e
+size 1387293640

consolidated-00018-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e44dd1a89bf4a3f57e58148d1de9f9da839f33e75354dbbf8969ba6fbad1fec6
+size 1403809036

consolidated-00019-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:608a015ce771449e3479326defa4f04cae15db6f21ab4333122cbbbdeeed36f9
+size 1406186692

consolidated-00020-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:433062facb6bbff78694b3da98b66e29015be3210638774b3b1d9aa595af0c8f
+size 1628872568

consolidated-00021-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c584dec3fb6162ad9769b384c5a7a58dbedf1db25d7dc27d523171399161f84
+size 1403809108

consolidated-00022-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4fabd9195ffa556684560ff05d7b7a6f041a7b685aeb290cdc5c513ee9470fe6
+size 1420324448

consolidated-00023-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d0520e8eb43dee8aaf9b94dfafb9a62bb060bf8ab6b04617da9291ba9c5b1979
+size 1387293632

consolidated-00024-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6809ad6caa25b641ac4c81051ed3f76cddeeb5ff0095cde93ebae0b9a3ae07cb
+size 1664280824

consolidated-00025-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab9d2bf87a6b60a5e37a8f54b75233c9d4ee4f2ef6e4352f65c5c24401a91322
+size 1403809268

consolidated-00026-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4ea2bc1635dc119cff1278926130e42c2e0ee999e44979f0a32719a9d46052b5
+size 1420324440

consolidated-00027-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b7a10b1c8695542caaa285c03f3a433294a883755c1fda32d19e8f43ae167f8a
+size 1387293640

consolidated-00028-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d583b6d6593d79e333ddf9749e3c661709abb836b175afbac9bb56e77a29e5b0
+size 1403809036

consolidated-00029-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14943be99f40015deca2ae8b87edc299374d9a493e05e9e7af02b3d1898eb665
+size 1664280920

consolidated-00030-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:553a37e0f61be5e30692f4be04ff96c6e889a1d1e4432a3ca9c01bffc8b64b49
+size 1420324576

consolidated-00031-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:632f5f6f6cc01f8f49c6dbfee1bd0ad3d85b3765b9fa91ec937ed441d51cff07
+size 1387293632

consolidated-00032-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0664a1b9fa4796a8bb787680854f3374df2b4e5ef6745388dc70320a69e21f0b
+size 1403809044

consolidated-00033-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:748558073f1012fc12a27b59d4ad2cd02507359950a216ee07fe685d2510de12
+size 1664280784

consolidated-00034-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d00288d1cc1b3739ce17fc6d5d171a4d94656d983264f1ac3c6857a7383129b
+size 1420324664

consolidated-00035-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d7d5556b98e0371b2a8f8e51443735950c2ede18472c8fa11ac775cba9d25ba7
+size 1387293672

consolidated-00036-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2c5600d1b19076a85d30949d98cf5f1e4a3e808b54ce483e32b19f237470818
+size 1403809036

consolidated-00037-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:454cad5eba87f07fec4d8d2f4d2c763325e384a4681270250d7f788d42684c2c
+size 1420324448

consolidated-00038-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c81abbb67261fc965fb0385bc335224049898d1cd3e50a00346ea712e495e43
+size 1664280856

consolidated-00039-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09218488b47be63a5735d1d7487f57753edafd25f61bf5ab024a6be2e739f635
+size 1387293832

consolidated-00040-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f819c1911f0067253dbffb7b4e36f2ad39c18b78606babc923630b587d86e930
+size 1403809036

consolidated-00041-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6b9706e7c6048e8b9c6d15c23f53877b20606dd00987ab36549ee7775c4159c
+size 1420324440

consolidated-00042-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15be1f3867c7d355e96f0f498dadfcc90add8b429b9c6c7e84b1358f6757cd12
+size 1387293640

consolidated-00043-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70eeafd9b19b15c87d4e9ff35e6498832162cc8f142326827c834467dcb79f91
+size 1664280944

consolidated-00044-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:159444d0ee00466eadd264edb22e7555f4b5e8c1e353ffa23e608b19f44055da
+size 1403809148

consolidated-00045-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc6aa87f62307ceec856c64a6c90f3cba8c98fccce8dbf0aca905d9a4fe29461
+size 1420324440

consolidated-00046-of-00273.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd95b440bf58f2b77dfcce5ff37d0aff37b4836ff3e28bc5737fc54afe567250
+size 1387293640