Hugging Face's logo

Yi30
/
Llama-4-Scout-17B-16E-Instruct-FP8_STATIC-916-G2

Safetensors
llama4
compressed-tensors
Model card Files
xet
Community
Yi30 commited on
Commit
9c0ceca
·
verified ·
1 Parent(s): 8204094

Add files using upload-large-folder tool

Browse files
Files changed (36) hide show
  1. .gitattributes +1 -0
  2. config.ar.json +1787 -0
  3. config.json +288 -0
  4. generation_config.json +13 -0
  5. model-00001-of-00025.safetensors +3 -0
  6. model-00002-of-00025.safetensors +3 -0
  7. model-00003-of-00025.safetensors +3 -0
  8. model-00004-of-00025.safetensors +3 -0
  9. model-00005-of-00025.safetensors +3 -0
  10. model-00006-of-00025.safetensors +3 -0
  11. model-00007-of-00025.safetensors +3 -0
  12. model-00008-of-00025.safetensors +3 -0
  13. model-00009-of-00025.safetensors +3 -0
  14. model-00010-of-00025.safetensors +3 -0
  15. model-00011-of-00025.safetensors +3 -0
  16. model-00012-of-00025.safetensors +3 -0
  17. model-00013-of-00025.safetensors +3 -0
  18. model-00014-of-00025.safetensors +3 -0
  19. model-00015-of-00025.safetensors +3 -0
  20. model-00016-of-00025.safetensors +3 -0
  21. model-00017-of-00025.safetensors +3 -0
  22. model-00018-of-00025.safetensors +3 -0
  23. model-00019-of-00025.safetensors +3 -0
  24. model-00020-of-00025.safetensors +3 -0
  25. model-00021-of-00025.safetensors +3 -0
  26. model-00022-of-00025.safetensors +3 -0
  27. model-00023-of-00025.safetensors +3 -0
  28. model-00024-of-00025.safetensors +3 -0
  29. model-00025-of-00025.safetensors +3 -0
  30. model.safetensors.index.json +0 -0
  31. preprocessor_config.json +34 -0
  32. processor_config.json +6 -0
  33. quantization_config.json +1554 -0
  34. special_tokens_map.json +23 -0
  35. tokenizer.json +3 -0
  36. tokenizer_config.json +0 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
config.ar.json ADDED
@@ -0,0 +1,1787 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Llama4ForConditionalGeneration"
4
+ ],
5
+ "boi_token_index": 200080,
6
+ "dtype": "bfloat16",
7
+ "eoi_token_index": 200081,
8
+ "image_token_index": 200092,
9
+ "model_type": "llama4",
10
+ "quantization_config": {
11
+ "act_bits": 8,
12
+ "act_data_type": "fp",
13
+ "act_dynamic": false,
14
+ "act_group_size": 0,
15
+ "act_sym": true,
16
+ "autoround_version": "0.7.0",
17
+ "bits": 8,
18
+ "block_name_to_quantize": "language_model.model.layers",
19
+ "data_type": "fp",
20
+ "extra_config": {
21
+ "language_model.model.layers.0.feed_forward.router": {
22
+ "act_bits": 16,
23
+ "bits": 16
24
+ },
25
+ "language_model.model.layers.0.feed_forward.shared_expert.down_proj": {
26
+ "act_bits": 16,
27
+ "bits": 16
28
+ },
29
+ "language_model.model.layers.0.feed_forward.shared_expert.gate_proj": {
30
+ "act_bits": 16,
31
+ "bits": 16
32
+ },
33
+ "language_model.model.layers.0.feed_forward.shared_expert.up_proj": {
34
+ "act_bits": 16,
35
+ "bits": 16
36
+ },
37
+ "language_model.model.layers.0.self_attn.k_proj": {
38
+ "act_bits": 16,
39
+ "bits": 16
40
+ },
41
+ "language_model.model.layers.0.self_attn.o_proj": {
42
+ "act_bits": 16,
43
+ "bits": 16
44
+ },
45
+ "language_model.model.layers.0.self_attn.q_proj": {
46
+ "act_bits": 16,
47
+ "bits": 16
48
+ },
49
+ "language_model.model.layers.0.self_attn.v_proj": {
50
+ "act_bits": 16,
51
+ "bits": 16
52
+ },
53
+ "language_model.model.layers.1.feed_forward.router": {
54
+ "act_bits": 16,
55
+ "bits": 16
56
+ },
57
+ "language_model.model.layers.1.feed_forward.shared_expert.down_proj": {
58
+ "act_bits": 16,
59
+ "bits": 16
60
+ },
61
+ "language_model.model.layers.1.feed_forward.shared_expert.gate_proj": {
62
+ "act_bits": 16,
63
+ "bits": 16
64
+ },
65
+ "language_model.model.layers.1.feed_forward.shared_expert.up_proj": {
66
+ "act_bits": 16,
67
+ "bits": 16
68
+ },
69
+ "language_model.model.layers.1.self_attn.k_proj": {
70
+ "act_bits": 16,
71
+ "bits": 16
72
+ },
73
+ "language_model.model.layers.1.self_attn.o_proj": {
74
+ "act_bits": 16,
75
+ "bits": 16
76
+ },
77
+ "language_model.model.layers.1.self_attn.q_proj": {
78
+ "act_bits": 16,
79
+ "bits": 16
80
+ },
81
+ "language_model.model.layers.1.self_attn.v_proj": {
82
+ "act_bits": 16,
83
+ "bits": 16
84
+ },
85
+ "language_model.model.layers.10.feed_forward.router": {
86
+ "act_bits": 16,
87
+ "bits": 16
88
+ },
89
+ "language_model.model.layers.10.feed_forward.shared_expert.down_proj": {
90
+ "act_bits": 16,
91
+ "bits": 16
92
+ },
93
+ "language_model.model.layers.10.feed_forward.shared_expert.gate_proj": {
94
+ "act_bits": 16,
95
+ "bits": 16
96
+ },
97
+ "language_model.model.layers.10.feed_forward.shared_expert.up_proj": {
98
+ "act_bits": 16,
99
+ "bits": 16
100
+ },
101
+ "language_model.model.layers.10.self_attn.k_proj": {
102
+ "act_bits": 16,
103
+ "bits": 16
104
+ },
105
+ "language_model.model.layers.10.self_attn.o_proj": {
106
+ "act_bits": 16,
107
+ "bits": 16
108
+ },
109
+ "language_model.model.layers.10.self_attn.q_proj": {
110
+ "act_bits": 16,
111
+ "bits": 16
112
+ },
113
+ "language_model.model.layers.10.self_attn.v_proj": {
114
+ "act_bits": 16,
115
+ "bits": 16
116
+ },
117
+ "language_model.model.layers.11.feed_forward.router": {
118
+ "act_bits": 16,
119
+ "bits": 16
120
+ },
121
+ "language_model.model.layers.11.feed_forward.shared_expert.down_proj": {
122
+ "act_bits": 16,
123
+ "bits": 16
124
+ },
125
+ "language_model.model.layers.11.feed_forward.shared_expert.gate_proj": {
126
+ "act_bits": 16,
127
+ "bits": 16
128
+ },
129
+ "language_model.model.layers.11.feed_forward.shared_expert.up_proj": {
130
+ "act_bits": 16,
131
+ "bits": 16
132
+ },
133
+ "language_model.model.layers.11.self_attn.k_proj": {
134
+ "act_bits": 16,
135
+ "bits": 16
136
+ },
137
+ "language_model.model.layers.11.self_attn.o_proj": {
138
+ "act_bits": 16,
139
+ "bits": 16
140
+ },
141
+ "language_model.model.layers.11.self_attn.q_proj": {
142
+ "act_bits": 16,
143
+ "bits": 16
144
+ },
145
+ "language_model.model.layers.11.self_attn.v_proj": {
146
+ "act_bits": 16,
147
+ "bits": 16
148
+ },
149
+ "language_model.model.layers.12.feed_forward.router": {
150
+ "act_bits": 16,
151
+ "bits": 16
152
+ },
153
+ "language_model.model.layers.12.feed_forward.shared_expert.down_proj": {
154
+ "act_bits": 16,
155
+ "bits": 16
156
+ },
157
+ "language_model.model.layers.12.feed_forward.shared_expert.gate_proj": {
158
+ "act_bits": 16,
159
+ "bits": 16
160
+ },
161
+ "language_model.model.layers.12.feed_forward.shared_expert.up_proj": {
162
+ "act_bits": 16,
163
+ "bits": 16
164
+ },
165
+ "language_model.model.layers.12.self_attn.k_proj": {
166
+ "act_bits": 16,
167
+ "bits": 16
168
+ },
169
+ "language_model.model.layers.12.self_attn.o_proj": {
170
+ "act_bits": 16,
171
+ "bits": 16
172
+ },
173
+ "language_model.model.layers.12.self_attn.q_proj": {
174
+ "act_bits": 16,
175
+ "bits": 16
176
+ },
177
+ "language_model.model.layers.12.self_attn.v_proj": {
178
+ "act_bits": 16,
179
+ "bits": 16
180
+ },
181
+ "language_model.model.layers.13.feed_forward.router": {
182
+ "act_bits": 16,
183
+ "bits": 16
184
+ },
185
+ "language_model.model.layers.13.feed_forward.shared_expert.down_proj": {
186
+ "act_bits": 16,
187
+ "bits": 16
188
+ },
189
+ "language_model.model.layers.13.feed_forward.shared_expert.gate_proj": {
190
+ "act_bits": 16,
191
+ "bits": 16
192
+ },
193
+ "language_model.model.layers.13.feed_forward.shared_expert.up_proj": {
194
+ "act_bits": 16,
195
+ "bits": 16
196
+ },
197
+ "language_model.model.layers.13.self_attn.k_proj": {
198
+ "act_bits": 16,
199
+ "bits": 16
200
+ },
201
+ "language_model.model.layers.13.self_attn.o_proj": {
202
+ "act_bits": 16,
203
+ "bits": 16
204
+ },
205
+ "language_model.model.layers.13.self_attn.q_proj": {
206
+ "act_bits": 16,
207
+ "bits": 16
208
+ },
209
+ "language_model.model.layers.13.self_attn.v_proj": {
210
+ "act_bits": 16,
211
+ "bits": 16
212
+ },
213
+ "language_model.model.layers.14.feed_forward.router": {
214
+ "act_bits": 16,
215
+ "bits": 16
216
+ },
217
+ "language_model.model.layers.14.feed_forward.shared_expert.down_proj": {
218
+ "act_bits": 16,
219
+ "bits": 16
220
+ },
221
+ "language_model.model.layers.14.feed_forward.shared_expert.gate_proj": {
222
+ "act_bits": 16,
223
+ "bits": 16
224
+ },
225
+ "language_model.model.layers.14.feed_forward.shared_expert.up_proj": {
226
+ "act_bits": 16,
227
+ "bits": 16
228
+ },
229
+ "language_model.model.layers.14.self_attn.k_proj": {
230
+ "act_bits": 16,
231
+ "bits": 16
232
+ },
233
+ "language_model.model.layers.14.self_attn.o_proj": {
234
+ "act_bits": 16,
235
+ "bits": 16
236
+ },
237
+ "language_model.model.layers.14.self_attn.q_proj": {
238
+ "act_bits": 16,
239
+ "bits": 16
240
+ },
241
+ "language_model.model.layers.14.self_attn.v_proj": {
242
+ "act_bits": 16,
243
+ "bits": 16
244
+ },
245
+ "language_model.model.layers.15.feed_forward.router": {
246
+ "act_bits": 16,
247
+ "bits": 16
248
+ },
249
+ "language_model.model.layers.15.feed_forward.shared_expert.down_proj": {
250
+ "act_bits": 16,
251
+ "bits": 16
252
+ },
253
+ "language_model.model.layers.15.feed_forward.shared_expert.gate_proj": {
254
+ "act_bits": 16,
255
+ "bits": 16
256
+ },
257
+ "language_model.model.layers.15.feed_forward.shared_expert.up_proj": {
258
+ "act_bits": 16,
259
+ "bits": 16
260
+ },
261
+ "language_model.model.layers.15.self_attn.k_proj": {
262
+ "act_bits": 16,
263
+ "bits": 16
264
+ },
265
+ "language_model.model.layers.15.self_attn.o_proj": {
266
+ "act_bits": 16,
267
+ "bits": 16
268
+ },
269
+ "language_model.model.layers.15.self_attn.q_proj": {
270
+ "act_bits": 16,
271
+ "bits": 16
272
+ },
273
+ "language_model.model.layers.15.self_attn.v_proj": {
274
+ "act_bits": 16,
275
+ "bits": 16
276
+ },
277
+ "language_model.model.layers.16.feed_forward.router": {
278
+ "act_bits": 16,
279
+ "bits": 16
280
+ },
281
+ "language_model.model.layers.16.feed_forward.shared_expert.down_proj": {
282
+ "act_bits": 16,
283
+ "bits": 16
284
+ },
285
+ "language_model.model.layers.16.feed_forward.shared_expert.gate_proj": {
286
+ "act_bits": 16,
287
+ "bits": 16
288
+ },
289
+ "language_model.model.layers.16.feed_forward.shared_expert.up_proj": {
290
+ "act_bits": 16,
291
+ "bits": 16
292
+ },
293
+ "language_model.model.layers.16.self_attn.k_proj": {
294
+ "act_bits": 16,
295
+ "bits": 16
296
+ },
297
+ "language_model.model.layers.16.self_attn.o_proj": {
298
+ "act_bits": 16,
299
+ "bits": 16
300
+ },
301
+ "language_model.model.layers.16.self_attn.q_proj": {
302
+ "act_bits": 16,
303
+ "bits": 16
304
+ },
305
+ "language_model.model.layers.16.self_attn.v_proj": {
306
+ "act_bits": 16,
307
+ "bits": 16
308
+ },
309
+ "language_model.model.layers.17.feed_forward.router": {
310
+ "act_bits": 16,
311
+ "bits": 16
312
+ },
313
+ "language_model.model.layers.17.feed_forward.shared_expert.down_proj": {
314
+ "act_bits": 16,
315
+ "bits": 16
316
+ },
317
+ "language_model.model.layers.17.feed_forward.shared_expert.gate_proj": {
318
+ "act_bits": 16,
319
+ "bits": 16
320
+ },
321
+ "language_model.model.layers.17.feed_forward.shared_expert.up_proj": {
322
+ "act_bits": 16,
323
+ "bits": 16
324
+ },
325
+ "language_model.model.layers.17.self_attn.k_proj": {
326
+ "act_bits": 16,
327
+ "bits": 16
328
+ },
329
+ "language_model.model.layers.17.self_attn.o_proj": {
330
+ "act_bits": 16,
331
+ "bits": 16
332
+ },
333
+ "language_model.model.layers.17.self_attn.q_proj": {
334
+ "act_bits": 16,
335
+ "bits": 16
336
+ },
337
+ "language_model.model.layers.17.self_attn.v_proj": {
338
+ "act_bits": 16,
339
+ "bits": 16
340
+ },
341
+ "language_model.model.layers.18.feed_forward.router": {
342
+ "act_bits": 16,
343
+ "bits": 16
344
+ },
345
+ "language_model.model.layers.18.feed_forward.shared_expert.down_proj": {
346
+ "act_bits": 16,
347
+ "bits": 16
348
+ },
349
+ "language_model.model.layers.18.feed_forward.shared_expert.gate_proj": {
350
+ "act_bits": 16,
351
+ "bits": 16
352
+ },
353
+ "language_model.model.layers.18.feed_forward.shared_expert.up_proj": {
354
+ "act_bits": 16,
355
+ "bits": 16
356
+ },
357
+ "language_model.model.layers.18.self_attn.k_proj": {
358
+ "act_bits": 16,
359
+ "bits": 16
360
+ },
361
+ "language_model.model.layers.18.self_attn.o_proj": {
362
+ "act_bits": 16,
363
+ "bits": 16
364
+ },
365
+ "language_model.model.layers.18.self_attn.q_proj": {
366
+ "act_bits": 16,
367
+ "bits": 16
368
+ },
369
+ "language_model.model.layers.18.self_attn.v_proj": {
370
+ "act_bits": 16,
371
+ "bits": 16
372
+ },
373
+ "language_model.model.layers.19.feed_forward.router": {
374
+ "act_bits": 16,
375
+ "bits": 16
376
+ },
377
+ "language_model.model.layers.19.feed_forward.shared_expert.down_proj": {
378
+ "act_bits": 16,
379
+ "bits": 16
380
+ },
381
+ "language_model.model.layers.19.feed_forward.shared_expert.gate_proj": {
382
+ "act_bits": 16,
383
+ "bits": 16
384
+ },
385
+ "language_model.model.layers.19.feed_forward.shared_expert.up_proj": {
386
+ "act_bits": 16,
387
+ "bits": 16
388
+ },
389
+ "language_model.model.layers.19.self_attn.k_proj": {
390
+ "act_bits": 16,
391
+ "bits": 16
392
+ },
393
+ "language_model.model.layers.19.self_attn.o_proj": {
394
+ "act_bits": 16,
395
+ "bits": 16
396
+ },
397
+ "language_model.model.layers.19.self_attn.q_proj": {
398
+ "act_bits": 16,
399
+ "bits": 16
400
+ },
401
+ "language_model.model.layers.19.self_attn.v_proj": {
402
+ "act_bits": 16,
403
+ "bits": 16
404
+ },
405
+ "language_model.model.layers.2.feed_forward.router": {
406
+ "act_bits": 16,
407
+ "bits": 16
408
+ },
409
+ "language_model.model.layers.2.feed_forward.shared_expert.down_proj": {
410
+ "act_bits": 16,
411
+ "bits": 16
412
+ },
413
+ "language_model.model.layers.2.feed_forward.shared_expert.gate_proj": {
414
+ "act_bits": 16,
415
+ "bits": 16
416
+ },
417
+ "language_model.model.layers.2.feed_forward.shared_expert.up_proj": {
418
+ "act_bits": 16,
419
+ "bits": 16
420
+ },
421
+ "language_model.model.layers.2.self_attn.k_proj": {
422
+ "act_bits": 16,
423
+ "bits": 16
424
+ },
425
+ "language_model.model.layers.2.self_attn.o_proj": {
426
+ "act_bits": 16,
427
+ "bits": 16
428
+ },
429
+ "language_model.model.layers.2.self_attn.q_proj": {
430
+ "act_bits": 16,
431
+ "bits": 16
432
+ },
433
+ "language_model.model.layers.2.self_attn.v_proj": {
434
+ "act_bits": 16,
435
+ "bits": 16
436
+ },
437
+ "language_model.model.layers.20.feed_forward.router": {
438
+ "act_bits": 16,
439
+ "bits": 16
440
+ },
441
+ "language_model.model.layers.20.feed_forward.shared_expert.down_proj": {
442
+ "act_bits": 16,
443
+ "bits": 16
444
+ },
445
+ "language_model.model.layers.20.feed_forward.shared_expert.gate_proj": {
446
+ "act_bits": 16,
447
+ "bits": 16
448
+ },
449
+ "language_model.model.layers.20.feed_forward.shared_expert.up_proj": {
450
+ "act_bits": 16,
451
+ "bits": 16
452
+ },
453
+ "language_model.model.layers.20.self_attn.k_proj": {
454
+ "act_bits": 16,
455
+ "bits": 16
456
+ },
457
+ "language_model.model.layers.20.self_attn.o_proj": {
458
+ "act_bits": 16,
459
+ "bits": 16
460
+ },
461
+ "language_model.model.layers.20.self_attn.q_proj": {
462
+ "act_bits": 16,
463
+ "bits": 16
464
+ },
465
+ "language_model.model.layers.20.self_attn.v_proj": {
466
+ "act_bits": 16,
467
+ "bits": 16
468
+ },
469
+ "language_model.model.layers.21.feed_forward.router": {
470
+ "act_bits": 16,
471
+ "bits": 16
472
+ },
473
+ "language_model.model.layers.21.feed_forward.shared_expert.down_proj": {
474
+ "act_bits": 16,
475
+ "bits": 16
476
+ },
477
+ "language_model.model.layers.21.feed_forward.shared_expert.gate_proj": {
478
+ "act_bits": 16,
479
+ "bits": 16
480
+ },
481
+ "language_model.model.layers.21.feed_forward.shared_expert.up_proj": {
482
+ "act_bits": 16,
483
+ "bits": 16
484
+ },
485
+ "language_model.model.layers.21.self_attn.k_proj": {
486
+ "act_bits": 16,
487
+ "bits": 16
488
+ },
489
+ "language_model.model.layers.21.self_attn.o_proj": {
490
+ "act_bits": 16,
491
+ "bits": 16
492
+ },
493
+ "language_model.model.layers.21.self_attn.q_proj": {
494
+ "act_bits": 16,
495
+ "bits": 16
496
+ },
497
+ "language_model.model.layers.21.self_attn.v_proj": {
498
+ "act_bits": 16,
499
+ "bits": 16
500
+ },
501
+ "language_model.model.layers.22.feed_forward.router": {
502
+ "act_bits": 16,
503
+ "bits": 16
504
+ },
505
+ "language_model.model.layers.22.feed_forward.shared_expert.down_proj": {
506
+ "act_bits": 16,
507
+ "bits": 16
508
+ },
509
+ "language_model.model.layers.22.feed_forward.shared_expert.gate_proj": {
510
+ "act_bits": 16,
511
+ "bits": 16
512
+ },
513
+ "language_model.model.layers.22.feed_forward.shared_expert.up_proj": {
514
+ "act_bits": 16,
515
+ "bits": 16
516
+ },
517
+ "language_model.model.layers.22.self_attn.k_proj": {
518
+ "act_bits": 16,
519
+ "bits": 16
520
+ },
521
+ "language_model.model.layers.22.self_attn.o_proj": {
522
+ "act_bits": 16,
523
+ "bits": 16
524
+ },
525
+ "language_model.model.layers.22.self_attn.q_proj": {
526
+ "act_bits": 16,
527
+ "bits": 16
528
+ },
529
+ "language_model.model.layers.22.self_attn.v_proj": {
530
+ "act_bits": 16,
531
+ "bits": 16
532
+ },
533
+ "language_model.model.layers.23.feed_forward.router": {
534
+ "act_bits": 16,
535
+ "bits": 16
536
+ },
537
+ "language_model.model.layers.23.feed_forward.shared_expert.down_proj": {
538
+ "act_bits": 16,
539
+ "bits": 16
540
+ },
541
+ "language_model.model.layers.23.feed_forward.shared_expert.gate_proj": {
542
+ "act_bits": 16,
543
+ "bits": 16
544
+ },
545
+ "language_model.model.layers.23.feed_forward.shared_expert.up_proj": {
546
+ "act_bits": 16,
547
+ "bits": 16
548
+ },
549
+ "language_model.model.layers.23.self_attn.k_proj": {
550
+ "act_bits": 16,
551
+ "bits": 16
552
+ },
553
+ "language_model.model.layers.23.self_attn.o_proj": {
554
+ "act_bits": 16,
555
+ "bits": 16
556
+ },
557
+ "language_model.model.layers.23.self_attn.q_proj": {
558
+ "act_bits": 16,
559
+ "bits": 16
560
+ },
561
+ "language_model.model.layers.23.self_attn.v_proj": {
562
+ "act_bits": 16,
563
+ "bits": 16
564
+ },
565
+ "language_model.model.layers.24.feed_forward.router": {
566
+ "act_bits": 16,
567
+ "bits": 16
568
+ },
569
+ "language_model.model.layers.24.feed_forward.shared_expert.down_proj": {
570
+ "act_bits": 16,
571
+ "bits": 16
572
+ },
573
+ "language_model.model.layers.24.feed_forward.shared_expert.gate_proj": {
574
+ "act_bits": 16,
575
+ "bits": 16
576
+ },
577
+ "language_model.model.layers.24.feed_forward.shared_expert.up_proj": {
578
+ "act_bits": 16,
579
+ "bits": 16
580
+ },
581
+ "language_model.model.layers.24.self_attn.k_proj": {
582
+ "act_bits": 16,
583
+ "bits": 16
584
+ },
585
+ "language_model.model.layers.24.self_attn.o_proj": {
586
+ "act_bits": 16,
587
+ "bits": 16
588
+ },
589
+ "language_model.model.layers.24.self_attn.q_proj": {
590
+ "act_bits": 16,
591
+ "bits": 16
592
+ },
593
+ "language_model.model.layers.24.self_attn.v_proj": {
594
+ "act_bits": 16,
595
+ "bits": 16
596
+ },
597
+ "language_model.model.layers.25.feed_forward.router": {
598
+ "act_bits": 16,
599
+ "bits": 16
600
+ },
601
+ "language_model.model.layers.25.feed_forward.shared_expert.down_proj": {
602
+ "act_bits": 16,
603
+ "bits": 16
604
+ },
605
+ "language_model.model.layers.25.feed_forward.shared_expert.gate_proj": {
606
+ "act_bits": 16,
607
+ "bits": 16
608
+ },
609
+ "language_model.model.layers.25.feed_forward.shared_expert.up_proj": {
610
+ "act_bits": 16,
611
+ "bits": 16
612
+ },
613
+ "language_model.model.layers.25.self_attn.k_proj": {
614
+ "act_bits": 16,
615
+ "bits": 16
616
+ },
617
+ "language_model.model.layers.25.self_attn.o_proj": {
618
+ "act_bits": 16,
619
+ "bits": 16
620
+ },
621
+ "language_model.model.layers.25.self_attn.q_proj": {
622
+ "act_bits": 16,
623
+ "bits": 16
624
+ },
625
+ "language_model.model.layers.25.self_attn.v_proj": {
626
+ "act_bits": 16,
627
+ "bits": 16
628
+ },
629
+ "language_model.model.layers.26.feed_forward.router": {
630
+ "act_bits": 16,
631
+ "bits": 16
632
+ },
633
+ "language_model.model.layers.26.feed_forward.shared_expert.down_proj": {
634
+ "act_bits": 16,
635
+ "bits": 16
636
+ },
637
+ "language_model.model.layers.26.feed_forward.shared_expert.gate_proj": {
638
+ "act_bits": 16,
639
+ "bits": 16
640
+ },
641
+ "language_model.model.layers.26.feed_forward.shared_expert.up_proj": {
642
+ "act_bits": 16,
643
+ "bits": 16
644
+ },
645
+ "language_model.model.layers.26.self_attn.k_proj": {
646
+ "act_bits": 16,
647
+ "bits": 16
648
+ },
649
+ "language_model.model.layers.26.self_attn.o_proj": {
650
+ "act_bits": 16,
651
+ "bits": 16
652
+ },
653
+ "language_model.model.layers.26.self_attn.q_proj": {
654
+ "act_bits": 16,
655
+ "bits": 16
656
+ },
657
+ "language_model.model.layers.26.self_attn.v_proj": {
658
+ "act_bits": 16,
659
+ "bits": 16
660
+ },
661
+ "language_model.model.layers.27.feed_forward.router": {
662
+ "act_bits": 16,
663
+ "bits": 16
664
+ },
665
+ "language_model.model.layers.27.feed_forward.shared_expert.down_proj": {
666
+ "act_bits": 16,
667
+ "bits": 16
668
+ },
669
+ "language_model.model.layers.27.feed_forward.shared_expert.gate_proj": {
670
+ "act_bits": 16,
671
+ "bits": 16
672
+ },
673
+ "language_model.model.layers.27.feed_forward.shared_expert.up_proj": {
674
+ "act_bits": 16,
675
+ "bits": 16
676
+ },
677
+ "language_model.model.layers.27.self_attn.k_proj": {
678
+ "act_bits": 16,
679
+ "bits": 16
680
+ },
681
+ "language_model.model.layers.27.self_attn.o_proj": {
682
+ "act_bits": 16,
683
+ "bits": 16
684
+ },
685
+ "language_model.model.layers.27.self_attn.q_proj": {
686
+ "act_bits": 16,
687
+ "bits": 16
688
+ },
689
+ "language_model.model.layers.27.self_attn.v_proj": {
690
+ "act_bits": 16,
691
+ "bits": 16
692
+ },
693
+ "language_model.model.layers.28.feed_forward.router": {
694
+ "act_bits": 16,
695
+ "bits": 16
696
+ },
697
+ "language_model.model.layers.28.feed_forward.shared_expert.down_proj": {
698
+ "act_bits": 16,
699
+ "bits": 16
700
+ },
701
+ "language_model.model.layers.28.feed_forward.shared_expert.gate_proj": {
702
+ "act_bits": 16,
703
+ "bits": 16
704
+ },
705
+ "language_model.model.layers.28.feed_forward.shared_expert.up_proj": {
706
+ "act_bits": 16,
707
+ "bits": 16
708
+ },
709
+ "language_model.model.layers.28.self_attn.k_proj": {
710
+ "act_bits": 16,
711
+ "bits": 16
712
+ },
713
+ "language_model.model.layers.28.self_attn.o_proj": {
714
+ "act_bits": 16,
715
+ "bits": 16
716
+ },
717
+ "language_model.model.layers.28.self_attn.q_proj": {
718
+ "act_bits": 16,
719
+ "bits": 16
720
+ },
721
+ "language_model.model.layers.28.self_attn.v_proj": {
722
+ "act_bits": 16,
723
+ "bits": 16
724
+ },
725
+ "language_model.model.layers.29.feed_forward.router": {
726
+ "act_bits": 16,
727
+ "bits": 16
728
+ },
729
+ "language_model.model.layers.29.feed_forward.shared_expert.down_proj": {
730
+ "act_bits": 16,
731
+ "bits": 16
732
+ },
733
+ "language_model.model.layers.29.feed_forward.shared_expert.gate_proj": {
734
+ "act_bits": 16,
735
+ "bits": 16
736
+ },
737
+ "language_model.model.layers.29.feed_forward.shared_expert.up_proj": {
738
+ "act_bits": 16,
739
+ "bits": 16
740
+ },
741
+ "language_model.model.layers.29.self_attn.k_proj": {
742
+ "act_bits": 16,
743
+ "bits": 16
744
+ },
745
+ "language_model.model.layers.29.self_attn.o_proj": {
746
+ "act_bits": 16,
747
+ "bits": 16
748
+ },
749
+ "language_model.model.layers.29.self_attn.q_proj": {
750
+ "act_bits": 16,
751
+ "bits": 16
752
+ },
753
+ "language_model.model.layers.29.self_attn.v_proj": {
754
+ "act_bits": 16,
755
+ "bits": 16
756
+ },
757
+ "language_model.model.layers.3.feed_forward.router": {
758
+ "act_bits": 16,
759
+ "bits": 16
760
+ },
761
+ "language_model.model.layers.3.feed_forward.shared_expert.down_proj": {
762
+ "act_bits": 16,
763
+ "bits": 16
764
+ },
765
+ "language_model.model.layers.3.feed_forward.shared_expert.gate_proj": {
766
+ "act_bits": 16,
767
+ "bits": 16
768
+ },
769
+ "language_model.model.layers.3.feed_forward.shared_expert.up_proj": {
770
+ "act_bits": 16,
771
+ "bits": 16
772
+ },
773
+ "language_model.model.layers.3.self_attn.k_proj": {
774
+ "act_bits": 16,
775
+ "bits": 16
776
+ },
777
+ "language_model.model.layers.3.self_attn.o_proj": {
778
+ "act_bits": 16,
779
+ "bits": 16
780
+ },
781
+ "language_model.model.layers.3.self_attn.q_proj": {
782
+ "act_bits": 16,
783
+ "bits": 16
784
+ },
785
+ "language_model.model.layers.3.self_attn.v_proj": {
786
+ "act_bits": 16,
787
+ "bits": 16
788
+ },
789
+ "language_model.model.layers.30.feed_forward.router": {
790
+ "act_bits": 16,
791
+ "bits": 16
792
+ },
793
+ "language_model.model.layers.30.feed_forward.shared_expert.down_proj": {
794
+ "act_bits": 16,
795
+ "bits": 16
796
+ },
797
+ "language_model.model.layers.30.feed_forward.shared_expert.gate_proj": {
798
+ "act_bits": 16,
799
+ "bits": 16
800
+ },
801
+ "language_model.model.layers.30.feed_forward.shared_expert.up_proj": {
802
+ "act_bits": 16,
803
+ "bits": 16
804
+ },
805
+ "language_model.model.layers.30.self_attn.k_proj": {
806
+ "act_bits": 16,
807
+ "bits": 16
808
+ },
809
+ "language_model.model.layers.30.self_attn.o_proj": {
810
+ "act_bits": 16,
811
+ "bits": 16
812
+ },
813
+ "language_model.model.layers.30.self_attn.q_proj": {
814
+ "act_bits": 16,
815
+ "bits": 16
816
+ },
817
+ "language_model.model.layers.30.self_attn.v_proj": {
818
+ "act_bits": 16,
819
+ "bits": 16
820
+ },
821
+ "language_model.model.layers.31.feed_forward.router": {
822
+ "act_bits": 16,
823
+ "bits": 16
824
+ },
825
+ "language_model.model.layers.31.feed_forward.shared_expert.down_proj": {
826
+ "act_bits": 16,
827
+ "bits": 16
828
+ },
829
+ "language_model.model.layers.31.feed_forward.shared_expert.gate_proj": {
830
+ "act_bits": 16,
831
+ "bits": 16
832
+ },
833
+ "language_model.model.layers.31.feed_forward.shared_expert.up_proj": {
834
+ "act_bits": 16,
835
+ "bits": 16
836
+ },
837
+ "language_model.model.layers.31.self_attn.k_proj": {
838
+ "act_bits": 16,
839
+ "bits": 16
840
+ },
841
+ "language_model.model.layers.31.self_attn.o_proj": {
842
+ "act_bits": 16,
843
+ "bits": 16
844
+ },
845
+ "language_model.model.layers.31.self_attn.q_proj": {
846
+ "act_bits": 16,
847
+ "bits": 16
848
+ },
849
+ "language_model.model.layers.31.self_attn.v_proj": {
850
+ "act_bits": 16,
851
+ "bits": 16
852
+ },
853
+ "language_model.model.layers.32.feed_forward.router": {
854
+ "act_bits": 16,
855
+ "bits": 16
856
+ },
857
+ "language_model.model.layers.32.feed_forward.shared_expert.down_proj": {
858
+ "act_bits": 16,
859
+ "bits": 16
860
+ },
861
+ "language_model.model.layers.32.feed_forward.shared_expert.gate_proj": {
862
+ "act_bits": 16,
863
+ "bits": 16
864
+ },
865
+ "language_model.model.layers.32.feed_forward.shared_expert.up_proj": {
866
+ "act_bits": 16,
867
+ "bits": 16
868
+ },
869
+ "language_model.model.layers.32.self_attn.k_proj": {
870
+ "act_bits": 16,
871
+ "bits": 16
872
+ },
873
+ "language_model.model.layers.32.self_attn.o_proj": {
874
+ "act_bits": 16,
875
+ "bits": 16
876
+ },
877
+ "language_model.model.layers.32.self_attn.q_proj": {
878
+ "act_bits": 16,
879
+ "bits": 16
880
+ },
881
+ "language_model.model.layers.32.self_attn.v_proj": {
882
+ "act_bits": 16,
883
+ "bits": 16
884
+ },
885
+ "language_model.model.layers.33.feed_forward.router": {
886
+ "act_bits": 16,
887
+ "bits": 16
888
+ },
889
+ "language_model.model.layers.33.feed_forward.shared_expert.down_proj": {
890
+ "act_bits": 16,
891
+ "bits": 16
892
+ },
893
+ "language_model.model.layers.33.feed_forward.shared_expert.gate_proj": {
894
+ "act_bits": 16,
895
+ "bits": 16
896
+ },
897
+ "language_model.model.layers.33.feed_forward.shared_expert.up_proj": {
898
+ "act_bits": 16,
899
+ "bits": 16
900
+ },
901
+ "language_model.model.layers.33.self_attn.k_proj": {
902
+ "act_bits": 16,
903
+ "bits": 16
904
+ },
905
+ "language_model.model.layers.33.self_attn.o_proj": {
906
+ "act_bits": 16,
907
+ "bits": 16
908
+ },
909
+ "language_model.model.layers.33.self_attn.q_proj": {
910
+ "act_bits": 16,
911
+ "bits": 16
912
+ },
913
+ "language_model.model.layers.33.self_attn.v_proj": {
914
+ "act_bits": 16,
915
+ "bits": 16
916
+ },
917
+ "language_model.model.layers.34.feed_forward.router": {
918
+ "act_bits": 16,
919
+ "bits": 16
920
+ },
921
+ "language_model.model.layers.34.feed_forward.shared_expert.down_proj": {
922
+ "act_bits": 16,
923
+ "bits": 16
924
+ },
925
+ "language_model.model.layers.34.feed_forward.shared_expert.gate_proj": {
926
+ "act_bits": 16,
927
+ "bits": 16
928
+ },
929
+ "language_model.model.layers.34.feed_forward.shared_expert.up_proj": {
930
+ "act_bits": 16,
931
+ "bits": 16
932
+ },
933
+ "language_model.model.layers.34.self_attn.k_proj": {
934
+ "act_bits": 16,
935
+ "bits": 16
936
+ },
937
+ "language_model.model.layers.34.self_attn.o_proj": {
938
+ "act_bits": 16,
939
+ "bits": 16
940
+ },
941
+ "language_model.model.layers.34.self_attn.q_proj": {
942
+ "act_bits": 16,
943
+ "bits": 16
944
+ },
945
+ "language_model.model.layers.34.self_attn.v_proj": {
946
+ "act_bits": 16,
947
+ "bits": 16
948
+ },
949
+ "language_model.model.layers.35.feed_forward.router": {
950
+ "act_bits": 16,
951
+ "bits": 16
952
+ },
953
+ "language_model.model.layers.35.feed_forward.shared_expert.down_proj": {
954
+ "act_bits": 16,
955
+ "bits": 16
956
+ },
957
+ "language_model.model.layers.35.feed_forward.shared_expert.gate_proj": {
958
+ "act_bits": 16,
959
+ "bits": 16
960
+ },
961
+ "language_model.model.layers.35.feed_forward.shared_expert.up_proj": {
962
+ "act_bits": 16,
963
+ "bits": 16
964
+ },
965
+ "language_model.model.layers.35.self_attn.k_proj": {
966
+ "act_bits": 16,
967
+ "bits": 16
968
+ },
969
+ "language_model.model.layers.35.self_attn.o_proj": {
970
+ "act_bits": 16,
971
+ "bits": 16
972
+ },
973
+ "language_model.model.layers.35.self_attn.q_proj": {
974
+ "act_bits": 16,
975
+ "bits": 16
976
+ },
977
+ "language_model.model.layers.35.self_attn.v_proj": {
978
+ "act_bits": 16,
979
+ "bits": 16
980
+ },
981
+ "language_model.model.layers.36.feed_forward.router": {
982
+ "act_bits": 16,
983
+ "bits": 16
984
+ },
985
+ "language_model.model.layers.36.feed_forward.shared_expert.down_proj": {
986
+ "act_bits": 16,
987
+ "bits": 16
988
+ },
989
+ "language_model.model.layers.36.feed_forward.shared_expert.gate_proj": {
990
+ "act_bits": 16,
991
+ "bits": 16
992
+ },
993
+ "language_model.model.layers.36.feed_forward.shared_expert.up_proj": {
994
+ "act_bits": 16,
995
+ "bits": 16
996
+ },
997
+ "language_model.model.layers.36.self_attn.k_proj": {
998
+ "act_bits": 16,
999
+ "bits": 16
1000
+ },
1001
+ "language_model.model.layers.36.self_attn.o_proj": {
1002
+ "act_bits": 16,
1003
+ "bits": 16
1004
+ },
1005
+ "language_model.model.layers.36.self_attn.q_proj": {
1006
+ "act_bits": 16,
1007
+ "bits": 16
1008
+ },
1009
+ "language_model.model.layers.36.self_attn.v_proj": {
1010
+ "act_bits": 16,
1011
+ "bits": 16
1012
+ },
1013
+ "language_model.model.layers.37.feed_forward.router": {
1014
+ "act_bits": 16,
1015
+ "bits": 16
1016
+ },
1017
+ "language_model.model.layers.37.feed_forward.shared_expert.down_proj": {
1018
+ "act_bits": 16,
1019
+ "bits": 16
1020
+ },
1021
+ "language_model.model.layers.37.feed_forward.shared_expert.gate_proj": {
1022
+ "act_bits": 16,
1023
+ "bits": 16
1024
+ },
1025
+ "language_model.model.layers.37.feed_forward.shared_expert.up_proj": {
1026
+ "act_bits": 16,
1027
+ "bits": 16
1028
+ },
1029
+ "language_model.model.layers.37.self_attn.k_proj": {
1030
+ "act_bits": 16,
1031
+ "bits": 16
1032
+ },
1033
+ "language_model.model.layers.37.self_attn.o_proj": {
1034
+ "act_bits": 16,
1035
+ "bits": 16
1036
+ },
1037
+ "language_model.model.layers.37.self_attn.q_proj": {
1038
+ "act_bits": 16,
1039
+ "bits": 16
1040
+ },
1041
+ "language_model.model.layers.37.self_attn.v_proj": {
1042
+ "act_bits": 16,
1043
+ "bits": 16
1044
+ },
1045
+ "language_model.model.layers.38.feed_forward.router": {
1046
+ "act_bits": 16,
1047
+ "bits": 16
1048
+ },
1049
+ "language_model.model.layers.38.feed_forward.shared_expert.down_proj": {
1050
+ "act_bits": 16,
1051
+ "bits": 16
1052
+ },
1053
+ "language_model.model.layers.38.feed_forward.shared_expert.gate_proj": {
1054
+ "act_bits": 16,
1055
+ "bits": 16
1056
+ },
1057
+ "language_model.model.layers.38.feed_forward.shared_expert.up_proj": {
1058
+ "act_bits": 16,
1059
+ "bits": 16
1060
+ },
1061
+ "language_model.model.layers.38.self_attn.k_proj": {
1062
+ "act_bits": 16,
1063
+ "bits": 16
1064
+ },
1065
+ "language_model.model.layers.38.self_attn.o_proj": {
1066
+ "act_bits": 16,
1067
+ "bits": 16
1068
+ },
1069
+ "language_model.model.layers.38.self_attn.q_proj": {
1070
+ "act_bits": 16,
1071
+ "bits": 16
1072
+ },
1073
+ "language_model.model.layers.38.self_attn.v_proj": {
1074
+ "act_bits": 16,
1075
+ "bits": 16
1076
+ },
1077
+ "language_model.model.layers.39.feed_forward.router": {
1078
+ "act_bits": 16,
1079
+ "bits": 16
1080
+ },
1081
+ "language_model.model.layers.39.feed_forward.shared_expert.down_proj": {
1082
+ "act_bits": 16,
1083
+ "bits": 16
1084
+ },
1085
+ "language_model.model.layers.39.feed_forward.shared_expert.gate_proj": {
1086
+ "act_bits": 16,
1087
+ "bits": 16
1088
+ },
1089
+ "language_model.model.layers.39.feed_forward.shared_expert.up_proj": {
1090
+ "act_bits": 16,
1091
+ "bits": 16
1092
+ },
1093
+ "language_model.model.layers.39.self_attn.k_proj": {
1094
+ "act_bits": 16,
1095
+ "bits": 16
1096
+ },
1097
+ "language_model.model.layers.39.self_attn.o_proj": {
1098
+ "act_bits": 16,
1099
+ "bits": 16
1100
+ },
1101
+ "language_model.model.layers.39.self_attn.q_proj": {
1102
+ "act_bits": 16,
1103
+ "bits": 16
1104
+ },
1105
+ "language_model.model.layers.39.self_attn.v_proj": {
1106
+ "act_bits": 16,
1107
+ "bits": 16
1108
+ },
1109
+ "language_model.model.layers.4.feed_forward.router": {
1110
+ "act_bits": 16,
1111
+ "bits": 16
1112
+ },
1113
+ "language_model.model.layers.4.feed_forward.shared_expert.down_proj": {
1114
+ "act_bits": 16,
1115
+ "bits": 16
1116
+ },
1117
+ "language_model.model.layers.4.feed_forward.shared_expert.gate_proj": {
1118
+ "act_bits": 16,
1119
+ "bits": 16
1120
+ },
1121
+ "language_model.model.layers.4.feed_forward.shared_expert.up_proj": {
1122
+ "act_bits": 16,
1123
+ "bits": 16
1124
+ },
1125
+ "language_model.model.layers.4.self_attn.k_proj": {
1126
+ "act_bits": 16,
1127
+ "bits": 16
1128
+ },
1129
+ "language_model.model.layers.4.self_attn.o_proj": {
1130
+ "act_bits": 16,
1131
+ "bits": 16
1132
+ },
1133
+ "language_model.model.layers.4.self_attn.q_proj": {
1134
+ "act_bits": 16,
1135
+ "bits": 16
1136
+ },
1137
+ "language_model.model.layers.4.self_attn.v_proj": {
1138
+ "act_bits": 16,
1139
+ "bits": 16
1140
+ },
1141
+ "language_model.model.layers.40.feed_forward.router": {
1142
+ "act_bits": 16,
1143
+ "bits": 16
1144
+ },
1145
+ "language_model.model.layers.40.feed_forward.shared_expert.down_proj": {
1146
+ "act_bits": 16,
1147
+ "bits": 16
1148
+ },
1149
+ "language_model.model.layers.40.feed_forward.shared_expert.gate_proj": {
1150
+ "act_bits": 16,
1151
+ "bits": 16
1152
+ },
1153
+ "language_model.model.layers.40.feed_forward.shared_expert.up_proj": {
1154
+ "act_bits": 16,
1155
+ "bits": 16
1156
+ },
1157
+ "language_model.model.layers.40.self_attn.k_proj": {
1158
+ "act_bits": 16,
1159
+ "bits": 16
1160
+ },
1161
+ "language_model.model.layers.40.self_attn.o_proj": {
1162
+ "act_bits": 16,
1163
+ "bits": 16
1164
+ },
1165
+ "language_model.model.layers.40.self_attn.q_proj": {
1166
+ "act_bits": 16,
1167
+ "bits": 16
1168
+ },
1169
+ "language_model.model.layers.40.self_attn.v_proj": {
1170
+ "act_bits": 16,
1171
+ "bits": 16
1172
+ },
1173
+ "language_model.model.layers.41.feed_forward.router": {
1174
+ "act_bits": 16,
1175
+ "bits": 16
1176
+ },
1177
+ "language_model.model.layers.41.feed_forward.shared_expert.down_proj": {
1178
+ "act_bits": 16,
1179
+ "bits": 16
1180
+ },
1181
+ "language_model.model.layers.41.feed_forward.shared_expert.gate_proj": {
1182
+ "act_bits": 16,
1183
+ "bits": 16
1184
+ },
1185
+ "language_model.model.layers.41.feed_forward.shared_expert.up_proj": {
1186
+ "act_bits": 16,
1187
+ "bits": 16
1188
+ },
1189
+ "language_model.model.layers.41.self_attn.k_proj": {
1190
+ "act_bits": 16,
1191
+ "bits": 16
1192
+ },
1193
+ "language_model.model.layers.41.self_attn.o_proj": {
1194
+ "act_bits": 16,
1195
+ "bits": 16
1196
+ },
1197
+ "language_model.model.layers.41.self_attn.q_proj": {
1198
+ "act_bits": 16,
1199
+ "bits": 16
1200
+ },
1201
+ "language_model.model.layers.41.self_attn.v_proj": {
1202
+ "act_bits": 16,
1203
+ "bits": 16
1204
+ },
1205
+ "language_model.model.layers.42.feed_forward.router": {
1206
+ "act_bits": 16,
1207
+ "bits": 16
1208
+ },
1209
+ "language_model.model.layers.42.feed_forward.shared_expert.down_proj": {
1210
+ "act_bits": 16,
1211
+ "bits": 16
1212
+ },
1213
+ "language_model.model.layers.42.feed_forward.shared_expert.gate_proj": {
1214
+ "act_bits": 16,
1215
+ "bits": 16
1216
+ },
1217
+ "language_model.model.layers.42.feed_forward.shared_expert.up_proj": {
1218
+ "act_bits": 16,
1219
+ "bits": 16
1220
+ },
1221
+ "language_model.model.layers.42.self_attn.k_proj": {
1222
+ "act_bits": 16,
1223
+ "bits": 16
1224
+ },
1225
+ "language_model.model.layers.42.self_attn.o_proj": {
1226
+ "act_bits": 16,
1227
+ "bits": 16
1228
+ },
1229
+ "language_model.model.layers.42.self_attn.q_proj": {
1230
+ "act_bits": 16,
1231
+ "bits": 16
1232
+ },
1233
+ "language_model.model.layers.42.self_attn.v_proj": {
1234
+ "act_bits": 16,
1235
+ "bits": 16
1236
+ },
1237
+ "language_model.model.layers.43.feed_forward.router": {
1238
+ "act_bits": 16,
1239
+ "bits": 16
1240
+ },
1241
+ "language_model.model.layers.43.feed_forward.shared_expert.down_proj": {
1242
+ "act_bits": 16,
1243
+ "bits": 16
1244
+ },
1245
+ "language_model.model.layers.43.feed_forward.shared_expert.gate_proj": {
1246
+ "act_bits": 16,
1247
+ "bits": 16
1248
+ },
1249
+ "language_model.model.layers.43.feed_forward.shared_expert.up_proj": {
1250
+ "act_bits": 16,
1251
+ "bits": 16
1252
+ },
1253
+ "language_model.model.layers.43.self_attn.k_proj": {
1254
+ "act_bits": 16,
1255
+ "bits": 16
1256
+ },
1257
+ "language_model.model.layers.43.self_attn.o_proj": {
1258
+ "act_bits": 16,
1259
+ "bits": 16
1260
+ },
1261
+ "language_model.model.layers.43.self_attn.q_proj": {
1262
+ "act_bits": 16,
1263
+ "bits": 16
1264
+ },
1265
+ "language_model.model.layers.43.self_attn.v_proj": {
1266
+ "act_bits": 16,
1267
+ "bits": 16
1268
+ },
1269
+ "language_model.model.layers.44.feed_forward.router": {
1270
+ "act_bits": 16,
1271
+ "bits": 16
1272
+ },
1273
+ "language_model.model.layers.44.feed_forward.shared_expert.down_proj": {
1274
+ "act_bits": 16,
1275
+ "bits": 16
1276
+ },
1277
+ "language_model.model.layers.44.feed_forward.shared_expert.gate_proj": {
1278
+ "act_bits": 16,
1279
+ "bits": 16
1280
+ },
1281
+ "language_model.model.layers.44.feed_forward.shared_expert.up_proj": {
1282
+ "act_bits": 16,
1283
+ "bits": 16
1284
+ },
1285
+ "language_model.model.layers.44.self_attn.k_proj": {
1286
+ "act_bits": 16,
1287
+ "bits": 16
1288
+ },
1289
+ "language_model.model.layers.44.self_attn.o_proj": {
1290
+ "act_bits": 16,
1291
+ "bits": 16
1292
+ },
1293
+ "language_model.model.layers.44.self_attn.q_proj": {
1294
+ "act_bits": 16,
1295
+ "bits": 16
1296
+ },
1297
+ "language_model.model.layers.44.self_attn.v_proj": {
1298
+ "act_bits": 16,
1299
+ "bits": 16
1300
+ },
1301
+ "language_model.model.layers.45.feed_forward.router": {
1302
+ "act_bits": 16,
1303
+ "bits": 16
1304
+ },
1305
+ "language_model.model.layers.45.feed_forward.shared_expert.down_proj": {
1306
+ "act_bits": 16,
1307
+ "bits": 16
1308
+ },
1309
+ "language_model.model.layers.45.feed_forward.shared_expert.gate_proj": {
1310
+ "act_bits": 16,
1311
+ "bits": 16
1312
+ },
1313
+ "language_model.model.layers.45.feed_forward.shared_expert.up_proj": {
1314
+ "act_bits": 16,
1315
+ "bits": 16
1316
+ },
1317
+ "language_model.model.layers.45.self_attn.k_proj": {
1318
+ "act_bits": 16,
1319
+ "bits": 16
1320
+ },
1321
+ "language_model.model.layers.45.self_attn.o_proj": {
1322
+ "act_bits": 16,
1323
+ "bits": 16
1324
+ },
1325
+ "language_model.model.layers.45.self_attn.q_proj": {
1326
+ "act_bits": 16,
1327
+ "bits": 16
1328
+ },
1329
+ "language_model.model.layers.45.self_attn.v_proj": {
1330
+ "act_bits": 16,
1331
+ "bits": 16
1332
+ },
1333
+ "language_model.model.layers.46.feed_forward.router": {
1334
+ "act_bits": 16,
1335
+ "bits": 16
1336
+ },
1337
+ "language_model.model.layers.46.feed_forward.shared_expert.down_proj": {
1338
+ "act_bits": 16,
1339
+ "bits": 16
1340
+ },
1341
+ "language_model.model.layers.46.feed_forward.shared_expert.gate_proj": {
1342
+ "act_bits": 16,
1343
+ "bits": 16
1344
+ },
1345
+ "language_model.model.layers.46.feed_forward.shared_expert.up_proj": {
1346
+ "act_bits": 16,
1347
+ "bits": 16
1348
+ },
1349
+ "language_model.model.layers.46.self_attn.k_proj": {
1350
+ "act_bits": 16,
1351
+ "bits": 16
1352
+ },
1353
+ "language_model.model.layers.46.self_attn.o_proj": {
1354
+ "act_bits": 16,
1355
+ "bits": 16
1356
+ },
1357
+ "language_model.model.layers.46.self_attn.q_proj": {
1358
+ "act_bits": 16,
1359
+ "bits": 16
1360
+ },
1361
+ "language_model.model.layers.46.self_attn.v_proj": {
1362
+ "act_bits": 16,
1363
+ "bits": 16
1364
+ },
1365
+ "language_model.model.layers.47.feed_forward.router": {
1366
+ "act_bits": 16,
1367
+ "bits": 16
1368
+ },
1369
+ "language_model.model.layers.47.feed_forward.shared_expert.down_proj": {
1370
+ "act_bits": 16,
1371
+ "bits": 16
1372
+ },
1373
+ "language_model.model.layers.47.feed_forward.shared_expert.gate_proj": {
1374
+ "act_bits": 16,
1375
+ "bits": 16
1376
+ },
1377
+ "language_model.model.layers.47.feed_forward.shared_expert.up_proj": {
1378
+ "act_bits": 16,
1379
+ "bits": 16
1380
+ },
1381
+ "language_model.model.layers.47.self_attn.k_proj": {
1382
+ "act_bits": 16,
1383
+ "bits": 16
1384
+ },
1385
+ "language_model.model.layers.47.self_attn.o_proj": {
1386
+ "act_bits": 16,
1387
+ "bits": 16
1388
+ },
1389
+ "language_model.model.layers.47.self_attn.q_proj": {
1390
+ "act_bits": 16,
1391
+ "bits": 16
1392
+ },
1393
+ "language_model.model.layers.47.self_attn.v_proj": {
1394
+ "act_bits": 16,
1395
+ "bits": 16
1396
+ },
1397
+ "language_model.model.layers.5.feed_forward.router": {
1398
+ "act_bits": 16,
1399
+ "bits": 16
1400
+ },
1401
+ "language_model.model.layers.5.feed_forward.shared_expert.down_proj": {
1402
+ "act_bits": 16,
1403
+ "bits": 16
1404
+ },
1405
+ "language_model.model.layers.5.feed_forward.shared_expert.gate_proj": {
1406
+ "act_bits": 16,
1407
+ "bits": 16
1408
+ },
1409
+ "language_model.model.layers.5.feed_forward.shared_expert.up_proj": {
1410
+ "act_bits": 16,
1411
+ "bits": 16
1412
+ },
1413
+ "language_model.model.layers.5.self_attn.k_proj": {
1414
+ "act_bits": 16,
1415
+ "bits": 16
1416
+ },
1417
+ "language_model.model.layers.5.self_attn.o_proj": {
1418
+ "act_bits": 16,
1419
+ "bits": 16
1420
+ },
1421
+ "language_model.model.layers.5.self_attn.q_proj": {
1422
+ "act_bits": 16,
1423
+ "bits": 16
1424
+ },
1425
+ "language_model.model.layers.5.self_attn.v_proj": {
1426
+ "act_bits": 16,
1427
+ "bits": 16
1428
+ },
1429
+ "language_model.model.layers.6.feed_forward.router": {
1430
+ "act_bits": 16,
1431
+ "bits": 16
1432
+ },
1433
+ "language_model.model.layers.6.feed_forward.shared_expert.down_proj": {
1434
+ "act_bits": 16,
1435
+ "bits": 16
1436
+ },
1437
+ "language_model.model.layers.6.feed_forward.shared_expert.gate_proj": {
1438
+ "act_bits": 16,
1439
+ "bits": 16
1440
+ },
1441
+ "language_model.model.layers.6.feed_forward.shared_expert.up_proj": {
1442
+ "act_bits": 16,
1443
+ "bits": 16
1444
+ },
1445
+ "language_model.model.layers.6.self_attn.k_proj": {
1446
+ "act_bits": 16,
1447
+ "bits": 16
1448
+ },
1449
+ "language_model.model.layers.6.self_attn.o_proj": {
1450
+ "act_bits": 16,
1451
+ "bits": 16
1452
+ },
1453
+ "language_model.model.layers.6.self_attn.q_proj": {
1454
+ "act_bits": 16,
1455
+ "bits": 16
1456
+ },
1457
+ "language_model.model.layers.6.self_attn.v_proj": {
1458
+ "act_bits": 16,
1459
+ "bits": 16
1460
+ },
1461
+ "language_model.model.layers.7.feed_forward.router": {
1462
+ "act_bits": 16,
1463
+ "bits": 16
1464
+ },
1465
+ "language_model.model.layers.7.feed_forward.shared_expert.down_proj": {
1466
+ "act_bits": 16,
1467
+ "bits": 16
1468
+ },
1469
+ "language_model.model.layers.7.feed_forward.shared_expert.gate_proj": {
1470
+ "act_bits": 16,
1471
+ "bits": 16
1472
+ },
1473
+ "language_model.model.layers.7.feed_forward.shared_expert.up_proj": {
1474
+ "act_bits": 16,
1475
+ "bits": 16
1476
+ },
1477
+ "language_model.model.layers.7.self_attn.k_proj": {
1478
+ "act_bits": 16,
1479
+ "bits": 16
1480
+ },
1481
+ "language_model.model.layers.7.self_attn.o_proj": {
1482
+ "act_bits": 16,
1483
+ "bits": 16
1484
+ },
1485
+ "language_model.model.layers.7.self_attn.q_proj": {
1486
+ "act_bits": 16,
1487
+ "bits": 16
1488
+ },
1489
+ "language_model.model.layers.7.self_attn.v_proj": {
1490
+ "act_bits": 16,
1491
+ "bits": 16
1492
+ },
1493
+ "language_model.model.layers.8.feed_forward.router": {
1494
+ "act_bits": 16,
1495
+ "bits": 16
1496
+ },
1497
+ "language_model.model.layers.8.feed_forward.shared_expert.down_proj": {
1498
+ "act_bits": 16,
1499
+ "bits": 16
1500
+ },
1501
+ "language_model.model.layers.8.feed_forward.shared_expert.gate_proj": {
1502
+ "act_bits": 16,
1503
+ "bits": 16
1504
+ },
1505
+ "language_model.model.layers.8.feed_forward.shared_expert.up_proj": {
1506
+ "act_bits": 16,
1507
+ "bits": 16
1508
+ },
1509
+ "language_model.model.layers.8.self_attn.k_proj": {
1510
+ "act_bits": 16,
1511
+ "bits": 16
1512
+ },
1513
+ "language_model.model.layers.8.self_attn.o_proj": {
1514
+ "act_bits": 16,
1515
+ "bits": 16
1516
+ },
1517
+ "language_model.model.layers.8.self_attn.q_proj": {
1518
+ "act_bits": 16,
1519
+ "bits": 16
1520
+ },
1521
+ "language_model.model.layers.8.self_attn.v_proj": {
1522
+ "act_bits": 16,
1523
+ "bits": 16
1524
+ },
1525
+ "language_model.model.layers.9.feed_forward.router": {
1526
+ "act_bits": 16,
1527
+ "bits": 16
1528
+ },
1529
+ "language_model.model.layers.9.feed_forward.shared_expert.down_proj": {
1530
+ "act_bits": 16,
1531
+ "bits": 16
1532
+ },
1533
+ "language_model.model.layers.9.feed_forward.shared_expert.gate_proj": {
1534
+ "act_bits": 16,
1535
+ "bits": 16
1536
+ },
1537
+ "language_model.model.layers.9.feed_forward.shared_expert.up_proj": {
1538
+ "act_bits": 16,
1539
+ "bits": 16
1540
+ },
1541
+ "language_model.model.layers.9.self_attn.k_proj": {
1542
+ "act_bits": 16,
1543
+ "bits": 16
1544
+ },
1545
+ "language_model.model.layers.9.self_attn.o_proj": {
1546
+ "act_bits": 16,
1547
+ "bits": 16
1548
+ },
1549
+ "language_model.model.layers.9.self_attn.q_proj": {
1550
+ "act_bits": 16,
1551
+ "bits": 16
1552
+ },
1553
+ "language_model.model.layers.9.self_attn.v_proj": {
1554
+ "act_bits": 16,
1555
+ "bits": 16
1556
+ }
1557
+ },
1558
+ "group_size": -1,
1559
+ "iters": 0,
1560
+ "packing_format": "auto_round:fp8_static",
1561
+ "quant_method": "auto-round",
1562
+ "sym": true
1563
+ },
1564
+ "text_config": {
1565
+ "_attn_implementation_autoset": true,
1566
+ "attention_bias": false,
1567
+ "attention_chunk_size": 8192,
1568
+ "attention_dropout": 0.0,
1569
+ "attn_scale": 0.1,
1570
+ "attn_temperature_tuning": true,
1571
+ "bos_token_id": 200000,
1572
+ "dtype": "bfloat16",
1573
+ "eos_token_id": [
1574
+ 200001,
1575
+ 200007,
1576
+ 200008
1577
+ ],
1578
+ "floor_scale": 8192,
1579
+ "for_llm_compressor": false,
1580
+ "head_dim": 128,
1581
+ "hidden_act": "silu",
1582
+ "hidden_size": 5120,
1583
+ "initializer_range": 0.02,
1584
+ "interleave_moe_layer_step": 1,
1585
+ "intermediate_size": 8192,
1586
+ "intermediate_size_mlp": 16384,
1587
+ "layer_types": [
1588
+ "chunked_attention",
1589
+ "chunked_attention",
1590
+ "chunked_attention",
1591
+ "full_attention",
1592
+ "chunked_attention",
1593
+ "chunked_attention",
1594
+ "chunked_attention",
1595
+ "full_attention",
1596
+ "chunked_attention",
1597
+ "chunked_attention",
1598
+ "chunked_attention",
1599
+ "full_attention",
1600
+ "chunked_attention",
1601
+ "chunked_attention",
1602
+ "chunked_attention",
1603
+ "full_attention",
1604
+ "chunked_attention",
1605
+ "chunked_attention",
1606
+ "chunked_attention",
1607
+ "full_attention",
1608
+ "chunked_attention",
1609
+ "chunked_attention",
1610
+ "chunked_attention",
1611
+ "full_attention",
1612
+ "chunked_attention",
1613
+ "chunked_attention",
1614
+ "chunked_attention",
1615
+ "full_attention",
1616
+ "chunked_attention",
1617
+ "chunked_attention",
1618
+ "chunked_attention",
1619
+ "full_attention",
1620
+ "chunked_attention",
1621
+ "chunked_attention",
1622
+ "chunked_attention",
1623
+ "full_attention",
1624
+ "chunked_attention",
1625
+ "chunked_attention",
1626
+ "chunked_attention",
1627
+ "full_attention",
1628
+ "chunked_attention",
1629
+ "chunked_attention",
1630
+ "chunked_attention",
1631
+ "full_attention",
1632
+ "chunked_attention",
1633
+ "chunked_attention",
1634
+ "chunked_attention",
1635
+ "full_attention"
1636
+ ],
1637
+ "max_position_embeddings": 10485760,
1638
+ "model_type": "llama4_text",
1639
+ "moe_layers": [
1640
+ 0,
1641
+ 1,
1642
+ 2,
1643
+ 3,
1644
+ 4,
1645
+ 5,
1646
+ 6,
1647
+ 7,
1648
+ 8,
1649
+ 9,
1650
+ 10,
1651
+ 11,
1652
+ 12,
1653
+ 13,
1654
+ 14,
1655
+ 15,
1656
+ 16,
1657
+ 17,
1658
+ 18,
1659
+ 19,
1660
+ 20,
1661
+ 21,
1662
+ 22,
1663
+ 23,
1664
+ 24,
1665
+ 25,
1666
+ 26,
1667
+ 27,
1668
+ 28,
1669
+ 29,
1670
+ 30,
1671
+ 31,
1672
+ 32,
1673
+ 33,
1674
+ 34,
1675
+ 35,
1676
+ 36,
1677
+ 37,
1678
+ 38,
1679
+ 39,
1680
+ 40,
1681
+ 41,
1682
+ 42,
1683
+ 43,
1684
+ 44,
1685
+ 45,
1686
+ 46,
1687
+ 47
1688
+ ],
1689
+ "no_rope_layers": [
1690
+ 1,
1691
+ 1,
1692
+ 1,
1693
+ 0,
1694
+ 1,
1695
+ 1,
1696
+ 1,
1697
+ 0,
1698
+ 1,
1699
+ 1,
1700
+ 1,
1701
+ 0,
1702
+ 1,
1703
+ 1,
1704
+ 1,
1705
+ 0,
1706
+ 1,
1707
+ 1,
1708
+ 1,
1709
+ 0,
1710
+ 1,
1711
+ 1,
1712
+ 1,
1713
+ 0,
1714
+ 1,
1715
+ 1,
1716
+ 1,
1717
+ 0,
1718
+ 1,
1719
+ 1,
1720
+ 1,
1721
+ 0,
1722
+ 1,
1723
+ 1,
1724
+ 1,
1725
+ 0,
1726
+ 1,
1727
+ 1,
1728
+ 1,
1729
+ 0,
1730
+ 1,
1731
+ 1,
1732
+ 1,
1733
+ 0,
1734
+ 1,
1735
+ 1,
1736
+ 1,
1737
+ 0
1738
+ ],
1739
+ "num_attention_heads": 40,
1740
+ "num_experts_per_tok": 1,
1741
+ "num_hidden_layers": 48,
1742
+ "num_key_value_heads": 8,
1743
+ "num_local_experts": 16,
1744
+ "output_router_logits": false,
1745
+ "pad_token_id": 200018,
1746
+ "rms_norm_eps": 1e-05,
1747
+ "rope_scaling": {
1748
+ "factor": 16.0,
1749
+ "high_freq_factor": 1.0,
1750
+ "low_freq_factor": 1.0,
1751
+ "original_max_position_embeddings": 8192,
1752
+ "rope_type": "llama3"
1753
+ },
1754
+ "rope_theta": 500000.0,
1755
+ "router_aux_loss_coef": 0.001,
1756
+ "router_jitter_noise": 0.0,
1757
+ "use_cache": true,
1758
+ "use_qk_norm": true,
1759
+ "vocab_size": 202048
1760
+ },
1761
+ "tie_word_embeddings": false,
1762
+ "transformers_version": "4.56.1",
1763
+ "vision_config": {
1764
+ "_attn_implementation_autoset": true,
1765
+ "attention_dropout": 0.0,
1766
+ "hidden_act": "gelu",
1767
+ "hidden_size": 1408,
1768
+ "image_size": 336,
1769
+ "initializer_range": 0.02,
1770
+ "intermediate_size": 5632,
1771
+ "model_type": "llama4_vision_model",
1772
+ "multi_modal_projector_bias": false,
1773
+ "norm_eps": 1e-05,
1774
+ "num_attention_heads": 16,
1775
+ "num_channels": 3,
1776
+ "num_hidden_layers": 34,
1777
+ "patch_size": 14,
1778
+ "pixel_shuffle_ratio": 0.5,
1779
+ "projector_dropout": 0.0,
1780
+ "projector_input_dim": 4096,
1781
+ "projector_output_dim": 4096,
1782
+ "rope_theta": 10000,
1783
+ "vision_feature_layer": -1,
1784
+ "vision_feature_select_strategy": "default",
1785
+ "vision_output_dim": 4096
1786
+ }
1787
+ }
config.json ADDED
@@ -0,0 +1,288 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Llama4ForConditionalGeneration"
4
+ ],
5
+ "boi_token_index": 200080,
6
+ "dtype": "bfloat16",
7
+ "eoi_token_index": 200081,
8
+ "image_token_index": 200092,
9
+ "model_type": "llama4",
10
+ "quantization_config": {
11
+ "config_groups": {
12
+ "group_0": {
13
+ "format": "float-quantized",
14
+ "input_activations": {
15
+ "actorder": null,
16
+ "block_structure": null,
17
+ "dynamic": false,
18
+ "group_size": null,
19
+ "num_bits": 8,
20
+ "observer": "minmax",
21
+ "observer_kwargs": {},
22
+ "strategy": "tensor",
23
+ "symmetric": true,
24
+ "type": "float"
25
+ },
26
+ "output_activations": null,
27
+ "targets": [
28
+ "Linear"
29
+ ],
30
+ "weights": {
31
+ "actorder": null,
32
+ "block_structure": null,
33
+ "dynamic": false,
34
+ "group_size": null,
35
+ "num_bits": 8,
36
+ "observer": "minmax",
37
+ "observer_kwargs": {},
38
+ "strategy": "channel",
39
+ "symmetric": true,
40
+ "type": "float"
41
+ }
42
+ }
43
+ },
44
+ "format": "float-quantized",
45
+ "global_compression_ratio": null,
46
+ "ignore": [
47
+ "re:.*router",
48
+ "re:.*shared_expert",
49
+ "re:.*feed_forward.down_proj",
50
+ "re:.*feed_forward.gate_proj",
51
+ "re:.*feed_forward.up_proj",
52
+ "re:.*k_proj",
53
+ "re:.*o_proj",
54
+ "re:.*q_proj",
55
+ "re:.*v_proj",
56
+ "re:.*qkv_proj",
57
+ "re:.*self_attn",
58
+ "re:.*lm_head",
59
+ "re:vision_mdoel.*"
60
+ ],
61
+ "kv_cache_scheme": null,
62
+ "quant_method": "compressed-tensors",
63
+ "quantization_status": "compressed"
64
+ },
65
+ "text_config": {
66
+ "_attn_implementation_autoset": true,
67
+ "attention_bias": false,
68
+ "attention_chunk_size": 8192,
69
+ "attention_dropout": 0.0,
70
+ "attn_scale": 0.1,
71
+ "attn_temperature_tuning": true,
72
+ "bos_token_id": 200000,
73
+ "dtype": "bfloat16",
74
+ "eos_token_id": [
75
+ 200001,
76
+ 200007,
77
+ 200008
78
+ ],
79
+ "floor_scale": 8192,
80
+ "for_llm_compressor": false,
81
+ "head_dim": 128,
82
+ "hidden_act": "silu",
83
+ "hidden_size": 5120,
84
+ "initializer_range": 0.02,
85
+ "interleave_moe_layer_step": 1,
86
+ "intermediate_size": 8192,
87
+ "intermediate_size_mlp": 16384,
88
+ "layer_types": [
89
+ "chunked_attention",
90
+ "chunked_attention",
91
+ "chunked_attention",
92
+ "full_attention",
93
+ "chunked_attention",
94
+ "chunked_attention",
95
+ "chunked_attention",
96
+ "full_attention",
97
+ "chunked_attention",
98
+ "chunked_attention",
99
+ "chunked_attention",
100
+ "full_attention",
101
+ "chunked_attention",
102
+ "chunked_attention",
103
+ "chunked_attention",
104
+ "full_attention",
105
+ "chunked_attention",
106
+ "chunked_attention",
107
+ "chunked_attention",
108
+ "full_attention",
109
+ "chunked_attention",
110
+ "chunked_attention",
111
+ "chunked_attention",
112
+ "full_attention",
113
+ "chunked_attention",
114
+ "chunked_attention",
115
+ "chunked_attention",
116
+ "full_attention",
117
+ "chunked_attention",
118
+ "chunked_attention",
119
+ "chunked_attention",
120
+ "full_attention",
121
+ "chunked_attention",
122
+ "chunked_attention",
123
+ "chunked_attention",
124
+ "full_attention",
125
+ "chunked_attention",
126
+ "chunked_attention",
127
+ "chunked_attention",
128
+ "full_attention",
129
+ "chunked_attention",
130
+ "chunked_attention",
131
+ "chunked_attention",
132
+ "full_attention",
133
+ "chunked_attention",
134
+ "chunked_attention",
135
+ "chunked_attention",
136
+ "full_attention"
137
+ ],
138
+ "max_position_embeddings": 10485760,
139
+ "model_type": "llama4_text",
140
+ "moe_layers": [
141
+ 0,
142
+ 1,
143
+ 2,
144
+ 3,
145
+ 4,
146
+ 5,
147
+ 6,
148
+ 7,
149
+ 8,
150
+ 9,
151
+ 10,
152
+ 11,
153
+ 12,
154
+ 13,
155
+ 14,
156
+ 15,
157
+ 16,
158
+ 17,
159
+ 18,
160
+ 19,
161
+ 20,
162
+ 21,
163
+ 22,
164
+ 23,
165
+ 24,
166
+ 25,
167
+ 26,
168
+ 27,
169
+ 28,
170
+ 29,
171
+ 30,
172
+ 31,
173
+ 32,
174
+ 33,
175
+ 34,
176
+ 35,
177
+ 36,
178
+ 37,
179
+ 38,
180
+ 39,
181
+ 40,
182
+ 41,
183
+ 42,
184
+ 43,
185
+ 44,
186
+ 45,
187
+ 46,
188
+ 47
189
+ ],
190
+ "no_rope_layers": [
191
+ 1,
192
+ 1,
193
+ 1,
194
+ 0,
195
+ 1,
196
+ 1,
197
+ 1,
198
+ 0,
199
+ 1,
200
+ 1,
201
+ 1,
202
+ 0,
203
+ 1,
204
+ 1,
205
+ 1,
206
+ 0,
207
+ 1,
208
+ 1,
209
+ 1,
210
+ 0,
211
+ 1,
212
+ 1,
213
+ 1,
214
+ 0,
215
+ 1,
216
+ 1,
217
+ 1,
218
+ 0,
219
+ 1,
220
+ 1,
221
+ 1,
222
+ 0,
223
+ 1,
224
+ 1,
225
+ 1,
226
+ 0,
227
+ 1,
228
+ 1,
229
+ 1,
230
+ 0,
231
+ 1,
232
+ 1,
233
+ 1,
234
+ 0,
235
+ 1,
236
+ 1,
237
+ 1,
238
+ 0
239
+ ],
240
+ "num_attention_heads": 40,
241
+ "num_experts_per_tok": 1,
242
+ "num_hidden_layers": 48,
243
+ "num_key_value_heads": 8,
244
+ "num_local_experts": 16,
245
+ "output_router_logits": false,
246
+ "pad_token_id": 200018,
247
+ "rms_norm_eps": 1e-05,
248
+ "rope_scaling": {
249
+ "factor": 16.0,
250
+ "high_freq_factor": 1.0,
251
+ "low_freq_factor": 1.0,
252
+ "original_max_position_embeddings": 8192,
253
+ "rope_type": "llama3"
254
+ },
255
+ "rope_theta": 500000.0,
256
+ "router_aux_loss_coef": 0.001,
257
+ "router_jitter_noise": 0.0,
258
+ "use_cache": true,
259
+ "use_qk_norm": true,
260
+ "vocab_size": 202048
261
+ },
262
+ "tie_word_embeddings": false,
263
+ "transformers_version": "4.56.1",
264
+ "vision_config": {
265
+ "_attn_implementation_autoset": true,
266
+ "attention_dropout": 0.0,
267
+ "hidden_act": "gelu",
268
+ "hidden_size": 1408,
269
+ "image_size": 336,
270
+ "initializer_range": 0.02,
271
+ "intermediate_size": 5632,
272
+ "model_type": "llama4_vision_model",
273
+ "multi_modal_projector_bias": false,
274
+ "norm_eps": 1e-05,
275
+ "num_attention_heads": 16,
276
+ "num_channels": 3,
277
+ "num_hidden_layers": 34,
278
+ "patch_size": 14,
279
+ "pixel_shuffle_ratio": 0.5,
280
+ "projector_dropout": 0.0,
281
+ "projector_input_dim": 4096,
282
+ "projector_output_dim": 4096,
283
+ "rope_theta": 10000,
284
+ "vision_feature_layer": -1,
285
+ "vision_feature_select_strategy": "default",
286
+ "vision_output_dim": 4096
287
+ }
288
+ }
generation_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 200000,
3
+ "do_sample": true,
4
+ "eos_token_id": [
5
+ 200001,
6
+ 200007,
7
+ 200008
8
+ ],
9
+ "pad_token_id": 200018,
10
+ "temperature": 0.6,
11
+ "top_p": 0.9,
12
+ "transformers_version": "4.56.1"
13
+ }
model-00001-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9f294539691a27e8e9f7d38041569c408bd02d249429f76cd0edf44486f2fc9
3
+ size 4988043012
model-00002-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d4b43d6ac2877749dd2f12b9e5ac3c8f8a6f7384849c77efb5732206e7f3feb1
3
+ size 4994528756
model-00003-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9e004d2396978c2ca4fe22692ac01c400c11c2f689bbf4042be90b9469c24ac
3
+ size 4994541052
model-00004-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ebb480bfc9456924da2e84b56ad8b047ea9912f091eb34d0c0589df1d284144
3
+ size 4994528772
model-00005-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e984daf35f9cff742653b15fd4cdeccdd6481383ca38bba45ead16e8234c9178
3
+ size 4994528764
model-00006-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0a2f33c8589c3ce593e01062b5433774e7bca4a16ea7bf2e1bfe6dfac1e2b87
3
+ size 4994638916
model-00007-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3f9158a9d8052296f29db906cdd55ffa7e2e0ccb0b4a4fb64f4a54d00acf6ec8
3
+ size 4952465632
model-00008-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f30d50c28d6baed1002071e31bde26c2cdd874f783488c6f03318a73fd7b9ce3
3
+ size 4994454560
model-00009-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c445e48ae8c6392e2fecdfbac500baa80d207d09f06aee591f75a1cf57241b97
3
+ size 4994529084
model-00010-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0cce5c31c9d4688cc3f7d328f2f1071a7877701d8f5fb69a053f875a21eb3d24
3
+ size 4994529076
model-00011-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a672a0009da3c5fd94fa22aa268d5e6a47003fc3eed35d9200581ad149195674
3
+ size 4994541364
model-00012-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b140d365271c34d8120a7e8b58bc538511b16d3c4709d01a7c29b9645c86ccb7
3
+ size 4994529084
model-00013-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42c2b3afba49b1de8b2d4c62a5cb896b474d004df2cb23bff42f7ac9a01ff5f6
3
+ size 4994529076
model-00014-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2efb8eec282aed74a44860a9a37dc8a7579d7ec8eb3ebc144fa2a710cb6bcf58
3
+ size 4994541372
model-00015-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:30ec4595e00836d442ccdfbf1daa61ddd80301d0db593e358825aa8c3d33ec1a
3
+ size 4994529100
model-00016-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1c960736a5c8bbea9ce09b625c75ae30a720eb52d230810e6240ac4850072ee
3
+ size 4994529092
model-00017-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65421d5d66e498eb7ddd3f457e0b9077065aa730edfda2f8d1051f8fb8232019
3
+ size 4994541380
model-00018-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3140099dee19a3e28949baf46ef21bcb737e233ffd196cabc1ae96dabdbdb180
3
+ size 4994572828
model-00019-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d17cb67646535efc2cf30770936e2498238888ff86e58920dbe7650c20100e72
3
+ size 4994409048
model-00020-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4ca67315bd3083dcb5aaa80cea8dea38b9019ebaa0035ebe2efec255998b6d36
3
+ size 4994541364
model-00021-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7f5d84f72f0951e4db3c160dc4dcd1b4e1c902c3cd55ff30831f937d995cf7b
3
+ size 4994529084
model-00022-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5a4630d5e4cfefb53e59afabfa7c2a785f0b041ccb0818e58f6cfd856e826b19
3
+ size 4994529076
model-00023-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa1a4d88f9d800feb3007d8bbffc0328d31d444c57b6885b9d84d7d50108417e
3
+ size 4994541364
model-00024-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac8dfe6521fe136b3a4e3ccbace78cf7dfb3bfe5f4a83bd705cd6c8da2a7da65
3
+ size 3819401928
model-00025-of-00025.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f5a464e8b9d4ee51a76765c231720b677c1c2fe9efde77b306310d9d5a846219
3
+ size 2068971632
model.safetensors.index.json ADDED
The diff for this file is too large to render. See raw diff
 
preprocessor_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "crop_size": null,
3
+ "data_format": "channels_first",
4
+ "default_to_square": true,
5
+ "device": null,
6
+ "disable_grouping": null,
7
+ "do_center_crop": null,
8
+ "do_convert_rgb": true,
9
+ "do_normalize": true,
10
+ "do_rescale": true,
11
+ "do_resize": true,
12
+ "image_mean": [
13
+ 0.5,
14
+ 0.5,
15
+ 0.5
16
+ ],
17
+ "image_processor_type": "Llama4ImageProcessorFast",
18
+ "image_std": [
19
+ 0.5,
20
+ 0.5,
21
+ 0.5
22
+ ],
23
+ "input_data_format": null,
24
+ "max_patches": 16,
25
+ "processor_class": "Llama4Processor",
26
+ "resample": 2,
27
+ "rescale_factor": 0.00392156862745098,
28
+ "resize_to_max_canvas": false,
29
+ "return_tensors": null,
30
+ "size": {
31
+ "height": 336,
32
+ "width": 336
33
+ }
34
+ }
processor_config.json ADDED
@@ -0,0 +1,6 @@
 
 
 
 
 
 
 
1
+ {
2
+ "fake_image_token": "<|image|>",
3
+ "image_token": "<|image|>",
4
+ "patch_size": 14,
5
+ "processor_class": "Llama4Processor"
6
+ }
quantization_config.json ADDED
@@ -0,0 +1,1554 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bits": 8,
3
+ "group_size": -1,
4
+ "sym": true,
5
+ "data_type": "fp",
6
+ "iters": 0,
7
+ "act_bits": 8,
8
+ "act_group_size": 0,
9
+ "act_sym": true,
10
+ "act_dynamic": false,
11
+ "act_data_type": "fp",
12
+ "autoround_version": "0.7.0",
13
+ "block_name_to_quantize": "language_model.model.layers",
14
+ "quant_method": "auto-round",
15
+ "packing_format": "auto_round:fp8_static",
16
+ "extra_config": {
17
+ "language_model.model.layers.0.self_attn.q_proj": {
18
+ "bits": 16,
19
+ "act_bits": 16
20
+ },
21
+ "language_model.model.layers.0.self_attn.k_proj": {
22
+ "bits": 16,
23
+ "act_bits": 16
24
+ },
25
+ "language_model.model.layers.0.self_attn.v_proj": {
26
+ "bits": 16,
27
+ "act_bits": 16
28
+ },
29
+ "language_model.model.layers.0.self_attn.o_proj": {
30
+ "bits": 16,
31
+ "act_bits": 16
32
+ },
33
+ "language_model.model.layers.0.feed_forward.router": {
34
+ "bits": 16,
35
+ "act_bits": 16
36
+ },
37
+ "language_model.model.layers.0.feed_forward.shared_expert.gate_proj": {
38
+ "bits": 16,
39
+ "act_bits": 16
40
+ },
41
+ "language_model.model.layers.0.feed_forward.shared_expert.up_proj": {
42
+ "bits": 16,
43
+ "act_bits": 16
44
+ },
45
+ "language_model.model.layers.0.feed_forward.shared_expert.down_proj": {
46
+ "bits": 16,
47
+ "act_bits": 16
48
+ },
49
+ "language_model.model.layers.1.self_attn.q_proj": {
50
+ "bits": 16,
51
+ "act_bits": 16
52
+ },
53
+ "language_model.model.layers.1.self_attn.k_proj": {
54
+ "bits": 16,
55
+ "act_bits": 16
56
+ },
57
+ "language_model.model.layers.1.self_attn.v_proj": {
58
+ "bits": 16,
59
+ "act_bits": 16
60
+ },
61
+ "language_model.model.layers.1.self_attn.o_proj": {
62
+ "bits": 16,
63
+ "act_bits": 16
64
+ },
65
+ "language_model.model.layers.1.feed_forward.router": {
66
+ "bits": 16,
67
+ "act_bits": 16
68
+ },
69
+ "language_model.model.layers.1.feed_forward.shared_expert.gate_proj": {
70
+ "bits": 16,
71
+ "act_bits": 16
72
+ },
73
+ "language_model.model.layers.1.feed_forward.shared_expert.up_proj": {
74
+ "bits": 16,
75
+ "act_bits": 16
76
+ },
77
+ "language_model.model.layers.1.feed_forward.shared_expert.down_proj": {
78
+ "bits": 16,
79
+ "act_bits": 16
80
+ },
81
+ "language_model.model.layers.2.self_attn.q_proj": {
82
+ "bits": 16,
83
+ "act_bits": 16
84
+ },
85
+ "language_model.model.layers.2.self_attn.k_proj": {
86
+ "bits": 16,
87
+ "act_bits": 16
88
+ },
89
+ "language_model.model.layers.2.self_attn.v_proj": {
90
+ "bits": 16,
91
+ "act_bits": 16
92
+ },
93
+ "language_model.model.layers.2.self_attn.o_proj": {
94
+ "bits": 16,
95
+ "act_bits": 16
96
+ },
97
+ "language_model.model.layers.2.feed_forward.router": {
98
+ "bits": 16,
99
+ "act_bits": 16
100
+ },
101
+ "language_model.model.layers.2.feed_forward.shared_expert.gate_proj": {
102
+ "bits": 16,
103
+ "act_bits": 16
104
+ },
105
+ "language_model.model.layers.2.feed_forward.shared_expert.up_proj": {
106
+ "bits": 16,
107
+ "act_bits": 16
108
+ },
109
+ "language_model.model.layers.2.feed_forward.shared_expert.down_proj": {
110
+ "bits": 16,
111
+ "act_bits": 16
112
+ },
113
+ "language_model.model.layers.3.self_attn.q_proj": {
114
+ "bits": 16,
115
+ "act_bits": 16
116
+ },
117
+ "language_model.model.layers.3.self_attn.k_proj": {
118
+ "bits": 16,
119
+ "act_bits": 16
120
+ },
121
+ "language_model.model.layers.3.self_attn.v_proj": {
122
+ "bits": 16,
123
+ "act_bits": 16
124
+ },
125
+ "language_model.model.layers.3.self_attn.o_proj": {
126
+ "bits": 16,
127
+ "act_bits": 16
128
+ },
129
+ "language_model.model.layers.3.feed_forward.router": {
130
+ "bits": 16,
131
+ "act_bits": 16
132
+ },
133
+ "language_model.model.layers.3.feed_forward.shared_expert.gate_proj": {
134
+ "bits": 16,
135
+ "act_bits": 16
136
+ },
137
+ "language_model.model.layers.3.feed_forward.shared_expert.up_proj": {
138
+ "bits": 16,
139
+ "act_bits": 16
140
+ },
141
+ "language_model.model.layers.3.feed_forward.shared_expert.down_proj": {
142
+ "bits": 16,
143
+ "act_bits": 16
144
+ },
145
+ "language_model.model.layers.4.self_attn.q_proj": {
146
+ "bits": 16,
147
+ "act_bits": 16
148
+ },
149
+ "language_model.model.layers.4.self_attn.k_proj": {
150
+ "bits": 16,
151
+ "act_bits": 16
152
+ },
153
+ "language_model.model.layers.4.self_attn.v_proj": {
154
+ "bits": 16,
155
+ "act_bits": 16
156
+ },
157
+ "language_model.model.layers.4.self_attn.o_proj": {
158
+ "bits": 16,
159
+ "act_bits": 16
160
+ },
161
+ "language_model.model.layers.4.feed_forward.router": {
162
+ "bits": 16,
163
+ "act_bits": 16
164
+ },
165
+ "language_model.model.layers.4.feed_forward.shared_expert.gate_proj": {
166
+ "bits": 16,
167
+ "act_bits": 16
168
+ },
169
+ "language_model.model.layers.4.feed_forward.shared_expert.up_proj": {
170
+ "bits": 16,
171
+ "act_bits": 16
172
+ },
173
+ "language_model.model.layers.4.feed_forward.shared_expert.down_proj": {
174
+ "bits": 16,
175
+ "act_bits": 16
176
+ },
177
+ "language_model.model.layers.5.self_attn.q_proj": {
178
+ "bits": 16,
179
+ "act_bits": 16
180
+ },
181
+ "language_model.model.layers.5.self_attn.k_proj": {
182
+ "bits": 16,
183
+ "act_bits": 16
184
+ },
185
+ "language_model.model.layers.5.self_attn.v_proj": {
186
+ "bits": 16,
187
+ "act_bits": 16
188
+ },
189
+ "language_model.model.layers.5.self_attn.o_proj": {
190
+ "bits": 16,
191
+ "act_bits": 16
192
+ },
193
+ "language_model.model.layers.5.feed_forward.router": {
194
+ "bits": 16,
195
+ "act_bits": 16
196
+ },
197
+ "language_model.model.layers.5.feed_forward.shared_expert.gate_proj": {
198
+ "bits": 16,
199
+ "act_bits": 16
200
+ },
201
+ "language_model.model.layers.5.feed_forward.shared_expert.up_proj": {
202
+ "bits": 16,
203
+ "act_bits": 16
204
+ },
205
+ "language_model.model.layers.5.feed_forward.shared_expert.down_proj": {
206
+ "bits": 16,
207
+ "act_bits": 16
208
+ },
209
+ "language_model.model.layers.6.self_attn.q_proj": {
210
+ "bits": 16,
211
+ "act_bits": 16
212
+ },
213
+ "language_model.model.layers.6.self_attn.k_proj": {
214
+ "bits": 16,
215
+ "act_bits": 16
216
+ },
217
+ "language_model.model.layers.6.self_attn.v_proj": {
218
+ "bits": 16,
219
+ "act_bits": 16
220
+ },
221
+ "language_model.model.layers.6.self_attn.o_proj": {
222
+ "bits": 16,
223
+ "act_bits": 16
224
+ },
225
+ "language_model.model.layers.6.feed_forward.router": {
226
+ "bits": 16,
227
+ "act_bits": 16
228
+ },
229
+ "language_model.model.layers.6.feed_forward.shared_expert.gate_proj": {
230
+ "bits": 16,
231
+ "act_bits": 16
232
+ },
233
+ "language_model.model.layers.6.feed_forward.shared_expert.up_proj": {
234
+ "bits": 16,
235
+ "act_bits": 16
236
+ },
237
+ "language_model.model.layers.6.feed_forward.shared_expert.down_proj": {
238
+ "bits": 16,
239
+ "act_bits": 16
240
+ },
241
+ "language_model.model.layers.7.self_attn.q_proj": {
242
+ "bits": 16,
243
+ "act_bits": 16
244
+ },
245
+ "language_model.model.layers.7.self_attn.k_proj": {
246
+ "bits": 16,
247
+ "act_bits": 16
248
+ },
249
+ "language_model.model.layers.7.self_attn.v_proj": {
250
+ "bits": 16,
251
+ "act_bits": 16
252
+ },
253
+ "language_model.model.layers.7.self_attn.o_proj": {
254
+ "bits": 16,
255
+ "act_bits": 16
256
+ },
257
+ "language_model.model.layers.7.feed_forward.router": {
258
+ "bits": 16,
259
+ "act_bits": 16
260
+ },
261
+ "language_model.model.layers.7.feed_forward.shared_expert.gate_proj": {
262
+ "bits": 16,
263
+ "act_bits": 16
264
+ },
265
+ "language_model.model.layers.7.feed_forward.shared_expert.up_proj": {
266
+ "bits": 16,
267
+ "act_bits": 16
268
+ },
269
+ "language_model.model.layers.7.feed_forward.shared_expert.down_proj": {
270
+ "bits": 16,
271
+ "act_bits": 16
272
+ },
273
+ "language_model.model.layers.8.self_attn.q_proj": {
274
+ "bits": 16,
275
+ "act_bits": 16
276
+ },
277
+ "language_model.model.layers.8.self_attn.k_proj": {
278
+ "bits": 16,
279
+ "act_bits": 16
280
+ },
281
+ "language_model.model.layers.8.self_attn.v_proj": {
282
+ "bits": 16,
283
+ "act_bits": 16
284
+ },
285
+ "language_model.model.layers.8.self_attn.o_proj": {
286
+ "bits": 16,
287
+ "act_bits": 16
288
+ },
289
+ "language_model.model.layers.8.feed_forward.router": {
290
+ "bits": 16,
291
+ "act_bits": 16
292
+ },
293
+ "language_model.model.layers.8.feed_forward.shared_expert.gate_proj": {
294
+ "bits": 16,
295
+ "act_bits": 16
296
+ },
297
+ "language_model.model.layers.8.feed_forward.shared_expert.up_proj": {
298
+ "bits": 16,
299
+ "act_bits": 16
300
+ },
301
+ "language_model.model.layers.8.feed_forward.shared_expert.down_proj": {
302
+ "bits": 16,
303
+ "act_bits": 16
304
+ },
305
+ "language_model.model.layers.9.self_attn.q_proj": {
306
+ "bits": 16,
307
+ "act_bits": 16
308
+ },
309
+ "language_model.model.layers.9.self_attn.k_proj": {
310
+ "bits": 16,
311
+ "act_bits": 16
312
+ },
313
+ "language_model.model.layers.9.self_attn.v_proj": {
314
+ "bits": 16,
315
+ "act_bits": 16
316
+ },
317
+ "language_model.model.layers.9.self_attn.o_proj": {
318
+ "bits": 16,
319
+ "act_bits": 16
320
+ },
321
+ "language_model.model.layers.9.feed_forward.router": {
322
+ "bits": 16,
323
+ "act_bits": 16
324
+ },
325
+ "language_model.model.layers.9.feed_forward.shared_expert.gate_proj": {
326
+ "bits": 16,
327
+ "act_bits": 16
328
+ },
329
+ "language_model.model.layers.9.feed_forward.shared_expert.up_proj": {
330
+ "bits": 16,
331
+ "act_bits": 16
332
+ },
333
+ "language_model.model.layers.9.feed_forward.shared_expert.down_proj": {
334
+ "bits": 16,
335
+ "act_bits": 16
336
+ },
337
+ "language_model.model.layers.10.self_attn.q_proj": {
338
+ "bits": 16,
339
+ "act_bits": 16
340
+ },
341
+ "language_model.model.layers.10.self_attn.k_proj": {
342
+ "bits": 16,
343
+ "act_bits": 16
344
+ },
345
+ "language_model.model.layers.10.self_attn.v_proj": {
346
+ "bits": 16,
347
+ "act_bits": 16
348
+ },
349
+ "language_model.model.layers.10.self_attn.o_proj": {
350
+ "bits": 16,
351
+ "act_bits": 16
352
+ },
353
+ "language_model.model.layers.10.feed_forward.router": {
354
+ "bits": 16,
355
+ "act_bits": 16
356
+ },
357
+ "language_model.model.layers.10.feed_forward.shared_expert.gate_proj": {
358
+ "bits": 16,
359
+ "act_bits": 16
360
+ },
361
+ "language_model.model.layers.10.feed_forward.shared_expert.up_proj": {
362
+ "bits": 16,
363
+ "act_bits": 16
364
+ },
365
+ "language_model.model.layers.10.feed_forward.shared_expert.down_proj": {
366
+ "bits": 16,
367
+ "act_bits": 16
368
+ },
369
+ "language_model.model.layers.11.self_attn.q_proj": {
370
+ "bits": 16,
371
+ "act_bits": 16
372
+ },
373
+ "language_model.model.layers.11.self_attn.k_proj": {
374
+ "bits": 16,
375
+ "act_bits": 16
376
+ },
377
+ "language_model.model.layers.11.self_attn.v_proj": {
378
+ "bits": 16,
379
+ "act_bits": 16
380
+ },
381
+ "language_model.model.layers.11.self_attn.o_proj": {
382
+ "bits": 16,
383
+ "act_bits": 16
384
+ },
385
+ "language_model.model.layers.11.feed_forward.router": {
386
+ "bits": 16,
387
+ "act_bits": 16
388
+ },
389
+ "language_model.model.layers.11.feed_forward.shared_expert.gate_proj": {
390
+ "bits": 16,
391
+ "act_bits": 16
392
+ },
393
+ "language_model.model.layers.11.feed_forward.shared_expert.up_proj": {
394
+ "bits": 16,
395
+ "act_bits": 16
396
+ },
397
+ "language_model.model.layers.11.feed_forward.shared_expert.down_proj": {
398
+ "bits": 16,
399
+ "act_bits": 16
400
+ },
401
+ "language_model.model.layers.12.self_attn.q_proj": {
402
+ "bits": 16,
403
+ "act_bits": 16
404
+ },
405
+ "language_model.model.layers.12.self_attn.k_proj": {
406
+ "bits": 16,
407
+ "act_bits": 16
408
+ },
409
+ "language_model.model.layers.12.self_attn.v_proj": {
410
+ "bits": 16,
411
+ "act_bits": 16
412
+ },
413
+ "language_model.model.layers.12.self_attn.o_proj": {
414
+ "bits": 16,
415
+ "act_bits": 16
416
+ },
417
+ "language_model.model.layers.12.feed_forward.router": {
418
+ "bits": 16,
419
+ "act_bits": 16
420
+ },
421
+ "language_model.model.layers.12.feed_forward.shared_expert.gate_proj": {
422
+ "bits": 16,
423
+ "act_bits": 16
424
+ },
425
+ "language_model.model.layers.12.feed_forward.shared_expert.up_proj": {
426
+ "bits": 16,
427
+ "act_bits": 16
428
+ },
429
+ "language_model.model.layers.12.feed_forward.shared_expert.down_proj": {
430
+ "bits": 16,
431
+ "act_bits": 16
432
+ },
433
+ "language_model.model.layers.13.self_attn.q_proj": {
434
+ "bits": 16,
435
+ "act_bits": 16
436
+ },
437
+ "language_model.model.layers.13.self_attn.k_proj": {
438
+ "bits": 16,
439
+ "act_bits": 16
440
+ },
441
+ "language_model.model.layers.13.self_attn.v_proj": {
442
+ "bits": 16,
443
+ "act_bits": 16
444
+ },
445
+ "language_model.model.layers.13.self_attn.o_proj": {
446
+ "bits": 16,
447
+ "act_bits": 16
448
+ },
449
+ "language_model.model.layers.13.feed_forward.router": {
450
+ "bits": 16,
451
+ "act_bits": 16
452
+ },
453
+ "language_model.model.layers.13.feed_forward.shared_expert.gate_proj": {
454
+ "bits": 16,
455
+ "act_bits": 16
456
+ },
457
+ "language_model.model.layers.13.feed_forward.shared_expert.up_proj": {
458
+ "bits": 16,
459
+ "act_bits": 16
460
+ },
461
+ "language_model.model.layers.13.feed_forward.shared_expert.down_proj": {
462
+ "bits": 16,
463
+ "act_bits": 16
464
+ },
465
+ "language_model.model.layers.14.self_attn.q_proj": {
466
+ "bits": 16,
467
+ "act_bits": 16
468
+ },
469
+ "language_model.model.layers.14.self_attn.k_proj": {
470
+ "bits": 16,
471
+ "act_bits": 16
472
+ },
473
+ "language_model.model.layers.14.self_attn.v_proj": {
474
+ "bits": 16,
475
+ "act_bits": 16
476
+ },
477
+ "language_model.model.layers.14.self_attn.o_proj": {
478
+ "bits": 16,
479
+ "act_bits": 16
480
+ },
481
+ "language_model.model.layers.14.feed_forward.router": {
482
+ "bits": 16,
483
+ "act_bits": 16
484
+ },
485
+ "language_model.model.layers.14.feed_forward.shared_expert.gate_proj": {
486
+ "bits": 16,
487
+ "act_bits": 16
488
+ },
489
+ "language_model.model.layers.14.feed_forward.shared_expert.up_proj": {
490
+ "bits": 16,
491
+ "act_bits": 16
492
+ },
493
+ "language_model.model.layers.14.feed_forward.shared_expert.down_proj": {
494
+ "bits": 16,
495
+ "act_bits": 16
496
+ },
497
+ "language_model.model.layers.15.self_attn.q_proj": {
498
+ "bits": 16,
499
+ "act_bits": 16
500
+ },
501
+ "language_model.model.layers.15.self_attn.k_proj": {
502
+ "bits": 16,
503
+ "act_bits": 16
504
+ },
505
+ "language_model.model.layers.15.self_attn.v_proj": {
506
+ "bits": 16,
507
+ "act_bits": 16
508
+ },
509
+ "language_model.model.layers.15.self_attn.o_proj": {
510
+ "bits": 16,
511
+ "act_bits": 16
512
+ },
513
+ "language_model.model.layers.15.feed_forward.router": {
514
+ "bits": 16,
515
+ "act_bits": 16
516
+ },
517
+ "language_model.model.layers.15.feed_forward.shared_expert.gate_proj": {
518
+ "bits": 16,
519
+ "act_bits": 16
520
+ },
521
+ "language_model.model.layers.15.feed_forward.shared_expert.up_proj": {
522
+ "bits": 16,
523
+ "act_bits": 16
524
+ },
525
+ "language_model.model.layers.15.feed_forward.shared_expert.down_proj": {
526
+ "bits": 16,
527
+ "act_bits": 16
528
+ },
529
+ "language_model.model.layers.16.self_attn.q_proj": {
530
+ "bits": 16,
531
+ "act_bits": 16
532
+ },
533
+ "language_model.model.layers.16.self_attn.k_proj": {
534
+ "bits": 16,
535
+ "act_bits": 16
536
+ },
537
+ "language_model.model.layers.16.self_attn.v_proj": {
538
+ "bits": 16,
539
+ "act_bits": 16
540
+ },
541
+ "language_model.model.layers.16.self_attn.o_proj": {
542
+ "bits": 16,
543
+ "act_bits": 16
544
+ },
545
+ "language_model.model.layers.16.feed_forward.router": {
546
+ "bits": 16,
547
+ "act_bits": 16
548
+ },
549
+ "language_model.model.layers.16.feed_forward.shared_expert.gate_proj": {
550
+ "bits": 16,
551
+ "act_bits": 16
552
+ },
553
+ "language_model.model.layers.16.feed_forward.shared_expert.up_proj": {
554
+ "bits": 16,
555
+ "act_bits": 16
556
+ },
557
+ "language_model.model.layers.16.feed_forward.shared_expert.down_proj": {
558
+ "bits": 16,
559
+ "act_bits": 16
560
+ },
561
+ "language_model.model.layers.17.self_attn.q_proj": {
562
+ "bits": 16,
563
+ "act_bits": 16
564
+ },
565
+ "language_model.model.layers.17.self_attn.k_proj": {
566
+ "bits": 16,
567
+ "act_bits": 16
568
+ },
569
+ "language_model.model.layers.17.self_attn.v_proj": {
570
+ "bits": 16,
571
+ "act_bits": 16
572
+ },
573
+ "language_model.model.layers.17.self_attn.o_proj": {
574
+ "bits": 16,
575
+ "act_bits": 16
576
+ },
577
+ "language_model.model.layers.17.feed_forward.router": {
578
+ "bits": 16,
579
+ "act_bits": 16
580
+ },
581
+ "language_model.model.layers.17.feed_forward.shared_expert.gate_proj": {
582
+ "bits": 16,
583
+ "act_bits": 16
584
+ },
585
+ "language_model.model.layers.17.feed_forward.shared_expert.up_proj": {
586
+ "bits": 16,
587
+ "act_bits": 16
588
+ },
589
+ "language_model.model.layers.17.feed_forward.shared_expert.down_proj": {
590
+ "bits": 16,
591
+ "act_bits": 16
592
+ },
593
+ "language_model.model.layers.18.self_attn.q_proj": {
594
+ "bits": 16,
595
+ "act_bits": 16
596
+ },
597
+ "language_model.model.layers.18.self_attn.k_proj": {
598
+ "bits": 16,
599
+ "act_bits": 16
600
+ },
601
+ "language_model.model.layers.18.self_attn.v_proj": {
602
+ "bits": 16,
603
+ "act_bits": 16
604
+ },
605
+ "language_model.model.layers.18.self_attn.o_proj": {
606
+ "bits": 16,
607
+ "act_bits": 16
608
+ },
609
+ "language_model.model.layers.18.feed_forward.router": {
610
+ "bits": 16,
611
+ "act_bits": 16
612
+ },
613
+ "language_model.model.layers.18.feed_forward.shared_expert.gate_proj": {
614
+ "bits": 16,
615
+ "act_bits": 16
616
+ },
617
+ "language_model.model.layers.18.feed_forward.shared_expert.up_proj": {
618
+ "bits": 16,
619
+ "act_bits": 16
620
+ },
621
+ "language_model.model.layers.18.feed_forward.shared_expert.down_proj": {
622
+ "bits": 16,
623
+ "act_bits": 16
624
+ },
625
+ "language_model.model.layers.19.self_attn.q_proj": {
626
+ "bits": 16,
627
+ "act_bits": 16
628
+ },
629
+ "language_model.model.layers.19.self_attn.k_proj": {
630
+ "bits": 16,
631
+ "act_bits": 16
632
+ },
633
+ "language_model.model.layers.19.self_attn.v_proj": {
634
+ "bits": 16,
635
+ "act_bits": 16
636
+ },
637
+ "language_model.model.layers.19.self_attn.o_proj": {
638
+ "bits": 16,
639
+ "act_bits": 16
640
+ },
641
+ "language_model.model.layers.19.feed_forward.router": {
642
+ "bits": 16,
643
+ "act_bits": 16
644
+ },
645
+ "language_model.model.layers.19.feed_forward.shared_expert.gate_proj": {
646
+ "bits": 16,
647
+ "act_bits": 16
648
+ },
649
+ "language_model.model.layers.19.feed_forward.shared_expert.up_proj": {
650
+ "bits": 16,
651
+ "act_bits": 16
652
+ },
653
+ "language_model.model.layers.19.feed_forward.shared_expert.down_proj": {
654
+ "bits": 16,
655
+ "act_bits": 16
656
+ },
657
+ "language_model.model.layers.20.self_attn.q_proj": {
658
+ "bits": 16,
659
+ "act_bits": 16
660
+ },
661
+ "language_model.model.layers.20.self_attn.k_proj": {
662
+ "bits": 16,
663
+ "act_bits": 16
664
+ },
665
+ "language_model.model.layers.20.self_attn.v_proj": {
666
+ "bits": 16,
667
+ "act_bits": 16
668
+ },
669
+ "language_model.model.layers.20.self_attn.o_proj": {
670
+ "bits": 16,
671
+ "act_bits": 16
672
+ },
673
+ "language_model.model.layers.20.feed_forward.router": {
674
+ "bits": 16,
675
+ "act_bits": 16
676
+ },
677
+ "language_model.model.layers.20.feed_forward.shared_expert.gate_proj": {
678
+ "bits": 16,
679
+ "act_bits": 16
680
+ },
681
+ "language_model.model.layers.20.feed_forward.shared_expert.up_proj": {
682
+ "bits": 16,
683
+ "act_bits": 16
684
+ },
685
+ "language_model.model.layers.20.feed_forward.shared_expert.down_proj": {
686
+ "bits": 16,
687
+ "act_bits": 16
688
+ },
689
+ "language_model.model.layers.21.self_attn.q_proj": {
690
+ "bits": 16,
691
+ "act_bits": 16
692
+ },
693
+ "language_model.model.layers.21.self_attn.k_proj": {
694
+ "bits": 16,
695
+ "act_bits": 16
696
+ },
697
+ "language_model.model.layers.21.self_attn.v_proj": {
698
+ "bits": 16,
699
+ "act_bits": 16
700
+ },
701
+ "language_model.model.layers.21.self_attn.o_proj": {
702
+ "bits": 16,
703
+ "act_bits": 16
704
+ },
705
+ "language_model.model.layers.21.feed_forward.router": {
706
+ "bits": 16,
707
+ "act_bits": 16
708
+ },
709
+ "language_model.model.layers.21.feed_forward.shared_expert.gate_proj": {
710
+ "bits": 16,
711
+ "act_bits": 16
712
+ },
713
+ "language_model.model.layers.21.feed_forward.shared_expert.up_proj": {
714
+ "bits": 16,
715
+ "act_bits": 16
716
+ },
717
+ "language_model.model.layers.21.feed_forward.shared_expert.down_proj": {
718
+ "bits": 16,
719
+ "act_bits": 16
720
+ },
721
+ "language_model.model.layers.22.self_attn.q_proj": {
722
+ "bits": 16,
723
+ "act_bits": 16
724
+ },
725
+ "language_model.model.layers.22.self_attn.k_proj": {
726
+ "bits": 16,
727
+ "act_bits": 16
728
+ },
729
+ "language_model.model.layers.22.self_attn.v_proj": {
730
+ "bits": 16,
731
+ "act_bits": 16
732
+ },
733
+ "language_model.model.layers.22.self_attn.o_proj": {
734
+ "bits": 16,
735
+ "act_bits": 16
736
+ },
737
+ "language_model.model.layers.22.feed_forward.router": {
738
+ "bits": 16,
739
+ "act_bits": 16
740
+ },
741
+ "language_model.model.layers.22.feed_forward.shared_expert.gate_proj": {
742
+ "bits": 16,
743
+ "act_bits": 16
744
+ },
745
+ "language_model.model.layers.22.feed_forward.shared_expert.up_proj": {
746
+ "bits": 16,
747
+ "act_bits": 16
748
+ },
749
+ "language_model.model.layers.22.feed_forward.shared_expert.down_proj": {
750
+ "bits": 16,
751
+ "act_bits": 16
752
+ },
753
+ "language_model.model.layers.23.self_attn.q_proj": {
754
+ "bits": 16,
755
+ "act_bits": 16
756
+ },
757
+ "language_model.model.layers.23.self_attn.k_proj": {
758
+ "bits": 16,
759
+ "act_bits": 16
760
+ },
761
+ "language_model.model.layers.23.self_attn.v_proj": {
762
+ "bits": 16,
763
+ "act_bits": 16
764
+ },
765
+ "language_model.model.layers.23.self_attn.o_proj": {
766
+ "bits": 16,
767
+ "act_bits": 16
768
+ },
769
+ "language_model.model.layers.23.feed_forward.router": {
770
+ "bits": 16,
771
+ "act_bits": 16
772
+ },
773
+ "language_model.model.layers.23.feed_forward.shared_expert.gate_proj": {
774
+ "bits": 16,
775
+ "act_bits": 16
776
+ },
777
+ "language_model.model.layers.23.feed_forward.shared_expert.up_proj": {
778
+ "bits": 16,
779
+ "act_bits": 16
780
+ },
781
+ "language_model.model.layers.23.feed_forward.shared_expert.down_proj": {
782
+ "bits": 16,
783
+ "act_bits": 16
784
+ },
785
+ "language_model.model.layers.24.self_attn.q_proj": {
786
+ "bits": 16,
787
+ "act_bits": 16
788
+ },
789
+ "language_model.model.layers.24.self_attn.k_proj": {
790
+ "bits": 16,
791
+ "act_bits": 16
792
+ },
793
+ "language_model.model.layers.24.self_attn.v_proj": {
794
+ "bits": 16,
795
+ "act_bits": 16
796
+ },
797
+ "language_model.model.layers.24.self_attn.o_proj": {
798
+ "bits": 16,
799
+ "act_bits": 16
800
+ },
801
+ "language_model.model.layers.24.feed_forward.router": {
802
+ "bits": 16,
803
+ "act_bits": 16
804
+ },
805
+ "language_model.model.layers.24.feed_forward.shared_expert.gate_proj": {
806
+ "bits": 16,
807
+ "act_bits": 16
808
+ },
809
+ "language_model.model.layers.24.feed_forward.shared_expert.up_proj": {
810
+ "bits": 16,
811
+ "act_bits": 16
812
+ },
813
+ "language_model.model.layers.24.feed_forward.shared_expert.down_proj": {
814
+ "bits": 16,
815
+ "act_bits": 16
816
+ },
817
+ "language_model.model.layers.25.self_attn.q_proj": {
818
+ "bits": 16,
819
+ "act_bits": 16
820
+ },
821
+ "language_model.model.layers.25.self_attn.k_proj": {
822
+ "bits": 16,
823
+ "act_bits": 16
824
+ },
825
+ "language_model.model.layers.25.self_attn.v_proj": {
826
+ "bits": 16,
827
+ "act_bits": 16
828
+ },
829
+ "language_model.model.layers.25.self_attn.o_proj": {
830
+ "bits": 16,
831
+ "act_bits": 16
832
+ },
833
+ "language_model.model.layers.25.feed_forward.router": {
834
+ "bits": 16,
835
+ "act_bits": 16
836
+ },
837
+ "language_model.model.layers.25.feed_forward.shared_expert.gate_proj": {
838
+ "bits": 16,
839
+ "act_bits": 16
840
+ },
841
+ "language_model.model.layers.25.feed_forward.shared_expert.up_proj": {
842
+ "bits": 16,
843
+ "act_bits": 16
844
+ },
845
+ "language_model.model.layers.25.feed_forward.shared_expert.down_proj": {
846
+ "bits": 16,
847
+ "act_bits": 16
848
+ },
849
+ "language_model.model.layers.26.self_attn.q_proj": {
850
+ "bits": 16,
851
+ "act_bits": 16
852
+ },
853
+ "language_model.model.layers.26.self_attn.k_proj": {
854
+ "bits": 16,
855
+ "act_bits": 16
856
+ },
857
+ "language_model.model.layers.26.self_attn.v_proj": {
858
+ "bits": 16,
859
+ "act_bits": 16
860
+ },
861
+ "language_model.model.layers.26.self_attn.o_proj": {
862
+ "bits": 16,
863
+ "act_bits": 16
864
+ },
865
+ "language_model.model.layers.26.feed_forward.router": {
866
+ "bits": 16,
867
+ "act_bits": 16
868
+ },
869
+ "language_model.model.layers.26.feed_forward.shared_expert.gate_proj": {
870
+ "bits": 16,
871
+ "act_bits": 16
872
+ },
873
+ "language_model.model.layers.26.feed_forward.shared_expert.up_proj": {
874
+ "bits": 16,
875
+ "act_bits": 16
876
+ },
877
+ "language_model.model.layers.26.feed_forward.shared_expert.down_proj": {
878
+ "bits": 16,
879
+ "act_bits": 16
880
+ },
881
+ "language_model.model.layers.27.self_attn.q_proj": {
882
+ "bits": 16,
883
+ "act_bits": 16
884
+ },
885
+ "language_model.model.layers.27.self_attn.k_proj": {
886
+ "bits": 16,
887
+ "act_bits": 16
888
+ },
889
+ "language_model.model.layers.27.self_attn.v_proj": {
890
+ "bits": 16,
891
+ "act_bits": 16
892
+ },
893
+ "language_model.model.layers.27.self_attn.o_proj": {
894
+ "bits": 16,
895
+ "act_bits": 16
896
+ },
897
+ "language_model.model.layers.27.feed_forward.router": {
898
+ "bits": 16,
899
+ "act_bits": 16
900
+ },
901
+ "language_model.model.layers.27.feed_forward.shared_expert.gate_proj": {
902
+ "bits": 16,
903
+ "act_bits": 16
904
+ },
905
+ "language_model.model.layers.27.feed_forward.shared_expert.up_proj": {
906
+ "bits": 16,
907
+ "act_bits": 16
908
+ },
909
+ "language_model.model.layers.27.feed_forward.shared_expert.down_proj": {
910
+ "bits": 16,
911
+ "act_bits": 16
912
+ },
913
+ "language_model.model.layers.28.self_attn.q_proj": {
914
+ "bits": 16,
915
+ "act_bits": 16
916
+ },
917
+ "language_model.model.layers.28.self_attn.k_proj": {
918
+ "bits": 16,
919
+ "act_bits": 16
920
+ },
921
+ "language_model.model.layers.28.self_attn.v_proj": {
922
+ "bits": 16,
923
+ "act_bits": 16
924
+ },
925
+ "language_model.model.layers.28.self_attn.o_proj": {
926
+ "bits": 16,
927
+ "act_bits": 16
928
+ },
929
+ "language_model.model.layers.28.feed_forward.router": {
930
+ "bits": 16,
931
+ "act_bits": 16
932
+ },
933
+ "language_model.model.layers.28.feed_forward.shared_expert.gate_proj": {
934
+ "bits": 16,
935
+ "act_bits": 16
936
+ },
937
+ "language_model.model.layers.28.feed_forward.shared_expert.up_proj": {
938
+ "bits": 16,
939
+ "act_bits": 16
940
+ },
941
+ "language_model.model.layers.28.feed_forward.shared_expert.down_proj": {
942
+ "bits": 16,
943
+ "act_bits": 16
944
+ },
945
+ "language_model.model.layers.29.self_attn.q_proj": {
946
+ "bits": 16,
947
+ "act_bits": 16
948
+ },
949
+ "language_model.model.layers.29.self_attn.k_proj": {
950
+ "bits": 16,
951
+ "act_bits": 16
952
+ },
953
+ "language_model.model.layers.29.self_attn.v_proj": {
954
+ "bits": 16,
955
+ "act_bits": 16
956
+ },
957
+ "language_model.model.layers.29.self_attn.o_proj": {
958
+ "bits": 16,
959
+ "act_bits": 16
960
+ },
961
+ "language_model.model.layers.29.feed_forward.router": {
962
+ "bits": 16,
963
+ "act_bits": 16
964
+ },
965
+ "language_model.model.layers.29.feed_forward.shared_expert.gate_proj": {
966
+ "bits": 16,
967
+ "act_bits": 16
968
+ },
969
+ "language_model.model.layers.29.feed_forward.shared_expert.up_proj": {
970
+ "bits": 16,
971
+ "act_bits": 16
972
+ },
973
+ "language_model.model.layers.29.feed_forward.shared_expert.down_proj": {
974
+ "bits": 16,
975
+ "act_bits": 16
976
+ },
977
+ "language_model.model.layers.30.self_attn.q_proj": {
978
+ "bits": 16,
979
+ "act_bits": 16
980
+ },
981
+ "language_model.model.layers.30.self_attn.k_proj": {
982
+ "bits": 16,
983
+ "act_bits": 16
984
+ },
985
+ "language_model.model.layers.30.self_attn.v_proj": {
986
+ "bits": 16,
987
+ "act_bits": 16
988
+ },
989
+ "language_model.model.layers.30.self_attn.o_proj": {
990
+ "bits": 16,
991
+ "act_bits": 16
992
+ },
993
+ "language_model.model.layers.30.feed_forward.router": {
994
+ "bits": 16,
995
+ "act_bits": 16
996
+ },
997
+ "language_model.model.layers.30.feed_forward.shared_expert.gate_proj": {
998
+ "bits": 16,
999
+ "act_bits": 16
1000
+ },
1001
+ "language_model.model.layers.30.feed_forward.shared_expert.up_proj": {
1002
+ "bits": 16,
1003
+ "act_bits": 16
1004
+ },
1005
+ "language_model.model.layers.30.feed_forward.shared_expert.down_proj": {
1006
+ "bits": 16,
1007
+ "act_bits": 16
1008
+ },
1009
+ "language_model.model.layers.31.self_attn.q_proj": {
1010
+ "bits": 16,
1011
+ "act_bits": 16
1012
+ },
1013
+ "language_model.model.layers.31.self_attn.k_proj": {
1014
+ "bits": 16,
1015
+ "act_bits": 16
1016
+ },
1017
+ "language_model.model.layers.31.self_attn.v_proj": {
1018
+ "bits": 16,
1019
+ "act_bits": 16
1020
+ },
1021
+ "language_model.model.layers.31.self_attn.o_proj": {
1022
+ "bits": 16,
1023
+ "act_bits": 16
1024
+ },
1025
+ "language_model.model.layers.31.feed_forward.router": {
1026
+ "bits": 16,
1027
+ "act_bits": 16
1028
+ },
1029
+ "language_model.model.layers.31.feed_forward.shared_expert.gate_proj": {
1030
+ "bits": 16,
1031
+ "act_bits": 16
1032
+ },
1033
+ "language_model.model.layers.31.feed_forward.shared_expert.up_proj": {
1034
+ "bits": 16,
1035
+ "act_bits": 16
1036
+ },
1037
+ "language_model.model.layers.31.feed_forward.shared_expert.down_proj": {
1038
+ "bits": 16,
1039
+ "act_bits": 16
1040
+ },
1041
+ "language_model.model.layers.32.self_attn.q_proj": {
1042
+ "bits": 16,
1043
+ "act_bits": 16
1044
+ },
1045
+ "language_model.model.layers.32.self_attn.k_proj": {
1046
+ "bits": 16,
1047
+ "act_bits": 16
1048
+ },
1049
+ "language_model.model.layers.32.self_attn.v_proj": {
1050
+ "bits": 16,
1051
+ "act_bits": 16
1052
+ },
1053
+ "language_model.model.layers.32.self_attn.o_proj": {
1054
+ "bits": 16,
1055
+ "act_bits": 16
1056
+ },
1057
+ "language_model.model.layers.32.feed_forward.router": {
1058
+ "bits": 16,
1059
+ "act_bits": 16
1060
+ },
1061
+ "language_model.model.layers.32.feed_forward.shared_expert.gate_proj": {
1062
+ "bits": 16,
1063
+ "act_bits": 16
1064
+ },
1065
+ "language_model.model.layers.32.feed_forward.shared_expert.up_proj": {
1066
+ "bits": 16,
1067
+ "act_bits": 16
1068
+ },
1069
+ "language_model.model.layers.32.feed_forward.shared_expert.down_proj": {
1070
+ "bits": 16,
1071
+ "act_bits": 16
1072
+ },
1073
+ "language_model.model.layers.33.self_attn.q_proj": {
1074
+ "bits": 16,
1075
+ "act_bits": 16
1076
+ },
1077
+ "language_model.model.layers.33.self_attn.k_proj": {
1078
+ "bits": 16,
1079
+ "act_bits": 16
1080
+ },
1081
+ "language_model.model.layers.33.self_attn.v_proj": {
1082
+ "bits": 16,
1083
+ "act_bits": 16
1084
+ },
1085
+ "language_model.model.layers.33.self_attn.o_proj": {
1086
+ "bits": 16,
1087
+ "act_bits": 16
1088
+ },
1089
+ "language_model.model.layers.33.feed_forward.router": {
1090
+ "bits": 16,
1091
+ "act_bits": 16
1092
+ },
1093
+ "language_model.model.layers.33.feed_forward.shared_expert.gate_proj": {
1094
+ "bits": 16,
1095
+ "act_bits": 16
1096
+ },
1097
+ "language_model.model.layers.33.feed_forward.shared_expert.up_proj": {
1098
+ "bits": 16,
1099
+ "act_bits": 16
1100
+ },
1101
+ "language_model.model.layers.33.feed_forward.shared_expert.down_proj": {
1102
+ "bits": 16,
1103
+ "act_bits": 16
1104
+ },
1105
+ "language_model.model.layers.34.self_attn.q_proj": {
1106
+ "bits": 16,
1107
+ "act_bits": 16
1108
+ },
1109
+ "language_model.model.layers.34.self_attn.k_proj": {
1110
+ "bits": 16,
1111
+ "act_bits": 16
1112
+ },
1113
+ "language_model.model.layers.34.self_attn.v_proj": {
1114
+ "bits": 16,
1115
+ "act_bits": 16
1116
+ },
1117
+ "language_model.model.layers.34.self_attn.o_proj": {
1118
+ "bits": 16,
1119
+ "act_bits": 16
1120
+ },
1121
+ "language_model.model.layers.34.feed_forward.router": {
1122
+ "bits": 16,
1123
+ "act_bits": 16
1124
+ },
1125
+ "language_model.model.layers.34.feed_forward.shared_expert.gate_proj": {
1126
+ "bits": 16,
1127
+ "act_bits": 16
1128
+ },
1129
+ "language_model.model.layers.34.feed_forward.shared_expert.up_proj": {
1130
+ "bits": 16,
1131
+ "act_bits": 16
1132
+ },
1133
+ "language_model.model.layers.34.feed_forward.shared_expert.down_proj": {
1134
+ "bits": 16,
1135
+ "act_bits": 16
1136
+ },
1137
+ "language_model.model.layers.35.self_attn.q_proj": {
1138
+ "bits": 16,
1139
+ "act_bits": 16
1140
+ },
1141
+ "language_model.model.layers.35.self_attn.k_proj": {
1142
+ "bits": 16,
1143
+ "act_bits": 16
1144
+ },
1145
+ "language_model.model.layers.35.self_attn.v_proj": {
1146
+ "bits": 16,
1147
+ "act_bits": 16
1148
+ },
1149
+ "language_model.model.layers.35.self_attn.o_proj": {
1150
+ "bits": 16,
1151
+ "act_bits": 16
1152
+ },
1153
+ "language_model.model.layers.35.feed_forward.router": {
1154
+ "bits": 16,
1155
+ "act_bits": 16
1156
+ },
1157
+ "language_model.model.layers.35.feed_forward.shared_expert.gate_proj": {
1158
+ "bits": 16,
1159
+ "act_bits": 16
1160
+ },
1161
+ "language_model.model.layers.35.feed_forward.shared_expert.up_proj": {
1162
+ "bits": 16,
1163
+ "act_bits": 16
1164
+ },
1165
+ "language_model.model.layers.35.feed_forward.shared_expert.down_proj": {
1166
+ "bits": 16,
1167
+ "act_bits": 16
1168
+ },
1169
+ "language_model.model.layers.36.self_attn.q_proj": {
1170
+ "bits": 16,
1171
+ "act_bits": 16
1172
+ },
1173
+ "language_model.model.layers.36.self_attn.k_proj": {
1174
+ "bits": 16,
1175
+ "act_bits": 16
1176
+ },
1177
+ "language_model.model.layers.36.self_attn.v_proj": {
1178
+ "bits": 16,
1179
+ "act_bits": 16
1180
+ },
1181
+ "language_model.model.layers.36.self_attn.o_proj": {
1182
+ "bits": 16,
1183
+ "act_bits": 16
1184
+ },
1185
+ "language_model.model.layers.36.feed_forward.router": {
1186
+ "bits": 16,
1187
+ "act_bits": 16
1188
+ },
1189
+ "language_model.model.layers.36.feed_forward.shared_expert.gate_proj": {
1190
+ "bits": 16,
1191
+ "act_bits": 16
1192
+ },
1193
+ "language_model.model.layers.36.feed_forward.shared_expert.up_proj": {
1194
+ "bits": 16,
1195
+ "act_bits": 16
1196
+ },
1197
+ "language_model.model.layers.36.feed_forward.shared_expert.down_proj": {
1198
+ "bits": 16,
1199
+ "act_bits": 16
1200
+ },
1201
+ "language_model.model.layers.37.self_attn.q_proj": {
1202
+ "bits": 16,
1203
+ "act_bits": 16
1204
+ },
1205
+ "language_model.model.layers.37.self_attn.k_proj": {
1206
+ "bits": 16,
1207
+ "act_bits": 16
1208
+ },
1209
+ "language_model.model.layers.37.self_attn.v_proj": {
1210
+ "bits": 16,
1211
+ "act_bits": 16
1212
+ },
1213
+ "language_model.model.layers.37.self_attn.o_proj": {
1214
+ "bits": 16,
1215
+ "act_bits": 16
1216
+ },
1217
+ "language_model.model.layers.37.feed_forward.router": {
1218
+ "bits": 16,
1219
+ "act_bits": 16
1220
+ },
1221
+ "language_model.model.layers.37.feed_forward.shared_expert.gate_proj": {
1222
+ "bits": 16,
1223
+ "act_bits": 16
1224
+ },
1225
+ "language_model.model.layers.37.feed_forward.shared_expert.up_proj": {
1226
+ "bits": 16,
1227
+ "act_bits": 16
1228
+ },
1229
+ "language_model.model.layers.37.feed_forward.shared_expert.down_proj": {
1230
+ "bits": 16,
1231
+ "act_bits": 16
1232
+ },
1233
+ "language_model.model.layers.38.self_attn.q_proj": {
1234
+ "bits": 16,
1235
+ "act_bits": 16
1236
+ },
1237
+ "language_model.model.layers.38.self_attn.k_proj": {
1238
+ "bits": 16,
1239
+ "act_bits": 16
1240
+ },
1241
+ "language_model.model.layers.38.self_attn.v_proj": {
1242
+ "bits": 16,
1243
+ "act_bits": 16
1244
+ },
1245
+ "language_model.model.layers.38.self_attn.o_proj": {
1246
+ "bits": 16,
1247
+ "act_bits": 16
1248
+ },
1249
+ "language_model.model.layers.38.feed_forward.router": {
1250
+ "bits": 16,
1251
+ "act_bits": 16
1252
+ },
1253
+ "language_model.model.layers.38.feed_forward.shared_expert.gate_proj": {
1254
+ "bits": 16,
1255
+ "act_bits": 16
1256
+ },
1257
+ "language_model.model.layers.38.feed_forward.shared_expert.up_proj": {
1258
+ "bits": 16,
1259
+ "act_bits": 16
1260
+ },
1261
+ "language_model.model.layers.38.feed_forward.shared_expert.down_proj": {
1262
+ "bits": 16,
1263
+ "act_bits": 16
1264
+ },
1265
+ "language_model.model.layers.39.self_attn.q_proj": {
1266
+ "bits": 16,
1267
+ "act_bits": 16
1268
+ },
1269
+ "language_model.model.layers.39.self_attn.k_proj": {
1270
+ "bits": 16,
1271
+ "act_bits": 16
1272
+ },
1273
+ "language_model.model.layers.39.self_attn.v_proj": {
1274
+ "bits": 16,
1275
+ "act_bits": 16
1276
+ },
1277
+ "language_model.model.layers.39.self_attn.o_proj": {
1278
+ "bits": 16,
1279
+ "act_bits": 16
1280
+ },
1281
+ "language_model.model.layers.39.feed_forward.router": {
1282
+ "bits": 16,
1283
+ "act_bits": 16
1284
+ },
1285
+ "language_model.model.layers.39.feed_forward.shared_expert.gate_proj": {
1286
+ "bits": 16,
1287
+ "act_bits": 16
1288
+ },
1289
+ "language_model.model.layers.39.feed_forward.shared_expert.up_proj": {
1290
+ "bits": 16,
1291
+ "act_bits": 16
1292
+ },
1293
+ "language_model.model.layers.39.feed_forward.shared_expert.down_proj": {
1294
+ "bits": 16,
1295
+ "act_bits": 16
1296
+ },
1297
+ "language_model.model.layers.40.self_attn.q_proj": {
1298
+ "bits": 16,
1299
+ "act_bits": 16
1300
+ },
1301
+ "language_model.model.layers.40.self_attn.k_proj": {
1302
+ "bits": 16,
1303
+ "act_bits": 16
1304
+ },
1305
+ "language_model.model.layers.40.self_attn.v_proj": {
1306
+ "bits": 16,
1307
+ "act_bits": 16
1308
+ },
1309
+ "language_model.model.layers.40.self_attn.o_proj": {
1310
+ "bits": 16,
1311
+ "act_bits": 16
1312
+ },
1313
+ "language_model.model.layers.40.feed_forward.router": {
1314
+ "bits": 16,
1315
+ "act_bits": 16
1316
+ },
1317
+ "language_model.model.layers.40.feed_forward.shared_expert.gate_proj": {
1318
+ "bits": 16,
1319
+ "act_bits": 16
1320
+ },
1321
+ "language_model.model.layers.40.feed_forward.shared_expert.up_proj": {
1322
+ "bits": 16,
1323
+ "act_bits": 16
1324
+ },
1325
+ "language_model.model.layers.40.feed_forward.shared_expert.down_proj": {
1326
+ "bits": 16,
1327
+ "act_bits": 16
1328
+ },
1329
+ "language_model.model.layers.41.self_attn.q_proj": {
1330
+ "bits": 16,
1331
+ "act_bits": 16
1332
+ },
1333
+ "language_model.model.layers.41.self_attn.k_proj": {
1334
+ "bits": 16,
1335
+ "act_bits": 16
1336
+ },
1337
+ "language_model.model.layers.41.self_attn.v_proj": {
1338
+ "bits": 16,
1339
+ "act_bits": 16
1340
+ },
1341
+ "language_model.model.layers.41.self_attn.o_proj": {
1342
+ "bits": 16,
1343
+ "act_bits": 16
1344
+ },
1345
+ "language_model.model.layers.41.feed_forward.router": {
1346
+ "bits": 16,
1347
+ "act_bits": 16
1348
+ },
1349
+ "language_model.model.layers.41.feed_forward.shared_expert.gate_proj": {
1350
+ "bits": 16,
1351
+ "act_bits": 16
1352
+ },
1353
+ "language_model.model.layers.41.feed_forward.shared_expert.up_proj": {
1354
+ "bits": 16,
1355
+ "act_bits": 16
1356
+ },
1357
+ "language_model.model.layers.41.feed_forward.shared_expert.down_proj": {
1358
+ "bits": 16,
1359
+ "act_bits": 16
1360
+ },
1361
+ "language_model.model.layers.42.self_attn.q_proj": {
1362
+ "bits": 16,
1363
+ "act_bits": 16
1364
+ },
1365
+ "language_model.model.layers.42.self_attn.k_proj": {
1366
+ "bits": 16,
1367
+ "act_bits": 16
1368
+ },
1369
+ "language_model.model.layers.42.self_attn.v_proj": {
1370
+ "bits": 16,
1371
+ "act_bits": 16
1372
+ },
1373
+ "language_model.model.layers.42.self_attn.o_proj": {
1374
+ "bits": 16,
1375
+ "act_bits": 16
1376
+ },
1377
+ "language_model.model.layers.42.feed_forward.router": {
1378
+ "bits": 16,
1379
+ "act_bits": 16
1380
+ },
1381
+ "language_model.model.layers.42.feed_forward.shared_expert.gate_proj": {
1382
+ "bits": 16,
1383
+ "act_bits": 16
1384
+ },
1385
+ "language_model.model.layers.42.feed_forward.shared_expert.up_proj": {
1386
+ "bits": 16,
1387
+ "act_bits": 16
1388
+ },
1389
+ "language_model.model.layers.42.feed_forward.shared_expert.down_proj": {
1390
+ "bits": 16,
1391
+ "act_bits": 16
1392
+ },
1393
+ "language_model.model.layers.43.self_attn.q_proj": {
1394
+ "bits": 16,
1395
+ "act_bits": 16
1396
+ },
1397
+ "language_model.model.layers.43.self_attn.k_proj": {
1398
+ "bits": 16,
1399
+ "act_bits": 16
1400
+ },
1401
+ "language_model.model.layers.43.self_attn.v_proj": {
1402
+ "bits": 16,
1403
+ "act_bits": 16
1404
+ },
1405
+ "language_model.model.layers.43.self_attn.o_proj": {
1406
+ "bits": 16,
1407
+ "act_bits": 16
1408
+ },
1409
+ "language_model.model.layers.43.feed_forward.router": {
1410
+ "bits": 16,
1411
+ "act_bits": 16
1412
+ },
1413
+ "language_model.model.layers.43.feed_forward.shared_expert.gate_proj": {
1414
+ "bits": 16,
1415
+ "act_bits": 16
1416
+ },
1417
+ "language_model.model.layers.43.feed_forward.shared_expert.up_proj": {
1418
+ "bits": 16,
1419
+ "act_bits": 16
1420
+ },
1421
+ "language_model.model.layers.43.feed_forward.shared_expert.down_proj": {
1422
+ "bits": 16,
1423
+ "act_bits": 16
1424
+ },
1425
+ "language_model.model.layers.44.self_attn.q_proj": {
1426
+ "bits": 16,
1427
+ "act_bits": 16
1428
+ },
1429
+ "language_model.model.layers.44.self_attn.k_proj": {
1430
+ "bits": 16,
1431
+ "act_bits": 16
1432
+ },
1433
+ "language_model.model.layers.44.self_attn.v_proj": {
1434
+ "bits": 16,
1435
+ "act_bits": 16
1436
+ },
1437
+ "language_model.model.layers.44.self_attn.o_proj": {
1438
+ "bits": 16,
1439
+ "act_bits": 16
1440
+ },
1441
+ "language_model.model.layers.44.feed_forward.router": {
1442
+ "bits": 16,
1443
+ "act_bits": 16
1444
+ },
1445
+ "language_model.model.layers.44.feed_forward.shared_expert.gate_proj": {
1446
+ "bits": 16,
1447
+ "act_bits": 16
1448
+ },
1449
+ "language_model.model.layers.44.feed_forward.shared_expert.up_proj": {
1450
+ "bits": 16,
1451
+ "act_bits": 16
1452
+ },
1453
+ "language_model.model.layers.44.feed_forward.shared_expert.down_proj": {
1454
+ "bits": 16,
1455
+ "act_bits": 16
1456
+ },
1457
+ "language_model.model.layers.45.self_attn.q_proj": {
1458
+ "bits": 16,
1459
+ "act_bits": 16
1460
+ },
1461
+ "language_model.model.layers.45.self_attn.k_proj": {
1462
+ "bits": 16,
1463
+ "act_bits": 16
1464
+ },
1465
+ "language_model.model.layers.45.self_attn.v_proj": {
1466
+ "bits": 16,
1467
+ "act_bits": 16
1468
+ },
1469
+ "language_model.model.layers.45.self_attn.o_proj": {
1470
+ "bits": 16,
1471
+ "act_bits": 16
1472
+ },
1473
+ "language_model.model.layers.45.feed_forward.router": {
1474
+ "bits": 16,
1475
+ "act_bits": 16
1476
+ },
1477
+ "language_model.model.layers.45.feed_forward.shared_expert.gate_proj": {
1478
+ "bits": 16,
1479
+ "act_bits": 16
1480
+ },
1481
+ "language_model.model.layers.45.feed_forward.shared_expert.up_proj": {
1482
+ "bits": 16,
1483
+ "act_bits": 16
1484
+ },
1485
+ "language_model.model.layers.45.feed_forward.shared_expert.down_proj": {
1486
+ "bits": 16,
1487
+ "act_bits": 16
1488
+ },
1489
+ "language_model.model.layers.46.self_attn.q_proj": {
1490
+ "bits": 16,
1491
+ "act_bits": 16
1492
+ },
1493
+ "language_model.model.layers.46.self_attn.k_proj": {
1494
+ "bits": 16,
1495
+ "act_bits": 16
1496
+ },
1497
+ "language_model.model.layers.46.self_attn.v_proj": {
1498
+ "bits": 16,
1499
+ "act_bits": 16
1500
+ },
1501
+ "language_model.model.layers.46.self_attn.o_proj": {
1502
+ "bits": 16,
1503
+ "act_bits": 16
1504
+ },
1505
+ "language_model.model.layers.46.feed_forward.router": {
1506
+ "bits": 16,
1507
+ "act_bits": 16
1508
+ },
1509
+ "language_model.model.layers.46.feed_forward.shared_expert.gate_proj": {
1510
+ "bits": 16,
1511
+ "act_bits": 16
1512
+ },
1513
+ "language_model.model.layers.46.feed_forward.shared_expert.up_proj": {
1514
+ "bits": 16,
1515
+ "act_bits": 16
1516
+ },
1517
+ "language_model.model.layers.46.feed_forward.shared_expert.down_proj": {
1518
+ "bits": 16,
1519
+ "act_bits": 16
1520
+ },
1521
+ "language_model.model.layers.47.self_attn.q_proj": {
1522
+ "bits": 16,
1523
+ "act_bits": 16
1524
+ },
1525
+ "language_model.model.layers.47.self_attn.k_proj": {
1526
+ "bits": 16,
1527
+ "act_bits": 16
1528
+ },
1529
+ "language_model.model.layers.47.self_attn.v_proj": {
1530
+ "bits": 16,
1531
+ "act_bits": 16
1532
+ },
1533
+ "language_model.model.layers.47.self_attn.o_proj": {
1534
+ "bits": 16,
1535
+ "act_bits": 16
1536
+ },
1537
+ "language_model.model.layers.47.feed_forward.router": {
1538
+ "bits": 16,
1539
+ "act_bits": 16
1540
+ },
1541
+ "language_model.model.layers.47.feed_forward.shared_expert.gate_proj": {
1542
+ "bits": 16,
1543
+ "act_bits": 16
1544
+ },
1545
+ "language_model.model.layers.47.feed_forward.shared_expert.up_proj": {
1546
+ "bits": 16,
1547
+ "act_bits": 16
1548
+ },
1549
+ "language_model.model.layers.47.feed_forward.shared_expert.down_proj": {
1550
+ "bits": 16,
1551
+ "act_bits": 16
1552
+ }
1553
+ }
1554
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<|begin_of_text|>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "<|eot|>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "<|finetune_right_pad|>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ }
23
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:172c9eb4beafc72601690da3ccfcede5c2e6806a8d5ec1fca33e22acea8023a4
3
+ size 27948578
tokenizer_config.json ADDED
The diff for this file is too large to render. See raw diff