uukuguy commited on
Commit
1650dc4
·
1 Parent(s): def312d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +39 -28
README.md CHANGED
@@ -23,7 +23,7 @@ model-index:
23
  metrics:
24
  - name: pass@1
25
  type: pass@1
26
- value: 0.0
27
  verified: false
28
  ---
29
 
@@ -38,6 +38,44 @@ Total 153,013 samples.
38
  - WizardLM/WizardLM_evol_instruct_V2_196k: Coding coversation part. 30,185 samples
39
 
40
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
41
  | | |
42
  |------ | ------ |
43
  | lr | 2e-4 |
@@ -73,33 +111,6 @@ A40-48G x 2
73
  | eeval_samples_per_second | 5.525 |
74
  | eeval_steps_per_second | 2.763 |
75
 
76
- | Metric | Value |
77
- | --- | --- |
78
- | humaneval-python | |
79
-
80
- [Big Code Models Leaderboard](https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard)
81
-
82
- CodeLlama-34B-Python: 53.29
83
-
84
- CodeLlama-34B-Instruct: 50.79
85
-
86
- CodeLlama-13B-Instruct: 50.6
87
-
88
- CodeLlama-34B: 45.11
89
-
90
- CodeLlama-13B-Python: 42.89
91
-
92
- CodeLlama-13B: 35.07
93
-
94
- [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
95
- | Metric | Value |
96
- | --- | --- |
97
- | ARC | |
98
- | HellaSwag | |
99
- | MMLU | |
100
- | TruthfulQA | |
101
- | Average | |
102
-
103
 
104
  # **Code Llama**
105
 
 
23
  metrics:
24
  - name: pass@1
25
  type: pass@1
26
+ value: 75.61
27
  verified: false
28
  ---
29
 
 
38
  - WizardLM/WizardLM_evol_instruct_V2_196k: Coding coversation part. 30,185 samples
39
 
40
 
41
+
42
+ ## HumanEval
43
+
44
+ | human-eval | pass@1 |
45
+ | --- | --- |
46
+ | humaneval-python | 75.61 |
47
+
48
+ [Big Code Models Leaderboard](https://huggingface.co/spaces/bigcode/bigcode-models-leaderboard)
49
+
50
+ | Models | pass@1 |
51
+ |------ | ------ |
52
+ | Phind-CodeLlama-34B-v2| 71.95|
53
+ | WizardCoder-Python-34B-V1.0| 70.73|
54
+ | Phind-CodeLlama-34B-Python-v1| 70.22|
55
+ | Phind-CodeLlama-34B-v1| 65.85|
56
+ | WizardCoder-Python-13B-V1.0| 62.19|
57
+ | WizardCoder-15B-V1.0| 58.12|
58
+ | CodeLlama-34B-Python| 53.29|
59
+ | CodeLlama-34B-Instruct| 50.79|
60
+ | CodeLlama-13B-Instruct| 50.6|
61
+ | CodeLlama-34B| 45.11|
62
+ | CodeLlama-13B-Python| 42.89|
63
+ | CodeLlama-13B| 35.07|
64
+
65
+
66
+ ## lm-evaluation-harness
67
+
68
+ [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
69
+ | Metric | Value |
70
+ | --- | --- |
71
+ | ARC | |
72
+ | HellaSwag | |
73
+ | MMLU | |
74
+ | TruthfulQA | |
75
+ | Average | |
76
+
77
+
78
+ ## Training Arguments
79
  | | |
80
  |------ | ------ |
81
  | lr | 2e-4 |
 
111
  | eeval_samples_per_second | 5.525 |
112
  | eeval_steps_per_second | 2.763 |
113
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114
 
115
  # **Code Llama**
116