Update README.md
Browse files
README.md
CHANGED
|
@@ -24,13 +24,13 @@ Trying to get better at medical Q & A
|
|
| 24 |
|
| 25 |
|
| 26 |
- **Developed by:** [Tonic](https://huggingface.co/Tonic)
|
| 27 |
-
- **Shared by
|
| 28 |
- **Model type:** Mistral Fine-Tune
|
| 29 |
- **Language(s) (NLP):** English
|
| 30 |
- **License:** MIT2.0
|
| 31 |
-
- **Finetuned from model
|
| 32 |
|
| 33 |
-
### Model Sources
|
| 34 |
|
| 35 |
|
| 36 |
- **Repository:** [Tonic/mistralmed](https://huggingface.co/Tonic/mistralmed)
|
|
@@ -45,7 +45,7 @@ This model can be used the same way you normally use mistral
|
|
| 45 |
|
| 46 |
This model can do better in medical question and answer scenarios.
|
| 47 |
|
| 48 |
-
### Downstream Use
|
| 49 |
|
| 50 |
This model is intended to be further fine tuned.
|
| 51 |
|
|
@@ -63,18 +63,16 @@ Users (both direct and downstream) should be made aware of the risks, biases and
|
|
| 63 |
|
| 64 |
Use the code below to get started with the model.
|
| 65 |
|
| 66 |
-
[
|
| 67 |
|
| 68 |
```python
|
| 69 |
-
from transformers import AutoTokenizer, MistralForCausalLM
|
| 70 |
-
import torch
|
| 71 |
-
import gradio as gr
|
| 72 |
-
import random
|
| 73 |
-
from textwrap import wrap
|
| 74 |
from transformers import AutoConfig, AutoTokenizer, AutoModelForSeq2SeqLM, AutoModelForCausalLM, MistralForCausalLM
|
| 75 |
from peft import PeftModel, PeftConfig
|
| 76 |
import torch
|
| 77 |
import gradio as gr
|
|
|
|
|
|
|
|
|
|
| 78 |
|
| 79 |
# Functions to Wrap the Prompt Correctly
|
| 80 |
def wrap_text(text, width=90):
|
|
@@ -195,14 +193,16 @@ iface.launch()
|
|
| 195 |
|
| 196 |
### Training Procedure
|
| 197 |
|
|
|
|
| 198 |
Dataset({
|
| 199 |
features: ['qtype', 'Question', 'Answer'],
|
| 200 |
num_rows: 16407
|
| 201 |
})
|
| 202 |
-
|
| 203 |
|
| 204 |
#### Preprocessing [optional]
|
| 205 |
|
|
|
|
| 206 |
MistralForCausalLM(
|
| 207 |
(model): MistralModel(
|
| 208 |
(embed_tokens): Embedding(32000, 4096)
|
|
@@ -229,11 +229,12 @@ MistralForCausalLM(
|
|
| 229 |
)
|
| 230 |
(lm_head): Linear(in_features=4096, out_features=32000, bias=False)
|
| 231 |
)
|
| 232 |
-
|
| 233 |
|
| 234 |
#### Training Hyperparameters
|
| 235 |
|
| 236 |
- **Training regime:**
|
|
|
|
| 237 |
config = LoraConfig(
|
| 238 |
r=8,
|
| 239 |
lora_alpha=16,
|
|
@@ -251,6 +252,7 @@ config = LoraConfig(
|
|
| 251 |
lora_dropout=0.05, # Conventional
|
| 252 |
task_type="CAUSAL_LM",
|
| 253 |
)
|
|
|
|
| 254 |
|
| 255 |
#### Speeds, Sizes, Times [optional]
|
| 256 |
|
|
@@ -260,7 +262,6 @@ config = LoraConfig(
|
|
| 260 |
|
| 261 |
## Environmental Impact
|
| 262 |
|
| 263 |
-
<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
|
| 264 |
|
| 265 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 266 |
|
|
@@ -300,6 +301,7 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
|
|
| 300 |
|
| 301 |
### Model Architecture and Objective
|
| 302 |
|
|
|
|
| 303 |
PeftModelForCausalLM(
|
| 304 |
(base_model): LoraModel(
|
| 305 |
(model): MistralForCausalLM(
|
|
@@ -434,6 +436,8 @@ PeftModelForCausalLM(
|
|
| 434 |
)
|
| 435 |
)
|
| 436 |
)
|
|
|
|
|
|
|
| 437 |
#### Hardware
|
| 438 |
|
| 439 |
A100
|
|
|
|
| 24 |
|
| 25 |
|
| 26 |
- **Developed by:** [Tonic](https://huggingface.co/Tonic)
|
| 27 |
+
- **Shared by :** [Tonic](https://huggingface.co/Tonic)
|
| 28 |
- **Model type:** Mistral Fine-Tune
|
| 29 |
- **Language(s) (NLP):** English
|
| 30 |
- **License:** MIT2.0
|
| 31 |
+
- **Finetuned from model :** [mistralai/Mistral-7B-v0.1](https://huggingface.com/Mistralai/Mistral-7B-v0.1)
|
| 32 |
|
| 33 |
+
### Model Sources
|
| 34 |
|
| 35 |
|
| 36 |
- **Repository:** [Tonic/mistralmed](https://huggingface.co/Tonic/mistralmed)
|
|
|
|
| 45 |
|
| 46 |
This model can do better in medical question and answer scenarios.
|
| 47 |
|
| 48 |
+
### Downstream Use
|
| 49 |
|
| 50 |
This model is intended to be further fine tuned.
|
| 51 |
|
|
|
|
| 63 |
|
| 64 |
Use the code below to get started with the model.
|
| 65 |
|
| 66 |
+
[pseudolab/MistralMED_Chat](https://huggingface.co/spaces/pseudolab/MistralMED_Chat)
|
| 67 |
|
| 68 |
```python
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
from transformers import AutoConfig, AutoTokenizer, AutoModelForSeq2SeqLM, AutoModelForCausalLM, MistralForCausalLM
|
| 70 |
from peft import PeftModel, PeftConfig
|
| 71 |
import torch
|
| 72 |
import gradio as gr
|
| 73 |
+
import random
|
| 74 |
+
from textwrap import wrap
|
| 75 |
+
|
| 76 |
|
| 77 |
# Functions to Wrap the Prompt Correctly
|
| 78 |
def wrap_text(text, width=90):
|
|
|
|
| 193 |
|
| 194 |
### Training Procedure
|
| 195 |
|
| 196 |
+
```json
|
| 197 |
Dataset({
|
| 198 |
features: ['qtype', 'Question', 'Answer'],
|
| 199 |
num_rows: 16407
|
| 200 |
})
|
| 201 |
+
```
|
| 202 |
|
| 203 |
#### Preprocessing [optional]
|
| 204 |
|
| 205 |
+
```json
|
| 206 |
MistralForCausalLM(
|
| 207 |
(model): MistralModel(
|
| 208 |
(embed_tokens): Embedding(32000, 4096)
|
|
|
|
| 229 |
)
|
| 230 |
(lm_head): Linear(in_features=4096, out_features=32000, bias=False)
|
| 231 |
)
|
| 232 |
+
```
|
| 233 |
|
| 234 |
#### Training Hyperparameters
|
| 235 |
|
| 236 |
- **Training regime:**
|
| 237 |
+
```json
|
| 238 |
config = LoraConfig(
|
| 239 |
r=8,
|
| 240 |
lora_alpha=16,
|
|
|
|
| 252 |
lora_dropout=0.05, # Conventional
|
| 253 |
task_type="CAUSAL_LM",
|
| 254 |
)
|
| 255 |
+
```
|
| 256 |
|
| 257 |
#### Speeds, Sizes, Times [optional]
|
| 258 |
|
|
|
|
| 262 |
|
| 263 |
## Environmental Impact
|
| 264 |
|
|
|
|
| 265 |
|
| 266 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
| 267 |
|
|
|
|
| 301 |
|
| 302 |
### Model Architecture and Objective
|
| 303 |
|
| 304 |
+
```json
|
| 305 |
PeftModelForCausalLM(
|
| 306 |
(base_model): LoraModel(
|
| 307 |
(model): MistralForCausalLM(
|
|
|
|
| 436 |
)
|
| 437 |
)
|
| 438 |
)
|
| 439 |
+
```
|
| 440 |
+
|
| 441 |
#### Hardware
|
| 442 |
|
| 443 |
A100
|