Update README.md
Browse files
README.md
CHANGED
|
@@ -39,11 +39,11 @@ Built on **Phi-3.5 Vision-Instruct (7 B)**, it introduces **Direct Preference Op
|
|
| 39 |
```python
|
| 40 |
from transformers import AutoModelForVision2Seq, AutoProcessor
|
| 41 |
|
| 42 |
-
model_id = "
|
| 43 |
model = AutoModelForVision2Seq.from_pretrained(model_id)
|
| 44 |
processor = AutoProcessor.from_pretrained(model_id)
|
| 45 |
|
| 46 |
-
inputs = processor(
|
| 47 |
images="example_frontal.jpg",
|
| 48 |
text="Question: What abnormalities are present?\nAnswer:",
|
| 49 |
return_tensors="pt"
|
|
@@ -54,17 +54,17 @@ print(processor.decode(generated[0], skip_special_tokens=True))
|
|
| 54 |
````
|
| 55 |
|
| 56 |
> **Dependencies** `pip install transformers>=4.41.0 timm accelerate`
|
| 57 |
-
> For batch inference or a Streamlit demo, see the scripts in the [GitHub repo](https://
|
| 58 |
|
| 59 |
---
|
| 60 |
|
| 61 |
## Available Checkpoints
|
| 62 |
|
| 63 |
-
| HF Repo
|
| 64 |
-
|
|
| 65 |
-
| [`CheX-Phi3.5-vision-instruct-DPO`](https://huggingface.co/
|
| 66 |
-
| [`CheX-Phi3.5-vision-instruct-SFT`](https://huggingface.co/
|
| 67 |
-
| [`Phi-3.5-vision-instruct`](https://huggingface.co/
|
| 68 |
|
| 69 |
---
|
| 70 |
|
|
@@ -112,7 +112,7 @@ Evaluation scripts are provided in `stage3_evaluate_mimic_ext_vqa.sh`.
|
|
| 112 |
|
| 113 |
## Resources
|
| 114 |
|
| 115 |
-
* **Code & training scripts** — [https://
|
| 116 |
* **Data utilities** — `tools/generate_visual_prompt.py`
|
| 117 |
* **Demo notebook** — `demo.py`
|
| 118 |
|
|
@@ -125,7 +125,7 @@ Evaluation scripts are provided in `stage3_evaluate_mimic_ext_vqa.sh`.
|
|
| 125 |
title = {CheX-Phi3.5V: Preference-Optimised Vision-Language Model for Chest X-ray Interpretation},
|
| 126 |
author = {Liu, Xiao and Others},
|
| 127 |
year = {2025},
|
| 128 |
-
howpublished = {\url{https://huggingface.co/
|
| 129 |
}
|
| 130 |
```
|
| 131 |
|
|
@@ -133,4 +133,3 @@ Evaluation scripts are provided in `stage3_evaluate_mimic_ext_vqa.sh`.
|
|
| 133 |
|
| 134 |
|
| 135 |
|
| 136 |
-
|
|
|
|
| 39 |
```python
|
| 40 |
from transformers import AutoModelForVision2Seq, AutoProcessor
|
| 41 |
|
| 42 |
+
model_id = "CheX-Phi-3.5-vision-instruct-DPO"
|
| 43 |
model = AutoModelForVision2Seq.from_pretrained(model_id)
|
| 44 |
processor = AutoProcessor.from_pretrained(model_id)
|
| 45 |
|
| 46 |
+
inputs = processor(remove4a
|
| 47 |
images="example_frontal.jpg",
|
| 48 |
text="Question: What abnormalities are present?\nAnswer:",
|
| 49 |
return_tensors="pt"
|
|
|
|
| 54 |
````
|
| 55 |
|
| 56 |
> **Dependencies** `pip install transformers>=4.41.0 timm accelerate`
|
| 57 |
+
> For batch inference or a Streamlit demo, see the scripts in the [GitHub repo](https://huggingface.co/HaiZhiYan/CheX-Phi35V).
|
| 58 |
|
| 59 |
---
|
| 60 |
|
| 61 |
## Available Checkpoints
|
| 62 |
|
| 63 |
+
| HF Repo | Stage | Recommended Use |
|
| 64 |
+
| ------------------------------------------------------------ | ------------- | ------------------------- |
|
| 65 |
+
| [`CheX-Phi3.5-vision-instruct-DPO`](https://huggingface.co/HaiZhiYan/CheX-Phi35V/tree/main/CheX-Phi-3.5-vision-instruct-DPO) | **DPO** | Production / evaluation |
|
| 66 |
+
| [`CheX-Phi3.5-vision-instruct-SFT`](https://huggingface.co/HaiZhiYan/CheX-Phi35V/tree/main/CheX-Phi-3.5-vision-instruct-SFT) | SFT (phase 2) | Further preference tuning |
|
| 67 |
+
| [`Phi-3.5-vision-instruct`](https://huggingface.co/HaiZhiYan/CheX-Phi35V/tree/main/Phi-3.5-vision-instruct) | Base | Custom fine-tuning |
|
| 68 |
|
| 69 |
---
|
| 70 |
|
|
|
|
| 112 |
|
| 113 |
## Resources
|
| 114 |
|
| 115 |
+
* **Code & training scripts** — [https://huggingface.co/HaiZhiYan/CheX-Phi35V](https://huggingface.co/HaiZhiYan/CheX-Phi35V)
|
| 116 |
* **Data utilities** — `tools/generate_visual_prompt.py`
|
| 117 |
* **Demo notebook** — `demo.py`
|
| 118 |
|
|
|
|
| 125 |
title = {CheX-Phi3.5V: Preference-Optimised Vision-Language Model for Chest X-ray Interpretation},
|
| 126 |
author = {Liu, Xiao and Others},
|
| 127 |
year = {2025},
|
| 128 |
+
howpublished = {\url{https://huggingface.co/HaiZhiYan/CheX-Phi35V/tree/main/CheX-Phi-3.5-vision-instruct-DPO}}
|
| 129 |
}
|
| 130 |
```
|
| 131 |
|
|
|
|
| 133 |
|
| 134 |
|
| 135 |
|
|
|