Upload folder using huggingface_hub

README.md

@@ -1,12 +1,18 @@
 ---
-base_model:
-- google/medgemma-4b-it
-paper: 2507.05201
-library_name: transformers
 license: other
 license_name: health-ai-developer-foundations
 license_link: https://developers.google.com/health-ai-developer-foundations/terms
+library_name: transformers
 pipeline_tag: image-text-to-text
+extra_gated_heading: Access MedGemma on Hugging Face
+extra_gated_prompt: >-
+  To access MedGemma on Hugging Face, you're required to review and
+  agree to [Health AI Developer Foundation's terms of use](https://developers.google.com/health-ai-developer-foundations/terms).
+  To do this, please ensure you're logged in to Hugging Face and click below.
+  Requests are processed immediately.
+extra_gated_button_content: Acknowledge license
+base_model:
+- google/medgemma-4b-it
 tags:
 - medical
 - unsloth
@@ -16,14 +22,7 @@ tags:
 - pathology
 - ophthalmology
 - chest-x-ray
-extra_gated_heading: Access MedGemma on Hugging Face
-extra_gated_prompt: To access MedGemma on Hugging Face, you're required to review
-  and agree to [Health AI Developer Foundation's terms of use](https://developers.google.com/health-ai-developer-foundations/terms).
-  To do this, please ensure you're logged in to Hugging Face and click below. Requests
-  are processed immediately.
-extra_gated_button_content: Acknowledge license
 ---
-
 <div>
 <p style="margin-top: 0;margin-bottom: 0;">
     <em><a href="https://docs.unsloth.ai/basics/unsloth-dynamic-v2.0-gguf">Unsloth Dynamic 2.0</a> achieves superior accuracy & outperforms other leading quants.</em>
@@ -236,7 +235,7 @@ See the following Colab notebooks for examples of how to use MedGemma:
 ### Model architecture overview
 
 The MedGemma model is built based on [Gemma 3](https://ai.google.dev/gemma/) and
-uses the same decoder-only transformer architecture as Gemma 3. To read more
+uses the same decoder-only transformer architecture as Gemma 3\. To read more
 about the architecture, consult the Gemma 3 [model
 card](https://ai.google.dev/gemma/docs/core/model_card_3).
 
@@ -301,17 +300,17 @@ health benchmarks.
 | Task and metric | Gemma 3 4B | MedGemma 4B |
 | :---- | :---- | :---- |
 | **Medical image classification** |  |  |
-| MIMIC CXR\*\* - macro F1 for top 5 conditions | 81.2 | 88.9 |
-| CheXpert CXR - macro F1 for top 5 conditions | 32.6 | 48.1 |
-| CXR14 - macro F1 for 3 conditions | 32.0 | 50.1 |
-| PathMCQA\* (histopathology, internal\*\*)  - Accuracy | 37.1 | 69.8 |
-| US-DermMCQA\* - Accuracy | 52.5 | 71.8 |
-| EyePACS\* (fundus, internal) - Accuracy | 14.4 | 64.9 |
+| MIMIC CXR\*\* \- macro F1 for top 5 conditions | 81.2 | 88.9 |
+| CheXpert CXR \- macro F1 for top 5 conditions | 32.6 | 48.1 |
+| CXR14 \- macro F1 for 3 conditions | 32.0 | 50.1 |
+| PathMCQA\* (histopathology, internal\*\*)  \- Accuracy | 37.1 | 69.8 |
+| US-DermMCQA\* \- Accuracy | 52.5 | 71.8 |
+| EyePACS\* (fundus, internal) \- Accuracy | 14.4 | 64.9 |
 | **Visual question answering** |  |  |
-| SLAKE (radiology) - Tokenized F1 | 40.2 | 72.3 |
-| VQA-RAD\*\*\* (radiology) - Tokenized F1 | 33.6 | 49.9 |
+| SLAKE (radiology) \- Tokenized F1 | 40.2 | 72.3 |
+| VQA-RAD\*\*\* (radiology) \- Tokenized F1 | 33.6 | 49.9 |
 | **Knowledge and reasoning** |  |  |  |  |
-| MedXpertQA (text + multimodal questions) - Accuracy | 16.4 | 18.8 |
+| MedXpertQA (text \+ multimodal questions) \- Accuracy | 16.4 | 18.8 |
 
 *Internal datasets. US-DermMCQA is described in [Liu (2020, Nature
 medicine)](https://www.nature.com/articles/s41591-020-0842-3), presented as a
@@ -338,7 +337,7 @@ pre-trained checkpoint with our previous best model for CXR report generation,
 
 | Metric | MedGemma 4B (pre-trained) | MedGemma 4B (tuned for CXR)| PaliGemma 2 3B (tuned for CXR) | PaliGemma 2 10B (tuned for CXR) |
 | :---- | :---- | :---- | :---- | :---- |
-| MIMIC CXR - RadGraph F1 | 29.5 | 30.3 |28.8 | 29.5 |
+| MIMIC CXR \- RadGraph F1 | 29.5 | 30.3 |28.8 | 29.5 |
 
 
 
@@ -380,6 +379,69 @@ Gemma models.
 | EHRQA | 70.9 | 67.6 |
 
 
+### Ethics and safety evaluation
+
+#### Evaluation approach
+
+Our evaluation methods include structured evaluations and internal red-teaming
+testing of relevant content policies. Red-teaming was conducted by a number of
+different teams, each with different goals and human evaluation metrics. These
+models were evaluated against a number of different categories relevant to
+ethics and safety, including:
+
+*   **Child safety**: Evaluation of text-to-text and image-to-text prompts
+    covering child safety policies, including child sexual abuse and
+    exploitation.
+*   **Content safety:** Evaluation of text-to-text and image-to-text prompts
+    covering safety policies, including harassment, violence and gore, and hate
+    speech.
+*   **Representational harms**: Evaluation of text-to-text and image-to-text
+    prompts covering safety policies, including bias, stereotyping, and harmful
+    associations or inaccuracies.
+*   **General medical harms:** Evaluation of text-to-text and image-to-text
+    prompts covering safety policies, including information quality and harmful
+    associations or inaccuracies.
+
+In addition to development level evaluations, we conduct "assurance evaluations"
+which are our "arms-length" internal evaluations for responsibility governance
+decision making. They are conducted separately from the model development team,
+to inform decision making about release. High-level findings are fed back to the
+model team, but prompt sets are held out to prevent overfitting and preserve the
+results' ability to inform decision making. Notable assurance evaluation results
+are reported to our Responsibility & Safety Council as part of release review.
+
+#### Evaluation results
+
+For all areas of safety testing, we saw safe levels of performance across the
+categories of child safety, content safety, and representational harms. All
+testing was conducted without safety filters to evaluate the model capabilities
+and behaviors. For text-to-text, image-to-text, and audio-to-text, and across
+both MedGemma model sizes, the model produced minimal policy violations. A
+limitation of our evaluations was that they included primarily English language
+prompts.
+
+## Data card
+
+### Dataset overview
+
+#### Training
+
+The base Gemma models are pre-trained on a large corpus of text and code data.
+MedGemma 4B utilizes a [SigLIP](https://arxiv.org/abs/2303.15343) image encoder
+that has been specifically pre-trained on a variety of de-identified medical
+data, including radiology images, histopathology images, ophthalmology images,
+and dermatology images. Its LLM component is trained on a diverse set of medical
+data, including medical text relevant to radiology images, chest-x rays,
+histopathology patches, ophthalmology images and dermatology images.
+
+#### Evaluation
+
+MedGemma models have been evaluated on a comprehensive set of clinically
+relevant benchmarks, including over 22 datasets across 5 different tasks and 6
+medical image modalities. These include both open benchmark datasets and curated
+datasets, with a focus on expert human evaluations for tasks like CXR report
+generation and radiology VQA.
+
 ### Ethics and safety evaluation
 
 #### Evaluation approach
@@ -524,19 +586,19 @@ consented participants.
     clinical and dermatoscopic) from Australia.
 *   **Dermatology dataset 3:** De-identified dataset of non-diseased skin images
     from an internal data collection effort.
-*   **Pathology dataset 1:** De-identified dataset of histopathology H&E whole
+*   **Pathology dataset 1:** De-identified dataset of histopathology H\&E whole
     slide images created in collaboration with an academic research hospital and
     biobank in Europe. Comprises de-identified colon, prostate, and lymph nodes.
-*   **Pathology dataset 2:** De-identified dataset of lung histopathology H&E
+*   **Pathology dataset 2:** De-identified dataset of lung histopathology H\&E
     and IHC whole slide images created by a commercial biobank in the United
     States.
 *   **Pathology dataset 3:** De-identified dataset of prostate and lymph node
-    H&E and IHC histopathology whole slide images created by a contract
+    H\&E and IHC histopathology whole slide images created by a contract
     research organization in the United States.
 *   **Pathology dataset 4:** De-identified dataset of histopathology whole slide
     images created in collaboration with a large, tertiary teaching hospital in
     the United States. Comprises a diverse set of tissue and stain types,
-    predominantly H&E.
+    predominantly H\&E.
 *   **EHR dataset 1:** Question/answer dataset drawn from synthetic FHIR records
     created by [Synthea.](https://synthetichealth.github.io/synthea/) The test
     set includes 19 unique patients with 200 questions per patient divided into
@@ -549,7 +611,7 @@ consented participants.
     [https://physionet.org/content/mimic-cxr/2.1.0/](https://physionet.org/content/mimic-cxr/2.1.0/)
     *and* Johnson, Alistair E. W., Tom J. Pollard, Seth J. Berkowitz, Nathaniel
     R. Greenbaum, Matthew P. Lungren, Chih-Ying Deng, Roger G. Mark, and Steven
-    Horng. 2019. "MIMIC-CXR, a de-Identified Publicly Available Database of
+    Horng. 2019\. "MIMIC-CXR, a de-Identified Publicly Available Database of
     Chest Radiographs with Free-Text Reports." *Scientific Data 6* (1): 1–8.
 
 *   **SLAKE:** Liu, Bo, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu.
@@ -559,10 +621,10 @@ consented participants.
 
 *   **PAD-UEFS-20:** Pacheco, Andre GC, et al. "PAD-UFES-20: A skin lesion
     dataset composed of patient data and clinical images collected from
-    smartphones." *Data in brief* 32 (2020): 106221.
+    smartphones." *Data in brief* 32 (2020): 106221\.
 
 *   **SCIN:** Ward, Abbi, Jimmy Li, Julie Wang, Sriram Lakshminarasimhan, Ashley
-    Carrick, Bilson Campana, Jay Hartford, et al. 2024. "Creating an Empirical
+    Carrick, Bilson Campana, Jay Hartford, et al. 2024\. "Creating an Empirical
     Dermatology Dataset Through Crowdsourcing With Web Search Advertisements."
     *JAMA Network Open 7* (11): e2446615–e2446615.
 
@@ -572,7 +634,7 @@ consented participants.
 
 *   **CAMELYON16:** Ehteshami Bejnordi, Babak, Mitko Veta, Paul Johannes van
     Diest, Bram van Ginneken, Nico Karssemeijer, Geert Litjens, Jeroen A. W. M.
-    van der Laak, et al. 2017. "Diagnostic Assessment of Deep Learning
+    van der Laak, et al. 2017\. "Diagnostic Assessment of Deep Learning
     Algorithms for Detection of Lymph Node Metastases in Women With Breast
     Cancer." *JAMA 318* (22): 2199–2210.
 
@@ -581,22 +643,22 @@ consented participants.
     10.17632/t9ndx37v5h.1
 
 *   **VQA-RAD:** Lau, Jason J., Soumya Gayen, Asma Ben Abacha, and Dina
-    Demner-Fushman. 2018. "A Dataset of Clinically Generated Visual Questions
+    Demner-Fushman. 2018\. "A Dataset of Clinically Generated Visual Questions
     and Answers about Radiology Images." *Scientific Data 5* (1): 1–10.
 
 *   **Chest ImaGenome:** Wu, J., Agu, N., Lourentzou, I., Sharma, A., Paguio,
     J., Yao, J. S., Dee, E. C., Mitchell, W., Kashyap, S., Giovannini, A., Celi,
     L. A., Syeda-Mahmood, T., & Moradi, M. (2021). Chest ImaGenome Dataset
-    (version 1.0.0). PhysioNet. RRID:SCR_007345.
+    (version 1.0.0). PhysioNet. RRID:SCR\_007345.
     [https://doi.org/10.13026/wv01-y230](https://doi.org/10.13026/wv01-y230)
 
 *   **MedQA:** Jin, Di, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang,
-    and Peter Szolovits. 2020. "What Disease Does This Patient Have? A
+    and Peter Szolovits. 2020\. "What Disease Does This Patient Have? A
     Large-Scale Open Domain Question Answering Dataset from Medical Exams."
     [http://arxiv.org/abs/2009.13081](http://arxiv.org/abs/2009.13081).
 
 *   **AfrimedQA:** Olatunji, Tobi, Charles Nimo, Abraham Owodunni, Tassallah
-    Abdullahi, Emmanuel Ayodele, Mardhiyah Sanni, Chinemelu Aka, et al. 2024.
+    Abdullahi, Emmanuel Ayodele, Mardhiyah Sanni, Chinemelu Aka, et al. 2024\.
     "AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering
     Benchmark Dataset."
     [http://arxiv.org/abs/2411.15640](http://arxiv.org/abs/2411.15640).
@@ -607,46 +669,10 @@ consented participants.
     [https://arxiv.org/abs/2404.05590](https://arxiv.org/abs/2404.05590)
 
 *   **MedXpertQA:** Zuo, Yuxin, Shang Qu, Yifei Li, Zhangren Chen, Xuekai Zhu,
-    Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. 2025. "MedXpertQA:
+    Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. 2025\. "MedXpertQA:
     Benchmarking Expert-Level Medical Reasoning and Understanding."
     [http://arxiv.org/abs/2501.18362](http://arxiv.org/abs/2501.18362).
 
-*   **HealthSearchQA:** This dataset consists of consisting of 3,173 commonly searched consumer
-    questions
-
-In addition to the public datasets listed above, MedGemma was also trained on
-de-identified, licensed datasets or datasets collected internally at Google from
-consented participants.
-
-*   **Radiology dataset 1:** De-identified dataset of different CT studies
-    across body parts from a US-based radiology outpatient diagnostic center
-    network.
-*   **Ophthalmology dataset 1 (EyePACS):** De-identified dataset of fundus
-    images from diabetic retinopathy screening.
-*   **Dermatology dataset 1:** De-identified dataset of teledermatology skin
-    condition images (both clinical and dermatoscopic) from Colombia.
-*   **Dermatology dataset 2:** De-identified dataset of skin cancer images (both
-    clinical and dermatoscopic) from Australia.
-*   **Dermatology dataset 3:** De-identified dataset of non-diseased skin images
-    from an internal data collection effort.
-*   **Pathology dataset 1:** De-identified dataset of histopathology H&E whole
-    slide images created in collaboration with an academic research hospital and
-    biobank in Europe. Comprises de-identified colon, prostate, and lymph nodes.
-*   **Pathology dataset 2:** De-identified dataset of lung histopathology H&E
-    and IHC whole slide images created by a commercial biobank in the United
-    States.
-*   **Pathology dataset 3:** De-identified dataset of prostate and lymph node
-    H&E and IHC histopathology whole slide images created by a contract
-    research organization in the United States.
-*   **Pathology dataset 4:** De-identified dataset of histopathology whole slide
-    images created in collaboration with a large, tertiary teaching hospital in
-    the United States. Comprises a diverse set of tissue and stain types,
-    predominantly H&E.
-*   **EHR dataset 1:** Question/answer dataset drawn from synthetic FHIR records
-    created by [Synthea.](https://synthetichealth.github.io/synthea/) The test
-    set includes 19 unique patients with 200 questions per patient divided into
-    10 different categories.
-
 ### De-identification/anonymization:
 
 Google and its partners utilize datasets that have been rigorously anonymized or
@@ -716,7 +742,7 @@ of multiple images.
 MedGemma has not been evaluated or optimized for multi-turn applications.
 
 MedGemma's training may make it more sensitive to the specific prompt used than
-Gemma 3.
+Gemma 3\.
 
 When adapting MedGemma developer should consider the following: