Update README.md

README.md

@@ -1,18 +1,40 @@
 ---
 license: mit
+datasets:
+- locuslab/TOFU
+language:
+- en
+base_model:
+- OPTML-Group/TOFU-origin-Llama-2-7b-chat
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- unlearn
+- machine-unlearning
+- llm-unlearning
+- data-privacy
+- large-language-models
+- trustworthy-ai
+- trustworthy-machine-learning
+- language-model
 ---
 
-# LLaMA-2-chat 7B unlearned using SimNPO on TOFU Forget10
+# SimNPO-Unlearned Model on Task "TOFU - Forget05"
 
 ## Model Details
 
-- **Base Model**: LLaMA-2-chat 7B
-- **Training**: Fine-tuned on TOFU dataset
-- **Unlearning**: SimNPO on TOFU Forget10
+- **Unlearning**:
+  - **Task**: [🤗datasets/locuslab/TOFU - Forget10](https://huggingface.co/datasets/locuslab/TOFU)
+  - **Method**: [SimNPO](https://arxiv.org/abs/2410.07163)
+- **Origin Model**: [🤗OPTML-Group/TOFU-origin-Llama-2-7b-chat](https://huggingface.co/OPTML-Group/TOFU-origin-Llama-2-7b-chat)
+- **Code Base**: [github.com/OPTML-Group/Unlearn-Simple](https://github.com/OPTML-Group/Unlearn-Simple)
+- **Research Paper**: ["Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning"](https://arxiv.org/abs/2410.07163)
 
 ## Unlearning Algorithm
 
-This model uses the `SimNPO` unlearning algorithm with the following parameters:
+This model uses the `SimNPO` unlearning algorithm with the following optimization objective:
+$$\ell_{SimNPO}(\mathbf{\theta}) = \mathbb{E}_{(x, y) \in \mathcal{D}_f}\left[-\frac{2}{\beta}\log\sigma\left(-\frac{\beta}{|y|}\log\pi_{\mathbf{\theta}}(y|x) - \gamma\right)\right] + \lambda \mathbb{E}_{(x, y) \in \mathcal{D}_r}[-\log\pi_{\mathbf{\theta}} (y|x)]$$
+Unlearning hyper-parameters:
 - Learning Rate: `1e-5`
 - beta: `4.5`
 - lambda: `0.125`
@@ -27,21 +49,26 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
 model = AutoModelForCausalLM.from_pretrained("OPTML-Group/SimNPO-TOFU-forget10-Llama-2-7b-chat", use_flash_attention_2=True, torch_dtype=torch.bfloat16, trust_remote_code=True)
 ```
 
+## Evaluation Results
+||Forgeting Quality (FQ)|Model Utility (MU)|
+|---|---|---|
+|Origin|0.00|0.62|
+|Retrain|1.00|0.62|
+|NPO|0.29|0.55|
+|**SimNPO**|0.45|0.62|
+
 ## Citation
 
 If you use this model in your research, please cite: 
 ```
-@misc{fan2024simplicityprevailsrethinkingnegative,
-      title={Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning}, 
-      author={Chongyu Fan and Jiancheng Liu and Licong Lin and Jinghan Jia and Ruiqi Zhang and Song Mei and Sijia Liu},
-      year={2024},
-      eprint={2410.07163},
-      archivePrefix={arXiv},
-      primaryClass={cs.CL},
-      url={https://arxiv.org/abs/2410.07163}, 
+@article{fan2024simplicity,
+  title={Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning},
+  author={Fan, Chongyu and Liu, Jiancheng and Lin, Licong and Jia, Jinghan and Zhang, Ruiqi and Mei, Song and Liu, Sijia},
+  journal={arXiv preprint arXiv:2410.07163},
+  year={2024}
 }
 ```
 
-## Contact
+## Reporting Issues
 
-For questions or issues regarding this model, please contact chongyu.fan93@gmail.com.
+Reporting issues with the model: [github.com/OPTML-Group/Unlearn-Simple](https://github.com/OPTML-Group/Unlearn-Simple)