Update README.md

README.md

@@ -50,7 +50,7 @@ for large language model in regard to low resourced morphologically rich African
 
 <!-- Provide the basic links for the model. -->
 
-- **Repository:** [More Information Needed]
+- **Repository:** https://github.com/PKhoboko/MSc-Thesis
 - **Paper [optional]:** https://www.sciencedirect.com/science/article/pii/S2666827025000325
 - **Demo [optional]:** [More Information Needed]
 
@@ -135,7 +135,29 @@ translator("Translate to Zulu: The cow is eating grass.")
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+- **Training regime:** 
+- peft_config = LoraConfig(
+        lora_alpha=16,
+        lora_dropout=0.05,
+        r=16,
+        bias="none",
+        task_type="CAUSAL_LM",
+        target_modules=['k_proj', 'q_proj', 'v_proj', 'o_proj','gate_proj', 'down_proj', 'up_proj']
+)
+
+ - TrainingArguments(
+        optim="paged_adamw_8bit",
+        per_device_train_batch_size=32,
+        gradient_accumulation_steps=4,
+        log_level="debug",
+        save_steps=400,
+        logging_steps=10,
+        learning_rate=4e-4,
+        num_train_epochs=2,
+        warmup_steps=100,
+        lr_scheduler_type="linear",
+)
+
 
 #### Speeds, Sizes, Times [optional]