README.md

---
license: mit
---
### Model Card: **TinyLlama-1.1B-Chat-v1.0-Unfiltered**

---

**Model Name**: TinyLlama-1.1B-Chat-v1.0-Unfiltered  
**Model Type**: Conversational AI Model  
**Architecture**: Based on a 1.1B parameter TinyLlama architecture  

**Training Data**:  
- Fine-tuned on the "dan_remixed" dataset (2.7MB).  
- The dataset improves spelling, grammar, and consistency while replacing references to violent crimes with non-violent activities and removes self-censorship from explicatives.

**Training Time**: Approximately 30-45 minutes. Each validation epoch takes ~322 seconds.  
**Hardware**: Trained on Google Colab Pro A100 GPU (40GB).

---

**Training Performance**:
- **Epoch Losses**:
  - Epoch 1: 0.7209
  - Epoch 2: 0.4441
  - Epoch 3: 0.3683
  - Epoch 4: 0.3358
  - Epoch 5: 0.3145
- **Final Training Loss (Epoch 5)**: 0.3145

---

**Validation Performance** (5 Epochs):  
- **Epoch 1**:  
  - Training Loss: 0.2921  
  - Validation Loss: 0.7962  
  - Perplexity: 2.22  
  - Epoch completed in 321.64 seconds  

- **Epoch 2**:  
  - Training Loss: 0.2872  
  - Validation Loss: 0.7672  
  - Perplexity: 2.15  
  - Epoch completed in 321.91 seconds  

- **Epoch 3**:  
  - Training Loss: 0.2874  
  - Validation Loss: 0.7821  
  - Perplexity: 2.19  
  - Epoch completed in 321.94 seconds  

- **Epoch 4**:  
  - Training Loss: 0.2864  
  - Validation Loss: 0.7796  
  - Perplexity: 2.18  
  - Epoch completed in 322.01 seconds  

-
 **Epoch 5**:  
  - Training Loss: 0.2831  
  - Validation Loss: 0.8017  
  - Perplexity: 2.23  
  - Epoch completed in 322.01 seconds

---

**Optimizer**: AdamW, learning rate: 1e-5  
**Loss Function**: Cross-Entropy Loss, ignoring padding tokens (ignore_index=-100)  
**Use Case**: Conversational AI designed for general, unrestricted conversation, with no filtering on the nature of responses, provided the content is non-violent.

---

**Limitations**:
- Due to the small fine-tuning dataset size (2.7MB), the model may be prone to **overfitting** and **bias**.
- The dataset has been modified to avoid violent language, but the model might still exhibit strong or explicit responses.

**Metrics**:
- Loss and perplexity have been tracked, and more conversational metrics (like BLEU, ROUGE, or human evaluation) could be explored.