|
--- |
|
license: creativeml-openrail-m |
|
datasets: |
|
- amphora/QwQ-LongCoT-130K |
|
language: |
|
- en |
|
base_model: |
|
- Qwen/Qwen2.5-3B-Instruct |
|
pipeline_tag: text-generation |
|
library_name: transformers |
|
tags: |
|
- text-generation-inference |
|
- long-CoT |
|
- safetensors |
|
- 3B |
|
- Instruct |
|
- QwQ |
|
- Qwen2.5 |
|
--- |
|
### **QwQ-LCoT-3B-Instruct Model Card** |
|
|
|
The **QwQ-LCoT-3B-Instruct** model is a lightweight, instruction-tuned language model designed for complex reasoning and explanation tasks. It is fine-tuned on the **Qwen2.5-3B-Instruct** base model using the **QwQ-LongCoT-130K** dataset, focusing on **long-chain-of-thought (LCoT)** reasoning for enhanced logical comprehension and detailed output generation. |
|
|
|
| **File Name** | **Size** | **Description** | **Upload Status** | |
|
|----------------------------------------|----------------|-------------------------------------------------|--------------------| |
|
| `.gitattributes` | 1.57 kB | Specifies LFS tracking for large files. | Uploaded | |
|
| `README.md` | 267 Bytes | Basic project information file. | Updated | |
|
| `added_tokens.json` | 657 Bytes | Custom tokens added to the tokenizer. | Uploaded | |
|
| `config.json` | 859 Bytes | Configuration file for the model. | Uploaded | |
|
| `generation_config.json` | 281 Bytes | Configuration file for text generation settings.| Uploaded | |
|
| `merges.txt` | 1.82 MB | Contains the byte-pair encoding (BPE) merges. | Uploaded | |
|
| `pytorch_model-00001-of-00002.bin` | 4.96 GB | First shard of the model weights in PyTorch format. | Uploaded (LFS) | |
|
| `pytorch_model-00002-of-00002.bin` | 1.21 GB | Second shard of the model weights in PyTorch format. | Uploaded (LFS) | |
|
| `pytorch_model.bin.index.json` | 36 kB | Index mapping for sharded model weights. | Uploaded | |
|
| `special_tokens_map.json` | 644 Bytes | Maps special tokens to their roles. | Uploaded | |
|
| `tokenizer.json` | 11.4 MB | Serialized tokenizer data. | Uploaded (LFS) | |
|
| `tokenizer_config.json` | 7.73 kB | Tokenizer configuration settings. | Uploaded | |
|
| `vocab.json` | 2.78 MB | Vocabulary file for the tokenizer. | Uploaded | |
|
|
|
### **Key Features:** |
|
|
|
1. **Long Chain-of-Thought Reasoning:** |
|
- Specifically designed to generate comprehensive, step-by-step explanations for complex queries. |
|
|
|
2. **Lightweight and Efficient:** |
|
- With only 3 billion parameters, it is optimized for systems with limited computational resources without compromising reasoning capabilities. |
|
|
|
3. **Instruction Optimization:** |
|
- Fine-tuned to follow prompts and provide concise, actionable, and structured responses. |
|
|
|
--- |
|
|
|
### **Training Details:** |
|
|
|
- **Base Model:** [Qwen2.5-3B-Instruct](#) |
|
- **Dataset:** [amphora/QwQ-LongCoT-130K](#) |
|
- Comprising 133,000 annotated samples focusing on logical tasks and structured thinking. |
|
--- |
|
|
|
### **Capabilities:** |
|
|
|
1. **Text Generation:** |
|
- Provides detailed, structured, and logical text outputs tailored to user prompts. |
|
|
|
2. **Reasoning Tasks:** |
|
- Solves step-by-step problems in math, logic, and science. |
|
|
|
3. **Educational Assistance:** |
|
- Generates coherent explanations for academic and research purposes. |
|
|
|
4. **Dialogue and Summarization:** |
|
- Handles conversational queries and summarizes long documents effectively. |
|
|
|
--- |
|
|
|
### **Usage Instructions:** |
|
|
|
1. **Setup:** |
|
Download all model files and ensure compatibility with the Hugging Face Transformers library. |
|
|
|
2. **Loading the Model:** |
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
model_name = "prithivMLmods/QwQ-LCoT-3B-Instruct" |
|
tokenizer = AutoTokenizer.from_pretrained(model_name) |
|
model = AutoModelForCausalLM.from_pretrained(model_name) |
|
``` |
|
|
|
3. **Generate Long-Chain Reasoning Outputs:** |
|
```python |
|
input_text = "Explain the process of photosynthesis step-by-step." |
|
inputs = tokenizer(input_text, return_tensors="pt") |
|
outputs = model.generate(**inputs, max_length=300, temperature=0.5) |
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
``` |
|
|
|
4. **Customize Output Generation:** |
|
Modify the `generation_config.json` file for different scenarios: |
|
- **`temperature`**: Controls randomness (lower = deterministic, higher = creative). |
|
- **`max_length`**: Sets response length. |
|
- **`top_p`**: Adjusts sampling for diversity in outputs. |
|
|
|
--- |
