|
--- |
|
library_name: transformers |
|
tags: |
|
- chess |
|
license: mit |
|
language: |
|
- en |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
The base model, Mitral-7B-v1, has been fine-tuned to improve its reasoning, game analysis, and chess understanding capabilities, including proficiency in Algebraic Notation and FEN (Forsyth-Edwards Notation). This enhancement aims to create a robust AI system architecture that can integrate various tools seamlessly, boosting cognitive abilities within the controlled environment of chess. |
|
The full work can be accessed [here](__link__to__add__) |
|
|
|
|
|
### Model Description |
|
|
|
- **Developed by:** Danny Xu, Carlos Kuhn, Muntasir Adnan |
|
- **Funded by:** OpenSI |
|
- **Model type:** Transformer based |
|
- **License:** MIT |
|
- **Finetuned from model:** Mistral-7B-v0.1 |
|
- |
|
|
|
### Model Sources |
|
|
|
|
|
- **Repository:** https://github.com/TheOpenSI/cognitive_AI_experiments |
|
- **Paper:** [Unleashing Artificial Cognition: Integrating Multiple AISystems](__link__to__add__) |
|
|
|
## Uses |
|
|
|
### Direct Use |
|
|
|
- Chess analysis |
|
- Meausre cognition qualities in a controlled environment |
|
|
|
### Downstream Use |
|
|
|
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app --> |
|
- AGI |
|
- Cognition capability of AI Systems |
|
|
|
|
|
## How to Get Started with the Model |
|
|
|
The model card contains only the LoRA adapter. To use it, load the adapter with the base Mistral model |
|
``` |
|
model = AutoModelForCausalLM.from_pretrained( |
|
base_model, |
|
quantization_config=bnb_config |
|
) |
|
|
|
lora_repo = "OpenSI/cognitive_AI_finetune_3" |
|
adapter_config = PeftConfig.from_pretrained(lora_repo) |
|
openSI_chess = PeftModel.from_pretrained(model, lora_model_name) |
|
``` |
|
|
|
## Training Details |
|
|
|
### Training Data |
|
|
|
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> |
|
- Analysis |
|
- Probable winner |
|
- Next move prediction |
|
- FEN parsing |
|
- Capture analysis |
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
- **Training regime:** |
|
``` |
|
bnb_config = BitsAndBytesConfig( |
|
load_in_4bit=True, |
|
bnb_4bit_use_double_quant=True, |
|
bnb_4bit_quant_type="nf4", |
|
bnb_4bit_compute_dtype=torch.bfloat16) |
|
|
|
|
|
model_args = TrainingArguments( |
|
output_dir="mistral_7b", |
|
num_train_epochs=3, |
|
# max_steps=50, |
|
per_device_train_batch_size=4, |
|
gradient_accumulation_steps=2, |
|
gradient_checkpointing=True, |
|
optim="paged_adamw_32bit", |
|
logging_steps=20, |
|
save_strategy="epoch", |
|
learning_rate=2e-4, |
|
bf16=True, |
|
tf32=True, |
|
max_grad_norm=0.3, |
|
warmup_ratio=0.03, |
|
lr_scheduler_type="constant", |
|
disable_tqdm=False |
|
) |
|
``` |
|
|
|
## Evaluation |
|
|
|
#### Testing Data |
|
Test dataset can be accessed here - [OpenSI Cognitive_AI](https://github.com/TheOpenSI/cognitive_AI_experiments/tree/master/data/test_framework) |
|
|
|
#### Metrics |
|
- Memory |
|
- Perception |
|
- Attention |
|
- Reasoning |
|
- Anticipation |
|
|
|
|
|
### Results |
|
|
|
<table> |
|
<thead> |
|
<tr> |
|
<th>Evaluation</th> |
|
</tr> |
|
</thead> |
|
<tbody> |
|
<tr> |
|
<td> |
|
<img src="./radar_plot.PNG" alt="Evaluation"> |
|
</td> |
|
</tr> |
|
</tbody> |
|
</table> |
|
|
|
|
|
#### Hardware |
|
|
|
Nvidia RTX 3090 |
|
|
|
|
|
## Citation |
|
``` |
|
@misc{Adnan2024, |
|
title = {Unleashing Artificial Cognition: Integrating Multiple AI Systems}, |
|
author = {Muntasir Adnan and Buddhi Gamage and Zhiwei Xu and Damith Herath and Carlos C. N. Kuhn}, |
|
year = {2024}, |
|
eprint = {2408.04910}, |
|
archivePrefix = {arXiv} |
|
} |
|
``` |
