CryptoBERT
This model is a fine-tuned version of ProsusAI/finbert on the Custom Crypto Market Sentiment dataset. It achieves the following results on the evaluation set:
- Loss: 0.3823
from transformers import BertTokenizer, BertForSequenceClassification
from transformers import pipeline
tokenizer = BertTokenizer.from_pretrained("kk08/CryptoBERT")
model = BertForSequenceClassification.from_pretrained("kk08/CryptoBERT")
classifier = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)
text = "Bitcoin (BTC) touches $29k, Ethereum (ETH) Set To Explode, RenQ Finance (RENQ) Crosses Massive Milestone"
result = classifier(text)
print(result)
[{'label': 'LABEL_1', 'score': 0.9678454399108887}]
Model description
This model fine-tunes the ProsusAI/finbert, which is a pre-trained NLP model to analyze the sentiment of the financial text. CryptoBERT model fine-tunes this by training the model as a downstream task on Custom Crypto Sentiment data to predict whether the given text related to the Crypto market is Positive (LABEL_1) or Negative (LABEL_0).
Intended uses & limitations
The model can perform well on Crypto-related data. The main limitation is that the fine-tuning was done using only a small corpus of data
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10
Training results
Training Loss | Epoch | Step | Validation Loss |
---|---|---|---|
0.4077 | 1.0 | 27 | 0.4257 |
0.2048 | 2.0 | 54 | 0.2479 |
0.0725 | 3.0 | 81 | 0.3068 |
0.0028 | 4.0 | 108 | 0.4120 |
0.0014 | 5.0 | 135 | 0.3566 |
0.0007 | 6.0 | 162 | 0.3495 |
0.0006 | 7.0 | 189 | 0.3645 |
0.0005 | 8.0 | 216 | 0.3754 |
0.0004 | 9.0 | 243 | 0.3804 |
0.0004 | 10.0 | 270 | 0.3823 |
Framework versions
- Transformers 4.28.0
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.3
- Downloads last month
- 1,665
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for kk08/CryptoBERT
Base model
ProsusAI/finbert