mrm8488's picture
Update README.md
3a62479
|
raw
history blame
2.51 kB
metadata
license: bigscience-bloom-rail-1.0
tags:
  - generated_from_trainer
model-index:
  - name: bloom-560m-finetuned-unnatural-instructions-6k-steps
    results: []
widget:
  - text: >-
      <s>You will be provided with a short text that you should read. After
      reading the text, answer the question 'Who is telling the story in first
      person?'

      Text: I went to the store and bought some milk.

      The output should be one of the two options: 'I' or 'Not I'.

      Output:
  - text: >-
      <s>'You are given the title and opening paragraph of an article. Your task
      is to find the main idea of the article using context clues from the text.

      Text: The first confirmed case of Zika in Africa has been found in Uganda,
      health officials said on Tuesday, more than 4,000 miles away from Brazil
      where a major outbreak began last year.

      Ablood test carried out at a private hospital showed that a five-year-old
      girl who had returned from Mozambique three weeks ago had contracted Zika,
      which can cause birth defects in babies born to infected mothers.

      Output:

bloom-560m-finetuned-unnatural-instructions-6k-steps

This model is a fine-tuned version of bigscience/bloom-560m on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4037

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 1
  • eval_batch_size: 2
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • training_steps: 6000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
1.6758 0.32 1000 1.6487
1.512 0.63 2000 1.5360
1.4658 0.95 3000 1.4580
1.0616 1.26 4000 1.4510
1.0291 1.58 5000 1.4199
0.9906 1.89 6000 1.4037

Framework versions

  • Transformers 4.25.1
  • Pytorch 1.13.0+cu116
  • Datasets 2.8.0
  • Tokenizers 0.13.2