metadata

license: bigscience-bloom-rail-1.0
tags:
  - generated_from_trainer
model-index:
  - name: bloom-560m-finetuned-unnatural-instructions-6k-steps
    results: []
widget:
  - text: >-
      <s>You will be provided with a short text that you should read. After
      reading the text, answer the question 'Who is telling the story in first
      person?'

      Text: I went to the store and bought some milk.

      The output should be one of the two options: 'I' or 'Not I'.

      Output:
  - text: >-
      <s>You are given the title and opening paragraph of an article. Your task
      is to find the main idea of the article using context clues from the text.

      Text: The first confirmed case of Zika in Africa has been found in Uganda,
      health officials said on Tuesday, more than 4,000 miles away from Brazil
      where a major outbreak began last year.

      Ablood test carried out at a private hospital showed that a five-year-old
      girl who had returned from Mozambique three weeks ago had contracted Zika,
      which can cause birth defects in babies born to infected mothers.

      Output:
  - text: >
      <s>We have given you a list of sentences. Your task is to go through this
      list and find all the unique words used in these sentences and store them
      in a dictionary.

      This is an example sentence., That is another example sentence., Here is
      an interesting sentence!

      The output should be a Python dictionary with keys as the unique words and
      values as the number of times that word occurs in the text.

      Output:
inference:
  parameters:
    max_length: 150

bloom-560m-finetuned-unnatural-instructions-6k-steps

This model is a fine-tuned version of bigscience/bloom-560m on the None dataset. It achieves the following results on the evaluation set:

Loss: 1.4037

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 1
eval_batch_size: 2
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
training_steps: 6000
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.6758	0.32	1000	1.6487
1.512	0.63	2000	1.5360
1.4658	0.95	3000	1.4580
1.0616	1.26	4000	1.4510
1.0291	1.58	5000	1.4199
0.9906	1.89	6000	1.4037

Framework versions

Transformers 4.25.1
Pytorch 1.13.0+cu116
Datasets 2.8.0
Tokenizers 0.13.2