metadata

license: bigscience-bloom-rail-1.0
tags:
  - generated_from_trainer
model-index:
  - name: bloom-560m-finetuned-unnatural-instructions-6k-steps
    results: []
widget:
  - text: >-
      <s>You will be provided with a short text that you should read. After
      reading the text, answer the question 'Who is telling the story in first
      person?'

      Text: I went to the store and bought some milk.

      The output should be one of the two options: 'I' or 'Not I'.

      Output:
  - text: >-
      <s>You are given the title and opening paragraph of an article. Your task
      is to find the main idea of the article using context clues from the text.

      Text: The first confirmed case of Zika in Africa has been found in Uganda,
      health officials said on Tuesday, more than 4,000 miles away from Brazil
      where a major outbreak began last year.

      Ablood test carried out at a private hospital showed that a five-year-old
      girl who had returned from Mozambique three weeks ago had contracted Zika,
      which can cause birth defects in babies born to infected mothers.

      Output:
  - text: >
      <s>We have given you a list of sentences. Your task is to go through this
      list and find all the unique words used in these sentences and store them
      in a dictionary.

      This is an example sentence., That is another example sentence., Here is
      an interesting sentence!

      The output should be a Python dictionary with keys as the unique words and
      values as the number of times that word occurs in the text.

      Output:
  - text: >
      <s>You will given a set of two or more words. Output the shortest word in
      the set. If there is a tie for the shortest word, output all tied words in
      alphabetical order, separated by space.

      Words: defenestrate, circumambulate, excommunication.

      The input will be lowercase and only contain alphanumeric characters and
      spaces.

      Output:
inference:
  parameters:
    max_length: 150

bloom-560m-finetuned-unnatural-instructions-6k-steps

This model is a fine-tuned version of bigscience/bloom-560m on the None dataset. It achieves the following results on the evaluation set:

Loss: 1.4037

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 1
eval_batch_size: 2
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
training_steps: 6000
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.6758	0.32	1000	1.6487
1.512	0.63	2000	1.5360
1.4658	0.95	3000	1.4580
1.0616	1.26	4000	1.4510
1.0291	1.58	5000	1.4199
0.9906	1.89	6000	1.4037

Framework versions

Transformers 4.25.1
Pytorch 1.13.0+cu116
Datasets 2.8.0
Tokenizers 0.13.2