ArtifactAI commited on
Commit
7ffd91a
1 Parent(s): 34375e7

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +77 -0
README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ pipeline_tag: summarization
6
+ widget:
7
+ - text: What is an LSTM?
8
+ example_title: Question Answering
9
+ tags:
10
+ - arxiv
11
+ ---
12
+ # Table of Contents
13
+
14
+ 0. [TL;DR](#TL;DR)
15
+ 1. [Model Details](#model-details)
16
+ 2. [Usage](#usage)
17
+ 3. [Uses](#uses)
18
+ 4. [Citation](#citation)
19
+
20
+ # TL;DR
21
+
22
+ This is a FLAN-T5-XXL model trained on [ArtifactAI/arxiv-cs-ml-instruct-50k](https://huggingface.co/datasets/ArtifactAI/arxiv-cs-ml-instruct-50k). This model is for research purposes only and ***should not be used in production settings***. The output it highly unreliable.
23
+
24
+
25
+ ## Model Description
26
+
27
+
28
+ - **Model type:** Language model
29
+ - **Language(s) (NLP):** English
30
+ - **License:** Apache 2.0
31
+ - **Related Models:** [All FLAN-T5 Checkpoints](https://huggingface.co/models?search=flan-t5)
32
+
33
+ # Usage
34
+
35
+ Find below some example scripts on how to use the model in `transformers`:
36
+
37
+ ## Using the Pytorch model
38
+
39
+ ```python
40
+
41
+ import torch
42
+ from peft import PeftModel, PeftConfig
43
+ from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
44
+
45
+ # Load peft config for pre-trained checkpoint etc.
46
+ peft_model_id = "ArtifactAI/flant5-xxl-math-full-training-run-one"
47
+ config = PeftConfig.from_pretrained(peft_model_id)
48
+
49
+ # load base LLM model and tokenizer
50
+ model = AutoModelForSeq2SeqLM.from_pretrained(config.base_model_name_or_path, load_in_8bit=True, device_map={"":0})
51
+ tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
52
+
53
+ # Load the Lora model
54
+ model = PeftModel.from_pretrained(model, peft_model_id, device_map={"":0})
55
+ model.eval()
56
+
57
+
58
+ input_ids = tokenizer("What is the peak phase of T-eV?", return_tensors="pt", truncation=True).input_ids.cuda()
59
+ # with torch.inference_mode():
60
+ outputs = model.generate(input_ids=input_ids, max_new_tokens=1000, do_sample=True, top_p=0.9)
61
+
62
+ print(f"summary: {tokenizer.batch_decode(outputs.detach().cpu().numpy(), skip_special_tokens=True)[0]}")
63
+ ```
64
+
65
+ ## Training Data
66
+
67
+ The model was trained on [ArtifactAI/arxiv-math-instruct-50k](https://huggingface.co/datasets/ArtifactAI/arxiv-cs-ml-instruct-50k), a dataset of question/answer pairs. Questions are generated using the t5-base model, while the answers are generated using the GPT-3.5-turbo model.
68
+
69
+ # Citation
70
+
71
+ ```
72
+ @misc{flan-t5-xxl-arxiv-cs-ml-zeroshot-qa,
73
+ title={flan-t5-xxl-arxiv-cs-ml-zeroshot-qa},
74
+ author={Matthew Kenney},
75
+ year={2023}
76
+ }
77
+ ```