Text Generation
Transformers
Safetensors
English
mistral
text-generation-inference
Inference Endpoints
instruction-pretrain commited on
Commit
0ecc04f
1 Parent(s): 457cad7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -16,6 +16,7 @@ We explore supervised multitask pre-training by proposing ***Instruction Pre-Tra
16
 
17
  **************************** **Updates** ****************************
18
  * 2024/9/20: Our paper has been accepted by EMNLP 2024 main conference🎉
 
19
  * 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/medicine-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
20
  * 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
21
  * 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. The performance trend on downstream tasks throughout the pre-training process:
@@ -224,7 +225,7 @@ text_ids = tokenizer(text, add_special_tokens=False, **kwargs).input_ids
224
  ## Citation
225
  If you find our work helpful, please cite us:
226
 
227
- Instruction Pre-Training (EMNLP 2024)
228
  ```bibtex
229
  @article{cheng2024instruction,
230
  title={Instruction Pre-Training: Language Models are Supervised Multitask Learners},
@@ -234,7 +235,7 @@ Instruction Pre-Training (EMNLP 2024)
234
  }
235
  ```
236
 
237
- [Adapt LLM to Domains](https://huggingface.co/papers/2309.09530)ICLR 2024
238
  ```bibtex
239
  @inproceedings{
240
  cheng2024adapting,
 
16
 
17
  **************************** **Updates** ****************************
18
  * 2024/9/20: Our paper has been accepted by EMNLP 2024 main conference🎉
19
+ * 2024/9/11: Updated [FAQ on continual pre-training from Llama3](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
20
  * 2024/8/29: Updated [guidelines](https://huggingface.co/instruction-pretrain/medicine-Llama3-8B) on evaluating any 🤗Huggingface models on the domain-specific tasks
21
  * 2024/7/31: Updated pre-training suggestions in the `Advanced Usage` section of [instruction-synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer)
22
  * 2024/7/15: We scaled up the pre-trained tokens from 100B to 250B, with the number of synthesized instruction-response pairs reaching 500M. The performance trend on downstream tasks throughout the pre-training process:
 
225
  ## Citation
226
  If you find our work helpful, please cite us:
227
 
228
+ [Instruction Pre-Training](https://huggingface.co/papers/2406.14491) (EMNLP 2024)
229
  ```bibtex
230
  @article{cheng2024instruction,
231
  title={Instruction Pre-Training: Language Models are Supervised Multitask Learners},
 
235
  }
236
  ```
237
 
238
+ [Adapt LLM to Domains](https://huggingface.co/papers/2309.09530) (ICLR 2024)
239
  ```bibtex
240
  @inproceedings{
241
  cheng2024adapting,