J38 commited on
Commit
39fea4e
·
1 Parent(s): 68d2933

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -4,17 +4,17 @@ widget:
4
  - text: 'Photosynthesis is'
5
  ---
6
 
7
- # Model Card for Pubmed GPT 2.7B
8
 
9
- PubMed GPT 2.7B is new language model trained exclusively on biomedical abstracts and papers from [The Pile](https://pile.eleuther.ai/). This GPT-style model can achieve strong results on a variety of biomedical NLP tasks, including a new state of the art performance of 50.3% accuracy on the MedQA biomedical question answering task.
10
 
11
- As an autoregressive language model, PubMed GPT 2.7B is also capable of natural language generation. However, we have only begun to explore the generation capabilities and limitations of this model, and we emphasize that this model’s generation capabilities are for research purposes only and not suitable for production. In releasing this model, we hope to advance both the development of biomedical NLP applications and best practices for responsibly training and utilizing domain-specific language models; issues of reliability, truthfulness, and explainability are top of mind for us.
12
 
13
  This model was a joint collaboration of [Stanford CRFM](https://crfm.stanford.edu/) and [MosaicML](https://www.mosaicml.com/).
14
 
15
  # Table of Contents
16
 
17
- - [Model Card for Pubmed GPT 2.7B](#model-card-for--model_id-)
18
  - [Table of Contents](#table-of-contents)
19
  - [Model Details](#model-details)
20
  - [Model Description](#model-description)
@@ -37,9 +37,9 @@ This model was a joint collaboration of [Stanford CRFM](https://crfm.stanford.ed
37
  ## Model Description
38
 
39
  <!-- Provide a longer summary of what this model is/does. -->
40
- PubMed GPT 2.7B is new language model trained exclusively on biomedical abstracts and papers from [The Pile](https://pile.eleuther.ai/). This GPT-style model can achieve strong results on a variety of biomedical NLP tasks, including a new state of the art performance of 50.3% accuracy on the MedQA biomedical question answering task.
41
 
42
- As an autoregressive language model, PubMed GPT 2.7B is also capable of natural language generation. However, we have only begun to explore the generation capabilities and limitations of this model, and we emphasize that this model’s generation capabilities are for research purposes only and not suitable for production. In releasing this model, we hope to advance both the development of biomedical NLP applications and best practices for responsibly training and utilizing domain-specific language models; issues of reliability, truthfulness, and explainability are top of mind for us.
43
 
44
  This model was a joint collaboration of [Stanford CRFM](https://crfm.stanford.edu/) and [MosaicML](https://www.mosaicml.com/).
45
 
@@ -131,7 +131,7 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
131
 
132
  ## Model Architecture and Objective
133
 
134
- Pubmed GPT 2.7B is a standard GPT-2 implementation (trained with Flash Attention) with the following hyperparameters:
135
 
136
  | | |
137
  | ----------- | ----- |
 
4
  - text: 'Photosynthesis is'
5
  ---
6
 
7
+ # Model Card for PubmedGPT 2.7B
8
 
9
+ PubMedGPT 2.7B is new language model trained exclusively on biomedical abstracts and papers from [The Pile](https://pile.eleuther.ai/). This GPT-style model can achieve strong results on a variety of biomedical NLP tasks, including a new state of the art performance of 50.3% accuracy on the MedQA biomedical question answering task.
10
 
11
+ As an autoregressive language model, PubMedGPT 2.7B is also capable of natural language generation. However, we have only begun to explore the generation capabilities and limitations of this model, and we emphasize that this model’s generation capabilities are for research purposes only and not suitable for production. In releasing this model, we hope to advance both the development of biomedical NLP applications and best practices for responsibly training and utilizing domain-specific language models; issues of reliability, truthfulness, and explainability are top of mind for us.
12
 
13
  This model was a joint collaboration of [Stanford CRFM](https://crfm.stanford.edu/) and [MosaicML](https://www.mosaicml.com/).
14
 
15
  # Table of Contents
16
 
17
+ - [Model Card for PubmedGPT 2.7B](#model-card-for--model_id-)
18
  - [Table of Contents](#table-of-contents)
19
  - [Model Details](#model-details)
20
  - [Model Description](#model-description)
 
37
  ## Model Description
38
 
39
  <!-- Provide a longer summary of what this model is/does. -->
40
+ PubMedGPT 2.7B is new language model trained exclusively on biomedical abstracts and papers from [The Pile](https://pile.eleuther.ai/). This GPT-style model can achieve strong results on a variety of biomedical NLP tasks, including a new state of the art performance of 50.3% accuracy on the MedQA biomedical question answering task.
41
 
42
+ As an autoregressive language model, PubMedGPT 2.7B is also capable of natural language generation. However, we have only begun to explore the generation capabilities and limitations of this model, and we emphasize that this model’s generation capabilities are for research purposes only and not suitable for production. In releasing this model, we hope to advance both the development of biomedical NLP applications and best practices for responsibly training and utilizing domain-specific language models; issues of reliability, truthfulness, and explainability are top of mind for us.
43
 
44
  This model was a joint collaboration of [Stanford CRFM](https://crfm.stanford.edu/) and [MosaicML](https://www.mosaicml.com/).
45
 
 
131
 
132
  ## Model Architecture and Objective
133
 
134
+ PubmedGPT 2.7B is a standard GPT-2 implementation (trained with Flash Attention) with the following hyperparameters:
135
 
136
  | | |
137
  | ----------- | ----- |