arnosimons commited on
Commit
579aaa7
1 Parent(s): d0100a5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -5,11 +5,13 @@ language:
5
  pipeline_tag: fill-mask
6
  ---
7
 
8
- # Model Card for **Astro-HEP-BERT**
9
 
10
  **Astro-HEP-BERT** is a bidirectional transformer designed primarily to generate contextualized word embeddings for analyzing epistemic change in astrophysics and high-energy physics (<a target="_blank" rel="noopener noreferrer" href="https://doi.org/10.3030/101044932" >NEPI project</a> at TU Berlin). Built upon Google's "bert-base-uncased," the model underwent additional training for three epochs using approximately 21.5 million paragraphs extracted from around 600,000 scholarly articles sourced from arXiv, all pertaining to astrophysics and/or high-energy physics (HEP). The sole training objective was masked language modeling.
11
 
12
- For further insights into the model and the corpus, please refer to the Astro-HEP-BERT paper [link coming soon].
 
 
13
 
14
  <!-- <a target="_blank" rel="noopener noreferrer" href="">Astro-HEP-BERT paper</a>. -->
15
 
 
5
  pipeline_tag: fill-mask
6
  ---
7
 
8
+ # Model Card for Astro-HEP-BERT
9
 
10
  **Astro-HEP-BERT** is a bidirectional transformer designed primarily to generate contextualized word embeddings for analyzing epistemic change in astrophysics and high-energy physics (<a target="_blank" rel="noopener noreferrer" href="https://doi.org/10.3030/101044932" >NEPI project</a> at TU Berlin). Built upon Google's "bert-base-uncased," the model underwent additional training for three epochs using approximately 21.5 million paragraphs extracted from around 600,000 scholarly articles sourced from arXiv, all pertaining to astrophysics and/or high-energy physics (HEP). The sole training objective was masked language modeling.
11
 
12
+ The Astro-HEP-BERT project embodies the spirit of a tabletop experiment or grassroots scientific effort. It exclusively utilized open-source inputs during training, and the entire training process was completed on a single MacBook Pro M2/96GB over a span of 6 weeks for 3 epochs. This project stands as a proof of concept, showcasing the viability of employing a bidirectional transformer for research ventures in the history, philosophy, and sociology of science (HPSS) even with limited financial resources.
13
+
14
+ For further insights into the model, the corpus, and the underlying research project please refer to the Astro-HEP-BERT paper [link coming soon].
15
 
16
  <!-- <a target="_blank" rel="noopener noreferrer" href="">Astro-HEP-BERT paper</a>. -->
17