arnosimons
commited on
Commit
•
28a2355
1
Parent(s):
e13c079
Update README.md
Browse files
README.md
CHANGED
@@ -39,7 +39,7 @@ tags:
|
|
39 |
|
40 |
# Model Card for Astro-HEP-BERT
|
41 |
|
42 |
-
**Astro-HEP-BERT** is a bidirectional transformer designed primarily to generate contextualized word embeddings for
|
43 |
|
44 |
The Astro-HEP-BERT project embodies the spirit of a tabletop experiment or grassroots scientific effort. It exclusively utilized open-source inputs during training, and the entire training process was completed on a single MacBook Pro M2/96GB in 48 days for 3 epochs. This project stands as a proof of concept, showcasing the viability of employing a bidirectional transformer for research ventures in the history, philosophy, and sociology of science (HPSS) even with limited financial resources.
|
45 |
|
@@ -50,7 +50,7 @@ For further insights into the model, the corpus, and the underlying research pro
|
|
50 |
## Model Details
|
51 |
|
52 |
- **Developer:** <a target="_blank" rel="noopener noreferrer" href="https://www.tu.berlin/en/hps-mod-sci/arno-simons">Arno Simons</a>
|
53 |
-
- **Funded by:** European
|
54 |
- **Language (NLP):** English
|
55 |
- **License:** apache-2.0
|
56 |
- **Parent model:** Google's <a target="_blank" rel="noopener noreferrer" href="https://github.com/google-research/bert">`bert-base-uncased`</a>
|
|
|
39 |
|
40 |
# Model Card for Astro-HEP-BERT
|
41 |
|
42 |
+
**Astro-HEP-BERT** is a bidirectional transformer designed primarily to generate contextualized word embeddings for computational conceptual analysis in astrophysics and high-energy physics (HEP). Built upon Google's `bert-base-uncased`, the model underwent additional training for three epochs using 21.84 million paragraphs found in more than 600,000 scholarly articles sourced from arXiv, all pertaining to astrophysics and/or high-energy physics (HEP). The sole training objective was masked language modeling.
|
43 |
|
44 |
The Astro-HEP-BERT project embodies the spirit of a tabletop experiment or grassroots scientific effort. It exclusively utilized open-source inputs during training, and the entire training process was completed on a single MacBook Pro M2/96GB in 48 days for 3 epochs. This project stands as a proof of concept, showcasing the viability of employing a bidirectional transformer for research ventures in the history, philosophy, and sociology of science (HPSS) even with limited financial resources.
|
45 |
|
|
|
50 |
## Model Details
|
51 |
|
52 |
- **Developer:** <a target="_blank" rel="noopener noreferrer" href="https://www.tu.berlin/en/hps-mod-sci/arno-simons">Arno Simons</a>
|
53 |
+
- **Funded by:** The European Union under Grant agreement ID: <a target="_blank" rel="noopener noreferrer" href="https://doi.org/10.3030/101044932" >101044932</a>
|
54 |
- **Language (NLP):** English
|
55 |
- **License:** apache-2.0
|
56 |
- **Parent model:** Google's <a target="_blank" rel="noopener noreferrer" href="https://github.com/google-research/bert">`bert-base-uncased`</a>
|