Loubna ben allal
commited on
Commit
•
a13c298
1
Parent(s):
d97087d
remove ipynb
Browse files
architectures/.ipynb_checkpoints/codeparrot-checkpoint.txt
DELETED
@@ -1,5 +0,0 @@
|
|
1 |
-
[CodeParrot](https://huggingface.co/lvwerra/codeparrot) uses GPT-2 architecture with BPE tokenizer trained on Python code.
|
2 |
-
|
3 |
-
|Model | # parameters |
|
4 |
-
| - | - |
|
5 |
-
| GPT2 | 1.5B |
|
|
|
|
|
|
|
|
|
|
|
|
architectures/.ipynb_checkpoints/incoder-checkpoint.txt
DELETED
@@ -1,5 +0,0 @@
|
|
1 |
-
[InCoder](https://huggingface.co/facebook/incoder-6B) uses a decoder-only Transformer with [Causal Masking objective](https://arxiv.org/abs/2201.07520), to train a left-to-right language model to fill in masked token segments.
|
2 |
-
|
3 |
-
|Model | # parameters |
|
4 |
-
| - | - |
|
5 |
-
| Decoder |6.7B |
|
|
|
|
|
|
|
|
|
|
|
|
architectures/.ipynb_checkpoints/opt-checkpoint.txt
DELETED
@@ -1,5 +0,0 @@
|
|
1 |
-
[OPT](https://huggingface.co/facebook/opt-30b) uses decoder-only models like GPT-3. It was trained on datasets with a small portion of code. In this demo we use the 30B parameters model. The largest model has 176B parameters.
|
2 |
-
|
3 |
-
|Model | # parameters |
|
4 |
-
| - | - |
|
5 |
-
| Decoder |30B |
|
|
|
|
|
|
|
|
|
|
|
|