update
Browse files- architectures/intro.txt +2 -0
architectures/intro.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Various architectures are used in code generation models, but most of them use the auto-regressive left-to-right setting, such as GPT. However [InCoder](https://huggingface.co/facebook/incoder-6B) used a decoder-only Transformer with Causal Masking objective,
|
| 2 |
+
that combines both next token prediction and bidirectional context through masking. AlphaCode used an encoder-decoder architecture. For model-specific information about the architecture, please select a model below:
|