macedonizer commited on
Commit
e72656b
1 Parent(s): 12b271b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -17
README.md CHANGED
@@ -1,29 +1,28 @@
1
  ---
2
  language:
3
- - mk
4
- thumbnail: https://huggingface.co/macedonizer/mk-roberta-base/blaze-koneski.jpg
5
  license: apache-2.0
6
  datasets:
7
- - wiki-mk
8
- - time-mk-news-2010-2015
9
  ---
10
 
11
- # mk-gpt2
12
  Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large
13
  Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in
14
  [this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)
15
  and first released at [this page](https://openai.com/blog/better-language-models/).
16
 
17
  ## Model description
18
- mk-gpt2 is a transformers model pretrained on a very large corpus of Macedonian data in a self-supervised fashion. This
19
- means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots
20
  of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely,
21
  it was trained to guess the next word in sentences.
22
- More precisely, inputs are sequences of continuous text of a certain length and the targets are the same sequence,
23
- shifted one token (word or piece of word) to the right. The model uses internally a mask-mechanism to make sure the
24
  predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens.
25
- This way, the model learns an inner representation of the Macedonian language that can then be used to extract features
26
- useful for downstream tasks. The model is best at what it was pretrained for however, which is generating texts from a
27
  prompt.
28
 
29
  ### How to use
@@ -32,14 +31,14 @@ Here is how to use this model to get the features of a given text in PyTorch:
32
  import random
33
  from transformers import AutoTokenizer, AutoModelWithLMHead
34
 
35
- tokenizer = AutoTokenizer.from_pretrained('macedonizer/mk-gpt2') \
36
- model = AutoModelWithLMHead.from_pretrained('macedonizer/mk-gpt2')
37
 
38
- input_text = 'Скопје е '
39
 
40
  if len(input_text) == 0: \
41
  encoded_input = tokenizer(input_text, return_tensors="pt") \
42
- output = model.generate( \
43
  bos_token_id=random.randint(1, 50000), \
44
  do_sample=True, \
45
  top_k=50, \
@@ -47,9 +46,9 @@ if len(input_text) == 0: \
47
  top_p=0.95, \
48
  num_return_sequences=1, \
49
  ) \
50
- else: \
51
  encoded_input = tokenizer(input_text, return_tensors="pt") \
52
- output = model.generate( \
53
  **encoded_input, \
54
  bos_token_id=random.randint(1, 50000), \
55
  do_sample=True, \
 
1
  ---
2
  language:
3
+ - al
4
+ thumbnail: https://huggingface.co/macedonizer/al-roberta-base/lets-talk-about-nlp-al.jpg
5
  license: apache-2.0
6
  datasets:
7
+ - wiki-al
 
8
  ---
9
 
10
+ # al-gpt2
11
  Test the whole generation capabilities here: https://transformer.huggingface.co/doc/gpt2-large
12
  Pretrained model on English language using a causal language modeling (CLM) objective. It was introduced in
13
  [this paper](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)
14
  and first released at [this page](https://openai.com/blog/better-language-models/).
15
 
16
  ## Model description
17
+ al-gpt2 is a transformers model pretrained on a very large corpus of Albanian data in a self-supervised fashion. This
18
+ means it was pretrained on the raw texts only, with no humans labeling them in any way (which is why it can use lots
19
  of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely,
20
  it was trained to guess the next word in sentences.
21
+ More precisely, inputs are sequences of the continuous text of a certain length and the targets are the same sequence,
22
+ shifted one token (word or piece of a word) to the right. The model uses internally a mask-mechanism to make sure the
23
  predictions for the token `i` only uses the inputs from `1` to `i` but not the future tokens.
24
+ This way, the model learns an inner representation of the Albania language that can then be used to extract features
25
+ useful for downstream tasks. The model is best at what it was pretrained for, however, which is generating texts from a
26
  prompt.
27
 
28
  ### How to use
 
31
  import random
32
  from transformers import AutoTokenizer, AutoModelWithLMHead
33
 
34
+ tokenizer = AutoTokenizer.from_pretrained('macedonizer/al-gpt2') \
35
+ model = AutoModelWithLMHead.from_pretrained('macedonizer/al-gpt2')
36
 
37
+ input_text = 'Tirana'
38
 
39
  if len(input_text) == 0: \
40
  encoded_input = tokenizer(input_text, return_tensors="pt") \
41
+ output = model.generate( \
42
  bos_token_id=random.randint(1, 50000), \
43
  do_sample=True, \
44
  top_k=50, \
 
46
  top_p=0.95, \
47
  num_return_sequences=1, \
48
  ) \
49
+ else: \\
50
  encoded_input = tokenizer(input_text, return_tensors="pt") \
51
+ output = model.generate( \
52
  **encoded_input, \
53
  bos_token_id=random.randint(1, 50000), \
54
  do_sample=True, \