Ambuj Varshney commited on
Commit
b71d618
1 Parent(s): 49d06ad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -1
README.md CHANGED
@@ -26,13 +26,28 @@ This repository hosts a small language model developed as part of the TinyLLM fr
26
 
27
  ## Acknowledgements
28
 
29
- We would like to acknowledge the open-source frameworks [llm.c](https://github.com/karpathy/llm.c) and [llama.cpp](https://github.com/ggerganov/llama.cpp), which were instrumental in training and testing these models.
30
 
31
  ## Usage
32
 
33
  The model can be used in two primary ways:
34
  1. **With Hugging Face’s Transformers Library**
 
 
 
 
 
 
 
 
 
 
 
35
  2. **With llama.cpp**
 
 
 
 
36
 
37
  ## Disclaimer
38
 
 
26
 
27
  ## Acknowledgements
28
 
29
+ We want to acknowledge the open-source frameworks [llm.c](https://github.com/karpathy/llm.c) and [llama.cpp](https://github.com/ggerganov/llama.cpp) and the sensor dataset provided by SHL, which were instrumental in training and testing these models.
30
 
31
  ## Usage
32
 
33
  The model can be used in two primary ways:
34
  1. **With Hugging Face’s Transformers Library**
35
+ ```python
36
+ from transformers import pipeline
37
+ import torch
38
+
39
+ path = "tinyllm/51M-0.4"
40
+ prompt = "The sea is blue but it's his red sea"
41
+
42
+ generator = pipeline("text-generation", model=path,max_new_tokens = 30, repetition_penalty=1.3, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto")
43
+ print(generator(prompt)[0]['generated_text'])
44
+ ```
45
+
46
  2. **With llama.cpp**
47
+ Generate a GGUF model file using this [tool](https://github.com/ggerganov/llama.cpp/blob/master/convert_hf_to_gguf.py) and use the generated GGUF file for inferencing.
48
+ ```python
49
+ python3 convert_hf_to_gguf.py models/mymodel/
50
+ ```
51
 
52
  ## Disclaimer
53