some1nostr commited on
Commit
1c5c584
·
verified ·
1 Parent(s): 24435a1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -15
README.md CHANGED
@@ -10,25 +10,30 @@ license: apache-2.0
10
 
11
  **Finetuned from model:** [Llama3-70B](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct)
12
 
13
- ## Fine Tuned With
14
 
15
- **Trained with Nostr notes**: Knowledge and ideas about bitcoin and also Nostr itself. You can read more about the effects of Nostr training here:
16
- https://habla.news/a/naddr1qvzqqqr4gupzp8lvwt2hnw42wu40nec7vw949ys4wgdvums0svs8yhktl8mhlpd3qqxnzde3xumrswfjx56rjwf4kkqhsx
 
 
 
 
 
 
 
17
 
18
  **Aligned a bit in these domains:**
19
  - Health
20
- - Permaculture
21
- - Phytochemicals
22
  - Alternative medicine
23
  - Herbs
24
- - Nutrition
 
25
 
26
  More to come around these topics.
27
 
28
- Fine tuned a bit in faith related topics.
29
-
30
 
31
- ## Uses
32
 
33
  Compared to other models this may know more about Nostr and Bitcoin. It is aligned with opinions of people on Nostr. It may have alternative ideas to mainstream because Nostr is very censorship resistant.
34
 
@@ -39,25 +44,25 @@ Use repeat penalty of 1.05 or more to avoid repetitions.
39
  I am using the model here as a ground truth for Nostr related questions: https://wikifreedia.xyz/based-llm-leaderboard/npub1nlk894teh248w2heuu0x8z6jjg2hyxkwdc8cxgrjtm9lnamlskcsghjm9c
40
 
41
 
42
- ## Warning
43
 
44
  Users (both direct and downstream) should be aware of the risks, biases and limitations of the model.
45
  The trainer, developer or uploader of this model does not assume any liability. Use it at your own risk.
46
 
47
- ## Training Details
48
 
49
- ### Training Data
50
 
51
  Nostr related info from web and nostr itself, bitcoin related info.
52
  Information that aligns well with humanity is preferred.
53
  About 80% comes from Nostr notes. The rest is my own curation.
54
 
55
- ### Training Procedure
56
 
57
  LLaMa-Factory is used to train on 2* RTX 3090! fsdp_qlora is the technique.
58
 
59
- The Nostr training took ~200 hours.
60
 
61
  After training it for a while with notes, I used it to analyze and decide what notes to take in further for training.
62
 
63
- The number in the filenames (like 9230) means the version. I take the training steps and add those to the version. There has been more than 9k training steps.
 
10
 
11
  **Finetuned from model:** [Llama3-70B](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct)
12
 
13
+ # Fine Tuned With
14
 
15
+ **Nostr notes**:
16
+
17
+ This fine tuning makes the model know more about bitcoin and topics on Nostr.
18
+
19
+ You can read more about the "LLM curation" via fine tuning with Nostr notes:
20
+ [here](https://habla.news/a/naddr1qvzqqqr4gupzp8lvwt2hnw42wu40nec7vw949ys4wgdvums0svs8yhktl8mhlpd3qqgx6cnjvfhhzet20pkhqdn2wenkvu6gy4y) and
21
+ [here](https://habla.news/a/naddr1qvzqqqr4gupzp8lvwt2hnw42wu40nec7vw949ys4wgdvums0svs8yhktl8mhlpd3qqxnzde3xumrswfjx56rjwf4kkqhsx).
22
+
23
+ **Fine tuned in faith related topics**
24
 
25
  **Aligned a bit in these domains:**
26
  - Health
27
+ - Nutrition
 
28
  - Alternative medicine
29
  - Herbs
30
+ - Permaculture
31
+ - Phytochemicals
32
 
33
  More to come around these topics.
34
 
 
 
35
 
36
+ # Uses
37
 
38
  Compared to other models this may know more about Nostr and Bitcoin. It is aligned with opinions of people on Nostr. It may have alternative ideas to mainstream because Nostr is very censorship resistant.
39
 
 
44
  I am using the model here as a ground truth for Nostr related questions: https://wikifreedia.xyz/based-llm-leaderboard/npub1nlk894teh248w2heuu0x8z6jjg2hyxkwdc8cxgrjtm9lnamlskcsghjm9c
45
 
46
 
47
+ # Warning
48
 
49
  Users (both direct and downstream) should be aware of the risks, biases and limitations of the model.
50
  The trainer, developer or uploader of this model does not assume any liability. Use it at your own risk.
51
 
52
+ # Training Details
53
 
54
+ ## Training Data
55
 
56
  Nostr related info from web and nostr itself, bitcoin related info.
57
  Information that aligns well with humanity is preferred.
58
  About 80% comes from Nostr notes. The rest is my own curation.
59
 
60
+ ## Training Procedure
61
 
62
  LLaMa-Factory is used to train on 2* RTX 3090! fsdp_qlora is the technique.
63
 
64
+ The Nostr training took more than 200 hours.
65
 
66
  After training it for a while with notes, I used it to analyze and decide what notes to take in further for training.
67
 
68
+ The number in the filenames (like 12345) means the version. I take the training steps and add those to the version. There has been more than 9k training steps.