tjadamlee commited on
Commit
c7ff67a
1 Parent(s): eab3fa6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -1
README.md CHANGED
@@ -54,12 +54,34 @@ cae7b4ee8d1ad4e4402632bb0600cc17 ./tokenizer_config.json.ef7ef410b9b909949e96f1
54
  848005d07146c31e73a10020b3a3099a ./tokenizer.model.9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347.enc
55
  ```
56
 
57
- 2. Decrypt the files using https://github.com/LianjiaTech/BELLE/tree/main/models#使用说明
 
 
 
 
 
58
  ```
59
  for f in "encrypted"/*; do if [ -f "$f" ]; then python3 decrypt.py "$f" "original/7B/consolidated.00.pth" "result/"; fi; done
60
  ```
61
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
62
  3. Check md5sum
 
 
63
  ```
64
  md5sum ./*
65
  0fa6ff8379308d40f090878593f085a9 ./config.json
 
54
  848005d07146c31e73a10020b3a3099a ./tokenizer.model.9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347.enc
55
  ```
56
 
57
+ 2. Decrypt the files using the scripts in https://github.com/LianjiaTech/BELLE/tree/main/models
58
+
59
+ This command for conversion requires approximately 60 GB of RAM.
60
+ In case you have insufficient memory, you can create a sizeable swap file,
61
+ which permits the operating system to utilize the disk as virtual memory.
62
+
63
  ```
64
  for f in "encrypted"/*; do if [ -f "$f" ]; then python3 decrypt.py "$f" "original/7B/consolidated.00.pth" "result/"; fi; done
65
  ```
66
 
67
+ After executing the aforementioned command, you will obtain the following files.
68
+
69
+ ```
70
+ ./config.json
71
+ ./generation_config.json
72
+ ./pytorch_model-00001-of-00003.bin
73
+ ./pytorch_model-00002-of-00003.bin
74
+ ./pytorch_model-00003-of-00003.bin
75
+ ./pytorch_model.bin.index.json
76
+ ./README.md
77
+ ./special_tokens_map.json
78
+ ./tokenizer_config.json
79
+ ./tokenizer.model
80
+ ```
81
+
82
  3. Check md5sum
83
+ You can verify the integrity of these files by performing an MD5 checksum to ensure their complete recovery.
84
+ Here are the MD5 checksums for the relevant files:
85
  ```
86
  md5sum ./*
87
  0fa6ff8379308d40f090878593f085a9 ./config.json