digitous commited on
Commit
50ca297
1 Parent(s): 2d73f3b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -0
README.md CHANGED
@@ -1,3 +1,55 @@
1
  ---
2
  license: llama2
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: llama2
3
  ---
4
+
5
+ ## 13B-Legerdemain-L2
6
+ 13B-Legerdemain-L2 is the first model merge of its kind in a series of LLaMaV2 models mixed using a custom script built in-house by CalderaAI called Model-REVOLVER.
7
+ M-REVOLVER is also the first in a series of custom scripts based on the concept of mixtuning - not only does the end user have contol over which models are mixed
8
+ and their percentages on a per-layer basis, we tackle the problem of overcomplexity that arises from such a level of control; this model is the first of its series.
9
+
10
+ ## The Model-REVOLVER Process Designed by CalderaAI
11
+ M-REVOLVER (Rapid Evolution Via Optimized-List Viewer Evaluated Response)
12
+ Per-layer merging between parent models is a nebulous inexact science, and therefore impractical to most users despite the raw power it offers. We propose an
13
+ entirely new approach that gives the user a clear looking glass into the impact vastly different layer merge configurations between selected parent models of
14
+ their choice will have on the potential offspring model - especially its inherited behaviors. We've developed solution MK.1 - A cyclic random pattern search
15
+ in place that determines all layer merge ratios, combines test models, infers prompt completions, and deletes a prototype after data collection is saved.
16
+ When the cyclic system has completed its entire run, nothing is left but the telemetry collected along with the cycle and layer merge ratios from every
17
+ single prototype merge. This data is then used to empower the user to choose which offspring is most fit to their desired outcome. This final step is
18
+ only initiated when all necessary data has been aggregated from all assembled-tested-erased prototypes sampled in the search space.
19
+
20
+ From here, the user is provided five 300 token prompt completions from each and every offspring contender that was created and tested during the cyclic process.
21
+ The user simply browses each prototype's series of responses and selects their desired outcome model by entering the cycle number associated with the prompt
22
+ completions they feel best suits their vision. That model is then instantly repatriated into the official offspring of its parent models and tokenizer files
23
+ found to be most relevant are instantly auto-copied from the parent model dir to the offspring.
24
+
25
+ That's it - the user instantly has a complete model based on the behavior they decided on, suggested from one of many potentials; all with their own unique
26
+ trait inheritence thanks to layer merge auto randomization inside an ordered system. One more thing - the user not only selects how many cycles to run,
27
+ the user can edit prompts.txt which the system reads as a single prompt - this means if the user desires to use any multiline instruct format to observe
28
+ all potential model outcomes from instruct, or desires simply their own prompt, it's up to them.. simply works.
29
+
30
+ Link to GitHub for M-REVOLVER are at the end of the model card. More advanced MergeTech toolsets and merge techniques are currently under internal testing
31
+ and development by Caldera.
32
+
33
+ ## 13B-Legerdemain-L2 Use
34
+ 13B-Legerdemain-L2 is capable of following Alpaca instructions however it seems far more receptive to the by-the-book method as seen here:
35
+
36
+ ```
37
+ Below is an instruction that describes a task. Write a response that appropriately completes the request.
38
+
39
+ ### Instruction:
40
+ {instruction}
41
+
42
+ ### Response:
43
+ {New Line}
44
+ ```
45
+
46
+ The primary model of choice for this model was a story-only model called Holodeck by KoboldAI. Traits preserved seem to be detailed descriptiveness, verbosity,
47
+ and characters with personality. The two other models selected were 13B-Nous-Hermes by NousResearch and 13B-orca-8k-3319 by OpenAssistant. I began the process by
48
+ providing an incredibly obscene prompt and simply ignored each and every guardrail or censorship laden prompt completion and accepted the offensive ones in turn -
49
+ intent wasn't to be crass but trigger censorship parts of the network to test if it's possible to completely undermine them. Second pass with offspring model and
50
+ Orca was a simple milquetoast prompt to gauge vocabulary, word flow, and intelligence as I selected the most fit in that category. Result model seems a bit of a
51
+ curiosity - different samplers and even a different UI (as I went from TGUI to KoboldAI) seem to uncover different facets of behavior. Godlike preset with Alpaca
52
+ Instruct in TGUI worked fine. In KoboldAI some tweaking was necessary to get the same experience. If you choose to test this model, have fun - it's got a mind of
53
+ its own.
54
+
55
+ https://github.com/Digitous/ModelREVOLVER