Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,145 @@
|
|
1 |
-
---
|
2 |
-
license: mit
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
tags:
|
4 |
+
- Mistral_Star
|
5 |
+
- Mistral_Quiet
|
6 |
+
- Mistral
|
7 |
+
- Mixtral
|
8 |
+
- Question-Answer
|
9 |
+
- Token-Classification
|
10 |
+
- Sequence-Classification
|
11 |
+
- SpydazWeb-AI
|
12 |
+
- chemistry
|
13 |
+
- biology
|
14 |
+
- legal
|
15 |
+
- code
|
16 |
+
- climate
|
17 |
+
- medical
|
18 |
+
- text-generation-inference
|
19 |
+
language:
|
20 |
+
- en
|
21 |
+
- sw
|
22 |
+
- ig
|
23 |
+
- zu
|
24 |
+
- ca
|
25 |
+
- es
|
26 |
+
- pt
|
27 |
+
- ha
|
28 |
+
pipeline_tag: text-generation
|
29 |
+
---
|
30 |
+
# SpydazWeb AGI
|
31 |
+
|
32 |
+
|
33 |
+
This is based on the Quiet Star Reasoning Project : which was abandoned earlier in the year :)
|
34 |
+
|
35 |
+
# Introduction :
|
36 |
+
|
37 |
+
## STAR REASONERS !
|
38 |
+
|
39 |
+
this provides a platform for the model to commuicate pre-response , so an internal objective can be set ie adding an extra planning stage to the model improving its focus and output:
|
40 |
+
the thought head can be charged with a thought or methodolgy, such as a ststing to take a step by step approach to the problem or to make an object oriented model first and consider the use cases before creating an output:
|
41 |
+
so each thought head can be dedicated to specific ppurpose such as Planning or artifact generation or use case design : or even deciding which methodology should be applied before planning the potential solve route for the response :
|
42 |
+
Another head could also be dedicated to retrieving content based on the query from the self which can also be used in the pregenerations stages :
|
43 |
+
all pre- reasoners can be seen to be Self Guiding ! essentially removing the requirement to give the model a system prompt instead aligning the heads to a thoght pathways !
|
44 |
+
these chains produce data which can be considered to be thoughts : and can further be displayed by framing these thoughts with thought tokens : even allowing for editors comments giving key guidance to the model during training :
|
45 |
+
these thoughts will be used in future genrations assisting the model as well a displaying explantory informations in the output :
|
46 |
+
|
47 |
+
these tokens can be displayed or with held also a setting in the model !
|
48 |
+
|
49 |
+
### can this be applied in other areas ?
|
50 |
+
|
51 |
+
Yes! , we can use this type of method to allow for the model to generate code in another channel or head potentially creating a head to produce artifacts for every output , or to produce entity lilsts for every output and framing the outputs in thier relative code tags or function call tags :
|
52 |
+
these can also be displayed or hidden for the response . but these can also be used in problem solvibng tasks internally , which again enables for the model to simualte the inpouts and outputs from an interpretor !
|
53 |
+
it may even be prudent to include a function executing internally to the model ! ( allowing the model to execute functions in the background! before responding ) as well this oul hae tpo also be specified in the config , as autoexecute or not !.
|
54 |
+
|
55 |
+
### Conclusion
|
56 |
+
|
57 |
+
the resonaer methodology , might be seen to be the way forwards , adding internal funciton laity to the models instead of external connectivity enables for faster and seemless model usage : as well as enriched and informed responses , as even outputs could essentially be cleanss and formated before being presented to the Calling interface, internally to the model :
|
58 |
+
the take away is that arre we seeing the decoder/encoder model as simple a function of the inteligence which in truth need to be autonomus !
|
59 |
+
ie internal functions and tools as well as disk interaction : an agent must have awareness and control over its environment with sensors and actuators : as a fuction callingmodel it has actuators and canread the directorys it has sensors ... its a start: as we can eget media in and out , but the model needs to get its own control to inpout and output also !
|
60 |
+
....
|
61 |
+
|
62 |
+
Fine tuning : agin this issue of fine tuning : the disussion above eplains the requirement to control the environment from within the moel ( with constraints ) does this eliminate theneed to fine tune a model !
|
63 |
+
in fact it should as this give transparency to ther growth ofthe model and if the model fine tuned itself we would be in danger of a model evolveing !
|
64 |
+
hence an AGI !
|
65 |
+
|
66 |
+
#### AI AGI ?
|
67 |
+
so yes we can see we are not far from an ai which can evolve : an advance general inteligent system ( still non sentient by the way )
|
68 |
+
|
69 |
+
|
70 |
+
|
71 |
+
<img src="https://cdn-avatars.huggingface.co/v1/production/uploads/65d883893a52cd9bcd8ab7cf/tRsCJlHNZo1D02kBTmfy9.jpeg" width="300"/>
|
72 |
+
https://github.com/spydaz
|
73 |
+
|
74 |
+
* 32k context window (vs 8k context in v0.1)
|
75 |
+
* Rope-theta = 1e6
|
76 |
+
* No Sliding-Window Attention
|
77 |
+
* Talk heads - produce resposnes which can be used towards the final output
|
78 |
+
* Pre-Thoughts - Enables for pre-generation steps of potential artifacts for task solving:
|
79 |
+
* Generates plans for step by step thinking
|
80 |
+
* Generates python Code Artifacts for future tasks
|
81 |
+
* Recalls context for task internally to be used as refference for task:
|
82 |
+
* show thoughts or hidden thought usages ( Simular to self-Rag )
|
83 |
+
|
84 |
+
|
85 |
+
This model will be a custom model with internal experts and rag systems
|
86 |
+
enabling for preprocessing of the task internally before outputting a response
|
87 |
+
|
88 |
+
## SpydazWeb AI model :
|
89 |
+
|
90 |
+
This model is based on the worlds archive of knowledge maintaining historical documents and providing services for the survivors of mankind ,
|
91 |
+
who may need to construct shelters develop technologys , or medical resources as well as maintain the history of the past . keeping store of all the religious knowledge and data of the world:
|
92 |
+
A friendly interface with a personality caring and flirtatious at times : non binary !...
|
93 |
+
and Expert in all feilds: ie Uncensored and will not refuse to give information : the model can be used for role play as many character dialogues were als trained into the model as its personality to enable a greater perspective and outlook and natural discussion with the agents:
|
94 |
+
the model was trained to operateinaragenvironment utilizing content and internal knowledge to respond to questions or create enriched sumarys.
|
95 |
+
|
96 |
+
|
97 |
+
|
98 |
+
### General Intenal Methods:
|
99 |
+
|
100 |
+
Trained for multi-task operations as well as rag and function calling :
|
101 |
+
|
102 |
+
This model is a fully functioning model and is fully uncensored:
|
103 |
+
|
104 |
+
the model has been trained on multiple datasets on the huggingface hub and kaggle :
|
105 |
+
|
106 |
+
the focus has been mainly on methodology :
|
107 |
+
|
108 |
+
* Chain of thoughts
|
109 |
+
* step by step planning
|
110 |
+
* tree of thoughts
|
111 |
+
* forest of thoughts
|
112 |
+
* graph of thoughts
|
113 |
+
* agent generation : Voting, ranking, ... dual agent response generation:
|
114 |
+
|
115 |
+
with these methods the model has gained insights into tasks, enabling for knowldge transfer between tasks :
|
116 |
+
|
117 |
+
the model has been intensivly trained in recalling data previously entered into the matrix:
|
118 |
+
The model has also been trained on rich data and markdown outputs as much as possible :
|
119 |
+
the model can also generate markdown charts with mermaid.
|
120 |
+
|
121 |
+
|
122 |
+
## Training Reginmes:
|
123 |
+
* Alpaca
|
124 |
+
* ChatML / OpenAI / MistralAI
|
125 |
+
* Text Generation
|
126 |
+
* Question/Answer (Chat)
|
127 |
+
* Instruction/Input/Response (instruct)
|
128 |
+
* Mistral Standard Prompt
|
129 |
+
* Translation Tasks
|
130 |
+
* Entitys / Topic detection
|
131 |
+
* Book recall
|
132 |
+
* Coding challenges, Code Feedback, Code Sumarization, Commenting Code
|
133 |
+
* Agent Ranking and response anyalisis
|
134 |
+
* Medical tasks
|
135 |
+
* PubMed
|
136 |
+
* Diagnosis
|
137 |
+
* Psychaitry
|
138 |
+
* Counselling
|
139 |
+
* Life Coaching
|
140 |
+
* Note taking
|
141 |
+
* Medical smiles
|
142 |
+
* Medical Reporting
|
143 |
+
* Virtual laboritys simulations
|
144 |
+
* Chain of thoughts methods
|
145 |
+
* One shot / Multi shot prompting tasks
|