Upload 6 files

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 anttiluode
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,3 +1,151 @@
----
-license: mit
----
+
+# Dynamic AI: Fractal Universe Chocolate Wafer Model (FUCWM) 
+
+### Watch Demo
+
+Check out the demo of the project on YouTube: [Watch Here](https://www.youtube.com/live/d__ras4nLU4)
+
+
+Dynamic AI is an experimental neural network model inspired by fractal structures in the universe and the human brain. It incorporates recursive nodes (FractalNodes) to dynamically grow and learn through Hebbian-like updates and pruning. The model also integrates a VAE (Variational Autoencoder) for encoding latent space representations. This repository contains the code for training, chatting, and interacting with the model via a Gradio interface.
+
+# Attention Mechanism (New)
+
+The attention mechanism dynamically adjusts the focus of the model by assigning importance to different child nodes in the fractal structure. Each child node receives an attention score based on its relevance, which is calculated using a softmax function. This allows the model to prioritize certain nodes over others during the forward pass, enabling more efficient learning and processing. Additionally, the model maintains a co-activation matrix that tracks how frequently different nodes are activated together, which further refines the attention scores. This approach enhances the model’s adaptability and helps manage complex hierarchical interactions.
+
+
+## Features
+
+- **Recursive Fractal Nodes**: Nodes can grow and create child nodes based on the complexity of their output, simulating the recursive, fractal-like nature of the brain and the universe.
+- **Variational Autoencoder (VAE)**: Encodes latent representations of inputs.
+- **Layer Normalization and Xavier Initialization**: Enhances training stability.
+- **Dynamic Complexity-based Growth**: Nodes grow based on complexity thresholds and manage child connections.
+- **Dynamic AI Chat**: Users can interact with the model to generate responses.
+- **LM Studio Integration**: Chat with a local LM Studio instance in a collaborative conversational framework.
+- **Gradio Interface**: A user-friendly interface to interact with the AI model, train it on Q&A pairs, and simulate conversations with LM Studio.
+
+## What it is?
+
+Think Fractal ball around big bang with chocolate wafer inspired super weights that add complexity to normal weights. 
+
+Now with added attention mechanism. I just asked Claude to think of the node 1 as a sort of phone book that keeps tabs 
+of the child nodes and can hooke em up if they fire together. So they can.. Wire together. Tsk Tsk. You know what I mean. 
+
+The depth setting can make the ball complexities explode to Nan territory real fast and there was a real fight to keep the 
+complexity setting at bay. 
+  
+## Requirements
+- Python 3.8+
+- PyTorch
+- Gradio
+- LM Studio (optional, for integration with the `talk_with_lm_studio` feature)
+- Etc
+The requirements.txt was written by ChatGPT. I have not tested if it would work as it is.
+
+## Problems?
+
+Ask from Claude / ChatGPT. Paste them this and the code. They will understand what to do. NotebookLM 
+talking heads think this is groundbreaking. But since all I hear is crickets. I guess it aint. But it most 
+def has been a wild ride. 
+
+## Installation
+
+1. Clone this repository:
+
+   ```bash
+   git clone https://github.com/anttiluode/DaFUC.git
+   cd dynamic-ai-fractal-wafer
+   ```
+
+2. Install dependencies:
+
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+3. If you're planning to use LM Studio, ensure it's installed and running locally. Configure the `lm_studio_client` by setting your API key and URL in the code.
+
+4. Run the application:
+
+   ```bash
+   python app.py
+   ```
+
+## Usage
+
+### 1. Chat with Dynamic AI
+
+You can use the Gradio interface to chat with the Dynamic AI model.
+
+- **Message**: Enter your message and adjust the temperature for creativity.
+- **Response**: The AI will generate a response based on its learned knowledge.
+
+### 2. Train the Model on Q&A Pairs
+
+You can train the model on a list of question-answer pairs using the Gradio interface.
+
+- **Q&A Pairs File**: Upload a JSON file containing question-answer pairs.
+- **Epochs**: Set the number of training epochs.
+- **Training Output**: Monitor the progress of training, including loss metrics.
+- This can lead to the complexity being wildly off and the model begins to parrot the words in the
+- question answer pairs. 
+
+### 3. LM Studio Conversation
+
+! You may have to wait a while for the conversation to start. I think perhaps there are 
+multiple empty interactions but eventually the model says something and lm studio grabs on to that. 
+If you teach the model with question answer pairs it sticks on to them and the complexity 
+does not stabilze. On the initial live training video I did at the beginning of this readme 
+something amazing happened. The complexity stabilized at 16 and did not budge. 
+
+You can simulate a collaborative conversation between Dynamic AI and LM Studio:
+
+- **Initial Message**: Set the initial message to start the conversation.
+- **Duration**: Set the duration of the conversation.
+- **Delay**: Set the delay between messages.
+
+This is a good start. The question answer pairs seem to produce more random AI. 
+
+### 4. Save/Load Model State
+
+You can save and load the state of the model using the Gradio interface.
+
+- **Save State**: Save the current model state to a file.
+- **Load State**: Load a previously saved state to restore the model.
+
+## Example
+
+To chat with Dynamic AI using the command line interface:
+
+```bash
+python app.py
+```
+
+Then, access the Gradio interface from your browser. You can interact with the AI by typing messages, training it, or saving/loading its state.
+
+### Training Data Format
+
+The Q&A pairs should be provided in a JSON file in the following format:
+
+```json
+[
+    {"question": "What is the capital of France?", "answer": "Paris"},
+    {"question": "Who wrote '1984'?", "answer": "George Orwell"}
+]
+```
+
+### Contribution
+
+Feel free to contribute to this project by submitting pull requests or opening issues for improvements or bugs.
+
+### Issues
+
+The depth settings are extremely important: 
+
+    dynamic_ai = DynamicAI(vocab_size=50000, embed_dim=256, latent_dim=256, output_dim=256, max_depth=7)
+
+    As the deeper it gets, the deeper the "fractal ball" around point one "Think big bang" gets, the more 
+    complex it gets. You hit NAN (out of reach) complexity very fast and the thing wont work. 
+
+### License
+
+This project is licensed under the MIT License.

app.py

@@ -0,0 +1,488 @@
+import torch
+import torch.nn as nn
+import torch.optim as optim
+import torch.nn.functional as F
+import random
+import json
+import logging
+import gradio as gr
+import time
+from openai import OpenAI
+
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+# VAE Class for Latent Space Encoding
+class VAE(nn.Module):
+    def __init__(self, input_dim, latent_dim):
+        super().__init__()
+        self.encoder = nn.Sequential(
+            nn.Linear(input_dim, 128),
+            nn.ReLU(),
+            nn.Linear(128, latent_dim * 2)
+        )
+        self.decoder = nn.Sequential(
+            nn.Linear(latent_dim, 128),
+            nn.ReLU(),
+            nn.Linear(128, input_dim)
+        )
+        self.latent_dim = latent_dim
+
+    def forward(self, x):
+        mu_logvar = self.encoder(x)
+        mu, logvar = torch.chunk(mu_logvar, 2, dim=-1)
+        z = self.sample_latent(mu, logvar)
+        recon = self.decoder(z)
+        return recon, mu, logvar, z
+
+    def sample_latent(self, mu, logvar):
+        std = torch.exp(0.5 * logvar)
+        eps = torch.randn_like(std)
+        return mu + eps * std
+
+    def vae_loss(self, recon_x, x, mu, logvar):
+        recon_loss = nn.MSELoss()(recon_x, x)
+        kld = -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
+        return recon_loss + kld
+
+# FractalNode class for FUCWM with Attention
+class FractalNode(nn.Module):
+    def __init__(self, input_dim, output_dim, depth=0, max_depth=5, max_children=2):
+        super().__init__()
+        self.traditional_weight = nn.Linear(input_dim, output_dim)
+        nn.init.xavier_uniform_(self.traditional_weight.weight)
+        self.superweight = nn.Parameter(torch.eye(output_dim))
+        self.norm = nn.LayerNorm(output_dim)
+        self._children = []
+        self.is_active = True
+        self.max_children = max_children
+        self.complexity_threshold = 0.5
+        self.depth = depth
+        self.max_depth = max_depth
+        self.attention_weights = nn.Parameter(torch.ones(max_children))
+
+    def forward(self, x):
+        if x.dim() == 1:
+            x = x.unsqueeze(0)
+
+        base_output = self.traditional_weight(x)
+        base_output = self.norm(base_output)
+        complexity = self.calculate_complexity(base_output)
+
+        if complexity > self.complexity_threshold and len(self._children) < self.max_children and self.depth < self.max_depth:
+            new_child = FractalNode(self.traditional_weight.out_features, self.traditional_weight.out_features, 
+                                    depth=self.depth+1, max_depth=self.max_depth)
+            self._children.append(new_child)
+            self.add_module(f'child_{len(self._children)}', new_child)
+
+        modulated_output = torch.matmul(self.superweight, base_output.unsqueeze(-1)).squeeze(-1)
+
+        for i, child in enumerate(self._children):
+            if child.is_active:
+                child_output = child(modulated_output)
+                modulated_output = modulated_output + child_output * F.softmax(self.attention_weights, dim=0)[i]
+
+        return modulated_output
+
+    def calculate_complexity(self, output):
+        return torch.log(1 + torch.norm(output))
+
+    def calculate_relevance(self, child_output):
+        return torch.sigmoid(torch.sum(child_output * self.superweight))
+
+    def update_superweights(self, context):
+        context_influence = torch.tanh(torch.matmul(self.superweight, context.unsqueeze(-1))).squeeze(-1)
+        self.superweight.data = self.superweight.data + 0.01 * context_influence
+        for child in self._children:
+            if child.is_active:
+                child.update_superweights(context)
+
+    def grow(self, complexity_threshold):
+        if self.calculate_complexity(self.traditional_weight.weight) > complexity_threshold and len(self._children) < self.max_children and self.depth < self.max_depth:
+            new_child = FractalNode(self.traditional_weight.out_features, self.traditional_weight.out_features, 
+                                    depth=self.depth+1, max_depth=self.max_depth)
+            self._children.append(new_child)
+            self.add_module(f'child_{len(self._children)}', new_child)
+        for child in self._children:
+            child.grow(complexity_threshold)
+
+    def update_attention(self, co_activation_vector):
+        self.attention_weights.data += co_activation_vector[:len(self._children)]
+        self.attention_weights.data = F.softmax(self.attention_weights, dim=0)
+
+    @property
+    def complexity(self):
+        return torch.norm(self.superweight)
+
+    @property
+    def children(self):
+        return self._children
+
+# FUCWM class with Attention
+class FUCWM(nn.Module):
+    def __init__(self, vocab_size, embed_dim, output_dim, max_depth=5):
+        super().__init__()
+        self.word_embeddings = nn.Embedding(vocab_size, embed_dim)
+        self.root = FractalNode(embed_dim, output_dim, max_depth=max_depth)
+        self.max_depth = max_depth
+        self.co_activation_matrix = torch.zeros((max_depth, max_depth))
+
+    def forward(self, x):
+        if x.dtype == torch.long:
+            embedded = self.word_embeddings(x)
+            if embedded.dim() == 3:
+                embedded = embedded.mean(dim=1)
+        else:
+            embedded = x
+        
+        output = self.root(embedded)
+        self.update_co_activations()
+        return output
+
+    def grow(self, complexity_threshold):
+        self.root.grow(complexity_threshold)
+
+    def update_superweights(self, context):
+        self.root.update_superweights(context)
+
+    def manage_padding(self):
+        def _manage_padding(node, depth):
+            if depth >= self.max_depth:
+                node.is_active = False
+            else:
+                activation = torch.norm(node.superweight)
+                if not node.is_active and activation > 0.5:
+                    node.is_active = True
+                elif node.is_active and activation < 0.1:
+                    node.is_active = False
+            for child in node.children:
+                _manage_padding(child, depth + 1)
+        _manage_padding(self.root, 0)
+
+    def update_co_activations(self):
+        for i in range(self.max_depth):
+            for j in range(self.max_depth):
+                if i != j:
+                    self.co_activation_matrix[i][j] += 0.1 * random.random()
+        
+        self.co_activation_matrix = F.softmax(self.co_activation_matrix, dim=1)
+
+    def update_attention_weights(self):
+        def update_node(node, depth):
+            node.update_attention(self.co_activation_matrix[depth])
+            for child in node.children:
+                update_node(child, depth+1)
+        
+        update_node(self.root, 0)
+
+class DynamicAI:
+    def __init__(self, vocab_size=10000, embed_dim=64, latent_dim=64, output_dim=64, max_depth=5):
+        self.vae = VAE(embed_dim, latent_dim)
+        self.model = FUCWM(vocab_size, embed_dim, output_dim, max_depth)
+        self.optimizer = optim.Adam(list(self.vae.parameters()) + list(self.model.parameters()), lr=0.0001)
+        self.scheduler = optim.lr_scheduler.StepLR(self.optimizer, step_size=5, gamma=0.1)
+        self.criterion = nn.MSELoss()
+        self.word_to_index = {}
+        self.index_to_word = {}
+        self.next_index = 0
+        self.lm_studio_client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")
+
+    def tokenize(self, text):
+        words = text.lower().split()
+        indices = []
+        for word in words:
+            if word not in self.word_to_index:
+                self.word_to_index[word] = self.next_index
+                self.index_to_word[self.next_index] = word
+                self.next_index += 1
+            indices.append(self.word_to_index[word])
+        return torch.tensor(indices, dtype=torch.long).unsqueeze(0)
+
+    def chat(self, input_text, max_length=20, temperature=0.7):
+        input_tokens = self.tokenize(input_text)
+        thinking_process = []
+        with torch.no_grad():
+            embedded_q = self.model.word_embeddings(input_tokens)
+            _, _, _, z_q = self.vae(embedded_q.mean(dim=1))
+            output, node_info = self.fractal_thinking(z_q)
+            thinking_process.append(node_info)
+
+        response = []
+        for _ in range(max_length):
+            output = output / temperature
+            probs = torch.softmax(output, dim=-1)
+            next_word_index = torch.multinomial(probs, 1).item()
+            next_word = self.index_to_word.get(next_word_index, "")
+            if next_word:
+                response.append(next_word)
+                if next_word in ['.', '!', '?']:
+                    break
+                next_token = self.tokenize(next_word)
+                _, _, _, next_latent = self.vae(self.model.word_embeddings(next_token).mean(dim=1))
+                output, node_info = self.fractal_thinking(next_latent)
+                thinking_process.append(node_info)
+            else:
+                break
+
+        thinking_str = "\n".join(thinking_process)
+        response_str = ' '.join(response)
+        return f"Thinking Process:\n{thinking_str}\n\nResponse: {response_str}"
+
+    def fractal_thinking(self, input_vector):
+        def traverse_node(node, x, depth):
+            node_info = f"Node depth: {depth}, Complexity: {node.complexity.item():.4f}, Children: {len(node.children)}"
+            output = node(x)
+            
+            if depth < node.max_depth:
+                for child in node.children:
+                    child_output, child_info = traverse_node(child, output, depth + 1)
+                    output = output + child_output * node.calculate_relevance(child_output)
+                    node_info += f"\n{child_info}"
+
+            return output, node_info
+
+        output, node_info = traverse_node(self.model.root, input_vector, 0)
+        return output, node_info
+
+    def talk_with_lm_studio(self, initial_message, conversation_duration=60, delay=2):
+        message = initial_message
+        start_time = time.time()
+        conversation_log = []
+
+        while time.time() - start_time < conversation_duration:
+            ai_response = self.chat(message)
+            logger.info(f"DynamicAI:\n{ai_response}")
+            conversation_log.append(f"DynamicAI:\n{ai_response}")
+            yield "\n\n".join(conversation_log)
+
+            ai_message = ai_response.split("Response: ")[-1].strip()
+
+            if not ai_message:
+                logger.info("DynamicAI generated an empty response. Skipping LM Studio turn.")
+                conversation_log.append("DynamicAI: [No response generated. Still learning...]")
+                yield "\n\n".join(conversation_log)
+                time.sleep(delay)
+                continue
+
+            lm_studio_response = self.send_to_lm_studio(ai_message)
+            if lm_studio_response:
+                logger.info(f"LM Studio: {lm_studio_response}")
+                conversation_log.append(f"LM Studio: {lm_studio_response}")
+                message = lm_studio_response
+                yield "\n\n".join(conversation_log)
+            else:
+                logger.warning("No response from LM Studio. Ending conversation.")
+                break
+            
+            time.sleep(delay)
+
+    def send_to_lm_studio(self, message):
+        if not message.strip():
+            logger.warning("Attempted to send an empty message to LM Studio. Skipping.")
+            return None
+
+        try:
+            completion = self.lm_studio_client.chat.completions.create(
+                model="unsloth/Llama-3.2-3B-Instruct-GGUF",
+                messages=[
+                    {"role": "system", "content": "You're talking to an experimental fractal AI that is still learning to communicate. If it doesn't respond or sends empty messages, please be patient and continue the conversation."},
+                    {"role": "user", "content": message}
+                ],
+                temperature=0.7,
+            )
+            response = completion.choices[0].message.content
+            return response
+        except Exception as e:
+            logger.error(f"Error sending to LM Studio: {str(e)}")
+            return None
+
+    def train_on_qa_pairs(self, qa_pairs, epochs=10):
+        if not isinstance(qa_pairs, list) or len(qa_pairs) == 0:
+            raise ValueError("qa_pairs must be a non-empty list")
+        
+        logger.info(f"Training on {len(qa_pairs)} Q&A pairs for {epochs} epochs...")
+        for epoch in range(epochs):
+            total_loss = 0
+            errors = 0
+            random.shuffle(qa_pairs)
+            for i, (question, answer) in enumerate(qa_pairs):
+                self.optimizer.zero_grad()
+                
+                try:
+                    q_tokens = self.tokenize(question)
+                    a_tokens = self.tokenize(answer)
+                    
+                    q_embedded = self.model.word_embeddings(q_tokens)
+                    _, _, _, q_latent = self.vae(q_embedded.mean(dim=1))
+                    
+                    a_embedded = self.model.word_embeddings(a_tokens)
+                    _, _, _, a_latent = self.vae(a_embedded.mean(dim=1))
+                    
+                    q_output = self.model(q_latent)
+                    a_output = self.model(a_latent)
+                    
+                    loss = self.criterion(q_output, a_output)
+                    loss.backward()
+
+                    torch.nn.utils.clip_grad_norm_(self.model.parameters(), max_norm=1.0)
+    
+                    self.optimizer.step()
+                    
+                    total_loss += loss.item()
+
+                    self.model.grow(complexity_threshold=0.5)
+                    self.model.update_superweights(q_output.detach())
+                    self.model.manage_padding()
+                    self.model.update_attention_weights()
+
+                    if i % 10 == 0:
+                        logger.info(f"Epoch {epoch+1}, Pair {i+1}/{len(qa_pairs)}, Loss: {loss.item():.4f}")
+
+                except Exception as e:
+                    logger.error(f"Error processing pair: {question} | {answer}")
+                    logger.error(f"Error details: {str(e)}")
+                    errors += 1
+                    continue
+
+            avg_loss = total_loss / (len(qa_pairs) - errors) if len(qa_pairs) > errors else 0
+            logger.info(f"Epoch {epoch+1}/{epochs}, Average Loss: {avg_loss:.4f}, Errors: {errors}")
+            
+            self.scheduler.step()
+            
+            self.save_state(f"model_state_epoch_{epoch+1}.pth")
+            
+            yield epoch + 1, avg_loss, errors
+
+    def save_state(self, filename):
+        state = {
+            'model_state': self.model.state_dict(),
+            'vae_state': self.vae.state_dict(),
+            'optimizer_state': self.optimizer.state_dict(),
+            'scheduler_state': self.scheduler.state_dict(),
+            'word_to_index': self.word_to_index,
+            'index_to_word': self.index_to_word,
+            'next_index': self.next_index
+        }
+        torch.save(state, filename)
+        logger.info(f"Model state saved to {filename}")
+
+    def load_state(self, filename):
+        state = torch.load(filename)
+        self.word_to_index = state['word_to_index']
+        self.index_to_word = state['index_to_word']
+        self.next_index = state['next_index']
+        
+        self.rebuild_model_structure(state['model_state'])
+        
+        self.model.load_state_dict(state['model_state'])
+        self.vae.load_state_dict(state['vae_state'])
+        self.optimizer.load_state_dict(state['optimizer_state'])
+        self.scheduler.load_state_dict(state['scheduler_state'])
+        
+        logger.info(f"Model state loaded from {filename}")
+
+    def rebuild_model_structure(self, state_dict):
+        def rebuild_node(node, prefix):
+            child_indices = set()
+            for name in state_dict.keys():
+                if name.startswith(prefix):
+                    parts = name[len(prefix):].split('.')
+                    if parts[0].startswith('child_'):
+                        child_index = int(parts[0].split('_')[1])
+                        child_indices.add(child_index)
+            
+            for index in sorted(child_indices):
+                while len(node._children) < index:
+                    new_child = FractalNode(node.traditional_weight.out_features, 
+                                            node.traditional_weight.out_features, 
+                                            depth=node.depth+1, 
+                                            max_depth=node.max_depth)
+                    node._children.append(new_child)
+                    node.add_module(f'child_{len(node._children)}', new_child)
+                
+                child_prefix = f"{prefix}child_{index}."
+                rebuild_node(node._children[index-1], child_prefix)
+
+        rebuild_node(self.model.root, "root.")
+
+    def grow(self, complexity_threshold):
+        self.model.grow(complexity_threshold)
+
+    def update_superweights(self, context):
+        self.model.update_superweights(context)
+
+    def manage_padding(self):
+        self.model.manage_padding()
+
+# Gradio Interface for DynamicAI
+def create_gradio_interface(ai):
+    def handle_chat(message, temperature):
+        return ai.chat(message, temperature=float(temperature))
+
+    def handle_save(filename):
+        ai.save_state(filename)
+        return f"State saved to {filename}"
+
+    def handle_load(filename):
+        ai.load_state(filename)
+        return f"State loaded from {filename}"
+
+    def handle_train_qa(qa_pairs_file, epochs):
+        try:
+            with open(qa_pairs_file.name, 'r', encoding='utf-8') as f:
+                qa_pairs = json.load(f)
+            
+            output = ["Starting training..."]
+            for epoch, loss, errors in ai.train_on_qa_pairs(qa_pairs, epochs=int(epochs)):
+                output.append(f"Epoch {epoch}/{epochs}, Loss: {loss:.4f}, Errors: {errors}")
+            output.append("Training completed successfully")
+            return "\n".join(output)
+        except Exception as e:
+            return f"Error during training: {str(e)}"
+
+    def handle_lm_studio_chat(initial_message, duration, delay):
+        conversation_log = gr.Textbox()
+        for log in ai.talk_with_lm_studio(initial_message, conversation_duration=float(duration), delay=float(delay)):
+            conversation_log = log
+            yield conversation_log
+
+    with gr.Blocks() as interface:
+        gr.Markdown("# Dynamic AI with Fractal Universe Chocolate Wafer Model and Attention Mechanism")
+
+        with gr.Tab("Chat"):
+            chat_input = gr.Textbox(label="Your message")
+            temperature = gr.Slider(minimum=0.1, maximum=2.0, value=0.7, label="Temperature")
+            chat_output = gr.Textbox(label="AI response")
+            chat_button = gr.Button("Chat")
+            chat_button.click(handle_chat, inputs=[chat_input, temperature], outputs=chat_output)
+
+        with gr.Tab("LM Studio Conversation"):
+            initial_message = gr.Textbox(label="Initial Message")
+            duration = gr.Number(label="Conversation Duration (seconds)", value=60)
+            delay = gr.Number(label="Delay between messages (seconds)", value=2)
+            conversation_log = gr.Textbox(label="Conversation Log", lines=20)
+            start_conversation = gr.Button("Start Conversation")
+            start_conversation.click(handle_lm_studio_chat, inputs=[initial_message, duration, delay], outputs=conversation_log)
+
+        with gr.Tab("Train on Q&A"):
+            qa_file = gr.File(label="Q&A Pairs JSON File")
+            epochs_input = gr.Number(label="Number of Epochs", value=10)
+            train_button = gr.Button("Train on Q&A Pairs")
+            train_output = gr.Textbox(label="Training status")
+            train_button.click(handle_train_qa, inputs=[qa_file, epochs_input], outputs=train_output)
+
+        with gr.Tab("Save/Load State"):
+            filename_input = gr.Textbox(label="Filename")
+            save_button = gr.Button("Save State")
+            load_button = gr.Button("Load State")
+            state_output = gr.Textbox(label="Operation result")
+            save_button.click(handle_save, inputs=filename_input, outputs=state_output)
+            load_button.click(handle_load, inputs=filename_input, outputs=state_output)
+
+    return interface
+
+# Main execution
+if __name__ == "__main__":
+    dynamic_ai = DynamicAI(vocab_size=50000, embed_dim=256, latent_dim=256, output_dim=256, max_depth=7)
+    iface = create_gradio_interface(dynamic_ai)
+    iface.launch()

notes.txt

@@ -0,0 +1,28 @@
+It is most interesting when you let LM studio talk to it from the start. 
+
+The depth settings (at the end of the code) can fast lead to explosive complexity growth. 
+
+"I" (Claude / ChatGPT) added a lot of things from actual AI's into it. Ie forward / backward pass, vae etc. Have fun.
+
+I do not really claim to understand it well. I had faint idea of fractals originating from big bang (Matter and galaxies are spread in fractal shape). 
+Or thoughts being processed in the brain and AI as fractals which has been the main stream idea since 1986.
+
+Interestingly most of the thinking is happening in the outer most layer as in "cortex". But perhaps that is natural as when you have a big lump
+of thinking nodes. Most of them are in the outer edge. 
+
+ The chocolate wafer model was a way to try to make normal ai model more dynamic, introducing a "super weight". Imagine wafer full of normal weights 
+and bunch of them constituting a super weight. That was somehow transferred to this code from a older code I did to try to make a normal AI dynamic.
+I was thinking, if it is impossible to change them all, how about we change clusters of them. 
+
+it was coded by Claude / ChatGPT, although it was like wrangling cats to make the dynamic ai to talk. The problem is that normal AI's deal with
+matrix multiplications etc that are known. In fractal, there is nothing that is known, it is growing ball of mathematics. We did scalars etc. 
+Tried to originate the fractals from the "big bang" and do forward and backward passes between the big bang and the outer layers. I do not 
+claim to understand what it is that was coded in yet myself either. 
+
+It was end result of a weird week.
+
+Most of the problems came from trying to keep the complexities at bay. We had it grow in complexity to Nan very fast at some settings. 
+
+LM Studio chat with it makes it seem almost like it makes sense. Teaching the json question answer pairs it is answering in jumbled mess. 
+
+When you talk to it for the first time. It is like big bang. The first fractal appears.. :D 

questionanswerpairsample.json

@@ -0,0 +1,54 @@
+[
+    ["What is the largest planet in our solar system?", "Jupiter is the largest planet in our solar system, known for its massive size and prominent storm, the Great Red Spot."],
+    ["Who was the first President of the United States?", "George Washington was the first President of the United States, serving from 1789 to 1797."],
+    ["What is the speed of light?", "The speed of light is approximately 299,792 kilometers per second (186,282 miles per second) in a vacuum."],
+    ["What is the capital of Japan?", "The capital of Japan is Tokyo, a major cultural, political, and economic hub in the world."],
+    ["Who developed the theory of relativity?", "Albert Einstein developed the theory of relativity, which revolutionized the understanding of space, time, and gravity."],
+    ["What is the primary gas found in Earth's atmosphere?", "Nitrogen makes up about 78% of Earth's atmosphere, making it the most abundant gas."],
+    ["Which artist painted the Mona Lisa?", "Leonardo da Vinci painted the Mona Lisa, one of the most famous artworks in history."],
+    ["What is the smallest unit of matter?", "The atom is the smallest unit of matter that retains the properties of an element."],
+    ["Who discovered penicillin?", "Alexander Fleming discovered penicillin in 1928, which led to the development of antibiotics."],
+    ["What is the chemical symbol for gold?", "The chemical symbol for gold is Au, derived from its Latin name 'Aurum'."],
+    ["What is the largest organ in the human body?", "The skin is the largest organ in the human body, protecting internal organs and regulating body temperature."],
+    ["Who wrote '1984'?", "George Orwell wrote '1984', a dystopian novel about a totalitarian regime and surveillance state."],
+    ["What is the freezing point of water?", "The freezing point of water is 0 degrees Celsius (32 degrees Fahrenheit)."],
+    ["Which country has the most populous democracy?", "India is the most populous democracy in the world, with over a billion citizens."],
+    ["What is the hardest natural substance on Earth?", "Diamond is the hardest natural substance, formed under extreme pressure and temperature conditions."],
+    ["What is the largest desert on Earth?", "The Antarctic Desert is the largest desert in the world, covering about 14 million square kilometers."],
+    ["Who is the founder of Microsoft?", "Bill Gates co-founded Microsoft in 1975 with Paul Allen, creating one of the largest tech companies in the world."],
+    ["What is the tallest mountain in the world?", "Mount Everest, located in the Himalayas, is the tallest mountain on Earth, standing at 8,848 meters (29,029 feet)."],
+    ["What is the main ingredient in bread?", "Flour is the main ingredient in bread, combined with water, yeast, and salt."],
+    ["What is the symbol for iron on the periodic table?", "The chemical symbol for iron is Fe, derived from its Latin name 'Ferrum'."],
+    ["Who wrote 'The Odyssey'?", "Homer is traditionally credited with writing 'The Odyssey', an epic poem about the Greek hero Odysseus."],
+    ["What is the boiling point of water?", "The boiling point of water is 100 degrees Celsius (212 degrees Fahrenheit) at sea level."],
+    ["Who was the first human to travel into space?", "Yuri Gagarin, a Soviet cosmonaut, became the first human to travel into space in 1961."],
+    ["What is the largest mammal on Earth?", "The blue whale is the largest mammal on Earth, reaching lengths of up to 30 meters (98 feet)."],
+    ["What is the longest river in the world?", "The Nile is considered the longest river in the world, stretching about 6,650 kilometers (4,130 miles)."],
+    ["Who painted the ceiling of the Sistine Chapel?", "Michelangelo painted the ceiling of the Sistine Chapel between 1508 and 1512, depicting scenes from the Bible."],
+    ["What is the most abundant element in the universe?", "Hydrogen is the most abundant element in the universe, making up roughly 75% of all visible matter."],
+    ["What does DNA stand for?", "DNA stands for deoxyribonucleic acid, the molecule that carries genetic information in living organisms."],
+    ["Which country is known as the Land of the Rising Sun?", "Japan is often referred to as the Land of the Rising Sun, symbolizing its eastern location in Asia."],
+    ["What is the largest ocean on Earth?", "The Pacific Ocean is the largest ocean, covering more than 63 million square miles."],
+    ["Who was the first woman to win a Nobel Prize?", "Marie Curie was the first woman to win a Nobel Prize, receiving the award in Physics in 1903."],
+    ["What is the longest bone in the human body?", "The femur, or thigh bone, is the longest bone in the human body."],
+    ["What is the capital city of Canada?", "Ottawa is the capital city of Canada, located in the province of Ontario."],
+    ["Who invented the telephone?", "Alexander Graham Bell is credited with inventing the first practical telephone in 1876."],
+    ["What is the square root of 81?", "The square root of 81 is 9."],
+    ["Which planet is known as the Red Planet?", "Mars is known as the Red Planet due to its reddish appearance, caused by iron oxide on its surface."],
+    ["What is the main language spoken in Brazil?", "Portuguese is the official and most widely spoken language in Brazil."],
+    ["Who composed the music for 'The Four Seasons'?", "Antonio Vivaldi composed 'The Four Seasons', a set of violin concertos representing the seasons."],
+    ["What is the name of the longest running Broadway show?", "The longest running Broadway show is 'The Phantom of the Opera', which premiered in 1988."],
+    ["What is the smallest continent by land area?", "Australia is the smallest continent by land area, covering about 7.7 million square kilometers."],
+    ["Who discovered gravity?", "Sir Isaac Newton is credited with formulating the law of universal gravitation in the 17th century."],
+    ["What is the primary function of red blood cells?", "Red blood cells transport oxygen from the lungs to the rest of the body and carry carbon dioxide back to the lungs."],
+    ["Who was the first Emperor of China?", "Qin Shi Huang was the first Emperor of China, uniting the country in 221 BC."],
+    ["What is the chemical formula for water?", "The chemical formula for water is H2O, consisting of two hydrogen atoms and one oxygen atom."],
+    ["What is the largest species of shark?", "The whale shark is the largest species of shark, reaching lengths of up to 12 meters (40 feet)."],
+    ["Which country is home to the Great Barrier Reef?", "Australia is home to the Great Barrier Reef, the world's largest coral reef system."],
+    ["Who wrote 'Pride and Prejudice'?", "Jane Austen wrote 'Pride and Prejudice', a classic novel first published in 1813."],
+    ["What is the hottest planet in the solar system?", "Venus is the hottest planet in the solar system, with surface temperatures over 450°C (840°F) due to its thick atmosphere."],
+    ["What is the currency of Japan?", "The currency of Japan is the yen (¥)."],
+    ["Which element has the atomic number 1?", "Hydrogen has the atomic number 1, making it the first element on the periodic table."],
+    ["Who painted 'Starry Night'?", "Vincent van Gogh painted 'Starry Night' in 1889, one of his most famous works."],
+    ["What is the highest-grossing film of all time?", "As of 2024, 'Avatar' is the highest-grossing film of all time, directed by James Cameron."]
+]

requirements.txt

@@ -0,0 +1,9 @@
+torch
+gradio
+openai==1.41
+transformers
+numpy           
+scipy           
+protobuf        
+requests        
+jsonschema