test

Dockerfile

@@ -10,6 +10,7 @@ WORKDIR /app
 
 USER root
 RUN chmod -R 777 /app
+
 # Install minimal required build tools and dependencies for Playwright
 RUN apt-get update && apt-get install -y \
     gcc \
@@ -44,9 +45,10 @@ COPY . .
 # COPY config/config.yaml /app/config/config.yaml
 # COPY config/config.yaml /config/config.yaml
 # COPY agent_memory/jar3d_final_response_previous_run.txt /app/agent_memory/jar3d_final_response_previous_run.txt
-COPY agent_memory/jar3d_final_response_previous_run.txt /home/user/app/agent_memory/jar3d_final_response_previous_run.txt
 
 
+ENTRYPOINT ["entrypoint.sh"]
+
 # Expose the port Chainlit runs on
 EXPOSE 8000
 

README.md

@@ -30,7 +30,7 @@ Thanks John Adeojo, who brings this wonderful project to open source community!
 
 ## TODO
 [] fix "/end" meta expert 503 error,maybe we should "Retry".
-[] deploy to Huggingface
+[x] deploy to Huggingface
 
 
 [] Long-term memory.
@@ -63,16 +63,30 @@ In `offline_graph_rag` file, we combine similarity search with
 
 ## Table of Contents
 
-1. [Core Concepts](#core-concepts)
-2. [Prerequisites](#prerequisites)
-3. [Configuration](#configuration)
-   - [API Key Configuration](#api-key-configuration)
-   - [Endpoints Configuration](#endpoints-configuration)
-4. [Setup for Basic Meta Agent](#setup-for-basic-meta-agent)
-5. [Setup for Jar3d](#setup-for-jar3d)
-   - [Docker Setup for Jar3d](#docker-setup-for-jar3d)
-   - [Interacting with Jar3d](#interacting-with-jar3d)
-6. [Roadmap for Jar3d](#roadmap-for-jar3d)
+- [Super Expert](#super-expert)
+  - [Tech Stack](#tech-stack)
+  - [TODO](#todo)
+  - [PMF - What problem this project has solved?](#pmf---what-problem-this-project-has-solved)
+  - [Business Logics](#business-logics)
+    - [LLM Application Workflow](#llm-application-workflow)
+  - [Bullet points](#bullet-points)
+  - [FAQ](#faq)
+  - [Table of Contents](#table-of-contents)
+  - [Core Concepts](#core-concepts)
+  - [Prerequisites](#prerequisites)
+    - [Environment Setup](#environment-setup)
+  - [Repository Setup](#repository-setup)
+  - [Configuration](#configuration)
+    - [API Key Configuration](#api-key-configuration)
+    - [Endpoints Configuration](#endpoints-configuration)
+  - [Setup for Basic Meta Agent](#setup-for-basic-meta-agent)
+    - [Run Your Query in Shell](#run-your-query-in-shell)
+  - [Setup for Jar3d](#setup-for-jar3d)
+    - [Docker Setup for Jar3d](#docker-setup-for-jar3d)
+      - [Prerequisites](#prerequisites-1)
+      - [Quick Start](#quick-start)
+      - [Notes](#notes)
+    - [Interacting with Jar3d](#interacting-with-jar3d)
 
 ## Core Concepts
 

agent_memory/jar3d_final_response_previous_run.txt

@@ -1,131 +0,0 @@
-# Literature Review on the Current State of Large Language Models (LLMs)
-
-## Introduction
-
-Large Language Models (LLMs) have revolutionized the field of Natural Language Processing (NLP) by demonstrating unprecedented capabilities in language understanding and generation. These models have significantly impacted various domains, including machine translation, question-answering systems, and content creation. This literature review provides a comprehensive overview of the advancements in LLMs up to 2023, focusing on architecture developments, training techniques, ethical considerations, and practical applications.
-
-## Architecture Advancements
-
-### Transformer Architecture
-
-The introduction of the Transformer architecture by Vaswani et al. (2017) marked a pivotal moment in NLP. By utilizing self-attention mechanisms, Transformers addressed the limitations of recurrent neural networks, particularly in handling long-range dependencies and parallelization during training.
-
-### GPT Series
-
-OpenAI's Generative Pre-trained Transformer (GPT) series has been instrumental in pushing the boundaries of LLMs:
-
-- **GPT-2** (Radford et al., 2019): Featured 1.5 billion parameters and demonstrated coherent text generation, raising awareness about the potential and risks of LLMs.
-- **GPT-3** (Brown et al., 2020): Expanded to 175 billion parameters, exhibiting remarkable few-shot learning abilities and setting new benchmarks in NLP tasks.
-
-### Scaling Laws and Large-Scale Models
-
-Kaplan et al. (2020) established empirical scaling laws, showing that model performance scales predictably with computational resources, model size, and dataset size. This led to the development of even larger models:
-
-- **Megatron-Turing NLG 530B** (Smith et al., 2022): A collaboration between NVIDIA and Microsoft, this model contains 530 billion parameters, enhancing language generation capabilities.
-- **PaLM** (Chowdhery et al., 2022): Google's 540-billion-parameter model showcased state-of-the-art performance in reasoning and language understanding tasks.
-
-## Training Techniques
-
-### Unsupervised and Self-Supervised Learning
-
-LLMs are primarily trained using vast amounts of unlabelled text data through unsupervised or self-supervised learning, enabling them to learn language patterns without explicit annotations (Devlin et al., 2019).
-
-### Fine-Tuning and Transfer Learning
-
-Fine-tuning allows LLMs to adapt to specific tasks by training on smaller, task-specific datasets. Techniques like Transfer Learning have been crucial in applying general language understanding to specialized domains (Howard & Ruder, 2018).
-
-### Instruction Tuning and Prompt Engineering
-
-Wei et al. (2021) introduced instruction tuning, enhancing LLMs' ability to follow human instructions by fine-tuning on datasets with task instructions. Prompt engineering has emerged as a method to elicit desired behaviors from LLMs without additional training.
-
-### Reinforcement Learning from Human Feedback (RLHF)
-
-RLHF incorporates human preferences to refine model outputs, aligning them with human values and improving safety (Christiano et al., 2017).
-
-## Ethical Considerations
-
-### Bias and Fairness
-
-LLMs can inadvertently perpetuate biases present in their training data. Studies have highlighted issues related to gender, race, and cultural stereotypes (Bender et al., 2021). Efforts are ongoing to mitigate biases through data curation and algorithmic adjustments (Bolukbasi et al., 2016).
-
-### Misinformation and Content Moderation
-
-The ability of LLMs to generate plausible but incorrect or harmful content poses risks in misinformation dissemination. OpenAI has explored content moderation strategies and responsible deployment practices (Solaiman et al., 2019).
-
-### Privacy Concerns
-
-Training on large datasets may include sensitive information, raising privacy issues. Techniques like differential privacy are being explored to protect individual data (Abadi et al., 2016).
-
-### Transparency and Interpretability
-
-Understanding the decision-making processes of LLMs is challenging due to their complexity. Research into explainable AI aims to make models more interpretable (Danilevsky et al., 2020), which is critical for trust and regulatory compliance.
-
-## Applications
-
-### Healthcare
-
-LLMs assist in clinical documentation, patient communication, and research data analysis. They facilitate faster diagnosis and personalized treatment plans (Jiang et al., 2020).
-
-### Finance
-
-In finance, LLMs are used for algorithmic trading, risk assessment, and customer service automation, enhancing efficiency and decision-making processes (Yang et al., 2020).
-
-### Education
-
-Educational technologies leverage LLMs for personalized learning experiences, automated grading, and language tutoring, contributing to improved learning outcomes (Zawacki-Richter et al., 2019).
-
-### Legal Sector
-
-LLMs aid in legal document analysis, contract review, and summarization, reducing manual workloads and increasing accuracy (Bommarito & Katz, 2018).
-
-### Customer Service and Virtual Assistants
-
-Chatbots and virtual assistants powered by LLMs provide customer support, handle inquiries, and perform tasks, improving user engagement and satisfaction (Xu et al., 2020).
-
-## Conclusion
-
-Advancements in Large Language Models up to 2023 have significantly influenced AI and NLP, leading to models capable of understanding and generating human-like text. Progress in model architectures and training techniques has expanded their applicability across diverse industries. However, ethical considerations regarding bias, misinformation, and transparency remain critical challenges. Addressing these concerns is essential for the responsible development and deployment of LLMs. Future research is expected to focus on enhancing model efficiency, interpretability, and alignment with human values.
-
-## References
-
-- Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016). **Deep Learning with Differential Privacy.** *Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security*, 308-318.
-
-- Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). **On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?** *Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency*, 610-623.
-
-- Bolukbasi, T., Chang, K. W., Zou, J. Y., Saligrama, V., & Kalai, A. T. (2016). **Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings.** *Advances in Neural Information Processing Systems*, 4349-4357.
-
-- Bommarito, M. J., & Katz, D. M. (2018). **A Statistical Analysis of the Predictive Technologies of Law and the Future of Legal Practice.** *Stanford Technology Law Review*, 21, 286.
-
-- Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). **Language Models are Few-Shot Learners.** *Advances in Neural Information Processing Systems*, 33, 1877-1901.
-
-- Chowdhery, A., Narang, S., Devlin, J., et al. (2022). **PaLM: Scaling Language Modeling with Pathways.** *arXiv preprint* arXiv:2204.02311.
-
-- Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., & Amodei, D. (2017). **Deep Reinforcement Learning from Human Preferences.** *Advances in Neural Information Processing Systems*, 30.
-
-- Danilevsky, M., Qian, Y., Aharon, R., et al. (2020). **A Survey of the State of Explainable AI for Natural Language Processing.** *arXiv preprint* arXiv:2010.00711.
-
-- Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). **BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.** *Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics*, 4171-4186.
-
-- Howard, J., & Ruder, S. (2018). **Universal Language Model Fine-tuning for Text Classification.** *Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics*, 328-339.
-
-- Jiang, F., Jiang, Y., Zhi, H., et al. (2020). **Artificial Intelligence in Healthcare: Past, Present and Future.** *Stroke and Vascular Neurology*, 5(2), 230-243.
-
-- Kaplan, J., McCandlish, S., Henighan, T., et al. (2020). **Scaling Laws for Neural Language Models.** *arXiv preprint* arXiv:2001.08361.
-
-- Radford, A., Wu, J., Child, R., et al. (2019). **Language Models are Unsupervised Multitask Learners.** *OpenAI Blog*, 1(8).
-
-- Smith, S., Gray, J., Forte, S., et al. (2022). **Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model.** *arXiv preprint* arXiv:2201.11990.
-
-- Solaiman, I., Brundage, M., Clark, J., et al. (2019). **Release Strategies and the Social Impacts of Language Models.** *arXiv preprint* arXiv:1908.09203.
-
-- Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). **Attention is All You Need.** *Advances in Neural Information Processing Systems*, 30.
-
-- Wei, J., Bosma, M., Zhao, V., et al. (2021). **Finetuned Language Models Are Zero-Shot Learners.** *arXiv preprint* arXiv:2109.01652.
-
-- Xu, P., Liu, Z., Ou, C., & Li, W. (2020). **Leveraging Pre-trained Language Model in Machine Reading Comprehension with Multi-task Learning.** *Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing*, 226-236.
-
-- Yang, X., Yin, Z., & Li, Y. (2020). **Application of Artificial Intelligence in Financial Industry.** *Journal of Physics: Conference Series*, 1486(4), 042047.
-
-- Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). **Systematic Review of Research on Artificial Intelligence Applications in Higher Education – Where Are the Educators?** *International Journal of Educational Technology in Higher Education*, 16(1), 1-27.
-
----

models/llms.py

@@ -23,7 +23,7 @@ class BaseModel:
         self.retry_delay = retry_delay
 
     # @retry(stop=stop_after_attempt(3), wait=wait_fixed(1), retry=retry_if_exception_type(requests.RequestException))
-    @retry(stop=stop_after_attempt(5), wait=wait_exponential(multiplier=1, min=2, max=10), reraise=True)
+    @retry(stop=stop_after_attempt(5), wait=wait_exponential(multiplier=1, min=10, max=15), reraise=True)
     def _make_request(self, url, headers, payload):
 
         response = requests.post(url, headers=headers, data=json.dumps(payload))

tools/legacy/offline_graph_rag_tool copy.py

@@ -331,7 +331,7 @@ def create_graph_index(
     if os.environ.get('LLM_SERVER') == "openai":
         # require hundreds calls to api
         # we create index for every small chunk
-        llm = ChatOpenAI(temperature=0, model_name="gpt-4o-mini")
+        llm = ChatOpenAI(temperature=0, model_name="gpt-4o-mini-2024-07-18")
 
     else:
         llm = ChatAnthropic(temperature=0, model_name="claude-3-haiku-20240307")

tools/legacy/rag_tool.py

@@ -318,7 +318,7 @@ def create_graph_index(
     graph: Neo4jGraph = None,
     max_threads: int = 5
 ) -> Neo4jGraph:
-    llm = ChatOpenAI(temperature=0, model_name="gpt-4o-mini")
+    llm = ChatOpenAI(temperature=0, model_name="gpt-4o-mini-2024-07-18")
 
     # llm = ChatAnthropic(temperature=0, model_name="claude-3-haiku-20240307")
 

tools/offline_graph_rag_tool.py

@@ -324,7 +324,7 @@ def create_graph_index(
 ) -> Neo4jGraph:
     
     if os.environ.get('LLM_SERVER') == "openai":
-        llm = ChatOpenAI(temperature=0, model_name="gpt-4o-mini")
+        llm = ChatOpenAI(temperature=0, model_name="gpt-4o-mini-2024-07-18")
     else:
         llm = ChatAnthropic(temperature=0, model_name="claude-3-haiku-20240307")
 

tools/offline_graph_rag_tool_with_async.py

@@ -321,7 +321,7 @@ def create_graph_index(
 ) -> Neo4jGraph:
     
     if os.environ.get('LLM_SERVER') == "openai":
-        llm = ChatOpenAI(temperature=0, model_name="gpt-4o-mini")
+        llm = ChatOpenAI(temperature=0, model_name="gpt-4o-mini-2024-07-18")
     else:
         llm = ChatAnthropic(temperature=0, model_name="claude-3-haiku-20240307")