feat: Add "Embeddings" section to website navigation

src/_quarto.yml

@@ -31,6 +31,8 @@ website:
             text: "Prompting"
           - href: theory/chainoftoughts.qmd
             text: "Chain of toughts"
+          - href: llms/embeddings.qmd
+            text: "Embeddings"
           - href: llms/index.qmd
             text: "LLM'xs"
 
@@ -44,6 +46,7 @@ website:
               text: "Advanced RAG"
             - href: notebooks/rag_evaluation.qmd
               text: "RAG Evaluation"
+
         - section: "Additional Techniques"
           contents:
             - href: notebooks/automatic_embedding.ipynb
@@ -53,6 +56,15 @@ website:
             - href: notebooks/single_gpu.ipynb
               text: "Single GPU Optimization"
 
+      - section: "Computer Vision"
+        contents:
+          - href: notebooks/automatic_embedding.ipynb
+            text: "Automatic Embedding"
+          - href: notebooks/faiss.ipynb
+            text: "FAISS for Efficient Search"
+          - href: notebooks/single_gpu.ipynb
+            text: "Single GPU Optimization"
+
 format:
   html:
     theme: cosmo

src/index.qmd

@@ -1,84 +1,11 @@
-qu---
-title: "About Quarto"
+---
+title: "About My AI Cookbook"
 ---
 
-[Quarto](https://quarto.org/) is a Markdown-based documentation system that lets you write documents in Markdown or Jupyter Notebooks, and render them to a variety of formats including HTML, PDF, PowerPoint, and more. 
-You can also use Quarto to write [books](https://quarto.org/docs/books/), create [dashboards](https://quarto.org/docs/dashboards/), and embed web applications with [Observable](https://quarto.org/docs/interactive/ojs/) and [Shinylive](https://quarto.org/docs/blog/posts/2022-10-25-shinylive-extension/).
-```{mermaid}
-flowchart LR
-  A[Hard edge] --> B(Round edge)
-  B --> C{Decision}
-  C --> D[Result one]
-  C --> E[Result two]
-```
-## Getting started with Quarto
-
-Once you've created the space, click on the `Files` tab in the top right to take a look at the files which make up this Space. 
-There are a couple of important files which you should pay attention to:
-
-- `Dockerfile`: This contains the system setup to build and serve the Quarto site on Hugging Face. You probably won't need to change this file that 
-often unless you need to add additional system dependencies or modify the Quarto version.
-- `requirements.txt`: This is where you should include any Python dependencies which you need for your website. 
-These are installed when the Dockerfile builds.
-- The `src` directory contains the source files for the Quarto website. You can include Jupyter notebooks or markdown (`.qmd` or `.md`) files. 
-- `src/_quarto.yml` defines the navigation for your website. If you want to add new pages or reorganize the existing ones, you'll need to change this file.
-
-
-## Recommended Workflow
-
-1. **Clone the space locally**
-2. **Install Quarto**: In order to render your Quarto site without Docker, we recommend installing Quarto by following the instructions on the [official Quarto website](https://quarto.org/docs/get-started/).
-3. **Install Quarto VS Code extension**: The [Quarto VS Code Extension](https://quarto.org/docs/tools/vscode.html) includes a number of productivity tools including YAML Autocomplete, a preview button, and a visual editor. Quarto works great with VS Code, but the extension does make it easier to get the most out of Quarto.
-4. **Edit the site**: The website files are contained in the `src` directory, and the site navigation is defined in `src/_quarto.yml`. Try editing these files and either clicking the "Preview" button in VS Code, or calling `quarto preview src` from the command line.
-5. **Learn more about Quarto**: You can do a lot of things with Quarto, and they are all documented on the [Quarto Website](https://quarto.org/guide/). In particular, you may be interested in:
-
-    - All about building [websites](https://quarto.org/docs/websites/)
-    - Building Static [Dashboards](https://quarto.org/docs/dashboards/)
-    - How to write [books](https://quarto.org/docs/books/index.html) and [manuscripts](https://quarto.org/docs/manuscripts/)
-    - Reproducible [presentations](https://quarto.org/docs/presentations/)
-    - Including [Observable](https://quarto.org/docs/interactive/ojs/) or [Shiny](https://quarto.org/docs/interactive/shiny/) applications in your Quarto site
-
-::: {.callout-warning}
-It can take a couple of minutes for the Space to deploy to Hugging Face after the Docker build process completes. Two see your changes you will need to do two things: 
-
-1) Wait for your space's status to go from 'Building' to 'Running'(this is visible in the status bar above the Space)
-2) Force-reload the web page by holding Shift and hitting the reload button in your browser.
-:::
-
-## Code Execution
-
-One of the main virtues of Quarto is that it lets you combine code and text in a single document.
-By default, if you include a code chunk in your document, Quarto will execute that code and include the output in the rendered document.
-This is great for reproducibility and for creating documents that are always up-to-date.
-For example you can include code which generates a plot like this:
-
-```{python}
-import seaborn as sns
-import matplotlib.pyplot as plt
-
-# Sample data
-tips = sns.load_dataset("tips")
-# Create a seaborn plot
-sns.set_style("whitegrid")
-g = sns.lmplot(x="total_bill", y="tip", data=tips, aspect=2)
-g = g.set_axis_labels("Total bill (USD)", "Tip").set(xlim=(0, 60), ylim=(0, 12))
-
-plt.title("Tip by Total Bill")
-plt.show()
-```
-
-When the website is built the Python code will run and the output will be included in the document. 
-
-You can also include [inline code](https://quarto.org/docs/computations/inline-code.html) to insert computed values into text. 
-For example we can include the maximum tip value in the `tips` data frame like this: ``{python} tips['tip'].max()``. 
-You can control [code execution](https://quarto.org/docs/computations/execution-options.html), or [freeze code output](https://quarto.org/docs/projects/code-execution.html#freeze) to capture the output of long running computations. 
-
-
-## About the Open Source AI Cookbook
-
-To provide a realistic example of how Quarto can help you organize long-form documentation, 
-we've implemented the Hugging Face [Open-Source AI Cookbook](https://github.com/huggingface/cookbook) in Quarto. 
-The Open-Source AI Cookbook is a collection of notebooks illustrating practical aspects of building AI applications and solving various machine learning tasks using open-source tools and models.
-You can read more about it, or contribute your own Notebook on the [Github Repo](https://github.com/huggingface/cookbook)
+This repository is my personal collection of recipes and notebooks, documenting my journey of learning and exploring various aspects of Artificial Intelligence (AI). As a self-taught AI enthusiast, I created this cookbook to serve as a knowledge base, a "how-to" guide, and a reference point for my own projects and experiments.
 
+::: {.callout-tip}
+## Tip with Title
 
+This is an example of a callout with a title.
+:::

src/llms/embeddings.qmd

@@ -0,0 +1,53 @@
+# Understanding Embeddings in Large Language Models (LLMs)
+Embeddings in Large Language Models (LLMs) like GPT, BERT, and others, are a foundational component in the field of natural language processing (NLP). These embeddings transform words, phrases, or even longer texts into a vector space, capturing the semantic meaning that enables LLMs to perform a variety of language-based tasks with remarkable proficiency. This article focuses on the role of embeddings in LLMs, how they are generated, and their impact on the performance of these models.
+
+## What are Embeddings in LLMs?
+In the context of LLMs, embeddings are dense vector representations of text. Each vector aims to encapsulate aspects of linguistic meaning such as syntax, semantics, and context. Unlike simpler models that might use one-hot encoding, LLM embeddings map words or tokens to vectors in a way that reflects their semantic and contextual relationships.
+
+## Role of Embeddings in LLMs
+Embeddings are the input layer of LLMs, where each word or token from the input text is converted into vectors. These vectors are then processed by the model’s deeper layers to perform tasks such as text classification, question answering, translation, and more. Here’s how embeddings contribute to the functionality of LLMs:
+
+## Pre-trained Word Embeddings
+Many LLMs start with a layer of pre-trained word embeddings obtained from vast amounts of text data. These pre-trained embeddings encapsulate general language features before being fine-tuned on specific tasks.
+
+## Contextual Embeddings
+Advanced models like BERT and GPT use embeddings that adjust according to the context of a word in a sentence, differing from static word embeddings used in earlier models. This means the embedding for the word "bank" would differ when used in the context of a river compared to a financial institution.
+
+## Generating Embeddings in LLMs
+LLMs typically generate embeddings using one of two architectures:
+
+### Transformer-Based Models
+These models, including BERT and GPT, utilize the transformer architecture that processes all words or tokens in parallel. This architecture allows for the generation of contextual embeddings where the meaning of a word can dynamically change based on the surrounding words.
+
+### Autoencoder Models
+Some LLMs employ autoencoders to generate embeddings that are then used to reconstruct the input data. This process helps in learning efficient representations of the data.
+
+## Applications of Embeddings in LLMs
+Embeddings in LLMs enable a wide range of applications:
+
+**Machine Translation**: Understanding and translating languages by capturing contextual nuances.
+
+**Sentiment Analysis**: Determining the sentiment of text by understanding the contextual meaning of words.
+
+**Content Recommendation**: Suggesting relevant articles or products by understanding textual descriptions.
+
+**Automated Question Answering**: Providing answers to questions based on understanding the context of the content.
+
+## Benefits of Embeddings in LLMs
+
+**Efficiency**: Embeddings significantly reduce the dimensionality of text data, making it manageable for neural networks.
+
+**Effectiveness**: By capturing deep semantic properties, embeddings allow models to perform complex language tasks.
+
+**Flexibility**: Once trained, the same embeddings can be used across different tasks and domains, enhancing the model's utility.
+
+## Challenges with Embeddings in LLMs
+
+**Resource Intensive**: Training LLMs to generate useful embeddings requires substantial computational resources and data.
+
+**Bias and Fairness**: Embeddings can perpetuate or amplify biases present in the training data, leading to fairness issues in model outputs.
+
+**Interpretability**: It can be difficult to interpret what specific dimensions of an embedding represent, complicating model debugging and transparency.
+
+## Conclusion
+Embeddings are a crucial part of the infrastructure that powers LLMs, enabling these models to process language with a depth and nuance that was previously unachievable. As NLP continues to evolve, the techniques for generating and refining embeddings will remain key to unlocking even more sophisticated language understanding and generation capabilities.