File size: 6,979 Bytes
871fbb1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 |
---
library_name: transformers
tags:
- finance
license: llama3
base_model: meta-llama/Meta-Llama-3-8B-Instruct
datasets:
- virattt/financial-qa-10K
language:
- en
pipeline_tag: text-generation
---
[](https://hf.co/QuantFactory)
# QuantFactory/Llama-3-8B-Instruct-Finance-RAG-GGUF
This is quantized version of [curiousily/Llama-3-8B-Instruct-Finance-RAG](https://huggingface.co/curiousily/Llama-3-8B-Instruct-Finance-RAG) created using llama.cpp
# Original Model Card
# Llama 3 8B Instruct (Financial RAG)
This model is a fine-tuned version of the original [Llama 3 8B Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) model
on 4000 examples from the [virattt/financial-qa-10K](https://huggingface.co/datasets/virattt/financial-qa-10K) dataset.
The model is fine-tuned using a LoRA adapter for RAG use cases. It is optimized to answer a question based on a context:
```txt
Answer the question:
{question}
Using the information:
{context}
```
## Usage
Load the model:
```py
MODEL_NAME = "curiousily/Llama-3-8B-Instruct-Finance-RAG"
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
MODEL_NAME,
device_map="auto"
)
pipe = pipeline(
task="text-generation",
model=model,
tokenizer=tokenizer,
max_new_tokens=128,
return_full_text=False,
)
```
Format the prompt (uses the original Instruct prompt format):
````py
prompt = """
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
Use only the information to answer the question<|eot_id|><|start_header_id|>user<|end_header_id|>
How much did the company's net earnings amount to in fiscal 2022?
Information:
```
Net earnings were $17.1 billion in fiscal 2022.
```<|eot_id|><|start_header_id|>assistant<|end_header_id|>
"""
````
And make a prediction:
```py
print(outputs[0]["generated_text"])
```
```
$17.1 billion
```
Here's a helper function to build your prompts:
```py
def create_test_prompt(data_row):
prompt = dedent(f"""
{data_row["question"]}
Information:
```
{data_row["context"]}
```
""")
messages = [
{"role": "system", "content": "Use only the information to answer the question"},
{"role": "user", "content": prompt},
]
return tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
```
Where `data_row` must be a dict:
```
data_row = {
"question": "...",
"context": "..."
}
```
## Sample Predictions
Here's a sample of the predictions from *trained* and *untrained* models
```txt
Example 1
answer: Delta Air Lines' agreements with its regional carriers typically last at least ten years with options for extensions. Delta controls operational aspects like scheduling and pricing.
trained: Delta Connection agreements typically last at least ten years with options for extensions.
untrained: According to the information, the terms of Delta Air Lines' agreements with its regional carriers through Delta Connection are:
1. The agreements typically last at least ten years.
2. There are options for extensions.
3. Delta controls major operational aspects like scheduling and pricing.
4. The regional carriers supply the services.
Example 2
answer: The company evaluates acquisition-related intangibles for impairment by comparing the asset's carrying amount to undiscounted future net cash flows expected from the asset. An impairment loss is recognized if the carrying amount exceeds the asset's recoverable amount.
trained: The company evaluates acquisition-related intangible and other long-lived assets for impairment whenever events or changes in circumstances indicate that the carrying amount may not be recoverable. This involves measuring the recoverability of the asset by comparing its carrying amount to the future undiscounted net cash flows expected to be generated by the asset group.
untrained: According to the information, the company uses the following criteria to determine whether an impairment loss should be recognized on acquisition-related intangible assets:
* Events or changes in circumstances that indicate the carrying amount of an asset may not be recoverable.
* A comparison of the carrying amount of an asset to future undiscounted net cash flows expected to be generated by the asset group.
In other words, the company evaluates whether the asset's carrying amount is recoverable by comparing it to the expected future cash flows, which involves assumptions about future prospects and computations of estimated future cash flows.
Example 3
answer: In the United States, the approval process for biosimilars is governed by the Public Health Service Act (PHSA) and the regulations implementing these statutes, specifically including provisions made under federal health care reform legislation enacted in March 2010.
trained: The Federal Food, Drug, and Cosmetic Act (the FFDCA) and the Public Health Service Act (PHSA)
untrained: The legal framework that governs the approval process for biosimilars in the United States is the Federal Food, Drug, and Cosmetic Act (FFDCA) and the Public Health Service Act (PHSA), as well as the regulations implementing these statutes.
Example 4
answer: Timothy S. Teter holds a B.S. degree in Mechanical Engineering from the University of California at Davis and a J.D. degree from Stanford Law School.
trained: B.S. in Mechanical Engineering from the University of California at Davis and a J.D. from Stanford Law School
untrained: According to the information, Timothy S. Teter holds:
1. A B.S. degree in Mechanical Engineering from the University of California at Davis.
2. A J.D. degree from Stanford Law School.
Example 5
answer: Beginning in fiscal year 2024, the company plans to exclude paused Connected Fitness subscriptions from its new 'Ending Paid Connected Fitness Subscriptions' metric and will treat a pause action as a churn event in its 'Average Net Monthly Paid Connected Fitness Subscription Churn' metric.
trained: Starting in fiscal year 2024, the company will no longer include paused Connected Fitness subscriptions in their Ending Paid Connected Fitness Subscriptions metric and will treat a pause action as a churn event in their Average Net Monthly Paid Connected Fitness Subscription Churn.
untrained: Starting in fiscal year 2024, the company will:
* No longer include paused Connected Fitness subscriptions in the Ending Paid Connected Fitness Subscriptions metric
* Treat a pause action as a churn event in the Average Net Monthly Paid Connected Fitness Subscription Churn
```
## License
Uses the original Llama 3 License.
A custom commercial license is available at: https://llama.meta.com/llama3/license
|