interstellarninja Rocketknight1 HF staff commited on
Commit
78a3194
1 Parent(s): 7591a57

Add example of chat templates for function calling (#3)

Browse files

- Add example of chat templates for function calling (975d07b5c9e889b8f6d9c3c8d84dc245d3a87022)


Co-authored-by: Matthew Carrigan <Rocketknight1@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +45 -0
README.md CHANGED
@@ -131,6 +131,51 @@ The stock fundamentals data for Tesla (TSLA) are as follows:
131
  This information provides a snapshot of Tesla's financial position and performance based on the fundamental data obtained from the yfinance API. It shows that Tesla has a substantial market capitalization and a relatively high P/E and P/B ratio compared to other stocks in its industry. The company does not pay a dividend at the moment, which is reflected by a 'Dividend Yield' of 'None'. The Beta value indicates that Tesla's stock has a moderate level of volatility relative to the market. The 52-week high and low prices give an idea of the stock's range over the past year. This data can be useful when assessing investment opportunities and making investment decisions.<|im_end|>
132
  ```
133
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
134
  ## Prompt Format for JSON Mode / Structured Outputs
135
 
136
  Our model was also trained on a specific system prompt for Structured Outputs, which should respond with **only** a json object response, in a specific json schema.
 
131
  This information provides a snapshot of Tesla's financial position and performance based on the fundamental data obtained from the yfinance API. It shows that Tesla has a substantial market capitalization and a relatively high P/E and P/B ratio compared to other stocks in its industry. The company does not pay a dividend at the moment, which is reflected by a 'Dividend Yield' of 'None'. The Beta value indicates that Tesla's stock has a moderate level of volatility relative to the market. The 52-week high and low prices give an idea of the stock's range over the past year. This data can be useful when assessing investment opportunities and making investment decisions.<|im_end|>
132
  ```
133
 
134
+ ## Chat Templates for function calling
135
+
136
+ You can also use chat templates for function calling. For more information, please see the relevant section of the [chat template documentation](https://huggingface.co/docs/transformers/en/chat_templating#advanced-tool-use--function-calling).
137
+
138
+ Here is a brief example of this approach:
139
+
140
+ ```python
141
+ def multiply(a: int, b: int):
142
+ """
143
+ A function that multiplies two numbers
144
+
145
+ Args:
146
+ a: The first number to multiply
147
+ b: The second number to multiply
148
+ """
149
+ return int(a) * int(b)
150
+
151
+ tools = [multiply] # Only one tool in this example, but you probably want multiple!
152
+
153
+ model_input = tokenizer.apply_chat_template(
154
+ messages,
155
+ tools=tools
156
+ )
157
+ ```
158
+
159
+ The docstrings and type hints of the functions will be used to generate a function schema that will be read by the chat template and passed to the model.
160
+ Please make sure you include a docstring in the same format as this example!
161
+
162
+ If the model makes a tool call, you can append the tool call to the conversation like so:
163
+
164
+ ```python
165
+ tool_call_id = "vAHdf3" # Random ID, should be unique for each tool call
166
+ tool_call = {"name": "multiply", "arguments": {"a": "6", "b": "7"}}
167
+ messages.append({"role": "assistant", "tool_calls": [{"id": tool_call_id, "type": "function", "function": tool_call}]})
168
+ ```
169
+
170
+ Next, call the tool function and append the tool result:
171
+
172
+ ```python
173
+ messages.append({"role": "tool", "tool_call_id": tool_call_id, "name": "multiply", "content": "42"})
174
+ ```
175
+
176
+ And finally apply the chat template to the updated `messages` list and `generate()` text once again to continue the conversation.
177
+
178
+
179
  ## Prompt Format for JSON Mode / Structured Outputs
180
 
181
  Our model was also trained on a specific system prompt for Structured Outputs, which should respond with **only** a json object response, in a specific json schema.