You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Oct 16, 2025. It is now read-only.
These scripts use the openai Python package to demonstrate how to use the OpenAI Chat Completions API.
8
24
In increasing order of complexity, the scripts are:
9
25
10
26
1.[`chat.py`](./chat.py): A simple script that demonstrates how to use the OpenAI API to generate chat completions.
@@ -17,15 +33,22 @@ Plus these scripts to demonstrate additional features:
17
33
*[`chat_safety.py`](./chat_safety.py): The simple script with exception handling for Azure AI Content Safety filter errors.
18
34
*[`chat_async.py`](./chat_async.py): Uses the async clients to make asynchronous calls, including an example of sending off multiple requests at once using `asyncio.gather`.
19
35
20
-
## Popular LLM libraries
36
+
### Function calling
37
+
38
+
These scripts demonstrate using the Chat Completions API "tools" (a.k.a. function calling) feature, which lets the model decide when to call developer-defined functions and return structured arguments instead of (or before) a natural language answer.
39
+
40
+
In all of these examples, a list of functions is declared in the `tools` parameter. The model may respond with `message.tool_calls` containing one or more tool calls. Each tool call includes the function `name` and a JSON string of `arguments` that match the declared schema. Your application is responsible for: (1) detecting tool calls, (2) executing the corresponding local / external logic, and (3) (optionally) sending the tool result back to the model for a final answer.
21
41
22
-
These scripts use popular LLM libraries to demonstrate how to use the OpenAI API with them:
42
+
Scripts (in increasing order of capability):
23
43
24
-
*[`chat_langchain.py`](./chat_langchain.py): Uses the Langchain package to generate chat completions. [Learn more from Langchain docs](https://python.langchain.com/docs/get_started/quickstart)
25
-
*[`chat_llamaindex.py`](./chat_llamaindex.py): Uses the LlamaIndex package to generate chat completions. [Learn more from LlamaIndex docs](https://docs.llamaindex.ai/en/stable/)
26
-
*[`chat_pydanticai.py`](./chat_pydanticai.py): Uses the PydanticAI package to generate chat completions. [Learn more from PydanticAI docs](https://ai.pydantic.dev/)
44
+
1.[`function_calling_basic.py`](./function_calling_basic.py): Declares a single `lookup_weather` function and prompts the model. It prints the tool call (if any) or falls back to the model's normal content. No actual function execution occurs.
45
+
2.[`function_calling_call.py`](./function_calling_call.py): Executes the `lookup_weather` function if the model requests it by parsing the returned arguments JSON and calling the local Python function.
46
+
3.[`function_calling_extended.py`](./function_calling_extended.py): Shows a full round‑trip: after executing the function, it appends a `tool` role message containing the function result and asks the model again so it can incorporate real data into a final user-facing response.
47
+
4.[`function_calling_multiple.py`](./function_calling_multiple.py): Exposes multiple functions (`lookup_weather`, `lookup_movies`) so you can see how the model chooses among them and how multiple tool calls could be returned.
27
48
28
-
## Retrieval-Augmented Generation (RAG)
49
+
You must use a model that supports function calling (such as the defaults `gpt-4o`, `gpt-4o-mini`, etc.). Some local or older models may not support the `tools` parameter.
50
+
51
+
### Retrieval-Augmented Generation (RAG)
29
52
30
53
These scripts demonstrate how to use the OpenAI API for Retrieval-Augmented Generation (RAG) tasks, where the model retrieves relevant information from a source and uses it to generate a response.
31
54
@@ -44,7 +67,7 @@ Then run the scripts (in order of increasing complexity):
44
67
*[`rag_documents_flow.py`](./rag_pdfs.py): A RAG flow that retrieves matching results from the local JSON file created by `rag_documents_ingestion.py`.
45
68
*[`rag_documents_hybrid.py`](./rag_documents_hybrid.py): A RAG flow that implements a hybrid retrieval with both vector and keyword search, merging with Reciprocal Rank Fusion (RRF), and semantic re-ranking with a cross-encoder model.
46
69
47
-
## Structured outputs with OpenAI
70
+
## Structured outputs
48
71
49
72
These scripts demonstrate how to use the OpenAI API to generate structured responses using Pydantic data models:
50
73
@@ -54,7 +77,7 @@ These scripts demonstrate how to use the OpenAI API to generate structured respo
54
77
*[`structured_outputs_function_calling.py`](./structured_outputs_function_calling.py): Demonstrates how to use functions defined with Pydantic for automatic function calling based on user queries.
55
78
*[`structured_outputs_nested.py`](./structured_outputs_nested.py): Uses nested Pydantic models to handle more complex structured responses, such as events with participants having multiple attributes.
56
79
57
-
## Setting up the environment
80
+
## Setting up the Python environment
58
81
59
82
If you open this up in a Dev Container or GitHub Codespaces, everything will be setup for you.
These scripts can be run with Azure OpenAI account, OpenAI.com, local Ollama server, or GitHub models,
73
-
depending on the environment variables you set.
96
+
depending on the environment variables you set. All the scripts reference the environment variables from a `.env` file, and an example `.env.sample` file is provided. Host-specific instructions are below.
74
97
75
-
1. Copy the `.env.sample` file to a new file called `.env`:
98
+
## Using GitHub Models
76
99
77
-
```bash
78
-
cp .env.sample .env
100
+
If you open this repository in GitHub Codespaces, you can run the scripts for free using GitHub Models without any additional steps, as your `GITHUB_TOKEN` is already configured in the Codespaces environment.
101
+
102
+
If you want to run the scripts locally, you need to set up the `GITHUB_TOKEN` environment variable with a GitHub [personal access token (PAT)](https://github.com/settings/tokens). You can create a PAT by following these steps:
103
+
104
+
1. Go to your GitHub account settings.
105
+
2. Click on "Developer settings" in the left sidebar.
106
+
3. Click on "Personal access tokens" in the left sidebar.
107
+
4. Click on "Tokens (classic)" or "Fine-grained tokens" depending on your preference.
108
+
5. Click on "Generate new token".
109
+
6. Give your token a name and select the scopes you want to grant. For this project, you don't need any specific scopes.
110
+
7. Click on "Generate token".
111
+
8. Copy the generated token.
112
+
9. Set the `GITHUB_TOKEN` environment variable in your terminal or IDE:
113
+
114
+
```shell
115
+
export GITHUB_TOKEN=your_personal_access_token
79
116
```
80
117
81
-
2. For Azure OpenAI, create an Azure OpenAI gpt-3.5 or gpt-4 deployment (perhaps using [this template](https://github.com/Azure-Samples/azure-openai-keyless)), and customize the `.env` file with your Azure OpenAI endpoint and deployment id.
118
+
10. Optionally, you can use a model other than "gpt-4o" by setting the `GITHUB_MODEL` environment variable. Use a model that supports functioncalling, such as: `gpt-4o`, `gpt-4o-mini`, `o3-mini`, `AI21-Jamba-1.5-Large`, `AI21-Jamba-1.5-Mini`, `Codestral-2501`, `Cohere-command-r`, `Ministral-3B`, `Mistral-Large-2411`, `Mistral-Nemo`, `Mistral-small`
You can run all examples in this repository using GitHub Models. If you want to run the examples using models from Azure OpenAI instead, you need to provision the Azure AI resources, which will incur costs.
123
+
124
+
This project includes infrastructure as code (IaC) to provision Azure OpenAI deployments of "gpt-4o" and "text-embedding-3-large". The IaC is defined in the `infra` directory and uses the Azure Developer CLI to provision the resources.
125
+
126
+
1. Make sure the [Azure Developer CLI (azd)](https://aka.ms/install-azd) is installed.
127
+
128
+
2. Login to Azure:
129
+
130
+
```shell
131
+
azd auth login
132
+
```
133
+
134
+
For GitHub Codespaces users, if the previous command fails, try:
135
+
136
+
```shell
137
+
azd auth login --use-device-code
138
+
```
139
+
140
+
3. Provision the OpenAI account:
141
+
142
+
```shell
143
+
azd provision
88
144
```
89
145
90
-
If you are not yet logged into the Azure account associated with that deployment, run this command to log in:
146
+
It will prompt you to provide an `azd` environment name (like "agents-demos"), selecta subscription from your Azure account, and selecta location. Then it will provision the resources in your account.
147
+
148
+
4. Once the resources are provisioned, you should now see a local`.env` file with all the environment variables needed to run the scripts.
149
+
5. To delete the resources, run:
91
150
92
151
```shell
93
-
az login
152
+
azd down
94
153
```
95
154
96
-
3. For OpenAI.com, customize the `.env` file with your OpenAI API key and desired model name.
155
+
156
+
## Using OpenAI.com models
157
+
158
+
1. Create a `.env` file by copying the `.env.sample` file and updating it with your OpenAI API key and desired model name.
97
159
98
160
```bash
99
-
API_HOST=openai
100
-
OPENAI_KEY=YOUR-OPENAI-API-KEY
101
-
OPENAI_MODEL=gpt-3.5-turbo
161
+
cp .env.sample .env
102
162
```
103
163
104
-
4. For Ollama, customize the `.env` file with your Ollama endpoint and model name (any model you've pulled).
164
+
2. Update the `.env` file with your OpenAI API key and desired model name:
105
165
106
166
```bash
107
-
API_HOST=ollama
108
-
OLLAMA_ENDPOINT=http://localhost:11434/v1
109
-
OLLAMA_MODEL=llama2
167
+
API_HOST=openai
168
+
OPENAI_API_KEY=your_openai_api_key
169
+
OPENAI_MODEL=gpt-4o-mini
110
170
```
111
171
112
-
If you're running inside the Dev Container, replace `localhost` with `host.docker.internal`.
172
+
## Using Ollama models
113
173
114
-
5. For GitHub models, customize the `.env` file with your GitHub model name.
174
+
1. Install [Ollama](https://ollama.com/) and follow the instructions to set it up on your local machine.
175
+
2. Pull a model, for example:
176
+
177
+
```shell
178
+
ollama pull llama3.1
179
+
```
180
+
181
+
3. Create a `.env` file by copying the `.env.sample` file and updating it with your Ollama endpoint and model name.
115
182
116
183
```bash
117
-
API_HOST=github
118
-
GITHUB_MODEL=gpt-4o
184
+
cp .env.sample .env
119
185
```
120
186
121
-
You'll need a `GITHUB_TOKEN` environment variable that stores a GitHub personal access token.
122
-
If you're running this inside a GitHub Codespace, the token will be automatically available.
123
-
If not, generate a new [personal access token](https://github.com/settings/tokens) and run this command to set the `GITHUB_TOKEN` environment variable:
187
+
4. Update the `.env` file with your Ollama endpoint and model name (any model you've pulled):
0 commit comments