Keep the original casing of model name for custom endpoint to enable Foundry Local #413
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Summary
Fix #410
This PR contains the following 2 fixes:
OpenAI models require lower case for their names, but the custom endpoint may be case-sensitive, such as the endpoint set up by Foundry Local. So, we should keep the original casing for custom endpoint configurations.
Use Foundry Local with
openai-gpt
agentUpdate the
openai.agent.json
file with the following GPT configuration:In PowerShell, run the following commands to start up the Foundry service and load the model (use
phi-3.5-mini
as an example)Be noted that:
foundry service start
plus/v1
.foundry service ps
command."OPENAI_API_KEY"
.References
Foundry Local CLI: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/reference/reference-cli
Foundry Local SDK: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/reference/reference-sdk?pivots=programming-language-csharp