Skip to content

Conversation

daxian-dbw
Copy link
Member

@daxian-dbw daxian-dbw commented Aug 28, 2025

PR Summary

Fix #410

This PR contains the following 2 fixes:

  1. Keep the original casing of the model's name for custom endpoint configuration.
  2. Fix the error reporting to handle exception messages that already contains newline at the end.

OpenAI models require lower case for their names, but the custom endpoint may be case-sensitive, such as the endpoint set up by Foundry Local. So, we should keep the original casing for custom endpoint configurations.

Use Foundry Local with openai-gpt agent

  1. Update the openai.agent.json file with the following GPT configuration:

    {
      "GPTs": [
        {
          "Name": "foundry-local",
          "Description": "A GPT instance using Foundry Local.",
          "Endpoint": "http://127.0.0.1:56952/v1",
          "ModelName": "Phi-3.5-mini-instruct-generic-cpu",
          "Key": "OPENAI_API_KEY"
        }
      ]
    
      "Active": "foundry-local"
    }
  2. In PowerShell, run the following commands to start up the Foundry service and load the model (use phi-3.5-mini as an example)

    PS> foundry service start
    🟢 Service is already running on http://127.0.0.1:56952/.
    
    PS> foundry model load phi-3.5-mini
    🕔 Loading model...
    🟢 Model phi-3.5-mini loaded successfully
    
    PS> foundry service ps
    Models running in service:
         Alias                          Model ID
    🟢  phi-3.5-mini                   Phi-3.5-mini-instruct-generic-cpu
    

Be noted that:

  1. The endpoint to use is the URL shown from foundry service start plus /v1.
  2. The model name to use is shown as the 'Model ID' from the foundry service ps command.
    • Make sure you use the exact casing as shown in 'Model ID'. Foundry Local is case-sensitive.
  3. The API key is hardcoded to be literally "OPENAI_API_KEY".

References

Foundry Local CLI: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/reference/reference-cli
Foundry Local SDK: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-local/reference/reference-sdk?pivots=programming-language-csharp

@daxian-dbw daxian-dbw changed the title Keep the original casing of model name for custom endpoint Keep the original casing of model name for custom endpoint to enable Foundry Local Aug 28, 2025
@daxian-dbw daxian-dbw merged commit 9103bd8 into PowerShell:main Aug 28, 2025
4 checks passed
@daxian-dbw daxian-dbw deleted the foundry-local branch August 28, 2025 01:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Foundry-Local as agent
2 participants