feat: add OpenAI JSON Schema structured output support

[dbuos]

• Add automatic model detection for JSON schema support
• Implement graceful fallback to function calling for unsupported models
• Maintain full backward compatibility with existing APIs

Description

Overview

This PR implements support for OpenAI's native JSON Schema structured output capabilities while maintaining minimal architectural changes and leveraging the existing program pattern.

Approach

1. Minimal Integration Changes

• Leverages existing structured_predict() and astructured_predict() methods
• No changes to public APIs - fully backward compatible

Fixes # (issue)

New Package?

Did I fill in the tool.llamahub section in the pyproject.toml and provide a detailed README.md for my new integration or package?

• Yes
• No

Version Bump?

Did I bump the version in the pyproject.toml file of the package I am updating? (Except for the llama-index-core package)

• Yes
• No

Type of Change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Your pull-request will likely not be merged unless it is covered by some form of impactful unit testing.

• I added new unit tests to cover this change
• I believe this change is already covered by existing unit tests

Suggested Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added Google Colab support for the newly added notebooks.
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I ran uv run make format; uv run make lint to appease the lint gods

[dbuos]

@logan-markewich It seems that it is now ready to merge, I remove all the added program logic and keep the modification local to the base openAI class methods, also added some unit tests. Lmk what you think

[logan-markewich]

Do we actually need to import/use a private function? I thought the openai client let you directly pass in a pydantic object?

[logan-markewich]

ah this is probably because we are using the chat api?

[dbuos]

yea, this is indeed because I'm using the chat and achat APIs.

I’m also not entirely comfortable with calling that _type_to_response_format function directly. It might be safer in the long run to extract that logic into our own utility function within the repo...

do you think that could be the best approach here?

[logan-markewich]

eh, I think this is fine for now. In a future PR we can extract out that logic into a util

[pazevedo-hyland]

@dbuos would the idea be for this to have an interface with FunctionCalling agents while being called during Workflows?

[logan-markewich]

@pazevedo-hyland this is kind of separate from function agents for now

I do have an idea to add output class/structured responses to the FunctionAgent class though

1. We recently added tool_required as an arg for all function calling LLMs
2. From there, you could imagine an output_cls argument added to the FunctionAgent class. When set, we set tool_required=True and use the output class as a tool. The agent loop ends when this "tool" is called. This gets you a structured output with minimal LLM calls

From what I can see, openai does not let you mix response_format with tools (I might be wrong though), so the code in this PR isn't quite relevant

[dbuos]

@logan-markewich I just added support for the two streaming methods (sync and async)