- 
                Notifications
    
You must be signed in to change notification settings  - Fork 2.1k
 
Description
Is your feature request related to a problem? Please describe.
Tools don't properly pass pydantic schema for the objects in their functions.
For example, my tool takes the following as its input parameter
class ReportType(Enum):
    REPORT_1 = "report_1"
    REPORT_2 = "report_2"
    @classmethod
    def values(cls) -> list[str]:
        return [type.value for type in cls]
class ReportConfig(BaseModel):
    sheet_name: str = Field(description="Name of the sheet")
    requested_start_date: date = Field(description="Start date")
    requested_end_date: date = Field(description="End date")
    report_type: str = Field(
        description="Type of the report",
        enum=ReportType.values()
    )
If I call
ReportConfig.model_json_schema() to be passed into, say, OpenAI,
tools=[
        {
            "type": "function",
            "function": {
                "name": "report_work",
                "description": "Adds a report. The configuration must follow the ReportConfig schema.",
                "parameters": ReportConfig.model_json_schema(),
            },
        }
]
I get a correct call
ReportConfig(sheet_name='ABC', requested_start_date=datetime.date(2024, 1, 1), requested_end_date=datetime.date(2024, 1, 31), report_type='report_1')
with the correct tool call
{'properties': {'sheet_name': {'description': 'Name of the sheet', 'title': 'Sheet Name', 'type': 'string'}, 'requested_start_date': {'description': 'Start date', 'format': 'date', 'title': 'Requested Start Date', 'type': 'string'}, 'requested_end_date': {'description': 'End date', 'format': 'date', 'title': 'Requested End Date', 'type': 'string'}, 'report_type': {'description': 'Type of the report', 'enum': ['report_1', 'report_2'], 'title': 'Report Type', 'type': 'string'}}, 'required': ['sheet_name', 'requested_start_date', 'requested_end_date', 'report_type'], 'title': 'ReportConfig', 'type': 'object'}
However, when I use LiteLLMModel with a tool with the following inputs:
inputs = {
"report_config": {
"type": "object",
"description": "Configuration for the new report.",
"properties": ReportConfig.model_json_schema()['properties'],
},
}
I get the call with
{'report_config': {'sheet_name': 'ABC', 'requested_start_date': '2024-01-01', 'requested_end_date': '2024-01-31'}}
So first, required fields are not passed, which results in an incomplete request. Moreover, the enum is not properly passed as get_json_schema function in ..._hint_utils.py actually uses strange choice way of defining enums
Describe the solution you'd like
I want to be able to specify ReportConfig.model_json_schema() in my tool input as a parameter, for a complex nested model. This will ensure much more consistent tool calling
Is this not possible with the current options.
At least I am not aware how to pass the schema differently
Describe alternatives you've considered
The only alternative I see is to write my own Tool class and overwrite most of the functions there, but it's possible this won't be enough as the LiteLLMModel might work with tools in a very specific way.
Additional context
Basically, it would be good to have at least parity with openai/anthropic tool calls capabilities in tool arguments