Initial Azure E2E Testing #13956
Open
Initial Azure E2E Testing #13956
+2,040
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Complete test coverage for all 4 Azure endpoints (/chat/completions, /messages, /responses, /batches) - 24 tests passing with full validation of OpenAI schema compliance - Automated test runner with environment setup and proxy management - Updated E2E testing roadmap marking Azure OpenAI as completed ✅ - Configured o4-mini model for Azure Responses API with 2025-03-01-preview - Comprehensive error handling and edge case testing - Ready for CI/CD integration
…test files - Replace hardcoded personal path with dynamic detection using BASH_SOURCE - Script now works for any user's litellm repository location - Move Azure test files from tests/azure_endpoints_tests/ to tests/openai_endpoints_tests/azure_endpoints_tests/ - Maintain proper directory structure for better organization
- Update AZURE_API_MODEL to use gpt-4.1 instead of gpt-4 - Configure all model aliases to use azure/gpt-4.1 deployment (except responses API) - Keep o4-mini for responses API tests only - Update max_tokens to 150+ across all tests for better token allocation - Switch API version to 2024-10-01-preview for GPT-4.1 compatibility - Fix streaming and function calling test compatibility with standard GPT behavior
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relevant issues
Initial PR for E2E testing of main routes and providers.
test_e2e_azure_batches.py - Azure Batches API validation
test_e2e_azure_chat_completions.py - Chat completions testing
test_e2e_azure_messages.py - Messages API functionality
test_e2e_azure_responses.py - Response format validation
Configuration & Documentation
azure_testing_config.yaml - LiteLLM proxy configuration for Azure
run_e2e_tests.sh - Main test runner script (improved in this PR)
README.md - Setup instructions and usage guide
E2E_Testing_Q4_2025_Roadmap.md - Testing roadmap and planned enhancements
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
📖 Documentation
🚄 Infrastructure
✅ Test
Changes