Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 158 additions & 0 deletions ADD_MODEL_CHANGES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# Add Model Interface Changes

This document details all changes made as part of the **LiteLLM Takehome: Redesign Model Add** assignment. The changes improve UX, fix critical issues, and enhance the overall model configuration experience.

NOTE: These are mainly UI fixes, and backend compatibility isn't thoroughly tested (Especially for updated advanced settings)


### Specific Concerns Addressed Briefly (Detailed explanation below)

#### What is 'public model'?
Renamed this section, added more thorough tooltip, and reorganized layout to properly guide user and help them understand better

#### How does load balancing work?
Added link within grouped "Model Mappings" description to redirect users that may be unaware of the concept

#### What is the difference between public model and litellm model?
Renamed table headers slightly, altered empty table to prompt users through setup, introduced properly styled placeholder text (with appropriate default behavior) to let users know intuitively that Public Model field is editable

#### What is ‘Team-BYOK model’?
Organized into descriptive group, Improved tooltip information, and linked to URL (placeholder) that provides a more ample description

#### Advanced Users
None of these solutions are functional, they are simply prototypes
- Solution A displayed in Advanced Options, under Add Models tab. However, this tab doesn't seem like the appropriate place to display this, which is why there are two options.
- Solution B in All Models tab, simply added as an exrta field

# Model Mappings
## Reconfigured Layout
- **Issue**: It was not obvious that “Model Mappings” option was dependent on the previous two options (Provider & LiteLLM Models).
- **Solution**: Layout was changed to ensure that Provider and LiteLLM Models visually appear to be required for filing in the table. Table text now (in its empty state) also prompts users to follow the desired flow of actions.

## Provider Option
- **Issue**: It was not obvious that "Model Mappings" option was dependent on the previous two options (Provider & LiteLLM Models)
- **Solution**: Layout was changed to ensure that Provider and LiteLLM Models visually appear to be required for filing in the table. Table text now (in its empty state) also prompts users to follow the desired flow of actions


## LiteLLM Model Name(s) Option

### 1. Fixed Input Type When No Provider Selected
- **Issue**: Field showed as text input when no provider was selected, allowing users to enter invalid data before selecting a provider
- **Solution**: Field now starts disabled until provider is selected with dependency checks and disabled state management to prevent invalid input and clarify field dependencies

### 2. Implemented Hybrid Input Approach
- **Issue**: All providers treated the same way causing inconsistent UX between deployment-based vs model-based providers that require different input patterns
- **Solution**: Different input types based on provider requirements (Azure, OpenAI_Compatible, Ollama use TextInput for custom names; other providers use dropdown with predefined models) providing appropriate input method for each provider's architecture

### 3. Enhanced Provider-Specific Tooltips
- **Issue**: Generic tooltip didn't provide specific guidance for different providers, leaving users unclear about expected input format
- **Solution**: Dynamic tooltips with provider-specific instructions (Azure: deployment name examples, OpenAI_Compatible: endpoint model names, Ollama: model name examples, Others: selection guidance) providing clear, contextual guidance for each provider type

### 4. Removed Redundant Description Text
- **Issue**: Static description "The model name LiteLLM will send to the LLM API" created visual clutter with information already covered by tooltips
- **Solution**: Removed static description and enhanced tooltips instead, creating cleaner interface with better contextual help

### 5. Implemented Exclusive ALL Selection Logic
- **Issue**: Users could select "ALL [provider] Models" alongside individual models, creating confusing and conflicting configurations
- **Solution**: Mutual exclusivity between ALL and individual selections with detection logic that replaces ALL with individual selections and vice versa, plus smart dropdown closing for clear, predictable selection behavior preventing conflicts


## Model Mappings Table
### 1. Fixed Table Visibility with Wildcard Selection
- **Issue**: Table disappeared when "ALL [provider] Models" was selected, preventing users from seeing or configuring available models with wildcard
- **Solution**: Show table with all provider models when wildcard is selected by modifying visibility logic and adding `providerModels` prop, allowing users to see and customize public names for all models

### 2. Eliminated Redundant Text Duplication
- **Issue**: "Public Model Name" and "LiteLLM Model Name" showed duplicate information with unclear distinction between editable and display fields
- **Solution**: Made LiteLLM Model Name read-only display and Public Model Name editable with dynamic placeholder text and backend defaults, providing clear separation with proper default behavior

### 3. Fixed Table Clearing Issue
- **Issue**: Table retained stale data when model selection was cleared, causing users to see outdated mappings after clearing selections
- **Solution**: Enhanced clearing logic across all input handlers with detection for empty selections, updated text input handlers, and enhanced `useEffect` to handle empty arrays for consistent table clearing when selections are removed

### 4. Enhanced Table Visual Design
- **Issue**: Table headers blended with section background with poor visual separation, making it difficult to distinguish headers and scan table content
- **Solution**: Professional styling with alternating colors, clear boundaries, rounded corners, header styling, and fixed column widths for professional appearance with excellent readability

### 5. Fixed Auto-Fill Issues
- **Issue**: Browser auto-filled credential fields with saved passwords/emails, causing unexpected values to appear in API Key and Existing Credentials fields
- **Solution**: Comprehensive auto-fill prevention using `autoComplete="new-password"` for TextInput fields and `autoComplete="off"` for Select components, preventing unwanted browser auto-fill while preserving intentional defaults

### 6. Improved Empty State Display
- **Issue**: Table completely disappeared when no models selected, causing users to see blank space instead of helpful guidance
- **Solution**: Always show table structure with contextual empty states using Ant Design `Empty` component with dynamic messages for different scenarios, providing clear guidance on required actions with appropriate visual feedback


# Authentication & Connection

## Mode Option

### 1. Converted Descriptions to Tooltips
- **Issue**: Standalone description text created visual clutter, making interface feel crowded with unnecessary text blocks
- **Solution**: Moved descriptions to tooltips (Mode Field health check description and Existing Credentials selection guidance) for cleaner, more compact interface with contextual help on hover

### 2. Added Mode Opt-Out Option
- **Issue**: Users couldn't clear Mode selection once chosen despite field being optional, providing no way to reset optional field after selection
- **Solution**: Added "None (Default)" option with `allowClear` prop and placeholder indicating optional nature, allowing users to easily opt out of optional selections

## Credentials Option

### 1. Reorganized Credentials into Clear Categories
- **Issue**: Credentials split with confusing OR divider that was present whether API key was used or if existing credentials was used
- **Solution**: Split credentials into new and existing categories with clear visual separation, eliminating the confusing OR divider and making the relationship between options obvious

### 2. Clarified API Key Requirements
- **Issue**: API Key being required was confusing to users who weren't sure when it was needed
- **Solution**: Made it visually clear that API Key is only required UNLESS you choose from an existing credential, providing clear conditional requirements based on credential selection

## Additional Model Info Settings

### 1. Grouped Team-BYOK Model and Model Access Group
- **Issue**: Team-BYOK Model and Model Access Group settings were scattered and lacked visual organization
- **Solution**: Grouped these related settings together with clear visual separation and logical organization for better user understanding of model access controls

### 2. Updated Team-BYOK Model Switch Icon
- **Issue**: Switch icon for Team-BYOK Model didn't match the existing implementation used in advanced settings
- **Solution**: Replaced switch icon with icon that matches existing implementation in advanced settings for consistent visual design across the interface

## Advanced Settings

### 1. Grouped Settings into Distinct Categories
- **Issue**: Advanced settings were disorganized and difficult to navigate, lacking clear categorization
- **Solution**: Grouped settings into distinct categories with clear visual separation and logical organization for improved navigation and user understanding

### 2. Fixed Custom Pricing Position Issue
- **Issue**: Custom pricing was opening below Guardrails option instead of in the correct position
- **Solution**: Fixed positioning logic to ensure Custom pricing appears in the appropriate location within the settings hierarchy

### 3. Implemented Weight Management for Load Balancing
- **Issue**: The LiteLLM admin UI didn't support setting weight parameters for load balancing. While weights could be configured in YAML files, there were limitations when managing models through the UI - no weight field in model creation, models with weight in litellm_params became uneditable, and users couldn't modify existing model weights through the UI
- **Solution**:
- **Added Weight Section**: Created a dedicated "Manage Model Weights" section in Advanced Settings with an interactive table showing model configurations
- **Editable Weight Table**: Implemented a table with editable weight fields where users can directly modify individual model weights and see real-time updates to request distribution
- **Real-time Calculations**: Added dynamic calculation of "Requests handled by Model" percentages that update immediately when weights are changed
- **Dual Model Name Display**: Split model information into "Public Model (Custom Name)" and "LiteLLM Model Name" columns for clarity
- **Form Integration**: Integrated weight field with existing form submission logic so weights are properly included in litellm_params when models are created
- **Edit Model Support**: Added weight field to the edit model modal so existing model weights can be modified through the UI
- **Table Display**: Added weight column to the main model list table showing current weight values with proper default handling
- **Validation**: Implemented positive number validation for weight values with appropriate error handling




## User Experience Impact

### Before Changes
- Confusing field dependencies
- Inconsistent input methods across providers
- Poor visual hierarchy
- Browser auto-fill issues
- Conflicting selection states

### After Changes
- Clear dependency relationships
- Provider-appropriate input methods
- Professional visual design
- Stable, predictable behavior
- Comprehensive guidance system

1 change: 1 addition & 0 deletions litellm/proxy/_experimental/out/model_hub_table/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
<!DOCTYPE html><html id="__next_error__"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="preload" as="script" fetchPriority="low" href="/litellm-asset-prefix/_next/static/chunks/webpack-a426aae3231a8df1.js"/><script src="/litellm-asset-prefix/_next/static/chunks/fd9d1056-205af899b895cbac.js" async=""></script><script src="/litellm-asset-prefix/_next/static/chunks/117-a0da667066d322b6.js" async=""></script><script src="/litellm-asset-prefix/_next/static/chunks/main-app-475d6efe4080647d.js" async=""></script><title>LiteLLM Dashboard</title><meta name="description" content="LiteLLM Proxy Admin UI"/><link rel="icon" href="/favicon.ico" type="image/x-icon" sizes="16x16"/><link rel="icon" href="./favicon.ico"/><meta name="next-size-adjust"/><script src="/litellm-asset-prefix/_next/static/chunks/polyfills-42372ed130431b0a.js" noModule=""></script></head><body><script src="/litellm-asset-prefix/_next/static/chunks/webpack-a426aae3231a8df1.js" async=""></script><script>(self.__next_f=self.__next_f||[]).push([0]);self.__next_f.push([2,null])</script><script>self.__next_f.push([1,"1:HL[\"/litellm-asset-prefix/_next/static/media/e4af272ccee01ff0-s.p.woff2\",\"font\",{\"crossOrigin\":\"\",\"type\":\"font/woff2\"}]\n2:HL[\"/litellm-asset-prefix/_next/static/css/31b7f215e119031e.css\",\"style\"]\n3:HL[\"/litellm-asset-prefix/_next/static/css/8cbc26d5aab3552a.css\",\"style\"]\n"])</script><script>self.__next_f.push([1,"4:I[12846,[],\"\"]\n6:I[19107,[],\"ClientPageRoot\"]\n7:I[22775,[\"50\",\"static/chunks/50-d0da2dd7acce2eb9.js\",\"487\",\"static/chunks/487-79ed94231812dae7.js\",\"866\",\"static/chunks/866-9e1803a09e9ae8da.js\",\"154\",\"static/chunks/154-fff436ed72b19a24.js\",\"162\",\"static/chunks/162-76acc071cdf524f9.js\",\"172\",\"static/chunks/172-0f7049c565983c4d.js\",\"25\",\"static/chunks/app/model_hub_table/page-aef41992dbd4614b.js\"],\"default\",1]\n8:I[4707,[],\"\"]\n9:I[36423,[],\"\"]\nb:I[61060,[],\"\"]\nc:[]\n"])</script><script>self.__next_f.push([1,"0:[\"$\",\"$L4\",null,{\"buildId\":\"n6l06PC0-4ergWGnWvh0w\",\"assetPrefix\":\"/litellm-asset-prefix\",\"urlParts\":[\"\",\"model_hub_table\"],\"initialTree\":[\"\",{\"children\":[\"model_hub_table\",{\"children\":[\"__PAGE__\",{}]}]},\"$undefined\",\"$undefined\",true],\"initialSeedData\":[\"\",{\"children\":[\"model_hub_table\",{\"children\":[\"__PAGE__\",{},[[\"$L5\",[\"$\",\"$L6\",null,{\"props\":{\"params\":{},\"searchParams\":{}},\"Component\":\"$7\"}],null],null],null]},[null,[\"$\",\"$L8\",null,{\"parallelRouterKey\":\"children\",\"segmentPath\":[\"children\",\"model_hub_table\",\"children\"],\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$L9\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":\"$undefined\",\"notFoundStyles\":\"$undefined\"}]],null]},[[[[\"$\",\"link\",\"0\",{\"rel\":\"stylesheet\",\"href\":\"/litellm-asset-prefix/_next/static/css/31b7f215e119031e.css\",\"precedence\":\"next\",\"crossOrigin\":\"$undefined\"}],[\"$\",\"link\",\"1\",{\"rel\":\"stylesheet\",\"href\":\"/litellm-asset-prefix/_next/static/css/8cbc26d5aab3552a.css\",\"precedence\":\"next\",\"crossOrigin\":\"$undefined\"}]],[\"$\",\"html\",null,{\"lang\":\"en\",\"children\":[\"$\",\"body\",null,{\"className\":\"__className_b0dd8a\",\"children\":[\"$\",\"$L8\",null,{\"parallelRouterKey\":\"children\",\"segmentPath\":[\"children\"],\"error\":\"$undefined\",\"errorStyles\":\"$undefined\",\"errorScripts\":\"$undefined\",\"template\":[\"$\",\"$L9\",null,{}],\"templateStyles\":\"$undefined\",\"templateScripts\":\"$undefined\",\"notFound\":[[\"$\",\"title\",null,{\"children\":\"404: This page could not be found.\"}],[\"$\",\"div\",null,{\"style\":{\"fontFamily\":\"system-ui,\\\"Segoe UI\\\",Roboto,Helvetica,Arial,sans-serif,\\\"Apple Color Emoji\\\",\\\"Segoe UI Emoji\\\"\",\"height\":\"100vh\",\"textAlign\":\"center\",\"display\":\"flex\",\"flexDirection\":\"column\",\"alignItems\":\"center\",\"justifyContent\":\"center\"},\"children\":[\"$\",\"div\",null,{\"children\":[[\"$\",\"style\",null,{\"dangerouslySetInnerHTML\":{\"__html\":\"body{color:#000;background:#fff;margin:0}.next-error-h1{border-right:1px solid rgba(0,0,0,.3)}@media (prefers-color-scheme:dark){body{color:#fff;background:#000}.next-error-h1{border-right:1px solid rgba(255,255,255,.3)}}\"}}],[\"$\",\"h1\",null,{\"className\":\"next-error-h1\",\"style\":{\"display\":\"inline-block\",\"margin\":\"0 20px 0 0\",\"padding\":\"0 23px 0 0\",\"fontSize\":24,\"fontWeight\":500,\"verticalAlign\":\"top\",\"lineHeight\":\"49px\"},\"children\":\"404\"}],[\"$\",\"div\",null,{\"style\":{\"display\":\"inline-block\"},\"children\":[\"$\",\"h2\",null,{\"style\":{\"fontSize\":14,\"fontWeight\":400,\"lineHeight\":\"49px\",\"margin\":0},\"children\":\"This page could not be found.\"}]}]]}]}]],\"notFoundStyles\":[]}]}]}]],null],null],\"couldBeIntercepted\":false,\"initialHead\":[null,\"$La\"],\"globalErrorComponent\":\"$b\",\"missingSlots\":\"$Wc\"}]\n"])</script><script>self.__next_f.push([1,"a:[[\"$\",\"meta\",\"0\",{\"name\":\"viewport\",\"content\":\"width=device-width, initial-scale=1\"}],[\"$\",\"meta\",\"1\",{\"charSet\":\"utf-8\"}],[\"$\",\"title\",\"2\",{\"children\":\"LiteLLM Dashboard\"}],[\"$\",\"meta\",\"3\",{\"name\":\"description\",\"content\":\"LiteLLM Proxy Admin UI\"}],[\"$\",\"link\",\"4\",{\"rel\":\"icon\",\"href\":\"/favicon.ico\",\"type\":\"image/x-icon\",\"sizes\":\"16x16\"}],[\"$\",\"link\",\"5\",{\"rel\":\"icon\",\"href\":\"./favicon.ico\"}],[\"$\",\"meta\",\"6\",{\"name\":\"next-size-adjust\"}]]\n5:null\n"])</script></body></html>
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ const AddAutoRouterTab: React.FC<AddAutoRouterTabProps> = ({
value: model_group,
label: model_group,
})),
{ value: 'custom', label: 'Enter custom model name' }
{ value: 'custom', label: 'Enter LiteLLM model name' }
]}
style={{ width: "100%" }}
showSearch={true}
Expand Down Expand Up @@ -234,7 +234,7 @@ const AddAutoRouterTab: React.FC<AddAutoRouterTabProps> = ({
value: model_group,
label: model_group,
})),
{ value: 'custom', label: 'Enter custom model name' }
{ value: 'custom', label: 'Enter LiteLLM model name' }
]}
style={{ width: "100%" }}
showSearch={true}
Expand Down
Loading