Skip to content

Commit 13c39f1

Browse files
committed
Add mutual exclusivity validation for maxTokens and maxCompletionTokens in OpenAI ChatOptions
- Add SLF4J logger to OpenAiChatOptions class for validation warnings - Implement 'last-set-wins' validation in builder methods for maxTokens() and maxCompletionTokens() - Add javadoc with model-specific usage guidance: * maxTokens: Use for non-reasoning models (gpt-4o, gpt-3.5-turbo) * maxCompletionTokens: Required for reasoning models (o1, o3, o4-mini series) - Add 8 unit tests covering mutual exclusivity scenarios: * Validation when setting conflicting parameters * Null value handling (no validation triggered) * Individual parameter setting * Direct setter behavior (no validation enforced) - Fix existing unit tests that set both parameters simultaneously - Update OpenAI documentation with detailed usage patterns and examples - Add model compatibility table and builder validation examples This ensures OpenAI API compatibility by preventing simultaneous use of mutually exclusive token limit parameters, matching the robust validation already implemented for Azure OpenAI integration. Signed-off-by: Mark Pollack <[email protected]>
1 parent 441d005 commit 13c39f1

File tree

3 files changed

+224
-7
lines changed

3 files changed

+224
-7
lines changed

models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiChatOptions.java

Lines changed: 85 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,9 @@
3030
import com.fasterxml.jackson.annotation.JsonInclude.Include;
3131
import com.fasterxml.jackson.annotation.JsonProperty;
3232

33+
import org.slf4j.Logger;
34+
import org.slf4j.LoggerFactory;
35+
3336
import org.springframework.ai.model.ModelOptionsUtils;
3437
import org.springframework.ai.model.tool.ToolCallingChatOptions;
3538
import org.springframework.ai.openai.api.OpenAiApi;
@@ -55,6 +58,8 @@
5558
@JsonInclude(Include.NON_NULL)
5659
public class OpenAiChatOptions implements ToolCallingChatOptions {
5760

61+
private static final Logger logger = LoggerFactory.getLogger(OpenAiChatOptions.class);
62+
5863
// @formatter:off
5964
/**
6065
* ID of the model to use.
@@ -84,13 +89,31 @@ public class OpenAiChatOptions implements ToolCallingChatOptions {
8489
*/
8590
private @JsonProperty("top_logprobs") Integer topLogprobs;
8691
/**
87-
* The maximum number of tokens to generate in the chat completion. The total length of input
88-
* tokens and generated tokens is limited by the model's context length.
92+
* The maximum number of tokens to generate in the chat completion.
93+
* The total length of input tokens and generated tokens is limited by the model's context length.
94+
*
95+
* <p><strong>Model-specific usage:</strong></p>
96+
* <ul>
97+
* <li><strong>Use for non-reasoning models</strong> (e.g., gpt-4o, gpt-3.5-turbo)</li>
98+
* <li><strong>Cannot be used with reasoning models</strong> (e.g., o1, o3, o4-mini series)</li>
99+
* </ul>
100+
*
101+
* <p><strong>Mutual exclusivity:</strong> This parameter cannot be used together with
102+
* {@link #maxCompletionTokens}. Setting both will result in an API error.</p>
89103
*/
90104
private @JsonProperty("max_tokens") Integer maxTokens;
91105
/**
92106
* An upper bound for the number of tokens that can be generated for a completion,
93107
* including visible output tokens and reasoning tokens.
108+
*
109+
* <p><strong>Model-specific usage:</strong></p>
110+
* <ul>
111+
* <li><strong>Required for reasoning models</strong> (e.g., o1, o3, o4-mini series)</li>
112+
* <li><strong>Cannot be used with non-reasoning models</strong> (e.g., gpt-4o, gpt-3.5-turbo)</li>
113+
* </ul>
114+
*
115+
* <p><strong>Mutual exclusivity:</strong> This parameter cannot be used together with
116+
* {@link #maxTokens}. Setting both will result in an API error.</p>
94117
*/
95118
private @JsonProperty("max_completion_tokens") Integer maxCompletionTokens;
96119
/**
@@ -678,12 +701,72 @@ public Builder topLogprobs(Integer topLogprobs) {
678701
return this;
679702
}
680703

704+
/**
705+
* Sets the maximum number of tokens to generate in the chat completion. The total
706+
* length of input tokens and generated tokens is limited by the model's context
707+
* length.
708+
*
709+
* <p>
710+
* <strong>Model-specific usage:</strong>
711+
* </p>
712+
* <ul>
713+
* <li><strong>Use for non-reasoning models</strong> (e.g., gpt-4o,
714+
* gpt-3.5-turbo)</li>
715+
* <li><strong>Cannot be used with reasoning models</strong> (e.g., o1, o3,
716+
* o4-mini series)</li>
717+
* </ul>
718+
*
719+
* <p>
720+
* <strong>Mutual exclusivity:</strong> This parameter cannot be used together
721+
* with {@link #maxCompletionTokens(Integer)}. If both are set, the last one set
722+
* will be used and the other will be cleared with a warning.
723+
* </p>
724+
* @param maxTokens the maximum number of tokens to generate, or null to unset
725+
* @return this builder instance
726+
*/
681727
public Builder maxTokens(Integer maxTokens) {
728+
if (maxTokens != null && this.options.maxCompletionTokens != null) {
729+
logger
730+
.warn("Both maxTokens and maxCompletionTokens are set. OpenAI API does not support setting both parameters simultaneously. "
731+
+ "The previously set maxCompletionTokens ({}) will be cleared and maxTokens ({}) will be used.",
732+
this.options.maxCompletionTokens, maxTokens);
733+
this.options.maxCompletionTokens = null;
734+
}
682735
this.options.maxTokens = maxTokens;
683736
return this;
684737
}
685738

739+
/**
740+
* Sets an upper bound for the number of tokens that can be generated for a
741+
* completion, including visible output tokens and reasoning tokens.
742+
*
743+
* <p>
744+
* <strong>Model-specific usage:</strong>
745+
* </p>
746+
* <ul>
747+
* <li><strong>Required for reasoning models</strong> (e.g., o1, o3, o4-mini
748+
* series)</li>
749+
* <li><strong>Cannot be used with non-reasoning models</strong> (e.g., gpt-4o,
750+
* gpt-3.5-turbo)</li>
751+
* </ul>
752+
*
753+
* <p>
754+
* <strong>Mutual exclusivity:</strong> This parameter cannot be used together
755+
* with {@link #maxTokens(Integer)}. If both are set, the last one set will be
756+
* used and the other will be cleared with a warning.
757+
* </p>
758+
* @param maxCompletionTokens the maximum number of completion tokens to generate,
759+
* or null to unset
760+
* @return this builder instance
761+
*/
686762
public Builder maxCompletionTokens(Integer maxCompletionTokens) {
763+
if (maxCompletionTokens != null && this.options.maxTokens != null) {
764+
logger
765+
.warn("Both maxTokens and maxCompletionTokens are set. OpenAI API does not support setting both parameters simultaneously. "
766+
+ "The previously set maxTokens ({}) will be cleared and maxCompletionTokens ({}) will be used.",
767+
this.options.maxTokens, maxCompletionTokens);
768+
this.options.maxTokens = null;
769+
}
687770
this.options.maxCompletionTokens = maxCompletionTokens;
688771
return this;
689772
}

models/spring-ai-openai/src/test/java/org/springframework/ai/openai/OpenAiChatOptionsTests.java

Lines changed: 81 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -91,7 +91,7 @@ void testBuilderWithAllFields() {
9191
"streamOptions", "seed", "stop", "temperature", "topP", "tools", "toolChoice", "user",
9292
"parallelToolCalls", "store", "metadata", "reasoningEffort", "internalToolExecutionEnabled",
9393
"httpHeaders", "toolContext")
94-
.containsExactly("test-model", 0.5, logitBias, true, 5, 100, 50, 2, outputModalities, outputAudio, 0.8,
94+
.containsExactly("test-model", 0.5, logitBias, true, 5, null, 50, 2, outputModalities, outputAudio, 0.8,
9595
responseFormat, streamOptions, 12345, stopSequences, 0.7, 0.9, tools, toolChoice, "test-user", true,
9696
false, metadata, "medium", false, Map.of("header1", "value1"), toolContext);
9797

@@ -120,8 +120,8 @@ void testCopy() {
120120
.logitBias(logitBias)
121121
.logprobs(true)
122122
.topLogprobs(5)
123-
.maxTokens(100)
124-
.maxCompletionTokens(50)
123+
.maxCompletionTokens(50) // Only set maxCompletionTokens to avoid validation
124+
// conflict
125125
.N(2)
126126
.outputModalities(outputModalities)
127127
.outputAudio(outputAudio)
@@ -449,4 +449,82 @@ void testCopyChangeIndependence() {
449449
assertThat(copied.getTemperature()).isEqualTo(0.5);
450450
}
451451

452+
@Test
453+
void testMaxTokensMutualExclusivityValidation() {
454+
// Test that setting maxTokens clears maxCompletionTokens
455+
OpenAiChatOptions options = OpenAiChatOptions.builder()
456+
.maxCompletionTokens(100)
457+
.maxTokens(50) // This should clear maxCompletionTokens
458+
.build();
459+
460+
assertThat(options.getMaxTokens()).isEqualTo(50);
461+
assertThat(options.getMaxCompletionTokens()).isNull();
462+
}
463+
464+
@Test
465+
void testMaxCompletionTokensMutualExclusivityValidation() {
466+
// Test that setting maxCompletionTokens clears maxTokens
467+
OpenAiChatOptions options = OpenAiChatOptions.builder()
468+
.maxTokens(50)
469+
.maxCompletionTokens(100) // This should clear maxTokens
470+
.build();
471+
472+
assertThat(options.getMaxTokens()).isNull();
473+
assertThat(options.getMaxCompletionTokens()).isEqualTo(100);
474+
}
475+
476+
@Test
477+
void testMaxTokensWithNullDoesNotClearMaxCompletionTokens() {
478+
// Test that setting maxTokens to null doesn't trigger validation
479+
OpenAiChatOptions options = OpenAiChatOptions.builder()
480+
.maxCompletionTokens(100)
481+
.maxTokens(null) // This should not clear maxCompletionTokens
482+
.build();
483+
484+
assertThat(options.getMaxTokens()).isNull();
485+
assertThat(options.getMaxCompletionTokens()).isEqualTo(100);
486+
}
487+
488+
@Test
489+
void testMaxCompletionTokensWithNullDoesNotClearMaxTokens() {
490+
// Test that setting maxCompletionTokens to null doesn't trigger validation
491+
OpenAiChatOptions options = OpenAiChatOptions.builder()
492+
.maxTokens(50)
493+
.maxCompletionTokens(null) // This should not clear maxTokens
494+
.build();
495+
496+
assertThat(options.getMaxTokens()).isEqualTo(50);
497+
assertThat(options.getMaxCompletionTokens()).isNull();
498+
}
499+
500+
@Test
501+
void testBuilderCanSetOnlyMaxTokens() {
502+
// Test that we can set only maxTokens without issues
503+
OpenAiChatOptions options = OpenAiChatOptions.builder().maxTokens(100).build();
504+
505+
assertThat(options.getMaxTokens()).isEqualTo(100);
506+
assertThat(options.getMaxCompletionTokens()).isNull();
507+
}
508+
509+
@Test
510+
void testBuilderCanSetOnlyMaxCompletionTokens() {
511+
// Test that we can set only maxCompletionTokens without issues
512+
OpenAiChatOptions options = OpenAiChatOptions.builder().maxCompletionTokens(150).build();
513+
514+
assertThat(options.getMaxTokens()).isNull();
515+
assertThat(options.getMaxCompletionTokens()).isEqualTo(150);
516+
}
517+
518+
@Test
519+
void testSettersMutualExclusivityNotEnforced() {
520+
// Test that direct setters do NOT enforce mutual exclusivity (only builder does)
521+
OpenAiChatOptions options = new OpenAiChatOptions();
522+
options.setMaxTokens(50);
523+
options.setMaxCompletionTokens(100);
524+
525+
// Both should be set when using setters directly
526+
assertThat(options.getMaxTokens()).isEqualTo(50);
527+
assertThat(options.getMaxCompletionTokens()).isEqualTo(100);
528+
}
529+
452530
}

spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chat/openai-chat.adoc

Lines changed: 58 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -150,8 +150,8 @@ The prefix `spring.ai.openai.chat` is the property prefix that lets you configur
150150
| spring.ai.openai.chat.options.temperature | The sampling temperature to use that controls the apparent creativity of generated completions. Higher values will make output more random while lower values will make results more focused and deterministic. It is not recommended to modify `temperature` and `top_p` for the same completions request as the interaction of these two settings is difficult to predict. | 0.8
151151
| spring.ai.openai.chat.options.frequencyPenalty | Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. | 0.0f
152152
| spring.ai.openai.chat.options.logitBias | Modify the likelihood of specified tokens appearing in the completion. | -
153-
| spring.ai.openai.chat.options.maxTokens | (Deprecated in favour of `maxCompletionTokens`) The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. | -
154-
| spring.ai.openai.chat.options.maxCompletionTokens | An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens. | -
153+
| spring.ai.openai.chat.options.maxTokens | The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. *Use for non-reasoning models* (e.g., gpt-4o, gpt-3.5-turbo). *Cannot be used with reasoning models* (e.g., o1, o3, o4-mini series). *Mutually exclusive with maxCompletionTokens* - setting both will result in an API error. | -
154+
| spring.ai.openai.chat.options.maxCompletionTokens | An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens. *Required for reasoning models* (e.g., o1, o3, o4-mini series). *Cannot be used with non-reasoning models* (e.g., gpt-4o, gpt-3.5-turbo). *Mutually exclusive with maxTokens* - setting both will result in an API error. | -
155155
| spring.ai.openai.chat.options.n | How many chat completion choices to generate for each input message. Note that you will be charged based on the number of generated tokens across all of the choices. Keep `n` as 1 to minimize costs. | 1
156156
| spring.ai.openai.chat.options.store | Whether to store the output of this chat completion request for use in our model | false
157157
| spring.ai.openai.chat.options.metadata | Developer-defined tags and values used for filtering completions in the chat completion dashboard | empty map
@@ -193,6 +193,62 @@ This is useful if you want to use different OpenAI accounts for different models
193193

194194
TIP: All properties prefixed with `spring.ai.openai.chat.options` can be overridden at runtime by adding request-specific <<chat-options>> to the `Prompt` call.
195195

196+
=== Token Limit Parameters: Model-Specific Usage
197+
198+
OpenAI provides two mutually exclusive parameters for controlling token generation limits:
199+
200+
[cols="2,3,3", stripes=even]
201+
|====
202+
| Parameter | Use Case | Compatible Models
203+
204+
| `maxTokens` | Non-reasoning models | gpt-4o, gpt-4o-mini, gpt-4-turbo, gpt-3.5-turbo
205+
| `maxCompletionTokens` | Reasoning models | o1, o1-mini, o1-preview, o3, o4-mini series
206+
|====
207+
208+
IMPORTANT: These parameters are **mutually exclusive**. Setting both will result in an API error from OpenAI.
209+
210+
==== Usage Examples
211+
212+
**For non-reasoning models (gpt-4o, gpt-3.5-turbo):**
213+
[source,java]
214+
----
215+
ChatResponse response = chatModel.call(
216+
new Prompt(
217+
"Explain quantum computing in simple terms.",
218+
OpenAiChatOptions.builder()
219+
.model("gpt-4o")
220+
.maxTokens(150) // Use maxTokens for non-reasoning models
221+
.build()
222+
));
223+
----
224+
225+
**For reasoning models (o1, o3 series):**
226+
[source,java]
227+
----
228+
ChatResponse response = chatModel.call(
229+
new Prompt(
230+
"Solve this complex math problem step by step: ...",
231+
OpenAiChatOptions.builder()
232+
.model("o1-preview")
233+
.maxCompletionTokens(1000) // Use maxCompletionTokens for reasoning models
234+
.build()
235+
));
236+
----
237+
238+
**Builder Pattern Validation:**
239+
The OpenAI ChatOptions builder automatically enforces mutual exclusivity with a "last-set-wins" approach:
240+
241+
[source,java]
242+
----
243+
// This will automatically clear maxTokens and use maxCompletionTokens
244+
OpenAiChatOptions options = OpenAiChatOptions.builder()
245+
.maxTokens(100) // Set first
246+
.maxCompletionTokens(200) // This clears maxTokens and logs a warning
247+
.build();
248+
249+
// Result: maxTokens = null, maxCompletionTokens = 200
250+
----
251+
196252
== Runtime Options [[chat-options]]
197253

198254
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/main/java/org/springframework/ai/openai/OpenAiChatOptions.java[OpenAiChatOptions.java] class provides model configurations such as the model to use, the temperature, the frequency penalty, etc.

0 commit comments

Comments
 (0)