v2 user input message fixes #45
Conversation
|
You have run out of free Bugbot PR reviews for this billing cycle. This will reset on November 16. To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial. |
📝 WalkthroughSummary by CodeRabbit
WalkthroughThis PR adds a comprehensive test suite for V2 AI SDK integration with Vercel (covering OpenAI and Anthropic models through Maxim), enhances the V2 wrapper to trace user input during logging setup, and includes minor formatting adjustments. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes The addition of the comprehensive test suite comprises a substantial amount of code, but follows established patterns and test structure. The wrapper enhancement is localized and straightforward. The whitespace-only change is trivial. Overall complexity stems from the breadth of test scenarios and need to verify each test case aligns with V2 specification expectations. Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
vercel-ai-sdk.ts (1)
3-4: Whitespace-only change.Two blank lines were added after the export statements. If this was unintentional, consider removing them for consistency.
src/lib/logger/vercel/vercelLoggerV2.test.ts (2)
37-90: Potential duplicate input logging.Line 58 manually calls
trace.input(query), but the enhancedwrapperV2.ts(changed in this PR) now automatically logs user input duringsetupLogging. This could result in duplicate input entries in traces.Consider removing the manual
trace.input(query)call unless double-logging is intentional for testing purposes.
284-340: Move sessionId declaration inside the test case.The
sessionIdis declared outside the test case at line 284. This could cause issues if tests are run in parallel or if the test suite is executed multiple times, as the same session ID would be reused.Apply this diff to scope the sessionId properly:
- const sessionId = uuid(); it("should handle V2 session and trace management", async () => { if (!repoId || !openAIKey) { throw new Error("MAXIM_LOG_REPO_ID and OPENAI_API_KEY environment variables are required"); } const logger = await maxim.logger({ id: repoId }); if (!logger) { throw new Error("Logger is not available"); } + const sessionId = uuid(); const model = wrapMaximAISDKModel(openai.chat("gpt-4o-mini"), logger);
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
src/lib/logger/vercel/vercelLoggerV2.test.ts(1 hunks)src/lib/logger/vercel/wrapperV2.ts(2 hunks)vercel-ai-sdk.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
src/lib/logger/vercel/vercelLoggerV2.test.ts (3)
src/lib/maxim.ts (2)
Maxim(149-1302)logger(1183-1214)src/lib/logger/components/error.ts (1)
Error(47-90)vercel-ai-sdk.ts (2)
wrapMaximAISDKModel(2-2)MaximVercelProviderMetadata(1-1)
🪛 Biome (2.1.2)
src/lib/logger/vercel/vercelLoggerV2.test.ts
[error] 690-690: Other switch clauses can erroneously access this declaration.
Wrap the declaration in a block to restrict its access to the switch clause.
The declaration is defined in this switch clause:
Safe fix: Wrap the declaration in a block.
(lint/correctness/noSwitchDeclarations)
[error] 691-691: Other switch clauses can erroneously access this declaration.
Wrap the declaration in a block to restrict its access to the switch clause.
The declaration is defined in this switch clause:
Safe fix: Wrap the declaration in a block.
(lint/correctness/noSwitchDeclarations)
🔇 Additional comments (6)
src/lib/logger/vercel/wrapperV2.ts (1)
2-3: LGTM!Import additions are appropriate for the user input tracing enhancement.
src/lib/logger/vercel/vercelLoggerV2.test.ts (5)
10-34: LGTM!Test setup and teardown are properly configured with environment validation and cleanup.
222-282: LGTM!Tool call test is well-structured and includes proper error handling for edge cases like division by zero.
342-393: LGTM!Multi-modal image analysis test is well-implemented with a publicly accessible image URL.
478-513: LGTM!Error handling test properly validates that invalid parameters trigger appropriate errors.
517-560: LGTM!Concurrent call test is well-structured and properly validates parallel execution with unique trace metadata for each call.
| execute: async ({ data, analysisType }) => { | ||
| switch (analysisType) { | ||
| case "mean": | ||
| return { | ||
| result: data.reduce((a, b) => a + b, 0) / data.length, | ||
| type: "mean", | ||
| dataPoints: data.length | ||
| }; | ||
| case "median": | ||
| const sorted = [...data].sort((a, b) => a - b); | ||
| const mid = Math.floor(sorted.length / 2); | ||
| return { | ||
| result: sorted.length % 2 === 0 ? (sorted[mid - 1] + sorted[mid]) / 2 : sorted[mid], | ||
| type: "median", | ||
| dataPoints: data.length | ||
| }; | ||
| case "range": | ||
| return { | ||
| result: Math.max(...data) - Math.min(...data), | ||
| type: "range", | ||
| min: Math.min(...data), | ||
| max: Math.max(...data), | ||
| dataPoints: data.length | ||
| }; | ||
| default: | ||
| return { result: "Analysis type not supported", type: "error" }; | ||
| } | ||
| }, |
There was a problem hiding this comment.
Wrap switch case declarations in blocks.
Lines 690-691 declare variables (sorted, mid) within switch cases without wrapping them in blocks. This can cause issues where these declarations are accessible to other switch cases, leading to potential bugs.
Based on static analysis hints.
Apply this diff to fix the issue:
switch (analysisType) {
case "mean":
return {
result: data.reduce((a, b) => a + b, 0) / data.length,
type: "mean",
dataPoints: data.length
};
- case "median":
- const sorted = [...data].sort((a, b) => a - b);
- const mid = Math.floor(sorted.length / 2);
+ case "median": {
+ const sorted = [...data].sort((a, b) => a - b);
+ const mid = Math.floor(sorted.length / 2);
return {
result: sorted.length % 2 === 0 ? (sorted[mid - 1] + sorted[mid]) / 2 : sorted[mid],
type: "median",
dataPoints: data.length
};
+ }
case "range":
return {
result: Math.max(...data) - Math.min(...data),
type: "range",
min: Math.min(...data),
max: Math.max(...data),
dataPoints: data.length
};
default:
return { result: "Analysis type not supported", type: "error" };
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| execute: async ({ data, analysisType }) => { | |
| switch (analysisType) { | |
| case "mean": | |
| return { | |
| result: data.reduce((a, b) => a + b, 0) / data.length, | |
| type: "mean", | |
| dataPoints: data.length | |
| }; | |
| case "median": | |
| const sorted = [...data].sort((a, b) => a - b); | |
| const mid = Math.floor(sorted.length / 2); | |
| return { | |
| result: sorted.length % 2 === 0 ? (sorted[mid - 1] + sorted[mid]) / 2 : sorted[mid], | |
| type: "median", | |
| dataPoints: data.length | |
| }; | |
| case "range": | |
| return { | |
| result: Math.max(...data) - Math.min(...data), | |
| type: "range", | |
| min: Math.min(...data), | |
| max: Math.max(...data), | |
| dataPoints: data.length | |
| }; | |
| default: | |
| return { result: "Analysis type not supported", type: "error" }; | |
| } | |
| }, | |
| execute: async ({ data, analysisType }) => { | |
| switch (analysisType) { | |
| case "mean": | |
| return { | |
| result: data.reduce((a, b) => a + b, 0) / data.length, | |
| type: "mean", | |
| dataPoints: data.length | |
| }; | |
| case "median": { | |
| const sorted = [...data].sort((a, b) => a - b); | |
| const mid = Math.floor(sorted.length / 2); | |
| return { | |
| result: sorted.length % 2 === 0 ? (sorted[mid - 1] + sorted[mid]) / 2 : sorted[mid], | |
| type: "median", | |
| dataPoints: data.length | |
| }; | |
| } | |
| case "range": | |
| return { | |
| result: Math.max(...data) - Math.min(...data), | |
| type: "range", | |
| min: Math.min(...data), | |
| max: Math.max(...data), | |
| dataPoints: data.length | |
| }; | |
| default: | |
| return { result: "Analysis type not supported", type: "error" }; | |
| } | |
| }, |
🧰 Tools
🪛 Biome (2.1.2)
[error] 690-690: Other switch clauses can erroneously access this declaration.
Wrap the declaration in a block to restrict its access to the switch clause.
The declaration is defined in this switch clause:
Safe fix: Wrap the declaration in a block.
(lint/correctness/noSwitchDeclarations)
[error] 691-691: Other switch clauses can erroneously access this declaration.
Wrap the declaration in a block to restrict its access to the switch clause.
The declaration is defined in this switch clause:
Safe fix: Wrap the declaration in a block.
(lint/correctness/noSwitchDeclarations)
🤖 Prompt for AI Agents
In src/lib/logger/vercel/vercelLoggerV2.test.ts around lines 681 to 708, the
switch cases declare consts (sorted, mid) without block scoping which can leak
those bindings across cases; wrap each case body that declares variables in its
own block (e.g., case "median": { const sorted = ...; const mid = ...; return {
... }; } ) and similarly wrap the other cases that declare variables so each
declaration is scoped to its case.
| const userMessage = promptMessages.find((msg) => msg.role === "user"); | ||
|
|
||
| const userInput = userMessage?.content; | ||
| if (userInput) { | ||
| if (typeof userInput === "string") { | ||
| trace.input(userInput); | ||
| } else { | ||
| const userMessage = userInput[0]; | ||
| switch (userMessage.type) { | ||
| case "text": | ||
| trace.input(userMessage.text); | ||
| break; | ||
| case "image_url": | ||
| trace.input(userMessage.image_url.url); | ||
| break; | ||
| default: | ||
| break; | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Variable shadowing and potential array bounds issue.
There are a few concerns with this user input tracing logic:
-
Variable shadowing: Line 82 declares
const userMessagewhich shadows the outeruserMessagefrom line 75. Consider renaming the inner variable tofirstContentorcontentItem. -
Missing array bounds check: Line 82 accesses
userInput[0]without verifying the array has elements. IfuserInputis an empty array, this will beundefinedand cause a runtime error in the switch statement. -
Incomplete multi-part logging: Only the first content item is logged for structured inputs. For multi-modal messages with multiple parts, subsequent items are silently ignored.
Apply this diff to fix the variable shadowing and add bounds checking:
const userMessage = promptMessages.find((msg) => msg.role === "user");
const userInput = userMessage?.content;
if (userInput) {
if (typeof userInput === "string") {
trace.input(userInput);
} else {
- const userMessage = userInput[0];
- switch (userMessage.type) {
+ if (userInput.length > 0) {
+ const firstContent = userInput[0];
+ switch (firstContent.type) {
case "text":
- trace.input(userMessage.text);
+ trace.input(firstContent.text);
break;
case "image_url":
- trace.input(userMessage.image_url.url);
+ trace.input(firstContent.image_url.url);
break;
default:
break;
+ }
}
}
}Note: This implementation only logs the first content item. If capturing all content parts is desired, consider iterating through the array or concatenating text parts.
🤖 Prompt for AI Agents
In src/lib/logger/vercel/wrapperV2.ts around lines 75 to 94, the inner const
userMessage shadows the outer variable and the code indexes userInput[0] without
checking the array length, risking runtime errors; rename the inner variable to
something like firstContent (or contentItem), add a guard that ensures
Array.isArray(userInput) && userInput.length > 0 before accessing index 0, and
handle the undefined case (skip tracing or log a fallback) so the switch only
runs on a valid object; keep current behavior of only logging the first content
item unless you choose to iterate through the array.
TL;DR
Added comprehensive test suite for Vercel AI SDK V2 integration and improved input logging in the V2 wrapper.
What changed?
vercelLoggerV2.test.tswith extensive tests for the Vercel AI SDK V2 specificationHow to test?
Ensure environment variables are set:
MAXIM_API_KEYMAXIM_BASE_URLMAXIM_LOG_REPO_IDOPENAI_API_KEYANTHROPIC_API_KEYRun the tests:
Verify that all test cases pass and traces are properly created in the Maxim dashboard
Why make this change?
This change ensures that our Vercel AI SDK V2 integration is thoroughly tested and properly captures user inputs in traces. The comprehensive test suite validates that our wrapper correctly handles all the features of the V2 specification, including multi-modal inputs, tool calls, and streaming responses. This will help maintain compatibility as the Vercel AI SDK evolves and ensure our logging functionality works correctly across different LLM providers.