Bug: Note Splitter fails with OpenRouter provider

[Cyb3rN8]

Issue: Note Splitter fails with OpenRouter provider

Description:

The "Note Splitter" feature, which utilizes the ai_sdk_structured function, is currently failing when using the OpenRouter provider. This issue prevents users from effectively splitting notes into logical sections when OpenRouter is selected as the LLM provider.

Steps to Reproduce:

1. Select OpenRouter as the LLM provider in the plugin settings.
2. Create a Canvas note with some text content.
3. Right-click on the node and select "Split Note."
4. Observe the error.

Expected Behavior:

The note should be split into logical sections, with each section creating a new node on the canvas.

Actual Behavior:

The note splitter fails to generate new nodes.

Technical Details:

The issue seems to stem from the ai_sdk_structured function's interaction with the OpenRouter provider. While setting the mode to "json" in ai_sdk_structured resolves the initial "Not Found" error from OpenRouter (likely due to the provider successfully returning a JSON response), a subsequent parsing error occurs. This suggests a problem with how the Vercel AI SDK handles the JSON response from OpenRouter.

Relevant Code:

• llm_calls.ts: ai_sdk_structured function
• main.ts: Note Splitter implementation within the addExtraActions method (specifically, the graphButtonEl.addEventListener("click", () => { ... }) block).

Possible Cause:

Incompatibility between the Vercel AI SDK's JSON parsing and the format of the JSON response returned by OpenRouter.

Suggested Solution:

1. Investigate the JSON response format from OpenRouter when using the generateObject function.
2. Implement custom parsing logic within the ai_sdk_structured function to handle the OpenRouter response format correctly.
3. Consider alternative methods for structured data extraction from OpenRouter if the Vercel AI SDK proves to be incompatible.

Additional context
The issue occurs because the generateObject function from the Vercel AI SDK, which is used in ai_sdk_structured, has trouble parsing the JSON response from OpenRouter, even when the mode is set to "json".

[jcollingj]

@Cyb3rN8 Awesome bug report here. TBD on exactly on when I will get to this but thank you for reporting!

[jcollingj]

@Cyb3rN8 What model where you using for this? Does it support function calling and/or structured outputs?

Please provide the link to it

[Cyb3rN8]

Provider-Specific Behavior Analysis

1. Model Independence

   • Issue persists across both GPT-o and GPT-4o model families
   • Root cause appears to be OpenRouter's parameter handling mechanism
   • Not specific to any particular model implementation

2. OpenRouter Implementation Details

   • Documentation indicates specific parameter requirements for structured outputs
   • Standard tools-based approach not recommended
   • Reference: https://openrouter.ai/docs/features/structured-outputs

3. Cross-Model Compatibility

   • Documentation primarily focuses on GPT model series support
   • Testing reveals functionality extends to other models (e.g., Gemini)
   • Suggests broader applicability than documented

Technical Analysis

1. SDK Interaction

   • Setting generateObject() mode to 'json' (vs. default 'auto') produces expected behavior
   • OpenRouter successfully processes request and returns JSON output
   • JSON response delivered as model text output
   • Local parsing required for text-to-JSON conversion

2. Validation Issues

   • Primary failure occurs during Vercel's JSON Schema Validation
   • Related upstream issue identified: Gemini flash 2.0 not compatible with vercel tool calling OpenRouterTeam/ai-sdk-provider#19

Interim Solution Proposal

Until Vercel implements official OpenRouter fixes:

1. Implement custom JSON response parsing
2. Bypass Vercel SDK's built-in validation
3. Handle JSON processing at application level

@jcollingj

[jcollingj]

@Cyb3rN8 Can you update to 0.2.70 and try again? I tried switching the mode for OpenRouter to be JSON specifically.