docs(weave): Add audio example and fix old intro example for Typescript #3682
Conversation
add working toggle button
fix plaintext for sentence about features not implemented in typescript
prevent overwriting of markdown files already generated
remove programming toggle
remove the try catch
WalkthroughThe changes introduce integration support for audio generation using OpenAI’s Audio API within W&B Weave. New documentation and TypeScript examples outline functionalities to track audio files, transcripts, and metadata. Key additions include type definitions for Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant WeaveProject
participant OpenAIClient
participant OpenAI_API
User->>WeaveProject: Initialize project
User->>OpenAIClient: Initialize client
User->>OpenAIClient: Request audio generation with prompt & history
OpenAIClient->>OpenAI_API: Send chat completion request
OpenAI_API-->>OpenAIClient: Return response (audio/text)
OpenAIClient->>WeaveProject: Log audio data, transcript, token usage
OpenAIClient-->>User: Return AudioResponse
Poem
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
|
Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=7ba8a9f587708a11b3b9cafe35888c5d964158e2 |
|
Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=937912e2c3e87bb1e7f4d9ba58e0405621a86a1a |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🔭 Outside diff range comments (2)
docs/scripts/generate_ts_examples/package.json (1)
11-18:⚠️ Potential issueAdd missing wav package and its type definitions.
The code imports the
wavpackage but it's not listed in the dependencies."dependencies": { "@types/node": "^22.10.2", "dotenv": "^16.4.7", "openai": "^4.76.1", "ts-node": "^10.9.2", "typescript": "^5.7.2", - "weave": "^0.7.3" + "weave": "^0.7.3", + "wav": "^1.0.2", + "@types/wav": "^1.0.4" },docs/scripts/generate_ts_examples/ts-examples/intro-notebook.ts (1)
165-178: 🛠️ Refactor suggestionAdd error handling for OpenAI API calls.
The OpenAI API call could fail for various reasons (rate limits, invalid API key, network issues). Consider adding proper error handling.
async predict(userInput: string): Promise<string> { + try { const response = await this.oaiClient.chat.completions.create({ model: 'gpt-4-turbo', messages: [ { role: "system", content: "You are a grammar checker, correct the following user input." }, { role: "user", content: userInput } ], temperature: 0 }); return response.choices[0].message.content ?? ''; + } catch (error) { + if (error instanceof OpenAI.APIError) { + // Handle specific API errors (rate limits, invalid API key, etc.) + throw new Error(`OpenAI API error: ${error.message}`); + } + // Handle other errors + throw new Error('Failed to correct grammar: ' + error); + } }
🧹 Nitpick comments (4)
docs/scripts/generate_ts_examples/ts-examples/audio-notebook.ts (3)
5-6: Remove unused imports.The
writeFileSyncfromfsand theWriterfromwavare imported but never used in the code.-import { writeFileSync, readFileSync } from 'fs'; -import { Writer } from 'wav';
37-51: Align type definitions for consistency.The
idproperty has inconsistent nullability betweenAudioResponseandMessage.audio. This could lead to type-safety issues when handling responses.interface AudioResponse { audioData: ReturnType<typeof weave.weaveAudio>; transcript?: string; - id?: string; + id?: string | null; }
155-165: Enhance error handling in main function.Consider adding more specific error handling to better diagnose issues.
async function main() { await weaveAudioExample(); } if (require.main === module) { main().catch(error => { - console.error('Error running audio example:', error); + if (error instanceof OpenAI.APIError) { + console.error('OpenAI API Error:', error.status, error.message); + } else { + console.error('Error running audio example:', error); + } process.exit(1); }); }docs/scripts/generate_ts_examples/ts-examples/intro-notebook.ts (1)
211-221: Enhance error handling and add cleanup.The main function could benefit from:
- More specific error handling for different types of failures
- Proper cleanup after demonstrations
async function main() { try { await initializeWeaveProject(); await demonstrateBasicTracking(); await demonstrateOpenAITracking(); await demonstrateNestedTracking(); await runEvaluation(); } catch (error) { - console.error('Error running demonstrations:', error); + if (error instanceof OpenAI.APIError) { + console.error('OpenAI API error:', error.message); + } else if (error instanceof weave.WeaveError) { + console.error('Weave error:', error.message); + } else { + console.error('Unexpected error:', error); + } + process.exit(1); + } finally { + // Cleanup resources if needed + await weave.shutdown(); } }
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
docs/scripts/generate_ts_examples/package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (5)
docs/docs/reference/generated_typescript_docs/audio-notebook.md(1 hunks)docs/scripts/generate_ts_examples/.gitignore(1 hunks)docs/scripts/generate_ts_examples/package.json(1 hunks)docs/scripts/generate_ts_examples/ts-examples/audio-notebook.ts(1 hunks)docs/scripts/generate_ts_examples/ts-examples/intro-notebook.ts(5 hunks)
✅ Files skipped from review due to trivial changes (1)
- docs/scripts/generate_ts_examples/.gitignore
🧰 Additional context used
📓 Path-based instructions (3)
`**/*.{yaml,yml,json,tf}`: Check for security best practices...
**/*.{yaml,yml,json,tf}: Check for security best practices.
Verify environment-specific configurations.
Look for hardcoded credentials or sensitive data.
Ensure proper resource limits and requests.
Verify proper versioning of dependencies.
Check for infrastructure best practices.
docs/scripts/generate_ts_examples/package.json
`**/*.{md,mdx}`: Focus on technical accuracy. Check for brok...
**/*.{md,mdx}: Focus on technical accuracy.
Check for broken links.
Verify code examples are up-to-date.
Look for clarity and completeness.
Don't focus on grammar/spelling unless significant.
docs/docs/reference/generated_typescript_docs/audio-notebook.md
`**/*.{js,jsx,ts,tsx}`: Focus on architectural and logical i...
**/*.{js,jsx,ts,tsx}: Focus on architectural and logical issues rather than style (assuming ESLint is in place).
Flag potential memory leaks and performance bottlenecks.
Check for proper error handling and async/await usage.
Avoid strict enforcement of try/catch blocks - accept Promise chains, early returns, and other clear error handling patterns. These are acceptable as long as they maintain clarity and predictability.
Ensure proper type usage in TypeScript files.
Look for security vulnerabilities in data handling.
Don't comment on formatting if prettier is configured.
Verify proper React hooks usage and component lifecycle.
Check for proper state management patterns.
docs/scripts/generate_ts_examples/ts-examples/intro-notebook.tsdocs/scripts/generate_ts_examples/ts-examples/audio-notebook.ts
⏰ Context from checks skipped due to timeout of 90000ms (40)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
- GitHub Check: Trace nox tests (3, 13, trace)
- GitHub Check: Trace nox tests (3, 12, trace)
- GitHub Check: Trace nox tests (3, 11, trace)
- GitHub Check: Trace nox tests (3, 10, trace)
🔇 Additional comments (3)
docs/scripts/generate_ts_examples/ts-examples/audio-notebook.ts (1)
119-153: Add error handling and utilize the second response.The example function lacks error handling and doesn't utilize
secondResponse.Consider adding error handling and demonstrating how to use the second response:
const weaveAudioExample = weave.op(async function AudioExample() { + try { await initializeWeaveProject(); // First message: Ask about golden retrievers const firstResponse = await generateAudioResponse({ userPrompt: "Is a golden retriever a good family dog?", previousMessages: [{ role: "system", content: "You are an expert on dogs especially golden retrievers." }] }); // Follow-up question using conversation history const secondResponse = await generateAudioResponse({ userPrompt: "Why do you say they are loyal?", previousMessages: [/*...*/] }); + + return { + firstResponse, + secondResponse + }; + } catch (error) { + console.error('Error in audio example:', error); + throw error; + } });docs/scripts/generate_ts_examples/ts-examples/intro-notebook.ts (2)
1-5: LGTM! Imports are well-organized.The imports are appropriate for the functionality being demonstrated, and dotenv is correctly configured before other imports.
69-69:✅ Verification successful
Verify the OpenAI model name.
The model name "gpt-4-turbo" might not be the official model identifier. Please verify the correct model name from OpenAI's documentation.
🌐 Web query:
What are the available GPT-4 model names in OpenAI API?💡 Result:
The available GPT-⁴ models in the OpenAI API include several variants optimized for different use cases:
Core Models
GPT‑⁴o (
gpt‑⁴o)
- Multimodal capabilities (text + image processing)
- Enhanced accuracy & responsiveness
- Supports features like JSON Mode & parallel function calling
- Matches GPT‑⁴ Turbo performance in English/text tasks while excelling at non-English languages/vision tasks[1][2]
GPT‑⁴ Turbo (
gpt‑⁴‑turbo)
- Latest GA replacement for previous preview models
- ¹²⁸k token context window
- Updated knowledge cutoff (Dec ²⁰²³)[1][7]
GPT‑⁴o mini (
gpt‑⁴o‑mini)
- Cost-effective alternative to GPT‑³.⁵ Turbo series
- Maintains core multimodal capabilities[1][10]
Version-Specific Models
Model ID Key Features Status gpt‑⁴(turbo‑²⁰²⁴‑⁰⁴‑⁰⁹)Vision integration & replaces earlier previews GA[1][2] gpt‑⁴(¹²⁵‑Preview)Improved code generation & UTF‑8 handling Preview[1][7] gpt‑⁴(vision‑preview)Legacy vision support Deprecated[1][7] Legacy Models
Older versions like:
Citations:
- 1: https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models
- 2: https://learn.microsoft.com/en-us/azure/ai-services/openai/whats-new
- 3: https://platform.openai.com/docs/models/gpt-4o
- 4: https://openai.com/index/gpt-4-api-general-availability/
- 5: https://platform.openai.com/docs/models/gp
- 6: https://platform.openai.com/docs/models
- 7: https://help.openai.com/en/articles/8555510-gpt-4-turbo-in-the-openai-api
- 8: https://help.openai.com/en/articles/7127966-what-is-the-difference-between-the-gpt-4-model-versions
- 9: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chatgpt
- 10: https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4-gpt-4-turbo-gpt-4o-and-gpt-4o-mini
OpenAI Model Name Verified
The OpenAI model name used here—
gpt-4-turbo—is confirmed as an official identifier based on the latest OpenAI API documentation. No changes are needed for this model naming.
| const response = await client.chat.completions.create({ | ||
| model: "gpt-4o-audio-preview", | ||
| modalities: ["text", "audio"], | ||
| audio: { | ||
| voice: "alloy", | ||
| format: "wav" | ||
| }, | ||
| messages, | ||
| store: true, | ||
| }); |
There was a problem hiding this comment.
Fix model name typo and add error handling.
The model name appears to have a typo, and the API call lacks error handling.
const response = await client.chat.completions.create({
- model: "gpt-4o-audio-preview",
+ model: "gpt-4-audio-preview",
modalities: ["text", "audio"],
audio: {
voice: "alloy",
format: "wav"
},
messages,
store: true,
- });
+ }).catch(error => {
+ console.error('Error generating audio response:', error);
+ throw error;
+ });📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const response = await client.chat.completions.create({ | |
| model: "gpt-4o-audio-preview", | |
| modalities: ["text", "audio"], | |
| audio: { | |
| voice: "alloy", | |
| format: "wav" | |
| }, | |
| messages, | |
| store: true, | |
| }); | |
| const response = await client.chat.completions.create({ | |
| model: "gpt-4-audio-preview", | |
| modalities: ["text", "audio"], | |
| audio: { | |
| voice: "alloy", | |
| format: "wav" | |
| }, | |
| messages, | |
| store: true, | |
| }).catch(error => { | |
| console.error('Error generating audio response:', error); | |
| throw error; | |
| }); |
| async function generateAudioResponse({ | ||
| userPrompt = "", | ||
| previousMessages = [] as Message[] | ||
| }) { | ||
|
|
||
| const client = initializeOpenAIClient(); | ||
| const messages = [ | ||
| ...previousMessages.map((msg: Message) => ({ | ||
| role: msg.role, | ||
| content: msg.content || msg.audio?.transcript || null, | ||
| })), | ||
| { | ||
| role: "user" as const, | ||
| content: userPrompt | ||
| } | ||
| ]; | ||
|
|
||
| const response = await client.chat.completions.create({ | ||
| model: "gpt-4o-audio-preview", | ||
| modalities: ["text", "audio"], | ||
| audio: { | ||
| voice: "alloy", | ||
| format: "wav" | ||
| }, | ||
| messages, | ||
| store: true, | ||
| }); | ||
|
|
||
| const message = response.choices[0].message; | ||
|
|
||
| // Check if the response contains audio data | ||
| if (message.audio?.data) { | ||
| const audioData = Buffer.from(message.audio.data, 'base64'); | ||
| return { | ||
| audioData: weave.weaveAudio({data: audioData}), | ||
| transcript: message.audio.transcript, | ||
| id: message.audio.id | ||
| } as AudioResponse; | ||
| } | ||
|
|
||
| // If no audio data, return the text content | ||
| return { | ||
| audioData: weave.weaveAudio({data: Buffer.from(message.content || '', 'utf-8')}), | ||
| transcript: message.content | ||
| } as AudioResponse; | ||
| } | ||
| ``` |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Remove duplicate code blocks from documentation.
The same example code is repeated multiple times in the documentation. This creates maintenance overhead and potential inconsistencies.
Keep only one version of each code example to maintain clarity and reduce maintenance burden.
Also applies to: 203-238
| * - Token usage | ||
| * - API costs | ||
| * - Request/response pairs | ||
| * - Model configurations | ||
| * | ||
| * Note: We also support other LLM providers like Anthropic and Mistral. | ||
| * Note: In addition to OpenAI, Weave supports automatic logging of other LLM providers, such as Anthropic and Mistral. | ||
| * For the full list, see [LLM Providers in the Integrations documentation](../../guides/integrations/index.md#llm-providers). |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Verify the relative documentation link.
Please ensure that the relative path ../../guides/integrations/index.md#llm-providers correctly points to the LLM providers documentation.
🏁 Script executed:
#!/bin/bash
# Description: Verify that the LLM providers documentation file exists
fd --type f "index.md" | grep "guides/integrations"Length of output: 132
Update the relative documentation link
The current link
../../guides/integrations/index.md#llm-providers
from docs/scripts/generate_ts_examples/ts-examples/intro-notebook.ts resolves to docs/scripts/guides/integrations/index.md, which doesn't match the location of the documentation file (docs/docs/guides/integrations/index.md). Please update the relative path accordingly—likely to something like:
../../../docs/guides/integrations/index.md#llm-providers
—to ensure it correctly points to the LLM Providers documentation.
| @@ -46,16 +46,16 @@ async function demonstrateBasicTracking() { | |||
| } | |||
|
|
|||
| /** | |||
| * ## 🔌 OpenAI Integration | |||
| * ## OpenAI integration | |||
There was a problem hiding this comment.
seems less fun without the emojies (:
Add audio example and fix old intro example for Typescript
References #3302
Summary by CodeRabbit