How to configure streaming Response isn't working with OpenAIClient from @azure/openai

[IsuruCreative]

Hi, I am trying to build a streaming chat response feature in my chat bot using @azure/openai client library for javascript. But after configuration i can't see a response stream in my UI. Instead response is showing as a block. Can someone help me to fix this issue.

Below is my chat route.

import { OpenAIClient, AzureKeyCredential } from "@azure/openai";
import { OpenAIStream, StreamingTextResponse } from "ai";
import { SearchClient } from "@azure/search-documents";
import { setLogLevel } from "@azure/logger";

const AOAI_KEY = process.env.AOAI_KEY;
const AOAI_ENDPOINT = process.env.AOAI_ENDPOINT;

const AAS_KEY = process.env.AAS_KEY;

const openai = new OpenAIClient(
  AOAI_ENDPOINT,
  new AzureKeyCredential(AOAI_KEY)
);

const search = new SearchClient(
  "https://chat-101-search.search.windows.net",
  "vector-1705422986077",
  new AzureKeyCredential(AAS_KEY)
);

// Export a POST function
export async function POST(req: Request) {
  // Get the messages from the request body
  const request = await req.json();
  const deploymentId = "chat";

  console.log(search);

  //message ques size
  const msg_length = request.messages.length;
  const user_msg = request.messages[msg_length - 1];

  //create embeddings for the message
  try {
    // setLogLevel("info");
    // Use the getEmbeddings method to get an array of numbers
    const msg_embeddings = await openai.getEmbeddings(
      "embedding",
      user_msg.content
    );

    console.log(msg_embeddings);

    // Pass the array as the value of the vector parameter
    const response = await search.search("*", {
      vector: {
        fields: ["vector"],
        value: msg_embeddings.data.embedding,
      },
    });

    for await (const result of response.results) {
      // These results are the nearest neighbors to the query vector
      console.log(result);
    }
  } catch (error) {
    console.log(error);
  }

  //prompts
  const messages = [
    {
      role: "system",
      content:
        "You are a helpful assistant. You will talk like a helpful assistant to provide solutions to user questions.",
    },
    { role: "user", content: user_msg.content },
    {
      role: "assistant",
      content: "What can I do for you?",
    },
  ];

  const events = await openai.streamChatCompletions(deploymentId, messages, {
    maxTokens: 140,
  });

  // Respond with the stream
  const stream = OpenAIStream(events);
  return new StreamingTextResponse(stream);
}

here's my chat ui

"use-client";
import clsx from "clsx";
import React, { ChangeEvent } from "react";
import { useChat } from "ai/react";
import { Avatar } from "../ui/avatar";
import { AvatarFallback, AvatarImage } from "@radix-ui/react-avatar";
import { Button } from "../ui/button";

const Chat = () => {
  const { messages, input, handleInputChange, handleSubmit } = useChat();

  return (
    <div className="flex flex-col h-[93vh]">
      <div className="overflow-y-auto">
        <div>
          {messages.map((m) => (
            <div
              className={clsx("p-8 flex justify-start", {
                "bg-gray-200 text-gray-800": m.role === "user",
                "bg-gray-100 text-gray-800": m.role !== "user",
              })}
              key={m.id}
            >
              <div className="max-w-[1024px] flex gap-4">
                <Avatar>
                  <AvatarImage src="https://github.com/shadcn.png" alt="user" />
                  <AvatarFallback>CN</AvatarFallback>
                </Avatar>
                <div className="mt-2">{m.content}</div>
              </div>
            </div>
          ))}
        </div>
      </div>

      <div className="mt-auto flex">
        {/* <ChatInput /> */}
        <form className="w-full flex gap-2 m-4" onSubmit={handleSubmit}>
          <input
            className="rounded border w-full border-gray-300 p-2 shadow-xl"
            value={input}
            onChange={handleInputChange}
          
          />
          <Button type="submit">Send</Button>
        </form>
      </div>
    </div>
  );
};

export default Chat;

[lgrammel]

We have released an Azure OpenAI provider: https://sdk.vercel.ai/providers/ai-sdk-providers/azure