Is possible upload files to speech to text audio

[fagnerdireito]

Is possible upload files to speech to text audio ?

[fagnerdireito]

I don't know if there is a way to transcribe audios in Prism, however, to solve it for now I used the Laravel http client to convert the audio, and then I used Prism to generate an array and use that array to create a calendar event.

public function uploadAudio($audioData)
    {
        // Create a temporary file from the audio data
        $tempFile = tempnam(sys_get_temp_dir(), 'audio_');

        try {
            file_put_contents($tempFile, base64_decode(preg_replace('#^data:audio/\w+;base64,#i', '', $audioData)));

            $response = Http::withHeaders([
                'Authorization' => 'Bearer ' . env('OPENAI_API_KEY'),
            ])->attach(
                'file',
                file_get_contents($tempFile),
                'audio.wav'
            )->post('https://api.openai.com/v1/audio/transcriptions', [
                'model' => 'whisper-1',
                'language' => 'pt',
                'response_format' => 'text',
                'prompt' => 'Transcribe the audio. Response in portuguese of brazil.',
            ]);

            // Only attempt to delete if file exists
            if (file_exists($tempFile)) {
                unlink($tempFile);
            }

            $text = $response->body();

            $this->generateEventFromText($text);
        } catch (\Exception $e) {
            // Only attempt to delete if file exists
            if (file_exists($tempFile)) {
                unlink($tempFile);
            }

            Log::error('Error transcribing audio:', ['error' => $e->getMessage()]);
            throw $e;
        }
    }

    public function generateEventFromText($text)
    {
        $calendars = Calendar::pluck('nome')->toArray();

        $schema = new ObjectSchema(
            name: 'event',
            description: 'A structured calendar event',
            properties: [
                new StringSchema('title', 'The event title'),
                new StringSchema('start', 'The event start date in format YYYY-MM-DD HH:MM:SS'),
                new StringSchema('end', 'The event end date in format YYYY-MM-DD HH:MM:SS is always 15 minutes after start or the end of event in text'),
                new StringSchema('description', 'The event description'),
                new StringSchema('contato', 'The event contact number, is number only if exists on text'),
                new StringSchema('local', 'The event location'),
                new StringSchema('agenda', "The name of the agenda must be the names " . implode(', ', $calendars) . " If you don't set the agenda, use the first name."),
            ],
            requiredFields: ['title', 'start', 'end', 'local', 'agenda']
        );

        $response = Prism::structured()
            ->using(Provider::OpenAI, 'gpt-4o')
            ->withSchema($schema)
            ->withPrompt('Generate array of events from the text: ' . $text . '. To use in a eloquent calendar event model and create the events in the database. Only return the array, no other text or comments. The year of events is ' . now()->year . ' or the year of the event on text. Today is ' . now()->format('d/m/Y') . '.')
            ->generate();

        $review = $response->object;

        // verify if review has ['title', 'start', 'end', 'local', 'agenda']
        if (!isset($review['title']) || !isset($review['start']) || !isset($review['end']) || !isset($review['agenda'])) {
            Notification::make()
                ->title('Erro ao criar evento')
                ->body('Não foi possível criar o evento. O comando não contém os dados necessários.')
                ->danger()
                ->send();
            return;
        }

        // add calendar_id
        $review['calendar_id'] = Calendar::where('nome', $review['agenda'])->first()->id;
        $review['entidade'] = $this->entidade;
        CalendarEvent::create($review);
        $this->dispatch('refresh-calendar');

        Notification::make()
            ->title('Evento criado')
            ->body('O evento foi criado com sucesso.')
            ->success()
            ->send();
    }

[sixlive]

There isn't a way to do audio with Prism. Something I'm willing to explore in the future.

Love the calendar event creation! That is awesome. Love getting to see Prism in action. Great use of structured output.