Poor AI output quality and tool usage issues with Groq integration

[SimonLeclere]

Hi,

I’ve integrated AI into the BlockNote editor following the official documentation, using Groq as the provider and the llama-3-70b-versatile model.

However, I’m encountering several issues:

1. Very short or incomplete responses
   When prompting the AI to help write content, it often returns just a single word or one short sentence, rather than providing meaningful assistance in drafting text. This happens consistently even with simple prompts.

2. Fails completely on empty documents
   If the document is completely empty and I ask the AI to generate content (e.g., "write a paragraph about x"), the request fails systematically: nothing is inserted and no helpful error message is returned.

3. Tool calling issues with other models
   I’ve also tried other models via Groq, but most of them fail to call the JSON tool correctly. Instead of invoking the tool, they either return nothing useful or trigger an error. This makes the AI integration essentially unusable in most cases.

Has anyone experienced similar issues? Is there a workaround or recommended model/provider combo that works well with BlockNote’s AI features?

Thanks in advance!

[YousefED]

Hi Simon,

See below:

Very short or incomplete responses

Which model were you using exactly? 3.3? can you give an example?

Fails completely on empty documents

Was able to reproduce and will investigate

Tool calling issues with other models

Which models did you try?

[SimonLeclere]

Hello Yousef, Thanks for your quick reply!

Which model were you using exactly? 3.3? Can you give an example?

I used the model llama-3.3-70b-versatile as described in the documentation.

Very short or incomplete responses

Here is an example of a very short response, where I would have preferred a detailed, multi-step explanation spanning several lines:

---

Tool calling issues with other models

Here is an example. when I try to use the llama-3.1-8b-instant model, I get:

{
    "error": {
        "message": "Failed to call a function. Please adjust your prompt. See 'failed_generation' for more details.",
        "type": "invalid_request_error",
        "code": "tool_use_failed",
        "failed_generation": "\u003cfunction=json\u003e {\"operations\": [{\"type\": \"add\", \"referenceId\": \"ca06e188-8794-4ff4-8b6f-2ada078eeca2$\", \"position\": \"after\", \"blocks\": [{\"block\": \"\u003cp\u003eStart by mixing together 1 cup of flour, 2 eggs, and 1/2 cup of milk in a large bowl.\u003c/p\u003e\"}]}]} \u003c/function\u003e"
    }
}

Which models did you try?

Model	Result
llama-3.3-70b-versatile	Works but outputs very short sentences; sometimes fails (tool_use_failed).
llama-3.1-8b-instant	Always fails (tool_use_failed).
gemma2-9b-it	Always fails (tool_use_failed).
deepseek-r1-distill-llama-70b	Fails or outputs only "# tools" (wtf?).
meta-llama/llama-4-maverick-17b-128e-instruct	Best output (2 or 3 sentences) but very costly to run.
meta-llama/llama-4-scout-17b-16e-instruct	Always fails (tool_use_failed).
qwen-qwq-32b	Produces one or two sentence replies but very slow.
qwen/qwen3-32b	Same as above.

For each of the tests above, I used the “Continue writing” button in a document containing only the title “# How to make crepes in 5 steps.”