Costly message caused by tokens exhausted limits breach for large messages.

[guywald]

Issue

Sent: a costly message with a big file.
Response:

Tokens: 1663k sent, 0 received. Cost: $4.99 message, $9.68 session.

Model claude-3-5-sonnet-20240620 has hit a token limit!
Token counts below are approximate.

Input tokens: ~1,663,395 of 200,000 -- possibly exhausted context window!
Output tokens: ~0 of 8,192
Total tokens: ~1,663,395 of 200,000 -- possibly exhausted context window!

A previously sent message with a larger file resulted in the following error:

BadRequestError: litellm.BadRequestError: AnthropicException - b'{"type":"error","error":{"type":"invalid_request_error","message":"too 
many total text bytes: 9256098 > 9000000"}}'

The different api limits should be considered before sending a message, preventing limits breaching.
Maybe a pre-send validation hook could solve such a scenario.

Version and model info

Aider v0.58.1
Main model: claude-3-5-sonnet-20240620 with architect edit format, infinite output
Editor model: claude-3-5-sonnet-20240620 with editor-diff edit format
Weak model: claude-3-haiku-20240307

[fry69]

Thank you for filing this issue.

Please check if those costly messages really got billed. If the API returns an error, those requests should not get billed. Do not trust aider's cost estimation, it is (as stated) not reliable.

Also note that you can (and should) check your token usage before you send a prompt/request with /tokens