Anthropic prompt caching #104
Conversation
✅ Deploy Preview for stunning-cannoli-238aa5 canceled.
|
|
|
||
| enum AnthropicCacheType | ||
| { | ||
| case ethemeral; |
There was a problem hiding this comment.
I see value of keeping the enum here but Imo keeping provider related settings in the prisms code can make maintenance way harder.
@sixlive can think different.
There was a problem hiding this comment.
Hilariously, having added the enum to avoid spelling mistakes, I misspelt it in the enum...! Now fixed.
Definitely not a huge loss if we don't include the enum. Really just a convenience thing.
|
Prompt caching is now out of beta, as is PDF processing (my next PR). The only live beta is now computer use, so we can probably just bin the config/header changes - unless you'd rather leave it in for future @sixlive? |
Preview deployments for prism ⚡️
Commit: Deployment ID: Static site name: |
|
Have had to close this to clean up our fork. Will resubmit when done. |
Implements Anthropic prompt caching
Choices made
providerMeta: [Provider::Anthropic->value -> [...]]. It ended up easy enough to do with a trait, shared with Tool.I don't think providerMeta is the nicest API for the user, and it might get messy in the map methods if more features become reliant on it. But it should be easy to add a nicer API in a backward compatible way in future if needed. My gut thought is a ->withFeature() method on Messages and Tools, which accepts an implementation of a contract with a handle method, which are then applied with a pipeline to mutate the payload.
To-do
P.S. could do away with the
AnthropicCacheTypeenum - however if anyone else is like me that can never spell "ephemeral" right - it is quite handy!