points by HeavenFox 1 day ago | 0 comments
If they are literally sending the entire transcript and asking for a summary, wouldn't almost all of the tokens be cache hit, and therefore fairly cheap?
[–] minraws 1 day ago
[–] odo1242 23 hours ago
But you also burn the exact same number of tokens (actually more) every time the model makes a tool call. It's an agent, it makes a lot of tool calls.
[–] kristjansson 23 hours ago
This is the right perspective. It costs as much as one additional tool call in the conversation. It’s a bit weird to do it on a timer, which inflates a bit over “just a tool call”, but not by much?
[–] brookst 18 hours ago
Tool calls don’t sent the full transcript? I think you mean one more turn?
[–] hedora 16 hours ago
Pretty much everything sends the full transcript. That’s how LLMs work, and why “compact before cache expiry” is now a default in Claude Code. Cache hits are incredibly cheap to service on the provider side.
[–] RockRobotRock 1 day ago
Who pays for their own tokens at full price?
[–] cheeseface 1 day ago
All Claude Enterprise customers.
[–] krzyk 23 hours ago
Most claude code users, minority can still use subscriptions.
[–] minraws 20 hours ago
I mean all big businesses (150+ peep) have to pay that much, it's the reality of the times. Getting some AI imho essential/good, one 200$ plan or spending 5000-7000$ per engineering team sure, 100$ plans or 3000$ for most semi technical staff definitely.
But if you are telling me I am wasting like a good 10-20% of my tokens on summaries I might as well stop using your service, worst part is claude doesn't let you use something else.
[–] simianwords 1 day ago
Isn’t it 0.2?
[–] buzer 1 day ago
Didn't they change the cache TTL to 5 minutes earlier this year? So depending on exact timing it could potentially be invalidated between the updates.
[–] maxbond 1 day ago
It's an hour. It'll tell you the expiry time on the status line now but that information is also available in /usage.
[–] simon83 22 hours ago
Subagents have a default TTL of 5 minutes though, with an opt in to increase it to 1h, for a premium for writing to cache.
[–] buzer 1 day ago
https://github.com/anthropics/claude-code/issues/46829
Looks like what I remembered was a bug.
[–] gringofyx 20 hours ago
No, not necessarily - the prompt-cache is only hit if everything prior matches... including MCP & tool definitions. So to benefit from prompt-caching themselves they would need to include the tools you define to the summariser.
If they were doing that, then a cheaper (prompt-cache friendly) option is just to silently send the whole conversation with the summariser prompt appended
[–] dietr1ch 1 day ago
Unless they prepend something like, "summarise this, "
[–] NewsaHackO 1 day ago
But they definitely wouldn’t do that.
[–] dietr1ch 13 hours ago
Yeah, this would have quadratic cost, but something silly has to be happening to waste a noticeable amount of tokens on summaries
[–] asaddhamani 21 hours ago
Wouldn’t they?
[–] skerit 20 hours ago
I just had a look, and they don't. It's really just a temporary fork of the conversation, so it's fully cached. But it does run every 30 seconds, so that is very bad especially if it's a subagent that's at 800k tokens.
And I don't really get the point of this feature, I've always found it super annoying that the main agent creates a subagent with the clear description of "Fix tile cache" for example, and then after a while it turns into something generic like "Running browser tests"
If it happens 10 times you have already accepted the same cost as the entire session fresh prefill.
People on HN are either too rich or wayyy too rich