points by lumost 15 hours ago | 0 comments
If I understand this clearly enough, what they really did was
A) add a post-processor which accepted a decoded tool call specifying what the processor should invoke.
B) had the post-processor perform the computation in tensor space.
The kv-cache/activations were not shared between qwen/the processor which is how they avoided fine tuning etc. I'm not clear how the tensor representation benefited the computation in this pipeline.
This will be interesting when something similar to a CPU can be triggered off of the LLM's own internal state representation - but imo we're far away from that.