Are there inference products focused on say, optimizing frequently used internal contexts, so that they are compact in the kv cache? Perhaps there rep could be optimized over hundreds+ of distinct real use cases?

Maybe your law firm doesn't want your data going outside your business. But you obviously want high quality analysis.