I don't think Grassmannian manifolds are necessarily a better representation than simple query/key inner products. It's a valid solution, and it does have advantages in some cases.
The Grassmannian is only preserving the subspace orientation, so we lose information about the actual linear map and basically throw out its basis.
Like everything, this is a trade off. You're trading information for a lower rank representation.
I'm still reading through the paper on the implementation side, but the other caveat I'd be wary of is compute. Does this projection add MACs to the overall model which would make inference slightly slower even with lower rank representations? I don't know. Like with any technique that trades constrained information representation for space/compute you _are_ widening your search space during training. I'd be curious what the impact of this is on relative data needed to train a model using this architecture versus "regular" attention.