Very nice! I'm actually running a autoresearch / modded-gpt comparision to try the complex math operation idea right now! I suspect it's too hard for small quick model training runs to figure out how to use them.

I'll put your knowledge graph nn idea into the mix of things to try.