And LLM "summarizes" the tool output as a completely unrelated and at times self-contradicting text.

If there are deterministic tools involved - just return their output as is. It's usually quite readable if needs a bit of adjusting