All valid points on being able to simulate well enough meaning we would have nearly solved everything to begin with.-

I wonder if, at some point, AI's will perform to such degree they are able to construct "black box" simulations as needed, that nobody understands BUT that are sufficient to empirically experiment on ...

... much in the same way LLM's function now: In an aggregate sense, they are bloack boxes.-