I am not sure you need something so complex. If objects on scene could be tagged with their location, and the camera movement tracked to identify when it brings the same objects back into scene, then the model can use its positional information to validate if the next rendered scene contains all the objects that should be expected.

Though, I am not sure well this retains temporal stability. I am just thinking about how in 3D rendering, objects are tracked as individual things and only culled, but all the data about them still exists.

Maybe the models need to reason as a 3D space first, then turn chosen objects into renderings?