Hacker News

points by cvoss 2 days ago | 0 comments

It's like its simulation model is confined to the renderable area. At the cost of increased computation, it would not be hard to generalize the technique to rendering a sphere centered around the POV to keep some of those artifacts at bay; however, the fundamental limitation of "object permanence" remains if objects can evolve chaotically as soon as they are blocked from view by another object. For that to be made stable, you really need a whole 3D model of the scene, and the cost increase there might be huge.

> While it can maintain coherent environments over short sessions, visual details and object permanence can still shift or drift over time—especially as a simulation runs longer, after multiple complex prompt edits, or when navigating throughout the scene. Complex physical interactions, real-time audio accuracy, rendering legible text, spatial accuracy, and long-term consistency are active areas of ongoing research.

[–] recursive 2 days ago

If it was actually not hard, they would have made their demo with it. I don't know the technical details, but I expect a product reveal like this to show the thing at its absolute best.

[–] msabalau 1 day ago

People seem to be leaping to the conclusion that this is "product" as opposed to, fundamentally, research work, posted on a lab blog post, showing off a capability they expect to be increasing interesting over time.

Just because they happen to be throwing in the option to sign up to to play with the Genie 4 into an Ultra plan, doesn't mean that this is a material part of the value prop for Gemini.

Presumably, they can learn something from how niche Ultra users interested in tech demos engage with it. If that's an upside, it's really unclear it why it would make sense for "a reveal" to promise something too expensive to deliver to such Ultra users. (None of which means that the suggestion that you are responding to is technically reasonable or possible--just that it is unclear that one should dismiss the idea because one views this through the lens of a consumer "product reveal"

Just because OpenAI had to crawl back from Sora to focus resources, and Anthropic proudly proclaims in their filing that they have been avoiding promising creative opportunities for the same reason, doesn't mean that Google can't continue to do lab/research stuff.

It would be odd if they didn't.

[–] busssard 1 day ago

didn't they show off an interactive demo with genie3 where you could paint on a blackboard and go out through the window and come back and the same was still written on the blackboard?

[–] david-gpu 1 day ago

It is possible that it is not hard, but they didn't have time to implement it for this release. It happens all the time in fast-moving industries.

[–] whatisthiseven 2 days ago

I am not sure you need something so complex. If objects on scene could be tagged with their location, and the camera movement tracked to identify when it brings the same objects back into scene, then the model can use its positional information to validate if the next rendered scene contains all the objects that should be expected.

Though, I am not sure well this retains temporal stability. I am just thinking about how in 3D rendering, objects are tracked as individual things and only culled, but all the data about them still exists.

Maybe the models need to reason as a 3D space first, then turn chosen objects into renderings?

[–] unsloppable 2 days ago

The trick is to render a larger frame in lower fidelity and then only render a smaller output frame in full fidelity.

Or render a low fidelity 3d model for coherence and then raytrace that onto a 2d frame that is then rendered at higher fidelity.

[–] sn0n 2 days ago

Shhhh!!! That’s the next release… you are spoiling the surprise!!

[–] everforward 2 days ago

> For that to be made stable, you really need a whole 3D model of the scene, and the cost increase there might be huge.

I suspect it is, because the 2D -> 3D models I've seen are terrible, but more importantly the 2D -> 3d models are terrible. Noteworthy because I suspect the number of people who have 3D scans laying around is tiny.

What you're describing is vaguely how video games work, to my understanding. There may be interesting parallels if that becomes a technique (like how distant objects are sometimes still rendered, but at a lower resolution/polygon count).

[–] david-gpu 1 day ago

> What you're describing is vaguely how video games work, to my understanding.

Your understanding is broadly correct. There is a 3D world, and the GPU computes how it looks like from a particular viewpoint.

In essence, objects are painted back to front, so that in the final image every pixel shows the nearest visible object. The details get complicated, but the idea is simple.

[–] 93po 2 days ago

> fundamental limitation of "object permanence" remains if objects can evolve chaotically as soon as they are blocked from view by another object.

they got this working decently well for video over a year ago, maybe closer to two years ago at this point, time goes fast. so i assume it'll translate to progress in Genie too

[–] selfmodruntime 1 day ago

I think it's exactly this. The rendering capabilities in a true world model would include the entire domain it exists in. This takes a ton of processing power.

[–] dominotw 2 days ago

This sounds like bunch of gibberish you made up