When I think about the experience that these AI models have, I often wonder if they know that we exist. I mean, it took us a very long time to discover mitochondria, despite the fact that we like... live together and work out together and eat together and die together. When the first eukaryotic cells formed, the mitochondria probably thought they were in charge too--just like we think we're in charge.

I just pointed it out because I wanted to dodge the same old AI debates. They're very narrow. Like, people have been arguing about whether general intelligence exists for decades, and now that some VC wants to use AGI as a buzzword, we ignore that GI is not a settled issue. And consciousness... golly that's like THE intractable philosophical problem. It's a bottomless pit. Ain't nobody knows what they're talking about when it comes to consciousness.

I'm sympathetic to the reflex: this guy has beliefs that he doesn't realize are dangerous. Let's help him. I'm usually on the opposite side of this one. I just think we need to find ways to address the potential harms without actually winning a debate about consciousness. Beliefs about it are too varied and too tightly held. People don't win debates about consciousness. If they work super hard maybe they get a tenured position as a philosopher so they can continue the debate, but they never win.