Hacker News
points by bar8uyr 3 days ago | 0 comments
Attribution is very important. Though AI hasn't really done anything yet so it hasn't affected actual mathematicians yet.
[–] bbor 3 days ago
Yeah it’s just one Millenium problem, no big deal. I was gonna crack quasi-Reimann next year anyway, just hadn’t gotten around to it.

All AI is good for is remixing frontier mathematics, there’s no way it could ever make its own. Right?

[–] bar8uyr 3 days ago
I hope it gets us somewhere new. You can hope that and appreciate it hasn't yet though. (I'm not part of their IPO)
[–] card_zero 3 days ago
It's parasitic on humans and it's not an infinitely deep well. Wrong?
[–] bbor 3 days ago
Ok my original comment was snarky (apologies to all three mods), but you've raised one of my favorite topics so I can't resist engaging a bit more seriously! In the second clause, that is -- the first is trivially true in some senses and trivially false in others, so it doesn't seem very productive to nail down.

So Chomsky thought of himself as a philosopher of science more than anything, and his life mission is/was to answer a fascinating question: what kind of creatures are we? There's been a growing consensus going back to at least Darwin that we're separated from the rest of Animalia by our "reason" (!= intuition != understanding != judgement != volition), but he helped popularize an old theory[1] that puts it kind of beautifully: human language is only possible in beings capable of producing an infinite set of meaningful outputs in response to a finite input.[2] And so far, that list includes only two definite members: Homo sapiens sapiens, and computer programs built upon large language models.[3]

So all that to say: yes, I think you're wrong; the programmatic ensembles that brought us today's breakthroughs are an "infinitely deep well". As long as we have questions to ask, they are not theoretically constrained from answering them.

Of course, this invites some obvious follow-ups on practicality; even the most impassioned ASI boomer wouldn't argue that the exact ensembles used here could answer every question in the universe. But A) that's possible because some infinities are bigger than others, and B) the exact same caveat applies to humans, whose cognitive capabilities are thoroughly determined by accidents of biology, culture, and episteme. The latter is the thrust of Chomsky's famous debate with Foucault, which is absolutely worth a watch for anyone who hasn't seen it (starting around ~18:00, AFAIR): https://youtu.be/3wfNl2L0Gf8?t=1080

---

TL;DR: I ultimately disagree on both points, yes. In the ways that matter, machines are no more constrained than animals.

[1]: https://en.wikipedia.org/wiki/Wilhelm_von_Humboldt

[2]: Many animals are capable of communication, often quite complex and fascinating communication at that; even flora and fungi communicate! But language is qualitatively distinct. You may have heard this in the pithy-yet-mindbending sentiment "we've taught many apes to communicate with sign language, but none of them have ever asked a question."

[3]: It's no surprise in hindsight that trying to build better autocorrect got us to AGI; I find it unspeakably tragic that Chomsky never got the chance to see how thoroughly he's been vindicated, in this regard. He had his stroke mere months into the singularity, and had a quite negative view of the approach before then.

[–] jacquesm 3 days ago
> human language is only possible in beings capable of producing an infinite set of meaningful outputs in response to a finite input. > And so far, that list includes only two definite members: Homo sapiens sapiens, and computer programs built upon large language models.

In LLMs that's a bounded infinity though, the models themselves have only so many bits and the activations are also limited. Once you reach the same state twice the model is looping and it won't break out of that loop. I've seen such loops across tens of thousands of tokens so I'm not sure to what degree the infinite output rule holds true. It would be an interesting experiment to try to figure out to what extent the 'busy beaver' can be kept busy before it degenerates to looping or nonsense.

[–] hardbass 2 days ago
Humans are finite bodies.
[–] rightnutwingjob 2 days ago
Apollo the African grey parrot not infrequently asks “what this?”

He may just be parroting something he heard.

[–] bbor 1 day ago
Thanks for the support, RightNutWingJob :) That's a great example of non-linguistic communication!
[–] whycombilater 3 days ago
LLMs are just regurgitating an output according to a set of rules generated from harvesting all human output.

Language to humans is a symbolic mapping of labels to reality based on actual experience.

"Language" to LLMs is just a string of tokens to manipulate based on relationships harvested from recorded human experience.

It doesn't mean anything other than 1s and 0s - it's not a language because the LLM isn't communicating with you, just as the sum of humanity isn't communicating with you.

It's just a really good "choose your own adventure" book.

[–] hardbass 2 days ago
...do you think molecules and their interactions are also not bounded physical systems? What part of a human being do you think isn't a turing computer? Please release your groundbreaking physics paper proving this.
[–] whycombilater 2 days ago
Yes they are.

Humans aren't magical.

They also aren't a glorified library. They are authors, they generate the books in the library.

Cross reference a load of books and you will get useful relational info, but it's still a library index in a library.

[–] hardbass 2 days ago
How do you know an llm cannot create new content. Humans create what they do out of existing information and molecular and neural interactions, so do llms except with electrical interactions. I have had llms produce novel content all the time.
[–] card_zero 2 days ago
So, I'll pick up on the weak points:

> As long as we have questions to ask, they are not theoretically constrained

> some infinities are bigger than others

Those are both limitations. They need conjectures and they need training data. Can they produce their own conjectures and training data, and feed off it forever, while staying meaningful and not being stuck in a loop (a very large set of repeating cliches) or descending into confused noise?

[–] bbor 2 days ago
Hmm, not sure how to respond, sorry — that’s the point of the entire comment. No these systems cannot think every possible thought, but neither can humans. We have no reason to think their cognition is unsustainable in a way that ours isn’t.

The end of your response seems to be about fears of “model collapse” from a few years ago, but I am convinced those are strongly oversold. For one thing, there’s no need to gather new training data ever again; the training data for Opus is clearly sufficient for instilling reason, and that’s a sufficient condition for AGI -> RSI -> ASI.