points by dylan604 2 days ago | 0 comments
> it becomes noisy and sameish at larger scale or the more you read it.
I absolutely understand where you are coming from, and I agree to a point. If you're of an age where you actually read a printed encyclopedia, the tone and voice of the entire set was sameish as well. The difference is that it didn't sound LLMish like LLM text does. I know that printed encyclopedias would have been vetted by editors, and I feel this is the key difference. Editors not only knew grammar rules, but had that sense of good writing practices that's hard to write down as rules. A piece of text passed through a good editorial review will read human while LLM text reads as LLM text.
[–] contrast 2 days ago
[–] zobzu 2 days ago
what's funny of that the llm learns from Wikipedia and then is used to put data back in Wikipedia. can't imagine that going well lol
[–] visarga 2 days ago
I think this is exactly what we should do. Send agents to collect evidence and write deep research reports, then train on those reports. DR reports are derived from hundreds of sources and can map out the distribution, what are the opinions on a topic, not necessarily what is the one correct answer. Agents could write one piece about every human, place, topic, event or entity there is under the sun. They could also try to generate the same without sources (closed-book mode) and report on LLM biases on that question.
[–] em3rgent0rdr 2 days ago
Please let's not clutter up wikipedia with new articles for every entity in the universe. It would be too much for humans to review. Though it sure would be funny to see an AI-peida as you describe.
[–] visarga 12 hours ago
It would not be inside Wikipedia, but separate, as a collection used in the training corpus.
When we say AI sounds same-ish we mean all sources of content can sound the same - fiction and non-fiction, news articles and blog posts, marketing pitches and ai drafted personal messages, all sound kinda the same. That is a different situation, and that's before you get to "and that one sound is LLMish".