Hacker News

points by clarionbell 2 days ago | 0 comments

There will be a point in the future, possibly near future, when AI will be capable of reliably producing encyclopedic content, and we'll have tools to verify that the content produced passes reasonable standards.

Today is not the day. Especially not regarding the verification.

[–] rkozik1989 2 days ago

That's really not how probabilistic systems work though. They're great at getting 80% of the way to the goal but the last 20% is always excruciatingly difficult. I really don't intend that a knock because LLMs are useful, but they also aren't magic boxes where anything is possible.

[–] dspillett 2 days ago

This is the problem that a lot of user-content based sites, and open software projects, are experiencing. The tools make prodiving 80% correct submissions so easy that many people are doing that and thinking that it is fine not to bother verifying for correctness (or not even knowing how to verify correctness). This means those projects are being inundated by submissions that are 20% tosh or worse.

[–] phoghed 2 days ago

Another problem with user content sites, these numbers are completely pulled from the ass and are a meme that just gets repeated ad nauseam. They mean nothing, but there will be people that take it seriously.

Is 80/20 ever generally correct? Why is it favored over the 90/10 meme? Why would it apply to this? Why wouldn’t the numbers be vastly different for different domains of content generation? Etc.

If you pay attention, so much user content is just repetition of something they read on the same fucking website and uncritically checked into their brain.

Current HN top post is Roblox downloaded 145GB models from us (including adult content) without asking. I fully expect to start seeing comments to the tune of “Roblox is stealing porno to train AI for children” within the day in any discussion Roblox comes up in.

Why would these folk’s contribution to any other user content site be any more thoughtful?

[–] dspillett 2 days ago

> Is 80/20 ever generally correct?

Sometimes. But it, or 90/10, or sometimes 75/25, is understood to main “a large part of the time”/“a significantly smaller but still crucial amount off the time”.

> If you pay attention, so much user content is just repetition of something they read on the same fucking website and uncritically checked into their brain.

Which is exactly the problem. We now have LLMs doing the same thing, flooding everywhere with more of that noise, further hiding what good signal remains. Even if some of that good signal is by chance almost entirely LLM generated too, it appears to be at best a vanishingly small amount.

[–] californical 2 days ago

80/20 doesn’t mean literally 80% and 20%

It’s just a rough way to say “mostly this with a decent bit of that” because 20% seems like a medium-small percent that could or could not matter depending on the context

That’s all anyone means when they say 80/20

To your second point, the 80% could be lots of basic things and the 20% could be the important facts, so it’s not always meant as a dismissal that the 20 doesn’t matter

[–] seamossfet 2 days ago

Coding and math are easily verifiable, language and prose is not.

As a result, I don't think LLMs will reach this state easily.

[–] steampoweredslo 2 days ago

Even putting aside accuracy, if the plan is for the tools that verify the written content to be in any shape or form outside of the human loop, then this feels like it glosses over the issues of both passive bias in data & active bias (e.g data filtering etc. ) when training such models that you'd be poisoning this information sources with.

Even subtle bias can manifest in significant ways. Given that the predominant market leaders in producing such models are currently : A country that's attempting to squeeze the word "america" in any geographic formation possible + rewriting history to put itself first, and a country that has perfected censorship en-mass and already actively doctors its own history.

I'm not convinced giving LLMs carte blanche to write authoritative sources on topics that - often are ideologically sensitive/controversial isn't just needlessly reckless and short-sighted. It's not a far reach to imagine such models reflecting the current views of administrations in these countries and becoming reluctant to mention certain colonial ambitions, environmental damage, political corruption, etc. where it might be relevant to authoritative content.

Wikipedia already struggles with issues in idealogical bias from authors - human validation (somewhat)helps mitigate that. You could argue that human validation is still prone to same bias - but you can always draw on a more diverse pool of editors with little ideological overlap. With LLM validation you're pretty much at the mercy of a pretty incestuous pool of data & a handful of self-serving corporates & governments.

[–] adsharma 2 days ago

Latentpedia.org will not have these problems. The content is human produced, but the extraction of a graph and creating the latent space is AI driven.

[–] charcircuit 2 days ago

Grokipedia serves as an existence proof that it is possible.

[–] Cthulhu_ 2 days ago

I don't think anyone's arguing against whether or not it's possible, the question is whether it's valuable, correct, and A Good Thing.

I have access to a number of LLMs and can order it to build me a wiki site with all knowledge of the world, just like everyone else with the same access. What would make mine any more valuable or better than another?

[–] adsharma 2 days ago

Grokipedia's mistake is that it generates text statically instead of dynamically from a machine curated data structure in latent space.

Latentpedia starts from wikidata, not Wikipedia.

[–] boxed 2 days ago

https://en.wikipedia.org/wiki/Grokipedia#:~:text=that%20arti... It literally states itself on the frontpage that it's bad.

Funny though, as that admission of error makes it more reliable than if it tried to hide this fact.

[–] MetalBreaksAndB 2 days ago

Grokipedia lacks the fundamental quality that makes WP great, collaboration.

[–] charcircuit 2 days ago

Saying there is no collaboration is like saying multiple people on a software project can't collaborate with Claude Code because it's really just Claude writing the code.

[–] MetalBreaksAndB 17 hours ago

There's no on-platform way of communicating to other editors and I don't think you get responses from the site when you report errors. It's like saying multiple people on a software project can't collaborate with Claude Code because they're all just yelling at a single claude with no responses from claude or communication between people.

[–] tomaskafka 2 days ago

Funny, I wanted to mention grokipedia as an example of why a pile of slop that’s subtly wrong about many details is a bad idea.

[–] charcircuit 2 days ago

So is Wikipedia, but with grokipedia you can at least correct it even if there is not a journalist pointing out the obvious.

[–] cowlevel 2 days ago

Does Grokipedia even accept contributions that don't align with Musk's political bias?

[–] charcircuit 1 day ago

You are free to give any suggestion that you want.

[–] Zardoz84 2 days ago

A parrot can be never a true source of information.

[–] yreg 2 days ago

Wikipedia is a tertiary source. Its editors are parroting secondary sources. (No disrespect intended.) There's no reason why LLMs couldn't be good at that.

I would rather be worried about the quality of all the primary and secondary sources going forward.

[–] kzrdude 2 days ago

There is a reason why LLMs can not be good at this: They don't know what they are quoting or citing. A different kind of model with factoid provenance could do it?

[–] derektank 2 days ago

>They don't know what they are quoting or citing.

What do you mean by this? I have passed very large documents into LLMs and they are able to accurately extract quotes and identify their location in the source documents. I don’t see any reason to think that they couldn’t accurately quote from and cite existing literature.

[–] kzrdude 2 days ago

LLMs cook up new sentences based on probability, based on everything in their training and their context. There's just nothing in their model that traces provenance of facts, they physically don't store that information. Once they have written a sentence, they can't tell you from which work they got this information.

There are tools where they try to retroactively add references, the LLM interfaces try to do this, but it's usually inaccurate.

I think of the way a scientific article works. "The sky is blue (Pearson, 1992)." The writer is fully aware what the factual statement is and which publication supports it (not just that, but which kind of publication - primary, secondary source, peer reviewed?). An LLM can try to imitate this, but the actual information to support this does not exist in its training.

[–] clord 2 days ago

They can. I had one doing original OCR of a scanned piece yesterday. It found relevant page numbers and understood the context. It read the whole book to find the relevant cites. Even went though the genealogy table and found ground breaking references no one noticed before. Total cost under 5 dollars and time was less than 20 mins.

The hallucination era is rapidly ending.

[–] hahn-kev 2 days ago

People reading a source can't know if it's true just by reading it either (though they could be fairly sure if it's false sometimes)

[–] kzrdude 2 days ago

That's not how academic literature works. The structure is like this: Statement (Citation). The citation is not an affirmation of truth but a content declaration; this statement is supported by this publication. The receipts are there, so that you can trace the argumentation and make your own conclusions if necessary.

'Knowledge' formed this way is a living, evolving graph, and sometimes links are broken (findings are reevaluated or disproven); having the content declaration so clearly in the open makes it easier to deal with this evolving landscape.

[–] Cthulhu_ 2 days ago

Sure they can, as long as they can access online resources they can give you an exact list of sources they're using in their responses.

[–] gilrain 2 days ago

Then what you've said isn't true: it parrots an existing talking point, adding nothing.