link? care to share? :) i often find myself starting with a tone guide.

yeah i work on this at https://groundedai.company/

we build a product called Veracity focused on citation and claim checking. legal is a major focus for us, also scientific and technical publishing, where we detect hallucinated citations in submitted works and all that jazz

planning to release something soon focused on the LLM harness layer

we've been working on the problem for years. gradually getting traction

all completely closed source atm, though we often revisit that discussion

i would contest a sibling comment - the solution is not RAG, at least not entirely. 1) because it doesn't necessarily involve 'generation' (e.g our grounding model doesn't depend on gen ai) and 2) RAG is just enriching the context. even if you can get the retrieval right (not a given) it doesn't guarentee that the LLM will faithfully or correctly use that context, or that all its claims are supported by sources. RAG is not the same as traceability and verification

I'm working on something around this that I'm hopeful I can open source in the next couple of weeks. It's just a research project, so not like a product that can be bought and shipped to prod right away, but I'm excited about it. I think a lot of people are working on this right now, at least in part, with things like memory layers and other approaches to "grounding", or with some of the more recent stuff around certification and governance. But there's sort of a split right now between:

- Groups focused strictly on model capability (i.e. benchmark scores)

- Groups really focused on governance, interpretability, control planes, etc

- A small but excitable minority of weirdos (me included) who think they're the same thing

A lot of the stuff that gets loosely clustered under neurosymbolic methods might be interesting if you're looking to dive in. The sort of pattern you're describing is something that a lot of people have taken stabs at from different directions, or at least have tangentially nipped at while working on related problems.

The name of the solution is RAG - retreival augumented generation. It is pretty easy to whip up your own over a corpus of material. I built one for myself within a week over a corpus of my favorite cook and homebrewing books. All you need is source material, postgress (pgvect) and some compute (llama.cpp with a decent chat model for chunking and labeling, an embedding and a rerank model. or some $ to use those via a router). When the corpus is indexed, you use your favorite coding agent and a system prompt that forces the model to answer only from search results.

Not sure why a solution is even needed. If I had to compile all the resources I had consulted for something like C programming, they wouldn't amount to a single shelf. I would say the same thing for research where the amount of sources is still very low. It's kinda reminded me of the big paper manuals and books before the internet. I don't think the whole library of congress was needed to write UNIX.

So AI being a source of truth and information is not a something that important. It's a generator and we should use it as such, getting it to map domains and repeating patterns. It should not be used as an encyclopedia or oracle.

I think it's more that the majority of humans DON'T REALIZE it's NOT doing that.

Most people think AI answers are carefully sourced, triple vetted, over-researched to death and can trust it with their life, despite the warnings, until they run into nonsense and then dismiss it as a one-off.

If there was a "always give the best possible, most researched and trusted answer" flag that was easily visible, or maybe "chatGpt" and "bestAnswersGpt", where the second one did all the verification and was considered much better overall, guess which one people would start using after a while and/or quote, because of improved accuracy?

It's not that people don't WANT honest information (well, most people) - they just don't KNOW they're not getting it right now. That's the problem with tech - they're trading on "computers were always right in the past" because there has been no generally available "fuzzy search" type application that needs handholding to give "good" results. Most software either gave the result of computations that are always the same, or enforced static rules (like spellcheck) which were perceived as "always right".