7 ms·

Yes, they are similar. Actually, our initial paper was presented around five months ago (https://arxiv.org/abs/2311.09000 https://arxiv.org/abs/2311.09000). Unf

by Xudong 2y ago

Yes, they are similar. Actually, our initial paper was presented around five months ago (https://arxiv.org/abs/2311.09000 https://arxiv.org/abs/2311.09000). Unfortunately, our paper isn't cited by the DeepMind paper, which you may see this discussion as an example: https://x.com/gregd_nlp/status/1773453723655696431 https://x.com/gregd_nlp/status/1773453723655696431 Compared with our initial version, we have mainly focused on its efficiency, with a 10X faster checking process without decreasing accuracy.

westurner 2y ago
> We further construct an open-domain document-level factuality benchmark in three-level granularity: claim, sentence and document A 2020 Meta paper [1] mentions FEVER [2], which was published in 2018. [1] "Language models as fact checkers?" (2020) https://scholar.google.com/scholar?cites=3466959631133385664 https://scholar.google.com/scholar?cites=3466959631133385664 [2] https://paperswithcode.com/dataset/fever https://paperswithcode.com/dataset/fever I've collected various ideas for publishing premises as linked data; "#StructuredPremises" "#nbmeta" https://www.google.com/search?q=%22structuredpremises%22 https://www.google.com/search?q=%22structuredpremises%22 From "GenAI and erroneous medical references" https://news.ycombinator.com/item?id=39497333 https://news.ycombinator.com/item?id=39497333 : >> Additional layers of these 'LLMs' could read the responses and determine whether their premises are valid and their logic is sound as necessary to support the presented conclusion(s), and then just suggest a different citation URL for the preceding text > [...] "Find tests for this code" > "Find citations for this bias" From https://news.ycombinator.com/item?id=38353285 https://news.ycombinator.com/item?id=38353285 : > "LLMs cannot find reasoning errors, but can correct them" https://news.ycombinator.com/item?id=38353285 https://news.ycombinator.com/item?id=38353285 > "Misalignment and [...]"

by Xudong 2y ago

westurner 2y ago

> We further construct an open-domain document-level factuality benchmark in three-level granularity: claim, sentence and document A 2020 Meta paper [1] mentions FEVER [2], which was published in 2018. [1] "Language models as fact checkers?" (2020) https://scholar.google.com/scholar?cites=3466959631133385664 https://scholar.google.com/scholar?cites=3466959631133385664 [2] https://paperswithcode.com/dataset/fever https://paperswithcode.com/dataset/fever I've collected various ideas for publishing premises as linked data; "#StructuredPremises" "#nbmeta" https://www.google.com/search?q=%22structuredpremises%22 https://www.google.com/search?q=%22structuredpremises%22 From "GenAI and erroneous medical references" https://news.ycombinator.com/item?id=39497333 https://news.ycombinator.com/item?id=39497333 : >> Additional layers of these 'LLMs' could read the responses and determine whether their premises are valid and their logic is sound as necessary to support the presented conclusion(s), and then just suggest a different citation URL for the preceding text > [...] "Find tests for this code" > "Find citations for this bias" From https://news.ycombinator.com/item?id=38353285 https://news.ycombinator.com/item?id=38353285 : > "LLMs cannot find reasoning errors, but can correct them" https://news.ycombinator.com/item?id=38353285 https://news.ycombinator.com/item?id=38353285 > "Misalignment and [...]"