8 ms·

Quality non-fiction books are the antithesis of AI slop

492 points by benbreen 2mo ago

https://book-prize-index.vercel.app/ https://book-prize-index.vercel.app/

damnesian 2mo ago
[flagged]
- dang 2mo ago
  Can you please review the site guidelines (https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html) and stick to them when posting? You broke several of them here. "Don't be snarky." "Omit internet tropes." (the stopped-reading-at bit) "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." Edit: it looks like your account has been posting quite a few flamebait and/or unsubstantive comments generally. Could you please not do that? It's not what this site is for, and destroys what it is for.
northhex 2mo ago
Thank you for sharing this. I do think book prizes are a better-than-average signal but I previously volunteered with a book award. I will caution that pretty much every publisher mass-submits these books for consideration in every remotely relevant prize. It is a cost of doing business (similar to how photographers pay to enter photography competitions to try and win the "award winning photographer" title, or businesses submit dossiers with consideration fees on why they're one of Michigan's top 100 places to work). There are often so many books and so few willing qualified readers that which books get an award can either be completely arbitrary, or comically easy. For example, the NCR Book Award in your book corpus faced a big scandal when it was revealed that the judges did not read the books themselves. [1] The PROSE award is so comically large that an ordinary category finalist or win is usually overstated in prestige and value. [1] https://www.theguardian.com/news/2013/may/19/literary-prize-judges-admit-failure-read-books https://www.theguardian.com/news/2013/may/19/literary-prize-...
- throwaway219450 2mo ago
  Awards are also a product of their time, representing biases among the judges, literary preferences from the readership, and straight up nepotism/favoritisim/elitism. Look at the Hugo/Nebula/Locus and you'll find a lot of so-so winning work throughout the years; especially when you look at the other nominees.
- stdbrouw 2mo ago
  Same thing with beer and wine awards, some are good but most are just a paid promotion lottery.
  - duttish 2mo ago
    Which are the good ones in your view?
    - raddan 2mo ago
      People’s tastes in beer and wine vary so dramatically (and there are so many styles) that it’s not clear to me that you can actually have a “best of.” The beer and wine I enjoy the best my wife finds completely unpalatable. For wine, I enjoy a dry Grenache; Mas de Gourgonnier is a good one. For beer, I enjoy a beer with body, a malty profile, and balanced hops; there are many mass-produced styles that fit this, but I like Sam Smith Organic Lager, and Leffe Brune. I’ve also brewed many beers myself and one of my all-time favorites was an all-grain formulation of a honey ale [1]. Such is the nature of beer and wine (and HN) that someone is going to “disagree” with me. That’s fine. Just go try some things and see what you like. [1] https://en.wikipedia.org/wiki/White_House_Honey_Ale https://en.wikipedia.org/wiki/White_House_Honey_Ale
      - duttish 2mo ago
        Sorry, I meant which awards not actual drinks :)
TMWNN 2mo ago
I read at night in bed. I love how, with ebooks (first Overdrive, now Libby), when I hear about an interesting book, I can within 30 seconds search for it across multiple libraries, borrow it, and send it to my Kindle.
DangitBobby 2mo ago
At least one of the links on the post is still to localhost:3000 instead of the vercel page. I don't want to make a substack account to tell the author so hopefully this information finds its way over there.
adamddev1 2mo ago
It's sad now to see a university library where students sit among endless shelves of amazing books, sitting on laptops, almost all of them with ChatGPT open. The library has become just a place to sit and open an LLM.
- DubiousPusher 2mo ago
  It's especially sad because on of the things those LLMs are best, almost purpose built for is to tell those students which books to go open, where to find nuggets people haven't bumped into for years, to make cross-connections that would take a PhD a decade to find.
  - nephihaha 2mo ago
    This is a very good point. If it should be used as anything, it is as a pointer to other things.
- Der_Einzige 2mo ago
  The most sad part of all is to think how little of that knowledge is digitized and available to be referenced by ChatGPT. Digitizing old books is one of the highest things you can do as a human for “good for the world”.
Planktonne 2mo ago
I think my issue with this project--and so many other similar ones--is that the provenance of the code does undermine the intention. If a project purports to be about quality, then knowing that the creator abdicated some of the responsibility for creating the thing they ostensibly care about makes it harder to put faith in them as having high standards elsewhere. Perhaps I am just old-fashioned, and vibe-coding is something that can be done with full focus and care for high quality, but I remain unconvinced. This is a project that needs to be cared about sincerely to be trustable/meaningful/useful, and the approach taken casts doubt on that.
zahirbmirza 2mo ago
Fascinated that the author shares an experience of discovery in libraries. The internet, in its early days felt like that. Of course, no longer. Libraries are certainly declining in their traditional form. I find it odd that everyone has a digital resources in their pocket yet libraries are squeezing out physical books to make way for more and more computers. Try to find paper copy of the Sony founder's book.... that will be £50 on amazon. Prohibitive. No library within a 10 mile radius has a copy. I remember joyfully discovering Tony Royce's book in my library a few years ago. That enlightenment will never happen now. Primary knowledge is being lost, churned crude will forever lubricate the delusions of those who have no facility to collate the basis of our understanding.
- sandspar 2mo ago
  AIs are fantastic for discovery. Just today I asked Chatgpt to find blogs 1) about old school newspaper cartoons 2) that have been writing for more than 10 years and 3) are run by one or two passionate people rather than a team. Within seconds Chatgpt found 7 candidates. Within another handful of seconds it gave me RSS links to the ones I wanted. This is incredible!
  - nephihaha 2mo ago
    Sometimes. I think this is partly because search engines are so poor nowadays. I have asked AI for certain websites that I know exist but it can't find them.
- zem 2mo ago
  > The internet, in its early days felt like that. Of course, no longer. back in the reasonably early days of the web (I would guess ~2000) I stumbled across a webpage that said "I used to collect random interesting snippets in a shoebox, here is the internet version of it". I was delighted enough that I actually wrote to the author to compliment them on helping keep the web interesting, and though I haven't thought about it in ages it clearly made enough of an impression on me that I remembered the author's name. and it's still up! https://www-users.york.ac.uk/~ss44/cyc/index.htm https://www-users.york.ac.uk/~ss44/cyc/index.htm
DubiousPusher 2mo ago
My bona-fides here amount to little more than being a big time reader of non-fiction. But IMO, there's a nice synthesis here. Rather than going long rounds of asking LLMs about a subject, I usually end up asking it for book recommendations. It's much better than a google search and you can push it into some deep corners if you go past the surface level recs. You can get quite specific. You can find texts you wouldn't discover unless you spent years studying the topic. Often these are completely approachable and give interesting perspectives they just get buried behind a wall of syllabi and listicles. This is how I ended up reading Thompson's 'The Making of the English Working Class' and Graves' 'Goodbye to all That' among others.
- philips 2mo ago
  I did this for a recent trip! I asked books related to or authored in the places I was visiting. Found some interesting titles.
jeffreyrogers 2mo ago
I would bet that how your brain stores information that you read from long-form text is very different from how it stores information you acquire from chatting with an LLM. When I read something challenging or new to me I spend a lot of time thinking about how what I'm reading matches my own experiences or knowledge. Although I'm a fairly fast reader, it often takes me a long time to get through difficult pages since I have to stop and think about what I'm reading. I seem to be doing a lot of integrating and reorganizing my thoughts. When interacting with LLMs it feels a lot more like I'm just receiving knowledge passively and I don't think it gets integrated as well. Not sure why this is and its somewhat counterintuitive since I don't think I'd have the same experience with a human tutor.
- m463 2mo ago
  I listen to audiobooks while I walk and hike, and sometimes when I recall some particular thing I've learned, I can also recall where I was hiking when I heard it. I think there's a lot to learn on how we really process information.
  - tony_cannistra 2mo ago
    Same here. I always marvel at this. These recollections can even be years later, and the memory of the place in which I heard the remembered thing is vivid.
  - annzabelle 2mo ago
    There's a podcast I listened to as it came out when I was a teenager. I'm now relistening to it over a decade later, and I distinctly remember walking home from the bus stop with it coming through wired earbuds from my ipod.
  - ahartman00 2mo ago
    "when events are represented in memory, contextual information is stored along with memory targets; the context can therefore cue memories containing that contextual information" https://en.wikipedia.org/wiki/Context-dependent_memory https://en.wikipedia.org/wiki/Context-dependent_memory
EdwardAF-IT 2mo ago
[dead]
adamtaylor_13 2mo ago
On a related note, I was just noting to my co-founder, as we struggle to write good case studies for our website, that I find LLMs are astoundingly bad at writing good prose. We all know the "AI-tics" that give away a sloppily AI-written piece, but even if you steer them, they still struggle to write consistently high-quality prose. Somehow I feel that the work of a good copywriter has never been more noticeable.
- mpalmer 2mo ago
  I've not really enjoyed finding out lately just how few people seem to notice what ought to be unmissable.
  - cyanydeez 2mo ago
    those people will also start adopting the AIsm and will become indistinguishable.
    - Mtinie 2mo ago
      AI adopted humanisms, we just weren’t used to seeing them at the same scale we do today. Diversity of writing styles was part of that, but I’d point to vernacular exposure as the larger component. We’re going to go through a period where we try to adapt to a form of “Universal English” for those of us who read primarily English writing. Other languages’ readers may be experiencing the same dissonance when they come across AI-generated prose in their native language (but I’ll let others validate /reject my hypothesis).
      - cwnyth 2mo ago
        This is right. AI is using human speech, but just doing so in a consistently peculiar way. The em-dash in particular is frustrating, because it's all over high-quality, pre-2022 academic work. But now instead of proper and erudite it's seen as AI-slop. Well, maybe if they didn't use it — all — time — ! It's even an auto-replacement in Word and can be (by one's choice) in LibreOffice, replacing three dashes (and two is replaced by an en-dash). But when I submit a novel with em-dashes, will sloppy agents and sloppy editors be able to tell that em-dash was deliberately put there by me?
paxys 2mo ago
Very neat site and write-up, but I found this part amusing: > There is really nothing “AI” about this aside from the tool that collected the data and coded it, and, crucially, semantic search […] So really, everything about it is AI. And that’s not a bad thing! It’s okay to simultaneously preach the superiority of award winning books over AI-generated garbage while also acknowledging the same AI as a valuable tool for other uses.
- therealpygon 2mo ago
  Key word being tool. People get lost in a fantasy that the AI was responsible for decisions and not the human who chose to use it as a replacement for effort rather than a tool to enhance effort, then apply their bias to all AI use.
- calvinmorrison 2mo ago
  so much of AI is like, to me, used as was to generate basically dynamic applicaions. if we ever got anywhere with semantics, objects, etc, pushed html further, maybe we'd naturally have web clients that could take objects, natively search, collate, display, put on a map, graph, etc... but AI is like a text only shortcut to the end result we want. I have a stupid AI app I use now for some bookkeeping which was previously, just an excel sheet. except it has a UI, it validates inputs, and does a bunch of other housekeeping stuff that makes it nice. Did the excel work? yes totally.
replatformradar 2mo ago
Is it even possible to find one these days? I would love to see some real honest amazon kindle stats on the amount of ebooks added over the last 3 years. Even a honest pie chart to show people its not worth it to get them to stop posting them.
wackget 2mo ago
> Decries AI for producing slop > Uses AI to produce website
- frollogaston 2mo ago
  The article isn't even about AI slop other than the title for some reason
  - llm_nerd 2mo ago
    It's a clever title because it's FREE MONEY on HN. There is a small but rabid group that upvotes everything that denigrates AI, sight unseen.
  - deleted 2mo ago
    [deleted]
OgsyedIE 2mo ago
Suggested additions: the Axiom Business Book Awards, Library Journal Best Books of the Year and Booklist Magazine's Editor's Choice Awards. There's also a few popular substacks that have big sideshows in regular book reviews, but I wouldn't even recommend my favourite for a public aggregator like this.
- deleted 2mo ago
  [deleted]
- benbreen 2mo ago
  Original author/creator of the site here - thank you! Will add the Axiom one. I am on the fence about whether/how to add "book of the year" type lists as they are somewhat distinct from book awards, but I do think that would probably be the next step to get more books in the corpus. And any other ideas that HN readers have for awards to add would be welcome. Currently it's probably too history-slanted since I'm a historian and knew those awards better.
  - OgsyedIE 2mo ago
    It may be hard to parse the lists but if they're still available somewhere online under the current administration the past annual State Department, CIA bookshelf recommendation, Army Chief of Staff, Navy CNO and Marine Commandant reading lists may have some valuable additions. To my knowledge there aren't any analogues to them in the rest of the Anglosphere but some LLMs will probably point out ones I haven't thought of. I can point out that the UN agencies' respective reading lists for their staffs' professional development are 99% internal UN papers of very limited interest to external audiences but I haven't had a professional reason to look at any EU agencies to confirm or deny value. Additionally, the Financial Times has a second set of book awards along their main awards which you've not listed, the FT reader's best books list.
maxdo 2mo ago
I participate book club for several years with friends, we casually navigate certain topics, and i can tell you human slop in literature is a real thing. - almost every book try to stretch core idea into book size format - unique ideas are rare people attack them under different angels - a later phenomena : their believes almost predict entire book, outcomes etc, brainwash impact is real so not sure, how ai slop is better vs book slop, at least with ai you can distill the idea, with the book, you have to spend 10-40 hours to digest average, absolutely non fresh ideas, that author brought in just to sell that book, otherwise it would be magazine article worth.
- tolugenius 2mo ago
  In your experience, how do you determine book slop? For me I don't really stray from authors and topics I like, and non-fiction isn't really my go to outside of sci-fi, and mystery/law books. Like on the stretch point, I can tell that in other media but not so much books.
  - jonfromsf 2mo ago
    Business books are 99% slop.
    - vkazanov 2mo ago
      Self improvement books! All of them! "Make good habits, not bad habits because this is what Lincoln did..."
- Der_Einzige 2mo ago
  This is sadly correct/true. This is one of the many reasons why I militantly reject carbon chauvinism. Most of the potshots people take against AI can be easily retorted with “have you seen how bad the average human is at X?”
- paxys 2mo ago
  They’ve used a pretty good filter for the human slop - prestigious awards.
diego_sandoval 2mo ago
The title is tautological. Quality is the opposite of slop, AI or not.
- janalsncm 2mo ago
  Not really. Quality nonfiction books are a subset of quality books.
titanomachy 2mo ago
Thank you for this. I sorted the "technology" and "science" sections and saw a few excellent books that I've read and a few that I would really like to read. This motivates me to start setting aside "reading time" again every day, since I've lost that habit. The recent "society & culture" books gave me some good book club ideas. Bug report: filtering by "award" appears to be broken for some awards. If I select Pulitzer or National Book Award, no books show up, but I can find books with these awards by browsing.
- benbreen 2mo ago
  Awesome, that's what I was going for. I originally made this for my own use and got kind of obsessed with building it out once I started finding unfamiliar books I enjoyed using the semantic search and category browsing. Glad to see it out in the world hopefully doing the same thing!
  - titanomachy 2mo ago
    One thing that would make this information even more useful and searchable would be a JSON dump of your dataset. You could serve it statically on S3 or similar, to keep hosting costs to a minimum. Of course, having created a cool thing generates no obligation for further work on your part! I'm grateful that you did this work and made it available for free. Did you build your semantic search index based on the full text of the books, or just the reviews and descriptions?
    - benbreen 2mo ago
      Already tried to do this in a basic form. Advice from experts would be welcome as it's my first time trying to make data and an API available: https://book-prize-index.vercel.app/data https://book-prize-index.vercel.app/data Also I just had to upgrade my Vercel plan due to traffic, which is very welcome and appreciated, but if anyone wants to donate, that would be really helpful! There's a Stripe link at the site: https://book-prize-index.vercel.app https://book-prize-index.vercel.app
why_at 2mo ago
This is pretty nice, I might use this if I'm looking to learn more on a topic. I'm always cautious when reading nonfiction because it's hard for me to tell if the author knows what they're talking about when I'm not an expert myself. Using awards is a good metric. Maybe. I am curious what the "score" for each book means. Is it calculated by giving each award a certain weight and adding them up?
zem 2mo ago
possibly a bug - I searched for "writers like lewis thomas" and the top two results were by thomas himself.
- olooney 2mo ago
  On a similar note: on Steam, the original Dark Souls game[1] is listed with the "souls-like" tag. That's not wrong, I guess, but just kind of a tautology. Grice's maxims[2] generally have such tautologies omitted in ordinary conversation as non-informative. [1]: https://store.steampowered.com/app/570940/DARK_SOULS_REMASTERED/ https://store.steampowered.com/app/570940/DARK_SOULS_REMASTE... [2]: https://en.wikipedia.org/wiki/Cooperative_principle https://en.wikipedia.org/wiki/Cooperative_principle
  - vkazanov 2mo ago
    For me it is surprisngly natural to include the original thing into "a set of thing-likes". Why does it feel wrong (or meaningless) to you?
  - inigyou 2mo ago
    I really thought ChatGPT would be able to tell me what this is called but it can't. I suggest "circular eponym". It's like calling a CRT monitor a really big flatscreen.
    - zem 2mo ago
      it's more like you would call a xerox-brand photocopier a "xerox machine" and mean that it was a photocopier rather than that it was made by xerox.
- yellow_lead 2mo ago
  For embeddings based search I think this is unavoidable
anvuong 2mo ago
Is it only me or the titles of top books from History, Nature, and Politics sound very click-baiting? I'm really taken off by these titles.
- DonHopkins 2mo ago
  Hey buddy, are you feeling all right?
janalsncm 2mo ago
Worth stating that this is one of the success stories from AI. Someone who has domain expertise outside of programming is able to create a really useful piece of software because the barrier to entry has been significantly lowered. Really nice.
- deleted 2mo ago
  [deleted]
762236 2mo ago
I've gotten AI to write things that made me teary eyed --- it manipulated me
- inigyou 2mo ago
  I made a sandwich yesterday and cried for an hour because it reminded me of the way my mom used to make school lunches. No AI necessary.
mrkpdl 2mo ago
This is wonderful. 30 seconds into trying the site and I’ve already found a couple of interesting books to read. The search I typed is ‘Stuff being made, how it was made, how history shaped its form’.
narrator 2mo ago
The antithesis of AI slop is when you've created a unique idea from first principles and AI keeps misunderstanding your idea and spell checking it back to something that's widely popular such that you have to write a whole extra alignment subsystem to check that it hasn't ignored half of what you've wrote in your first principles document and just thought you were talking about something distantly related in its training set. To work on something truly original in non-fiction in the future against AI pushing everything to the mean of its training set and fighting against that when doing research, as opposed to generating a whole book of slop from familiar ideas, is going to be quite the unrecognized effort of writing those books.
hahahaa 2mo ago
Literally reading the books like an LLM would be trained on them!
jocelyner 2mo ago
[flagged]
Balgair 2mo ago
Awesome stuff! Now, can this interface with Libby? The worldcat interface is, well, confusing. I can't figure out if it knows libraries near me exist.
fuddle 2mo ago
Nice site, but most of the top biographies are US figures: Edgar Hoover, Frederick Douglass, Lyndon Johnson etc. I'd like to see more non-US biographies.
Nevermark 2mo ago
I don't think we would have AI slop, if they were just trained on high quality media. Less relatable AI, maybe, but not sloppy AI. AI trained on other AI (i.e. distilling), can be better than the AI they were trained on, likely because of this reason. Each iteration can raise the bar. As apposed to the deep spiral of generating shiny attention crystals for humans, and then training on that.
- deleted 2mo ago
  [deleted]
- inigyou 2mo ago
  The slop isn't a property of the LLM but the workflow it's utilized in. Just now I asked ChatGPT about the linguistic effect where a CRT could be described as a giant flatscreen, a landline could be described as a wired cellphone, or the original Rogue could be described as a roguelike. Unfortunately it doesn't know what that's called either, but that is a non-slop use of an LLM, like a semantic search engine. On the other hand if you print LLM output into a book it's almost certainly slop no matter how the model was trained.
- dncornholio 2mo ago
  Training AI on AI will most likely increase slop. LLM's add randomness in their outputs, books usually don't.
  - Nevermark 2mo ago
    That isn't correct. Obviously, you can handicap any model with an indiscriminate dataset. Are you training on slop? Why? More data is not automatically better data. Just three (of many) advantages of quality data selection: (1) It gives a clearer signal. (2) It, ironically, reduces the complexity of what needs to be learned, since slop adds its own complexity. (3) It reduces dataset size which means more learning per watt (and per just about any other cost). The advantages compound. Humans are no different. Children surprise with their ability to absorb sophisticated relationships and skills, while people who had bad examples struggle to recover. 99% of human effective intelligence is higher quality cultural knowledge learned as children. None of us had to spend centuries deciding whether zero, negative numbers, the square root of -1 are numbers. Slop + quality is not quality.
whattheheckheck 2mo ago
And what pray tell is everyone doing with their privileged leisure time? You have access to every single wicked problem in the world and you choose to do nothing. Shame
nttylock 2mo ago
[flagged]
seizethecheese 2mo ago
The article resonates, there is great serendipity in somewhat random exploration of books. But then the web app seems to be the antithesis of this? I clicked a few categories and the top ranked books all had the “popularity contest” smell, sort of the antithesis of random exploration.
abc42 2mo ago
I like very very much that there's no user voting system here. I just recently read a really mediocre book with a 4.7/5 rating on Goodreads. Shows being sloppy is not exclusively a privilege for neither the AIs or producers.
rincewind 2mo ago
I'm not sure how I feel about "Entangled Life" on the first page of books in "science". Isn't that slop? I mean, it's pop-science bullshit, and it's based on feelgood extrapolation from a thin evidence base. Maybe that doesn't quite fit the definition of slop, but it's not a signal of quality for the rest of the list.
insane_dreamer 2mo ago
Thank you! Wonderful idea, and very well executed. Useful discovery tool. I understand your passion is non-fiction, but would be wonderful to have the equivalent for fiction, of which there is a great deal of excellent literature. Perhaps the same site could incorporate both fiction and non-fiction, with a filter?
gabriben 2mo ago
Thanks for this! I've been looking for a good book on the Spanish Civil War, in either Spanish or English. The one American book I was recommended by someone, I didn't like. I'd like to keep searching regardless of the book's language or what country it has won prizes in. Could you also include book awards from other non-english speaking countries?
- nsmog767 2mo ago
  +1 on this!
- gorkaerana 2mo ago
  Books on the Spanish Civil War I have read and can wholeheartedly recommend: - "The Spanish Civil War" by Paul Preston - "Homage to Catalonia" by George Orwell Books on the same topic on my reading list (and quite frankly can't wait to get hold of them): - "The Spanish Civil War" by Hugh Thomas - "The Fight for Spain" by Antony Beevor
  - embedding-shape 2mo ago
    I've only read Homage to Catalonia out of those, which is a personal retelling from Orwell, are the other books in the same style? What's the difference you'd say between the ones you list?
    - gorkaerana 2mo ago
      Paul Preston's "The Spanish Civil War" would be the poster child of the "quality non-fiction" category specified in this post's title. He is, in my opinion, the leading Hispanist historian [1], and I hold his work in the highest esteem. I can also very much recommend his book "A People Betrayed". Regarding the other two authors in my previous comment: Hugh Thomas was also a leading Hispanist historian [2], and Antony Beevor is a prominent military historian [3]. I expect both books to be in the same vein as Preston's. (Quick aside for the astute reader that noticed all of the recommended authors being non-Spanish: I find Spanish politics extremely polarized; the Spanish Civil War being perhaps the most polarizing topic. As such, I find outsider's perspectives to be more balanced and objective.) References: 1. https://en.wikipedia.org/wiki/Paul_Preston https://en.wikipedia.org/wiki/Paul_Preston 2. https://en.wikipedia.org/wiki/Hugh_Thomas,_Baron_Thomas_of_Swynnerton https://en.wikipedia.org/wiki/Hugh_Thomas,_Baron_Thomas_of_S... 3. https://en.wikipedia.org/wiki/Antony_Beevor https://en.wikipedia.org/wiki/Antony_Beevor
codedump 2mo ago
[flagged]
theoneone 2mo ago
real nice work!
sawrtagy 2mo ago
[dead]
rimworld 2mo ago
why read when you can flick on the tube?
dash2 2mo ago
I would really like this, but using academic reading lists instead of prizes. What are the important historians of (e.g.) medieval Italy?
Elkanumanos 2mo ago
Thank you for sharing. This will come in handy for my research!
alansaber 2mo ago
Fiction genre catching strays as usual
claudiacsf 2mo ago
But I get triggered by the em dashes. :(
mrich 2mo ago
Could you please add a link to the Goodreads page of each book?
keiferski 2mo ago
Not to be intentionally contrarian, but I have thought a lot about this: I think good fiction is actually the antithesis of AI, fundamentally. LLMs are really only capable of combining different things into output. This can give you some creative results, but it is ultimately kind of limited. I cannot imagine that prompting an AI a thousand times would give you something truly original in the way many high-quality fictional stories are. The magic and absurdity of life is a necessary thing to create such works.
- newguy1000 2mo ago
  you have a bad imagination then
- brrrrrm 2mo ago
  Models are increasingly showing their ability to extrapolate into the human unexplored (math proofs being the most apparent). What gives you confidence the absurdity of life is uniquely difficult for models to source?
  - eddythompson80 2mo ago
    Mathematical proofs are hard to come to, but easy (or “cheap”) to verify. Also having a math proof processed through a transformer to derive an output that matches existing proof patterns kid of makes sense. The former allows you to generate hundreds or thousands of iteration and cheaply verifying its correctness until you hit the right randomness level. Generating thousands of fiction books then verify them doesn’t really work. Most good fiction authors are trying to communicate a story that’s forming in their minds. Even when they get stuck, they are usually not stuck because they just don’t know anyone to ask. It would’t be their voice anymore. Some do seek input and those might experiment with llms for ideas. Using llms for research, technical questions, grammar, synonym sentences, word selection, etc makes sense too. I’d agree with OP for people generating full stories through llms. While I say that, I know a very young person who told a while back (like early ChatGPT 4 time) they use it to write fan fiction for books or characters they like.
  - keiferski 2mo ago
glasffordd 2mo ago
[dead]
deton3991 2mo ago
This hits the nail on the head regarding the real utility of AI right now. The magic isn’t in using LLMs to generate endless reams of synthetic text; it’s in using embeddings and semantic search to navigate high-signal, human-curated data.
- deleted 2mo ago
  [deleted]
haritha-j 2mo ago
I think your amazon links are broken, the ones i tried all 404.
dustypotato 2mo ago
Tangentially related, but what a snappy webpage
abrezas 2mo ago
The history section of the book price index is incredibly america-biased. History of america, american natives, japan after ww2 and vietnam in the last century.
- bluGill 2mo ago
  That is what Sells. The USA has a lot of people so selling to us is going to make money even in a niche subject like history. Historians who are not trying to sell mass market books have a much broader set of books. You won't see them in a normal bookstore.
- diego_moita 2mo ago
  And the irony of it is that the U.S. is actually quite history-poor, when compared to Europe, North Africa, Middle-East, Persia, China, Central Asia and India.
rntksi 2mo ago
Thanks for making this, I searched up economics and found at least 3 titles that , when reading through first pages, are really really interesting.
chegra 2mo ago
If you are interested in a non-fiction book, check out Veritas: Truth Across Cultures[1]. It's a collection of quotes/saying/proverbs that are similar between cultures, which I call truths. I am in interested in long term thinking, things that stood the test of time. This was primarily put together for myself as a lens to view the world. [1] https://www.chestergrant.com/veritas https://www.chestergrant.com/veritas
gavin_gee 2mo ago
the cope from authors.
zkiihne 2mo ago
Not to be that guy but your site is terrible. It does not do the most important function well, giving me a list of award winning books that is easy to sort through. It is not clear what is a link and what is not. It looks fine though somewhat generic. But it is not actually functional in a real sense.
diego_moita 2mo ago
When I see lists like this I am surprised that they always live out school textbooks. School textbooks are the kind of books that have the very precise purpose of giving you a foundation and roadmap to explore and understand a whole field of knowledge. And they're an industry that has been improving them on successive iterations to produce highly organized, comprehensive and accessible reading material through years.
- gwbas1c 2mo ago
  School textbooks also have a lot of bias, and may oversimplify issues in ways that are necessary with children, but lead to longing misconceptions. IE: Was the US Civil War fought over slavery or states' rights? Someone's answer may come from the biases of the textbooks they grew up with.
  - diego_moita 2mo ago
    > School textbooks also have For me the keyword in your comment is "also". What this word means to me is that, whatever faults textbooks might have, there is no alternative solution. They replicate a problem that all books have. However, within the domain of textbooks, you still might find good solutions for these bias. These bias are mostly within history and social sciences and good and reputable Universities are renowned for publishing books in these domains that are quite balanced, comprehensive, deep and rigorous. Oxford and Cambridge are the 2 best examples that come to my mind. The U.S. has good Universities too, but after their government declared a war on science and knowledge I'd be carefull before trusting these universities.
throwaway27448 2mo ago
I think the antithesis of ai slop is meditation on a nice beach. Or eating like a really good meal. Or holding your child.
weird-eye-issue 2mo ago
Sweet, I think I found a few to read > And before you wonder, yes this is actually free. I am paying for the hosting and the API costs entirely because I just want people to find and read more good non-fiction books. Just to note, you do have affiliate links so it isn't 100% out of the goodness of your heart ;) Not blaming you or anything, just saying...
JKCalhoun 2mo ago
"To avoid the tedium, I decided that I would also flip to a random page of every book I shelved and read a random sentence from it." Also, this is about the lost joy of "browsing". I wonder how much serendipity played when wandering through a library (or other things in life—even window shopping). But now, everything on the internet, you more or less have to know what it is you're looking for—are rarely "surprised or delighted".
- tombert 2mo ago
  I’m not sure I agree with that. I’ve found plenty of interesting things by randomly wandering around on Gutenberg.org, completely undirected.
  - JKCalhoun 2mo ago
    If they have a sort of "wander UI", that's cool. I'm not sure what that looks like though.
sparklingmango 2mo ago
Can I make a feature request? I would love to be able to search by author. I recently started Caro's LBJ series (it's amazing so far) and I'm curious how many awards he's won. Anywho, I love non-fiction so I plan to visit this website often. Thank you and great work!
- FabHK 2mo ago
  More generally, just sort by any of the column headers (year, title, publisher, author, score, ...) by clicking on it. In some views it works, in some it doesn't. A bit unintuitive. The toggle between "traditional" text and "semantic" search is cool and new, but maybe the toggle could stay visible (it's there on the home tab, but then not anymore on the books tab).
VoodooJuJu 2mo ago
[dead]
32-ai-toasters 2mo ago
maybe I am being too small brained black/white about this, but something feels amiss about the idea of vibecoding a solution to an AI generated problem. How can we get better, doing the same thing that made us sick?
keithbucklen80 2mo ago
I love the "books for dads who like Pavement" prompt! It would be great if you could incorporate the same experience of "browsing" random page of some of the results that are returned
MillironX 2mo ago
I very much have had the same experience as the author with research libraries. I have learned so many life-changing techniques and derivations from books that were two or three down from the book that I was looking for. When I became a research associate, I would often have my librarians ship the books to the branch closest to my office, and I stopped browsing the shelves myself. It wasn't until I went to the main library that I checked out a book several over from the one I had found in the catalog that I had a major breakthrough. Now that same library has collapsible shelving that makes browsing feel like a chore, and it makes me think we lost something valuable. That said, I had high hopes for the catalog that the author had put together, but for my technical fields, I found it lacking. Trying to find books that would be close to van Soest's Nutritional Ecology of the Ruminant or van der Werf's Animal Breeding: Use of New Technologies, yields a whole bunch of pop-sci titles. The joy of finding these titles at the library in the past was how remarkably understandable and engaging they were while remaining scientifically rigorous. I immediately feel that the pop-sci titles that I was recommended are going to try to sell me something or some idea, rather than letting me explore ideas myself. I guess that means I need to go back to browsing library shelves myself.
subset 2mo ago
As it happens, I had the same job as the author at uni. I shelved books at the biomedical and sciences library. As a life-long book lover, it was a dream uni job being surrounded by all sorts of historical treatises on medical arcana. I've been really exercising my non-fiction reading muscles of late. I'm trying weaponise it as an antidote to AI slop and doom-scrolling. My last read was Designing Data-Intensive Applications by Martin Klepmann (the newly released second edition), it was a wonderful read!
jschveibinz 2mo ago
Serious question: can we define the top 5 parameters that distinguish human-generated non-fiction writing from AI "slop" writing (I don't like the term "slop" because it carries a built-in bias)? Since AI output is based on the most likely next word/token given the context, let's consider how humans do this, i.e. choose the next word based on context. Is one of the parameters the degree of "randomization"? Are these other parameters? - Specific experience? - Recent exposure to relevant materials, i.e. paraphrasing? - Borrowed knowledge from a specific resource? - Writing style, i.e. choice of words or sentence structure? I'd like to hear what others think.
naiveter 2mo ago
Penguin Random House seems to be dominating the leaderboard. What's the story here? Are there only very few non-fiction publishers capable of publishing high-quality work, or is Penguin Random House exceptionally good at it?

492 points by benbreen 2mo ago

https://book-prize-index.vercel.app/ https://book-prize-index.vercel.app/

damnesian 2mo ago

[flagged]

dang 2mo ago
Can you please review the site guidelines (https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html) and stick to them when posting? You broke several of them here. "Don't be snarky." "Omit internet tropes." (the stopped-reading-at bit) "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." Edit: it looks like your account has been posting quite a few flamebait and/or unsubstantive comments generally. Could you please not do that? It's not what this site is for, and destroys what it is for.

northhex 2mo ago

Thank you for sharing this. I do think book prizes are a better-than-average signal but I previously volunteered with a book award. I will caution that pretty much every publisher mass-submits these books for consideration in every remotely relevant prize. It is a cost of doing business (similar to how photographers pay to enter photography competitions to try and win the "award winning photographer" title, or businesses submit dossiers with consideration fees on why they're one of Michigan's top 100 places to work). There are often so many books and so few willing qualified readers that which books get an award can either be completely arbitrary, or comically easy. For example, the NCR Book Award in your book corpus faced a big scandal when it was revealed that the judges did not read the books themselves. [1] The PROSE award is so comically large that an ordinary category finalist or win is usually overstated in prestige and value. [1] https://www.theguardian.com/news/2013/may/19/literary-prize-judges-admit-failure-read-books https://www.theguardian.com/news/2013/may/19/literary-prize-...

throwaway219450 2mo ago
Awards are also a product of their time, representing biases among the judges, literary preferences from the readership, and straight up nepotism/favoritisim/elitism. Look at the Hugo/Nebula/Locus and you'll find a lot of so-so winning work throughout the years; especially when you look at the other nominees.
stdbrouw 2mo ago
Same thing with beer and wine awards, some are good but most are just a paid promotion lottery.
- duttish 2mo ago
  Which are the good ones in your view?
  - raddan 2mo ago
    People’s tastes in beer and wine vary so dramatically (and there are so many styles) that it’s not clear to me that you can actually have a “best of.” The beer and wine I enjoy the best my wife finds completely unpalatable. For wine, I enjoy a dry Grenache; Mas de Gourgonnier is a good one. For beer, I enjoy a beer with body, a malty profile, and balanced hops; there are many mass-produced styles that fit this, but I like Sam Smith Organic Lager, and Leffe Brune. I’ve also brewed many beers myself and one of my all-time favorites was an all-grain formulation of a honey ale [1]. Such is the nature of beer and wine (and HN) that someone is going to “disagree” with me. That’s fine. Just go try some things and see what you like. [1] https://en.wikipedia.org/wiki/White_House_Honey_Ale https://en.wikipedia.org/wiki/White_House_Honey_Ale
    - duttish 2mo ago
      Sorry, I meant which awards not actual drinks :)

TMWNN 2mo ago

I read at night in bed. I love how, with ebooks (first Overdrive, now Libby), when I hear about an interesting book, I can within 30 seconds search for it across multiple libraries, borrow it, and send it to my Kindle.

DangitBobby 2mo ago

At least one of the links on the post is still to localhost:3000 instead of the vercel page. I don't want to make a substack account to tell the author so hopefully this information finds its way over there.

adamddev1 2mo ago

It's sad now to see a university library where students sit among endless shelves of amazing books, sitting on laptops, almost all of them with ChatGPT open. The library has become just a place to sit and open an LLM.

DubiousPusher 2mo ago
It's especially sad because on of the things those LLMs are best, almost purpose built for is to tell those students which books to go open, where to find nuggets people haven't bumped into for years, to make cross-connections that would take a PhD a decade to find.
- nephihaha 2mo ago
  This is a very good point. If it should be used as anything, it is as a pointer to other things.
Der_Einzige 2mo ago
The most sad part of all is to think how little of that knowledge is digitized and available to be referenced by ChatGPT. Digitizing old books is one of the highest things you can do as a human for “good for the world”.

Planktonne 2mo ago

I think my issue with this project--and so many other similar ones--is that the provenance of the code does undermine the intention. If a project purports to be about quality, then knowing that the creator abdicated some of the responsibility for creating the thing they ostensibly care about makes it harder to put faith in them as having high standards elsewhere. Perhaps I am just old-fashioned, and vibe-coding is something that can be done with full focus and care for high quality, but I remain unconvinced. This is a project that needs to be cared about sincerely to be trustable/meaningful/useful, and the approach taken casts doubt on that.

zahirbmirza 2mo ago

Fascinated that the author shares an experience of discovery in libraries. The internet, in its early days felt like that. Of course, no longer. Libraries are certainly declining in their traditional form. I find it odd that everyone has a digital resources in their pocket yet libraries are squeezing out physical books to make way for more and more computers. Try to find paper copy of the Sony founder's book.... that will be £50 on amazon. Prohibitive. No library within a 10 mile radius has a copy. I remember joyfully discovering Tony Royce's book in my library a few years ago. That enlightenment will never happen now. Primary knowledge is being lost, churned crude will forever lubricate the delusions of those who have no facility to collate the basis of our understanding.

sandspar 2mo ago
AIs are fantastic for discovery. Just today I asked Chatgpt to find blogs 1) about old school newspaper cartoons 2) that have been writing for more than 10 years and 3) are run by one or two passionate people rather than a team. Within seconds Chatgpt found 7 candidates. Within another handful of seconds it gave me RSS links to the ones I wanted. This is incredible!
- nephihaha 2mo ago
  Sometimes. I think this is partly because search engines are so poor nowadays. I have asked AI for certain websites that I know exist but it can't find them.
zem 2mo ago
> The internet, in its early days felt like that. Of course, no longer. back in the reasonably early days of the web (I would guess ~2000) I stumbled across a webpage that said "I used to collect random interesting snippets in a shoebox, here is the internet version of it". I was delighted enough that I actually wrote to the author to compliment them on helping keep the web interesting, and though I haven't thought about it in ages it clearly made enough of an impression on me that I remembered the author's name. and it's still up! https://www-users.york.ac.uk/~ss44/cyc/index.htm https://www-users.york.ac.uk/~ss44/cyc/index.htm

DubiousPusher 2mo ago

My bona-fides here amount to little more than being a big time reader of non-fiction. But IMO, there's a nice synthesis here. Rather than going long rounds of asking LLMs about a subject, I usually end up asking it for book recommendations. It's much better than a google search and you can push it into some deep corners if you go past the surface level recs. You can get quite specific. You can find texts you wouldn't discover unless you spent years studying the topic. Often these are completely approachable and give interesting perspectives they just get buried behind a wall of syllabi and listicles. This is how I ended up reading Thompson's 'The Making of the English Working Class' and Graves' 'Goodbye to all That' among others.

philips 2mo ago
I did this for a recent trip! I asked books related to or authored in the places I was visiting. Found some interesting titles.

jeffreyrogers 2mo ago

I would bet that how your brain stores information that you read from long-form text is very different from how it stores information you acquire from chatting with an LLM. When I read something challenging or new to me I spend a lot of time thinking about how what I'm reading matches my own experiences or knowledge. Although I'm a fairly fast reader, it often takes me a long time to get through difficult pages since I have to stop and think about what I'm reading. I seem to be doing a lot of integrating and reorganizing my thoughts. When interacting with LLMs it feels a lot more like I'm just receiving knowledge passively and I don't think it gets integrated as well. Not sure why this is and its somewhat counterintuitive since I don't think I'd have the same experience with a human tutor.

m463 2mo ago
I listen to audiobooks while I walk and hike, and sometimes when I recall some particular thing I've learned, I can also recall where I was hiking when I heard it. I think there's a lot to learn on how we really process information.
- tony_cannistra 2mo ago
  Same here. I always marvel at this. These recollections can even be years later, and the memory of the place in which I heard the remembered thing is vivid.
- annzabelle 2mo ago
  There's a podcast I listened to as it came out when I was a teenager. I'm now relistening to it over a decade later, and I distinctly remember walking home from the bus stop with it coming through wired earbuds from my ipod.
- ahartman00 2mo ago
  "when events are represented in memory, contextual information is stored along with memory targets; the context can therefore cue memories containing that contextual information" https://en.wikipedia.org/wiki/Context-dependent_memory https://en.wikipedia.org/wiki/Context-dependent_memory

EdwardAF-IT 2mo ago

[dead]

adamtaylor_13 2mo ago

On a related note, I was just noting to my co-founder, as we struggle to write good case studies for our website, that I find LLMs are astoundingly bad at writing good prose. We all know the "AI-tics" that give away a sloppily AI-written piece, but even if you steer them, they still struggle to write consistently high-quality prose. Somehow I feel that the work of a good copywriter has never been more noticeable.

mpalmer 2mo ago
I've not really enjoyed finding out lately just how few people seem to notice what ought to be unmissable.
- cyanydeez 2mo ago
  those people will also start adopting the AIsm and will become indistinguishable.
  - Mtinie 2mo ago
    AI adopted humanisms, we just weren’t used to seeing them at the same scale we do today. Diversity of writing styles was part of that, but I’d point to vernacular exposure as the larger component. We’re going to go through a period where we try to adapt to a form of “Universal English” for those of us who read primarily English writing. Other languages’ readers may be experiencing the same dissonance when they come across AI-generated prose in their native language (but I’ll let others validate /reject my hypothesis).
    - cwnyth 2mo ago
      This is right. AI is using human speech, but just doing so in a consistently peculiar way. The em-dash in particular is frustrating, because it's all over high-quality, pre-2022 academic work. But now instead of proper and erudite it's seen as AI-slop. Well, maybe if they didn't use it — all — time — ! It's even an auto-replacement in Word and can be (by one's choice) in LibreOffice, replacing three dashes (and two is replaced by an en-dash). But when I submit a novel with em-dashes, will sloppy agents and sloppy editors be able to tell that em-dash was deliberately put there by me?

paxys 2mo ago

Very neat site and write-up, but I found this part amusing: > There is really nothing “AI” about this aside from the tool that collected the data and coded it, and, crucially, semantic search […] So really, everything about it is AI. And that’s not a bad thing! It’s okay to simultaneously preach the superiority of award winning books over AI-generated garbage while also acknowledging the same AI as a valuable tool for other uses.

therealpygon 2mo ago
Key word being tool. People get lost in a fantasy that the AI was responsible for decisions and not the human who chose to use it as a replacement for effort rather than a tool to enhance effort, then apply their bias to all AI use.
calvinmorrison 2mo ago
so much of AI is like, to me, used as was to generate basically dynamic applicaions. if we ever got anywhere with semantics, objects, etc, pushed html further, maybe we'd naturally have web clients that could take objects, natively search, collate, display, put on a map, graph, etc... but AI is like a text only shortcut to the end result we want. I have a stupid AI app I use now for some bookkeeping which was previously, just an excel sheet. except it has a UI, it validates inputs, and does a bunch of other housekeeping stuff that makes it nice. Did the excel work? yes totally.

replatformradar 2mo ago

Is it even possible to find one these days? I would love to see some real honest amazon kindle stats on the amount of ebooks added over the last 3 years. Even a honest pie chart to show people its not worth it to get them to stop posting them.

wackget 2mo ago

> Decries AI for producing slop > Uses AI to produce website

frollogaston 2mo ago
The article isn't even about AI slop other than the title for some reason
- llm_nerd 2mo ago
  It's a clever title because it's FREE MONEY on HN. There is a small but rabid group that upvotes everything that denigrates AI, sight unseen.
- deleted 2mo ago
  [deleted]

OgsyedIE 2mo ago

Suggested additions: the Axiom Business Book Awards, Library Journal Best Books of the Year and Booklist Magazine's Editor's Choice Awards. There's also a few popular substacks that have big sideshows in regular book reviews, but I wouldn't even recommend my favourite for a public aggregator like this.

deleted 2mo ago
[deleted]
benbreen 2mo ago
Original author/creator of the site here - thank you! Will add the Axiom one. I am on the fence about whether/how to add "book of the year" type lists as they are somewhat distinct from book awards, but I do think that would probably be the next step to get more books in the corpus. And any other ideas that HN readers have for awards to add would be welcome. Currently it's probably too history-slanted since I'm a historian and knew those awards better.
- OgsyedIE 2mo ago
  It may be hard to parse the lists but if they're still available somewhere online under the current administration the past annual State Department, CIA bookshelf recommendation, Army Chief of Staff, Navy CNO and Marine Commandant reading lists may have some valuable additions. To my knowledge there aren't any analogues to them in the rest of the Anglosphere but some LLMs will probably point out ones I haven't thought of. I can point out that the UN agencies' respective reading lists for their staffs' professional development are 99% internal UN papers of very limited interest to external audiences but I haven't had a professional reason to look at any EU agencies to confirm or deny value. Additionally, the Financial Times has a second set of book awards along their main awards which you've not listed, the FT reader's best books list.

maxdo 2mo ago

I participate book club for several years with friends, we casually navigate certain topics, and i can tell you human slop in literature is a real thing. - almost every book try to stretch core idea into book size format - unique ideas are rare people attack them under different angels - a later phenomena : their believes almost predict entire book, outcomes etc, brainwash impact is real so not sure, how ai slop is better vs book slop, at least with ai you can distill the idea, with the book, you have to spend 10-40 hours to digest average, absolutely non fresh ideas, that author brought in just to sell that book, otherwise it would be magazine article worth.

tolugenius 2mo ago
In your experience, how do you determine book slop? For me I don't really stray from authors and topics I like, and non-fiction isn't really my go to outside of sci-fi, and mystery/law books. Like on the stretch point, I can tell that in other media but not so much books.
- jonfromsf 2mo ago
  Business books are 99% slop.
  - vkazanov 2mo ago
    Self improvement books! All of them! "Make good habits, not bad habits because this is what Lincoln did..."
Der_Einzige 2mo ago
This is sadly correct/true. This is one of the many reasons why I militantly reject carbon chauvinism. Most of the potshots people take against AI can be easily retorted with “have you seen how bad the average human is at X?”
paxys 2mo ago
They’ve used a pretty good filter for the human slop - prestigious awards.

diego_sandoval 2mo ago

The title is tautological. Quality is the opposite of slop, AI or not.

janalsncm 2mo ago
Not really. Quality nonfiction books are a subset of quality books.

titanomachy 2mo ago

Thank you for this. I sorted the "technology" and "science" sections and saw a few excellent books that I've read and a few that I would really like to read. This motivates me to start setting aside "reading time" again every day, since I've lost that habit. The recent "society & culture" books gave me some good book club ideas. Bug report: filtering by "award" appears to be broken for some awards. If I select Pulitzer or National Book Award, no books show up, but I can find books with these awards by browsing.

benbreen 2mo ago
Awesome, that's what I was going for. I originally made this for my own use and got kind of obsessed with building it out once I started finding unfamiliar books I enjoyed using the semantic search and category browsing. Glad to see it out in the world hopefully doing the same thing!
- titanomachy 2mo ago
  One thing that would make this information even more useful and searchable would be a JSON dump of your dataset. You could serve it statically on S3 or similar, to keep hosting costs to a minimum. Of course, having created a cool thing generates no obligation for further work on your part! I'm grateful that you did this work and made it available for free. Did you build your semantic search index based on the full text of the books, or just the reviews and descriptions?
  - benbreen 2mo ago
    Already tried to do this in a basic form. Advice from experts would be welcome as it's my first time trying to make data and an API available: https://book-prize-index.vercel.app/data https://book-prize-index.vercel.app/data Also I just had to upgrade my Vercel plan due to traffic, which is very welcome and appreciated, but if anyone wants to donate, that would be really helpful! There's a Stripe link at the site: https://book-prize-index.vercel.app https://book-prize-index.vercel.app

why_at 2mo ago

This is pretty nice, I might use this if I'm looking to learn more on a topic. I'm always cautious when reading nonfiction because it's hard for me to tell if the author knows what they're talking about when I'm not an expert myself. Using awards is a good metric. Maybe. I am curious what the "score" for each book means. Is it calculated by giving each award a certain weight and adding them up?

zem 2mo ago

possibly a bug - I searched for "writers like lewis thomas" and the top two results were by thomas himself.

olooney 2mo ago
On a similar note: on Steam, the original Dark Souls game[1] is listed with the "souls-like" tag. That's not wrong, I guess, but just kind of a tautology. Grice's maxims[2] generally have such tautologies omitted in ordinary conversation as non-informative. [1]: https://store.steampowered.com/app/570940/DARK_SOULS_REMASTERED/ https://store.steampowered.com/app/570940/DARK_SOULS_REMASTE... [2]: https://en.wikipedia.org/wiki/Cooperative_principle https://en.wikipedia.org/wiki/Cooperative_principle
- vkazanov 2mo ago
  For me it is surprisngly natural to include the original thing into "a set of thing-likes". Why does it feel wrong (or meaningless) to you?
- inigyou 2mo ago
  I really thought ChatGPT would be able to tell me what this is called but it can't. I suggest "circular eponym". It's like calling a CRT monitor a really big flatscreen.
  - zem 2mo ago
    it's more like you would call a xerox-brand photocopier a "xerox machine" and mean that it was a photocopier rather than that it was made by xerox.
yellow_lead 2mo ago
For embeddings based search I think this is unavoidable

anvuong 2mo ago

Is it only me or the titles of top books from History, Nature, and Politics sound very click-baiting? I'm really taken off by these titles.

DonHopkins 2mo ago
Hey buddy, are you feeling all right?

janalsncm 2mo ago

Worth stating that this is one of the success stories from AI. Someone who has domain expertise outside of programming is able to create a really useful piece of software because the barrier to entry has been significantly lowered. Really nice.

deleted 2mo ago
[deleted]

762236 2mo ago

I've gotten AI to write things that made me teary eyed --- it manipulated me

inigyou 2mo ago
I made a sandwich yesterday and cried for an hour because it reminded me of the way my mom used to make school lunches. No AI necessary.

mrkpdl 2mo ago

This is wonderful. 30 seconds into trying the site and I’ve already found a couple of interesting books to read. The search I typed is ‘Stuff being made, how it was made, how history shaped its form’.

narrator 2mo ago

The antithesis of AI slop is when you've created a unique idea from first principles and AI keeps misunderstanding your idea and spell checking it back to something that's widely popular such that you have to write a whole extra alignment subsystem to check that it hasn't ignored half of what you've wrote in your first principles document and just thought you were talking about something distantly related in its training set. To work on something truly original in non-fiction in the future against AI pushing everything to the mean of its training set and fighting against that when doing research, as opposed to generating a whole book of slop from familiar ideas, is going to be quite the unrecognized effort of writing those books.

hahahaa 2mo ago

Literally reading the books like an LLM would be trained on them!

jocelyner 2mo ago

[flagged]

Balgair 2mo ago

Awesome stuff! Now, can this interface with Libby? The worldcat interface is, well, confusing. I can't figure out if it knows libraries near me exist.

fuddle 2mo ago

Nice site, but most of the top biographies are US figures: Edgar Hoover, Frederick Douglass, Lyndon Johnson etc. I'd like to see more non-US biographies.

Nevermark 2mo ago

I don't think we would have AI slop, if they were just trained on high quality media. Less relatable AI, maybe, but not sloppy AI. AI trained on other AI (i.e. distilling), can be better than the AI they were trained on, likely because of this reason. Each iteration can raise the bar. As apposed to the deep spiral of generating shiny attention crystals for humans, and then training on that.

deleted 2mo ago
[deleted]
inigyou 2mo ago
The slop isn't a property of the LLM but the workflow it's utilized in. Just now I asked ChatGPT about the linguistic effect where a CRT could be described as a giant flatscreen, a landline could be described as a wired cellphone, or the original Rogue could be described as a roguelike. Unfortunately it doesn't know what that's called either, but that is a non-slop use of an LLM, like a semantic search engine. On the other hand if you print LLM output into a book it's almost certainly slop no matter how the model was trained.
dncornholio 2mo ago
Training AI on AI will most likely increase slop. LLM's add randomness in their outputs, books usually don't.
- Nevermark 2mo ago
  That isn't correct. Obviously, you can handicap any model with an indiscriminate dataset. Are you training on slop? Why? More data is not automatically better data. Just three (of many) advantages of quality data selection: (1) It gives a clearer signal. (2) It, ironically, reduces the complexity of what needs to be learned, since slop adds its own complexity. (3) It reduces dataset size which means more learning per watt (and per just about any other cost). The advantages compound. Humans are no different. Children surprise with their ability to absorb sophisticated relationships and skills, while people who had bad examples struggle to recover. 99% of human effective intelligence is higher quality cultural knowledge learned as children. None of us had to spend centuries deciding whether zero, negative numbers, the square root of -1 are numbers. Slop + quality is not quality.

whattheheckheck 2mo ago

And what pray tell is everyone doing with their privileged leisure time? You have access to every single wicked problem in the world and you choose to do nothing. Shame

nttylock 2mo ago

[flagged]

seizethecheese 2mo ago

The article resonates, there is great serendipity in somewhat random exploration of books. But then the web app seems to be the antithesis of this? I clicked a few categories and the top ranked books all had the “popularity contest” smell, sort of the antithesis of random exploration.

abc42 2mo ago

I like very very much that there's no user voting system here. I just recently read a really mediocre book with a 4.7/5 rating on Goodreads. Shows being sloppy is not exclusively a privilege for neither the AIs or producers.

rincewind 2mo ago

I'm not sure how I feel about "Entangled Life" on the first page of books in "science". Isn't that slop? I mean, it's pop-science bullshit, and it's based on feelgood extrapolation from a thin evidence base. Maybe that doesn't quite fit the definition of slop, but it's not a signal of quality for the rest of the list.

insane_dreamer 2mo ago

Thank you! Wonderful idea, and very well executed. Useful discovery tool. I understand your passion is non-fiction, but would be wonderful to have the equivalent for fiction, of which there is a great deal of excellent literature. Perhaps the same site could incorporate both fiction and non-fiction, with a filter?

gabriben 2mo ago

Thanks for this! I've been looking for a good book on the Spanish Civil War, in either Spanish or English. The one American book I was recommended by someone, I didn't like. I'd like to keep searching regardless of the book's language or what country it has won prizes in. Could you also include book awards from other non-english speaking countries?

nsmog767 2mo ago
+1 on this!
gorkaerana 2mo ago
Books on the Spanish Civil War I have read and can wholeheartedly recommend: - "The Spanish Civil War" by Paul Preston - "Homage to Catalonia" by George Orwell Books on the same topic on my reading list (and quite frankly can't wait to get hold of them): - "The Spanish Civil War" by Hugh Thomas - "The Fight for Spain" by Antony Beevor
- embedding-shape 2mo ago
  I've only read Homage to Catalonia out of those, which is a personal retelling from Orwell, are the other books in the same style? What's the difference you'd say between the ones you list?
  - gorkaerana 2mo ago
    Paul Preston's "The Spanish Civil War" would be the poster child of the "quality non-fiction" category specified in this post's title. He is, in my opinion, the leading Hispanist historian [1], and I hold his work in the highest esteem. I can also very much recommend his book "A People Betrayed". Regarding the other two authors in my previous comment: Hugh Thomas was also a leading Hispanist historian [2], and Antony Beevor is a prominent military historian [3]. I expect both books to be in the same vein as Preston's. (Quick aside for the astute reader that noticed all of the recommended authors being non-Spanish: I find Spanish politics extremely polarized; the Spanish Civil War being perhaps the most polarizing topic. As such, I find outsider's perspectives to be more balanced and objective.) References: 1. https://en.wikipedia.org/wiki/Paul_Preston https://en.wikipedia.org/wiki/Paul_Preston 2. https://en.wikipedia.org/wiki/Hugh_Thomas,_Baron_Thomas_of_Swynnerton https://en.wikipedia.org/wiki/Hugh_Thomas,_Baron_Thomas_of_S... 3. https://en.wikipedia.org/wiki/Antony_Beevor https://en.wikipedia.org/wiki/Antony_Beevor

codedump 2mo ago

[flagged]

theoneone 2mo ago

real nice work!

sawrtagy 2mo ago

[dead]

rimworld 2mo ago

why read when you can flick on the tube?

dash2 2mo ago

I would really like this, but using academic reading lists instead of prizes. What are the important historians of (e.g.) medieval Italy?

Elkanumanos 2mo ago

Thank you for sharing. This will come in handy for my research!

alansaber 2mo ago

Fiction genre catching strays as usual

claudiacsf 2mo ago

But I get triggered by the em dashes. :(

mrich 2mo ago

Could you please add a link to the Goodreads page of each book?

keiferski 2mo ago

Not to be intentionally contrarian, but I have thought a lot about this: I think good fiction is actually the antithesis of AI, fundamentally. LLMs are really only capable of combining different things into output. This can give you some creative results, but it is ultimately kind of limited. I cannot imagine that prompting an AI a thousand times would give you something truly original in the way many high-quality fictional stories are. The magic and absurdity of life is a necessary thing to create such works.

newguy1000 2mo ago
you have a bad imagination then
brrrrrm 2mo ago
Models are increasingly showing their ability to extrapolate into the human unexplored (math proofs being the most apparent). What gives you confidence the absurdity of life is uniquely difficult for models to source?
- eddythompson80 2mo ago
  Mathematical proofs are hard to come to, but easy (or “cheap”) to verify. Also having a math proof processed through a transformer to derive an output that matches existing proof patterns kid of makes sense. The former allows you to generate hundreds or thousands of iteration and cheaply verifying its correctness until you hit the right randomness level. Generating thousands of fiction books then verify them doesn’t really work. Most good fiction authors are trying to communicate a story that’s forming in their minds. Even when they get stuck, they are usually not stuck because they just don’t know anyone to ask. It would’t be their voice anymore. Some do seek input and those might experiment with llms for ideas. Using llms for research, technical questions, grammar, synonym sentences, word selection, etc makes sense too. I’d agree with OP for people generating full stories through llms. While I say that, I know a very young person who told a while back (like early ChatGPT 4 time) they use it to write fan fiction for books or characters they like.
- keiferski 2mo ago

glasffordd 2mo ago

[dead]

deton3991 2mo ago

This hits the nail on the head regarding the real utility of AI right now. The magic isn’t in using LLMs to generate endless reams of synthetic text; it’s in using embeddings and semantic search to navigate high-signal, human-curated data.

deleted 2mo ago
[deleted]

haritha-j 2mo ago

I think your amazon links are broken, the ones i tried all 404.

dustypotato 2mo ago

Tangentially related, but what a snappy webpage

abrezas 2mo ago

The history section of the book price index is incredibly america-biased. History of america, american natives, japan after ww2 and vietnam in the last century.

bluGill 2mo ago
That is what Sells. The USA has a lot of people so selling to us is going to make money even in a niche subject like history. Historians who are not trying to sell mass market books have a much broader set of books. You won't see them in a normal bookstore.
diego_moita 2mo ago
And the irony of it is that the U.S. is actually quite history-poor, when compared to Europe, North Africa, Middle-East, Persia, China, Central Asia and India.

rntksi 2mo ago

Thanks for making this, I searched up economics and found at least 3 titles that , when reading through first pages, are really really interesting.

chegra 2mo ago

If you are interested in a non-fiction book, check out Veritas: Truth Across Cultures[1]. It's a collection of quotes/saying/proverbs that are similar between cultures, which I call truths. I am in interested in long term thinking, things that stood the test of time. This was primarily put together for myself as a lens to view the world. [1] https://www.chestergrant.com/veritas https://www.chestergrant.com/veritas

gavin_gee 2mo ago

the cope from authors.

zkiihne 2mo ago

Not to be that guy but your site is terrible. It does not do the most important function well, giving me a list of award winning books that is easy to sort through. It is not clear what is a link and what is not. It looks fine though somewhat generic. But it is not actually functional in a real sense.

diego_moita 2mo ago

When I see lists like this I am surprised that they always live out school textbooks. School textbooks are the kind of books that have the very precise purpose of giving you a foundation and roadmap to explore and understand a whole field of knowledge. And they're an industry that has been improving them on successive iterations to produce highly organized, comprehensive and accessible reading material through years.

gwbas1c 2mo ago
School textbooks also have a lot of bias, and may oversimplify issues in ways that are necessary with children, but lead to longing misconceptions. IE: Was the US Civil War fought over slavery or states' rights? Someone's answer may come from the biases of the textbooks they grew up with.
- diego_moita 2mo ago
  > School textbooks also have For me the keyword in your comment is "also". What this word means to me is that, whatever faults textbooks might have, there is no alternative solution. They replicate a problem that all books have. However, within the domain of textbooks, you still might find good solutions for these bias. These bias are mostly within history and social sciences and good and reputable Universities are renowned for publishing books in these domains that are quite balanced, comprehensive, deep and rigorous. Oxford and Cambridge are the 2 best examples that come to my mind. The U.S. has good Universities too, but after their government declared a war on science and knowledge I'd be carefull before trusting these universities.

throwaway27448 2mo ago

I think the antithesis of ai slop is meditation on a nice beach. Or eating like a really good meal. Or holding your child.

weird-eye-issue 2mo ago

Sweet, I think I found a few to read > And before you wonder, yes this is actually free. I am paying for the hosting and the API costs entirely because I just want people to find and read more good non-fiction books. Just to note, you do have affiliate links so it isn't 100% out of the goodness of your heart ;) Not blaming you or anything, just saying...

JKCalhoun 2mo ago

"To avoid the tedium, I decided that I would also flip to a random page of every book I shelved and read a random sentence from it." Also, this is about the lost joy of "browsing". I wonder how much serendipity played when wandering through a library (or other things in life—even window shopping). But now, everything on the internet, you more or less have to know what it is you're looking for—are rarely "surprised or delighted".

tombert 2mo ago
I’m not sure I agree with that. I’ve found plenty of interesting things by randomly wandering around on Gutenberg.org, completely undirected.
- JKCalhoun 2mo ago
  If they have a sort of "wander UI", that's cool. I'm not sure what that looks like though.

sparklingmango 2mo ago

Can I make a feature request? I would love to be able to search by author. I recently started Caro's LBJ series (it's amazing so far) and I'm curious how many awards he's won. Anywho, I love non-fiction so I plan to visit this website often. Thank you and great work!

FabHK 2mo ago
More generally, just sort by any of the column headers (year, title, publisher, author, score, ...) by clicking on it. In some views it works, in some it doesn't. A bit unintuitive. The toggle between "traditional" text and "semantic" search is cool and new, but maybe the toggle could stay visible (it's there on the home tab, but then not anymore on the books tab).

VoodooJuJu 2mo ago

[dead]

32-ai-toasters 2mo ago

maybe I am being too small brained black/white about this, but something feels amiss about the idea of vibecoding a solution to an AI generated problem. How can we get better, doing the same thing that made us sick?

keithbucklen80 2mo ago

I love the "books for dads who like Pavement" prompt! It would be great if you could incorporate the same experience of "browsing" random page of some of the results that are returned

MillironX 2mo ago

I very much have had the same experience as the author with research libraries. I have learned so many life-changing techniques and derivations from books that were two or three down from the book that I was looking for. When I became a research associate, I would often have my librarians ship the books to the branch closest to my office, and I stopped browsing the shelves myself. It wasn't until I went to the main library that I checked out a book several over from the one I had found in the catalog that I had a major breakthrough. Now that same library has collapsible shelving that makes browsing feel like a chore, and it makes me think we lost something valuable. That said, I had high hopes for the catalog that the author had put together, but for my technical fields, I found it lacking. Trying to find books that would be close to van Soest's Nutritional Ecology of the Ruminant or van der Werf's Animal Breeding: Use of New Technologies, yields a whole bunch of pop-sci titles. The joy of finding these titles at the library in the past was how remarkably understandable and engaging they were while remaining scientifically rigorous. I immediately feel that the pop-sci titles that I was recommended are going to try to sell me something or some idea, rather than letting me explore ideas myself. I guess that means I need to go back to browsing library shelves myself.

subset 2mo ago

As it happens, I had the same job as the author at uni. I shelved books at the biomedical and sciences library. As a life-long book lover, it was a dream uni job being surrounded by all sorts of historical treatises on medical arcana. I've been really exercising my non-fiction reading muscles of late. I'm trying weaponise it as an antidote to AI slop and doom-scrolling. My last read was Designing Data-Intensive Applications by Martin Klepmann (the newly released second edition), it was a wonderful read!

jschveibinz 2mo ago

Serious question: can we define the top 5 parameters that distinguish human-generated non-fiction writing from AI "slop" writing (I don't like the term "slop" because it carries a built-in bias)? Since AI output is based on the most likely next word/token given the context, let's consider how humans do this, i.e. choose the next word based on context. Is one of the parameters the degree of "randomization"? Are these other parameters? - Specific experience? - Recent exposure to relevant materials, i.e. paraphrasing? - Borrowed knowledge from a specific resource? - Writing style, i.e. choice of words or sentence structure? I'd like to hear what others think.

naiveter 2mo ago

Penguin Random House seems to be dominating the leaderboard. What's the story here? Are there only very few non-fiction publishers capable of publishing high-quality work, or is Penguin Random House exceptionally good at it?