8 ms·

Show HN: MCP server for searching and downloading documents from Anna's Archive

256 points by iosifache 1y ago

I was looking around for an MCP server that could connect Anna's Archive to Claude Desktop, as I wanted to be able to search and download books directly through the interface. I couldn't find any public implementations, so ended up building one myself. What it does? - It searches Anna's Archive by keywords. - It downloads books from search results. - It works directly in Claude Desktop through MCP. Check out the repository's README for detailed installation and configuration instructions. The code is fully open source and builds run on GitHub Actions for transparency. I figured I'd share, since I couldn't be the only one wanting this functionality!

neilv 1y ago
> This software does not endorse unauthorized acquisition of copyrighted content and should be regarded solely as a utility. Users are urged to respect the intellectual property rights of authors and acknowledge the considerable effort invested in document creation. How sincere is that statement?
- bigyabai 1y ago
  As sincere as the user takes it.
- iosifache 1y ago
  I just provide a hammer. Users decide whether they're hitting their own nail or the metal one. The comparison might be loose, but the problem is similar to releasing a browser. Do you prevent users from accessing websites you think are malicious or illegal? Or do you delegate that responsibility? I was hesitant about releasing the MCP server as open source software, but I hope (1) it proves useful for others and (2) people understand that the authors of the books they're reading need money to eat, live, and support their families.
  - TylerE 1y ago
    > The comparison might be loose, but the problem is similar to releasing a browser. Do you prevent users from accessing websites you think are malicious or illegal? Or do you delegate that responsibility? I might liken the situation more to releasing a browser and setting thepiratebay as the homepage.
    - iosifache 1y ago
      That would imply constantly reminding users of an available action, which isn't the case since the MCP server is just a dormant capability that needs to be triggered.
  - thedevilslawyer 1y ago
    IMO, you're needlessly taking a defensive stand. It's ok to take a forward looking stand on how access to knowledge should be.
    - 9dev 1y ago
      Oh come on. We all know there’s pretty much every novel you’ll find at Barnes&Noble on Anna's Archive as a pirated copy, not just scientific papers. At least be honest; it’s as much a mundane piracy tool as it is a knowledge repository.
      - thedevilslawyer 1y ago
        I was absolutely saying that. Novels are part of the knowledge too - a scientific paper and a novel have equal weight
        9dev 1y ago
        Phrasing "here's a way to pirate novels using an LLM" as "I'm on a mission to grant everyone access to the wealth of humanity's knowledge" is just disingenuous. Sure do novels count as knowledge, but there's a moral difference between making scientific content available to researchers for use in research versus saving money by pirating books.
        thedevilslawyer 1y ago
        Why is scientific content not saving money, and why can't novels be useful?
  - recitedropper 1y ago
    Hilariously disingenuous.
- ASalazarMX 1y ago
  I'd bet you won't find a single string containing "acquire copyrighted content arrr!", so pretty sincere. The software doesn't endorse it.
- deleted 1y ago
  [deleted]
- hereme888 1y ago
  As sincere as LLM providers not wanting to get sued for the copyrighted content they used.
thorum 1y ago
Interesting project! I’m a little surprised that Claude is willing to call these functions. The demo screenshot is downloading a public domain work, I wonder if it would also happily go along with requests for Harry Potter or other copyrighted material?
- throwaway314155 1y ago
  Last I checked downloading isn't the issue. It's distributing. Not an expert though.
  - iosifache 1y ago
    > I wonder if it would also happily go along with requests for Harry Potter or other copyrighted material? There's no way to protect against this. Anna's Archive doesn't include licence information in their data fields. It would be helpful to integrate with another data source that could warn MCP server users when they're attempting potentially risky actions. Please let me know if you have ideas on how to achieve this. On a related note, please see this reply: https://news.ycombinator.com/reply?id=44515205 https://news.ycombinator.com/reply?id=44515205
    - throwaway314155 1y ago
      Thanks, good context.
    - kevindamm 1y ago
      It's worse than you'd expect, there's a sizable subset of works with unknown or uncertain license status. https://wikipedia.org/wiki/Orphan_works_in_the_United_States https://wikipedia.org/wiki/Orphan_works_in_the_United_States https://martincopenhaver.github.io/files/orphanworks.pdf https://martincopenhaver.github.io/files/orphanworks.pdf [PDF] Estimated 25-50 million books are orphaned works, and their copyright holder may step forward at any time after you've treated it as unlicensed, it's perpetually uncertain (but showing due diligence in finding and contacting the rights holder is considered adequate). For US works published after 1977 and most works between 1898-1945 the US copyright office has a database: https://publicrecords.copyright.gov/ https://publicrecords.copyright.gov/ but I don't know a good catalog for non-US.
puppycodes 1y ago
love this. god bless anna's archive
- iosifache 1y ago
  Cheers!
toomuchtodo 1y ago
Edit: Would you accept a PR to override the search and download endpoint hostnames with env vars? For someone who has their own copy and ES index, it might be helpful to support overriding the public endpoint hostnames (/internal/anna/anna.go#L22-L23).
- iosifache 1y ago
  > local copy of this corpus Are you referring to the JSON index (https://annas-archive.org/faq#api https://annas-archive.org/faq#api)?
  - deleted 1y ago
    [deleted]
xmonkee 1y ago
I'm an LLM noob, but how feasible it is to make a research agent that can not only download articles, but read and reference them in it's process?
- iosifache 1y ago
  It's doable, as you'll also find MCP servers for reading files [1]. Claude Desktop also has a built-in file reader [2], so you can ask it to read the file and process the content (e.g., generate a summary or even a meta-summary [3]). [1] https://github.com/sylphxltd/pdf-reader-mcp https://github.com/sylphxltd/pdf-reader-mcp [2] https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem https://github.com/modelcontextprotocol/servers/tree/main/sr... [3] https://x.com/iosifache/status/1942247320302547175 https://x.com/iosifache/status/1942247320302547175
- conception 1y ago
  Firecrawl -> Rag -> mcp is the general path
irskep 1y ago
What advantage do you get from this being an MCP server rather than simply a command line tool? Genuinely curious, as I'm trying to develop my mental model of when to use one or the other. Lovely project!
- iosifache 1y ago
  Cheers, glad you like it! I justified the hours I invested by thinking I could search, download, and explore books directly from Claude Desktop. While the initial steps are achievable with a CLI tool, the integration opens up new possibilities. Some general thoughts: - You’ll find the MCP mental model similar to the API one. - MCP integrations make it easier for non-technical users to access tools that were previously too technical. - An MCP integration implicitly respects a contract, unlike CLIs and GUIs which involve human aspects (aesthetics, information organisation, etc.). - MCP is an excuse for people to democratize data access. I wrote about this aspect here: https://x.com/iosifache/status/1941049600162574676?s=46 https://x.com/iosifache/status/1941049600162574676?s=46 And BTW, that’s a good idea! The functionality should probably also be exposed via CLI.
  - btown 1y ago
    https://neon.com/blog/building-a-cli-client-for-model-context-protocol-servers https://neon.com/blog/building-a-cli-client-for-model-contex... might be of interest. An MCP server provides enough metadata and self-documentation that it's quite straightforward to make a MCP-agnostic CLI client that adapts an arbitrary MCP server into a set of flags that allow you to call its explicit tools with explicit arguments - without ever needing to involve an LLM in the mix! You could even have that CLI tool launch the MCP server as a local subprocess, if you wanted - again, all deterministically. And if you want to have an SDK in any language under the sun, once you have an MCP outputting reasonable tool descriptions, any LLM could make a best-in-class SDK for you in a heartbeat following that language's best practices. So it's not unreasonable for someone working on a greenfield project to make an MCP server first nowadays!
    - iosifache 1y ago
profsummergig 1y ago
My understanding of Anna's Archive is that one has to download large zip files (>10 Gb) containing thousands of books even if one wants only a single book. Am I correct here? Does this MCP server allow one to download just a single book? I remember once using an IPFS based tool to download a single 200-year-old, out-of-copyright copy of "Last of the Mohicans" from Anna's Archive. It worked, but was very very complicated to figure out how to make it work.
- Qazar 1y ago
  [dead]
- LeratoAustini 1y ago
  I've downloaded single books several times recently (annas-archive.org in the browser): - search for book - tap a result - see a list of links to download mirrors (under 'slow downloads'), tap a link - get a countdown timer - timer expires, download links appear - click a link, book downloads just like any other download
  - iosifache 1y ago
    The waiting part is nonexistent if you have an active donation (which is also required by this MCP server for API access). The fast downloads mean you request a book and start downloading it immediately.
    - IshKebab 1y ago
      "Donation". Let's call it what it is - a subscription.
      - iosifache 1y ago
        [flagged]
  - 0points 1y ago
    So just 8 or 9 steps to grab a e-book, instead of just letting the LLM recall it from training data... /s
iosifache 1y ago
UPDATE: Following a suggestion from @irskep, I've added CLI command support for the search and download features. This raised a valid concern - while we're focused on building MCP servers, we shouldn't overlook whether users already have preferred (T/G)UIs available. When they don't exist, we should consider user experience and make our functionality accessible through multiple interfaces beyond just MCP. https://github.com/iosifache/annas-mcp/releases/tag/v0.0.2 https://github.com/iosifache/annas-mcp/releases/tag/v0.0.2
johnjames87 1y ago
[flagged]
- iosifache 1y ago
  https://hn.algolia.com/?q=anna%27s+archive https://hn.algolia.com/?q=anna%27s+archive
- eMPee584 1y ago
  The real crime is an economic system that limits the spread of knowledge and access to other "human rights" by requiring everyone to hustle to survive (and, if possible, increase capital gains for the financial overlords) when we would already be technologically equipped to feed and house well all of mankind, instead of letting thousands of children starve to death each day and restricting access to education so that billions miss out on their intellectual development - a void easily filled with addictive media full of rage and distraction. Pirating books is just a symptom of this wretched system. And it is not enough - RISE, HN! .. towards RBE & beyond..
- mandw 1y ago
  I recently downloaded a number of magazines froma now defunct publication from late eighties. No onesells them and annas archive was the only place i found them. its not exclusivly pirating, its a source for a lot of out of print.
- mrweasel 1y ago
  It's not piracy if you download the books to train your LLM.... Apparently.
- pjc50 1y ago
  Ai is dependent on piracy .. and apparently that's fine now.
ospider 1y ago
Why do people keep building servers for such a silly protocol?
- zipping1549 1y ago
  It's easy and has the word AI in it.
  - chisleu 1y ago
    It's infinitely useful for people who's workflows involve LLM agents.
- UncleEntity 1y ago
  People may do things you have no need for or don't agree with. Welcome to Planet Earth.
- aussieguy1234 1y ago
  Let's say you want to use the tool `ssh` and let's imagine ssh has just been released and it's not in the AI agents training set yet. It won't know in this case how to use the ssh command line tool. MCP provides a standard way to tell an AI agent how and when to use tools. So if you had an SSH MCP server, you'd simply plug that in, now your AI agent automagically has SSH capabilities.
snickerdoodle12 1y ago
Insane how HN cheers on piracy nowadays, all because it helps them train their LLMs
- artninja1988 1y ago
  I only see comments complaining about imaginary property "violations" on ai threads funnily enough.
- hombre_fatal 1y ago
  This isn’t about training LLMs at all. Also, HN like the rest of the world was always pro-piracy and getting the fruits of your labor without paying for it. The only time I’ve seen anti-piracy comments has been been wrt LLM training. Suddenly people pretend to care but it feels performative.
  - GuinansEyebrows 1y ago
    i wonder if you're considering the huge difference between: - individuals pirating copies of things for sole consumption or relatively-miniscule distribution; and - large highly-funded institutions that pirate content for the sole purpose of generating revenue from it ...and why that might lead people to feel differently about one and the other (not to mention the outsized punitive response to the former compared to the latter).
- xboxnolifes 1y ago
  I don't think HN was ever much against piracy. At least not personal use piracy.
- MisterTea 1y ago
  Information wants to be free.
- CamperBob2 1y ago
  The evolution of intelligence and its intersection with universal access to knowledge is more important than copyright. To the extent copyright interests want to pick a fight with AI, they must lose, and decisively so.

256 points by iosifache 1y ago

neilv 1y ago

> This software does not endorse unauthorized acquisition of copyrighted content and should be regarded solely as a utility. Users are urged to respect the intellectual property rights of authors and acknowledge the considerable effort invested in document creation. How sincere is that statement?

bigyabai 1y ago
As sincere as the user takes it.
iosifache 1y ago
I just provide a hammer. Users decide whether they're hitting their own nail or the metal one. The comparison might be loose, but the problem is similar to releasing a browser. Do you prevent users from accessing websites you think are malicious or illegal? Or do you delegate that responsibility? I was hesitant about releasing the MCP server as open source software, but I hope (1) it proves useful for others and (2) people understand that the authors of the books they're reading need money to eat, live, and support their families.
- TylerE 1y ago
  > The comparison might be loose, but the problem is similar to releasing a browser. Do you prevent users from accessing websites you think are malicious or illegal? Or do you delegate that responsibility? I might liken the situation more to releasing a browser and setting thepiratebay as the homepage.
  - iosifache 1y ago
    That would imply constantly reminding users of an available action, which isn't the case since the MCP server is just a dormant capability that needs to be triggered.
- thedevilslawyer 1y ago
  IMO, you're needlessly taking a defensive stand. It's ok to take a forward looking stand on how access to knowledge should be.
  - 9dev 1y ago
    Oh come on. We all know there’s pretty much every novel you’ll find at Barnes&Noble on Anna's Archive as a pirated copy, not just scientific papers. At least be honest; it’s as much a mundane piracy tool as it is a knowledge repository.
    - thedevilslawyer 1y ago
      I was absolutely saying that. Novels are part of the knowledge too - a scientific paper and a novel have equal weight
      - 9dev 1y ago
        Phrasing "here's a way to pirate novels using an LLM" as "I'm on a mission to grant everyone access to the wealth of humanity's knowledge" is just disingenuous. Sure do novels count as knowledge, but there's a moral difference between making scientific content available to researchers for use in research versus saving money by pirating books.
        thedevilslawyer 1y ago
        Why is scientific content not saving money, and why can't novels be useful?
- recitedropper 1y ago
  Hilariously disingenuous.
ASalazarMX 1y ago
I'd bet you won't find a single string containing "acquire copyrighted content arrr!", so pretty sincere. The software doesn't endorse it.
deleted 1y ago
[deleted]
hereme888 1y ago
As sincere as LLM providers not wanting to get sued for the copyrighted content they used.

thorum 1y ago

Interesting project! I’m a little surprised that Claude is willing to call these functions. The demo screenshot is downloading a public domain work, I wonder if it would also happily go along with requests for Harry Potter or other copyrighted material?

throwaway314155 1y ago
Last I checked downloading isn't the issue. It's distributing. Not an expert though.
- iosifache 1y ago
  > I wonder if it would also happily go along with requests for Harry Potter or other copyrighted material? There's no way to protect against this. Anna's Archive doesn't include licence information in their data fields. It would be helpful to integrate with another data source that could warn MCP server users when they're attempting potentially risky actions. Please let me know if you have ideas on how to achieve this. On a related note, please see this reply: https://news.ycombinator.com/reply?id=44515205 https://news.ycombinator.com/reply?id=44515205
  - throwaway314155 1y ago
    Thanks, good context.
  - kevindamm 1y ago
    It's worse than you'd expect, there's a sizable subset of works with unknown or uncertain license status. https://wikipedia.org/wiki/Orphan_works_in_the_United_States https://wikipedia.org/wiki/Orphan_works_in_the_United_States https://martincopenhaver.github.io/files/orphanworks.pdf https://martincopenhaver.github.io/files/orphanworks.pdf [PDF] Estimated 25-50 million books are orphaned works, and their copyright holder may step forward at any time after you've treated it as unlicensed, it's perpetually uncertain (but showing due diligence in finding and contacting the rights holder is considered adequate). For US works published after 1977 and most works between 1898-1945 the US copyright office has a database: https://publicrecords.copyright.gov/ https://publicrecords.copyright.gov/ but I don't know a good catalog for non-US.

puppycodes 1y ago

love this. god bless anna's archive

iosifache 1y ago
Cheers!

toomuchtodo 1y ago

Edit: Would you accept a PR to override the search and download endpoint hostnames with env vars? For someone who has their own copy and ES index, it might be helpful to support overriding the public endpoint hostnames (/internal/anna/anna.go#L22-L23).

iosifache 1y ago
> local copy of this corpus Are you referring to the JSON index (https://annas-archive.org/faq#api https://annas-archive.org/faq#api)?
- deleted 1y ago
  [deleted]

xmonkee 1y ago

I'm an LLM noob, but how feasible it is to make a research agent that can not only download articles, but read and reference them in it's process?

iosifache 1y ago
It's doable, as you'll also find MCP servers for reading files [1]. Claude Desktop also has a built-in file reader [2], so you can ask it to read the file and process the content (e.g., generate a summary or even a meta-summary [3]). [1] https://github.com/sylphxltd/pdf-reader-mcp https://github.com/sylphxltd/pdf-reader-mcp [2] https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem https://github.com/modelcontextprotocol/servers/tree/main/sr... [3] https://x.com/iosifache/status/1942247320302547175 https://x.com/iosifache/status/1942247320302547175
conception 1y ago
Firecrawl -> Rag -> mcp is the general path

irskep 1y ago

What advantage do you get from this being an MCP server rather than simply a command line tool? Genuinely curious, as I'm trying to develop my mental model of when to use one or the other. Lovely project!

iosifache 1y ago
Cheers, glad you like it! I justified the hours I invested by thinking I could search, download, and explore books directly from Claude Desktop. While the initial steps are achievable with a CLI tool, the integration opens up new possibilities. Some general thoughts: - You’ll find the MCP mental model similar to the API one. - MCP integrations make it easier for non-technical users to access tools that were previously too technical. - An MCP integration implicitly respects a contract, unlike CLIs and GUIs which involve human aspects (aesthetics, information organisation, etc.). - MCP is an excuse for people to democratize data access. I wrote about this aspect here: https://x.com/iosifache/status/1941049600162574676?s=46 https://x.com/iosifache/status/1941049600162574676?s=46 And BTW, that’s a good idea! The functionality should probably also be exposed via CLI.
- btown 1y ago
  https://neon.com/blog/building-a-cli-client-for-model-context-protocol-servers https://neon.com/blog/building-a-cli-client-for-model-contex... might be of interest. An MCP server provides enough metadata and self-documentation that it's quite straightforward to make a MCP-agnostic CLI client that adapts an arbitrary MCP server into a set of flags that allow you to call its explicit tools with explicit arguments - without ever needing to involve an LLM in the mix! You could even have that CLI tool launch the MCP server as a local subprocess, if you wanted - again, all deterministically. And if you want to have an SDK in any language under the sun, once you have an MCP outputting reasonable tool descriptions, any LLM could make a best-in-class SDK for you in a heartbeat following that language's best practices. So it's not unreasonable for someone working on a greenfield project to make an MCP server first nowadays!
  - iosifache 1y ago

profsummergig 1y ago

My understanding of Anna's Archive is that one has to download large zip files (>10 Gb) containing thousands of books even if one wants only a single book. Am I correct here? Does this MCP server allow one to download just a single book? I remember once using an IPFS based tool to download a single 200-year-old, out-of-copyright copy of "Last of the Mohicans" from Anna's Archive. It worked, but was very very complicated to figure out how to make it work.

Qazar 1y ago
[dead]
LeratoAustini 1y ago
I've downloaded single books several times recently (annas-archive.org in the browser): - search for book - tap a result - see a list of links to download mirrors (under 'slow downloads'), tap a link - get a countdown timer - timer expires, download links appear - click a link, book downloads just like any other download
- iosifache 1y ago
  The waiting part is nonexistent if you have an active donation (which is also required by this MCP server for API access). The fast downloads mean you request a book and start downloading it immediately.
  - IshKebab 1y ago
    "Donation". Let's call it what it is - a subscription.
    - iosifache 1y ago
      [flagged]
- 0points 1y ago
  So just 8 or 9 steps to grab a e-book, instead of just letting the LLM recall it from training data... /s

iosifache 1y ago

UPDATE: Following a suggestion from @irskep, I've added CLI command support for the search and download features. This raised a valid concern - while we're focused on building MCP servers, we shouldn't overlook whether users already have preferred (T/G)UIs available. When they don't exist, we should consider user experience and make our functionality accessible through multiple interfaces beyond just MCP. https://github.com/iosifache/annas-mcp/releases/tag/v0.0.2 https://github.com/iosifache/annas-mcp/releases/tag/v0.0.2

johnjames87 1y ago

[flagged]

iosifache 1y ago
https://hn.algolia.com/?q=anna%27s+archive https://hn.algolia.com/?q=anna%27s+archive
eMPee584 1y ago
The real crime is an economic system that limits the spread of knowledge and access to other "human rights" by requiring everyone to hustle to survive (and, if possible, increase capital gains for the financial overlords) when we would already be technologically equipped to feed and house well all of mankind, instead of letting thousands of children starve to death each day and restricting access to education so that billions miss out on their intellectual development - a void easily filled with addictive media full of rage and distraction. Pirating books is just a symptom of this wretched system. And it is not enough - RISE, HN! .. towards RBE & beyond..
mandw 1y ago
I recently downloaded a number of magazines froma now defunct publication from late eighties. No onesells them and annas archive was the only place i found them. its not exclusivly pirating, its a source for a lot of out of print.
mrweasel 1y ago
It's not piracy if you download the books to train your LLM.... Apparently.
pjc50 1y ago
Ai is dependent on piracy .. and apparently that's fine now.

ospider 1y ago

Why do people keep building servers for such a silly protocol?

zipping1549 1y ago
It's easy and has the word AI in it.
- chisleu 1y ago
  It's infinitely useful for people who's workflows involve LLM agents.
UncleEntity 1y ago
People may do things you have no need for or don't agree with. Welcome to Planet Earth.
aussieguy1234 1y ago
Let's say you want to use the tool `ssh` and let's imagine ssh has just been released and it's not in the AI agents training set yet. It won't know in this case how to use the ssh command line tool. MCP provides a standard way to tell an AI agent how and when to use tools. So if you had an SSH MCP server, you'd simply plug that in, now your AI agent automagically has SSH capabilities.

snickerdoodle12 1y ago

Insane how HN cheers on piracy nowadays, all because it helps them train their LLMs

artninja1988 1y ago
I only see comments complaining about imaginary property "violations" on ai threads funnily enough.
hombre_fatal 1y ago
This isn’t about training LLMs at all. Also, HN like the rest of the world was always pro-piracy and getting the fruits of your labor without paying for it. The only time I’ve seen anti-piracy comments has been been wrt LLM training. Suddenly people pretend to care but it feels performative.
- GuinansEyebrows 1y ago
  i wonder if you're considering the huge difference between: - individuals pirating copies of things for sole consumption or relatively-miniscule distribution; and - large highly-funded institutions that pirate content for the sole purpose of generating revenue from it ...and why that might lead people to feel differently about one and the other (not to mention the outsized punitive response to the former compared to the latter).
xboxnolifes 1y ago
I don't think HN was ever much against piracy. At least not personal use piracy.
MisterTea 1y ago
Information wants to be free.
CamperBob2 1y ago
The evolution of intelligence and its intersection with universal access to knowledge is more important than copyright. To the extent copyright interests want to pick a fight with AI, they must lose, and decisively so.