7 ms·

Show HN: I built a tiny-VPS friendly RSS aggregator and reader

187 points by rook1e_dev 2y ago

Hi, folks. As an RSS user, I tried Inoreader and Feedly, then ended up self-hosting a Miniflux instance on my homelab. A few months ago, I moved to another city and had to shut down my homelab for a long time, so I couldn't access my local miniflux. It was quite inconvenient. I decided to self-host my RSS aggregator on a tiny VPS or PaaS such as fly.io. However, Miniflux requires a PostgreSQL database, which may isn't suitable for a tiny VPS instance. So I built fusion with Golang and SQLite. It contains basic features such as Group, Bookmark, Search, Automatically feeds sniffing, Import/Export OPML file, etc. It uses about 80MB of Mem and negligible CPU usage (metrics here: https://imgur.com/a/EJIdevn https://imgur.com/a/EJIdevn). Feel free to share your questions and suggestions. BTW, I also built an online tool to sniff RSS links from a URL. (https://rss-finder.rook1e.com/ https://rss-finder.rook1e.com/)

QasimK 2y ago
This is what self-hosted software should be. An app, self-contained, (essentially) a single file with minimal dependencies. Not something so complex that it requires docker. Not something that requires you to install a separate database. Not something that depends on redis and other external services. I’ve turned down many self-hosted options due to the complexity of the setup and maintenance.
- holsta 2y ago
  > Node 21 with NPM > Sveltekit, shadcn-svelte > Echo, GORM > gofeed And the documentation literally specifies it requires docker. How did this become "minimal dependencies"?
  - bArray 2y ago
    Almost guaranteed this doesn't build in ~2 years and you would likely need to know all of those languages to fix it.
    - Klonoar 2y ago
      I don't think it's that bad, but skimming that repo leaves me feeling like the frontend stack used makes things far more complex than they need to be. Still, seems like a great project. I added it to my list of things to check out whenever I have a free weekend... if I ever get there.
    - skydhash 2y ago
      Encountered the same problem last year. My tech stack was React Native as I was mostly building stuff that has few interactions and it was easy to get something good on both platforms. Then, I got a notice from Google about API version. Updating the project was such a nightmare due to compatibility issue. Some libraries were abandoned. Some had breaking changes. It was easier to write the two native versions than to deal with npm mess.
      - cxr 2y ago
        For any project that you depend on that has opted into the NPM quagmire, you really should be running `git add --force ./node_modules` and periodically pushing a copy of this to a branch/repo that you control instead of depending on upstream, since most projects that ostensibly use Git tend to thwart its entire raison d'etre—hobbling its ability to do effective version control by abusing .gitignore for their overlay VCS of choice (i.e. the "package manager").
        skydhash 2y ago
        The issue was not finding the libraries code. The issue was the churn. So one day, you have a (in my case, small) set of libraries to get things going. Then 2 years after, compilation issues as they all have different requirements. So you have to find another common intersection between them and the node/react-native versions. I should have vendored some in the project. In Android, libraries are much stabler. Deprecated functions are picked by the IDE and an alternative is often presented in the comments. I'd much prefer to have big libraries (as we have tree-shaking) especially when dealing with frameworks instead of the bazillion packages when trying to do anything with npm.
  - eviks 2y ago
    The docs literally specify it doesn't "Deploy with a single binary"
    - holsta 2y ago
      It would be nice if people realised that not everything is Linux (or Windows or Mac).
      - mynegation 2y ago
        It explicitly states small VPS as a target, so - yes - Linux. I realize you can deploy non-Linux image on VPS, but hosting docker containers on Linux is the default
      - royjacobs 2y ago
        Where would it end, though? Should the developer also provide build instructions for Haiku and AmigaOS?
  - rook1e_dev 2y ago
    Hi. Just to clarify. > Node 21 with NPM > Sveltekit, shadcn-svelte The front-end things are used during the build process. You don't need them when deploy. > And the documentation literally specifies it requires docker. Fusion can be deployed as a *single binary*. Docker is just the recommended way. Maybe we need to make documentation clearer.
    - drusklo 2y ago
      Looks good! I am curious on why you recommend to deploy using docker, being a single binary with no external dependencies I find the deployment simple enough. I write all my personal projects using Go and one of the things I most like is that it compiles to a binary without external dependencies.
      - rook1e_dev 2y ago
        That's mostly for Windows users. The SQLite driver uses cgo, so we use both Ubuntu and Windows Server in CI to avoid cross-compiling. However, we still can't confirm that it's 100% ok on Windows. If any weird bugs occur on Windows, we don't have much experience or energy to deal with them. The Docker image is based on Debian, we are more familary with it.
        drusklo 2y ago
        That makes a lot of sense, thanks for taking the time to explain it.
        _lvbh 2y ago
        For the SQLite driver, check out the one by modernc for pure Go
    - holsta 2y ago
      It's an interesting difference in backgrounds, maybe? I tried to build this on OpenBSD where we don't have docker and we use 'make' instead of random shell scripts. For what it's worth, this is from the build log: error code 1 error path /home/holsta/3rdparty/fusion/frontend/node_modules/@sveltejs/kit error command failed error command sh -c node postinstall.js error /home/holsta/3rdparty/fusion/frontend/node_modules/rollup/dist/native.js:84 error throw new Error( error ^ error error Error: Your current platform "openbsd" and architecture "x64" combination is not yet supported by the native Rollup build. Please use the WASM build "@rollup/wasm-node" instead.
- generallee5686 2y ago
  Agreed. Lately I'm coming across self-hosted services that not only require docker, they essentially require a docker-compose file full of many other dependencies. The other thing that blows my mind is how many of these self-hosted applications require mysql/postgres instead of just using sqlite.
phubbard 2y ago
This is awesome! I currently run FreshRSS on a pi5 and this looks like a nice, non-PHP alternative.
beepbooptheory 2y ago
Looks really great. I can't tell from the readme, but can this produce its own feed which is the aggregation of the other feeds? That is the feature I always want and don't see too much with apps like this.
- rook1e_dev 2y ago
  Hi, thanks for the suggestion! Currently it doesn't have that feature. You can create an issue about it on GitHub so we can evaluate how much it is needed.
pglevy 2y ago
I love this idea! As a code hobbyist, I tried to do something like this for DIY podcast aggregation using Jekyll (https://github.com/lowcodelounge/pod-force https://github.com/lowcodelounge/pod-force). Look forward to checking this out to learn more. Thanks for sharing!
skrause 2y ago
I can recommend https://github.com/nkanaev/yarr https://github.com/nkanaev/yarr which is also a single Go binary, consumes only 40 MB of RAM and uses SQLite as well.
nedrylandJP 2y ago
I've been using Slack /feed (ugh) to get RSS notifications because I want pings from certain feeds. I have not found a different, easy lightweight way to do this.
visil 2y ago
Looks cool! I'm currently using Selfoss, which is fine in terms of requirements/maintenance, but if I didn't, I'd probably give this a try!
gcascio 2y ago
Cool project! Also it's welcome resource to learn, since I just started learning Go and was trying myself at an RSS reader (https://stille.app https://stille.app).
- rsolva 2y ago
  Looks really nice! Not sure if it is intentional, but "stille" means "silent" in Norwegian :)
pacifika 2y ago
Does it produce a rss feed itself?
- rook1e_dev 2y ago
  Currently it doesn't have that feature. You can create an issue about it on GitHub so we can evaluate how much it is needed.
Xeoncross 2y ago
Simple, no-fuss self-hosted server software should really be Rust, or Go / C++ if needed. All three of them allow you embed the UI assets (media, JS, CSS) into the binary and all three work great with key/value stores (badger, leveldb, rocksdb, etc..) or SQLite. There is no install. No setup. No packages to download first. Just a simple binary that respects OS signals, has crazy good throughput, and uses so little memory that your router can run it. Please, consider moving your JVM/Node.js/Electron project to one of these as a chance to jump into really performant software. (There are also a lot of RSS servers written in Rust/Go/C on Github: https://github.com/search?q=rss+host+language%3AGo+language%3ARust+&type=repositories https://github.com/search?q=rss+host+language%3AGo+language%...)
- _lvbh 2y ago
  This one is written in Go. The front end in Svelte is built into static files and served by Go. The docker container just makes it easy to build but they do distribute a single static binary
- watermelon0 2y ago
  Node.js / PHP / dotnet / JVM projects can be bundled to a single binary file. Sure, the binaries are bigger, and they cannot run on routers, but you can easily run many of them simultaneously on RaspberryPi, which is usually the bare minimum HW people use to self-host. Performance wise, difference should be negligible for most use cases, and allows people to write software in the language that's most suited for them and the project.
  - Xeoncross 2y ago
    > Performance wise, difference should be negligible for most use cases I think you mean unnoticeable (or similar); the difference in performance (memory usage and CPU cycles) is certainly not negligible.
bigiain 2y ago
You might have seen this already, but Rachel By The bay has a few recent blog posts on how feed readers commonly get things wrong, and a service she's got to "score" your feed reader based on it's observed behaviour: https://rachelbythebay.com/w/2024/05/27/feed/ https://rachelbythebay.com/w/2024/05/27/feed/ https://rachelbythebay.com/w/2024/05/29/score/ https://rachelbythebay.com/w/2024/05/29/score/ https://rachelbythebay.com/w/2024/05/30/fs/ https://rachelbythebay.com/w/2024/05/30/fs/ Don't be "that guy" who releases software that once deployed widely creates "thundering herds" that knock unsuspecting websites down <glares at Mastodon>
djfergus 2y ago
Great project. Spun the docker version up on an Oracle always free VPS instance (server class: potato) in a few seconds and it runs great with my imported OPML with a few hundred feeds. As the enshittification of feedly continues I'm keen to find a replacement. Am I missing the "mark all as read" button? That would be helpful vs going through 13 years of old posts and marking read page by page (albeit quickly at 500 at a time).
- rook1e_dev 2y ago
  You can custom the page size by params in url ( ?page_size=1000). But large page size can cause page performance problems.

187 points by rook1e_dev 2y ago

QasimK 2y ago

This is what self-hosted software should be. An app, self-contained, (essentially) a single file with minimal dependencies. Not something so complex that it requires docker. Not something that requires you to install a separate database. Not something that depends on redis and other external services. I’ve turned down many self-hosted options due to the complexity of the setup and maintenance.

holsta 2y ago
> Node 21 with NPM > Sveltekit, shadcn-svelte > Echo, GORM > gofeed And the documentation literally specifies it requires docker. How did this become "minimal dependencies"?
- bArray 2y ago
  Almost guaranteed this doesn't build in ~2 years and you would likely need to know all of those languages to fix it.
  - Klonoar 2y ago
    I don't think it's that bad, but skimming that repo leaves me feeling like the frontend stack used makes things far more complex than they need to be. Still, seems like a great project. I added it to my list of things to check out whenever I have a free weekend... if I ever get there.
  - skydhash 2y ago
    Encountered the same problem last year. My tech stack was React Native as I was mostly building stuff that has few interactions and it was easy to get something good on both platforms. Then, I got a notice from Google about API version. Updating the project was such a nightmare due to compatibility issue. Some libraries were abandoned. Some had breaking changes. It was easier to write the two native versions than to deal with npm mess.
    - cxr 2y ago
      For any project that you depend on that has opted into the NPM quagmire, you really should be running `git add --force ./node_modules` and periodically pushing a copy of this to a branch/repo that you control instead of depending on upstream, since most projects that ostensibly use Git tend to thwart its entire raison d'etre—hobbling its ability to do effective version control by abusing .gitignore for their overlay VCS of choice (i.e. the "package manager").
      - skydhash 2y ago
        The issue was not finding the libraries code. The issue was the churn. So one day, you have a (in my case, small) set of libraries to get things going. Then 2 years after, compilation issues as they all have different requirements. So you have to find another common intersection between them and the node/react-native versions. I should have vendored some in the project. In Android, libraries are much stabler. Deprecated functions are picked by the IDE and an alternative is often presented in the comments. I'd much prefer to have big libraries (as we have tree-shaking) especially when dealing with frameworks instead of the bazillion packages when trying to do anything with npm.
- eviks 2y ago
  The docs literally specify it doesn't "Deploy with a single binary"
  - holsta 2y ago
    It would be nice if people realised that not everything is Linux (or Windows or Mac).
    - mynegation 2y ago
      It explicitly states small VPS as a target, so - yes - Linux. I realize you can deploy non-Linux image on VPS, but hosting docker containers on Linux is the default
    - royjacobs 2y ago
      Where would it end, though? Should the developer also provide build instructions for Haiku and AmigaOS?
- rook1e_dev 2y ago
  Hi. Just to clarify. > Node 21 with NPM > Sveltekit, shadcn-svelte The front-end things are used during the build process. You don't need them when deploy. > And the documentation literally specifies it requires docker. Fusion can be deployed as a *single binary*. Docker is just the recommended way. Maybe we need to make documentation clearer.
  - drusklo 2y ago
    Looks good! I am curious on why you recommend to deploy using docker, being a single binary with no external dependencies I find the deployment simple enough. I write all my personal projects using Go and one of the things I most like is that it compiles to a binary without external dependencies.
    - rook1e_dev 2y ago
      That's mostly for Windows users. The SQLite driver uses cgo, so we use both Ubuntu and Windows Server in CI to avoid cross-compiling. However, we still can't confirm that it's 100% ok on Windows. If any weird bugs occur on Windows, we don't have much experience or energy to deal with them. The Docker image is based on Debian, we are more familary with it.
      - drusklo 2y ago
        That makes a lot of sense, thanks for taking the time to explain it.
      - _lvbh 2y ago
        For the SQLite driver, check out the one by modernc for pure Go
  - holsta 2y ago
    It's an interesting difference in backgrounds, maybe? I tried to build this on OpenBSD where we don't have docker and we use 'make' instead of random shell scripts. For what it's worth, this is from the build log: error code 1 error path /home/holsta/3rdparty/fusion/frontend/node_modules/@sveltejs/kit error command failed error command sh -c node postinstall.js error /home/holsta/3rdparty/fusion/frontend/node_modules/rollup/dist/native.js:84 error throw new Error( error ^ error error Error: Your current platform "openbsd" and architecture "x64" combination is not yet supported by the native Rollup build. Please use the WASM build "@rollup/wasm-node" instead.
generallee5686 2y ago
Agreed. Lately I'm coming across self-hosted services that not only require docker, they essentially require a docker-compose file full of many other dependencies. The other thing that blows my mind is how many of these self-hosted applications require mysql/postgres instead of just using sqlite.

phubbard 2y ago

This is awesome! I currently run FreshRSS on a pi5 and this looks like a nice, non-PHP alternative.

beepbooptheory 2y ago

Looks really great. I can't tell from the readme, but can this produce its own feed which is the aggregation of the other feeds? That is the feature I always want and don't see too much with apps like this.

rook1e_dev 2y ago
Hi, thanks for the suggestion! Currently it doesn't have that feature. You can create an issue about it on GitHub so we can evaluate how much it is needed.

pglevy 2y ago

I love this idea! As a code hobbyist, I tried to do something like this for DIY podcast aggregation using Jekyll (https://github.com/lowcodelounge/pod-force https://github.com/lowcodelounge/pod-force). Look forward to checking this out to learn more. Thanks for sharing!

skrause 2y ago

I can recommend https://github.com/nkanaev/yarr https://github.com/nkanaev/yarr which is also a single Go binary, consumes only 40 MB of RAM and uses SQLite as well.

nedrylandJP 2y ago

I've been using Slack /feed (ugh) to get RSS notifications because I want pings from certain feeds. I have not found a different, easy lightweight way to do this.

visil 2y ago

Looks cool! I'm currently using Selfoss, which is fine in terms of requirements/maintenance, but if I didn't, I'd probably give this a try!

gcascio 2y ago

Cool project! Also it's welcome resource to learn, since I just started learning Go and was trying myself at an RSS reader (https://stille.app https://stille.app).

rsolva 2y ago
Looks really nice! Not sure if it is intentional, but "stille" means "silent" in Norwegian :)

pacifika 2y ago

Does it produce a rss feed itself?

rook1e_dev 2y ago
Currently it doesn't have that feature. You can create an issue about it on GitHub so we can evaluate how much it is needed.

Xeoncross 2y ago

Simple, no-fuss self-hosted server software should really be Rust, or Go / C++ if needed. All three of them allow you embed the UI assets (media, JS, CSS) into the binary and all three work great with key/value stores (badger, leveldb, rocksdb, etc..) or SQLite. There is no install. No setup. No packages to download first. Just a simple binary that respects OS signals, has crazy good throughput, and uses so little memory that your router can run it. Please, consider moving your JVM/Node.js/Electron project to one of these as a chance to jump into really performant software. (There are also a lot of RSS servers written in Rust/Go/C on Github: https://github.com/search?q=rss+host+language%3AGo+language%3ARust+&type=repositories https://github.com/search?q=rss+host+language%3AGo+language%...)

_lvbh 2y ago
This one is written in Go. The front end in Svelte is built into static files and served by Go. The docker container just makes it easy to build but they do distribute a single static binary
watermelon0 2y ago
Node.js / PHP / dotnet / JVM projects can be bundled to a single binary file. Sure, the binaries are bigger, and they cannot run on routers, but you can easily run many of them simultaneously on RaspberryPi, which is usually the bare minimum HW people use to self-host. Performance wise, difference should be negligible for most use cases, and allows people to write software in the language that's most suited for them and the project.
- Xeoncross 2y ago
  > Performance wise, difference should be negligible for most use cases I think you mean unnoticeable (or similar); the difference in performance (memory usage and CPU cycles) is certainly not negligible.

bigiain 2y ago

You might have seen this already, but Rachel By The bay has a few recent blog posts on how feed readers commonly get things wrong, and a service she's got to "score" your feed reader based on it's observed behaviour: https://rachelbythebay.com/w/2024/05/27/feed/ https://rachelbythebay.com/w/2024/05/27/feed/ https://rachelbythebay.com/w/2024/05/29/score/ https://rachelbythebay.com/w/2024/05/29/score/ https://rachelbythebay.com/w/2024/05/30/fs/ https://rachelbythebay.com/w/2024/05/30/fs/ Don't be "that guy" who releases software that once deployed widely creates "thundering herds" that knock unsuspecting websites down <glares at Mastodon>

djfergus 2y ago

Great project. Spun the docker version up on an Oracle always free VPS instance (server class: potato) in a few seconds and it runs great with my imported OPML with a few hundred feeds. As the enshittification of feedly continues I'm keen to find a replacement. Am I missing the "mark all as read" button? That would be helpful vs going through 13 years of old posts and marking read page by page (albeit quickly at 500 at a time).

rook1e_dev 2y ago
You can custom the page size by params in url ( ?page_size=1000). But large page size can cause page performance problems.