8 ms·

Mike Perham, Creator of Sidekiq: From Employment to Independence

201 points by derekkraan 3y ago

davidw 3y ago
He's also a standup guy in terms of contributing back to worthy things here in Oregon.
john_fushi 3y ago
I guess Matsumoto should have charged for Ruby. I'm sure Mike Perham and Derek Kraan would have been glad to pay for that and that the Ruby community would be in a strong and healthy shape.
- jorge-d 3y ago
  Well Sidekiq is free to use. It's only the pro version that he charges and the free version code is open source. I don't see the problem in having that kind of business model, it still allows the community to thrive and offers entreprises a way to have premium support. Plus it allows him to invest more time in maintaining the free version.
  - phamilton 3y ago
    I have no problem paying for the Pro version, but one if its marketing pitches is "enhanced reliability", which is a wild marketing spin on "the free version will lose jobs in fairly common scenarios". In sidekiq without super_fetch (a paid feature), any jobs in progress when a worker crashes are lost forever. If a worker merely encounters an exception the job will be put back on the queue and retried but a crash means the job is lost. Again, no problem paying for Pro, but I would prefer a little more transparency on how big a gap that is.
    - durkie 3y ago
      how often do your workers crash? i rely heavily on sidekiq and don't think I see this very often, if ever.
      - Colex 3y ago
        it’s not uncommon to lose jobs in sidekiq if you heavily rely on it and have a lot of jobs running. If using the free version for mission critical jobs, I usually run that task as a cron job to ensure that it will re-try if the job is lost. I have in the past monitored how many jobs were lost and, although a small percentage, it was still recurring thing.
      - phamilton 3y ago
        We process around 50M sidekiq jobs a day across a few hundred workers on a heavily autoscaled infrastructure. Over the past week there were 2 jobs that would have been lost if not for superfetch. It's not a ton, but it's not zero. And when it comes to data durability the difference between zero and not zero is usually all that matters. Edit for additional color: One of the most common crashes we'll see is OutOfMemory. We run in a containerized environment and if a rogue job uses too much memory (or a deploy drastically changes our memory footprint) the container will be killed. In that scenario, the job is not placed back into the queue. SuperFetch is able to recover them, albeit with really lose guarantees around "when".
        ZephyrBlu 3y ago
        Let me get this straight, you're complaining about eight 9s of reliability? 50,000,000 * 7 = 350,000,000 2 / 350,000,000 = 0.000000005714286 1 - (2 / 350,000,000) = 0.999999994285714 = 99.999999% > It's not a ton, but it's not zero. And when it comes to data durability the difference between zero and not zero is usually all that matters. If your system isn't resilient to 2 in 350,000,000 jobs failing I think there is something wrong with your system.
        deleted 3y ago
        [deleted]
        phamilton 3y ago
        This isn't about 2 in 350,000,000 jobs failing. It's about 2 jobs disappearing entirely. It's not reliability we're talking about, it's about durability. For reference, S3 has eleven 9s of durability. Every major queuing system solves this problem. RabbitMQ uses unacknowledged messages which are pinned to a tcp connection, so when that connection drops before acknowledging them they get picked up by another worker. SQS uses visibility timeouts, where if the message hasn't been successfully processed within a time frame it's made available to other workers. Sidekiq free edition chooses not to solve it. And that's a fine stance for a free product, but just one I wish was made clearer.
    - aqme28 3y ago
      When we used Sidekiq in production, not only did I never see crashes that lost us jobs, but there are also ways to protect yourself from that. I highly recommend writing your jobs to be idempotent.
      - phamilton 3y ago
        Idempotence doesn't solve this problem. The jobs are all idempotent. The problem is that jobs will never be retried if a crash occurs. This doesn't happen at a high rate, but it happens more than zero times per week for us. We pay for Sidekiq Pro and have superfetch enabled so we are protected. If we didn't do so we'd need to create some additional infra to detect jobs that were never properly run and re-run them.
        aqme28 3y ago
        Fair enough about idempotence. I'm still confused about what you're saying though. You're saying that the language of "enhanced reliability" doesn't reflect losing 2 jobs over about 50*7 million (from your other comment)? And that if you didn't pay for the service, you'd have to add some checks to make up for this? That all seems incredibly reasonable to me.
        ransackdev 3y ago
        Or install an opensource gem[1] that recreates the functionality using the same redis rpoplpush[2] command [1] https://gitlab.com/gitlab-org/ruby/gems/sidekiq-reliable-fetch https://gitlab.com/gitlab-org/ruby/gems/sidekiq-reliable-fet... [2] https://redis.io/commands/rpoplpush/#pattern-reliable-queue https://redis.io/commands/rpoplpush/#pattern-reliable-queue
        danenania 3y ago
        Crashes are under your control though. They’re not caused by sidekiq. And you could always add your own crash recovery logic, as you say. To me that makes it a reasonable candidate for a pro feature. It’s hard to get this right though. No matter where the line gets drawn, free users will complain that they don’t get everything for free.
    - arkasan 3y ago
      I wish this was prominently documented. Most people new to Sidekiq have no idea that the job will be lost forever if you simply hard kill the worker. I have seen a couple of instances where the team had Sidekiq Pro, but they had not enabled reliable fetch because they were unaware of this problem
    - tebbers 3y ago
      Exactly why we refuse to use Sidekiq. “Hey, you have to pay to guarantee your jobs won’t just vanish”. No thanks.
    - mperham 3y ago
      The free version acts exactly like Resque, the previous market leader in Ruby background jobs. If it was good enough reliability for GitHub and Shopify to use for years, it was good enough for Sidekiq OSS too. Here's Resque literally using `lpop` which is destructive and will lose jobs. https://github.com/resque/resque/blob/7623b8dfbdd0a07eb04b19fb25b16a8d6f087f9a/lib/resque/data_store.rb#L113 https://github.com/resque/resque/blob/7623b8dfbdd0a07eb04b19...
      - phamilton 3y ago
        > If it was good enough reliability for GitHub and Shopify to use for years, it was good enough for Sidekiq OSS too. Great point, and thanks for chiming in. I wonder if containerization has made this more painful (due to cgroups and OOMs). The comments here are basically some people saying it's never been a problem for them and some people saying they encounter it a lot (in containerized environments) and have had to add mitigations. Either way, my observation is a lot of people not paying for Sidekiq Pro should. I hope you can agree with that.
- the_gastropod 3y ago
  To be fair to Mike, Sidekiq is absolutely free. He sells an enterprise version for money, that comes with support.
  - 1123581321 3y ago
    And he only started because companies were telling him it would be easier if they could just pay him.
    - bombcar 3y ago
      Anyone who has a free thing, sell something that produces an expendable invoice or receipt. Makes it easy to use company funds to pay for company work. “Buy me a coffee” buttons don’t really cut it.
    - toomanyrichies 3y ago
      This is great. Reminds me of the Derek Sivers post [1], "Don't start a business until people are asking you to." 1. https://sive.rs/asking https://sive.rs/asking
      - mcsniff 3y ago
        Sorry, this is bogus. "If I had asked people what they wanted, they would have said faster horses." or whatever the quote is. There are tons of businesses selling products that are ideas brought to life from scratching their own itch or simply a desire to make something and put it out there. Personally, I am one of those people who started a business based on an idea with no validation before launching it. I built it in its entirety, then went to places and people who I expected would want it and low-and-behold, I am making a living doing it.
        toomanyrichies 3y ago
        > There are tons of businesses selling products that are ideas brought to life from scratching their own itch or simply a desire to make something and put it out there. I'd bet money that the number of businesses which fail because they boil down to "a solution in search of a problem" is vastly larger than the number of businesses that succeeded despite performing "no validation". That said, "making something" and "starting a business" are two different things. I would challenge you to point out where in the post he argues against making something, especially for the reasons you mention. > I am one of those people who started a business based on an idea with no validation before launching it. I built it in its entirety, then went to places and people who I expected would want it and low-and-behold, I am making a living doing it. Consider the possibility that you're in the minority there, and that you succeeded despite performing no validation. > Sorry, this is bogus. So if it doesn't apply to you personally, or in all cases, then you dismiss it as "bogus", full stop? Are you in the habit of doing this often? Also, for future reference, it's "lo-and-behold", not "low-and-behold". [1] 1. https://en.wiktionary.org/wiki/lo_and_behold https://en.wiktionary.org/wiki/lo_and_behold
- ianbutler 3y ago
  People need to eat man, doing something for free is a luxury not many possess. We can see how messed up the financial situation is for a lot of Open Source devs and their lives would be better if they were charging money instead of subsisting off grants and donations. I don't even necessarily disagree with your point that without being free those things wouldn't have taken off but we need to find a way to strike a balance in the developer community. Sidekiq having a free version and an enterprise version walks an okay middle line imo.
  - wyldberry 3y ago
    I personally try to spend money or use ad supported anything that is open-source just to help someone else eat. It really hit home for me about five years ago when the author of a WoW addon[0] couldn't develop anymore because of his financial and life situation. So many communities across the web rely on people putting in their spare hours for free just to enjoy things. Whether it's spreadsheets in Eve, Addons and Weak Auras in WoW, forum analysis posts, or whatever goes on in the depths of pvpoke, so much free labor underpins massive parts of the world today. I would love something that I could donate x money to per month and then based on usage, have it dole out to all the content providers with perhaps a minimum per month. It just seems daunting to do that as a) not a crypto scheme and b.) across all the various creator landscapes. [0]https://www.polygon.com/2018/9/25/17901552/world-of-warcraft-deadly-boss-mods-patreon https://www.polygon.com/2018/9/25/17901552/world-of-warcraft...
- bdcravens 3y ago
  Not by Matz, but there are paid versions of Ruby out there.
- ch4s3 3y ago
  This is a very bad take. From an OSS perspective languages can attract large communities of contributors and corporate sponsors because of their broad appeal and utility. Specialized libraries will have more trouble doing both and may need alternate models to sustain themselves. From a business perspective, Mike offers not only a free version but the paid enterprise version comes with support from Mike and his team, which is something you can't get from a language owner unless you outright hire them or they run a consultancy.
- mattgreenrocks 3y ago
  Hilarious take given the fact that there are plenty of orgs making an order of magnitude more than Mike is that rely on Ruby.
shubhamjain 3y ago
This is one of the best businesses I have heard of. Even 6-7 years back, Sidekiq was grossing $80k/month [1]. I imagine it must be much more now. No servers to maintain, no employees, minimal support work and Mike has the complete freedom to work on updates whenever he prefers. Almost entire revenue minus payment processing costs must be profit. Imagine making $1M / yr working only 10-20hrs per week (Don't wish to presume here, but that's my estimate of the man-hours it costs to run). > I still have 0 employees and don't plan to hire. I tune my business processes to run as lean as possible: Most of my customers are on credit card so their payments are automatic. The gem servers take about one day of maintenance per year. I can't really outsource much of my support work because it is so technical and specialized. > My gem server is a $6 droplet on DigitalOcean. Because gems are just static files, that little droplet can handle millions of requests per day with just Apache. Oh, and I run three servers in parallel for failover purposes for a grand total of $18/mo. I wonder what makes Sidekiq special? I don't imagine any equivalent product (background job processing) in the other languages is raking in that much, if making any revenue in the first place. [1]: https://www.indiehackers.com/product/sidekiq https://www.indiehackers.com/product/sidekiq
- iudqnolq 3y ago
  Oban is a roughly equivalent elixir library. They claim to have hundreds of customers and their cheapest plan is $500/yr, so maybe they make an order of magnitude less? Considering the relative language popularity that seems pretty good.
- sparker72678 3y ago
  Sidekiq is special because it's crazy performant, and it _works_. Mike has built a system that's rock solid, and he doesn't twiddle around with it willy nilly. He keeps it super stable. The people paying for it (I'm one of those people) are paying for the reliability, in addition to the features. Mike's also managed to find a real sweet-spot with the features he holds out for Pro/Enterprise. The stratification is just right to nudge you up, but it always feels like you're getting a good deal when you pay.
- sorentwo 3y ago
  > I wonder what makes Sidekiq special? Time, know-how, and long term dedication are the key ingredients from my perspective. Maybe he's working 10-20 hours now, but there's support and development every day for the past 11 years. > I don't imagine any equivalent product (background job processing) in the other languages is raking in that much, if making any revenue in the first place. I guarantee, there are background job processors in other languages making revenue.
jack_riminton 3y ago
"I toyed with Sidekiq.js but decided to kill that idea real quick because, as we all know, JavaScript is terrible" This is why I love the ruby community, so much sense :)
- rektide 3y ago
  From the JS community, this is sadly the level of snark we expect. It doesn't have to be like this. Your community tried to be better once. Have you forgotten? https://jasonfleetwoodboldt.com/courses/stepping-up-rails/matz-is-nice-and-so-we-are-nice/ https://jasonfleetwoodboldt.com/courses/stepping-up-rails/ma...
  - jack_riminton 3y ago
    The other thing I like about the ruby community is the sense of humour
    - rektide 3y ago
      At others expense tbough? Is this the form of merriment we want? Gl;hf.
  - pmarreck 3y ago
    You act as if some percentage less than 100% of Ruby users have extensive experience with JS as well... OK, maybe it's only, say, 96% of Ruby users, or 100% of Rails users at least
    - rektide 3y ago
      See like, this isn't constructive maligning at all, it's more undirected blanket unniceness. It costs nothing to actually put down a reason for being upset, versus having totally generic downputs. Very few things are truly rotten to the core, most bad things have some bad aspects but could be much better if ___. If something is rotten to the core the central articles of faith for why that's so should be declared. Plenty of Ruby folks have gone on to do great friendly Rails ish JS works. Ember.js is very batteries included, tries to show that yes all the potential is there to pave a nice well integrated happy-path system. To assume misery seems unreasonable; many have done fine. I think we have a real obligation to do better to ourselves & one another than to foster shallow prejudices. Trump in his April 11th Tucker Carlson interview was asked why he thought Dems weren't worried about nuclear weapons. "That's because they don't understand life. That's because they don't understand what it is that you have to understand." This is an irrefutable claim. There's no point to start discussing here, no possibility that the other side might change or have some nuances. Let's not do this. Let's rise to higher places where we take real appraising concerned looks at things, rather than just dismiss stuff out of hand. We don't even have to be nice, but let's at least strive to be somewhat useful.
aqme28 3y ago
Love Sidekiq and loved how he runs it. Really feels like the right way to make money on OSS. Enterprise never crippled the free version.
adrianthedev 3y ago
Mike is an inspiration to me and my project!
- ezekg 3y ago
  Same. He's always been a role model for my business as well, especially when I was just starting out. My business uses Sidekiq every day. It really is a great piece of software. He's also one of the reasons I think open source software can be highly profitable if people just figure out a business model that works.
pmarreck 3y ago
Spoke with Mike at a couple of Railsconfs a few years ago (before I got into Elixir). Really great guy.
fidrelity 3y ago
Mike Perham is an inspiration to me both in terms of Ruby/tech as well as business. I've been using sidekiq now for almost a decade and had the pleasure to meet Mike at a Ruby conf many years ago.
sandstrom 3y ago
Sidekiq Pro is great, we're paying for it! 10k a year I think. But for people who are interested in alternatives, I'd also suggest Good Job (runs on Postgresql). https://github.com/bensheldon/good_job https://github.com/bensheldon/good_job
sickcodebruh 3y ago
Sidekiq is one of my favorite things about using Ruby, I include it in my short list every time a "Why use Ruby?" question gets asked. I miss it desperately when I work in other languages. I plan on giving Faktory a shot ASAP. Mike has always struck me as a great guy, really the kind of person who you're happy to know is building one of your favorite tools. In a world of starving open source contributors and mega corps throwing around weight, his success with Sidekiq stands out. It makes me very happy to see.
andris9 3y ago
Mike Perham's Sidekiq was what I modeled my own business, EmailEngine, after. I took my decade's worth of expertise in email protocol implementations (I also run the open-source Nodemailer library) and built an email gateway app my customers can download and run on their machines or servers. Just like Sidekiq Pro, the customers need an active subscription to run it, but the app runs on their premises, and they maintain it. I only run the key generation and validation server, which costs me around $50 per month, including all the backups and so on. I quit my job to go full in with EmailEngine exactly one year ago, when MRR for EmailEngine was $500, now it is $3,900 and steadily growing. I do work full time on it, but the result is async - the work I put into EmailEngine today brings me income sometime later, and the recurring income I receive today is unrelated to any effort I put into the company right now.
- mperham 3y ago
  Congratulations, that’s great. I’m always happy to hear of more successful indie developers.
  - andris9 3y ago
    Thank you for your inspiration!
- albertgoeswoof 3y ago
  I was just about to post emailengine here, the business model is so similar to sidekiq. Any other great ideas?
  - andris9 3y ago
    Yeah, it is so similar because I blatantly copied it. I had the old Changelog podcast episode with Mike Perham pretty much on repeat when I tried to figure out how exatly to set up my business.
- alberth 3y ago
  Unsolicited feedback, always link to your business in your posts. I googled “emailengine” and it’s wasn’t super obvious which was your business since so much paid ads exist for that search.
JustLurking2022 3y ago
It's always interesting to find out about these niche subcultures. Having worked with batch schedulers of all varieties for a couple decades, I've never run across Sidekiq but, then again, I didn't realize anyone had actually built anything in Ruby in the past decade.
joshmn 3y ago
I've said this to Mike and I'll say it again: I hope he never gets a job.
everybodyknows 3y ago
Naive question: How do you deter thieves who would pose as legit customers, but instead redistribute the library?
- atonse 3y ago
  I know tons of other companies will share my view that we would never stake anything remotely important on a stolen library to save a minuscule amount of money. It would be absolutely idiotic. Instead, we shell over the negligible monthly fees, spend no time thinking about it, and move on to solve real problems. There are enough of us that it probably simply doesn’t matter.
thaumasiotes 3y ago
> I was Director of Technical Operations at The Clymb, which was outdoor-focused e-commerce vendor (think REI without the stores). I had a team of four and we were responsible for site devops and infrastructure. This was perfect because they were my alpha customer and I was in charge of the tech stack. The Clymb switched to Sidekiq immediately, of course, and every Sidekiq Pro feature I wrote was running in production before it was released. This makes me curious about either (a) conflict-of-interest rules at The Clymb that may or may not have governed the DTO directing the company to use his own separately-owned (commercial?) software. (b) negotiating private ownership of software written for The Clymb. One of those must have been relevant?
- mperham 3y ago
  I started work on Sidekiq right before I joined so I made sure my employment contract had a IP carve out for anything related to Sidekiq. I never charged The Clymb to use the software. Previously they had been using delayed_job and were suffering badly from scale issues due to the use of MySQL as their queue store.
polishdude20 3y ago
Can someone explain what Sidekiq actually does?

201 points by derekkraan 3y ago

He's also a standup guy in terms of contributing back to worthy things here in Oregon.

I guess Matsumoto should have charged for Ruby. I'm sure Mike Perham and Derek Kraan would have been glad to pay for that and that the Ruby community would be in a strong and healthy shape.

jorge-d 3y ago
Well Sidekiq is free to use. It's only the pro version that he charges and the free version code is open source. I don't see the problem in having that kind of business model, it still allows the community to thrive and offers entreprises a way to have premium support. Plus it allows him to invest more time in maintaining the free version.
- phamilton 3y ago
  I have no problem paying for the Pro version, but one if its marketing pitches is "enhanced reliability", which is a wild marketing spin on "the free version will lose jobs in fairly common scenarios". In sidekiq without super_fetch (a paid feature), any jobs in progress when a worker crashes are lost forever. If a worker merely encounters an exception the job will be put back on the queue and retried but a crash means the job is lost. Again, no problem paying for Pro, but I would prefer a little more transparency on how big a gap that is.
  - durkie 3y ago
    how often do your workers crash? i rely heavily on sidekiq and don't think I see this very often, if ever.
    - Colex 3y ago
      it’s not uncommon to lose jobs in sidekiq if you heavily rely on it and have a lot of jobs running. If using the free version for mission critical jobs, I usually run that task as a cron job to ensure that it will re-try if the job is lost. I have in the past monitored how many jobs were lost and, although a small percentage, it was still recurring thing.
    - phamilton 3y ago
      We process around 50M sidekiq jobs a day across a few hundred workers on a heavily autoscaled infrastructure. Over the past week there were 2 jobs that would have been lost if not for superfetch. It's not a ton, but it's not zero. And when it comes to data durability the difference between zero and not zero is usually all that matters. Edit for additional color: One of the most common crashes we'll see is OutOfMemory. We run in a containerized environment and if a rogue job uses too much memory (or a deploy drastically changes our memory footprint) the container will be killed. In that scenario, the job is not placed back into the queue. SuperFetch is able to recover them, albeit with really lose guarantees around "when".
      - ZephyrBlu 3y ago
        Let me get this straight, you're complaining about eight 9s of reliability? 50,000,000 * 7 = 350,000,000 2 / 350,000,000 = 0.000000005714286 1 - (2 / 350,000,000) = 0.999999994285714 = 99.999999% > It's not a ton, but it's not zero. And when it comes to data durability the difference between zero and not zero is usually all that matters. If your system isn't resilient to 2 in 350,000,000 jobs failing I think there is something wrong with your system.
        deleted 3y ago
        [deleted]
        phamilton 3y ago
        This isn't about 2 in 350,000,000 jobs failing. It's about 2 jobs disappearing entirely. It's not reliability we're talking about, it's about durability. For reference, S3 has eleven 9s of durability. Every major queuing system solves this problem. RabbitMQ uses unacknowledged messages which are pinned to a tcp connection, so when that connection drops before acknowledging them they get picked up by another worker. SQS uses visibility timeouts, where if the message hasn't been successfully processed within a time frame it's made available to other workers. Sidekiq free edition chooses not to solve it. And that's a fine stance for a free product, but just one I wish was made clearer.
  - aqme28 3y ago
    When we used Sidekiq in production, not only did I never see crashes that lost us jobs, but there are also ways to protect yourself from that. I highly recommend writing your jobs to be idempotent.
    - phamilton 3y ago
      Idempotence doesn't solve this problem. The jobs are all idempotent. The problem is that jobs will never be retried if a crash occurs. This doesn't happen at a high rate, but it happens more than zero times per week for us. We pay for Sidekiq Pro and have superfetch enabled so we are protected. If we didn't do so we'd need to create some additional infra to detect jobs that were never properly run and re-run them.
      - aqme28 3y ago
        Fair enough about idempotence. I'm still confused about what you're saying though. You're saying that the language of "enhanced reliability" doesn't reflect losing 2 jobs over about 50*7 million (from your other comment)? And that if you didn't pay for the service, you'd have to add some checks to make up for this? That all seems incredibly reasonable to me.
      - ransackdev 3y ago
        Or install an opensource gem[1] that recreates the functionality using the same redis rpoplpush[2] command [1] https://gitlab.com/gitlab-org/ruby/gems/sidekiq-reliable-fetch https://gitlab.com/gitlab-org/ruby/gems/sidekiq-reliable-fet... [2] https://redis.io/commands/rpoplpush/#pattern-reliable-queue https://redis.io/commands/rpoplpush/#pattern-reliable-queue
      - danenania 3y ago
        Crashes are under your control though. They’re not caused by sidekiq. And you could always add your own crash recovery logic, as you say. To me that makes it a reasonable candidate for a pro feature. It’s hard to get this right though. No matter where the line gets drawn, free users will complain that they don’t get everything for free.
  - arkasan 3y ago
    I wish this was prominently documented. Most people new to Sidekiq have no idea that the job will be lost forever if you simply hard kill the worker. I have seen a couple of instances where the team had Sidekiq Pro, but they had not enabled reliable fetch because they were unaware of this problem
  - tebbers 3y ago
    Exactly why we refuse to use Sidekiq. “Hey, you have to pay to guarantee your jobs won’t just vanish”. No thanks.
  - mperham 3y ago
    The free version acts exactly like Resque, the previous market leader in Ruby background jobs. If it was good enough reliability for GitHub and Shopify to use for years, it was good enough for Sidekiq OSS too. Here's Resque literally using `lpop` which is destructive and will lose jobs. https://github.com/resque/resque/blob/7623b8dfbdd0a07eb04b19fb25b16a8d6f087f9a/lib/resque/data_store.rb#L113 https://github.com/resque/resque/blob/7623b8dfbdd0a07eb04b19...
    - phamilton 3y ago
      > If it was good enough reliability for GitHub and Shopify to use for years, it was good enough for Sidekiq OSS too. Great point, and thanks for chiming in. I wonder if containerization has made this more painful (due to cgroups and OOMs). The comments here are basically some people saying it's never been a problem for them and some people saying they encounter it a lot (in containerized environments) and have had to add mitigations. Either way, my observation is a lot of people not paying for Sidekiq Pro should. I hope you can agree with that.
the_gastropod 3y ago
To be fair to Mike, Sidekiq is absolutely free. He sells an enterprise version for money, that comes with support.
- 1123581321 3y ago
  And he only started because companies were telling him it would be easier if they could just pay him.
  - bombcar 3y ago
    Anyone who has a free thing, sell something that produces an expendable invoice or receipt. Makes it easy to use company funds to pay for company work. “Buy me a coffee” buttons don’t really cut it.
  - toomanyrichies 3y ago
    This is great. Reminds me of the Derek Sivers post [1], "Don't start a business until people are asking you to." 1. https://sive.rs/asking https://sive.rs/asking
    - mcsniff 3y ago
      Sorry, this is bogus. "If I had asked people what they wanted, they would have said faster horses." or whatever the quote is. There are tons of businesses selling products that are ideas brought to life from scratching their own itch or simply a desire to make something and put it out there. Personally, I am one of those people who started a business based on an idea with no validation before launching it. I built it in its entirety, then went to places and people who I expected would want it and low-and-behold, I am making a living doing it.
      - toomanyrichies 3y ago
        > There are tons of businesses selling products that are ideas brought to life from scratching their own itch or simply a desire to make something and put it out there. I'd bet money that the number of businesses which fail because they boil down to "a solution in search of a problem" is vastly larger than the number of businesses that succeeded despite performing "no validation". That said, "making something" and "starting a business" are two different things. I would challenge you to point out where in the post he argues against making something, especially for the reasons you mention. > I am one of those people who started a business based on an idea with no validation before launching it. I built it in its entirety, then went to places and people who I expected would want it and low-and-behold, I am making a living doing it. Consider the possibility that you're in the minority there, and that you succeeded despite performing no validation. > Sorry, this is bogus. So if it doesn't apply to you personally, or in all cases, then you dismiss it as "bogus", full stop? Are you in the habit of doing this often? Also, for future reference, it's "lo-and-behold", not "low-and-behold". [1] 1. https://en.wiktionary.org/wiki/lo_and_behold https://en.wiktionary.org/wiki/lo_and_behold
ianbutler 3y ago
People need to eat man, doing something for free is a luxury not many possess. We can see how messed up the financial situation is for a lot of Open Source devs and their lives would be better if they were charging money instead of subsisting off grants and donations. I don't even necessarily disagree with your point that without being free those things wouldn't have taken off but we need to find a way to strike a balance in the developer community. Sidekiq having a free version and an enterprise version walks an okay middle line imo.
- wyldberry 3y ago
  I personally try to spend money or use ad supported anything that is open-source just to help someone else eat. It really hit home for me about five years ago when the author of a WoW addon[0] couldn't develop anymore because of his financial and life situation. So many communities across the web rely on people putting in their spare hours for free just to enjoy things. Whether it's spreadsheets in Eve, Addons and Weak Auras in WoW, forum analysis posts, or whatever goes on in the depths of pvpoke, so much free labor underpins massive parts of the world today. I would love something that I could donate x money to per month and then based on usage, have it dole out to all the content providers with perhaps a minimum per month. It just seems daunting to do that as a) not a crypto scheme and b.) across all the various creator landscapes. [0]https://www.polygon.com/2018/9/25/17901552/world-of-warcraft-deadly-boss-mods-patreon https://www.polygon.com/2018/9/25/17901552/world-of-warcraft...
bdcravens 3y ago
Not by Matz, but there are paid versions of Ruby out there.
ch4s3 3y ago
This is a very bad take. From an OSS perspective languages can attract large communities of contributors and corporate sponsors because of their broad appeal and utility. Specialized libraries will have more trouble doing both and may need alternate models to sustain themselves. From a business perspective, Mike offers not only a free version but the paid enterprise version comes with support from Mike and his team, which is something you can't get from a language owner unless you outright hire them or they run a consultancy.
mattgreenrocks 3y ago
Hilarious take given the fact that there are plenty of orgs making an order of magnitude more than Mike is that rely on Ruby.

shubhamjain 3y ago

This is one of the best businesses I have heard of. Even 6-7 years back, Sidekiq was grossing $80k/month [1]. I imagine it must be much more now. No servers to maintain, no employees, minimal support work and Mike has the complete freedom to work on updates whenever he prefers. Almost entire revenue minus payment processing costs must be profit. Imagine making $1M / yr working only 10-20hrs per week (Don't wish to presume here, but that's my estimate of the man-hours it costs to run). > I still have 0 employees and don't plan to hire. I tune my business processes to run as lean as possible: Most of my customers are on credit card so their payments are automatic. The gem servers take about one day of maintenance per year. I can't really outsource much of my support work because it is so technical and specialized. > My gem server is a $6 droplet on DigitalOcean. Because gems are just static files, that little droplet can handle millions of requests per day with just Apache. Oh, and I run three servers in parallel for failover purposes for a grand total of $18/mo. I wonder what makes Sidekiq special? I don't imagine any equivalent product (background job processing) in the other languages is raking in that much, if making any revenue in the first place. [1]: https://www.indiehackers.com/product/sidekiq https://www.indiehackers.com/product/sidekiq

iudqnolq 3y ago
Oban is a roughly equivalent elixir library. They claim to have hundreds of customers and their cheapest plan is $500/yr, so maybe they make an order of magnitude less? Considering the relative language popularity that seems pretty good.
sparker72678 3y ago
Sidekiq is special because it's crazy performant, and it _works_. Mike has built a system that's rock solid, and he doesn't twiddle around with it willy nilly. He keeps it super stable. The people paying for it (I'm one of those people) are paying for the reliability, in addition to the features. Mike's also managed to find a real sweet-spot with the features he holds out for Pro/Enterprise. The stratification is just right to nudge you up, but it always feels like you're getting a good deal when you pay.
sorentwo 3y ago
> I wonder what makes Sidekiq special? Time, know-how, and long term dedication are the key ingredients from my perspective. Maybe he's working 10-20 hours now, but there's support and development every day for the past 11 years. > I don't imagine any equivalent product (background job processing) in the other languages is raking in that much, if making any revenue in the first place. I guarantee, there are background job processors in other languages making revenue.

jack_riminton 3y ago

"I toyed with Sidekiq.js but decided to kill that idea real quick because, as we all know, JavaScript is terrible" This is why I love the ruby community, so much sense :)

rektide 3y ago
From the JS community, this is sadly the level of snark we expect. It doesn't have to be like this. Your community tried to be better once. Have you forgotten? https://jasonfleetwoodboldt.com/courses/stepping-up-rails/matz-is-nice-and-so-we-are-nice/ https://jasonfleetwoodboldt.com/courses/stepping-up-rails/ma...
- jack_riminton 3y ago
  The other thing I like about the ruby community is the sense of humour
  - rektide 3y ago
    At others expense tbough? Is this the form of merriment we want? Gl;hf.
- pmarreck 3y ago
  You act as if some percentage less than 100% of Ruby users have extensive experience with JS as well... OK, maybe it's only, say, 96% of Ruby users, or 100% of Rails users at least
  - rektide 3y ago
    See like, this isn't constructive maligning at all, it's more undirected blanket unniceness. It costs nothing to actually put down a reason for being upset, versus having totally generic downputs. Very few things are truly rotten to the core, most bad things have some bad aspects but could be much better if ___. If something is rotten to the core the central articles of faith for why that's so should be declared. Plenty of Ruby folks have gone on to do great friendly Rails ish JS works. Ember.js is very batteries included, tries to show that yes all the potential is there to pave a nice well integrated happy-path system. To assume misery seems unreasonable; many have done fine. I think we have a real obligation to do better to ourselves & one another than to foster shallow prejudices. Trump in his April 11th Tucker Carlson interview was asked why he thought Dems weren't worried about nuclear weapons. "That's because they don't understand life. That's because they don't understand what it is that you have to understand." This is an irrefutable claim. There's no point to start discussing here, no possibility that the other side might change or have some nuances. Let's not do this. Let's rise to higher places where we take real appraising concerned looks at things, rather than just dismiss stuff out of hand. We don't even have to be nice, but let's at least strive to be somewhat useful.

aqme28 3y ago

Love Sidekiq and loved how he runs it. Really feels like the right way to make money on OSS. Enterprise never crippled the free version.

adrianthedev 3y ago

Mike is an inspiration to me and my project!

ezekg 3y ago
Same. He's always been a role model for my business as well, especially when I was just starting out. My business uses Sidekiq every day. It really is a great piece of software. He's also one of the reasons I think open source software can be highly profitable if people just figure out a business model that works.

pmarreck 3y ago

Spoke with Mike at a couple of Railsconfs a few years ago (before I got into Elixir). Really great guy.

fidrelity 3y ago

Mike Perham is an inspiration to me both in terms of Ruby/tech as well as business. I've been using sidekiq now for almost a decade and had the pleasure to meet Mike at a Ruby conf many years ago.

sandstrom 3y ago

Sidekiq Pro is great, we're paying for it! 10k a year I think. But for people who are interested in alternatives, I'd also suggest Good Job (runs on Postgresql). https://github.com/bensheldon/good_job https://github.com/bensheldon/good_job

sickcodebruh 3y ago

Sidekiq is one of my favorite things about using Ruby, I include it in my short list every time a "Why use Ruby?" question gets asked. I miss it desperately when I work in other languages. I plan on giving Faktory a shot ASAP. Mike has always struck me as a great guy, really the kind of person who you're happy to know is building one of your favorite tools. In a world of starving open source contributors and mega corps throwing around weight, his success with Sidekiq stands out. It makes me very happy to see.

andris9 3y ago

Mike Perham's Sidekiq was what I modeled my own business, EmailEngine, after. I took my decade's worth of expertise in email protocol implementations (I also run the open-source Nodemailer library) and built an email gateway app my customers can download and run on their machines or servers. Just like Sidekiq Pro, the customers need an active subscription to run it, but the app runs on their premises, and they maintain it. I only run the key generation and validation server, which costs me around $50 per month, including all the backups and so on. I quit my job to go full in with EmailEngine exactly one year ago, when MRR for EmailEngine was $500, now it is $3,900 and steadily growing. I do work full time on it, but the result is async - the work I put into EmailEngine today brings me income sometime later, and the recurring income I receive today is unrelated to any effort I put into the company right now.

mperham 3y ago
Congratulations, that’s great. I’m always happy to hear of more successful indie developers.
- andris9 3y ago
  Thank you for your inspiration!
albertgoeswoof 3y ago
I was just about to post emailengine here, the business model is so similar to sidekiq. Any other great ideas?
- andris9 3y ago
  Yeah, it is so similar because I blatantly copied it. I had the old Changelog podcast episode with Mike Perham pretty much on repeat when I tried to figure out how exatly to set up my business.
alberth 3y ago
Unsolicited feedback, always link to your business in your posts. I googled “emailengine” and it’s wasn’t super obvious which was your business since so much paid ads exist for that search.

JustLurking2022 3y ago

It's always interesting to find out about these niche subcultures. Having worked with batch schedulers of all varieties for a couple decades, I've never run across Sidekiq but, then again, I didn't realize anyone had actually built anything in Ruby in the past decade.

joshmn 3y ago

I've said this to Mike and I'll say it again: I hope he never gets a job.

everybodyknows 3y ago

Naive question: How do you deter thieves who would pose as legit customers, but instead redistribute the library?

atonse 3y ago
I know tons of other companies will share my view that we would never stake anything remotely important on a stolen library to save a minuscule amount of money. It would be absolutely idiotic. Instead, we shell over the negligible monthly fees, spend no time thinking about it, and move on to solve real problems. There are enough of us that it probably simply doesn’t matter.

thaumasiotes 3y ago

> I was Director of Technical Operations at The Clymb, which was outdoor-focused e-commerce vendor (think REI without the stores). I had a team of four and we were responsible for site devops and infrastructure. This was perfect because they were my alpha customer and I was in charge of the tech stack. The Clymb switched to Sidekiq immediately, of course, and every Sidekiq Pro feature I wrote was running in production before it was released. This makes me curious about either (a) conflict-of-interest rules at The Clymb that may or may not have governed the DTO directing the company to use his own separately-owned (commercial?) software. (b) negotiating private ownership of software written for The Clymb. One of those must have been relevant?

mperham 3y ago
I started work on Sidekiq right before I joined so I made sure my employment contract had a IP carve out for anything related to Sidekiq. I never charged The Clymb to use the software. Previously they had been using delayed_job and were suffering badly from scale issues due to the use of MySQL as their queue store.

polishdude20 3y ago

Can someone explain what Sidekiq actually does?