7 ms·

I guess Matsumoto should have charged for Ruby. I'm sure Mike Perham and Derek Kraan would have been glad to pay for that and that the Ruby community would be i

I guess Matsumoto should have charged for Ruby. I'm sure Mike Perham and Derek Kraan would have been glad to pay for that and that the Ruby community would be in a strong and healthy shape.

jorge-d 3y ago
Well Sidekiq is free to use. It's only the pro version that he charges and the free version code is open source. I don't see the problem in having that kind of business model, it still allows the community to thrive and offers entreprises a way to have premium support. Plus it allows him to invest more time in maintaining the free version.
- phamilton 3y ago
  I have no problem paying for the Pro version, but one if its marketing pitches is "enhanced reliability", which is a wild marketing spin on "the free version will lose jobs in fairly common scenarios". In sidekiq without super_fetch (a paid feature), any jobs in progress when a worker crashes are lost forever. If a worker merely encounters an exception the job will be put back on the queue and retried but a crash means the job is lost. Again, no problem paying for Pro, but I would prefer a little more transparency on how big a gap that is.
  - durkie 3y ago
    how often do your workers crash? i rely heavily on sidekiq and don't think I see this very often, if ever.
    - Colex 3y ago
      it’s not uncommon to lose jobs in sidekiq if you heavily rely on it and have a lot of jobs running. If using the free version for mission critical jobs, I usually run that task as a cron job to ensure that it will re-try if the job is lost. I have in the past monitored how many jobs were lost and, although a small percentage, it was still recurring thing.
    - phamilton 3y ago
      We process around 50M sidekiq jobs a day across a few hundred workers on a heavily autoscaled infrastructure. Over the past week there were 2 jobs that would have been lost if not for superfetch. It's not a ton, but it's not zero. And when it comes to data durability the difference between zero and not zero is usually all that matters. Edit for additional color: One of the most common crashes we'll see is OutOfMemory. We run in a containerized environment and if a rogue job uses too much memory (or a deploy drastically changes our memory footprint) the container will be killed. In that scenario, the job is not placed back into the queue. SuperFetch is able to recover them, albeit with really lose guarantees around "when".
      - ZephyrBlu 3y ago
        Let me get this straight, you're complaining about eight 9s of reliability? 50,000,000 * 7 = 350,000,000 2 / 350,000,000 = 0.000000005714286 1 - (2 / 350,000,000) = 0.999999994285714 = 99.999999% > It's not a ton, but it's not zero. And when it comes to data durability the difference between zero and not zero is usually all that matters. If your system isn't resilient to 2 in 350,000,000 jobs failing I think there is something wrong with your system.
        deleted 3y ago
        [deleted]
        phamilton 3y ago
        This isn't about 2 in 350,000,000 jobs failing. It's about 2 jobs disappearing entirely. It's not reliability we're talking about, it's about durability. For reference, S3 has eleven 9s of durability. Every major queuing system solves this problem. RabbitMQ uses unacknowledged messages which are pinned to a tcp connection, so when that connection drops before acknowledging them they get picked up by another worker. SQS uses visibility timeouts, where if the message hasn't been successfully processed within a time frame it's made available to other workers. Sidekiq free edition chooses not to solve it. And that's a fine stance for a free product, but just one I wish was made clearer.
        3y ago
    - kubectl_h 3y ago
      In containerized environments it may happen more often due to OOM kills or if you leverage autoscalers and have long running sidekiq jobs that have a runtime that exceeds your configured grace period for shutting down a container during a downscale and the process is eventually terminated without prejudice. OOM kills are particularly pernicious as they can get into a vicious cycle of retry-killed-retry loops. The individual job causing the OOM isn't that important (we will identify it, log it and noop it), it's the blast radius effect on other sidekiq threads (we use up to 20 threads on some of our workers), so you want to be able to recover and re-run any jobs that are innocent victims of a misbehaving job.
  - aqme28 3y ago
    When we used Sidekiq in production, not only did I never see crashes that lost us jobs, but there are also ways to protect yourself from that. I highly recommend writing your jobs to be idempotent.
    - phamilton 3y ago
      Idempotence doesn't solve this problem. The jobs are all idempotent. The problem is that jobs will never be retried if a crash occurs. This doesn't happen at a high rate, but it happens more than zero times per week for us. We pay for Sidekiq Pro and have superfetch enabled so we are protected. If we didn't do so we'd need to create some additional infra to detect jobs that were never properly run and re-run them.
      - aqme28 3y ago
        Fair enough about idempotence. I'm still confused about what you're saying though. You're saying that the language of "enhanced reliability" doesn't reflect losing 2 jobs over about 50*7 million (from your other comment)? And that if you didn't pay for the service, you'd have to add some checks to make up for this? That all seems incredibly reasonable to me.
      - ransackdev 3y ago
        Or install an opensource gem[1] that recreates the functionality using the same redis rpoplpush[2] command [1] https://gitlab.com/gitlab-org/ruby/gems/sidekiq-reliable-fetch https://gitlab.com/gitlab-org/ruby/gems/sidekiq-reliable-fet... [2] https://redis.io/commands/rpoplpush/#pattern-reliable-queue https://redis.io/commands/rpoplpush/#pattern-reliable-queue
      - danenania 3y ago
        Crashes are under your control though. They’re not caused by sidekiq. And you could always add your own crash recovery logic, as you say. To me that makes it a reasonable candidate for a pro feature. It’s hard to get this right though. No matter where the line gets drawn, free users will complain that they don’t get everything for free.
  - arkasan 3y ago
    I wish this was prominently documented. Most people new to Sidekiq have no idea that the job will be lost forever if you simply hard kill the worker. I have seen a couple of instances where the team had Sidekiq Pro, but they had not enabled reliable fetch because they were unaware of this problem
  - tebbers 3y ago
    Exactly why we refuse to use Sidekiq. “Hey, you have to pay to guarantee your jobs won’t just vanish”. No thanks.
  - mperham 3y ago
    The free version acts exactly like Resque, the previous market leader in Ruby background jobs. If it was good enough reliability for GitHub and Shopify to use for years, it was good enough for Sidekiq OSS too. Here's Resque literally using `lpop` which is destructive and will lose jobs. https://github.com/resque/resque/blob/7623b8dfbdd0a07eb04b19fb25b16a8d6f087f9a/lib/resque/data_store.rb#L113 https://github.com/resque/resque/blob/7623b8dfbdd0a07eb04b19...
    - phamilton 3y ago
      > If it was good enough reliability for GitHub and Shopify to use for years, it was good enough for Sidekiq OSS too. Great point, and thanks for chiming in. I wonder if containerization has made this more painful (due to cgroups and OOMs). The comments here are basically some people saying it's never been a problem for them and some people saying they encounter it a lot (in containerized environments) and have had to add mitigations. Either way, my observation is a lot of people not paying for Sidekiq Pro should. I hope you can agree with that.
the_gastropod 3y ago
To be fair to Mike, Sidekiq is absolutely free. He sells an enterprise version for money, that comes with support.
- 1123581321 3y ago
  And he only started because companies were telling him it would be easier if they could just pay him.
  - bombcar 3y ago
    Anyone who has a free thing, sell something that produces an expendable invoice or receipt. Makes it easy to use company funds to pay for company work. “Buy me a coffee” buttons don’t really cut it.
  - toomanyrichies 3y ago
    This is great. Reminds me of the Derek Sivers post [1], "Don't start a business until people are asking you to." 1. https://sive.rs/asking https://sive.rs/asking
    - mcsniff 3y ago
      Sorry, this is bogus. "If I had asked people what they wanted, they would have said faster horses." or whatever the quote is. There are tons of businesses selling products that are ideas brought to life from scratching their own itch or simply a desire to make something and put it out there. Personally, I am one of those people who started a business based on an idea with no validation before launching it. I built it in its entirety, then went to places and people who I expected would want it and low-and-behold, I am making a living doing it.
      - toomanyrichies 3y ago
        > There are tons of businesses selling products that are ideas brought to life from scratching their own itch or simply a desire to make something and put it out there. I'd bet money that the number of businesses which fail because they boil down to "a solution in search of a problem" is vastly larger than the number of businesses that succeeded despite performing "no validation". That said, "making something" and "starting a business" are two different things. I would challenge you to point out where in the post he argues against making something, especially for the reasons you mention. > I am one of those people who started a business based on an idea with no validation before launching it. I built it in its entirety, then went to places and people who I expected would want it and low-and-behold, I am making a living doing it. Consider the possibility that you're in the minority there, and that you succeeded despite performing no validation. > Sorry, this is bogus. So if it doesn't apply to you personally, or in all cases, then you dismiss it as "bogus", full stop? Are you in the habit of doing this often? Also, for future reference, it's "lo-and-behold", not "low-and-behold". [1] 1. https://en.wiktionary.org/wiki/lo_and_behold https://en.wiktionary.org/wiki/lo_and_behold
ianbutler 3y ago
People need to eat man, doing something for free is a luxury not many possess. We can see how messed up the financial situation is for a lot of Open Source devs and their lives would be better if they were charging money instead of subsisting off grants and donations. I don't even necessarily disagree with your point that without being free those things wouldn't have taken off but we need to find a way to strike a balance in the developer community. Sidekiq having a free version and an enterprise version walks an okay middle line imo.
- wyldberry 3y ago
  I personally try to spend money or use ad supported anything that is open-source just to help someone else eat. It really hit home for me about five years ago when the author of a WoW addon[0] couldn't develop anymore because of his financial and life situation. So many communities across the web rely on people putting in their spare hours for free just to enjoy things. Whether it's spreadsheets in Eve, Addons and Weak Auras in WoW, forum analysis posts, or whatever goes on in the depths of pvpoke, so much free labor underpins massive parts of the world today. I would love something that I could donate x money to per month and then based on usage, have it dole out to all the content providers with perhaps a minimum per month. It just seems daunting to do that as a) not a crypto scheme and b.) across all the various creator landscapes. [0]https://www.polygon.com/2018/9/25/17901552/world-of-warcraft-deadly-boss-mods-patreon https://www.polygon.com/2018/9/25/17901552/world-of-warcraft...
bdcravens 3y ago
Not by Matz, but there are paid versions of Ruby out there.
ch4s3 3y ago
This is a very bad take. From an OSS perspective languages can attract large communities of contributors and corporate sponsors because of their broad appeal and utility. Specialized libraries will have more trouble doing both and may need alternate models to sustain themselves. From a business perspective, Mike offers not only a free version but the paid enterprise version comes with support from Mike and his team, which is something you can't get from a language owner unless you outright hire them or they run a consultancy.
mattgreenrocks 3y ago
Hilarious take given the fact that there are plenty of orgs making an order of magnitude more than Mike is that rely on Ruby.

I guess Matsumoto should have charged for Ruby. I'm sure Mike Perham and Derek Kraan would have been glad to pay for that and that the Ruby community would be in a strong and healthy shape.

jorge-d 3y ago

Well Sidekiq is free to use. It's only the pro version that he charges and the free version code is open source. I don't see the problem in having that kind of business model, it still allows the community to thrive and offers entreprises a way to have premium support. Plus it allows him to invest more time in maintaining the free version.

phamilton 3y ago
I have no problem paying for the Pro version, but one if its marketing pitches is "enhanced reliability", which is a wild marketing spin on "the free version will lose jobs in fairly common scenarios". In sidekiq without super_fetch (a paid feature), any jobs in progress when a worker crashes are lost forever. If a worker merely encounters an exception the job will be put back on the queue and retried but a crash means the job is lost. Again, no problem paying for Pro, but I would prefer a little more transparency on how big a gap that is.
- durkie 3y ago
  how often do your workers crash? i rely heavily on sidekiq and don't think I see this very often, if ever.
  - Colex 3y ago
    it’s not uncommon to lose jobs in sidekiq if you heavily rely on it and have a lot of jobs running. If using the free version for mission critical jobs, I usually run that task as a cron job to ensure that it will re-try if the job is lost. I have in the past monitored how many jobs were lost and, although a small percentage, it was still recurring thing.
  - phamilton 3y ago
    We process around 50M sidekiq jobs a day across a few hundred workers on a heavily autoscaled infrastructure. Over the past week there were 2 jobs that would have been lost if not for superfetch. It's not a ton, but it's not zero. And when it comes to data durability the difference between zero and not zero is usually all that matters. Edit for additional color: One of the most common crashes we'll see is OutOfMemory. We run in a containerized environment and if a rogue job uses too much memory (or a deploy drastically changes our memory footprint) the container will be killed. In that scenario, the job is not placed back into the queue. SuperFetch is able to recover them, albeit with really lose guarantees around "when".
    - ZephyrBlu 3y ago
      Let me get this straight, you're complaining about eight 9s of reliability? 50,000,000 * 7 = 350,000,000 2 / 350,000,000 = 0.000000005714286 1 - (2 / 350,000,000) = 0.999999994285714 = 99.999999% > It's not a ton, but it's not zero. And when it comes to data durability the difference between zero and not zero is usually all that matters. If your system isn't resilient to 2 in 350,000,000 jobs failing I think there is something wrong with your system.
      - deleted 3y ago
        [deleted]
      - phamilton 3y ago
        This isn't about 2 in 350,000,000 jobs failing. It's about 2 jobs disappearing entirely. It's not reliability we're talking about, it's about durability. For reference, S3 has eleven 9s of durability. Every major queuing system solves this problem. RabbitMQ uses unacknowledged messages which are pinned to a tcp connection, so when that connection drops before acknowledging them they get picked up by another worker. SQS uses visibility timeouts, where if the message hasn't been successfully processed within a time frame it's made available to other workers. Sidekiq free edition chooses not to solve it. And that's a fine stance for a free product, but just one I wish was made clearer.
        3y ago
  - kubectl_h 3y ago
    In containerized environments it may happen more often due to OOM kills or if you leverage autoscalers and have long running sidekiq jobs that have a runtime that exceeds your configured grace period for shutting down a container during a downscale and the process is eventually terminated without prejudice. OOM kills are particularly pernicious as they can get into a vicious cycle of retry-killed-retry loops. The individual job causing the OOM isn't that important (we will identify it, log it and noop it), it's the blast radius effect on other sidekiq threads (we use up to 20 threads on some of our workers), so you want to be able to recover and re-run any jobs that are innocent victims of a misbehaving job.
- aqme28 3y ago
  When we used Sidekiq in production, not only did I never see crashes that lost us jobs, but there are also ways to protect yourself from that. I highly recommend writing your jobs to be idempotent.
  - phamilton 3y ago
    Idempotence doesn't solve this problem. The jobs are all idempotent. The problem is that jobs will never be retried if a crash occurs. This doesn't happen at a high rate, but it happens more than zero times per week for us. We pay for Sidekiq Pro and have superfetch enabled so we are protected. If we didn't do so we'd need to create some additional infra to detect jobs that were never properly run and re-run them.
    - aqme28 3y ago
      Fair enough about idempotence. I'm still confused about what you're saying though. You're saying that the language of "enhanced reliability" doesn't reflect losing 2 jobs over about 50*7 million (from your other comment)? And that if you didn't pay for the service, you'd have to add some checks to make up for this? That all seems incredibly reasonable to me.
    - ransackdev 3y ago
      Or install an opensource gem[1] that recreates the functionality using the same redis rpoplpush[2] command [1] https://gitlab.com/gitlab-org/ruby/gems/sidekiq-reliable-fetch https://gitlab.com/gitlab-org/ruby/gems/sidekiq-reliable-fet... [2] https://redis.io/commands/rpoplpush/#pattern-reliable-queue https://redis.io/commands/rpoplpush/#pattern-reliable-queue
    - danenania 3y ago
      Crashes are under your control though. They’re not caused by sidekiq. And you could always add your own crash recovery logic, as you say. To me that makes it a reasonable candidate for a pro feature. It’s hard to get this right though. No matter where the line gets drawn, free users will complain that they don’t get everything for free.
- arkasan 3y ago
  I wish this was prominently documented. Most people new to Sidekiq have no idea that the job will be lost forever if you simply hard kill the worker. I have seen a couple of instances where the team had Sidekiq Pro, but they had not enabled reliable fetch because they were unaware of this problem
- tebbers 3y ago
  Exactly why we refuse to use Sidekiq. “Hey, you have to pay to guarantee your jobs won’t just vanish”. No thanks.
- mperham 3y ago
  The free version acts exactly like Resque, the previous market leader in Ruby background jobs. If it was good enough reliability for GitHub and Shopify to use for years, it was good enough for Sidekiq OSS too. Here's Resque literally using `lpop` which is destructive and will lose jobs. https://github.com/resque/resque/blob/7623b8dfbdd0a07eb04b19fb25b16a8d6f087f9a/lib/resque/data_store.rb#L113 https://github.com/resque/resque/blob/7623b8dfbdd0a07eb04b19...
  - phamilton 3y ago
    > If it was good enough reliability for GitHub and Shopify to use for years, it was good enough for Sidekiq OSS too. Great point, and thanks for chiming in. I wonder if containerization has made this more painful (due to cgroups and OOMs). The comments here are basically some people saying it's never been a problem for them and some people saying they encounter it a lot (in containerized environments) and have had to add mitigations. Either way, my observation is a lot of people not paying for Sidekiq Pro should. I hope you can agree with that.

the_gastropod 3y ago

To be fair to Mike, Sidekiq is absolutely free. He sells an enterprise version for money, that comes with support.

1123581321 3y ago
And he only started because companies were telling him it would be easier if they could just pay him.
- bombcar 3y ago
  Anyone who has a free thing, sell something that produces an expendable invoice or receipt. Makes it easy to use company funds to pay for company work. “Buy me a coffee” buttons don’t really cut it.
- toomanyrichies 3y ago
  This is great. Reminds me of the Derek Sivers post [1], "Don't start a business until people are asking you to." 1. https://sive.rs/asking https://sive.rs/asking
  - mcsniff 3y ago
    Sorry, this is bogus. "If I had asked people what they wanted, they would have said faster horses." or whatever the quote is. There are tons of businesses selling products that are ideas brought to life from scratching their own itch or simply a desire to make something and put it out there. Personally, I am one of those people who started a business based on an idea with no validation before launching it. I built it in its entirety, then went to places and people who I expected would want it and low-and-behold, I am making a living doing it.
    - toomanyrichies 3y ago
      > There are tons of businesses selling products that are ideas brought to life from scratching their own itch or simply a desire to make something and put it out there. I'd bet money that the number of businesses which fail because they boil down to "a solution in search of a problem" is vastly larger than the number of businesses that succeeded despite performing "no validation". That said, "making something" and "starting a business" are two different things. I would challenge you to point out where in the post he argues against making something, especially for the reasons you mention. > I am one of those people who started a business based on an idea with no validation before launching it. I built it in its entirety, then went to places and people who I expected would want it and low-and-behold, I am making a living doing it. Consider the possibility that you're in the minority there, and that you succeeded despite performing no validation. > Sorry, this is bogus. So if it doesn't apply to you personally, or in all cases, then you dismiss it as "bogus", full stop? Are you in the habit of doing this often? Also, for future reference, it's "lo-and-behold", not "low-and-behold". [1] 1. https://en.wiktionary.org/wiki/lo_and_behold https://en.wiktionary.org/wiki/lo_and_behold

ianbutler 3y ago

People need to eat man, doing something for free is a luxury not many possess. We can see how messed up the financial situation is for a lot of Open Source devs and their lives would be better if they were charging money instead of subsisting off grants and donations. I don't even necessarily disagree with your point that without being free those things wouldn't have taken off but we need to find a way to strike a balance in the developer community. Sidekiq having a free version and an enterprise version walks an okay middle line imo.

wyldberry 3y ago
I personally try to spend money or use ad supported anything that is open-source just to help someone else eat. It really hit home for me about five years ago when the author of a WoW addon[0] couldn't develop anymore because of his financial and life situation. So many communities across the web rely on people putting in their spare hours for free just to enjoy things. Whether it's spreadsheets in Eve, Addons and Weak Auras in WoW, forum analysis posts, or whatever goes on in the depths of pvpoke, so much free labor underpins massive parts of the world today. I would love something that I could donate x money to per month and then based on usage, have it dole out to all the content providers with perhaps a minimum per month. It just seems daunting to do that as a) not a crypto scheme and b.) across all the various creator landscapes. [0]https://www.polygon.com/2018/9/25/17901552/world-of-warcraft-deadly-boss-mods-patreon https://www.polygon.com/2018/9/25/17901552/world-of-warcraft...

bdcravens 3y ago

Not by Matz, but there are paid versions of Ruby out there.

ch4s3 3y ago

This is a very bad take. From an OSS perspective languages can attract large communities of contributors and corporate sponsors because of their broad appeal and utility. Specialized libraries will have more trouble doing both and may need alternate models to sustain themselves. From a business perspective, Mike offers not only a free version but the paid enterprise version comes with support from Mike and his team, which is something you can't get from a language owner unless you outright hire them or they run a consultancy.

mattgreenrocks 3y ago

Hilarious take given the fact that there are plenty of orgs making an order of magnitude more than Mike is that rely on Ruby.