5 ms·

How could you possibly say this is doing it wrong? The only way you could batch requests in the way you describe is if you have 1 (or very small number) compute

How could you possibly say this is doing it wrong? The only way you could batch requests in the way you describe is if you have 1 (or very small number) compute nodes. You would need all those requests to hit same node so you could try and batch. With serverless compute infrastructure (which is what this blog is demonstrating by using lambda) you can have 1 isolated process per request and therefore need a database that can actually handle this kind of load.

vbezhenar 4y ago
Introduce intermediate server which accepts multiple requests and groups them into single batch requests. With enough trickery you can even implement it using postgres wire protocol, I guess, so it'll be transparent.
- cbrewster 4y ago
  Now you need to batch your requests to your intermediate server
  - yamtaddle 4y ago
    No, you don't. You have one per 100 nodes or whatever. Not a single one for all nodes to talk to.
- mjb 4y ago
  But why? Why have an additional component if your database can do it?
  - vbezhenar 4y ago
    twawaaay claimed that similar approach allowed him to improve throughput by factor of 50. Apparently database couldn't do it efficiently in that case.
    - deleted 4y ago
      [deleted]
qaq 4y ago
if you are at load level when you have million lambdas executing concurrently your monthly bill will make even Uncle Sam cry.
- ignoramous 4y ago
  In the case of the blog post at least: It is a 1,000 lambdas making 1,000 queries each.
  - qaq 4y ago
    Well thats very unrealistic workload
    - aarondf 4y ago
      Unrealistic from the Lambda side, yes! But we were just trying to generate connections to the DB, not test the capabilities of Lambda
      - qaq 4y ago
        sure I am not criticizing the methodology of load testing, just pointing out that at extreme high loads lambda is too expensive to use.
twawaaay 4y ago
> With serverless compute infrastructure Here is your problem. You are trying to build a huge application using inadequate technical building blocks. Lambdas are super inefficient in many different ways. It is a good tool but as with every tool you do need to know how to use it. If you try to build heavy compute app in Python and then complain at your electricity bill -- that really is on you. If your database is overloaded with hundreds of thousands of connections from your lambdas, it means it is end of the road for your lambdas. Do not put effort into scaling your database up, put effort into reducing the number of your connections and efficiency of your application.
- aarondf 4y ago
  I think you can start to hit connection limit walls with RDS at several hundred connections, depending on your instance size. Running an even moderately busy app you could hit those pretty quickly. I would hate to have to change my entire infrastructure at such an early stage because the DB was hitting connection limits! Would you ever need a million open connections? Probably not! But you'll likely want more than 500 at some point. And if your entire stack is serverless already, it'd be nice if the DB could handle that relatively low number of connections too.
  - twawaaay 4y ago
    I look at the database connections the following way: how many connections can a database really serve effectively? For a connection to be actively served the database really needs to have a cpu core working on it or waiting for IO from the storage. And I am completely omitting the fact that databases really need a sizeable amount of memory to be able to do things efficiently. Even if you have a server with hundreds of cores your database probably can't be actively working on more than a small multiple of the number of the cores.
    - aarondf 4y ago
      I'm not sure if you've seen the other post, but we can totally handle one million queries per second: https://planetscale.com/blog/one-million-queries-per-second-with-mysql https://planetscale.com/blog/one-million-queries-per-second-.... And previous HN discussion: https://news.ycombinator.com/item?id=32680957 https://news.ycombinator.com/item?id=32680957.

vbezhenar 4y ago

Introduce intermediate server which accepts multiple requests and groups them into single batch requests. With enough trickery you can even implement it using postgres wire protocol, I guess, so it'll be transparent.

cbrewster 4y ago
Now you need to batch your requests to your intermediate server
- yamtaddle 4y ago
  No, you don't. You have one per 100 nodes or whatever. Not a single one for all nodes to talk to.
mjb 4y ago
But why? Why have an additional component if your database can do it?
- vbezhenar 4y ago
  twawaaay claimed that similar approach allowed him to improve throughput by factor of 50. Apparently database couldn't do it efficiently in that case.
  - deleted 4y ago
    [deleted]

qaq 4y ago

if you are at load level when you have million lambdas executing concurrently your monthly bill will make even Uncle Sam cry.

ignoramous 4y ago
In the case of the blog post at least: It is a 1,000 lambdas making 1,000 queries each.
- qaq 4y ago
  Well thats very unrealistic workload
  - aarondf 4y ago
    Unrealistic from the Lambda side, yes! But we were just trying to generate connections to the DB, not test the capabilities of Lambda
    - qaq 4y ago
      sure I am not criticizing the methodology of load testing, just pointing out that at extreme high loads lambda is too expensive to use.

twawaaay 4y ago

> With serverless compute infrastructure Here is your problem. You are trying to build a huge application using inadequate technical building blocks. Lambdas are super inefficient in many different ways. It is a good tool but as with every tool you do need to know how to use it. If you try to build heavy compute app in Python and then complain at your electricity bill -- that really is on you. If your database is overloaded with hundreds of thousands of connections from your lambdas, it means it is end of the road for your lambdas. Do not put effort into scaling your database up, put effort into reducing the number of your connections and efficiency of your application.

aarondf 4y ago
I think you can start to hit connection limit walls with RDS at several hundred connections, depending on your instance size. Running an even moderately busy app you could hit those pretty quickly. I would hate to have to change my entire infrastructure at such an early stage because the DB was hitting connection limits! Would you ever need a million open connections? Probably not! But you'll likely want more than 500 at some point. And if your entire stack is serverless already, it'd be nice if the DB could handle that relatively low number of connections too.
- twawaaay 4y ago
  I look at the database connections the following way: how many connections can a database really serve effectively? For a connection to be actively served the database really needs to have a cpu core working on it or waiting for IO from the storage. And I am completely omitting the fact that databases really need a sizeable amount of memory to be able to do things efficiently. Even if you have a server with hundreds of cores your database probably can't be actively working on more than a small multiple of the number of the cores.
  - aarondf 4y ago
    I'm not sure if you've seen the other post, but we can totally handle one million queries per second: https://planetscale.com/blog/one-million-queries-per-second-with-mysql https://planetscale.com/blog/one-million-queries-per-second-.... And previous HN discussion: https://news.ycombinator.com/item?id=32680957 https://news.ycombinator.com/item?id=32680957.