6 ms·

The "How it works" section surprised me - I did not expect to see Websockets there. Other serverless drivers or database APIs (PlanetScale Serverless Driver, A

by Sujan 4y ago

The "How it works" section surprised me - I did not expect to see Websockets there. Other serverless drivers or database APIs (PlanetScale Serverless Driver, AWS RDS Proxy or Prisma Data Proxy) usually use HTTP to replace the stateful TCP connection. The post explains a bit why this might be beneficial (can use existing tools, and use connection state) - but what about the other benefits and use cases of going with HTTP? For example in that architecture the connection pool also moves from the application to the component in between and all that complexity can completely be removed from the app. Especially on serverless platforms, where in theory each request gets its own application with its own pool, that is huge. How does that work here? Or does the pool also move to the "WebSocket-to-TCP proxy" and you just did not highlight it in the post? (The graphic shows a "Connection Pool" but I can not really make sense of what that means.)

gmac 4y ago
Right: at a basic level, using WebSockets lets us change as little as possible from the user perspective. You get an ordinary Postgres session with an ordinary Postgres driver, full control over transactions, and so on. At this point, your serverless function establishes a new Postgres connection on each call. We do pooling on the server side with pgBouncer, which means we can handle lots of simultaneous connections (which is what this approach generates). It's true that this approach doesn't fully optimise for low latencies. But, as Nikita has mentioned elsewhere in this thread, we have a roadmap for bringing latency down in a number of different ways over time.
- Sujan 4y ago
  Ok, so to make that explicit: If I want to do 5 parallel queries on my serverless function I should still have a connection pool size of 5 in my application, which will be fine as PgBouncer ensures there are plenty connections to open and use from the database server side. Correct?
  - gmac 4y ago
    In principle, yes, but as things stand you'll be starting 5 separate TLS connections to the server that way. My feeling is that this would be an unusual way to use serverless functions. Is it something you think you'd do?
    - Sujan 4y ago
      Yes. You only have to open these connections once on the first execution of that function (cold start), any future request that hit this warm function will have 5 open connections and can instantly execute these queries in parallel. No overhead at all to open the connections. What would be the alternative? Only execute the queries in sequence, one by one?
mattashii 4y ago
HTTP does not (easily) allow for using answers of earlier queries in the transaction. E.g. BEGIN; INSERT INTO my_table (...) RETURNING (id); SELECT count(\*) AS my_count FROM my_table; INSERT INTO historical (new_id, value_derived_from_count, now()); COMMIT; is difficult (or potentially impossible) to do transactionally using single HTTP queries. Sure, you can rewrite your queries to use single-statement queries if you're lucky, but for others that may take a lot of work. Keeping a transaction alive over WebSocket allows you to easily have transaction states that last longer than the lifetime of the first request, which allows for transactions of which state is processed in more than one place.
- Sujan 4y ago
  That I understand. But of course for the user of the driver it might be fine if that is 2 or more HTTP requests. I expect that is how PlanetScale does it in their transaction implementation, https://github.com/planetscale/database-js#transactions https://github.com/planetscale/database-js#transactions, and I know that is how Prisma Data Proxy handles it - the transaction is identified with an ID which is returned to the Client and then included in further requests for the same transaction. It's valid tradeoff to make to prefer a persistent connection to keep the overhead for multiple queries in a transaction as low as possible - which seems what Neon has done here.
  - throwdown220 4y ago
    It’s not a trade off. They’ve not been able to do something that is doable and are trying to tell you it is a trade off.
  - mattashii 4y ago
    > the transaction is identified with an ID which is returned to the Client and then included in further requests for the same transaction. Yes, and there's a catch there that people might not notice: HTTP-based queries utilize this ID to identify which active transaction to use, but this is vulnerable to concurrent HTTP requests on the same transaction ID, thus allowing query injection (early COMMIT, SELECT sleep(1000000), etc. by attackers) due to requests being re-routed every time you send the query. Keeping a direct connection (albeit proxied) to the PostgreSQL instance prevents this kind of attack, while also allowing for better state keeping control in systems that don't have access to raw TCP sockets, and reducing per-query overheads.
kelvich 4y ago
Another angle here is compatibility. With our current driver one can use ordinary node-postgres package, as we can substitute TCP-related calls with WebSocket calls during the build time. With that it possible to use all the packages that do require node-postgres like Prisma, Zapatos, etc.
- Sujan 4y ago
  Prisma does not actually use node-postgres, but a Rust PostgreSQL driver. Prisma will not be able to use the Neon serverless driver.
  - kelvich 4y ago
    https://github.com/prisma/prisma/blob/main/packages/client/package.json#L122 https://github.com/prisma/prisma/blob/main/packages/client/p...
    - Sujan 4y ago
      That is only used in tests :) The query engine uses this: https://github.com/prisma/quaint/blob/6532d69b5aec007ad06ac60a331c20af11f293fa/Cargo.toml#L149-L153 https://github.com/prisma/quaint/blob/6532d69b5aec007ad06ac6... (I work at Prisma, could have mentioned that earlier)
      - kelvich 4y ago
        Gotcha. I drew my conclusion based on the mentioned package.json. Now wonder why did you decide to go with rust for query engine? Do you compile it into wasm?
        Sujan 4y ago
        It made sense at the time. We do not only support Node, but also have community Clients in Go, Python or Rust. Right now we are moving more and more parts from a Node-API library or binary engine (the two variants we support until now) over to Wasm modules where it is possible for our Node/TS/JS Client. Socket/TCP connections itself are unfortunately not supported yet, so this will only be partial. And maybe there is also a future where we support Node based databases drives. As this blog post we are commenting on shows, sometimes we have to combine the weirdest things together to achieve our goal.

by Sujan 4y ago

gmac 4y ago

Right: at a basic level, using WebSockets lets us change as little as possible from the user perspective. You get an ordinary Postgres session with an ordinary Postgres driver, full control over transactions, and so on. At this point, your serverless function establishes a new Postgres connection on each call. We do pooling on the server side with pgBouncer, which means we can handle lots of simultaneous connections (which is what this approach generates). It's true that this approach doesn't fully optimise for low latencies. But, as Nikita has mentioned elsewhere in this thread, we have a roadmap for bringing latency down in a number of different ways over time.

Sujan 4y ago
Ok, so to make that explicit: If I want to do 5 parallel queries on my serverless function I should still have a connection pool size of 5 in my application, which will be fine as PgBouncer ensures there are plenty connections to open and use from the database server side. Correct?
- gmac 4y ago
  In principle, yes, but as things stand you'll be starting 5 separate TLS connections to the server that way. My feeling is that this would be an unusual way to use serverless functions. Is it something you think you'd do?
  - Sujan 4y ago
    Yes. You only have to open these connections once on the first execution of that function (cold start), any future request that hit this warm function will have 5 open connections and can instantly execute these queries in parallel. No overhead at all to open the connections. What would be the alternative? Only execute the queries in sequence, one by one?

mattashii 4y ago

HTTP does not (easily) allow for using answers of earlier queries in the transaction. E.g. BEGIN; INSERT INTO my_table (...) RETURNING (id); SELECT count(\*) AS my_count FROM my_table; INSERT INTO historical (new_id, value_derived_from_count, now()); COMMIT; is difficult (or potentially impossible) to do transactionally using single HTTP queries. Sure, you can rewrite your queries to use single-statement queries if you're lucky, but for others that may take a lot of work. Keeping a transaction alive over WebSocket allows you to easily have transaction states that last longer than the lifetime of the first request, which allows for transactions of which state is processed in more than one place.

Sujan 4y ago
That I understand. But of course for the user of the driver it might be fine if that is 2 or more HTTP requests. I expect that is how PlanetScale does it in their transaction implementation, https://github.com/planetscale/database-js#transactions https://github.com/planetscale/database-js#transactions, and I know that is how Prisma Data Proxy handles it - the transaction is identified with an ID which is returned to the Client and then included in further requests for the same transaction. It's valid tradeoff to make to prefer a persistent connection to keep the overhead for multiple queries in a transaction as low as possible - which seems what Neon has done here.
- throwdown220 4y ago
  It’s not a trade off. They’ve not been able to do something that is doable and are trying to tell you it is a trade off.
- mattashii 4y ago
  > the transaction is identified with an ID which is returned to the Client and then included in further requests for the same transaction. Yes, and there's a catch there that people might not notice: HTTP-based queries utilize this ID to identify which active transaction to use, but this is vulnerable to concurrent HTTP requests on the same transaction ID, thus allowing query injection (early COMMIT, SELECT sleep(1000000), etc. by attackers) due to requests being re-routed every time you send the query. Keeping a direct connection (albeit proxied) to the PostgreSQL instance prevents this kind of attack, while also allowing for better state keeping control in systems that don't have access to raw TCP sockets, and reducing per-query overheads.

kelvich 4y ago

Another angle here is compatibility. With our current driver one can use ordinary node-postgres package, as we can substitute TCP-related calls with WebSocket calls during the build time. With that it possible to use all the packages that do require node-postgres like Prisma, Zapatos, etc.

Sujan 4y ago
Prisma does not actually use node-postgres, but a Rust PostgreSQL driver. Prisma will not be able to use the Neon serverless driver.
- kelvich 4y ago
  https://github.com/prisma/prisma/blob/main/packages/client/package.json#L122 https://github.com/prisma/prisma/blob/main/packages/client/p...
  - Sujan 4y ago
    That is only used in tests :) The query engine uses this: https://github.com/prisma/quaint/blob/6532d69b5aec007ad06ac60a331c20af11f293fa/Cargo.toml#L149-L153 https://github.com/prisma/quaint/blob/6532d69b5aec007ad06ac6... (I work at Prisma, could have mentioned that earlier)
    - kelvich 4y ago
      Gotcha. I drew my conclusion based on the mentioned package.json. Now wonder why did you decide to go with rust for query engine? Do you compile it into wasm?
      - Sujan 4y ago
        It made sense at the time. We do not only support Node, but also have community Clients in Go, Python or Rust. Right now we are moving more and more parts from a Node-API library or binary engine (the two variants we support until now) over to Wasm modules where it is possible for our Node/TS/JS Client. Socket/TCP connections itself are unfortunately not supported yet, so this will only be partial. And maybe there is also a future where we support Node based databases drives. As this blog post we are commenting on shows, sometimes we have to combine the weirdest things together to achieve our goal.