7 ms·

The PlanetScale serverless driver for JavaScript

144 points by tbarn 4y ago

alexcroox 4y ago
This is huge for me. I went down the path of trying to build an entire API on CF Workers and the biggest stumbling block was no easy access to external relational databases due to a lack of v8 compatible connectors. This was before the D1 announcement of course.
- higgins 4y ago
  did you look into fauandb (https://fauna.com/ https://fauna.com/) ? if so and you used it, what was you experience running fauna in CF workers?
- _ben_ 4y ago
  Does CF workers support TCP yet?
  - tbarn 4y ago
    They still require external connections to be made over HTTP and not other networking protocols. This is part of why this new driver is useful. Before today, it would have been impossible to use PlanetScale directly from a worker without it.
- asadawadia 4y ago
  is it because you couldn't find a DB that you could talk to over http[s]?
theobr 4y ago
HUGE launch so hyped
brundolf 4y ago
Curious what the performance impact is of replacing TCP with HTTP for this kind of thing
- mattrobenolt 4y ago
  Great question! Author of a lot of the infrastructure surrounding this here. I plan on following up with a more technical deep dive on some of these aspects, but it's quite a bit hard to do a 1:1 comparison. While, obviously, HTTP also uses TCP, I'm going to assume you're asking more about the binary MySQL protocol vs HTTP. On the surface, yes, HTTP is going to have more overhead with headers and the other aspects that come with HTTP. Also, in this case, the payload is JSON, both on the request and response, which is going to be slightly more bulky than the payloads over the MySQL protocol. So where things get interesting is the real world performance implications. Networks and CPUs are really fast. HTTP has been extremely scrutinized and JSON as well. While they are admittedly not the most efficient standards, they are used so heavily and parsers are heavily optimized for these protocols. Mixing in modern TLS 1.3 with modern ciphers, which is something you're very unlikely to get with a traditional MySQL client, we can achieve a much faster first connection time. Pairing with modern ciphers that are demanded for TLS 1.3, and the transport itself can be significantly faster than a slower cipher. This isn't the best comparison since typically you're using MySQL without TLS, but when talking to a service provider like us, we require it for obvious reasons. Next, with HTTP, we get to leverage compression. In our case, we can use browser level native compression with gzip, or if using something server side, we support compressions like snappy. Combine something like gzip or snappy with HTTP/2 now, and given that the bulk of a query result is typically the result itself, and not the protocol, compression can make up a decent amount of the difference. Again, I'm being hand wavy since every query pattern and results are going to be wildly different based on your data, so it's not anything fair to say "this is x% more or less efficient." And lastly, with HTTP, especially HTTP/2, we can multiplex connections. So similar to the gains with TLS 1.3, you can do many many concurrent database sessions across 1 actual TCP connection with HTTP/2. And similarly to TLS 1.3, the cost here is in connection timings, and management of a connection pool. You don't need a pool of connections on the client, so your startup times can be reduced. As a stretch goal, HTTP/3 (with QUIC) is in the crosshairs, which should eliminate some more transport cost since it uses UDP rather than TCP. My hunch is all of this combined together, an optimized driver leveraging HTTP/3 might beat out a MySQL client overall. Time will tell though! So there's no simple answer here, everything has tradeoffs. :)
deleted 4y ago
[deleted]
gsanderson 4y ago
This is great! Potentially the missing piece in the world of "serverless": a serverless SQL database accessible over HTTP with no minimum price. I saw AWS Aurora has a Data API but currently that does have a minimum monthly cost. Fauna is possibly the closest, but that appears to involve translating SQL to its FQL. How stable is the beta?
- k__ 4y ago
  But it doesn't seem to habe serverless/pay-as-you-go pricing.
  - samlambert 4y ago
    We do have pay as you go pricing.
    - deleted 4y ago
      [deleted]
    - tobyjsullivan 4y ago
      I believe they meant pay-per-use.
  - joshstrange 4y ago
    That's fair and since my use of PlanetScale is very burst-y I do wish there was a cheaper option for the vast majority of the time when I don't need to scale but at $30/mo it's much cheaper than Aurora Serverless and I'm not aware of many other "serverless" db's, let alone cheaper ones (that are still MySQL).
  - gsanderson 4y ago
    I believe it does. From their pricing page, beyond the included quota it is $1 per billion reads and $1.50 per million writes
- mattrobenolt 4y ago
  Our beta for this is pretty stable. The only real instabilities are around the underlying APIs we use, which is part of why we're not ready to document it just yet. But the underlying tech is exactly the same as we use for handling traditional MySQL connections, so there isn't anything to fear.
joshstrange 4y ago
Very cool and lightweight way to talk to PlanetScale but for now I'll stick with Prisma. Prisma is much heavier (engine weighs in at ~50MB) and that can be a non-starter for serverless in some cases but it works for me on AWS Lambda. The nice thing about PlanetScale is you get nearly unlimited connections (soft limit of like 250K IIRC) so making 1 connection per active lambda isn't a problem at all. I've been using PlanetScale since shortly after they went GA and I've been very happy so far. Cheaper than Aurora Serverless, less hassle, and the branching feature is super cool. Zero-downtime deploys, with rollback support, feel magical.
- mattrobenolt 4y ago
  We didn't talk much about this yet, but the underlying tech for this will help you within Lambda too reduce latency even over normal MySQL. This specifically is targeting environments where a MySQL client isn't able to run.
  - michaellee8 4y ago
    Is the HTTP API technically a wrapper of the vtgate's grpc interface?
    - mattrobenolt 4y ago
      It's a good chunk of it, but is a lot more suitable for public consumption.
  - joshstrange 4y ago
    > This specifically is targeting environments where a MySQL client isn't able to run. Oh, absolutely and I didn't mean to imply it wasn't useful, I was just saying I went down the Prisma path and will be sticking with that. Even so I'm glad this exists for times that I don't need the full weight/power of Prisma but do want to talk to a PlanetScale DB.
    - mattrobenolt 4y ago
tdy721 4y ago
This is really cool, but you missed Deno. I have a hunch this would mix well with Fresh
- mattrobenolt 4y ago
  Netlify uses Deno as it's runtime. :) https://github.com/planetscale/f1-championship-stats/blob/main/examples/netlify/netlify/edge-functions/data.json.ts https://github.com/planetscale/f1-championship-stats/blob/ma... So we work there just fine.
rsweeney21 4y ago
Has anyone tested edge functions + planetscale (or similar) vs edge functions + a database read replica also located at the edge? It seems like you lose most of the benefit of your code running at the edge if your database is still in an AWS region.
- mattrobenolt 4y ago
  :see_no_evil: This is an obvious next step for us.
- athammer 4y ago
  To my knowledge you can use planetscale’s read replicas in different regions which the user is then routed to the closest one? I was going to use fly.io until I realized PlanetScale does this already - or do you mean writes as well? https://planetscale.com/docs/concepts/read-only-regions https://planetscale.com/docs/concepts/read-only-regions
tmikaeld 4y ago
While this is cool tech and convenient, I worry about run-away costs and what happens if the company goes down? Seems like a LOT of risk.
- mandeepj 4y ago
  > what happens if the company goes down? It's open-source. What's your concern now? https://github.com/planetscale/database-js https://github.com/planetscale/database-js
  - tmikaeld 4y ago
    The server is NOT open source.
asadawadia 4y ago
do people want a SQL DB over HTTP?
- mattrobenolt 4y ago
  Yeah.
- astrobe_ 4y ago
  It seems to me that it is the whole story about REST APIs and GraphQL. But I'm no player, just watching from the bench.
mandeepj 4y ago
Please forgive my ignorance. Is it right if I say the password seems to be available on the client side? import { connect } from '@planetscale/database' const config = { host: '<host>', username: '<user>', password: '<password>' } https://github.com/planetscale/database-js https://github.com/planetscale/database-js
- mattrobenolt 4y ago
  Correct. I would not recommend doing that. This is analogous to a database driver. It's raw access to the database.
- thundergolfer 4y ago
  They say in the post to use environment variables when deploying. I think they’re talking about server-side JS, not JS running in an end-user’s browser.
- jakear 4y ago
  The intended runtime platform is serverless/edge compute, not client side. Think Cloudflare Workers, Vercel Edge Functions, AWS Lambda, etc. The idea is these runtimes typically only allow HTTP outbound requests, not whatever protocol is typically used to connect to a planetscale db.
jackblemming 4y ago
Please consider using a custom string interpolation function to sanitize SQL, not an additional array of parameters. See styled components for inspiration.

144 points by tbarn 4y ago

This is huge for me. I went down the path of trying to build an entire API on CF Workers and the biggest stumbling block was no easy access to external relational databases due to a lack of v8 compatible connectors. This was before the D1 announcement of course.

higgins 4y ago
did you look into fauandb (https://fauna.com/ https://fauna.com/) ? if so and you used it, what was you experience running fauna in CF workers?
_ben_ 4y ago
Does CF workers support TCP yet?
- tbarn 4y ago
  They still require external connections to be made over HTTP and not other networking protocols. This is part of why this new driver is useful. Before today, it would have been impossible to use PlanetScale directly from a worker without it.
asadawadia 4y ago
is it because you couldn't find a DB that you could talk to over http[s]?

theobr 4y ago

HUGE launch so hyped

brundolf 4y ago

Curious what the performance impact is of replacing TCP with HTTP for this kind of thing

mattrobenolt 4y ago
Great question! Author of a lot of the infrastructure surrounding this here. I plan on following up with a more technical deep dive on some of these aspects, but it's quite a bit hard to do a 1:1 comparison. While, obviously, HTTP also uses TCP, I'm going to assume you're asking more about the binary MySQL protocol vs HTTP. On the surface, yes, HTTP is going to have more overhead with headers and the other aspects that come with HTTP. Also, in this case, the payload is JSON, both on the request and response, which is going to be slightly more bulky than the payloads over the MySQL protocol. So where things get interesting is the real world performance implications. Networks and CPUs are really fast. HTTP has been extremely scrutinized and JSON as well. While they are admittedly not the most efficient standards, they are used so heavily and parsers are heavily optimized for these protocols. Mixing in modern TLS 1.3 with modern ciphers, which is something you're very unlikely to get with a traditional MySQL client, we can achieve a much faster first connection time. Pairing with modern ciphers that are demanded for TLS 1.3, and the transport itself can be significantly faster than a slower cipher. This isn't the best comparison since typically you're using MySQL without TLS, but when talking to a service provider like us, we require it for obvious reasons. Next, with HTTP, we get to leverage compression. In our case, we can use browser level native compression with gzip, or if using something server side, we support compressions like snappy. Combine something like gzip or snappy with HTTP/2 now, and given that the bulk of a query result is typically the result itself, and not the protocol, compression can make up a decent amount of the difference. Again, I'm being hand wavy since every query pattern and results are going to be wildly different based on your data, so it's not anything fair to say "this is x% more or less efficient." And lastly, with HTTP, especially HTTP/2, we can multiplex connections. So similar to the gains with TLS 1.3, you can do many many concurrent database sessions across 1 actual TCP connection with HTTP/2. And similarly to TLS 1.3, the cost here is in connection timings, and management of a connection pool. You don't need a pool of connections on the client, so your startup times can be reduced. As a stretch goal, HTTP/3 (with QUIC) is in the crosshairs, which should eliminate some more transport cost since it uses UDP rather than TCP. My hunch is all of this combined together, an optimized driver leveraging HTTP/3 might beat out a MySQL client overall. Time will tell though! So there's no simple answer here, everything has tradeoffs. :)

deleted 4y ago

[deleted]

gsanderson 4y ago

This is great! Potentially the missing piece in the world of "serverless": a serverless SQL database accessible over HTTP with no minimum price. I saw AWS Aurora has a Data API but currently that does have a minimum monthly cost. Fauna is possibly the closest, but that appears to involve translating SQL to its FQL. How stable is the beta?

k__ 4y ago
But it doesn't seem to habe serverless/pay-as-you-go pricing.
- samlambert 4y ago
  We do have pay as you go pricing.
  - deleted 4y ago
    [deleted]
  - tobyjsullivan 4y ago
    I believe they meant pay-per-use.
- joshstrange 4y ago
  That's fair and since my use of PlanetScale is very burst-y I do wish there was a cheaper option for the vast majority of the time when I don't need to scale but at $30/mo it's much cheaper than Aurora Serverless and I'm not aware of many other "serverless" db's, let alone cheaper ones (that are still MySQL).
- gsanderson 4y ago
  I believe it does. From their pricing page, beyond the included quota it is $1 per billion reads and $1.50 per million writes
mattrobenolt 4y ago
Our beta for this is pretty stable. The only real instabilities are around the underlying APIs we use, which is part of why we're not ready to document it just yet. But the underlying tech is exactly the same as we use for handling traditional MySQL connections, so there isn't anything to fear.

joshstrange 4y ago

Very cool and lightweight way to talk to PlanetScale but for now I'll stick with Prisma. Prisma is much heavier (engine weighs in at ~50MB) and that can be a non-starter for serverless in some cases but it works for me on AWS Lambda. The nice thing about PlanetScale is you get nearly unlimited connections (soft limit of like 250K IIRC) so making 1 connection per active lambda isn't a problem at all. I've been using PlanetScale since shortly after they went GA and I've been very happy so far. Cheaper than Aurora Serverless, less hassle, and the branching feature is super cool. Zero-downtime deploys, with rollback support, feel magical.

mattrobenolt 4y ago
We didn't talk much about this yet, but the underlying tech for this will help you within Lambda too reduce latency even over normal MySQL. This specifically is targeting environments where a MySQL client isn't able to run.
- michaellee8 4y ago
  Is the HTTP API technically a wrapper of the vtgate's grpc interface?
  - mattrobenolt 4y ago
    It's a good chunk of it, but is a lot more suitable for public consumption.
- joshstrange 4y ago
  > This specifically is targeting environments where a MySQL client isn't able to run. Oh, absolutely and I didn't mean to imply it wasn't useful, I was just saying I went down the Prisma path and will be sticking with that. Even so I'm glad this exists for times that I don't need the full weight/power of Prisma but do want to talk to a PlanetScale DB.
  - mattrobenolt 4y ago

tdy721 4y ago

This is really cool, but you missed Deno. I have a hunch this would mix well with Fresh

mattrobenolt 4y ago
Netlify uses Deno as it's runtime. :) https://github.com/planetscale/f1-championship-stats/blob/main/examples/netlify/netlify/edge-functions/data.json.ts https://github.com/planetscale/f1-championship-stats/blob/ma... So we work there just fine.

rsweeney21 4y ago

Has anyone tested edge functions + planetscale (or similar) vs edge functions + a database read replica also located at the edge? It seems like you lose most of the benefit of your code running at the edge if your database is still in an AWS region.

mattrobenolt 4y ago
:see_no_evil: This is an obvious next step for us.
athammer 4y ago
To my knowledge you can use planetscale’s read replicas in different regions which the user is then routed to the closest one? I was going to use fly.io until I realized PlanetScale does this already - or do you mean writes as well? https://planetscale.com/docs/concepts/read-only-regions https://planetscale.com/docs/concepts/read-only-regions

tmikaeld 4y ago

While this is cool tech and convenient, I worry about run-away costs and what happens if the company goes down? Seems like a LOT of risk.

mandeepj 4y ago
> what happens if the company goes down? It's open-source. What's your concern now? https://github.com/planetscale/database-js https://github.com/planetscale/database-js
- tmikaeld 4y ago
  The server is NOT open source.

asadawadia 4y ago

do people want a SQL DB over HTTP?

mattrobenolt 4y ago
Yeah.
astrobe_ 4y ago
It seems to me that it is the whole story about REST APIs and GraphQL. But I'm no player, just watching from the bench.

mandeepj 4y ago

Please forgive my ignorance. Is it right if I say the password seems to be available on the client side? import { connect } from '@planetscale/database' const config = { host: '<host>', username: '<user>', password: '<password>' } https://github.com/planetscale/database-js https://github.com/planetscale/database-js

mattrobenolt 4y ago
Correct. I would not recommend doing that. This is analogous to a database driver. It's raw access to the database.
thundergolfer 4y ago
They say in the post to use environment variables when deploying. I think they’re talking about server-side JS, not JS running in an end-user’s browser.
jakear 4y ago
The intended runtime platform is serverless/edge compute, not client side. Think Cloudflare Workers, Vercel Edge Functions, AWS Lambda, etc. The idea is these runtimes typically only allow HTTP outbound requests, not whatever protocol is typically used to connect to a planetscale db.

jackblemming 4y ago

Please consider using a custom string interpolation function to sanitize SQL, not an additional array of parameters. See styled components for inspiration.