5 ms·

"...8-node DC1.8XL Amazon Redshift cluster for the tests." Well, yeah. That's 28,108.80 a month if you're running queries on demand and don't want a delay/coor

by Cidan 10y ago

"...8-node DC1.8XL Amazon Redshift cluster for the tests." Well, yeah. That's 28,108.80 a month if you're running queries on demand and don't want a delay/coordination in Amazon instance creation/destruction. BQ may or may not be as fast, but it's truly a managed service; I give it our data and it just works. I don't have to worry about instances, boot up time, maintenance, hourly costs, etc. It's silly to focus just on query speed when there's a whole layer of management and cost that comes with it.

beachstartup 10y ago
> 28,108.80 a month maybe i'm completely out of touch, but i'm really wondering who can afford this kind of stuff without $20M series A money in the bank. and i'm also wondering what they're going to do when they run out of that money and have zero database expertise in-house because they outsourced everything to amazon.
- earino 10y ago
  Maybe I'm jaded, but 330k a year or so for the very backbone of your entire real time query analytics infrastructure just doesn't seem unreasonable? This is highly specialized software, tuned and architected for the purpose of running relatively complex analytics queries and aggregates fast enough to allow interactive real time exploratory analysis. I dont think it's insane to believe that a profitable mature data informed business should be expected to spend 1/3 of a million a year for the ability to have all that data live and available without deep latency. I can also agree that this is quite a bit for a startup, but if the unit economics of the startup require that this kind of data be available for interactive exploring, there's going to be a deep challenge down the road in scaling it I think? To your point, if a startup runs through 20 million and their biggest problem is paying AWS bills, then they are probably a failed startup and folks should move on towards whatever next exciting thing is on the horizon.
  - tw04 10y ago
    With 10TB of data, you could also do it a hell of a lot cheaper and faster on premise.
    - trhway 10y ago
      One engineer/DBA to deal with your cluster is those 330k/year.
      - brianwawok 10y ago
        Baby startups don't have a DBA, they have a dev that kinda knows SQL.
      - manigandham 10y ago
        You dont really need a full time DBA to handle 10TB these days.
      - foobarian 10y ago
        Don't those computers cost a lot? Like $10k per blade? Granted it's probably amortized but still... On the other hand the Redshift cluster doesn't run itself, despite what Amazon says. You still need at least a part time DBA guy.
        eitland 10y ago
        Last time I worked with physical servers the rule was to avoid blades and run rackmount until space or termal was an issue. And as long as you were not into GPUs you could get some really decent HP servers at somewhere around $2000 - $7000 depending on your storage needs.
      - beachstartup 10y ago
        there are DBA services that do an excellent job for a fraction of that price. they also do AWS stuff too. full-time DBA's work for either enormous corporations, or database consulting firms. it's very rare you will see some random small-medium company with even one DBA on payroll.
      - 10y ago
- neom 10y ago
  https://en.wikipedia.org/wiki/Fortune_500 https://en.wikipedia.org/wiki/Fortune_500
- StreamBright 10y ago
  I give you just one example: a security vendor collects multiple terabytes per day worth of antivirus data and sells the insight that can be gathered from this to large enterprises to support their security operations. I do not know exactly how much they make out of this but is guess it is in the 200K-500K / year / customer range. Startups on the other side, rarely have the amount of data that could not be hosted on anything for more than 1000-2000 USD/month.
- deleted 10y ago
  [deleted]
dahdum 10y ago
Redshift absolutely dominated the other benchmarks, so while the author chose the largest instance, it's not likely to be necessary to still beat BQ. I don't work for or have any relationship with either, but I do use Redshift.
mooman219 10y ago
edit: My original comment's data was misleading due to conversion error. I edited out the comment to avoid confusion for anyone else who might not spot the error and be misinformed. Thank you for those who pointed it out!
- mej10 10y ago
  > and a DC1.8XL costs $4800/hour to "compute" the data. Is this a typo? A single DC1.8XL is $4.80/hour. 8, like the article, would be $38.40/hour.
  - mooman219 10y ago
    As stated, on demand pricing was used. Both products have special pricing at higher usages and flat-pricing tiers. It would start to get compilicated to compare them at that point. Numbers on https://aws.amazon.com/redshift/pricing/ https://aws.amazon.com/redshift/pricing/ used. dc1.8xlarge - $4.800 per Hour
    - deleted 10y ago
      [deleted]
    - zerocrates 10y ago
      > $4.800 per Hour That's four point eight dollars, not four thousand eight hundred.
      - mooman219 10y ago
        Fixed! Used to seeing 4.800 being 4800 apologies!
        zerocrates 10y ago
        I figured. It's easy to forget that there's really no international/intercultural agreement on number punctuation, particularly combined with a service that wants to price down to tenths of cents.
StreamBright 10y ago
> That's 28,108.80 a month That is right. We had ~100K / month with on prem Hadoop (energy, networking, OPS, enterprise support, etc. included) that could have been moved to Redshift to save money. However, when we did the comparison a different cloud based solution came out as the winer for our DWH needs. The point is, showing a single number without showing how much it would be with other solutions is pointless. There are several companies that can easily afford a ~29K/month DWH in the cloud.
paulasmuth 10y ago
>> Well, yeah. That's 28,108.80 a month If you can't (or don't want to!) afford that kind of money for data analytics, please consider giving the FOSS alternative EventQL [0] a try some time. It's super simple to set up and tries to be efficient on commodity hardware, so you can run large clusters (>100TB scale) for a couple hundred dollars a month. [0] https://eventql.io/ https://eventql.io/ DISC: I'm one of the EventQL authors
- mej10 10y ago
  How fast are group bys? Like say I have 120 billion rows, 25 not-sparse columns, and want to group by between 2-20 columns (5 of which are varchar), aggregating the other 5 columns? What kind of hardware would I need to do that interactively? Or consistently sub-10 second, with 100s of queries per minute. I have built a thing on Redshift that can do some of this, but it has been new territory for me and I am not sure I've done it "right". Constantly looking for alternatives.
  - Jweb_Guru 10y ago
    What you're asking for is just really difficult. Even with a fairly good compression ratio (e.g. 57 bits / tuple, which I chose at random but is similar to what's achievable on real data according to http://i.stanford.edu/~adityagp/courses/cs598/papers/constant.pdf http://i.stanford.edu/~adityagp/courses/cs598/papers/constan...) you're talking 83.4 GB / s of read memory bandwidth just to read the tuples at all within 10 seconds for a single query. The state of the art in terms of what you're likely to be able to actually buy (IBM Power8 machines) can do about 91.5 GB / s. in Stream Triad benchmarks. That's still nowhere close to supporting 100s of queries a minute (more like 6 or 7) and you haven't even started calculating anything yet (Triad is pretty simplistic). It also assumes you can actually achieve that data rate while keeping everything in memory (which is why GPUs probably won't help much for your use case, despite NVLink; they don't have enough RAM). Your biggest issue with aggregation / group by is going to be the memory bandwidth to/from a hash table to store all the results. Once the hash table no longer fits in L3, its access time will rise dramatically, so your query's performance will depend heavily on how many buckets you need (if you're grouping by 2 columns, maybe not that many; if you're grouping by 20, you'll probably have almost no buckets with more than one entry). Another potential issue is going to be having to look up the values for that column in a hash table if they are highly compressed (since you need to perform aggregation on them, presumably something like sum; if it's count, this doesn't matter). If you can find a total attribute order such that the columns you group by are always to the left of the columns you aggregate by, you can sort the rows to enable efficient delta encoding; you can also then perform aggregation in the same order as your scan, which eliminates the hash table lookup problem for output (not input, though). You can also pre-materialize the results for common subsets. To do this with multiple queries at once, you'd probably need to batch query execution (because of the memory bandwidth issue I alluded to earlier). While the data access requirements would remain similar on read, they'd get worse and worse on write (again, depending on how many buckets you had and how cleverly sorted your data were). Alternately, you could get a bunch of Power8s (or commodity machines, but to maximize bang for your hardware buck you really want stuff with tons of memory bandwidth) and give each of them a slice of the data (but still apply all the above optimizations). The commodity version of this is the Redshift solution. If you went this route, you could also look at specialized solutions like GPUs with NVLink or the KNL Xeon Phis, which have "fast" memory with tons of extra bandwidth which help mitigate the aforementioned hash table / query result access costs. I still haven't talked about how you're supposed to actually get the results back. If you're trying to do ethernet through anything commodity you're going to be very limited in terms of data rate. Even 100 Gbps Infiniband only gets you 12.5 GB/s out, and even if you hook two of those up to each machine you're still only at 25 GB/s out. So either you get even more machines, your clients process the data on the machine, or you have to limit the output somehow... and if you're thinking "we'll sort it!" guess what that's going to be bound by (unless you can store the input data presorted)? Probably memory bandwidth (assuming radix sort)! tl;dr One way or another, you're paying out the nose to satisfy the requirements you just outlined. Also worth noting that you're paying for this read performance on write.
vijayr 10y ago
I give it our data and it just works Could you please talk a bit more about your use cases? what domain, volume of data, how you manage updates etc, if it is possible for you to share?

by Cidan 10y ago

beachstartup 10y ago

> 28,108.80 a month maybe i'm completely out of touch, but i'm really wondering who can afford this kind of stuff without $20M series A money in the bank. and i'm also wondering what they're going to do when they run out of that money and have zero database expertise in-house because they outsourced everything to amazon.

earino 10y ago
Maybe I'm jaded, but 330k a year or so for the very backbone of your entire real time query analytics infrastructure just doesn't seem unreasonable? This is highly specialized software, tuned and architected for the purpose of running relatively complex analytics queries and aggregates fast enough to allow interactive real time exploratory analysis. I dont think it's insane to believe that a profitable mature data informed business should be expected to spend 1/3 of a million a year for the ability to have all that data live and available without deep latency. I can also agree that this is quite a bit for a startup, but if the unit economics of the startup require that this kind of data be available for interactive exploring, there's going to be a deep challenge down the road in scaling it I think? To your point, if a startup runs through 20 million and their biggest problem is paying AWS bills, then they are probably a failed startup and folks should move on towards whatever next exciting thing is on the horizon.
- tw04 10y ago
  With 10TB of data, you could also do it a hell of a lot cheaper and faster on premise.
  - trhway 10y ago
    One engineer/DBA to deal with your cluster is those 330k/year.
    - brianwawok 10y ago
      Baby startups don't have a DBA, they have a dev that kinda knows SQL.
    - manigandham 10y ago
      You dont really need a full time DBA to handle 10TB these days.
    - foobarian 10y ago
      Don't those computers cost a lot? Like $10k per blade? Granted it's probably amortized but still... On the other hand the Redshift cluster doesn't run itself, despite what Amazon says. You still need at least a part time DBA guy.
      - eitland 10y ago
        Last time I worked with physical servers the rule was to avoid blades and run rackmount until space or termal was an issue. And as long as you were not into GPUs you could get some really decent HP servers at somewhere around $2000 - $7000 depending on your storage needs.
    - beachstartup 10y ago
      there are DBA services that do an excellent job for a fraction of that price. they also do AWS stuff too. full-time DBA's work for either enormous corporations, or database consulting firms. it's very rare you will see some random small-medium company with even one DBA on payroll.
    - 10y ago
neom 10y ago
https://en.wikipedia.org/wiki/Fortune_500 https://en.wikipedia.org/wiki/Fortune_500
StreamBright 10y ago
I give you just one example: a security vendor collects multiple terabytes per day worth of antivirus data and sells the insight that can be gathered from this to large enterprises to support their security operations. I do not know exactly how much they make out of this but is guess it is in the 200K-500K / year / customer range. Startups on the other side, rarely have the amount of data that could not be hosted on anything for more than 1000-2000 USD/month.
deleted 10y ago
[deleted]

dahdum 10y ago

Redshift absolutely dominated the other benchmarks, so while the author chose the largest instance, it's not likely to be necessary to still beat BQ. I don't work for or have any relationship with either, but I do use Redshift.

mooman219 10y ago

edit: My original comment's data was misleading due to conversion error. I edited out the comment to avoid confusion for anyone else who might not spot the error and be misinformed. Thank you for those who pointed it out!

mej10 10y ago
> and a DC1.8XL costs $4800/hour to "compute" the data. Is this a typo? A single DC1.8XL is $4.80/hour. 8, like the article, would be $38.40/hour.
- mooman219 10y ago
  As stated, on demand pricing was used. Both products have special pricing at higher usages and flat-pricing tiers. It would start to get compilicated to compare them at that point. Numbers on https://aws.amazon.com/redshift/pricing/ https://aws.amazon.com/redshift/pricing/ used. dc1.8xlarge - $4.800 per Hour
  - deleted 10y ago
    [deleted]
  - zerocrates 10y ago
    > $4.800 per Hour That's four point eight dollars, not four thousand eight hundred.
    - mooman219 10y ago
      Fixed! Used to seeing 4.800 being 4800 apologies!
      - zerocrates 10y ago
        I figured. It's easy to forget that there's really no international/intercultural agreement on number punctuation, particularly combined with a service that wants to price down to tenths of cents.

StreamBright 10y ago

> That's 28,108.80 a month That is right. We had ~100K / month with on prem Hadoop (energy, networking, OPS, enterprise support, etc. included) that could have been moved to Redshift to save money. However, when we did the comparison a different cloud based solution came out as the winer for our DWH needs. The point is, showing a single number without showing how much it would be with other solutions is pointless. There are several companies that can easily afford a ~29K/month DWH in the cloud.

paulasmuth 10y ago

>> Well, yeah. That's 28,108.80 a month If you can't (or don't want to!) afford that kind of money for data analytics, please consider giving the FOSS alternative EventQL [0] a try some time. It's super simple to set up and tries to be efficient on commodity hardware, so you can run large clusters (>100TB scale) for a couple hundred dollars a month. [0] https://eventql.io/ https://eventql.io/ DISC: I'm one of the EventQL authors

mej10 10y ago
How fast are group bys? Like say I have 120 billion rows, 25 not-sparse columns, and want to group by between 2-20 columns (5 of which are varchar), aggregating the other 5 columns? What kind of hardware would I need to do that interactively? Or consistently sub-10 second, with 100s of queries per minute. I have built a thing on Redshift that can do some of this, but it has been new territory for me and I am not sure I've done it "right". Constantly looking for alternatives.
- Jweb_Guru 10y ago
  What you're asking for is just really difficult. Even with a fairly good compression ratio (e.g. 57 bits / tuple, which I chose at random but is similar to what's achievable on real data according to http://i.stanford.edu/~adityagp/courses/cs598/papers/constant.pdf http://i.stanford.edu/~adityagp/courses/cs598/papers/constan...) you're talking 83.4 GB / s of read memory bandwidth just to read the tuples at all within 10 seconds for a single query. The state of the art in terms of what you're likely to be able to actually buy (IBM Power8 machines) can do about 91.5 GB / s. in Stream Triad benchmarks. That's still nowhere close to supporting 100s of queries a minute (more like 6 or 7) and you haven't even started calculating anything yet (Triad is pretty simplistic). It also assumes you can actually achieve that data rate while keeping everything in memory (which is why GPUs probably won't help much for your use case, despite NVLink; they don't have enough RAM). Your biggest issue with aggregation / group by is going to be the memory bandwidth to/from a hash table to store all the results. Once the hash table no longer fits in L3, its access time will rise dramatically, so your query's performance will depend heavily on how many buckets you need (if you're grouping by 2 columns, maybe not that many; if you're grouping by 20, you'll probably have almost no buckets with more than one entry). Another potential issue is going to be having to look up the values for that column in a hash table if they are highly compressed (since you need to perform aggregation on them, presumably something like sum; if it's count, this doesn't matter). If you can find a total attribute order such that the columns you group by are always to the left of the columns you aggregate by, you can sort the rows to enable efficient delta encoding; you can also then perform aggregation in the same order as your scan, which eliminates the hash table lookup problem for output (not input, though). You can also pre-materialize the results for common subsets. To do this with multiple queries at once, you'd probably need to batch query execution (because of the memory bandwidth issue I alluded to earlier). While the data access requirements would remain similar on read, they'd get worse and worse on write (again, depending on how many buckets you had and how cleverly sorted your data were). Alternately, you could get a bunch of Power8s (or commodity machines, but to maximize bang for your hardware buck you really want stuff with tons of memory bandwidth) and give each of them a slice of the data (but still apply all the above optimizations). The commodity version of this is the Redshift solution. If you went this route, you could also look at specialized solutions like GPUs with NVLink or the KNL Xeon Phis, which have "fast" memory with tons of extra bandwidth which help mitigate the aforementioned hash table / query result access costs. I still haven't talked about how you're supposed to actually get the results back. If you're trying to do ethernet through anything commodity you're going to be very limited in terms of data rate. Even 100 Gbps Infiniband only gets you 12.5 GB/s out, and even if you hook two of those up to each machine you're still only at 25 GB/s out. So either you get even more machines, your clients process the data on the machine, or you have to limit the output somehow... and if you're thinking "we'll sort it!" guess what that's going to be bound by (unless you can store the input data presorted)? Probably memory bandwidth (assuming radix sort)! tl;dr One way or another, you're paying out the nose to satisfy the requirements you just outlined. Also worth noting that you're paying for this read performance on write.

vijayr 10y ago

I give it our data and it just works Could you please talk a bit more about your use cases? what domain, volume of data, how you manage updates etc, if it is possible for you to share?