qoega

4 ms·

1.
by qoega 1y ago
It is meant for single reader/writer workload so not meant to be used as a service
2.
by qoega 2y ago
In ClickHouse it is just `INSERT INTO t FROM INFILE 'data.csv.gz'`. Any supported format, any encryption, autodetected from file name and sample data piece to get column types, delimeters etc. Separate tools to convert CSV are not
3.
by qoega 3y ago
Are you guy comparing 16vCPU/32GB vs 8vCPU/32GB and say yours is only 1.6 faster?
4.
by qoega 3y ago
Now you rarely use basic MapReduce primitives, you have another layer of abstraction that can run on infrastructure that was running MR jobs before. This infrastructure allows to efficiently allocate some compute resources for "long&qu
5.
by qoega 3y ago
You can even migrate your zookeeper to ClickHouse keeper. It requires small downtime, but you will have all your zookeeper data inside and your clients will just work when your keeper will be back
6.
by qoega 3y ago
Did not expect to see issue I created
7.
by qoega 3y ago
Open-source ClickHouse also allows both real-time and large historical data.
8.
by qoega 3y ago
I think atwong just promotes his product https://news.ycombinator.com/threads?id=atwong
9.
by qoega 3y ago
It is nice to have an image of expected dashboard in readme.
10.
by qoega 3y ago
Passion and experience
11.
by qoega 4y ago
>It would also be good to see GEOS, GDAL and PROJ integrated into ClickHouse. You can just add feature requests like this to ClickHouse issue tracker. https://github.com/ClickHouse/ClickHouse/issues/45129
12.
by qoega 4y ago
There is no need to use clickhouse-local for 'local' analysis. It just the same ClickHouse, but it will not store any local data. Just use it as ETL tool. Something like this will work: INSERT INTO FUNCTION s3(' https:/&
13.
by qoega 4y ago
I don't know why exactly you need BigQuery in your scenario. If I wanted to have Postgres->ClickHouse it would take single INSERT SELECT from ClickHouse. If you need just CSV result dumped to GCS you can use clickhouse-local mode th
14.
by qoega 4y ago
Sure you can read from S3. You can define table that will look at your files like this https://clickhouse.com/docs/en/integrations/s3/s3-table-engi... Or just use s3 table functions https://cl
15.
by qoega 4y ago
Can you provide SingleStore result for TPC-H and TPC-DS? I can't find it. Why? [1] https://www.tpc.org/tpcds/results/tpcds_results5.asp?orderby... [2] https://www.tpc.org/tpch/results&#x2
16.
by qoega 4y ago
I think optimizations that you focus on for self-hosted ClickHouse are the same as for Cloud. In self-hosted it helps to improve your throughput/capacity with fixed allocated resources. In cloud it directly affects cost. For those comp
17.
by qoega 4y ago
License states the following. All other modifications are not standardized and you can't just compare systems. Otherwise there would be another standardized benchmark in the list you propose to run and publish. >c. Public Disclosure
18.
by qoega 4y ago
I think ClickHouse Cloud Beta addressed exactly this concern. It separates storage and compute and deals with scaling. There is no sharding so you don't need to deal with scaling.
19.
by qoega 4y ago
You can't post results of TPC benchmarks without official audit. So it complicates posting results. You can't find common names that are usually compared with ClickHouse there [1]. So open standardized ClickBench tries to encourag
20.
by qoega 4y ago
I may assure you that 1WU is not 1 part. Not even close. You can check it using trial credits with your data. For example, I just checked that uploading 1.1GB example table(cell_towers with 14 columns) cost me 0.38 write units.
21.
by qoega 4y ago
You can consider that ClickHouse allows both to query a lot of supported external data sources(s3/hdfs/mysql/postgre/...) and to store data in pretty efficient columnar way with compression, indexes and all the bells and
22.
by qoega 4y ago
Many queries that did do not run had aggregations over strings like MIN/MAX. I don't know specifics why many Java based DBMS lack this aggregation functions.
23.
by qoega 4y ago
You can see that setup used is the one provided in a package: single-server/medium. It makes sense to improve setup, but I recommend to provide better configuration by default. I think it is a common courtesy that system should just wo
24.
by qoega 4y ago
There are several existing benchmarks that test query optimisers with a lot of joins. It does not show performance of query engine, but more likely how good is your optimiser was tailored for this queries.
25.
by qoega 4y ago
It is that big only with debug symbols. If you take packaged version of ClickHouse it will be much smaller. Or just strip large binary manually if you already have it.
26.
by qoega 4y ago
Did you miss previous try of TimescaleDB to show that they are better than ClickHouse https://news.ycombinator.com/item?id=28945903 ?
27.
by qoega 4y ago
It is partially true, but this benchmarks force schema. You can't reorganise data for example in wide table or add indices. So it actually does not show you how to use the system to solve this type of problems in a best way possible, b
28.
by qoega 4y ago
Agree. And for a blog post it can even have a story like: "We compared with ClickHouse and we were 10x slower, than we looked at this case and made it 100x faster. Thank you, benchmark and ClickHouse developers that showed us use case
29.
by qoega 4y ago
There is a small list of vendors that do not forbid to run benchmarks with their systems. https://cube.dev/blog/dewitt-clause-or-can-you-benchmark-a-d... That is why there is a small subset of vendors that are being &
30.
by qoega 4y ago
If you will use S3 as a storage for cold data it can be become just S3(or other object storage) cost. And ClickHouse compression rates for Twitter-like data can be 10x or better as you do not store raw data and data in stored in columns. Co

More ›