rxin

6 ms·

1.
From monolith to Lakebase to LTAP: rethinking the database from storage up(databricks.com)
15 points by rxin 3mo ago|0 comments
2.
by rxin 2y ago
This is coming to Serverless SQL warehouses soon!
3.
From Spark to Databricks: Spark's Origins, Innovations, and What's Next(sudipchakrabarti.substack.com)
4 points by rxin 3y ago|0 comments
4.
Best-in-Class Open Source Generative Models(databricks.com)
3 points by rxin 3y ago|0 comments
5.
by rxin 3y ago
(OP / Databricks cofounder here) I'm not sure what happened but I just sent an email out internally to ask people not to do this. The team might have gotten overly excited by this because they were all part of the creation of the
6.
by rxin 3y ago
(Databricks cofounder here) All datasets are biased, including this specific one. However, we believe it's still very valuable to open source, for a few reasons: - This dataset is primarily used to train instruction reasoning, not for
7.
Free Dolly: First truly open instruction-tuned LLM(databricks.com)
181 points by rxin 3y ago|49 comments
8.
by rxin 4y ago
Databricks founder here. Doesn’t seem like DM work on HN anymore. Do you mind shooting me an email? Would love to follow up and understand the issues. rxin@databricks.com
9.
by rxin 5y ago
You should try Databricks, especially the new Photon engine powering Spark. In general more performant than Snowflake in SQL and a lot more flexible. (There are some cases in which Databricks would be slower but the perf is improving rapidl
10.
by rxin 5y ago
Isn’t that what the official TPC does?
11.
Databricks response to Snowflake's accusation of lacking integrity(databricks.com)
217 points by rxin 5y ago|156 comments
12.
by rxin 5y ago
Totally. Simplicity is critical. That’s why we built Databricks SQL not based on Spark. As a matter of fact, we took the extreme approach of not allowing customers (or ourselves) to set any of the known knobs. We want to force ourselves to
13.
by rxin 5y ago
Geometric mean is commonly used in benchmarks when the workloads consists of queries that have large (often orders of magnitude) differences in runtime. Consider 4 queries. Two run for 1sec, and the other two 1000sec. If we look at arithmet
14.
by rxin 5y ago
Ah ok. Wasn't clear. I think some repro scripts will be available soon.
15.
by rxin 5y ago
Check my reply, Leo.
16.
by rxin 5y ago
Exactly. Not sure about Netflix special, but there are experts that have dedicated their professional careers to creating fair benchmarks. Snowflake should just participate in the official TPC benchmark. Disclaimer: Databricks cofounder who
17.
by rxin 5y ago
There's an official TPC process to audit and review the benchmark process. This debate can be easiest settled by everybody participating in the official benchmark, like we (Databricks) did. The official review process is significantly
18.
by rxin 5y ago
Co-author of the paper here. I don't think your argument holds here at all. It's a common misconception to think high performance would require tight coupling of storage and query processing. "Think columnar storage, high com
19.
by rxin 5y ago
We have taken a very different approach with the Unity Catalog. It is designed opposite to the "cluster-centric" access control model, and will be user and role centric. Disclosure: I'm a Databricks cofounder and have contrib
20.
The Launch of Databricks on Google Cloud(databricks.com)
3 points by rxin 6y ago|0 comments
21.
by rxin 7y ago
You should take a look at Databricks, in all three dimensions.
22.
China’s All-Out War on the Coronavirus(nytimes.com)
5 points by rxin 7y ago|1 comments
23.
by rxin 7y ago
Disclaimer: I designed the system that won the 2014 GraySort based on Apache Spark, and the same system was extended by a different team to win the 2016 record in cloud sort. "Merge sort is the only algorithm that can effectively be pa
24.
by rxin 7y ago
What's the reason you need to abandon scikit-learn? You can run scikit-learn on Databricks, and many of our customers do. Disclaimer: Databricks cofounder.
25.
Delta Lake Now Hosted by the Linux Foundation(databricks.com)
3 points by rxin 7y ago|0 comments
26.
Uncovering performance regressions in the TCP SACKs vulnerability fixes(databricks.com)
32 points by rxin 7y ago|3 comments
27.
by rxin 7y ago
We posted more details just now. Will share more next week too.
28.
by rxin 7y ago
More coming next week. We believe we got to the bottom of this last night.
29.
Network performance regressions from TCP SACK vulnerability fixes(databricks.com)
31 points by rxin 7y ago|5 comments
30.
by rxin 8y ago
This is definitely not the case for China.

More ›