goodwanghan

5 ms·

1. 1.▲
   by goodwanghan 3y ago

   I guess you want to say MPI and C++ are better for distributed computation?
2. 2.▲
   by goodwanghan 3y ago

   Yes we do. We try to leverage Arrow as much as we can. And different from many other computing frameworks, Fugue doesn't invent new data types, it just uses Arrow data types.
3. 3.▲
   by goodwanghan 3y ago

   Yes, we plan to integrate with Snowflake and Snowpark.
4. 4.▲
   by goodwanghan 3y ago

   Hello whinvik, I totally agree the debugging experience of Spark is suboptimal. But why? I don't think the main reason is too much 'magic', I think it is because of the coupling of user's logic and the complicated backen
5. 5.▲
   by goodwanghan 5y ago

   Fugue SQL extends standard SQL so you can use SQL-ish language end to end. For example, let's assume you use duckdb or mysql or other standard sql for data processing, how do you deal with intermediate tables? You may have to frequentl
6. 6.▲
   by goodwanghan 5y ago

   No, we use antlr, we have no dependency on Java.
7. 7.▲
   by goodwanghan 5y ago

   That is very true. Thank you. Fugue SQL is one way, and it also has functional API. They both can be translated into the underlying runtime. You can choose based your preference and real need.
8. 8.▲
   by goodwanghan 5y ago

   Hey, I am the author of Fugue. Fugue is a higher level abstraction compared to Ray. It provides unified and non-invasive interfaces for people to use Spark, Dask and Pandas. Ray/Modin is also on our roadmap. It provides both Python int