5 ms·

I suggest https://perspective.finos.org/ https://perspective.finos.org/ for data viz to be built in. We use DuckDB paired with Perspective for client-side BI us

I suggest https://perspective.finos.org/ https://perspective.finos.org/ for data viz to be built in. We use DuckDB paired with Perspective for client-side BI use case, and it's been great.

dleeftink 2y ago
Why Perspective? If going for a D3 wrapper, Plot would offer more flexibility.
- summer_avery 2y ago
  We've built a nice integration for Plot + DuckDB, found here: https://www.duckplot.com/ https://www.duckplot.com/!
mritchie712 2y ago
+1 we're using Perspective in crabwalk[0] (it's like dbt specifically built for duckdb and written in rust) and it's amazing paired with duckdb. Near instant loads for hundreds of thousands of rows and you can keep everything in arrow. 0 - https://github.com/definite-app/crabwalk https://github.com/definite-app/crabwalk
- 0cf8612b2e1e 2y ago
  Where are you using/advocating crabwalk? It does look interesting, but for the local ETL use case, I am missing the pitch on just having my own collection of SQL scripts. Presumably the all-local case needs less complexity. Unless the idea is that this will eventually support more connectors/backends and work as a full dbt replacement?
  - mritchie712 2y ago
    A few features: * Built-in column level lineage (i.e. dump in 20 .sql files and crabwalk automatically figures out lineage) * Visualize the lineage * Clean handling of input / output (e.g. simply specify @config output and you can export results to parquet, csv, etc.) * Tests are not yet implemented, but crabwalk will have built-in support for tests (e.g. uniqueness, joins, etc.) we're using it in our product (https://www.definite.app/ https://www.definite.app/), but only for lineage right now.
texodus 2y ago
Glad you dig it! Check out our pro version to - it also support DuckDB, Python/Pyodide and more! https://prospective.co https://prospective.co
- pepperonipboy 2y ago
  Wow that's really cool! Part of my PhD thesis was about writing stable treemapping algorithms for temporal data. The idea being that you want your treemap cells not to fly around like what I'm seeing in your demo, but to remain more or less in the same position without sacrificing too much on the cells aspect ratios. We've come up with a pretty effective and fast method to do that, check out the paper and a demo down below. Maybe we could even do a collaboration to get this implemented in perspective. https://github.com/EduardoVernier/eduardovernier.github.io/blob/master/pdf/SIBGRAPI18.pdf https://github.com/EduardoVernier/eduardovernier.github.io/b... https://youtu.be/Bf-MRxhNMdI?list=PLy5Y4CMtJ7mKaUBrSZ3YgwrFYBb3xNpHg https://youtu.be/Bf-MRxhNMdI?list=PLy5Y4CMtJ7mKaUBrSZ3YgwrFY... (see the GIT method)
  - texodus 2y ago
    That looks much better, thanks I will read up.
ciupicri 2y ago
The online demo looks great and promising, too bad it's unusable for me. I've tried installing it with conda from conda-forge and no luck. I've tried installing it with pip, the same. I've also cloned the repository from github, tried to build it and failed, but I don't remember the details. Why is some software so difficult to install beats me.
- timkpaine 2y ago
  Have you ever reported an issue? I use perspective heavily on a variety of platforms both conda and pypi without any problems.
  - ciupicri 2y ago
    Not yet, because I wanted to give it one more try while documenting all the steps.
tobilg 2y ago
Have a look at https://sql-workbench.com https://sql-workbench.com eventually, as it's using DuckDB WASM & Perspective to render the query results. Let me know what you think!
- markhalonen 2y ago
  This is actually how I discovered Perspective!
  - tobilg 2y ago
    Hahaha, nice. It's a small world.
deleted 2y ago
[deleted]

I suggest https://perspective.finos.org/ https://perspective.finos.org/ for data viz to be built in. We use DuckDB paired with Perspective for client-side BI use case, and it's been great.

dleeftink 2y ago

Why Perspective? If going for a D3 wrapper, Plot would offer more flexibility.

summer_avery 2y ago
We've built a nice integration for Plot + DuckDB, found here: https://www.duckplot.com/ https://www.duckplot.com/!

mritchie712 2y ago

+1 we're using Perspective in crabwalk[0] (it's like dbt specifically built for duckdb and written in rust) and it's amazing paired with duckdb. Near instant loads for hundreds of thousands of rows and you can keep everything in arrow. 0 - https://github.com/definite-app/crabwalk https://github.com/definite-app/crabwalk

0cf8612b2e1e 2y ago
Where are you using/advocating crabwalk? It does look interesting, but for the local ETL use case, I am missing the pitch on just having my own collection of SQL scripts. Presumably the all-local case needs less complexity. Unless the idea is that this will eventually support more connectors/backends and work as a full dbt replacement?
- mritchie712 2y ago
  A few features: * Built-in column level lineage (i.e. dump in 20 .sql files and crabwalk automatically figures out lineage) * Visualize the lineage * Clean handling of input / output (e.g. simply specify @config output and you can export results to parquet, csv, etc.) * Tests are not yet implemented, but crabwalk will have built-in support for tests (e.g. uniqueness, joins, etc.) we're using it in our product (https://www.definite.app/ https://www.definite.app/), but only for lineage right now.

texodus 2y ago

Glad you dig it! Check out our pro version to - it also support DuckDB, Python/Pyodide and more! https://prospective.co https://prospective.co

pepperonipboy 2y ago
Wow that's really cool! Part of my PhD thesis was about writing stable treemapping algorithms for temporal data. The idea being that you want your treemap cells not to fly around like what I'm seeing in your demo, but to remain more or less in the same position without sacrificing too much on the cells aspect ratios. We've come up with a pretty effective and fast method to do that, check out the paper and a demo down below. Maybe we could even do a collaboration to get this implemented in perspective. https://github.com/EduardoVernier/eduardovernier.github.io/blob/master/pdf/SIBGRAPI18.pdf https://github.com/EduardoVernier/eduardovernier.github.io/b... https://youtu.be/Bf-MRxhNMdI?list=PLy5Y4CMtJ7mKaUBrSZ3YgwrFYBb3xNpHg https://youtu.be/Bf-MRxhNMdI?list=PLy5Y4CMtJ7mKaUBrSZ3YgwrFY... (see the GIT method)
- texodus 2y ago
  That looks much better, thanks I will read up.

ciupicri 2y ago

The online demo looks great and promising, too bad it's unusable for me. I've tried installing it with conda from conda-forge and no luck. I've tried installing it with pip, the same. I've also cloned the repository from github, tried to build it and failed, but I don't remember the details. Why is some software so difficult to install beats me.

timkpaine 2y ago
Have you ever reported an issue? I use perspective heavily on a variety of platforms both conda and pypi without any problems.
- ciupicri 2y ago
  Not yet, because I wanted to give it one more try while documenting all the steps.

tobilg 2y ago

Have a look at https://sql-workbench.com https://sql-workbench.com eventually, as it's using DuckDB WASM & Perspective to render the query results. Let me know what you think!

markhalonen 2y ago
This is actually how I discovered Perspective!
- tobilg 2y ago
  Hahaha, nice. It's a small world.

deleted 2y ago

[deleted]