kylebarron

6 ms·

1.
by kylebarron 4mo ago
See recent discussion in [0] [0]: https://news.ycombinator.com/item?id=47482185
2.
by kylebarron 10mo ago
Looks great! Another seemingly extremely similar project released in the last few days: https://github.com/raulcd/datanomy
3.
How to Build High-Speed Rail on the Northeast Corridor(nec.transitcosts.com)
3 points by kylebarron 11mo ago|0 comments
4.
Making performant Python bindings to compiled code(kylebarron.dev)
2 points by kylebarron 1y ago|0 comments
5.
by kylebarron 1y ago
https://github.com/manzt/juv
6.
by kylebarron 1y ago
Agreed, I find this to be a super productive environment, because you get all of vscode's IDE plus the niceties of Jupyter and IPython. I wrote a small vscode extension that builds upon this to automatically infer code blocks via inden
7.
by kylebarron 2y ago
Equal distances to each adjacent neighbor: https://www.uber.com/blog/h3/
8.
Obstore: Efficient streaming object store operations in Python(github.com)
2 points by kylebarron 2y ago|0 comments
9.
by kylebarron 2y ago
This worked for me last month on a Mac M2: https://www.portingkit.com/game/278
10.
Literate Flatbush: Understanding a fast, elegant RTree implementation(kylebarron.dev)
1 points by kylebarron 2y ago|0 comments
11.
by kylebarron 2y ago
https://visgl.github.io/react-map-gl/ is a react interface to MapLibre
12.
by kylebarron 2y ago
I develop Lonboard [0], a Python library for plotting large geospatial data. If you have small data (~max 30,000 coordinates), leaflet-based Python libraries like folium and ipyleaflet can be fine, but because Lonboard uses deck.gl for GPU-
13.
by kylebarron 2y ago
It can read from HTTP urls, but you'd need to manage signing the URLs yourself. On the writing side, it currently writes to an ArrayBuffer, which then you could upload to a server or save on the user's machine.
14.
by kylebarron 2y ago
Arrow JS is just ArrayBuffers underneath. You do want to amortize some operations to avoid unnecessary conversions. I.e. Arrow JS stores strings as UTF-8, but native JS strings are UTF-16 I believe. Arrow is especially powerful across the W
15.
Parquet-WASM: Rust-based WebAssembly bindings to read and write Parquet data(github.com)
179 points by kylebarron 2y ago|14 comments
16.
by kylebarron 2y ago
That's why I'm working on the GeoParquet spec [0]! It gives you both compression-by-default and super fast reads and writes! So it's usually as small as gzipped CSV, if not smaller, while being faster to read and write than G
17.
by kylebarron 2y ago
Your issue is that you're using the default (old) binding to GDAL, based on Fiona [0]. You need to use pyogrio [1], its vectorized counterpart, instead. Make sure you use `engine="pyogrio"` when calling `to_file` [2]. Fiona d
18.
by kylebarron 2y ago
Sorry, this is not true _at all_ for geospatial data. A quick benchmark [0] shows that saving to GeoPackage, FlatGeobuf, and GeoParquet are roughly 10x faster than saving to CSV. Additionally, the CSV is much larger than any other format. [
19.
by kylebarron 3y ago
Is this open? Is there a way to read more about it? Issues/pull requests?
20.
by kylebarron 3y ago
nodejs-polars is node-specific and uses native FFI. polars can be compiled to Wasm but doesn't yet have a js API out of the box. As for the fastest way to serialize data to Pandas data to the browser, you should use Parquet; it's
21.
by kylebarron 3y ago
pmtiles are only the counterpart to the "overviews" part of COG. pmtiles doesn't give you analysis-ready data. For that, look to FlatGeobuf or GeoParquet (once it gets a spatial index in v1.1 of the spec)
22.
by kylebarron 3y ago
10,000 coordinates are certainly not enough to see the difference, but at some scale this would be faster on the GPU. This is implemented in an nvidia geospatial library call cuspatial: https://docs.rapids.ai/api/cuspat
23.
by kylebarron 3y ago
I'm working on bringing GeoRust algorithms [0] (which include boolean operations) to the web via WebAssembly. See this blog post [1] for an intro. There's also separate work on binding GEOS to Wasm [2], but I'm more excited a
24.
by kylebarron 3y ago
I'm surprised this link works and doesn't redirect given https://macwright.com/2022/09/15/hacker-news.html
25.
DuckDB for Swift(duckdb.org)
3 points by kylebarron 3y ago|0 comments
26.
by kylebarron 3y ago
The line visuals at the bottom are not using Mapbox for the data rendering. Rather they're using the open source Kepler.gl [0], (a user-friendly wrapping of the deck.gl library [1]). These can use Mapbox for the underlying basemap, but
27.
by kylebarron 3y ago
DuckDB is not backed by Arrow, but by something similar enough that it can be zero copy to Arrow in many cases https://duckdb.org/2021/12/03/duck-arrow.html
28.
by kylebarron 4y ago
Most of these examples go up to zoom 10 or so because they cover a wide area. It's not prohibitive to increase that zoom in a small area because the total number of tiles will still be low. Also keep in mind that for vector data it&#x2
29.
by kylebarron 4y ago
Maybe due to SIMD?
30.
by kylebarron 4y ago
pandas added a copy-on-write API in 2.0, so under the hood the Arrow columns are still effectively immutable. https://pandas.pydata.org/docs/dev/whatsnew/v2.0.0.html#copy...

More ›