sonthonax

5 ms·

1.
Show HN: DemandMap – Memory Mapping S3 into Polars on macOS Without FUSE(github.com)
2 points by sonthonax 2mo ago|1 comments
2.
by sonthonax 4mo ago
The thing that people forget about databases is that they are just really complicated ways of writing to a file. The crux of all these “you only need sqlite” posts isn’t that you need SQLite, it’s that your architecture only ever needed fla
3.
by sonthonax 11mo ago
They’re extremely temperamental vehicles because they’re not fit for purpose. The original Range Rover is an upmarket agricultural vehicle, and used the same platform from 1970 until the 2000s. When you actually use them as intended, for of
4.
by sonthonax 11mo ago
You just set the alembic version yourself in the alembic.version table. It’s not rocket science.
5.
by sonthonax 1y ago
To the contrary, a simple expression language is one of those things that can easily be done in that size.
6.
by sonthonax 1y ago
Yeah, parent comment is deeply unserious. Imagine you're writing trading software, you have an algo go haywire and it machine guns the whole order book, and then you refuse to put a "max order size" outside of the algo to sto
7.
by sonthonax 2y ago
Just not true. Lots of oil companies have a vested interest in renewables now. A lot of the infrastructure for offshore oil is being redeployed for offshore wind.
8.
by sonthonax 2y ago
600 people on a median 200k salary in New York, when including taxes and obligatory benefits is about $150 million dollars a year just in payroll. This is an enormous spend. Unlike a game studio or a software company like Adobe, these peo
9.
by sonthonax 2y ago
I know they have a larger range of services, but are they 20x the breadth of services? Think of what other companies do with 600 tech people and compare what the profitability per employee should be at an tech organisation of such a scale.
10.
by sonthonax 2y ago
They have at least 600 employees, get some perspective on just how much that is. It's least $150 million dollars a year in technology payroll alone. They have more technology workers than some major newspapers have in staff alone. Wh
11.
by sonthonax 2y ago
If it's low for the market why choose to work at the NYT? Being a tech worker isn't like being a steelworker where you're kinda screwed if the plant shuts down. You're fully capable of moving job if you're dissatis
12.
by sonthonax 2y ago
The New York Times is a glorified blogging platform. Not to long ago it was a Wordpress site. I'm fully aware of how jaring it is for the median HN reader to hear this, but maintenance of a news website isn't the kind of skilled
13.
by sonthonax 2y ago
Linux outages and upgrade fuckups happen all the time. The difference is that Linux isn't on a unified upgrade cycle so issues come as a trickle rather than a dulge, so the problems are usually localised to a corporation and don'
14.
by sonthonax 2y ago
Surely you've worked with other traders who get confused by import statements. Whole point of this file is to provide a common interface for common math operations that might be exposed by either plain python or pandas/numpy func
15.
by sonthonax 2y ago
Wait until you learn that they believe they need Kafka. Their engineers are probably bitter they work at a media company and not a FAANG. https://www.confluent.io/en-gb/blog/publishing-apache-kafka-... > The M
16.
by sonthonax 2y ago
Thanks for the thorough response. But firstly: > If your firm has had KDB for ages there's a good chance it's big enough to be signed up to one of the research groups who maintain a set of test-suites they will run over a vendo
17.
by sonthonax 2y ago
What did you do for your backtesting testing?
18.
by sonthonax 2y ago
> Loading your rust code into your existing KDB data lake and periodically updating it will be a significantly smaller lift than rewriting your data lake. This is probably going to be what I do until KDB creaks over. > You sound very
19.
by sonthonax 2y ago
So I’ve actually contributed to this project. My concern with adding custom libraries into KDB, while better than writing duplicative Q code is the maintenance nightmare of keeping them up to date in KDB. It’s still an investment but I need
20.
by sonthonax 2y ago
If it’s partitioned this should be even faster.
21.
by sonthonax 2y ago
> What does "move the code to the data" mean in practice? This grasps at why I’m finding KDB so hard to use. I’ve written a pricing and risk library in Rust. Historical data really needs to be taken processed in Rust rather tha
22.
by sonthonax 2y ago
> Unless there's some significant new data that might change that decision, it is what it is. This is what I’m grasping at. Are the challenges of writing a KDB system for back testing derivatives data which needs to work in tandem w
23.
by sonthonax 2y ago
Is that a pun?
24.
by sonthonax 2y ago
I care about my job, enjoy my work, and I take pride in being able to deliver things, but I don't know how to deliver value for my traders here. I'd really appreciate some perspective from seasoned data-engineers who might have se
25.
Ask HN: Is KDB a sane choice for a datalake in 2024?
40 points by sonthonax 2y ago|40 comments
26.
by sonthonax 2y ago
> Service workers, which require a completely different mode of thinking from the rest of front-end development and have more in common with web server programming than DOM manipulation. This is spoken like a true front end developer who
27.
by sonthonax 3y ago
> Severe developmental disorders or severe cognitive impairments where significant support is required, including but not exclusive to: physical disability, intellectual disability, autistic spectrum disorders, brain injury. I think that
28.
by sonthonax 3y ago
I can see you're already being downvoted because Hacker News has a really juvenile attitude to DRM. No one has the right to YouTube (or free media for that matter).
29.
by sonthonax 3y ago
I've noticed that M series Macs have extremely fast disk drives that the OS uses as swap quite efficiently. I've frequently used all my RAM on my Mac and barely noticed any slowdown when it starts swapping.
30.
by sonthonax 3y ago
Isn't PyO3 just a set of macros and helpers around FFI and ctypes?

More ›