luispedrocoelho

4 ms·

1.
Technical cantilevers (a better term than "technical debt"?)(luispedro.substack.com)
2 points by luispedrocoelho 3y ago|0 comments
2.
A small YouTube channel is still pretty large: Internet numbers are weird(luispedro.substack.com)
1 points by luispedrocoelho 4y ago|0 comments
3.
Grit: A non-existent alternative interface to Git(metarabbit.wordpress.com)
1 points by luispedrocoelho 6y ago|0 comments
4.
Towards Typed Pipelines(metarabbit.wordpress.com)
2 points by luispedrocoelho 7y ago|0 comments
5.
How (Jupyter) Notebooks Should Work(metarabbit.wordpress.com)
2 points by luispedrocoelho 8y ago|0 comments
6.
by luispedrocoelho 8y ago
It's not clear to me if google is allowed to demand that non-google app phones not be called Android or if that would also be considered anti-competitive. If they must allow Android branded phones that do not come with google apps, the
7.
by luispedrocoelho 8y ago
As a matter of law, the EC may be right, but as a matter of user experience, the idea that the Android market suffers from not having enough OEM-installed software on devices is a bit of a stretch. In all android phones/tablets I have
8.
by luispedrocoelho 8y ago
But he did not purchase health insurance, he purchased travel insurance. It seems to me that the whole thing is "I bought travel insurance and I thought I would be getting global health insurance, but it turns out it's really trav
9.
by luispedrocoelho 8y ago
> In reality, most claimants are going to spend much less than 1% of that £5 million limit before they get flown home. It only really applies to people who are at death's door and end up with a long stay in intensive care, or need a
10.
by luispedrocoelho 8y ago
I never claimed it was in bad faith and I don't think it was. Considering incentives and conflicts of interest is perfectly normal and does not mean that someone is operating in bad faith. I just claim that it's reasonable for the
11.
by luispedrocoelho 8y ago
The incentive to say so is pretty clear: if he flies away, not only does the doctor lose a 15,000 costumer, he can also be liable if anything does go wrong. Maybe he was indeed in no shape to fly, but he should have argued it properly in th
12.
by luispedrocoelho 8y ago
To be sure, travel insurance is very often a very good idea. But it's travel insurance not "whole-world, any hospital, health insurance" (which I am sure you can buy, but not at the same price).
13.
by luispedrocoelho 8y ago
From his telling, the company was not very communicative when he contacted them, but, fundamentally, saying he should fly back to the UK for treatment and they'll cover the flight does not seem unreasonable. He complains that "unl
14.
by luispedrocoelho 9y ago
The reply in the uber blog is less "angry tweet" than "good peer review": it points to a very specific methodological weakness which has bearing on the final conclusions.
15.
by luispedrocoelho 9y ago
For some things, we do. But databases are not magical and setting up a good table/index system &c is also work and there is overhead. Thus, if we are talking about having (for example) a webservice where queries have a form that is
16.
by luispedrocoelho 9y ago
`isin` is worse in terms of performance as it does linear iteration of the array. Reading in chunks is not bad (and you can just use `chunksize=...` as a parameter to `read_csv`), but pandas `read_csv` is not so efficient either. Furthemore
17.
by luispedrocoelho 9y ago
1. No, that function was the bottleneck, by far, and I can tell you that >10,000x was what we got between the initial version and the final one. 2. I don't care about faster at this point. The function is fast enough. Maybe there is
18.
by luispedrocoelho 9y ago
That is the _FAST_ version of the code (people keep saying "of course, it's slow", when it's the fast version). Here is an earlier version (intermediate speed): https://git.embl.de/costea/metaSNV&#x2
19.
by luispedrocoelho 9y ago
If I can fit the code into numpy-like structure, then Python is typically fine. The issue is when I cannot.
20.
by luispedrocoelho 9y ago
That may be the case. However, my point is that we started with a rather direct implementation of a formula in a paper. This was very easy to write but took hours on a test set (which we could extrapolate to taking weeks on real data!). The
21.
by luispedrocoelho 9y ago
You are commenting on the variant of the code that is fast enough that it doesn't matter.
22.
by luispedrocoelho 9y ago
"spending an hour figuring out what arcane incantation I need to pass to np.einsum to get the operation I want" Yes, I have also had this experience and I hate how in the end, the code is very hard to read, while the for loop was
23.
by luispedrocoelho 9y ago
I think this is part of my argument: as datasets grow faster than single-core CPU speed, performance matters more and more.
24.
by luispedrocoelho 9y ago
Yep, my Haskell usage is "conduit all the way down". I even wrote up a few utilities to make use of multiple threads while working at a high level: https://hackage.haskell.org/package/conduit-algorithms-0.0.7.
25.
by luispedrocoelho 9y ago
OP here. Speed is the main motivation, but total time is TimeToWriteCode + TimeToRunCode. Python has the lowest TimeToWriteCode, but very high TimeToRunCode. C++ has lowest TimeToRunCode, but high TimeTowWriteCode. Haskell is often a good c
26.
by luispedrocoelho 9y ago
There are variations where this works out in google's favor, but there is a big difference to the ebay situation where you just keep relisting the same item until it sells for a "good enough" price: each ad impression is only
27.
by luispedrocoelho 9y ago
Not as difficult, but consuming large amounts of energy on short notice is also not trivial. "connect a resistor to the end of a power cord" is supposed to make it sound trivial, but operating the resistors needed for absorbing th
28.
by luispedrocoelho 9y ago
At some cost to portability, IIRC, on Linux you can ask the operating system to keep pages in memory. Other systems will probably have similar functionality.
29.
by luispedrocoelho 9y ago
OP here. Use redis if you can. Use something like diskhash if you cannot afford the extra overhead of sqlite/redis. In my case, the alternative we were using before is an in-memory hash table built at startup from a text-based represen
30.
by luispedrocoelho 9y ago
Yes, it does give you a lot more and is great. Unfortunately, on my initial tests, it just was not fast enough.

More ›