6 ms·

> And yes, I also hate when I misplace my billions. Especially since I have yet to relocate them... I think you're joking, but I'm not sure. I work in trading,

> And yes, I also hate when I misplace my billions. Especially since I have yet to relocate them... I think you're joking, but I'm not sure. I work in trading, and I've been on the receiving end of that phone call. As I recall, it was around 9PM in the US, my work phone rang and could see from the caller ID that it was from our London office. There were no greetings, first words I hear were "We're missing over a billion dollars. You need to find it...NOW." When I received that call, it was in the middle of the 2008 financial crisis. The daily PnL swings were wild, and it wasn't always clear on the cause. FX volatility was insane. We did all of our PNL reporting in USD, but held a lot of foreign assets. There was no malfeasance; I'd just taken ownership of the system a week or two before, and a nightly job had silently failed. Perl job on Windows, extracting data from a 3rd party trading system that wasn't built w/ integrations in mind, feeding it into in-house systems. It was a very flimsy house of cards. A gentle breeze in the night would knock it over. Rewrote the integration in Python, hooked everything up into our monitored job scheduler. Had to do some janky UI automations in Python until we got the vendor to add a proper CLI-based reporting mechanisms. It was a "fun" ride, but I eventually got my evenings back. Did cause the end of a relationship, though, so there's that.

Andrex 3y ago
Would have been nice to have seen some of that billion you found, I'm sure.
- pas 3y ago
  it was not lost after all, so it couldn't even have been found. how ironic.
- mistrial9 3y ago
  welcome to your new career in professional finance services
hef19898 3y ago
I was joking, the max I ever "lost" was a couple 100k of inventory, the majority of it was recovered and the reminder, as far as I know, covered by insurance. Also a, surprisingly similar, fun story involving just slightly different interpetation and handling of messages between our and the service providers WMS, which screwed up things in ways I never thought possible. And almost went unnoticed, after all even with top notch metrics and my borderline paranoia the issue went on for almost three weeks before we caught it. I can only imagine so the slight shock you had after that phone call so! I love those stories from the trenches so you hear on HN, thanks for sharing!
paganel 3y ago
A lot less money involved, but I remember my boss at the time (we were a small mortgage broker in Eastern Europe) asking me to write a quick Python script that would automatically get the daily Libor number and save it into our DB. Seeing as I was hearing about Libor all day, every day (almost all of our clients had their mortgages computed on that piece of info), I had expected it to be something “automatic” (like at least an XML thingie) and well documented. Instead I had to parse some html on a page somewhere (I remember some yellow background) and hope that the HTML structure around that Libor figure would remain unchanged. This was all happening around 2007 - early 2008, suffice is to say that when all the Libor scandal happened a little later on I was not at all surprised.
- Scoundreller 3y ago
  I remember some internet arguments on Libor when I researched how it was determined and everyone disagreed with me because of how much was based on it. It's just survey data, and no verification of whether a loan can happen at that rate? And you throw away the lowest rates, which should be the market-clearing rate all-the-things-being-equal if lenders are fungible? And if lenders aren't fungible, then isn't it all apples and oranges?
  - smallnamespace 3y ago
    High finance even fairly recently (80s) was based on handshakes and trust. The value of contracts tied to LIBOR grew by an order of magnitude or two while the definition of LIBOR wasn’t adjusted. Why wasn’t it fixed? Because replacing it would require an enormous amount of coordination and there was no clear evidence that it was broken. When that changed it finally got replaced by SOFR.
    - rvba 3y ago
      LIBOR not broken? It was a club used by the big boys from banks to screw everyone else. With a madeup number that benefitted them. https://en.wikipedia.org/wiki/Libor_scandal https://en.wikipedia.org/wiki/Libor_scandal

Andrex 3y ago

Would have been nice to have seen some of that billion you found, I'm sure.

pas 3y ago
it was not lost after all, so it couldn't even have been found. how ironic.
mistrial9 3y ago
welcome to your new career in professional finance services

hef19898 3y ago

I was joking, the max I ever "lost" was a couple 100k of inventory, the majority of it was recovered and the reminder, as far as I know, covered by insurance. Also a, surprisingly similar, fun story involving just slightly different interpetation and handling of messages between our and the service providers WMS, which screwed up things in ways I never thought possible. And almost went unnoticed, after all even with top notch metrics and my borderline paranoia the issue went on for almost three weeks before we caught it. I can only imagine so the slight shock you had after that phone call so! I love those stories from the trenches so you hear on HN, thanks for sharing!

paganel 3y ago

A lot less money involved, but I remember my boss at the time (we were a small mortgage broker in Eastern Europe) asking me to write a quick Python script that would automatically get the daily Libor number and save it into our DB. Seeing as I was hearing about Libor all day, every day (almost all of our clients had their mortgages computed on that piece of info), I had expected it to be something “automatic” (like at least an XML thingie) and well documented. Instead I had to parse some html on a page somewhere (I remember some yellow background) and hope that the HTML structure around that Libor figure would remain unchanged. This was all happening around 2007 - early 2008, suffice is to say that when all the Libor scandal happened a little later on I was not at all surprised.

Scoundreller 3y ago
I remember some internet arguments on Libor when I researched how it was determined and everyone disagreed with me because of how much was based on it. It's just survey data, and no verification of whether a loan can happen at that rate? And you throw away the lowest rates, which should be the market-clearing rate all-the-things-being-equal if lenders are fungible? And if lenders aren't fungible, then isn't it all apples and oranges?
- smallnamespace 3y ago
  High finance even fairly recently (80s) was based on handshakes and trust. The value of contracts tied to LIBOR grew by an order of magnitude or two while the definition of LIBOR wasn’t adjusted. Why wasn’t it fixed? Because replacing it would require an enormous amount of coordination and there was no clear evidence that it was broken. When that changed it finally got replaced by SOFR.
  - rvba 3y ago
    LIBOR not broken? It was a club used by the big boys from banks to screw everyone else. With a madeup number that benefitted them. https://en.wikipedia.org/wiki/Libor_scandal https://en.wikipedia.org/wiki/Libor_scandal