13 ms·

I built Excel for Uber and they ditched it

753 points by robdimarco 3y ago

mrintegrity 3y ago
I get a strangely dystopian feeling from this article, like it's almost about a character in a black mirror episode
- pagnol 3y ago
  I reflexively swiped left when the inevitable newsletter modal started to appear at the bottom, so didn’t finish reading the article, but would like to know what gave you the dystopian vibe?
  - Tao3300 3y ago
    You should have kept going. Spoiler alert: It turns out they were fulfilling a Babylonian prophecy the whole time. The whole development cycle was a complicated sacrifice to Marduk.
- bastawhiz 3y ago
  Man, what a commentary on my career
  - lcnPylGDnU4H9OF 3y ago
    Alternatively, they just accidentally (I assume) called you a good storyteller.
- Vt71fcAqt7 3y ago
  Is it possible that you thought this because the words "black mirror(ed)" are mentioned in the article iteslf?
- Tao3300 3y ago
  Starring Excel as the pig.
- jhaenchen 3y ago
  That's because everyone in this comment section is giving wisdom about how to deal with the emotional reality of being abstracted from the fruits of their own labor. "Don't get attached to your work" is dystopian.
dublinben 3y ago
Is it common to take code written for one employer and reuse it for another?
- arcbyte 3y ago
  Some of us will strategically write generic code on our own time and machines and import it to employers machines when needed and customize it. For instance, how many times do you really need to write a spring OAuth server that integrates with LDAP? Or the guts of a simple CRUD app?
  - Kwpolska 3y ago
    > how many times do you really need to write a spring OAuth server that integrates with LDAP? I would expect this to be a library somewhere. > import it to employers machines How would you prove that you write the code long before getting hired by them?
    - dartos 3y ago
      Someone wrote the library for a reason, right? Many are made bc people needed it for their job.
    - sokoloff 3y ago
      > I would expect this to be a library somewhere. Indeed. That library is often written off-the-clock by person X and imported on-the-clock by person Y. Sometimes X === Y.
      - bastawhiz 3y ago
        And sometimes that person even writes about it on substack
    - soperj 3y ago
      > How would you prove that you write the code long before getting hired by them? Commit it to a repo somewhere.
Znafon 3y ago
"Nothing came of it, but I took the code and shoved it into my back pocket for a rainy day. My idea was to take this code and spruce it up for Uber’s use case." "My first reaction was to publish the code on Github." I’m very surprised by this, isn’t the code property of Box, or Uber? The author does not mention their authorisation before releasing it under MIT license.
- sidewndr46 3y ago
  I believe this kind of story is the kind that gives most legal counsel nightmares.
  - pc86 3y ago
    Especially the brazenness with which the author basically says "if they want to sue me for this verified and admitted IP theft, they can." Sure, they probably won't. But they might. And if they do, you'll lose immediately. Seems like a pretty high risk no reward scenario.
    - sokoloff 3y ago
      And if you lose immediately, you likely owe damages. Those damages, even if trebled, appear to be $0 here.
      - pc86 3y ago
        Which would be a great solace to someone who just spent $10k or more like 2-3x that responding to a lawsuit. But that's also why I agree the odds of actually getting sued are near-zero.
        Alupis 3y ago
        Some companies, armed with floors of attorneys and retained outside counsel, do that sort of thing just for the message alone. It costs them next to nothing, relatively, ruins the defendant regardless of outcome, and makes it clear for others to not mess around with IP.
renewiltord 3y ago
That's a tremendous war story. I _love_ it! Doing a thing that's considered unreasonable. Knowing that total value is integral of instantaneous value over time (so short high-impact projects can easily outvalue long-lived projects).
constantly 3y ago
I wouldn’t try moving code written for one employer and using it another employer. Unfortunately for me my employers have been somewhat litigious; thankfully Uber doesn’t have that reputation.
itsthecourier 3y ago
"When I met the members of the Crystal Ball team, it was about roughly ... people" how many people there were at that point, @bastawhiz?
- romnon 3y ago
  people be rough
- bastawhiz 3y ago
  Sorry, obvious typo! It was four people.
erulabs 3y ago
I read this article looking forward to the complex bespoke code to be ripped out and deleted - but the author clearly grew as an engineer in a way I didn’t expect: > Sometimes that’s just how it is. The devops saying “Cattle, not pets” is apt here: code (and by proxy, the products built with that code) is cattle. It does a job for you, and when that job is no longer useful, the code is ready to be retired. If you treat the code like a pet for sentimental reasons, you’re working in direct opposition to the interests of the business. A lot of code is fun to write. A lot of problems are fun to solve. But a business, especially a startup, needs to stay razor focused. My entire career is effectively to sit in meetings and tell young, passionate engineers not to build things. It’s a bit depressing, but it’s also vital. A good engineer can solve any problem with clever code. A great engineer knows what problems aren’t really problems and probably an XLS download link updated daily would have been fine.
- Retric 3y ago
  Exactly, in 2016 there was several off the shelf options for doing the exact same thing. It’s a perfect example of a young engineer feeling a huge accomplishment from reinventing the wheel, and then realizing the clever solution wasn’t actually worth anything like the effort required to create it. I had a long conversation to convince someone not to go down that path in 2006, and I am sure someone’s going to do it in 2026. Pausing to think: I wonder how someone else solved this exact problem is such a huge part of how you grow as a developer I wish schools would focus more on it.
  - yard2010 3y ago
    I would say what you talk about is experience. To have an experience, you must go this path to realize what to not do. It feels like a catch 22 kind of thing.
    - Retric 3y ago
      Doing it well comes down to experience, but doing it at all comes down to asking which of your unconscious assumptions are hard requirements. Nobody is actually saying you have skills X, Y, Z, which you must use to solve this problem, which is a huge difference from how schools prepare people for the workforce.
itsthecourier 3y ago
I'm glad I read this article and learned about the circ
sr228822 3y ago
ironically, I was backend at Uber 2014 - 2018 and used to ask a simple version of implementing an excel formula engine as my go-to coding interview question. Its got a nice mix of data-structures, algorithms, complexity, and implementation. Good candidates can get a reasonably efficient implementation handling cell references in an hour. nice read. made me nostalgic for the wild days of Uber-China hacking.
- bastawhiz 3y ago
  Amazing! I independently had an extremely similar interview question (because of this work) that also gave me great signal. That's really funny!
Reubend 3y ago
Love the part about circular references. I gotta say though - I'm having a difficult time imagining how complex these fomulas are that reimplementing Excel's formula engine is easier than just porting the formulas into JS.
- bastawhiz 3y ago
  The good news is that formulajs had a huge number of those functions implemented in JS already. Almost all of the time was spent on the engine, which wasn't a huge amount of code. The problem with porting the code to JS is that a.) nothing is named, b.) there's no real way to organize the code you've written because you're going from a spatial way of organizing code to imperative script, and c.) the actual design of the spreadsheet wasn't known to any engineers (it was designed by a data scientist, or perhaps an analyst). The work of translating would have meant really understanding what the thing is so that it can be turned into functions and modules. It also would have still required getting Excel function equivalents, since there's not a 1:1 equivalence between Excel and what's available in the JS standard lib.
- xen0 3y ago
  Honestly, if you're at 'thousands of formulae' across multiple sheets, I'd probably suggest writing a graph based execution engine too. The circular reference thing would have definitely thrown me for a loop though.
  - kirubakaran 3y ago
    I just want to let you know that the pun in the last line didn't go unappreciated.
Vt71fcAqt7 3y ago
I love this article. It begins setting the scene with Uber's grand expectations for the chinese market and then shows just a small piece of the work Uber spent on Uber China. Meanwhile Uber China itself failed spectacularly, with factories of phones claiming the free rides money.[0] I don't think I've ever experienced dramatic irony in a blog before, certainly not a technical blog. The post itself is a masterpiece. [0]https://www.forbes.com/sites/ywang/2016/09/27/ghost-drivers-are-just-one-of-uber-chinas-problems-following-didi-takeover https://www.forbes.com/sites/ywang/2016/09/27/ghost-drivers-...
gobrrrmeme 3y ago
There's a service in SharePoint and SharePoint Online that let you program against Excel. Just gonna leave this here. https://learn.microsoft.com/en-us/sharepoint/dev/general-development/getting-started-with-excel-services https://learn.microsoft.com/en-us/sharepoint/dev/general-dev...
acchow 3y ago
> Over the summer of 2016, we came up against a new twist on the project. We had a model that ran overnight to generate data for anticipated ridership in China. That data wasn’t useful on its own, but if you fed it into a tab on a special Excel spreadsheet, you’d get a little interactive Excel tool for choosing driver incentives. Our job was to take that spreadsheet and make it available as the interface for this model’s data. They eventually built a homegrown "Excel" clone as the UI for their model because "city teams only know how to use Excel". I would have done it the other way around - connected Excel to the data output by the model so the "city teams" could continue to use real excel. I think most finance teams do something like this.
- ftxbro 3y ago
  > I would have done it the other way around - connected Excel to the data output by the model so the "city teams" could continue to use real excel. Yeah except: “When you click in the cells of the spreadsheet you can see the formulas. You shouldn’t be able to do that.” “You said to make it just like Excel.” “People working for Didi apply for intern jobs at Uber China and then exfiltrate our data. We can’t let them see the formulas or they’ll just copy what we do!”
- bastawhiz 3y ago
  Because the city teams were in China, we didn't have this luxury. Everything had to be behind Uber's beyondcorp equivalent, and there was no real way to auth folks from the Chinese mainland. Our only surface was the browser.
  - MichaelZuo 3y ago
    A dedicated satellite link cross pacific is not that much money. Then again maybe the CFO didn't know so he took the easiest known option.
    - bastawhiz 3y ago
      I've always been fond of container ships loaded up with entangled qbits. Much lower latency!
pc486 3y ago
What a fantastic perspective from the former Uber BI team. I was on the Vertica team during this time period and the amount of effort was spent on incentives mind boggling. Millions a day lost on downtime, product features, or engineering bandwidth was a common theme. A director asking for an exact spreadsheet to be the UI would have been par for the course, especially during the Uber China days. Heck, I personally loaded FX prices into Vertica from a spreadsheet emailed every month to the team. That process remained for more than a year as there just wasn't enough bandwidth to invert the control as automated ingestion. Thanks for digging up these memories, @bastawhiz. I'd love to see more. :)
Macha 3y ago
So a lot of my time as a more junior engineer was spent on a similar project that I've described as "rebuilding excel". In my case, it was in the form of a table widget. It was an inhouse widget for displaying tabular data. We were working on AngularJS 1, and had moved from a prior iteration of our application in ExtJS. Now ExtJS had a reasonably featureful table widget out of the box, and so when we were porting to AngularJS, it was considered important that we continued to have these UI features, but at the time (the Angular ecosystem was relatively immature at that time) there were not very many advanced table widgets available in open source, so we ended up building our on. However, while the ExtJS table widget had been treated by product management as pretty immutable "this is what ExtJS gives us, we can customise the colours and wording and that's it", the idea that we could customise the table widget started something amongst one of the PMs. And so we would get a constant stream of feature requests for the table widget to add stuff and enhance it and soon we were significantly more featureful than the ExtJS widget. It's still, to this day, the most featureful table widget I've seen in a web app. Everything Excel had in terms of resizing tables, sticky columns, scrolling behaviours, sorting, filtering, searching, etc., all saved in your config so it was synced across all your devices, as well as all the performance goodies like recycled rendering etc. The constant stream of feature requests meant there were dev team years invested in this table widget. As a more mature engineer looking back, it's clear that at some point this had stopped being about customer value and more about one PM's obsession with getting excel like functionality in a in-browser reporting tool, but at the time we just kept building those features. Now at this time, our company had acquired a sort of competitor of ours. This competitor had what was effectively the same product, but in a different market. And so the first merger of the functionality was basically to reskin both applications so they would pretend to be tabs in a unified application, and change some terminology, etc. They actually did happen to have a pretty similar tech stack to us, so some newer components were available in both applications. But it became clear that our users were not happy with two applications pretending to be one. They wanted to know why this other market was not accessible by, say, a dropdown in the configuration, and not an entire application which worked in different parts from subtly differently to entirely differently. So the discussion became about building a ground up unified interface for both of them. Of course, this ignited the discussion of "which table component do we use?". On the one hand, the acquired team were looking at our table, with it's fifty billion options and single handedly accounting for half the page weight of the minified JS of our application and did not want something so bloated. On the other hand, our team were looking at their table widget which was effectively "for row in data, for column in row, print td" and dreading having to rebuild all these features for product management again. Ultimately the conflict was resolved by choosing to use an open grid that had less features than ours but more than theirs and telling the PM in question that table features were going to be prioritised much less heavily from then on unless there was a real user need for it.
deleted 3y ago
[deleted]
abeppu 3y ago
I'm confused about the circular reference thing. Like, was there a reason to do the linear regression that way? Is there a secret story in a story where next week, in a spinoff / sequel episode, the data scientist responsible will explain why they took the weird/surprising choices they did?
- andy81 3y ago
  I used it once to run a Monte-Carlo simulation in a spreadsheet. After enabling iterative calculation and manual calculation, every press of refresh runs a loop. Fun stuff.
- DwnVoteHoneyPot 3y ago
  It's a common excel trick in finance. For example, let say you have $0. If you borrow $1,000,000 at 5% interest, by end of year you'll be short $50k. That means you actually needed to borrow $1,050,000. But the extra $50K causes more interest ($2,500)... so you needed to borrow $1,052,500, which causes more interest... and so on. Instead of doing some Excel Goal Seek or Solver or VBA macro, it's nice to let the excel "reactivity" handle it for you.
  - hu3 3y ago
    Your explanation is brilliant!
- DrDroop 3y ago
  I didn't know you could do this in excel but it reminded me of the converge adverb in k: https://xpqz.github.io/kbook/Adverbs.html#converge-f-f https://xpqz.github.io/kbook/Adverbs.html#converge-f-f
  - itigtohft 3y ago
    https://en.wikipedia.org/wiki/Fixed-point_iteration https://en.wikipedia.org/wiki/Fixed-point_iteration
brap 3y ago
> He simply couldn’t believe that I’d written a full spreadsheet engine that ran in the browser. I can't believe it either, and I don't mean this in a good way. Apache POI lets you run headless Excel. You import and interact with sheets programmatically in Java. We used this in my old workplace for exactly the same reason (functions, cell references, the whole thing), it worked great. You found the ‘circ’ problem with a bit of luck. What about all of the other hidden little quirks of Excel that you would ultimately run into down the road? Are you really going to build and maintain a full blown Excel clone in JS? Is this really the objective of the frontend team? It seems to me like a bit of googling and >90% of the work here could have been avoided. As an added bonus it would have been done by the backend team instead.
- rkangel 3y ago
  To be fair, he wrote a spreadsheet engine that could run one particular spreadsheet. Admittedly a complex one, but it was a fixed set of functions that he needed to implement and not and endless tail of things that people expect Excel to do. I think I'd probably have argued more about the UI spec and down some Excel behind the scenes but it is a familiar UI if you've got lots of number inputs all over the place. I've always enjoyed this article about building a spreadsheet in 100 lines of F#: https://tomasp.net/blog/2018/write-your-own-excel/ https://tomasp.net/blog/2018/write-your-own-excel/ The expansion from that to the feature set needed here is manageable.
  - brap 3y ago
    Project requirements are never fixed, they’re always evolving. It was only a matter of time before users would’ve complained about features being missing/broken, especially since what they’re used to is Excel and this was meant to replace it.
    - bastawhiz 3y ago
      > this was meant to replace it. I think that's a very generous read of what I said the requirements for this product were.
kazinator 3y ago
> You see, when formulas create a circular reference, Excel will run that computation up to a number of times. Like almost every spreadsheet before it: Lotus 1-2-3, Borland Quattro Pro, VP Planner ... Spreadsheets iterating on circ references goes back to the 1980s. The first spreadsheet application, VisiCalc, didn't track dependencies: it evaluated cells left to right, top to bottom, IIRC. Microsoft had a product called Multiplan that competed with VisiCalc. Not sure if that did iteration. I think it used to be a setting in some programs whether circular references are flagged as errors, or iterate. Maybe it's still that way in Excel?
- itigtohft 3y ago
  There are a couple of nice minimal spreadsheets in js: http://web.archive.org/web/20130606222859/http://thomasstreet.net/blog/spreadsheet.html http://web.archive.org/web/20130606222859/http://thomasstree... Inspired by that, an even smaller one: https://jsfiddle.net/ondras/hYfN3/ https://jsfiddle.net/ondras/hYfN3/ The latter catches circular references rather than trying to calculate the fixpoint
ChrisMarshallNY 3y ago
> It’s easy to treat a particularly clever or elegant piece of code as a masterpiece. It might very well be a beautiful trinket! But we engineers are not in the business of beautiful trinkets, we’re in the business of outcomes. This spoke to me. However, as anyone that has looked at my code can attest, I tend to also want my code (and its functionality) to be very pretty. I'm generally writing code that I will be maintaining, so it needs to be something that I can look at, in a year, and understand. I'm currently in the final phases of a project that I will never announce here, and don't plan on taking much credit for, but it really is da schizz. It's that way, because no one is paying for it, and no one will make money from it. Money both spoils everything, and also makes it all happen.
allanrbo 3y ago
Another approach would be to use the Excel APIs. Both the classic desktop Excel and the web version has APIs to read/write cells and recompute. Rebuilding is more fun of course :-)
bjornlouser 3y ago
"'Growing as an engineer' means becoming a better engineer, and becoming a better engineer (directly or indirectly) means getting better at using your skills to create business value." Learning how the Excel model worked and then reimplementing it would have been a better example of 'getting better at using your skills to create business value'.
bsder 3y ago
Side question: Why the hell is it so stupidly difficult to display and edit tabular data from programming code? You can't drive Excel from Python/Rust/etc (Microsoft just announced to great fanfare that Excel can call Python--which is the wrong way around). All the editable table widgets for the web seem to suck. Nobody seems to have a TUI which you can drive from an external program. A poorly written spreadsheet with a driveable API seems like a component that has been built multiple times by lots of people yet seems to be unavailable. Is there some solution that I'm missing?
- kubanczyk 3y ago
  > Microsoft just announced to great fanfare that Excel can call Python--which is the wrong way around How do you cause users to even discover that Python app, or feed an input to it? Of course the spreadsheet software is an obvious candidate for the primary UI.
easton 3y ago
Excel circular reference doc, for the curious: https://support.microsoft.com/en-us/office/remove-or-allow-a-circular-reference-8540bd0f-6e97-4483-bcf7-1b49cd50d123 https://support.microsoft.com/en-us/office/remove-or-allow-a... > Unless you're familiar with iterative calculations, you probably won't want to keep any circular references intact. If you do, you can enable iterative calculations, but you need to determine how many times the formula should recalculate. When you turn on iterative calculations without changing the values for maximum iterations or maximum change, Excel stops calculating after 100 iterations, or after all values in the circular reference change by less than 0.001 between iterations, whichever comes first. However, you can control the maximum number of iterations and the amount of acceptable change.
cush 3y ago
Any PM that walks in with “Just make it like Excel” hasn’t spent a moment writing code.
- bastawhiz 3y ago
  Even worse: he was a director
  - cush 3y ago
    Arguably the most battle-tested, feature-rich program on the planet. Yeah just make it like that.
    - bastawhiz 3y ago
      I should tell the story of the PM (also at Uber!) who demanded our web app be "faster than Excel". I couldn't make it up if I tried.
sha16 3y ago
> “take this [the spreadsheet] and put it in on the website [Wesley].” This should be taught in classrooms.
DubiousPusher 3y ago
Just a heads up for anyone who finds themselves with a similar requirement, there's a very robust set of office APIs available in .net. I would be surprised if you couldn't open and run a whole workbook with them though I have only used them for more tangential tasks.
JohnMakin 3y ago
Reading this and having China cloud experience on my resume I wonder how many chinese data laws they may have violated, you have to really strictly segregate data when it comes to chinese users.
- blincoln 3y ago
  I was wondering the same thing, especially since it arguably involves mapping/geolocation information.
dmd 3y ago
> So the data scientists have multiple laptops that they download the data to, then run the models overnight. This haphazard way of running compute jobs really stuck out to me. I can't imagine doing things this way (rather than having a central compute cluster running SLURM or similar) at a company bigger than, say, a dozen people - much less the scale of Uber. What's the rationale? Even if it's just a cluster of 3 or 4 machines in a rack shoved in the corner, isn't that better than ... laptops?
- bastawhiz 3y ago
  It was easy to go to IT and say "get us a pile of laptops" and let the data scientists do their thing. It was hard to hire engineers to solve the problem (I was one of the engineers). They particular problem was far enough down the priority list that it took until 2016 to solve.
  - dmd 3y ago
    Heh. Well, I guess... sorry I didn't come help :) (I interviewed but turned down my offer.)
    - bastawhiz 3y ago
      You dodged a bullet
lph 3y ago
Excel is an ancient and complex beast. I get the appeal of building this project---it sounds fun---but trying to duplicate the Excel engine to the level of producing identical outputs is, frankly, bonkers. The author caught the one discrepancy they noticed, circular reference handling, but how many did they miss? How do they know different inputs won't cause it to deviate from Excel? I didn't get the sense from the blog post that this had extensive test coverage. Putting it into use for a business-critical financial calculation is a massive risk, but I guess that's how Uber rolls ¯\_(ツ)_/¯ It would have been less fun but way, way less risky to wire a headless Excel up to a javascript front-end.
- bastawhiz 3y ago
  No Windows, no Excel license, hundreds of concurrent users. How confident are you that each user is getting the calculations that they triggered and not someone else's? How are you deploying that spreadsheet? How are you version controlling the mutable parts of the system? How confident are you that the solution would be robust in the face of bad inputs and high load? I won't say there wasn't risk, but there was quite a bit of testing and a human always made the final call anyway (I never fully understood why we didn't eliminate humans from the processes altogether).
ryukoposting 3y ago
I wonder if the author would view this situation differently had Uber/Box decided to claim the code as their own. It has to bring some catharsis to know that, even if the code never actually met its potential, at least the whole world can see and appreciate it. I created a whole programming language as an intern for <defense megacorp>. It was lazily evaluated and garbage collected. Unquoted MAC addresses were valid syntax, among other application-specific oddities. No bytecode or JIT shenanigans - the interpreter just pushed and popped stuff from a stack as it traversed the parse tree, and that was fast enough for what we were doing with it. The interpreter was written in pure ANSI C, and Valgrind was very happy with it. Maybe it has been totally forgotten, or maybe it became critical to their technical infrastructure. That code never left the airgapped lab where I wrote it, so I have no way of knowing. 3 years ago, as a recent college grad, that was by far the coolest piece of "actually useful software" I had ever written. It's still high on the list. Sometimes I wonder whatever happened to it.
- lowbloodsugar 3y ago
  The part the author is missing is that Box and Uber have already claimed the code as their own. It will be in the employment contract. The author appears to be under the impression that asking a mid-level manager (or even a senior manager) "Hey, do you want this?" and that manager answering "No", is in any way legally binding on the company.
  - ryukoposting 3y ago
    I've only ever agreed to those terms kicking and screaming. I wish there was a larger part of the community willing to advocate for a "love of the art" clause that would explicitly state that code I wrote on my own time is mine.
joshu 3y ago
i feel like “learn vba” would have been a much easier path here
OJFord 3y ago
> having answers that were very, very close is objectively worse than having numbers that are wildly wrong: very wrong numbers usually always mean a simply logic problem. Almost-correct numbers mean something more insidious. Ha! I massively identify with this. 'The market-wide median is off by £3.07' is a much harder but to crack than the 'worse' report that 'the median is twice the max and the min is null'.
- OJFord 3y ago
  *bug to crack
stevebmark 3y ago
A good and related article about the dangers and poison of domain driven design, and why spreadsheets are used at every level of every company. https://dev.to/cheetah100/domain-driven-disaster-147i https://dev.to/cheetah100/domain-driven-disaster-147i
ineedasername 3y ago
These are the times that, if you at least enjoy the process of coding, you should try to look at your work as a videogame. You enjoyed the game, you even got paid for it, and you beat the final boss, it doesn't matter that no one will ever see your final score.
ipython 3y ago
Best quote: "People working for Didi apply for intern jobs at Uber China and then exfiltrate our data. We can’t let them see the formulas or they’ll just copy what we do!” This is so true. People in the US just don't understand the level of economic and industrial espionage that happens in China on a daily basis. I was responding to an unrelated breach at an unnamed tech company back in mid-2000s time frame and had a side bar conversation that went like the following: Them: "Yeah, we just opened a tech center in Xinjiang and ... wow, we've had quite the rash of lost ID badges there recently" Me: "Have you considered that they're not 'lost', but rather 'sold' for profit?" ... silence ... I don't know if executives are aware but just don't care, or if they're simply incompetent, but China has productized industrial espionage on a massive scale. GE Aviation was a victim more recently: https://www.cincinnati.com/story/news/2022/11/16/accused-chinese-spy-who-targeted-ge-aviation-sentenced-to-20-years/69653460007/ https://www.cincinnati.com/story/news/2022/11/16/accused-chi...
- cflewis 3y ago
  Well, not just corporate espionage, right? State espionage is going to be at every large US company too. To speak to the article, I can only imagine how excited an agency would be to get real-time updates on the Uber movements of a target.
  - ipython 3y ago
    Two things. First, this isn't the same thing; you're talking about embedding a resource in a company to read out real time telemetry from internal systems (information with a short shelf life), not stealing industrial trade secrets (information with a very long shelf life). Second, I can assure you that, even with our imperfect systems, there is actually a set of checks and balances in place to prevent rampant (note I said rampant) abuse of this kind. Yes, agencies would be very excited for this sort of capability. Do they get it as a matter of course that easily? No. There are layers of accountability, legal authorities, and (warranted) push back from commercial entities.
    - BlueTemplar 3y ago
nardi 3y ago
I do not like working with people like this, because I don't want to be anywhere near the maintenance burden of a pile of hacks that technically can run a single Excel spreadsheet. I don't even want to be within earshot of people complaining about this. If I even thought about it for too long I'm sure I'd have a giant headache.
- ndesaulniers 3y ago
  I worked with the author for maybe 3 years, I thought he was pretty awesome. Not surprised he implemented a partial implementation of excel so quickly. I'm disappointed in myself that I somehow got through most of the article without realizing who the author was (thanks Firefox reader mode); especially since the naming of Wesley and Crusher is too good. Of course it was Basta! Of course it was!
- abigail95 3y ago
  what would you have done instead?
- prasoonds 3y ago
  In a startup, you either build, ship and you survive another day, or, you die. If I were hiring for an early stage startup, I know exactly which of you two I would want to hire.
CodeWriter23 3y ago
> Someone will take joy in ripping it out someday.
purpleblue 3y ago
1) This was the type of effort and level of coding that I saw when I was Uber during its heyday. I love it and I miss it every day. 2) You stole Box code, used it at Uber, and then stole Uber code and posted it on github. I understand no one's using the code or missing that code, but you really did steal that code. It belongs to the company, not to you. I would be careful not to do that in the future, because technically that's a trade secret and people have gone to jail over that, like that programmer at Goldman Sachs (wrongfully) and ironically Levandowski who took Google code and tried to use it at Uber.
lowbloodsugar 3y ago
IANAL but I've paid lawyers to answer this question in the US. The question is, if you do some work, in your own time, on your own equipment, does your employer own it just because the employment contract says they do? In California: if the work in any way relates to the employer's business, then yes, they own it. One way to guarantee that it relates to the employer's business is to bring it into the office and use it as part of your job. If your employer is Apple or Google or AWS or Microsoft, then probably anything you write would in some way relate to their business. Write spreadsheets by day, but games by night? All of those companies make games, or are in the games business. I would love to hear a lawyer say, "Well, it doesn't matter what the employer does, it only matters what your job duties are, so writing games at night is fine if they don't pay you to work on games related things during they day." But I've never been told that by a lawyer, whether I paid them or otherwise. Everywhere else in the USA: they probably own it. You could write software for washing machines, and write a video game, and if your contract says they own everything you write then they do. You signed it. There's no "but surely not!" defense.
mkii 3y ago
> "In 2015, I had built a prototype of a tool at Box. Box had a collaborative note-taking product called Box Notes (based on Hackpad). I had the idea to make a similar project for working with numbers: sometimes you didn’t need a full spreadsheet, you just needed a place to put together a handful of formulas, format it with some headings and text, and share it with other people. Sort of like an ipython notebook for spreadsheets. I called it Box Sums." > "Nothing came of it, but I took the code and shoved it into my back pocket for a rainy day." ... > "Apparently that was a thing. I remember being only half-surprised at the time. I hadn’t considered that our threat model might include employees leaking the computations used to produce the numbers in question." ... > "My first reaction was to publish the code on Github." Perhaps the author feels "only half-surprised" due to their own disregard for corporate legal ownership of code. The hypocrisy is strong here.
lxe 3y ago
> To this day, I’ve never encountered an in-house application system as well-designed as Uber’s. You could go from start to Hello World running on a *.uberinternal.com subdomain in under 30 minutes with full CI/CD. I worked on all this at Uber back then, and this comment warmed my heart a bit. Thank you.
charles_f 3y ago
> “Every day that we don’t have this tool as specced, we’re losing millions of dollars.” There was no budging on the spec. If this is life or death, then the first reflex should be to find ways to make things simpler to ship. I understand that the requester in this case was a head of finance, but sh*t, if you want something, try to compromise once in a while. This is a great story, but at some point as an engineer in a gig-selling company, I would have asked myself if my job was really to re-build excel.
alzaeem 3y ago
I interviewed with Uber in 2016 with the "crystal ball" team. Reading about R-crusher, I am glad that offer never came through!
StopHammoTime 3y ago
I find it remarkable that they used laptops for this. Surely, a Dell server for $100k with good specs could have crushed a lot of these models. Could have just given them all Remote Desktop sessions too.
schlowmo 3y ago
> “The city teams only know how to use Excel, just make it like Excel.” > “Why can you see the formulas?” > “You said to make it just like Excel.” I can't keep up with the espionage story that followed but I had this conversation more than once in my professional career. The first time I sat as a junior dev on a multi-month project replacing a excel spreadsheet for financial controlling of data centers that only one person who was going to retire understood with a web-based solution. They were quite proud that they were going to get a "modern" solution. Then they wanted me to make it like excel. What followed was evaluating every fricking excel JavaScript library out there at this time, going for one and started duct taping all the missing pieces. They were pleased but the look was off. It wasn't excel. I slapped some styles on it coming quite close. I was not prepared for what happened during the next presentation: They hated it because they wanted a modern web based solution (their words, not mine) and what they got was a poor excel knockoff running in their Internet Explorers. Tables are so 90s. I remember the pain so vividly that I regard "just make it like excel" as some kind of forming meme for my career till today.
ww520 3y ago
I had a somewhat similar requirement in a past work. The Excel file had all the formulas and data, and the analysts only work with Excel files. The result of the calculation in Excel needed to be put on the web. The solution was not to recreate Excel in the browser, but ran the Excel file with its formulas and its data plus the input parameters from end users at the backend server. Apache POI was a nice Java library that could do everything on an Excel file. Once it finished the formula calculation, I just read the cells from the Excel file to extract the result data and generated the web pages to present the data and graphs. One nice benefit was the analysts could update their work in the Excel file, uploaded it to the server, and got the new calculation reflected on the web pages right the way.
whimsicalism 3y ago
Funny that this article is talking about interns exhilarating data and yet the author blatantly recycles code between employers.
ec109685 3y ago
This was an amazing read. One of my favorite memories in college was building a spreadsheet in Java (some of the guts were there) in third semester CS.
jengler 3y ago
Great to see @bastawhiz on here. Box had a collaborative note-taking product called Box Notes (based on Hackpad). Slight correction: Box Notes was forked off etherpad-lite. Hackpad was a parallel fork of Etherpad. [context: I was lead engineer on Box Notes till around 2014/2015. I left before the events @bastawhiz described]
NavinF 3y ago
>You see, when formulas create a circular reference, Excel will run that computation up to a number of times. If, in those computations, the magnitude of the difference between the most recent and previous computed values for the cell falls below some pre-defined epsilon value (usually a very small number, like 0.00001), Excel will stop recomputing the cell and pretend like it finished successfully. W T F
hannofcart 3y ago
On reading this article, I realised that maybe the most noticeable change that experience brings to a developer is... laziness? When I was 24, I was pretty much exactly this person. "Building a spreadsheet engine that runs in a browser. Sounds like so much fun!" And then I'd slog away at it for a month and get something working. Now though, with age, I know I can't chug away coffee working late into the night. My back hurts when I have to sit too long. My wrists aren't being too kind to me these days either. Now if someone asked me to build Excel I'd first laugh in their face. And if I don't see them smiling, I'd ask if they can afford any of these A,B,C...Z COTS products that are doing this that we can just buy. And if none of those work out, I'd look for how I can take something like ethercalc and repurpose it for our use case. Looks like the older I get, rather than writing code, I seem to be getting more adept at how not to write it.
mathgladiator 3y ago
Beyond the questionable IP theft. The lesson of what you build for a company as something that may be thrown away is a good one. However, I believe the only way to actually have a sacred masterpiece is to own the company and make the company's mission to build that sacred masterpiece. For example, there are definitely business opportunity for converting excel sheets to online apps that are collaborative. The core problem is alignment between the tech investment and the company's goal.
nperez 3y ago
I've worked on a project that was intended to replace Excel spreadsheets with a web front end. It was presented like we were moving our processes out of the stone age - as if we were replacing a pen-and-paper process that "doesn't scale", or whatever. I will never do that again. We didn't build a whole spreadsheet engine, but we did build a web UI that simply doesn't do as much as Excel. Excel is powerful. Sometimes it's a fine tool for the job. Our team was laid off before we could roll it out, but I remember the growing sinking feeling of "I would hate using this if I already had a bunch of habits built up in Excel"
PeterStuer 3y ago
Running the models on laptops overnight? They know desktops and servers exist right?
- choppaface 3y ago
  But when you have to be your own IT / devops... and when maybe all you know is R, some stats, and how to admin a laptop... The head of finance may have burned most of the opex budget and devops bandwidth on the Vertica and/or other DB service(s) they were using.
- avidiax 3y ago
  Uber, at that time, was "scrappy". Every operating city was a different market with different rules, so Uber was akin to a franchiser that gave their market managers wide leeway to setup whatever business processes are needed to succeed in that market. Excel is the easiest way for business-oriented people to make numerical decisions, run incentive campaigns, segment riders and drivers, produce reports required by cities, counties, states, etc. The data teams providing market intelligence to the various markets, of course, knew about servers. But they would not have the time or skill to setup something like this, and the business still had daily data needs. So the laptops it was, for a time.
  - PeterStuer 3y ago
    A laptop is basically a desktop with sustained performance thrown out the window in favor of sleek portability.
quijoteuniv 3y ago
Is interesting what people focus on when they read, ultimately I think one needs to ask oneself why you take some jobs. Is the money, the challenge or the purpose. I find that money and challenge has little to go for me if the purpose is skew.
mcintyre1994 3y ago
I’m really curious how Uber China was actually supposed to work. I once worked for a company that built server software and my understanding was that they were partnered with a Chinese company to sell into that market. We built a branded, localised installer for them. I’m not sure what the business arrangement was or what equity relationships (if any) there were between the companies. But I can’t imagine Uber would ever be allowed to have a fully owned Chinese subsidiary competing in that market?
- avidiax 3y ago
  > I can’t imagine Uber would ever be allowed to have a fully owned Chinese subsidiary competing in that market You imagine correctly, to my knowledge. Western companies in China have to be 51% Chinese owned, on paper. In reality, they own 100% (plus or minus diplomatic relationship issues), they just haven't asserted this right yet. This is just my opinion: Uber could never have won in China, and Uber's management likely knew this, but the story of China was needed at the time to drive company valuation and allow the "hockey stick" dashed line of future growth to be plausible.
hddqsb 3y ago
Tangential, but regarding footnote 7 (efficiently recomputing cells, assuming no cycles), the correct algorithm for determining the order is topological sorting (https://en.wikipedia.org/wiki/Topological_sorting https://en.wikipedia.org/wiki/Topological_sorting). Using a breadth-first search gives a good approximation in many cases, but there are pathological cases where it can produce an inefficient order. > Uber Sheets Hee hee, this name is funny on a number of levels :)
mihaaly 3y ago
Too many eager geeks are pushing through with realization of things that are a raw (not even half cooked!) droppings of a clueless mind with authority, instead of just simply saying 'No!' - or giving alternatives at least. Driven by pride doing things that should not have been attempted the first place. Lots of waste, lots of waste, lots of half cooked thingies interacting into unreliable solutions.
MrDresden 3y ago
This just gave me chills. Never would I want to work on a tool like this from scratch, and alone for that matter, that was underpinning crucial financial planning for a giant operation like the one described in the article. There are simply too many potential problems that could come up in this scenario and be the cause for catastrophic financial results in the end. And most of them wouldn't nessecarily be easily found via unit tests.
fnwbr 3y ago
given this website’s name it’s incredible how deeply neoliberal posts’ contents, comments and discourse have gotten… y’all seriously believe that the only valid reason for an engineer or a programmer to exist is to “create *bUsiNess~ value”? i’m shaking my head. this blog post is full of ideological blindness.
oq_pmg 3y ago
Questions at the end of the article, are the ones, every engineering manager should ask
junon 3y ago
Hey I worked with Matt Basta at Uber, cool guy and great engineer. His interview question for me was about this exact thing - how one might build excel from scratch. Neat to see him here :)
roarcher 3y ago
The intended lesson of this article may be the "cattle, not pets" point made near the end, but I think there's another lesson hidden in it. This is the story of literally every single large project I've done under a tight deadline in my career: - Some non-technical department (usually sales) promises a customer something very big very soon, with no regard to how much time might actually be needed to build it. Fulfilling this promise "on time" is now your problem, Engineer. - Months are spent frantically building said Thing, working overtime, burning out engineers, and bastardizing the previously clean codebase in the rush. - The deadline is met, and no customer uses Thing for at least several months. Or the deadline is not met and customer waits for Thing as long as it takes, because they're not going to blow up a big contract or integrate with your competitor instead of waiting another month. They have too much invested and the deadline wasn't that important anyway. The third possibility, as it was in OP's case, is that Thing gets thrown away in its entirety due to some fateful turning of the organizational wheels. I swear, almost as if by some karmic law of the cosmos, literally every frantic deadline turns out to be irrelevant in the end, and you should have saved your sanity and gone home at 5 every day for the last 6 months.
camgunz 3y ago
I just finished doing this. I built an Excel formula parser and interpreter, and then a transpiler into Python. I even had the experience of having the values off by "just a little" and there's so many things that can cause it, it's just whack-a-mole honestly. Makes me wonder how many times this has been done haha.
smrtinsert 3y ago
My favorite line: > On the other hand, we as engineers need to be real with ourselves. Every piece of code you write as an engineer is legacy code. Maybe not right now, but it will be. which of course leads to what some people are saying, did this have to be built? Sometimes I think our job is to optimize ourselves away.

753 points by robdimarco 3y ago

mrintegrity 3y ago

I get a strangely dystopian feeling from this article, like it's almost about a character in a black mirror episode

pagnol 3y ago
I reflexively swiped left when the inevitable newsletter modal started to appear at the bottom, so didn’t finish reading the article, but would like to know what gave you the dystopian vibe?
- Tao3300 3y ago
  You should have kept going. Spoiler alert: It turns out they were fulfilling a Babylonian prophecy the whole time. The whole development cycle was a complicated sacrifice to Marduk.
bastawhiz 3y ago
Man, what a commentary on my career
- lcnPylGDnU4H9OF 3y ago
  Alternatively, they just accidentally (I assume) called you a good storyteller.
Vt71fcAqt7 3y ago
Is it possible that you thought this because the words "black mirror(ed)" are mentioned in the article iteslf?
Tao3300 3y ago
Starring Excel as the pig.
jhaenchen 3y ago
That's because everyone in this comment section is giving wisdom about how to deal with the emotional reality of being abstracted from the fruits of their own labor. "Don't get attached to your work" is dystopian.

dublinben 3y ago

Is it common to take code written for one employer and reuse it for another?

arcbyte 3y ago
Some of us will strategically write generic code on our own time and machines and import it to employers machines when needed and customize it. For instance, how many times do you really need to write a spring OAuth server that integrates with LDAP? Or the guts of a simple CRUD app?
- Kwpolska 3y ago
  > how many times do you really need to write a spring OAuth server that integrates with LDAP? I would expect this to be a library somewhere. > import it to employers machines How would you prove that you write the code long before getting hired by them?
  - dartos 3y ago
    Someone wrote the library for a reason, right? Many are made bc people needed it for their job.
  - sokoloff 3y ago
    > I would expect this to be a library somewhere. Indeed. That library is often written off-the-clock by person X and imported on-the-clock by person Y. Sometimes X === Y.
    - bastawhiz 3y ago
      And sometimes that person even writes about it on substack
  - soperj 3y ago
    > How would you prove that you write the code long before getting hired by them? Commit it to a repo somewhere.

Znafon 3y ago

"Nothing came of it, but I took the code and shoved it into my back pocket for a rainy day. My idea was to take this code and spruce it up for Uber’s use case." "My first reaction was to publish the code on Github." I’m very surprised by this, isn’t the code property of Box, or Uber? The author does not mention their authorisation before releasing it under MIT license.

sidewndr46 3y ago
I believe this kind of story is the kind that gives most legal counsel nightmares.
- pc86 3y ago
  Especially the brazenness with which the author basically says "if they want to sue me for this verified and admitted IP theft, they can." Sure, they probably won't. But they might. And if they do, you'll lose immediately. Seems like a pretty high risk no reward scenario.
  - sokoloff 3y ago
    And if you lose immediately, you likely owe damages. Those damages, even if trebled, appear to be $0 here.
    - pc86 3y ago
      Which would be a great solace to someone who just spent $10k or more like 2-3x that responding to a lawsuit. But that's also why I agree the odds of actually getting sued are near-zero.
      - Alupis 3y ago
        Some companies, armed with floors of attorneys and retained outside counsel, do that sort of thing just for the message alone. It costs them next to nothing, relatively, ruins the defendant regardless of outcome, and makes it clear for others to not mess around with IP.

renewiltord 3y ago

That's a tremendous war story. I _love_ it! Doing a thing that's considered unreasonable. Knowing that total value is integral of instantaneous value over time (so short high-impact projects can easily outvalue long-lived projects).

constantly 3y ago

I wouldn’t try moving code written for one employer and using it another employer. Unfortunately for me my employers have been somewhat litigious; thankfully Uber doesn’t have that reputation.

itsthecourier 3y ago

"When I met the members of the Crystal Ball team, it was about roughly ... people" how many people there were at that point, @bastawhiz?

romnon 3y ago
people be rough
bastawhiz 3y ago
Sorry, obvious typo! It was four people.

erulabs 3y ago

I read this article looking forward to the complex bespoke code to be ripped out and deleted - but the author clearly grew as an engineer in a way I didn’t expect: > Sometimes that’s just how it is. The devops saying “Cattle, not pets” is apt here: code (and by proxy, the products built with that code) is cattle. It does a job for you, and when that job is no longer useful, the code is ready to be retired. If you treat the code like a pet for sentimental reasons, you’re working in direct opposition to the interests of the business. A lot of code is fun to write. A lot of problems are fun to solve. But a business, especially a startup, needs to stay razor focused. My entire career is effectively to sit in meetings and tell young, passionate engineers not to build things. It’s a bit depressing, but it’s also vital. A good engineer can solve any problem with clever code. A great engineer knows what problems aren’t really problems and probably an XLS download link updated daily would have been fine.

Retric 3y ago
Exactly, in 2016 there was several off the shelf options for doing the exact same thing. It’s a perfect example of a young engineer feeling a huge accomplishment from reinventing the wheel, and then realizing the clever solution wasn’t actually worth anything like the effort required to create it. I had a long conversation to convince someone not to go down that path in 2006, and I am sure someone’s going to do it in 2026. Pausing to think: I wonder how someone else solved this exact problem is such a huge part of how you grow as a developer I wish schools would focus more on it.
- yard2010 3y ago
  I would say what you talk about is experience. To have an experience, you must go this path to realize what to not do. It feels like a catch 22 kind of thing.
  - Retric 3y ago
    Doing it well comes down to experience, but doing it at all comes down to asking which of your unconscious assumptions are hard requirements. Nobody is actually saying you have skills X, Y, Z, which you must use to solve this problem, which is a huge difference from how schools prepare people for the workforce.

itsthecourier 3y ago

I'm glad I read this article and learned about the circ

sr228822 3y ago

ironically, I was backend at Uber 2014 - 2018 and used to ask a simple version of implementing an excel formula engine as my go-to coding interview question. Its got a nice mix of data-structures, algorithms, complexity, and implementation. Good candidates can get a reasonably efficient implementation handling cell references in an hour. nice read. made me nostalgic for the wild days of Uber-China hacking.

bastawhiz 3y ago
Amazing! I independently had an extremely similar interview question (because of this work) that also gave me great signal. That's really funny!

Reubend 3y ago

Love the part about circular references. I gotta say though - I'm having a difficult time imagining how complex these fomulas are that reimplementing Excel's formula engine is easier than just porting the formulas into JS.

bastawhiz 3y ago
The good news is that formulajs had a huge number of those functions implemented in JS already. Almost all of the time was spent on the engine, which wasn't a huge amount of code. The problem with porting the code to JS is that a.) nothing is named, b.) there's no real way to organize the code you've written because you're going from a spatial way of organizing code to imperative script, and c.) the actual design of the spreadsheet wasn't known to any engineers (it was designed by a data scientist, or perhaps an analyst). The work of translating would have meant really understanding what the thing is so that it can be turned into functions and modules. It also would have still required getting Excel function equivalents, since there's not a 1:1 equivalence between Excel and what's available in the JS standard lib.
xen0 3y ago
Honestly, if you're at 'thousands of formulae' across multiple sheets, I'd probably suggest writing a graph based execution engine too. The circular reference thing would have definitely thrown me for a loop though.
- kirubakaran 3y ago
  I just want to let you know that the pun in the last line didn't go unappreciated.

Vt71fcAqt7 3y ago

I love this article. It begins setting the scene with Uber's grand expectations for the chinese market and then shows just a small piece of the work Uber spent on Uber China. Meanwhile Uber China itself failed spectacularly, with factories of phones claiming the free rides money.[0] I don't think I've ever experienced dramatic irony in a blog before, certainly not a technical blog. The post itself is a masterpiece. [0]https://www.forbes.com/sites/ywang/2016/09/27/ghost-drivers-are-just-one-of-uber-chinas-problems-following-didi-takeover https://www.forbes.com/sites/ywang/2016/09/27/ghost-drivers-...

gobrrrmeme 3y ago

There's a service in SharePoint and SharePoint Online that let you program against Excel. Just gonna leave this here. https://learn.microsoft.com/en-us/sharepoint/dev/general-development/getting-started-with-excel-services https://learn.microsoft.com/en-us/sharepoint/dev/general-dev...

acchow 3y ago

> Over the summer of 2016, we came up against a new twist on the project. We had a model that ran overnight to generate data for anticipated ridership in China. That data wasn’t useful on its own, but if you fed it into a tab on a special Excel spreadsheet, you’d get a little interactive Excel tool for choosing driver incentives. Our job was to take that spreadsheet and make it available as the interface for this model’s data. They eventually built a homegrown "Excel" clone as the UI for their model because "city teams only know how to use Excel". I would have done it the other way around - connected Excel to the data output by the model so the "city teams" could continue to use real excel. I think most finance teams do something like this.

ftxbro 3y ago
> I would have done it the other way around - connected Excel to the data output by the model so the "city teams" could continue to use real excel. Yeah except: “When you click in the cells of the spreadsheet you can see the formulas. You shouldn’t be able to do that.” “You said to make it just like Excel.” “People working for Didi apply for intern jobs at Uber China and then exfiltrate our data. We can’t let them see the formulas or they’ll just copy what we do!”
bastawhiz 3y ago
Because the city teams were in China, we didn't have this luxury. Everything had to be behind Uber's beyondcorp equivalent, and there was no real way to auth folks from the Chinese mainland. Our only surface was the browser.
- MichaelZuo 3y ago
  A dedicated satellite link cross pacific is not that much money. Then again maybe the CFO didn't know so he took the easiest known option.
  - bastawhiz 3y ago
    I've always been fond of container ships loaded up with entangled qbits. Much lower latency!

pc486 3y ago

What a fantastic perspective from the former Uber BI team. I was on the Vertica team during this time period and the amount of effort was spent on incentives mind boggling. Millions a day lost on downtime, product features, or engineering bandwidth was a common theme. A director asking for an exact spreadsheet to be the UI would have been par for the course, especially during the Uber China days. Heck, I personally loaded FX prices into Vertica from a spreadsheet emailed every month to the team. That process remained for more than a year as there just wasn't enough bandwidth to invert the control as automated ingestion. Thanks for digging up these memories, @bastawhiz. I'd love to see more. :)

Macha 3y ago

So a lot of my time as a more junior engineer was spent on a similar project that I've described as "rebuilding excel". In my case, it was in the form of a table widget. It was an inhouse widget for displaying tabular data. We were working on AngularJS 1, and had moved from a prior iteration of our application in ExtJS. Now ExtJS had a reasonably featureful table widget out of the box, and so when we were porting to AngularJS, it was considered important that we continued to have these UI features, but at the time (the Angular ecosystem was relatively immature at that time) there were not very many advanced table widgets available in open source, so we ended up building our on. However, while the ExtJS table widget had been treated by product management as pretty immutable "this is what ExtJS gives us, we can customise the colours and wording and that's it", the idea that we could customise the table widget started something amongst one of the PMs. And so we would get a constant stream of feature requests for the table widget to add stuff and enhance it and soon we were significantly more featureful than the ExtJS widget. It's still, to this day, the most featureful table widget I've seen in a web app. Everything Excel had in terms of resizing tables, sticky columns, scrolling behaviours, sorting, filtering, searching, etc., all saved in your config so it was synced across all your devices, as well as all the performance goodies like recycled rendering etc. The constant stream of feature requests meant there were dev team years invested in this table widget. As a more mature engineer looking back, it's clear that at some point this had stopped being about customer value and more about one PM's obsession with getting excel like functionality in a in-browser reporting tool, but at the time we just kept building those features. Now at this time, our company had acquired a sort of competitor of ours. This competitor had what was effectively the same product, but in a different market. And so the first merger of the functionality was basically to reskin both applications so they would pretend to be tabs in a unified application, and change some terminology, etc. They actually did happen to have a pretty similar tech stack to us, so some newer components were available in both applications. But it became clear that our users were not happy with two applications pretending to be one. They wanted to know why this other market was not accessible by, say, a dropdown in the configuration, and not an entire application which worked in different parts from subtly differently to entirely differently. So the discussion became about building a ground up unified interface for both of them. Of course, this ignited the discussion of "which table component do we use?". On the one hand, the acquired team were looking at our table, with it's fifty billion options and single handedly accounting for half the page weight of the minified JS of our application and did not want something so bloated. On the other hand, our team were looking at their table widget which was effectively "for row in data, for column in row, print td" and dreading having to rebuild all these features for product management again. Ultimately the conflict was resolved by choosing to use an open grid that had less features than ours but more than theirs and telling the PM in question that table features were going to be prioritised much less heavily from then on unless there was a real user need for it.

deleted 3y ago

[deleted]

abeppu 3y ago

I'm confused about the circular reference thing. Like, was there a reason to do the linear regression that way? Is there a secret story in a story where next week, in a spinoff / sequel episode, the data scientist responsible will explain why they took the weird/surprising choices they did?

andy81 3y ago
I used it once to run a Monte-Carlo simulation in a spreadsheet. After enabling iterative calculation and manual calculation, every press of refresh runs a loop. Fun stuff.
DwnVoteHoneyPot 3y ago
It's a common excel trick in finance. For example, let say you have $0. If you borrow $1,000,000 at 5% interest, by end of year you'll be short $50k. That means you actually needed to borrow $1,050,000. But the extra $50K causes more interest ($2,500)... so you needed to borrow $1,052,500, which causes more interest... and so on. Instead of doing some Excel Goal Seek or Solver or VBA macro, it's nice to let the excel "reactivity" handle it for you.
- hu3 3y ago
  Your explanation is brilliant!
DrDroop 3y ago
I didn't know you could do this in excel but it reminded me of the converge adverb in k: https://xpqz.github.io/kbook/Adverbs.html#converge-f-f https://xpqz.github.io/kbook/Adverbs.html#converge-f-f
- itigtohft 3y ago
  https://en.wikipedia.org/wiki/Fixed-point_iteration https://en.wikipedia.org/wiki/Fixed-point_iteration

brap 3y ago

> He simply couldn’t believe that I’d written a full spreadsheet engine that ran in the browser. I can't believe it either, and I don't mean this in a good way. Apache POI lets you run headless Excel. You import and interact with sheets programmatically in Java. We used this in my old workplace for exactly the same reason (functions, cell references, the whole thing), it worked great. You found the ‘circ’ problem with a bit of luck. What about all of the other hidden little quirks of Excel that you would ultimately run into down the road? Are you really going to build and maintain a full blown Excel clone in JS? Is this really the objective of the frontend team? It seems to me like a bit of googling and >90% of the work here could have been avoided. As an added bonus it would have been done by the backend team instead.

rkangel 3y ago
To be fair, he wrote a spreadsheet engine that could run one particular spreadsheet. Admittedly a complex one, but it was a fixed set of functions that he needed to implement and not and endless tail of things that people expect Excel to do. I think I'd probably have argued more about the UI spec and down some Excel behind the scenes but it is a familiar UI if you've got lots of number inputs all over the place. I've always enjoyed this article about building a spreadsheet in 100 lines of F#: https://tomasp.net/blog/2018/write-your-own-excel/ https://tomasp.net/blog/2018/write-your-own-excel/ The expansion from that to the feature set needed here is manageable.
- brap 3y ago
  Project requirements are never fixed, they’re always evolving. It was only a matter of time before users would’ve complained about features being missing/broken, especially since what they’re used to is Excel and this was meant to replace it.
  - bastawhiz 3y ago
    > this was meant to replace it. I think that's a very generous read of what I said the requirements for this product were.

kazinator 3y ago

> You see, when formulas create a circular reference, Excel will run that computation up to a number of times. Like almost every spreadsheet before it: Lotus 1-2-3, Borland Quattro Pro, VP Planner ... Spreadsheets iterating on circ references goes back to the 1980s. The first spreadsheet application, VisiCalc, didn't track dependencies: it evaluated cells left to right, top to bottom, IIRC. Microsoft had a product called Multiplan that competed with VisiCalc. Not sure if that did iteration. I think it used to be a setting in some programs whether circular references are flagged as errors, or iterate. Maybe it's still that way in Excel?

itigtohft 3y ago
There are a couple of nice minimal spreadsheets in js: http://web.archive.org/web/20130606222859/http://thomasstreet.net/blog/spreadsheet.html http://web.archive.org/web/20130606222859/http://thomasstree... Inspired by that, an even smaller one: https://jsfiddle.net/ondras/hYfN3/ https://jsfiddle.net/ondras/hYfN3/ The latter catches circular references rather than trying to calculate the fixpoint

ChrisMarshallNY 3y ago

> It’s easy to treat a particularly clever or elegant piece of code as a masterpiece. It might very well be a beautiful trinket! But we engineers are not in the business of beautiful trinkets, we’re in the business of outcomes. This spoke to me. However, as anyone that has looked at my code can attest, I tend to also want my code (and its functionality) to be very pretty. I'm generally writing code that I will be maintaining, so it needs to be something that I can look at, in a year, and understand. I'm currently in the final phases of a project that I will never announce here, and don't plan on taking much credit for, but it really is da schizz. It's that way, because no one is paying for it, and no one will make money from it. Money both spoils everything, and also makes it all happen.

allanrbo 3y ago

Another approach would be to use the Excel APIs. Both the classic desktop Excel and the web version has APIs to read/write cells and recompute. Rebuilding is more fun of course :-)

bjornlouser 3y ago

"'Growing as an engineer' means becoming a better engineer, and becoming a better engineer (directly or indirectly) means getting better at using your skills to create business value." Learning how the Excel model worked and then reimplementing it would have been a better example of 'getting better at using your skills to create business value'.

bsder 3y ago

Side question: Why the hell is it so stupidly difficult to display and edit tabular data from programming code? You can't drive Excel from Python/Rust/etc (Microsoft just announced to great fanfare that Excel can call Python--which is the wrong way around). All the editable table widgets for the web seem to suck. Nobody seems to have a TUI which you can drive from an external program. A poorly written spreadsheet with a driveable API seems like a component that has been built multiple times by lots of people yet seems to be unavailable. Is there some solution that I'm missing?

kubanczyk 3y ago
> Microsoft just announced to great fanfare that Excel can call Python--which is the wrong way around How do you cause users to even discover that Python app, or feed an input to it? Of course the spreadsheet software is an obvious candidate for the primary UI.

easton 3y ago

Excel circular reference doc, for the curious: https://support.microsoft.com/en-us/office/remove-or-allow-a-circular-reference-8540bd0f-6e97-4483-bcf7-1b49cd50d123 https://support.microsoft.com/en-us/office/remove-or-allow-a... > Unless you're familiar with iterative calculations, you probably won't want to keep any circular references intact. If you do, you can enable iterative calculations, but you need to determine how many times the formula should recalculate. When you turn on iterative calculations without changing the values for maximum iterations or maximum change, Excel stops calculating after 100 iterations, or after all values in the circular reference change by less than 0.001 between iterations, whichever comes first. However, you can control the maximum number of iterations and the amount of acceptable change.

cush 3y ago

Any PM that walks in with “Just make it like Excel” hasn’t spent a moment writing code.

bastawhiz 3y ago
Even worse: he was a director
- cush 3y ago
  Arguably the most battle-tested, feature-rich program on the planet. Yeah just make it like that.
  - bastawhiz 3y ago
    I should tell the story of the PM (also at Uber!) who demanded our web app be "faster than Excel". I couldn't make it up if I tried.

sha16 3y ago

> “take this [the spreadsheet] and put it in on the website [Wesley].” This should be taught in classrooms.

DubiousPusher 3y ago

Just a heads up for anyone who finds themselves with a similar requirement, there's a very robust set of office APIs available in .net. I would be surprised if you couldn't open and run a whole workbook with them though I have only used them for more tangential tasks.

JohnMakin 3y ago

Reading this and having China cloud experience on my resume I wonder how many chinese data laws they may have violated, you have to really strictly segregate data when it comes to chinese users.

blincoln 3y ago
I was wondering the same thing, especially since it arguably involves mapping/geolocation information.

dmd 3y ago

> So the data scientists have multiple laptops that they download the data to, then run the models overnight. This haphazard way of running compute jobs really stuck out to me. I can't imagine doing things this way (rather than having a central compute cluster running SLURM or similar) at a company bigger than, say, a dozen people - much less the scale of Uber. What's the rationale? Even if it's just a cluster of 3 or 4 machines in a rack shoved in the corner, isn't that better than ... laptops?

bastawhiz 3y ago
It was easy to go to IT and say "get us a pile of laptops" and let the data scientists do their thing. It was hard to hire engineers to solve the problem (I was one of the engineers). They particular problem was far enough down the priority list that it took until 2016 to solve.
- dmd 3y ago
  Heh. Well, I guess... sorry I didn't come help :) (I interviewed but turned down my offer.)
  - bastawhiz 3y ago
    You dodged a bullet

lph 3y ago

Excel is an ancient and complex beast. I get the appeal of building this project---it sounds fun---but trying to duplicate the Excel engine to the level of producing identical outputs is, frankly, bonkers. The author caught the one discrepancy they noticed, circular reference handling, but how many did they miss? How do they know different inputs won't cause it to deviate from Excel? I didn't get the sense from the blog post that this had extensive test coverage. Putting it into use for a business-critical financial calculation is a massive risk, but I guess that's how Uber rolls ¯\_(ツ)_/¯ It would have been less fun but way, way less risky to wire a headless Excel up to a javascript front-end.

bastawhiz 3y ago
No Windows, no Excel license, hundreds of concurrent users. How confident are you that each user is getting the calculations that they triggered and not someone else's? How are you deploying that spreadsheet? How are you version controlling the mutable parts of the system? How confident are you that the solution would be robust in the face of bad inputs and high load? I won't say there wasn't risk, but there was quite a bit of testing and a human always made the final call anyway (I never fully understood why we didn't eliminate humans from the processes altogether).

ryukoposting 3y ago

I wonder if the author would view this situation differently had Uber/Box decided to claim the code as their own. It has to bring some catharsis to know that, even if the code never actually met its potential, at least the whole world can see and appreciate it. I created a whole programming language as an intern for <defense megacorp>. It was lazily evaluated and garbage collected. Unquoted MAC addresses were valid syntax, among other application-specific oddities. No bytecode or JIT shenanigans - the interpreter just pushed and popped stuff from a stack as it traversed the parse tree, and that was fast enough for what we were doing with it. The interpreter was written in pure ANSI C, and Valgrind was very happy with it. Maybe it has been totally forgotten, or maybe it became critical to their technical infrastructure. That code never left the airgapped lab where I wrote it, so I have no way of knowing. 3 years ago, as a recent college grad, that was by far the coolest piece of "actually useful software" I had ever written. It's still high on the list. Sometimes I wonder whatever happened to it.

lowbloodsugar 3y ago
The part the author is missing is that Box and Uber have already claimed the code as their own. It will be in the employment contract. The author appears to be under the impression that asking a mid-level manager (or even a senior manager) "Hey, do you want this?" and that manager answering "No", is in any way legally binding on the company.
- ryukoposting 3y ago
  I've only ever agreed to those terms kicking and screaming. I wish there was a larger part of the community willing to advocate for a "love of the art" clause that would explicitly state that code I wrote on my own time is mine.

joshu 3y ago

i feel like “learn vba” would have been a much easier path here

OJFord 3y ago

> having answers that were very, very close is objectively worse than having numbers that are wildly wrong: very wrong numbers usually always mean a simply logic problem. Almost-correct numbers mean something more insidious. Ha! I massively identify with this. 'The market-wide median is off by £3.07' is a much harder but to crack than the 'worse' report that 'the median is twice the max and the min is null'.

OJFord 3y ago
*bug to crack

stevebmark 3y ago

A good and related article about the dangers and poison of domain driven design, and why spreadsheets are used at every level of every company. https://dev.to/cheetah100/domain-driven-disaster-147i https://dev.to/cheetah100/domain-driven-disaster-147i

ineedasername 3y ago

These are the times that, if you at least enjoy the process of coding, you should try to look at your work as a videogame. You enjoyed the game, you even got paid for it, and you beat the final boss, it doesn't matter that no one will ever see your final score.

ipython 3y ago

Best quote: "People working for Didi apply for intern jobs at Uber China and then exfiltrate our data. We can’t let them see the formulas or they’ll just copy what we do!” This is so true. People in the US just don't understand the level of economic and industrial espionage that happens in China on a daily basis. I was responding to an unrelated breach at an unnamed tech company back in mid-2000s time frame and had a side bar conversation that went like the following: Them: "Yeah, we just opened a tech center in Xinjiang and ... wow, we've had quite the rash of lost ID badges there recently" Me: "Have you considered that they're not 'lost', but rather 'sold' for profit?" ... silence ... I don't know if executives are aware but just don't care, or if they're simply incompetent, but China has productized industrial espionage on a massive scale. GE Aviation was a victim more recently: https://www.cincinnati.com/story/news/2022/11/16/accused-chinese-spy-who-targeted-ge-aviation-sentenced-to-20-years/69653460007/ https://www.cincinnati.com/story/news/2022/11/16/accused-chi...

cflewis 3y ago
Well, not just corporate espionage, right? State espionage is going to be at every large US company too. To speak to the article, I can only imagine how excited an agency would be to get real-time updates on the Uber movements of a target.
- ipython 3y ago
  Two things. First, this isn't the same thing; you're talking about embedding a resource in a company to read out real time telemetry from internal systems (information with a short shelf life), not stealing industrial trade secrets (information with a very long shelf life). Second, I can assure you that, even with our imperfect systems, there is actually a set of checks and balances in place to prevent rampant (note I said rampant) abuse of this kind. Yes, agencies would be very excited for this sort of capability. Do they get it as a matter of course that easily? No. There are layers of accountability, legal authorities, and (warranted) push back from commercial entities.
  - BlueTemplar 3y ago

nardi 3y ago

I do not like working with people like this, because I don't want to be anywhere near the maintenance burden of a pile of hacks that technically can run a single Excel spreadsheet. I don't even want to be within earshot of people complaining about this. If I even thought about it for too long I'm sure I'd have a giant headache.

ndesaulniers 3y ago
I worked with the author for maybe 3 years, I thought he was pretty awesome. Not surprised he implemented a partial implementation of excel so quickly. I'm disappointed in myself that I somehow got through most of the article without realizing who the author was (thanks Firefox reader mode); especially since the naming of Wesley and Crusher is too good. Of course it was Basta! Of course it was!
abigail95 3y ago
what would you have done instead?
prasoonds 3y ago
In a startup, you either build, ship and you survive another day, or, you die. If I were hiring for an early stage startup, I know exactly which of you two I would want to hire.

CodeWriter23 3y ago

> Someone will take joy in ripping it out someday.

purpleblue 3y ago

1) This was the type of effort and level of coding that I saw when I was Uber during its heyday. I love it and I miss it every day. 2) You stole Box code, used it at Uber, and then stole Uber code and posted it on github. I understand no one's using the code or missing that code, but you really did steal that code. It belongs to the company, not to you. I would be careful not to do that in the future, because technically that's a trade secret and people have gone to jail over that, like that programmer at Goldman Sachs (wrongfully) and ironically Levandowski who took Google code and tried to use it at Uber.

lowbloodsugar 3y ago

IANAL but I've paid lawyers to answer this question in the US. The question is, if you do some work, in your own time, on your own equipment, does your employer own it just because the employment contract says they do? In California: if the work in any way relates to the employer's business, then yes, they own it. One way to guarantee that it relates to the employer's business is to bring it into the office and use it as part of your job. If your employer is Apple or Google or AWS or Microsoft, then probably anything you write would in some way relate to their business. Write spreadsheets by day, but games by night? All of those companies make games, or are in the games business. I would love to hear a lawyer say, "Well, it doesn't matter what the employer does, it only matters what your job duties are, so writing games at night is fine if they don't pay you to work on games related things during they day." But I've never been told that by a lawyer, whether I paid them or otherwise. Everywhere else in the USA: they probably own it. You could write software for washing machines, and write a video game, and if your contract says they own everything you write then they do. You signed it. There's no "but surely not!" defense.

mkii 3y ago

> "In 2015, I had built a prototype of a tool at Box. Box had a collaborative note-taking product called Box Notes (based on Hackpad). I had the idea to make a similar project for working with numbers: sometimes you didn’t need a full spreadsheet, you just needed a place to put together a handful of formulas, format it with some headings and text, and share it with other people. Sort of like an ipython notebook for spreadsheets. I called it Box Sums." > "Nothing came of it, but I took the code and shoved it into my back pocket for a rainy day." ... > "Apparently that was a thing. I remember being only half-surprised at the time. I hadn’t considered that our threat model might include employees leaking the computations used to produce the numbers in question." ... > "My first reaction was to publish the code on Github." Perhaps the author feels "only half-surprised" due to their own disregard for corporate legal ownership of code. The hypocrisy is strong here.

lxe 3y ago

> To this day, I’ve never encountered an in-house application system as well-designed as Uber’s. You could go from start to Hello World running on a *.uberinternal.com subdomain in under 30 minutes with full CI/CD. I worked on all this at Uber back then, and this comment warmed my heart a bit. Thank you.

charles_f 3y ago

> “Every day that we don’t have this tool as specced, we’re losing millions of dollars.” There was no budging on the spec. If this is life or death, then the first reflex should be to find ways to make things simpler to ship. I understand that the requester in this case was a head of finance, but sh*t, if you want something, try to compromise once in a while. This is a great story, but at some point as an engineer in a gig-selling company, I would have asked myself if my job was really to re-build excel.

alzaeem 3y ago

I interviewed with Uber in 2016 with the "crystal ball" team. Reading about R-crusher, I am glad that offer never came through!

StopHammoTime 3y ago

I find it remarkable that they used laptops for this. Surely, a Dell server for $100k with good specs could have crushed a lot of these models. Could have just given them all Remote Desktop sessions too.

schlowmo 3y ago

> “The city teams only know how to use Excel, just make it like Excel.” > “Why can you see the formulas?” > “You said to make it just like Excel.” I can't keep up with the espionage story that followed but I had this conversation more than once in my professional career. The first time I sat as a junior dev on a multi-month project replacing a excel spreadsheet for financial controlling of data centers that only one person who was going to retire understood with a web-based solution. They were quite proud that they were going to get a "modern" solution. Then they wanted me to make it like excel. What followed was evaluating every fricking excel JavaScript library out there at this time, going for one and started duct taping all the missing pieces. They were pleased but the look was off. It wasn't excel. I slapped some styles on it coming quite close. I was not prepared for what happened during the next presentation: They hated it because they wanted a modern web based solution (their words, not mine) and what they got was a poor excel knockoff running in their Internet Explorers. Tables are so 90s. I remember the pain so vividly that I regard "just make it like excel" as some kind of forming meme for my career till today.

ww520 3y ago

I had a somewhat similar requirement in a past work. The Excel file had all the formulas and data, and the analysts only work with Excel files. The result of the calculation in Excel needed to be put on the web. The solution was not to recreate Excel in the browser, but ran the Excel file with its formulas and its data plus the input parameters from end users at the backend server. Apache POI was a nice Java library that could do everything on an Excel file. Once it finished the formula calculation, I just read the cells from the Excel file to extract the result data and generated the web pages to present the data and graphs. One nice benefit was the analysts could update their work in the Excel file, uploaded it to the server, and got the new calculation reflected on the web pages right the way.

whimsicalism 3y ago

Funny that this article is talking about interns exhilarating data and yet the author blatantly recycles code between employers.

ec109685 3y ago

This was an amazing read. One of my favorite memories in college was building a spreadsheet in Java (some of the guts were there) in third semester CS.

jengler 3y ago

Great to see @bastawhiz on here. Box had a collaborative note-taking product called Box Notes (based on Hackpad). Slight correction: Box Notes was forked off etherpad-lite. Hackpad was a parallel fork of Etherpad. [context: I was lead engineer on Box Notes till around 2014/2015. I left before the events @bastawhiz described]

NavinF 3y ago

>You see, when formulas create a circular reference, Excel will run that computation up to a number of times. If, in those computations, the magnitude of the difference between the most recent and previous computed values for the cell falls below some pre-defined epsilon value (usually a very small number, like 0.00001), Excel will stop recomputing the cell and pretend like it finished successfully. W T F

hannofcart 3y ago

On reading this article, I realised that maybe the most noticeable change that experience brings to a developer is... laziness? When I was 24, I was pretty much exactly this person. "Building a spreadsheet engine that runs in a browser. Sounds like so much fun!" And then I'd slog away at it for a month and get something working. Now though, with age, I know I can't chug away coffee working late into the night. My back hurts when I have to sit too long. My wrists aren't being too kind to me these days either. Now if someone asked me to build Excel I'd first laugh in their face. And if I don't see them smiling, I'd ask if they can afford any of these A,B,C...Z COTS products that are doing this that we can just buy. And if none of those work out, I'd look for how I can take something like ethercalc and repurpose it for our use case. Looks like the older I get, rather than writing code, I seem to be getting more adept at how not to write it.

mathgladiator 3y ago

Beyond the questionable IP theft. The lesson of what you build for a company as something that may be thrown away is a good one. However, I believe the only way to actually have a sacred masterpiece is to own the company and make the company's mission to build that sacred masterpiece. For example, there are definitely business opportunity for converting excel sheets to online apps that are collaborative. The core problem is alignment between the tech investment and the company's goal.

nperez 3y ago

I've worked on a project that was intended to replace Excel spreadsheets with a web front end. It was presented like we were moving our processes out of the stone age - as if we were replacing a pen-and-paper process that "doesn't scale", or whatever. I will never do that again. We didn't build a whole spreadsheet engine, but we did build a web UI that simply doesn't do as much as Excel. Excel is powerful. Sometimes it's a fine tool for the job. Our team was laid off before we could roll it out, but I remember the growing sinking feeling of "I would hate using this if I already had a bunch of habits built up in Excel"

PeterStuer 3y ago

Running the models on laptops overnight? They know desktops and servers exist right?

choppaface 3y ago
But when you have to be your own IT / devops... and when maybe all you know is R, some stats, and how to admin a laptop... The head of finance may have burned most of the opex budget and devops bandwidth on the Vertica and/or other DB service(s) they were using.
avidiax 3y ago
Uber, at that time, was "scrappy". Every operating city was a different market with different rules, so Uber was akin to a franchiser that gave their market managers wide leeway to setup whatever business processes are needed to succeed in that market. Excel is the easiest way for business-oriented people to make numerical decisions, run incentive campaigns, segment riders and drivers, produce reports required by cities, counties, states, etc. The data teams providing market intelligence to the various markets, of course, knew about servers. But they would not have the time or skill to setup something like this, and the business still had daily data needs. So the laptops it was, for a time.
- PeterStuer 3y ago
  A laptop is basically a desktop with sustained performance thrown out the window in favor of sleek portability.

quijoteuniv 3y ago

Is interesting what people focus on when they read, ultimately I think one needs to ask oneself why you take some jobs. Is the money, the challenge or the purpose. I find that money and challenge has little to go for me if the purpose is skew.

mcintyre1994 3y ago

I’m really curious how Uber China was actually supposed to work. I once worked for a company that built server software and my understanding was that they were partnered with a Chinese company to sell into that market. We built a branded, localised installer for them. I’m not sure what the business arrangement was or what equity relationships (if any) there were between the companies. But I can’t imagine Uber would ever be allowed to have a fully owned Chinese subsidiary competing in that market?

avidiax 3y ago
> I can’t imagine Uber would ever be allowed to have a fully owned Chinese subsidiary competing in that market You imagine correctly, to my knowledge. Western companies in China have to be 51% Chinese owned, on paper. In reality, they own 100% (plus or minus diplomatic relationship issues), they just haven't asserted this right yet. This is just my opinion: Uber could never have won in China, and Uber's management likely knew this, but the story of China was needed at the time to drive company valuation and allow the "hockey stick" dashed line of future growth to be plausible.

hddqsb 3y ago

Tangential, but regarding footnote 7 (efficiently recomputing cells, assuming no cycles), the correct algorithm for determining the order is topological sorting (https://en.wikipedia.org/wiki/Topological_sorting https://en.wikipedia.org/wiki/Topological_sorting). Using a breadth-first search gives a good approximation in many cases, but there are pathological cases where it can produce an inefficient order. > Uber Sheets Hee hee, this name is funny on a number of levels :)

mihaaly 3y ago

Too many eager geeks are pushing through with realization of things that are a raw (not even half cooked!) droppings of a clueless mind with authority, instead of just simply saying 'No!' - or giving alternatives at least. Driven by pride doing things that should not have been attempted the first place. Lots of waste, lots of waste, lots of half cooked thingies interacting into unreliable solutions.

MrDresden 3y ago

This just gave me chills. Never would I want to work on a tool like this from scratch, and alone for that matter, that was underpinning crucial financial planning for a giant operation like the one described in the article. There are simply too many potential problems that could come up in this scenario and be the cause for catastrophic financial results in the end. And most of them wouldn't nessecarily be easily found via unit tests.

fnwbr 3y ago

given this website’s name it’s incredible how deeply neoliberal posts’ contents, comments and discourse have gotten… y’all seriously believe that the only valid reason for an engineer or a programmer to exist is to “create *bUsiNess~ value”? i’m shaking my head. this blog post is full of ideological blindness.

oq_pmg 3y ago

Questions at the end of the article, are the ones, every engineering manager should ask

junon 3y ago

Hey I worked with Matt Basta at Uber, cool guy and great engineer. His interview question for me was about this exact thing - how one might build excel from scratch. Neat to see him here :)

roarcher 3y ago

The intended lesson of this article may be the "cattle, not pets" point made near the end, but I think there's another lesson hidden in it. This is the story of literally every single large project I've done under a tight deadline in my career: - Some non-technical department (usually sales) promises a customer something very big very soon, with no regard to how much time might actually be needed to build it. Fulfilling this promise "on time" is now your problem, Engineer. - Months are spent frantically building said Thing, working overtime, burning out engineers, and bastardizing the previously clean codebase in the rush. - The deadline is met, and no customer uses Thing for at least several months. Or the deadline is not met and customer waits for Thing as long as it takes, because they're not going to blow up a big contract or integrate with your competitor instead of waiting another month. They have too much invested and the deadline wasn't that important anyway. The third possibility, as it was in OP's case, is that Thing gets thrown away in its entirety due to some fateful turning of the organizational wheels. I swear, almost as if by some karmic law of the cosmos, literally every frantic deadline turns out to be irrelevant in the end, and you should have saved your sanity and gone home at 5 every day for the last 6 months.

camgunz 3y ago

I just finished doing this. I built an Excel formula parser and interpreter, and then a transpiler into Python. I even had the experience of having the values off by "just a little" and there's so many things that can cause it, it's just whack-a-mole honestly. Makes me wonder how many times this has been done haha.

smrtinsert 3y ago

My favorite line: > On the other hand, we as engineers need to be real with ourselves. Every piece of code you write as an engineer is legacy code. Maybe not right now, but it will be. which of course leads to what some people are saying, did this have to be built? Sometimes I think our job is to optimize ourselves away.