7 ms·

This, 100%. That said, most data scientists don't do what you would consider real work (meaning, I assume, interesting work with significant mathematical/analyt

by psi75 4y ago

This, 100%. That said, most data scientists don't do what you would consider real work (meaning, I assume, interesting work with significant mathematical/analytical meat). There just isn't a lot that's both interesting and useful to private-sector rent-seekers whose opinions of your work determine whether or not you advance. Most of the people doing real ML in industry are prestige hires--they're hired because their names draw people in, but basically get to work on whatever they want--and you need a top-10 PhD at an absolute minimum to be eligible for those. The ugly truth about industry is that 99.9997% of it is flow capture based on power relationships, found artifacts (i.e., corruption opportunities) within the state, and the implementation of very simple processes but in a way such that the threat to executive reputations as a first priority, and profit as an important second one, are minimized. This doesn't exactly make a market for ML innovation, unless your boss for some weird reason still cares about being a co-author on your papers (which his bosses will pressure him not to let you publish, because after all, this publishing is a distraction from your paid work). On the other hand, if you want to be able to afford a house in the Bay Area, and to be tapped for (indeed, most likely forced into, both due to losing interest in and being unhireable for IC work) management in your mid-30s... then go for industry. The poison carrot will make you sick but it will kill you more slowly than poverty, so that ain't so bad, now is it?

cassepipe 4y ago
I really like your style. I think you should write a book (this is not sarcasm)
urthor 4y ago
Strongly disagree. There's a vast amount of work that doesn't involve unethical recommendation systems. Expand your horizon outside the Bay Area. The plurality of work I see is straightforward computer vision/NLP applications.
- natch 4y ago
  I suspect the work you're talking about could be easily handled by an intern working with Core ML and a MacBook. The landscape is varied. There are companies doing real actual big leading edge stuff, there are companies where ML is sprinkled onto projects as a buzzword but no real interesting work happens, and companies that just need a practical small solution like the ones you mentioned, and could get by with Core ML, but don't because they hire a PhD who isn't aware of Core ML.
  - carom 4y ago
    What does productionizing coreml look like if I wanted to stand a model up as an rpc service?
mrcartmeneses 4y ago
I’ve kinda developed the view that large organisations come to mirror the Russian Communist Party. I’m interested in “flow capture based on power relationships”. Do you have any recommended reading on this?
- badpun 4y ago
  > I’ve kinda developed the view that large organisations come to mirror the Russian Communist Party. Only the ones which have an unkillable cash cow. So, I suspect Google or large banks are mostly like that, but places like SpaceX or even large consulting firms (Delloitte, IBM etc., where managers essentially eat what they kill) cannot allow themselves to degenerate into a Chinese court.
  - bsenftner 4y ago
    Now this is interesting. I've always found it fascinating that when profit is on the table, democracy is nowhere to be found. I've looked, not too hard TBH, for essays and literature discussing the correlations to business model management structures and government/nation political hierarchies - not education level (propaganda), but critical analysis. I've been an employee of several of the top corporations on our planet, and the idea that corruption is not rampant is a farce. One simply lives within the environmental constraints and leaves when it gets to be too much. Does caring about corporate (and the larger realm of ethics) cast one incompatible with a modern corporate hierarchy?
    - psi75 4y ago
      Unfortunately, the only way to prevent hierarchy is to create a limited hierarchy (this is the purpose of constitutions) a priori; hierarchically naive organizations fail on this account. External parties will demand hierarchy simply because they want to know your organization (or nation) isn't wasting their time--no one wants to deliver a sales pitch to people who can't authorize purchases. If they're not careful, a group of people can end up in a state where the necessary-for-external-relations hierarchy becomes a total one. You see this with startup founders; the one who talks to the investors the most ends up in charge, and the ones who deal with employees or low-status counterparties lose power. This is why "flat" organizations can't really work; people who need things from the organization demand to know who to talk to in order to actually get things done, and eventually those "who to talk to" people end up with informal, then formal, power and it's very difficult to get them to give it back. The large-scale failure of democracy that's happening all over the world is something different, though. Regulation is struggling to keep up with technology, and it doesn't help that nation-states have already been doing a piss-poor job of protecting people from their employers. If the US falls in the next 20 years, it won't be due to Covid or Trump or nation-level adversaries; it'll be due to the obscene power given to employers, who can literally ruin an employee's life--not just fire him, but anally ravage him in perpetuity with bad references--for any reason or none. Eventually, unless national governments start dropping serious lead pipe on employers' heads, people are going to tire of paying 30+ percent of their incomes to a government that lets bosses get away with this shit.

by psi75 4y ago

cassepipe 4y ago

I really like your style. I think you should write a book (this is not sarcasm)

urthor 4y ago

Strongly disagree. There's a vast amount of work that doesn't involve unethical recommendation systems. Expand your horizon outside the Bay Area. The plurality of work I see is straightforward computer vision/NLP applications.

natch 4y ago
I suspect the work you're talking about could be easily handled by an intern working with Core ML and a MacBook. The landscape is varied. There are companies doing real actual big leading edge stuff, there are companies where ML is sprinkled onto projects as a buzzword but no real interesting work happens, and companies that just need a practical small solution like the ones you mentioned, and could get by with Core ML, but don't because they hire a PhD who isn't aware of Core ML.
- carom 4y ago
  What does productionizing coreml look like if I wanted to stand a model up as an rpc service?

mrcartmeneses 4y ago

I’ve kinda developed the view that large organisations come to mirror the Russian Communist Party. I’m interested in “flow capture based on power relationships”. Do you have any recommended reading on this?

badpun 4y ago
> I’ve kinda developed the view that large organisations come to mirror the Russian Communist Party. Only the ones which have an unkillable cash cow. So, I suspect Google or large banks are mostly like that, but places like SpaceX or even large consulting firms (Delloitte, IBM etc., where managers essentially eat what they kill) cannot allow themselves to degenerate into a Chinese court.
- bsenftner 4y ago
  Now this is interesting. I've always found it fascinating that when profit is on the table, democracy is nowhere to be found. I've looked, not too hard TBH, for essays and literature discussing the correlations to business model management structures and government/nation political hierarchies - not education level (propaganda), but critical analysis. I've been an employee of several of the top corporations on our planet, and the idea that corruption is not rampant is a farce. One simply lives within the environmental constraints and leaves when it gets to be too much. Does caring about corporate (and the larger realm of ethics) cast one incompatible with a modern corporate hierarchy?
  - psi75 4y ago
    Unfortunately, the only way to prevent hierarchy is to create a limited hierarchy (this is the purpose of constitutions) a priori; hierarchically naive organizations fail on this account. External parties will demand hierarchy simply because they want to know your organization (or nation) isn't wasting their time--no one wants to deliver a sales pitch to people who can't authorize purchases. If they're not careful, a group of people can end up in a state where the necessary-for-external-relations hierarchy becomes a total one. You see this with startup founders; the one who talks to the investors the most ends up in charge, and the ones who deal with employees or low-status counterparties lose power. This is why "flat" organizations can't really work; people who need things from the organization demand to know who to talk to in order to actually get things done, and eventually those "who to talk to" people end up with informal, then formal, power and it's very difficult to get them to give it back. The large-scale failure of democracy that's happening all over the world is something different, though. Regulation is struggling to keep up with technology, and it doesn't help that nation-states have already been doing a piss-poor job of protecting people from their employers. If the US falls in the next 20 years, it won't be due to Covid or Trump or nation-level adversaries; it'll be due to the obscene power given to employers, who can literally ruin an employee's life--not just fire him, but anally ravage him in perpetuity with bad references--for any reason or none. Eventually, unless national governments start dropping serious lead pipe on employers' heads, people are going to tire of paying 30+ percent of their incomes to a government that lets bosses get away with this shit.