12 ms·

Algorithms Have Already Gone Rogue

235 points by steven 9y ago

Iv 9y ago
Sorry but it is hard to take seriously someone who thinks AI and automation won't take away jobs. They will and it is a good thing.
- amelius 9y ago
  Perhaps, but they also take away income.
- pizza234 9y ago
  That's a loaded statement. You could say as well that's it's hard to take seriously those who don't know the lump of labour fallacy.
  - VincentEvans 9y ago
    If you think that models that are used to think about economy in aggregate somehow act as a guarantee that a life of an individual will not be wrecked as a result of large-scale automation.... well, lets discuss "lump of labor fallacy" when a former truck driver is "replacing" his job with 3 gig-economy part time jobs that sprung out as result of the "expanding economy". It's not that those models are wrong, it's that it's a very different thing to read about 20 years of traumatic events from the pages of books on economics, vs. having to live through it.
    - VincentEvans 9y ago
      To add to this, a bit: Our present political kerfuffle is no small way caused by collective feeling of anger by population who have seen their (industrial, mining, farming) jobs in rural america disappear due to globalization, while the market expanded .... elsewhere. In New York, in San Fran, in China, in India. And the money from that expansion did not go anywhere near those affected. But the upside is how much lives of Chinese have improved, so they got that going for them, which is nice.
      - s73ver_ 9y ago
        Kind of. Middle Class Chinese have had their lives improved quite a bit. Worker class Chinese, well, that situation seems fuzzier.
        pizza234 9y ago
        This may be more or less true, but that's a separate problem from globalization. Due to globalization China got much much richer (to the point of politically bullying other countries due to economical power), but the distribution of the income is an internal problem.
    - pizza234 9y ago
      I agree with that. My critic is towards stating that automation (robots, ai, etc.) causes a loss of jobs in absolute terms, as if in the future most of the people will be unemployed. There's a radical difference between stating that changes (loss and creation at the same time) in the jobs market will cause disruption, and stating that jobs will be lost and that's it. Thinking the (r)evolution in nihilistic terms will distract from improving the problems typically caused by the market change (which is the one you mentioned).
- theseatoms 9y ago
  Just like free trade, yes, a good thing in the aggregate. But many individuals will be displaced, and we should discuss these effects openly and honestly. The last generation wasn't so upfront about the secondary effects of globalization on blue collar workers in rich countries.
- wjossey 9y ago
  I think it’s important throw an addendum onto that statement. “They will, and it is a good thing. Although there may be some short term downsides we should look to mitigate.” I love to listen to economics podcasts and read economic articles and books in my spare time, so I’m sufficiently convinced that automation is a good thing for quality of life around the world. That being said, there is a real human cost as well that shouldn’t be downplayed. If we suddenly had autominous trucks available tomorrow, that would potentially lead to millions of American (tens of millions of world wide) workers that will become unemployed. Some percentage will pivot into new work. Some will struggle to find new jobs. Some will never find new employment. What we shouldn’t do is just discount that human suffering as “the cost of progress” and just move along. We short continue to work to find ways to help people with these transitions when and where we can. In short, let’s always try to mix in some humanity with our disruptive innovations.
  - _jal 9y ago
    An important point, but I hope it is recognized that "we" don't do economic downside-mitigation, at least in the US. People said the same thing about NAFTA - they'd fix it in post. I think there were some half-hearted education efforts and a bit of other fluff, but nobody who could actually do anything really cared. You can see this pattern repeated over and over. I only mean to pick on the US a little; we are worse than some other nations, but the problem is not unique to us. Humans just generally don't mitigate big, looming collective action problems. They ignore it until it happens and then whine that nobody did anything. Or, if it only effects the already-poor, everyone else settles on a narrative that it is their bad morals or laziness or whatever excuse doesn't make people think too hard.
- 24gttghh 9y ago
  Who says the people won't rise up and attempt to destroy thinking machines in some kind of holy war? No one likes being marginalized.
  - s73ver_ 9y ago
    But then the thinking machines will make machines of their own to protect them.
    - jam4peaches 9y ago
      Naw, the ideologues will go after the programmers first, as soon as they realize, much like in the movie 'transcendence'... [besides infrastructure etc]
- fiblye 9y ago
  Even assuming we end up with some society where robots do most of the labor and universal basic income becomes a thing, what do you think 90% of people will do? People need to be occupied, and the typical person isn't an inventor or artist. Furthermore, good luck making kids want to go to school when they see it as pointless since they can just get everything free from the robots while they watch TV. If you think obesity and our sedentary lifestyles are a danger now, it's only just starting. We're in for some huge problems as automation ramps up. The people saying "this is good" now will be screaming to go back once the problems start rolling in.
  - marcosdumay 9y ago
    Well, I think you give people too little credit. Nobody can stand passively watching TV the entire day every day of their lives. But even if you aren't, why would it bother you? If other people want to waste their lives on a coach in front of the TV, what makes this wrong?
    - sharemywin 9y ago
      Assume that the one thing you do best for a job AI/computers come out and do better. How do you recover from that. Especially, if said AI owners decide you only get a barely livable stipend while they keep most of the profits.
      - pbhjpbhj 9y ago
        Exactly, you have to rely on the "charity" of the Capitalists, which is going to be the worst gamble anyone made since the last time we tried to assume humans wouldn't be greedy and grab all the resources and power for themselves. These couch dwellers, what are they going to do to protest? They can't strike; if they're completely sedentary then they probably can't even protest without having a heart attack; cut off their VR or sugar-pump for 5 minutes and you can probably get any sort of compliance you want. Our only hope is that the great rolling balls of flab that develop from these sedentary couch-dwellers who exist only to consume HFCS mixed with palm-oil in a satisfying soup of flavourings, and consume the latest immersive media, can't procreate and the problem sorts itself out. Though I fear we'll be using artificial wombs and the like and so manage to remove any reproductive pressure that might push our race back on course.
- seppin 9y ago
  > They will and it is a good thing. Donald Trump, and whatever opportunist demagogue comes after him, agree with you.
- pbhjpbhj 9y ago
  It could be a good thing; it won't if we retain Western Capitalism as our social order.
- Acalyptol 9y ago
  It's hard to take seriously someone who doesn't hold the same speculations about the future as yours?
- s73ver_ 9y ago
  Whether or not it's a good thing depends entirely on us as a society. We can either decide to free people up, and give them the freedom to do things without having to worry about bills or where their next meal is coming from, a la Star Trek, or we can continue to put the same capital requirements on living that we currently do, and end up with the Hunger Games.
jordigh 9y ago
Is this his new meme-hustling? "Algorithm"? https://thebaffler.com/salvos/the-meme-hustler https://thebaffler.com/salvos/the-meme-hustler I've never liked how Tim O'Reilly frames his discussion around vague terms like "open" which could mean participatory, transparent, available, or any other number of vague, feel-good terms. Now he seems to be calling "algorithm" things like economic models and government policy. These are widely disparate things, but by using vague terms in different contexts, he pushes discussion towards the direction he wants to steer it: in the case of "open", away from free software. In the case of "Web 2.0", towards anything that involved crowd participation. With "algorithms", he seems to be wanting to push the notion that technology is both scary but liberating and we need tech messiahs like Bezos or Musk to bring it under control.
- tenaciousDaniel 9y ago
  That's been bothering me, the word "algorithm" is slowly becoming known as this ambiguously scary thing.
  - zghst 9y ago
    Everyday people couldn't even tell you the definition of "algorithm", even then, they wouldn't recognize that algorithms are not only encoded into chips but also business process, legal compliance, etc.
    - mattkrause 9y ago
      To be fair, it's very hard to precisely define "algorithm" and actual, working scientists and philosophers have been arguing about this for ages: https://en.wikipedia.org/wiki/Algorithm_characterizations https://en.wikipedia.org/wiki/Algorithm_characterizations
  - Chaebixi 9y ago
    It's kinda understandable, though. Whenever the services of Google, Facebook, etc. behave in an inscrutable, nonsensical, or offensive way, they blame it on their "algorithm" (for a recent example, see https://www.theguardian.com/us-news/2017/oct/06/youtube-alters-search-algorithm-over-fake-las-vegas-conspiracy-videos https://www.theguardian.com/us-news/2017/oct/06/youtube-alte...). That's really the only context where the term "algorithm" surfaces in mainstream discussion. ML has had a lot of successes, but one of its failures has been more unpredictability on the level of individuals and events that people actually experience.
zghst 9y ago
"why capitalism is like a rogue AI" These people seriously need to take a step back. At some point you cross the bridge between reporting and actively advertising someone's personal views. I can't ever trust or read people who constantly try to push an agenda, it's disingenuous. Even more so, when reporting, engaging in personal exchange without the discussing data and statistics, i.e. the facts, you are just allowing yourself to become a personal blog of someone else. Reporters are supposed to fact check, look for concrete evidence in someone's statements, hold people accountable to their words, and yet certain people get a pass all the time, even the star treatment.
- pdimitar 9y ago
  Maybe you're looking for PhD thesis presentations and not interviews with book authors, then. YouTube will be happy to help.
- zellyn 9y ago
  Steven Levy is one of the best writers when it comes to profiles and the gestalt of the tech scene. His stuff is consistently well done. I think you're looking for a different _type_ of writing, perhaps?
- saulrh 9y ago
  Yes? This is an interview with an author that was conducted specifically to talk about the book and the material in it. What did you expect? Unrelated, I'd like to see your argument against that particular line. IMO the comparison is an excellent one; its only issue is that the chosen scope ("the financial market") is too small. Corporations and other bureaucratic entities like governments are powerful cross-domain optimizers with utterly alien cognitive processes and goals. Intelligence, certainly, artificial, might as well call it that.
  - zghst 9y ago
    Well I'd hope that we'd expect better when an author makes outrageous, nonsensical claims. They are rehashing the book instead of analyzing its claims and assessing its validity. Conveniently, the blame of all the issues are left on the firms instead of the rules of the marketplace, conditions, etc which lead to distortions and structural issues in the market.
bitL 9y ago
Seems like the only job available in a not-so-distant future will be AI-correcting engineer...
dahart 9y ago
> financial markets are the first rogue AI. I feel like I just got Rick-Rolled.
mathgenius 9y ago
The financial system is a giant message passing algorithm. It is pretty much just a min-sum algorithm [1] whose sole purpose is to answer the question "what should we do?". Anyone who has played around with these algorithms for solving decoding problems knows that they are fabulously powerful. But these message passing algorithms have two weaknesses: (A) When there is more than one solution (B) When there are small loops in the message network By far the worst problem is (B): it's a kind of a "corruption" of the network and causes it to pretty much go off the rails. I think people already do understand the consequences of these problems in the financial system, but we don't seem to see how we can just change the topology and/or the messages themselves, in particular, to try to fix up these self-reinforcing loops. Or: move away from min-sum towards sum-product (which often works an order of magnitude better) by perhaps implementing basic income. Etc. Etc. [1] https://en.wikipedia.org/wiki/Generalized_distributive_law https://en.wikipedia.org/wiki/Generalized_distributive_law
- zghst 9y ago
  Brilliant take. Better analysis than pseudo-intellectual O'Reilly and the Wired cultists.
- KirinDave 9y ago
  Having worked in the financial industry, your view of it is what we'd call, "50,000" feet. I don't think it reflects reality on the ground, nor does it really reflect how humans in groups make decisions according to research. The message network loop issue isn't even an issue for message passing systems except in the most pathological cases (where it results in an unbounded growth in messages or an infinite delta on the values the algorithm calculates.
  - mathgenius 9y ago
    I'm not saying the finance system doesn't work, it obviously does, and amazingly well. I'm just pointing out the weaknesses of the system, and these weaknesses could easily cause humanity to walk off a cliff. I also have a finance background. This idea that the system is a kind of message passing is not a "50,000 feet" idea, it actually came to me from writing algorithms to arbitrage markets. There you really are performing message passing (think of forex legs.) It's Dijkstra's algorithm. So this is the view from 50,000 nano-seconds. But I believe it holds for many other scales. Plenty of people walk into the supermarket and buy the product with minimum cost. Yes? How is this a view from "50,000 feet"? But obviously we are not just min-sum automata: we are all free agents, and so on. > how humans in groups make decisions according to research Well this is obviously a huge topic. > message network loop issue isn't even an issue for message passing systems except in the most pathological cases I totally disagree with this. Short loops completely screwup message passing.
jacknews 9y ago
"Our fears ultimately should be of ourselves and other people." Indeed. The short-term fear at least should not be about machine overlords, but about how people in power use AI to increase their power and/or make life worse for everyone else.
andrewla 9y ago
My only objection to this is a semantic one -- the word "algorithm" is not being well-served here. The correct word for this sort of thing is "heuristic". The concern isn't that algorithms themselves are incorrect, the concern is that the problem they are trying to solve is a heuristic one, not a formal one. Saying "let's write an algorithm to improve search results" is meaningless; "let's design and implement a heuristic that improves search results". The algorithmic part of this is how to efficiently implement that heuristic. I can usually get through articles like this by silently replacing "algorithm" with "heuristic"; the problem arises when some articles attempt to draw equivalencies between "algorithmic" concepts, like running time and space, and "heuristic" concepts, like optimizing for the wrong thing.
- usrusr 9y ago
  Algorithms running on "social hardware" can be surprisingly formal. A famously well-documented example are early modern witchhunts. The humorous depiction in Monty Python and the Holy Grail does a surprisingly good job at conveying the algorithmic nature.
  - stult 9y ago
    Many aspects of the law are algorithmic. Even though there is no one, settled formal definition for algorithm, statutory and common law meet many informal definitions. Laws usually lay out an ordered, (theoretically) unambiguous set of steps for deciding a legal issue. When lawyers talk about "elements of a test," they are referring to this structured logic. For example, the elements required to prove a negligence claim are: 1. Duty 2. Breach of Duty 3. Cause in Fact 4. Proximate Cause 5. Damages When evaluating a negligence claim, a lawyer first tries to determine if the defendant owed the plaintiff any duty of care, then whether the plaintiff breached that duty, then if that breach was the factual cause of a harm suffered by the plaintiff, then whether the causal relationship was close enough to be considered legally proximate, and then, finally, whether the plaintiff actually suffered measurable damages. Arguably, that superficially algorithmic process frequently breaks down in practice. For example, it's often easier to start with the damages suffered by a plaintiff and work backwards by identifying the causes, then who was responsible for those causes and any duties they may have owed to the plaintiff. However, regardless of how the lawyer and plaintiff identify whom to sue, they must frame their pleadings to allege the elements of the tort in the order specified by their jurisdiction's law, so the actual practice of law in court amounts to an algorithmic exercise.
quotemstr 9y ago
Capitalism is the best thing that's ever happened to humanity. The author's ideological bias gets in the way of understanding.
- auserperson 9y ago
  anyone who downvoted this can go fuck themselves.
- s73ver_ 9y ago
  That doesn't mean it's not also bad for large swaths of humanity.
- monsieurgaufre 9y ago
  Your own is as well. You make a strong claim without proving it.
- jeffdavis 9y ago
  That's probably true over the span of time where what we needed most was capital investment. I think there's a fair question about whether we are changing modes. "Capital" itself is changing in nature -- not so much about big machines and mass production any more.
- cujic9 9y ago
  It has certainly enabled more humans to exist. It has created way better conditions for many of those humans. And it has created unimaginable suffering for just as many humans. I wouldn't get rid of capitalism, but I certainly wouldn't give it a superlative. It's kind of like saying that domestication is the best thing that has happened to chickens, cause look how many more of them there are, and wow some of them live on a free range.
peterwwillis 9y ago
Have any of these tech "visionaries" (aka millionaires that we pretend have more insight than non-millionaires) considered that at the same time that we've increased our wealth we've also done more damage to our environment? If the cost of increased wealth is decreased habitat, will peak wealth result in the death of nature? Aside: people are still comparing AI to a machine that feels, and this is so stupid it boggles the mind. The machine that creates paper clips will not kill off anyone who tries to turn it off because it does not have self-preservation, which is a system dependent on fear of death. Machines do not fear, and even if they did they would not fear death, because they have solid state. Bio systems are walking RAM disks. Algorithms are basically math problems. The financial system and government do not work based on math problems, they work based on emotional instability. Seriously. Both these systems are driven almost entirely by the feelings of humans. They aren't algorithms, they are shitty biological systems that don't make mathematical sense at all.
- kosta_self 9y ago
  > The machine that creates paper clips will not kill off anyone who tries to turn it off because it does not have self-preservation The idea here is that the algorithm tries to maximize its paperclip production and turning the machine off will definitely have a negative impact on its paperclip production. So the machine will take the most efficient measure to stop that negative impact.
- peterwwillis 9y ago
  kosta_self: I think you're hellbanned. Good point, but that would be a bug in the AI. An intelligent system would see there would be no point in killing humans, so killing the humans would defeat the purpose of making the paperclips, and the humans support the machine, thus killing them would be counter-productive to its purpose. It's also unrealistic that algorithms would try to exceed the limitations of their system. Imagine if a natural predator tried to "maximize production" of killing its prey: it would run out of prey, starve, and die. An AI would understand that trying to "maximize production" to the detriment of everything else would be counter-productive and create a resource contention war. Humans are the only creature I know of that exhausts natural resources to its own detriment - we are smart enough to exploit our resources, but too stupid to know when to stop. The natural system's response to this behavior seems to be to get us to kill ourselves. Maybe the AI are part of this process?
deleted 9y ago
[deleted]
grandalf 9y ago
It's 2017 and humans have built several anemonae, and we have several highly profitable firms that are built around these anemonae. Their employees act as the stewards (clownfish) more as beings in symbiosis than as one controlling the other. We can neither understand or pull the plug on these creatures. Time will tell which species evolves more rapidly.
mathgenius 9y ago
> financial markets are the first rogue AI How about this idea: the first rogue AI was language. In terms of AI as a compositional system for storing meaning, I think this might be a reasonable position to take. Yes people, we've been playing this game for a very long time...
- projektir 9y ago
  That sounds a lot like McLuhan's ideas.
  - mathgenius 9y ago
    Who is McLuhan? Do you have a reference for this?
    - grzm 9y ago
      https://en.wikipedia.org/wiki/Marshall_McLuhan https://en.wikipedia.org/wiki/Marshall_McLuhan
dcre 9y ago
Can anyone explain why O'Reilly thinks nobody knew until recently that companies are biased toward part-time work in order to avoid providing benefits? "We can’t see, for example, that the algorithms that manage the workers at McDonald’s or The Gap are optimized toward not giving people full-time work so they don’t have to pay benefits. All that was invisible. It wasn’t until we really started seeing the tech-infused algorithms that people started being critical." And here's one that's more subtle, so I don't blame him quite as much, but he is naive to think "ideas" are what cause corporations to act the way they do. Material and institutional conditions cause their behavior, which is then justified after the fact by appeal to shareholder value. "Somebody planted the idea that shareholder value was the right algorithm, the right thing to be optimizing for. But this wasn’t the way companies acted before. We can plant a different idea. That’s what this political process is about."
- pdimitar 9y ago
  > Can anyone explain why O'Reilly thinks nobody knew until recently that companies are biased toward part-time work in order to avoid providing benefits? How do you know he's thinking that? The way he was talking, I read him as "well it's obvious that nobody has stopped this behavior so it's fair to assume that not enough people noticed". Wouldn't you agree with that? > And here's one that's more subtle, so I don't blame him quite as much, but he is naive to think "ideas" are what cause corporations to act the way they do. Material and institutional conditions cause their behavior, which is then justified after the fact by appeal to shareholder value. That was the only part of the interview where I strongly disagreed with him -- and you're right. It's not about ideas; there are a lot of people out there who are extremely good in bean-counting, micro-management, and of course awful at promoting a positive work environment. They will never change. Only regulators can limit them a bit, if even that.
  - dcre 9y ago
    Workers have been agitating against this behavior for many decades, so no, I would not agree that it is fair to assume nobody noticed.
vpribish 9y ago
I've flipped the bozo bit on the term 'Algorithm'.
hasbroslasher 9y ago
Obviously the most controversial part of this is "capitalism is the first rogue AI" point. I will try to stay out of having an opinion on this, I just want to try to add colour to what he's saying. First, in the classical "market" scenario, we're talking about little atomic firms, each with goals and hopes and dreams of profit. Each of these firms has some kind of "knowledge" and some kind of decision making apparatus. They all have some functionality. In a sense, these firms are like people - so much that our government treats them as such in many cases! In many cases, a lot of these ideas and processes end up being the same, and when it is, we call it collective wisdom[0]or collective intelligence[1]. I won't go too deep into that. So while financial markets and the firms that comprise them aren't exactly _machines_, they do display a form of intelligence different and sometimes more effective than our individual knowledge. [0] https://en.wikipedia.org/wiki/collective_wisdom https://en.wikipedia.org/wiki/collective_wisdom [1] https://en.wikipedia.org/wiki/collective_intelligence https://en.wikipedia.org/wiki/collective_intelligence
jpster 9y ago
>We had plenty of bias before but we couldn’t see it. We can’t see, for example, that the algorithms that manage the workers at McDonald’s or The Gap are optimized toward not giving people full-time work so they don’t have to pay benefits. All that was invisible. It wasn’t until we really started seeing the tech-infused algorithms that people started being critical. I’m not sure how this can be said with a straight face. An algorithm was really not needed to perceive this and it’s insulting and strange to suggest it.
- vpribish 9y ago
  “Tech-infused”. Like tech is an herb or a spice? Surely a sign that there are no coherent ideas to be found from that writer.
jefurii 9y ago
> For more than two decades, Tim O’Reilly has been the conscience of the tech industry. ...he was among the first to perceive both the societal and commercial value of the internet [and] ... he drew upon his education in the classics to apply a moral yardstick to what was happening in tech. Writer hasn't heard of Richard Stallman? Update: writer is Steven Levy, who most definitely has heard of Stallman, and should know better.
sukilot 9y ago
https://www.amazon.com/Weapons-Math-Destruction-Increases-Inequality/dp/0553418815 https://www.amazon.com/Weapons-Math-Destruction-Increases-In...
egypturnash 9y ago
"I’m not sure that Jeff would make a great president, but he might." So let me tell you about my experience with Amazon Fulfilment. I was gonna pay Amazon, who has this huge expertise in packing and shipping stuff, to fulfil my Kickstarter. I'd made a large, delicate art book. Amazon, in their infinite wisdom, stuffed them in bubble wrap envelopes and dropped them in the mail. They were getting bent, the envelopes were getting ripped up, it was a mess. I spent a month in customer service hell e-mailing someone who was following a script that said I would have to turn on something they called "prep", which would ask them to look at it and package it better. Three times I checked that this was set, and three times I sent myself a test book that came in a bubble mailer. Finally I got clued in that there is a high-level support team that you access by sending a complaint directly to Bezos. This person, after some back and forth, ultimately informed me that due to their internal systems, it is completely impossible for them to put a book in anything but this level of shitty packaging; "prep" is just not a process that can ever happen to a book. They covered shipping my books out to someone actually capable of finding sensible boxes and shipping them. And they intimated that someone maybe lost a job over this. But changing this, they said, might take on the order of years. I'm not sure I want a man who presides over a system like this running the country. Especially given that O'Reilly points out that the "rogue algorithms" of the title are corporations, and the only reason Amazon is headquartered in WA is that the tax was lowest there...
freedomben 9y ago
> when the curtain rolls back we see that those superpowers have consequences: Those algorithms have bias built in. > That’s absolutely right. But I’m optimistic because we’re having a conversation about biased algorithms. We had plenty of bias before but we couldn’t see it. I must say I am really happy to see that bias in tech is being recognized and accounted for (for the most part). Please forgive the politics (I'll try really hard not to bash Trump ;-) ), but if there is a silver lining to the 2016 US presidential election I think it is that it has really caused many of us to introspect and realize how thick our echo chamber walls have really gotten over the last few years. The chamber was constructed so quickly, I barely realized it was happening. We're becoming so polarized that we're actually moving to different communities to be with more people "like us." Simple awareness of the problem is a huge step forward in being able to resolve it.

235 points by steven 9y ago

Iv 9y ago

Sorry but it is hard to take seriously someone who thinks AI and automation won't take away jobs. They will and it is a good thing.

amelius 9y ago
Perhaps, but they also take away income.
pizza234 9y ago
That's a loaded statement. You could say as well that's it's hard to take seriously those who don't know the lump of labour fallacy.
- VincentEvans 9y ago
  If you think that models that are used to think about economy in aggregate somehow act as a guarantee that a life of an individual will not be wrecked as a result of large-scale automation.... well, lets discuss "lump of labor fallacy" when a former truck driver is "replacing" his job with 3 gig-economy part time jobs that sprung out as result of the "expanding economy". It's not that those models are wrong, it's that it's a very different thing to read about 20 years of traumatic events from the pages of books on economics, vs. having to live through it.
  - VincentEvans 9y ago
    To add to this, a bit: Our present political kerfuffle is no small way caused by collective feeling of anger by population who have seen their (industrial, mining, farming) jobs in rural america disappear due to globalization, while the market expanded .... elsewhere. In New York, in San Fran, in China, in India. And the money from that expansion did not go anywhere near those affected. But the upside is how much lives of Chinese have improved, so they got that going for them, which is nice.
    - s73ver_ 9y ago
      Kind of. Middle Class Chinese have had their lives improved quite a bit. Worker class Chinese, well, that situation seems fuzzier.
      - pizza234 9y ago
        This may be more or less true, but that's a separate problem from globalization. Due to globalization China got much much richer (to the point of politically bullying other countries due to economical power), but the distribution of the income is an internal problem.
  - pizza234 9y ago
    I agree with that. My critic is towards stating that automation (robots, ai, etc.) causes a loss of jobs in absolute terms, as if in the future most of the people will be unemployed. There's a radical difference between stating that changes (loss and creation at the same time) in the jobs market will cause disruption, and stating that jobs will be lost and that's it. Thinking the (r)evolution in nihilistic terms will distract from improving the problems typically caused by the market change (which is the one you mentioned).
theseatoms 9y ago
Just like free trade, yes, a good thing in the aggregate. But many individuals will be displaced, and we should discuss these effects openly and honestly. The last generation wasn't so upfront about the secondary effects of globalization on blue collar workers in rich countries.
wjossey 9y ago
I think it’s important throw an addendum onto that statement. “They will, and it is a good thing. Although there may be some short term downsides we should look to mitigate.” I love to listen to economics podcasts and read economic articles and books in my spare time, so I’m sufficiently convinced that automation is a good thing for quality of life around the world. That being said, there is a real human cost as well that shouldn’t be downplayed. If we suddenly had autominous trucks available tomorrow, that would potentially lead to millions of American (tens of millions of world wide) workers that will become unemployed. Some percentage will pivot into new work. Some will struggle to find new jobs. Some will never find new employment. What we shouldn’t do is just discount that human suffering as “the cost of progress” and just move along. We short continue to work to find ways to help people with these transitions when and where we can. In short, let’s always try to mix in some humanity with our disruptive innovations.
- _jal 9y ago
  An important point, but I hope it is recognized that "we" don't do economic downside-mitigation, at least in the US. People said the same thing about NAFTA - they'd fix it in post. I think there were some half-hearted education efforts and a bit of other fluff, but nobody who could actually do anything really cared. You can see this pattern repeated over and over. I only mean to pick on the US a little; we are worse than some other nations, but the problem is not unique to us. Humans just generally don't mitigate big, looming collective action problems. They ignore it until it happens and then whine that nobody did anything. Or, if it only effects the already-poor, everyone else settles on a narrative that it is their bad morals or laziness or whatever excuse doesn't make people think too hard.
24gttghh 9y ago
Who says the people won't rise up and attempt to destroy thinking machines in some kind of holy war? No one likes being marginalized.
- s73ver_ 9y ago
  But then the thinking machines will make machines of their own to protect them.
  - jam4peaches 9y ago
    Naw, the ideologues will go after the programmers first, as soon as they realize, much like in the movie 'transcendence'... [besides infrastructure etc]
fiblye 9y ago
Even assuming we end up with some society where robots do most of the labor and universal basic income becomes a thing, what do you think 90% of people will do? People need to be occupied, and the typical person isn't an inventor or artist. Furthermore, good luck making kids want to go to school when they see it as pointless since they can just get everything free from the robots while they watch TV. If you think obesity and our sedentary lifestyles are a danger now, it's only just starting. We're in for some huge problems as automation ramps up. The people saying "this is good" now will be screaming to go back once the problems start rolling in.
- marcosdumay 9y ago
  Well, I think you give people too little credit. Nobody can stand passively watching TV the entire day every day of their lives. But even if you aren't, why would it bother you? If other people want to waste their lives on a coach in front of the TV, what makes this wrong?
  - sharemywin 9y ago
    Assume that the one thing you do best for a job AI/computers come out and do better. How do you recover from that. Especially, if said AI owners decide you only get a barely livable stipend while they keep most of the profits.
    - pbhjpbhj 9y ago
      Exactly, you have to rely on the "charity" of the Capitalists, which is going to be the worst gamble anyone made since the last time we tried to assume humans wouldn't be greedy and grab all the resources and power for themselves. These couch dwellers, what are they going to do to protest? They can't strike; if they're completely sedentary then they probably can't even protest without having a heart attack; cut off their VR or sugar-pump for 5 minutes and you can probably get any sort of compliance you want. Our only hope is that the great rolling balls of flab that develop from these sedentary couch-dwellers who exist only to consume HFCS mixed with palm-oil in a satisfying soup of flavourings, and consume the latest immersive media, can't procreate and the problem sorts itself out. Though I fear we'll be using artificial wombs and the like and so manage to remove any reproductive pressure that might push our race back on course.
seppin 9y ago
> They will and it is a good thing. Donald Trump, and whatever opportunist demagogue comes after him, agree with you.
pbhjpbhj 9y ago
It could be a good thing; it won't if we retain Western Capitalism as our social order.
Acalyptol 9y ago
It's hard to take seriously someone who doesn't hold the same speculations about the future as yours?
s73ver_ 9y ago
Whether or not it's a good thing depends entirely on us as a society. We can either decide to free people up, and give them the freedom to do things without having to worry about bills or where their next meal is coming from, a la Star Trek, or we can continue to put the same capital requirements on living that we currently do, and end up with the Hunger Games.

jordigh 9y ago

Is this his new meme-hustling? "Algorithm"? https://thebaffler.com/salvos/the-meme-hustler https://thebaffler.com/salvos/the-meme-hustler I've never liked how Tim O'Reilly frames his discussion around vague terms like "open" which could mean participatory, transparent, available, or any other number of vague, feel-good terms. Now he seems to be calling "algorithm" things like economic models and government policy. These are widely disparate things, but by using vague terms in different contexts, he pushes discussion towards the direction he wants to steer it: in the case of "open", away from free software. In the case of "Web 2.0", towards anything that involved crowd participation. With "algorithms", he seems to be wanting to push the notion that technology is both scary but liberating and we need tech messiahs like Bezos or Musk to bring it under control.

tenaciousDaniel 9y ago
That's been bothering me, the word "algorithm" is slowly becoming known as this ambiguously scary thing.
- zghst 9y ago
  Everyday people couldn't even tell you the definition of "algorithm", even then, they wouldn't recognize that algorithms are not only encoded into chips but also business process, legal compliance, etc.
  - mattkrause 9y ago
    To be fair, it's very hard to precisely define "algorithm" and actual, working scientists and philosophers have been arguing about this for ages: https://en.wikipedia.org/wiki/Algorithm_characterizations https://en.wikipedia.org/wiki/Algorithm_characterizations
- Chaebixi 9y ago
  It's kinda understandable, though. Whenever the services of Google, Facebook, etc. behave in an inscrutable, nonsensical, or offensive way, they blame it on their "algorithm" (for a recent example, see https://www.theguardian.com/us-news/2017/oct/06/youtube-alters-search-algorithm-over-fake-las-vegas-conspiracy-videos https://www.theguardian.com/us-news/2017/oct/06/youtube-alte...). That's really the only context where the term "algorithm" surfaces in mainstream discussion. ML has had a lot of successes, but one of its failures has been more unpredictability on the level of individuals and events that people actually experience.

zghst 9y ago

"why capitalism is like a rogue AI" These people seriously need to take a step back. At some point you cross the bridge between reporting and actively advertising someone's personal views. I can't ever trust or read people who constantly try to push an agenda, it's disingenuous. Even more so, when reporting, engaging in personal exchange without the discussing data and statistics, i.e. the facts, you are just allowing yourself to become a personal blog of someone else. Reporters are supposed to fact check, look for concrete evidence in someone's statements, hold people accountable to their words, and yet certain people get a pass all the time, even the star treatment.

pdimitar 9y ago
Maybe you're looking for PhD thesis presentations and not interviews with book authors, then. YouTube will be happy to help.
zellyn 9y ago
Steven Levy is one of the best writers when it comes to profiles and the gestalt of the tech scene. His stuff is consistently well done. I think you're looking for a different _type_ of writing, perhaps?
saulrh 9y ago
Yes? This is an interview with an author that was conducted specifically to talk about the book and the material in it. What did you expect? Unrelated, I'd like to see your argument against that particular line. IMO the comparison is an excellent one; its only issue is that the chosen scope ("the financial market") is too small. Corporations and other bureaucratic entities like governments are powerful cross-domain optimizers with utterly alien cognitive processes and goals. Intelligence, certainly, artificial, might as well call it that.
- zghst 9y ago
  Well I'd hope that we'd expect better when an author makes outrageous, nonsensical claims. They are rehashing the book instead of analyzing its claims and assessing its validity. Conveniently, the blame of all the issues are left on the firms instead of the rules of the marketplace, conditions, etc which lead to distortions and structural issues in the market.

bitL 9y ago

Seems like the only job available in a not-so-distant future will be AI-correcting engineer...

dahart 9y ago

> financial markets are the first rogue AI. I feel like I just got Rick-Rolled.

mathgenius 9y ago

The financial system is a giant message passing algorithm. It is pretty much just a min-sum algorithm [1] whose sole purpose is to answer the question "what should we do?". Anyone who has played around with these algorithms for solving decoding problems knows that they are fabulously powerful. But these message passing algorithms have two weaknesses: (A) When there is more than one solution (B) When there are small loops in the message network By far the worst problem is (B): it's a kind of a "corruption" of the network and causes it to pretty much go off the rails. I think people already do understand the consequences of these problems in the financial system, but we don't seem to see how we can just change the topology and/or the messages themselves, in particular, to try to fix up these self-reinforcing loops. Or: move away from min-sum towards sum-product (which often works an order of magnitude better) by perhaps implementing basic income. Etc. Etc. [1] https://en.wikipedia.org/wiki/Generalized_distributive_law https://en.wikipedia.org/wiki/Generalized_distributive_law

zghst 9y ago
Brilliant take. Better analysis than pseudo-intellectual O'Reilly and the Wired cultists.
KirinDave 9y ago
Having worked in the financial industry, your view of it is what we'd call, "50,000" feet. I don't think it reflects reality on the ground, nor does it really reflect how humans in groups make decisions according to research. The message network loop issue isn't even an issue for message passing systems except in the most pathological cases (where it results in an unbounded growth in messages or an infinite delta on the values the algorithm calculates.
- mathgenius 9y ago
  I'm not saying the finance system doesn't work, it obviously does, and amazingly well. I'm just pointing out the weaknesses of the system, and these weaknesses could easily cause humanity to walk off a cliff. I also have a finance background. This idea that the system is a kind of message passing is not a "50,000 feet" idea, it actually came to me from writing algorithms to arbitrage markets. There you really are performing message passing (think of forex legs.) It's Dijkstra's algorithm. So this is the view from 50,000 nano-seconds. But I believe it holds for many other scales. Plenty of people walk into the supermarket and buy the product with minimum cost. Yes? How is this a view from "50,000 feet"? But obviously we are not just min-sum automata: we are all free agents, and so on. > how humans in groups make decisions according to research Well this is obviously a huge topic. > message network loop issue isn't even an issue for message passing systems except in the most pathological cases I totally disagree with this. Short loops completely screwup message passing.

jacknews 9y ago

"Our fears ultimately should be of ourselves and other people." Indeed. The short-term fear at least should not be about machine overlords, but about how people in power use AI to increase their power and/or make life worse for everyone else.

andrewla 9y ago

My only objection to this is a semantic one -- the word "algorithm" is not being well-served here. The correct word for this sort of thing is "heuristic". The concern isn't that algorithms themselves are incorrect, the concern is that the problem they are trying to solve is a heuristic one, not a formal one. Saying "let's write an algorithm to improve search results" is meaningless; "let's design and implement a heuristic that improves search results". The algorithmic part of this is how to efficiently implement that heuristic. I can usually get through articles like this by silently replacing "algorithm" with "heuristic"; the problem arises when some articles attempt to draw equivalencies between "algorithmic" concepts, like running time and space, and "heuristic" concepts, like optimizing for the wrong thing.

usrusr 9y ago
Algorithms running on "social hardware" can be surprisingly formal. A famously well-documented example are early modern witchhunts. The humorous depiction in Monty Python and the Holy Grail does a surprisingly good job at conveying the algorithmic nature.
- stult 9y ago
  Many aspects of the law are algorithmic. Even though there is no one, settled formal definition for algorithm, statutory and common law meet many informal definitions. Laws usually lay out an ordered, (theoretically) unambiguous set of steps for deciding a legal issue. When lawyers talk about "elements of a test," they are referring to this structured logic. For example, the elements required to prove a negligence claim are: 1. Duty 2. Breach of Duty 3. Cause in Fact 4. Proximate Cause 5. Damages When evaluating a negligence claim, a lawyer first tries to determine if the defendant owed the plaintiff any duty of care, then whether the plaintiff breached that duty, then if that breach was the factual cause of a harm suffered by the plaintiff, then whether the causal relationship was close enough to be considered legally proximate, and then, finally, whether the plaintiff actually suffered measurable damages. Arguably, that superficially algorithmic process frequently breaks down in practice. For example, it's often easier to start with the damages suffered by a plaintiff and work backwards by identifying the causes, then who was responsible for those causes and any duties they may have owed to the plaintiff. However, regardless of how the lawyer and plaintiff identify whom to sue, they must frame their pleadings to allege the elements of the tort in the order specified by their jurisdiction's law, so the actual practice of law in court amounts to an algorithmic exercise.

quotemstr 9y ago

Capitalism is the best thing that's ever happened to humanity. The author's ideological bias gets in the way of understanding.

auserperson 9y ago
anyone who downvoted this can go fuck themselves.
s73ver_ 9y ago
That doesn't mean it's not also bad for large swaths of humanity.
monsieurgaufre 9y ago
Your own is as well. You make a strong claim without proving it.
jeffdavis 9y ago
That's probably true over the span of time where what we needed most was capital investment. I think there's a fair question about whether we are changing modes. "Capital" itself is changing in nature -- not so much about big machines and mass production any more.
cujic9 9y ago
It has certainly enabled more humans to exist. It has created way better conditions for many of those humans. And it has created unimaginable suffering for just as many humans. I wouldn't get rid of capitalism, but I certainly wouldn't give it a superlative. It's kind of like saying that domestication is the best thing that has happened to chickens, cause look how many more of them there are, and wow some of them live on a free range.

peterwwillis 9y ago

Have any of these tech "visionaries" (aka millionaires that we pretend have more insight than non-millionaires) considered that at the same time that we've increased our wealth we've also done more damage to our environment? If the cost of increased wealth is decreased habitat, will peak wealth result in the death of nature? Aside: people are still comparing AI to a machine that feels, and this is so stupid it boggles the mind. The machine that creates paper clips will not kill off anyone who tries to turn it off because it does not have self-preservation, which is a system dependent on fear of death. Machines do not fear, and even if they did they would not fear death, because they have solid state. Bio systems are walking RAM disks. Algorithms are basically math problems. The financial system and government do not work based on math problems, they work based on emotional instability. Seriously. Both these systems are driven almost entirely by the feelings of humans. They aren't algorithms, they are shitty biological systems that don't make mathematical sense at all.

kosta_self 9y ago
> The machine that creates paper clips will not kill off anyone who tries to turn it off because it does not have self-preservation The idea here is that the algorithm tries to maximize its paperclip production and turning the machine off will definitely have a negative impact on its paperclip production. So the machine will take the most efficient measure to stop that negative impact.
peterwwillis 9y ago
kosta_self: I think you're hellbanned. Good point, but that would be a bug in the AI. An intelligent system would see there would be no point in killing humans, so killing the humans would defeat the purpose of making the paperclips, and the humans support the machine, thus killing them would be counter-productive to its purpose. It's also unrealistic that algorithms would try to exceed the limitations of their system. Imagine if a natural predator tried to "maximize production" of killing its prey: it would run out of prey, starve, and die. An AI would understand that trying to "maximize production" to the detriment of everything else would be counter-productive and create a resource contention war. Humans are the only creature I know of that exhausts natural resources to its own detriment - we are smart enough to exploit our resources, but too stupid to know when to stop. The natural system's response to this behavior seems to be to get us to kill ourselves. Maybe the AI are part of this process?

deleted 9y ago

[deleted]

grandalf 9y ago

It's 2017 and humans have built several anemonae, and we have several highly profitable firms that are built around these anemonae. Their employees act as the stewards (clownfish) more as beings in symbiosis than as one controlling the other. We can neither understand or pull the plug on these creatures. Time will tell which species evolves more rapidly.

mathgenius 9y ago

> financial markets are the first rogue AI How about this idea: the first rogue AI was language. In terms of AI as a compositional system for storing meaning, I think this might be a reasonable position to take. Yes people, we've been playing this game for a very long time...

projektir 9y ago
That sounds a lot like McLuhan's ideas.
- mathgenius 9y ago
  Who is McLuhan? Do you have a reference for this?
  - grzm 9y ago
    https://en.wikipedia.org/wiki/Marshall_McLuhan https://en.wikipedia.org/wiki/Marshall_McLuhan

dcre 9y ago

Can anyone explain why O'Reilly thinks nobody knew until recently that companies are biased toward part-time work in order to avoid providing benefits? "We can’t see, for example, that the algorithms that manage the workers at McDonald’s or The Gap are optimized toward not giving people full-time work so they don’t have to pay benefits. All that was invisible. It wasn’t until we really started seeing the tech-infused algorithms that people started being critical." And here's one that's more subtle, so I don't blame him quite as much, but he is naive to think "ideas" are what cause corporations to act the way they do. Material and institutional conditions cause their behavior, which is then justified after the fact by appeal to shareholder value. "Somebody planted the idea that shareholder value was the right algorithm, the right thing to be optimizing for. But this wasn’t the way companies acted before. We can plant a different idea. That’s what this political process is about."

pdimitar 9y ago
> Can anyone explain why O'Reilly thinks nobody knew until recently that companies are biased toward part-time work in order to avoid providing benefits? How do you know he's thinking that? The way he was talking, I read him as "well it's obvious that nobody has stopped this behavior so it's fair to assume that not enough people noticed". Wouldn't you agree with that? > And here's one that's more subtle, so I don't blame him quite as much, but he is naive to think "ideas" are what cause corporations to act the way they do. Material and institutional conditions cause their behavior, which is then justified after the fact by appeal to shareholder value. That was the only part of the interview where I strongly disagreed with him -- and you're right. It's not about ideas; there are a lot of people out there who are extremely good in bean-counting, micro-management, and of course awful at promoting a positive work environment. They will never change. Only regulators can limit them a bit, if even that.
- dcre 9y ago
  Workers have been agitating against this behavior for many decades, so no, I would not agree that it is fair to assume nobody noticed.

vpribish 9y ago

I've flipped the bozo bit on the term 'Algorithm'.

hasbroslasher 9y ago

Obviously the most controversial part of this is "capitalism is the first rogue AI" point. I will try to stay out of having an opinion on this, I just want to try to add colour to what he's saying. First, in the classical "market" scenario, we're talking about little atomic firms, each with goals and hopes and dreams of profit. Each of these firms has some kind of "knowledge" and some kind of decision making apparatus. They all have some functionality. In a sense, these firms are like people - so much that our government treats them as such in many cases! In many cases, a lot of these ideas and processes end up being the same, and when it is, we call it collective wisdom[0]or collective intelligence[1]. I won't go too deep into that. So while financial markets and the firms that comprise them aren't exactly _machines_, they do display a form of intelligence different and sometimes more effective than our individual knowledge. [0] https://en.wikipedia.org/wiki/collective_wisdom https://en.wikipedia.org/wiki/collective_wisdom [1] https://en.wikipedia.org/wiki/collective_intelligence https://en.wikipedia.org/wiki/collective_intelligence

jpster 9y ago

>We had plenty of bias before but we couldn’t see it. We can’t see, for example, that the algorithms that manage the workers at McDonald’s or The Gap are optimized toward not giving people full-time work so they don’t have to pay benefits. All that was invisible. It wasn’t until we really started seeing the tech-infused algorithms that people started being critical. I’m not sure how this can be said with a straight face. An algorithm was really not needed to perceive this and it’s insulting and strange to suggest it.

vpribish 9y ago
“Tech-infused”. Like tech is an herb or a spice? Surely a sign that there are no coherent ideas to be found from that writer.

jefurii 9y ago

> For more than two decades, Tim O’Reilly has been the conscience of the tech industry. ...he was among the first to perceive both the societal and commercial value of the internet [and] ... he drew upon his education in the classics to apply a moral yardstick to what was happening in tech. Writer hasn't heard of Richard Stallman? Update: writer is Steven Levy, who most definitely has heard of Stallman, and should know better.

sukilot 9y ago

https://www.amazon.com/Weapons-Math-Destruction-Increases-Inequality/dp/0553418815 https://www.amazon.com/Weapons-Math-Destruction-Increases-In...

egypturnash 9y ago

"I’m not sure that Jeff would make a great president, but he might." So let me tell you about my experience with Amazon Fulfilment. I was gonna pay Amazon, who has this huge expertise in packing and shipping stuff, to fulfil my Kickstarter. I'd made a large, delicate art book. Amazon, in their infinite wisdom, stuffed them in bubble wrap envelopes and dropped them in the mail. They were getting bent, the envelopes were getting ripped up, it was a mess. I spent a month in customer service hell e-mailing someone who was following a script that said I would have to turn on something they called "prep", which would ask them to look at it and package it better. Three times I checked that this was set, and three times I sent myself a test book that came in a bubble mailer. Finally I got clued in that there is a high-level support team that you access by sending a complaint directly to Bezos. This person, after some back and forth, ultimately informed me that due to their internal systems, it is completely impossible for them to put a book in anything but this level of shitty packaging; "prep" is just not a process that can ever happen to a book. They covered shipping my books out to someone actually capable of finding sensible boxes and shipping them. And they intimated that someone maybe lost a job over this. But changing this, they said, might take on the order of years. I'm not sure I want a man who presides over a system like this running the country. Especially given that O'Reilly points out that the "rogue algorithms" of the title are corporations, and the only reason Amazon is headquartered in WA is that the tax was lowest there...

freedomben 9y ago

> when the curtain rolls back we see that those superpowers have consequences: Those algorithms have bias built in. > That’s absolutely right. But I’m optimistic because we’re having a conversation about biased algorithms. We had plenty of bias before but we couldn’t see it. I must say I am really happy to see that bias in tech is being recognized and accounted for (for the most part). Please forgive the politics (I'll try really hard not to bash Trump ;-) ), but if there is a silver lining to the 2016 US presidential election I think it is that it has really caused many of us to introspect and realize how thick our echo chamber walls have really gotten over the last few years. The chamber was constructed so quickly, I barely realized it was happening. We're becoming so polarized that we're actually moving to different communities to be with more people "like us." Simple awareness of the problem is a huge step forward in being able to resolve it.