6 ms·

Ok, so let's say discuss your formulation instead: Why can't we 'invert' it, just because it's statistical effect? Yes, in your formulation, some of the new th

by feral 4y ago

Ok, so let's say discuss your formulation instead: Why can't we 'invert' it, just because it's statistical effect? Yes, in your formulation, some of the new things in the pool will go on to live a long time, while others will be selected out. But so what? We are talking about the expected lifetime of an item in the pool, conditioned only on it's age. There's no fundamental problem making a statement that this expected lifetime is short for new things, even if some fraction of those new things will last a long time, right? After all, we don't know any one individual item that's been around a long time will last a lot longer. We only know we expect it to. Because even long lived items have finite lifetime, hence they'll eventually die (and when they do it'll be really surprising, because they've been around so long; but it will happen eventually.) And so the statement is always talking about expected lifetime, whether for items that have already lasted a long or short time. (Hence I still don't think there's really any logical 'inversion' here.)

PaulDavisThe1st 4y ago
> There's no fundamental problem making a statement that this expected lifetime is short for new things, even if some fraction of those new things will last a long time, right? Everything that's new in the pool might be better than everything that's old. All you can say about the old stuff is that it was better (by some metric(s)) than anything it had to compete with so far. But you can't say anything about the new stuff. Sure, statistically it is likely that it will some blend of bad, middling and good, but you don't actually know the mix, or which term describes which items, until after the selection process (i.e. time) has taken place.
- feral 4y ago
  Absolutely there are real situations where the new stuff is going to last longer than the old stuff, even the old stuff that's been through a selection process. E.g. modern manufacturing techniques have improved overall longevity of all 2022 models. But I think you are outside the Lindy effect model at that point. To put this in the example of the original post: The author says visual studio code is expected to last less time than VIM. You could counter by saying: "hey, maybe, uh, the rise of Product Management as a discipline has meant that modern software overall will have longer lifetimes, and hence it's not fair to guess that VIM will outlive VSCode". And that'd be a fine position. But imo the right way to frame that isn't "the author did an incorrect logical inversion of the Lindy effect model"; rather it would be "I don't think the Lindy effect model applies to this domain". (No one is saying the Lindy model is universal.) I guess you could say you want to apply it only within a given year of software; so, we're happy to look backwards and apply the Lindy model to software written in 2011, but we've no idea how to think about the lifespan of software written in 2022, and aren't allowed make any inferences from software written before 2022. That's fine, but that's an additional constraint we've added, is outside the Lindy model, and, really, we're in "all models are wrong, some are useful" territory here, where I'd ask "is it really useful to throw away all that previous data? Wouldn't it be a better starting point to use the lifetimes of previous years as at least a prior?" And if you grant that, then I think theres no logical error here.
- coldtea 4y ago
  >Everything that's new in the pool might be better than everything that's old. It "might", but the empirical observation behind the Lindy effect points that this is unlikely (if we take "better" to mean "more fit to live and grow old and still used"). Sure, we haven't seen the new things develop yet. But we have seen that most new things dont survice time: the things that survive are a small subset of each "new things" (say, vi and emacs, and not one of 100+ 70s programming editors).
killjoywashere 4y ago
If you sampled from a pool of now defunct software projects, or all software projects that were made in some year, you could make some estimate of the survival probability of software projects that are current. But if you can't draw an Kaplan-Meier curve, it's unclear to me how you would assert there exists a survival function that could be inverted.
- pmcp 4y ago
  I’m talking out of ignorance her, so please educate me: if you take into account the black swan theory both kaplan meier and Lindy effect can’t say anything about anything, no?
  - killjoywashere 4y ago
    The point of the black swan is you can't model it. You can't predict COVID-19. You can't predict Russia renouncing it's debt, leading to the collapse of LTCM. You can't predict Hurricane Katrina hitting just so as to push feet of water into Lake Pontchartrain. You can't predict the the 1989 Loma Prieta earthquake. You can know everything there is to know about epidemics, finance, weather, and seismology, and you still can't predict those events. So don't. Do you best to build robust systems, looks for places with excessive efficiencies, and plan mitigation strategies in advance.
osigurdson 4y ago
Quite funny that the concept did not appear to apply to Lindy's restaurant itself which operated from 1921 to 1969 - shutting down 5 years after the term was coined in 1964. https://en.wikipedia.org/wiki/Lindy%27s https://en.wikipedia.org/wiki/Lindy%27s

by feral 4y ago

PaulDavisThe1st 4y ago

> There's no fundamental problem making a statement that this expected lifetime is short for new things, even if some fraction of those new things will last a long time, right? Everything that's new in the pool might be better than everything that's old. All you can say about the old stuff is that it was better (by some metric(s)) than anything it had to compete with so far. But you can't say anything about the new stuff. Sure, statistically it is likely that it will some blend of bad, middling and good, but you don't actually know the mix, or which term describes which items, until after the selection process (i.e. time) has taken place.

feral 4y ago
Absolutely there are real situations where the new stuff is going to last longer than the old stuff, even the old stuff that's been through a selection process. E.g. modern manufacturing techniques have improved overall longevity of all 2022 models. But I think you are outside the Lindy effect model at that point. To put this in the example of the original post: The author says visual studio code is expected to last less time than VIM. You could counter by saying: "hey, maybe, uh, the rise of Product Management as a discipline has meant that modern software overall will have longer lifetimes, and hence it's not fair to guess that VIM will outlive VSCode". And that'd be a fine position. But imo the right way to frame that isn't "the author did an incorrect logical inversion of the Lindy effect model"; rather it would be "I don't think the Lindy effect model applies to this domain". (No one is saying the Lindy model is universal.) I guess you could say you want to apply it only within a given year of software; so, we're happy to look backwards and apply the Lindy model to software written in 2011, but we've no idea how to think about the lifespan of software written in 2022, and aren't allowed make any inferences from software written before 2022. That's fine, but that's an additional constraint we've added, is outside the Lindy model, and, really, we're in "all models are wrong, some are useful" territory here, where I'd ask "is it really useful to throw away all that previous data? Wouldn't it be a better starting point to use the lifetimes of previous years as at least a prior?" And if you grant that, then I think theres no logical error here.
coldtea 4y ago
>Everything that's new in the pool might be better than everything that's old. It "might", but the empirical observation behind the Lindy effect points that this is unlikely (if we take "better" to mean "more fit to live and grow old and still used"). Sure, we haven't seen the new things develop yet. But we have seen that most new things dont survice time: the things that survive are a small subset of each "new things" (say, vi and emacs, and not one of 100+ 70s programming editors).

killjoywashere 4y ago

If you sampled from a pool of now defunct software projects, or all software projects that were made in some year, you could make some estimate of the survival probability of software projects that are current. But if you can't draw an Kaplan-Meier curve, it's unclear to me how you would assert there exists a survival function that could be inverted.

pmcp 4y ago
I’m talking out of ignorance her, so please educate me: if you take into account the black swan theory both kaplan meier and Lindy effect can’t say anything about anything, no?
- killjoywashere 4y ago
  The point of the black swan is you can't model it. You can't predict COVID-19. You can't predict Russia renouncing it's debt, leading to the collapse of LTCM. You can't predict Hurricane Katrina hitting just so as to push feet of water into Lake Pontchartrain. You can't predict the the 1989 Loma Prieta earthquake. You can know everything there is to know about epidemics, finance, weather, and seismology, and you still can't predict those events. So don't. Do you best to build robust systems, looks for places with excessive efficiencies, and plan mitigation strategies in advance.

osigurdson 4y ago

Quite funny that the concept did not appear to apply to Lindy's restaurant itself which operated from 1921 to 1969 - shutting down 5 years after the term was coined in 1964. https://en.wikipedia.org/wiki/Lindy%27s https://en.wikipedia.org/wiki/Lindy%27s