5 ms·

Furthermore, follow https://twitter.com/_brohrer_/status/1425770502321283073 https://twitter.com/_brohrer_/status/1425770502321283073 "When you have a problem,

Furthermore, follow https://twitter.com/_brohrer_/status/1425770502321283073 https://twitter.com/_brohrer_/status/1425770502321283073 "When you have a problem, build two solutions - a deep Bayesian transformer running on multicloud Kubernetes and a SQL query built on a stack of egregiously oversimplifying assumptions. Put one on your resume, the other in production. Everyone goes home happy."

arketyp 5y ago
Furthermore in the article, yes.
hughrr 5y ago
This reminds me of an experience I had watching a company trying to replace a system with ML. First they marketed it heavily before even thinking. During test cycle they fed the entire data corpus in and ran some of the original test cases and found some business destroying results pop out. The entire system ended up a verbatim port of the VB6 crap which was a verbatim port of the original AS400 crap that actually worked. The marketing to this day says it’s ML based and everyone buys into the hype. It’s not. It was a complete failure. But the original system has 30 years of human experience codified in it.
- thecopy 5y ago
  If it worked, why was it crap?
  - bryanrasmussen 5y ago
    VB6 is looked down upon.
    - deleted 5y ago
      [deleted]
    - DonHopkins 5y ago
      For very good reason, by an overwhelming majority of developers. The fact that a few developers thought VB.NET was even worse than VB6 doesn't lessen VB6's dreadfulness, it just highlights VB.NET's dreadfulness. https://en.wikipedia.org/wiki/Visual_Basic_(classic) https://en.wikipedia.org/wiki/Visual_Basic_(classic) >The final release was version 6 in 1998. On April 8, 2008, Microsoft stopped supporting Visual Basic 6.0 IDE. The Microsoft Visual Basic team still maintains compatibility for Visual Basic 6.0 applications through its "It Just Works" program on supported Windows operating systems. >In 2014, some software developers still preferred Visual Basic 6.0 over its successor, Visual Basic .NET. Visual Basic 6.0 was selected as the most dreaded programming language by respondents of Stack Overflow's annual developer survey in 2016, 2017, and 2018. Stack Overflow Developer Survey 2016: Most Dreaded: Visual Basic: 79.5% https://insights.stackoverflow.com/survey/2016#technology-most-loved-dreaded-and-wanted https://insights.stackoverflow.com/survey/2016#technology-mo... Stack Overflow Developer Survey 2017: Most Dreaded: Visual Basic 6: 88.3% https://insights.stackoverflow.com/survey/2017#most-loved-dreaded-and-wanted https://insights.stackoverflow.com/survey/2017#most-loved-dr... Stack Overflow Developer Survey 2018: Most Dreaded: Visual Basic 6: 89.9% https://insights.stackoverflow.com/survey/2018#most-loved-dreaded-and-wanted https://insights.stackoverflow.com/survey/2018#most-loved-dr...
      - cm2187 5y ago
        I think VB6’s bad reputation is that it is stuck in the 90s. I don’t know if any version of a language from that time would be popular now (let’s leave FORTRAN and COBOL aside). I disagree that VB.net was dreadful. But it broke backward compatibility but I think for good reasons: arguments being byref by default in VB6, collections being inconsistently 0 based or 1 based, the SET keyword that wasn’t really serving any purpose and was inconsistently applied, having to provide parameters within brackets or between spaces depending on whether the return value is assigned to a variable or not, etc... I have a lot of sympathy for the frustration of someone who has to maintain a huge code base when backward compatibility is broken, but I think the changes VB.net introduced were necessary.
        relaxing 5y ago
        Plenty of people still write C99.
    - moonchrome 5y ago
      It's been unsupported for 12+ years ? If you have code relying on it and haven't migrated to something supported it means your code is not maintained (don't care what your excuse is, using VB6 in 2020 means you're not actively maintaining the project), written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired and since nobody is actively maintaining it nobody has much knowledge about how it works. So yeah anything that's still running on VB6 is very likely crap.
      - wdb 5y ago
        There is RO Mercury?
      - bryanrasmussen 5y ago
        yeah it wasn't said when it was moved from though? The reference to Machine learning implies more modern, but not necessarily so. At any rate even when it was maintained still looked down on, I guess a Dijkstra based side-effect.
      - joshuaissac 5y ago
        > If you have code relying on it and haven't migrated to something supported it means your code is not maintained No, it does not. It means that Microsoft no longer provides support for the IDE. That does not prevent the developer from maintaining their own VB6 code. With some extra steps, the official IDE and compiler for VB6 can still be installed on Windows 10. Running programs built from VB6 is still supported. > written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired This applies regardless of the programming language to any codebase that has been around for long enough.
        moonchrome 5y ago
        >No, it does not. It means that Microsoft no longer provides support for the IDE. That does not prevent the developer from maintaining their own VB6 code. With some extra steps, the official IDE and compiler for VB6 can still be installed on Windows 10. Running programs built from VB6 is still supported. If you're comfortable with this then I don't think you're actively investing in your software. >This applies regardless of the programming language to any codebase that has been around for long enough. No, if you have a team actively maintaining the project you have the knowledge transfer in-house which is the second part of that sentence.
  - rusk 5y ago
    Presumably what’s meant is it hasn’t improved in 40 years, and even then it was probably just “barely good enough”. This might be considered MVP but that depends on whether you have to actually use it or not.
  - hughrr 5y ago
    People don’t know when something is done, finished and complete. They have to go and fuck around with it. Look at windows for example. Image how good that would be if they didn’t keep trying to fuck around with it and actually finished something.
    - bonoboTP 5y ago
      How do you get promoted and what will you put on your CV if you don't change stuff and just keep the lights on? This is a dilemma all the way from top management to developers. How does a project manager build a career if there are no projects? Change is needed because people want to have jobs and they will make work for themselves if none exists.
      - Ieghaehia9 5y ago
        >Change is needed because people want to have jobs and they will make work for themselves if none exists. Traditional jobs must have solved this problem somehow. You don't usually see e.g. windowmakers or installers coming up with windows in the shapes of superellipses because square windows are already solved, or stoves coming with integrated fridges because "just an oven and a top" is already solved.
        bonoboTP 5y ago
        Most of the window making jobs are just production. In software you don't have the same friction of production. You don't have to type in the source code each time you install the program. But you have to manufacture a new window and physically install it each time which is labor intensive. Also, the window designer job is not hyped as much as IT jobs are. Furthermore, you do see household appliances getting fitted with useless feature bloat and shoddy software and wireless and touchscreens on microwaves etc. It happens. IoT, subscription based software updates for power drills etc... Tractors that can't be repaired and contain a jumble of proprietary software as a service etc.
        raducu 5y ago
  - atoav 5y ago
    There is a certain value in understanding why something works and how you can either continously improve it or adjust a few dials when there is an exceptional situation. Part of the fascination with ML is the (dangerous) myth that you don't have to wrap your head around a complicated problem anymore, instead the solution will just magically fall out on the other side of the blackbox if you just feed it enough data. Understand ing the intricates of the problems you are dealing with however is a value in itself.
    - mdp2021 5y ago
      Yes, but (please allow me): «Part of the fascination with ML is» solving the mystery behind the ability to automatically build functions and behind those functions. Surely, both in practice and axiologically, understanding and deterministically solving have a great value. Also because of that, the fact that systems exist that can adapt into solutions, but contain a transparency problem ("yes, but why"), contains an immensely fascinating theoretical challenge, in the learning that may come from the attempt to understand the "grown, spawned" (as if a natural phenomenon) system. The laziness is not necessary: there is a great deal of fascination in unveiling the mysteries in the blackbox. Then of course, when you have a practical problem to solve (instead of that intellectual challenge and promise), pick your best solution. And surely it is sensible to call it dangerous to rely on something not properly understood, which may hide the potential faults ("yes, we found out it fails here, and it may be that we kind of assumed it "saw" shapes, while really it "sees" textures..."). In professional practice those "active" fascinations (understanding the spawned) may be luxury.
      - deleted 5y ago
        [deleted]
      - wnkrshm 5y ago
        While the methods are very interesting, I often wonder about assumptions about what can be modeled. We already know that it's not possible to correctly predict an arbitrary nonlinear system numerically (since that system could be chaotic). It's one of the reasons why for specific problems, heuristics or statistics are way better than any attempt at nonlinear modeling / ML prediction (e.g. highly accurate climate models vs. struggling weather models).
  - patall 5y ago
    One thing that I could imagine: it bases it's decision only on few (or the 'wrong') features while you (or marketing) want to consider more. We have had a project where we were asked if our model would consider X. So we added X to the model but this didn't increase performance. Now the sane, simple answer would be to just ignore X. But then people come and ask why, doubt that it doesn't improve results, competition without ML considers X. That doesn't happen (or is hidden) in a none ML situation where some decisions aren't questioned by a benchmark.
- ethbr0 5y ago
  If I had a nickel for every time I've seen "business rules engine" turned into "AI" in the last few years... But I guess if we complain that half of our colleagues and the media don't understand ML, why should we expect management to? When the command from C-level is "We need some AI projects to tell our shareholders about," we shouldn't be surprised when middle management suddenly has successful AI projects in their slide decks.
  - mjburgess 5y ago
    Well really-existing AI is just either "taking a mean()" or "programming a rule". So all of programming actually counts as (symbolic) AI.
  - cm2187 5y ago
    In the same way the terms “blockchain” being used for “digital signing” or “cloud” for a server...
  - i_am_proteus 5y ago
    If you have an existing rules-based decision-tree system, and you compare its performance with a bunch of other decision trees, and it does better, you are implementing a random forest that happens to be identical to your original system. Artificial Intelligence.
    - mumblemumble 5y ago
      If you talk enough all the models and hyperparameters you compared and suchlike that you experimented with, you can probably sufficiently impress people with the talk about the enormous deep learning model you spent several months developing that they won't even remember you mentioning the two-clause Boolean expression that you actually put into production. And of course it's AI. You used k-fold cross validation to select it.
- np_tedious 5y ago
  Isn't your organization itself a machine the learned these rules over time? Maybe the marketing checks out
- pedrocr 5y ago
  The AI taxonomy includes the term "Expert Systems" for these kinds of things. On the one hand it's definitely not of the new wave of ML AI so hyping those things as innovative is off. On the other hand we should definitely give more attention to that kind of setup and understand how to build/maintain/test it properly. Otherwise often it ends up being ran by a few hundred Excel sheets and a few severely underpaid people and that's a disaster waiting to happen. The AS400->VB6->NewShiny path actually sounds like a success case given the messes that are out there.
  - urthor 5y ago
    Rule engine is the term I believe. Often the biggest benefit is that the ML version is good at catching when the experts hadn't had their cup of coffee as well. Most experts are like family doctors, they get the correct diagnosis 70% of the time. And even if you juice them up real good, they will ALWAYS lose 5% to human error. The ML also hits the 70% mark, but it's a different 70%, so it'll fix 70% of the errors. Then you're batting at 0.91 instead of 0.70.
- knodi123 5y ago
  We did the same thing when I worked for a resume search/sort/share site. Built a big ML tool that could look at job listings and resumes and pick who was best for each job. Our training data set was millions of resumes, hundreds of thousands of jobs, and in most of those jobs, we could say which resumes got shortlisted and which resumes got hired. In the end, it gave basically the same results as keyword searching. But we marketed the shit out of it.
q-base 5y ago
That quote is seriously brilliant! Thanks for sharing.
DonHopkins 5y ago
Just don't build one solution to your problem with regular expressions: then you have two problems.
smichel17 5y ago
TFA ends with that quote.
bostonpete 5y ago
Well, the article does conclude with that exact tweet...
NumberCruncher 5y ago
The "the right tool for the right job" applies for ML topics too. If the job involves "looking smart and innovative" for whatever reasons, people tend to err on the side of overly complex solutions. On the other hand if the advice "let's just go with an SQL query built on a stack of egregiously oversimplifying assumptions" comes from someone, who doesn't know how SQL and linear regression / logistic regression with binning/bucketing / simple decision trees work, I would ask for a second opinion. Because a huge part of the retail banking, non-life insurance and marketing business is running on this simple stack. Obviously profitable. If the same advice comes from someone, who knows when to use deep learning instead of XGBoost and why, I would go with his/her advice. And I would try to keep him happy and on my team.
marcosdumay 5y ago
My workplace has got all kinds of attention for building a blockchain based data collection system that encompasses an entire sector of the economy. It's "almost done", so we are right now starting a simple set of REST services that write into a badly normalized transactional database just in case it stays "almost done" for too long.
arnaudsm 5y ago
This isn't ironic, I've actually done that multiple times in a large company. No one noticed, everyone went home happy.

arketyp 5y ago

Furthermore in the article, yes.

hughrr 5y ago

This reminds me of an experience I had watching a company trying to replace a system with ML. First they marketed it heavily before even thinking. During test cycle they fed the entire data corpus in and ran some of the original test cases and found some business destroying results pop out. The entire system ended up a verbatim port of the VB6 crap which was a verbatim port of the original AS400 crap that actually worked. The marketing to this day says it’s ML based and everyone buys into the hype. It’s not. It was a complete failure. But the original system has 30 years of human experience codified in it.

thecopy 5y ago
If it worked, why was it crap?
- bryanrasmussen 5y ago
  VB6 is looked down upon.
  - deleted 5y ago
    [deleted]
  - DonHopkins 5y ago
    For very good reason, by an overwhelming majority of developers. The fact that a few developers thought VB.NET was even worse than VB6 doesn't lessen VB6's dreadfulness, it just highlights VB.NET's dreadfulness. https://en.wikipedia.org/wiki/Visual_Basic_(classic) https://en.wikipedia.org/wiki/Visual_Basic_(classic) >The final release was version 6 in 1998. On April 8, 2008, Microsoft stopped supporting Visual Basic 6.0 IDE. The Microsoft Visual Basic team still maintains compatibility for Visual Basic 6.0 applications through its "It Just Works" program on supported Windows operating systems. >In 2014, some software developers still preferred Visual Basic 6.0 over its successor, Visual Basic .NET. Visual Basic 6.0 was selected as the most dreaded programming language by respondents of Stack Overflow's annual developer survey in 2016, 2017, and 2018. Stack Overflow Developer Survey 2016: Most Dreaded: Visual Basic: 79.5% https://insights.stackoverflow.com/survey/2016#technology-most-loved-dreaded-and-wanted https://insights.stackoverflow.com/survey/2016#technology-mo... Stack Overflow Developer Survey 2017: Most Dreaded: Visual Basic 6: 88.3% https://insights.stackoverflow.com/survey/2017#most-loved-dreaded-and-wanted https://insights.stackoverflow.com/survey/2017#most-loved-dr... Stack Overflow Developer Survey 2018: Most Dreaded: Visual Basic 6: 89.9% https://insights.stackoverflow.com/survey/2018#most-loved-dreaded-and-wanted https://insights.stackoverflow.com/survey/2018#most-loved-dr...
    - cm2187 5y ago
      I think VB6’s bad reputation is that it is stuck in the 90s. I don’t know if any version of a language from that time would be popular now (let’s leave FORTRAN and COBOL aside). I disagree that VB.net was dreadful. But it broke backward compatibility but I think for good reasons: arguments being byref by default in VB6, collections being inconsistently 0 based or 1 based, the SET keyword that wasn’t really serving any purpose and was inconsistently applied, having to provide parameters within brackets or between spaces depending on whether the return value is assigned to a variable or not, etc... I have a lot of sympathy for the frustration of someone who has to maintain a huge code base when backward compatibility is broken, but I think the changes VB.net introduced were necessary.
      - relaxing 5y ago
        Plenty of people still write C99.
  - moonchrome 5y ago
    It's been unsupported for 12+ years ? If you have code relying on it and haven't migrated to something supported it means your code is not maintained (don't care what your excuse is, using VB6 in 2020 means you're not actively maintaining the project), written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired and since nobody is actively maintaining it nobody has much knowledge about how it works. So yeah anything that's still running on VB6 is very likely crap.
    - wdb 5y ago
      There is RO Mercury?
    - bryanrasmussen 5y ago
      yeah it wasn't said when it was moved from though? The reference to Machine learning implies more modern, but not necessarily so. At any rate even when it was maintained still looked down on, I guess a Dijkstra based side-effect.
    - joshuaissac 5y ago
      > If you have code relying on it and haven't migrated to something supported it means your code is not maintained No, it does not. It means that Microsoft no longer provides support for the IDE. That does not prevent the developer from maintaining their own VB6 code. With some extra steps, the official IDE and compiler for VB6 can still be installed on Windows 10. Running programs built from VB6 is still supported. > written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired This applies regardless of the programming language to any codebase that has been around for long enough.
      - moonchrome 5y ago
        >No, it does not. It means that Microsoft no longer provides support for the IDE. That does not prevent the developer from maintaining their own VB6 code. With some extra steps, the official IDE and compiler for VB6 can still be installed on Windows 10. Running programs built from VB6 is still supported. If you're comfortable with this then I don't think you're actively investing in your software. >This applies regardless of the programming language to any codebase that has been around for long enough. No, if you have a team actively maintaining the project you have the knowledge transfer in-house which is the second part of that sentence.
- rusk 5y ago
  Presumably what’s meant is it hasn’t improved in 40 years, and even then it was probably just “barely good enough”. This might be considered MVP but that depends on whether you have to actually use it or not.
- hughrr 5y ago
  People don’t know when something is done, finished and complete. They have to go and fuck around with it. Look at windows for example. Image how good that would be if they didn’t keep trying to fuck around with it and actually finished something.
  - bonoboTP 5y ago
    How do you get promoted and what will you put on your CV if you don't change stuff and just keep the lights on? This is a dilemma all the way from top management to developers. How does a project manager build a career if there are no projects? Change is needed because people want to have jobs and they will make work for themselves if none exists.
    - Ieghaehia9 5y ago
      >Change is needed because people want to have jobs and they will make work for themselves if none exists. Traditional jobs must have solved this problem somehow. You don't usually see e.g. windowmakers or installers coming up with windows in the shapes of superellipses because square windows are already solved, or stoves coming with integrated fridges because "just an oven and a top" is already solved.
      - bonoboTP 5y ago
        Most of the window making jobs are just production. In software you don't have the same friction of production. You don't have to type in the source code each time you install the program. But you have to manufacture a new window and physically install it each time which is labor intensive. Also, the window designer job is not hyped as much as IT jobs are. Furthermore, you do see household appliances getting fitted with useless feature bloat and shoddy software and wireless and touchscreens on microwaves etc. It happens. IoT, subscription based software updates for power drills etc... Tractors that can't be repaired and contain a jumble of proprietary software as a service etc.
        raducu 5y ago
- atoav 5y ago
  There is a certain value in understanding why something works and how you can either continously improve it or adjust a few dials when there is an exceptional situation. Part of the fascination with ML is the (dangerous) myth that you don't have to wrap your head around a complicated problem anymore, instead the solution will just magically fall out on the other side of the blackbox if you just feed it enough data. Understand ing the intricates of the problems you are dealing with however is a value in itself.
  - mdp2021 5y ago
    Yes, but (please allow me): «Part of the fascination with ML is» solving the mystery behind the ability to automatically build functions and behind those functions. Surely, both in practice and axiologically, understanding and deterministically solving have a great value. Also because of that, the fact that systems exist that can adapt into solutions, but contain a transparency problem ("yes, but why"), contains an immensely fascinating theoretical challenge, in the learning that may come from the attempt to understand the "grown, spawned" (as if a natural phenomenon) system. The laziness is not necessary: there is a great deal of fascination in unveiling the mysteries in the blackbox. Then of course, when you have a practical problem to solve (instead of that intellectual challenge and promise), pick your best solution. And surely it is sensible to call it dangerous to rely on something not properly understood, which may hide the potential faults ("yes, we found out it fails here, and it may be that we kind of assumed it "saw" shapes, while really it "sees" textures..."). In professional practice those "active" fascinations (understanding the spawned) may be luxury.
    - deleted 5y ago
      [deleted]
    - wnkrshm 5y ago
      While the methods are very interesting, I often wonder about assumptions about what can be modeled. We already know that it's not possible to correctly predict an arbitrary nonlinear system numerically (since that system could be chaotic). It's one of the reasons why for specific problems, heuristics or statistics are way better than any attempt at nonlinear modeling / ML prediction (e.g. highly accurate climate models vs. struggling weather models).
- patall 5y ago
  One thing that I could imagine: it bases it's decision only on few (or the 'wrong') features while you (or marketing) want to consider more. We have had a project where we were asked if our model would consider X. So we added X to the model but this didn't increase performance. Now the sane, simple answer would be to just ignore X. But then people come and ask why, doubt that it doesn't improve results, competition without ML considers X. That doesn't happen (or is hidden) in a none ML situation where some decisions aren't questioned by a benchmark.
ethbr0 5y ago
If I had a nickel for every time I've seen "business rules engine" turned into "AI" in the last few years... But I guess if we complain that half of our colleagues and the media don't understand ML, why should we expect management to? When the command from C-level is "We need some AI projects to tell our shareholders about," we shouldn't be surprised when middle management suddenly has successful AI projects in their slide decks.
- mjburgess 5y ago
  Well really-existing AI is just either "taking a mean()" or "programming a rule". So all of programming actually counts as (symbolic) AI.
- cm2187 5y ago
  In the same way the terms “blockchain” being used for “digital signing” or “cloud” for a server...
- i_am_proteus 5y ago
  If you have an existing rules-based decision-tree system, and you compare its performance with a bunch of other decision trees, and it does better, you are implementing a random forest that happens to be identical to your original system. Artificial Intelligence.
  - mumblemumble 5y ago
    If you talk enough all the models and hyperparameters you compared and suchlike that you experimented with, you can probably sufficiently impress people with the talk about the enormous deep learning model you spent several months developing that they won't even remember you mentioning the two-clause Boolean expression that you actually put into production. And of course it's AI. You used k-fold cross validation to select it.
np_tedious 5y ago
Isn't your organization itself a machine the learned these rules over time? Maybe the marketing checks out
pedrocr 5y ago
The AI taxonomy includes the term "Expert Systems" for these kinds of things. On the one hand it's definitely not of the new wave of ML AI so hyping those things as innovative is off. On the other hand we should definitely give more attention to that kind of setup and understand how to build/maintain/test it properly. Otherwise often it ends up being ran by a few hundred Excel sheets and a few severely underpaid people and that's a disaster waiting to happen. The AS400->VB6->NewShiny path actually sounds like a success case given the messes that are out there.
- urthor 5y ago
  Rule engine is the term I believe. Often the biggest benefit is that the ML version is good at catching when the experts hadn't had their cup of coffee as well. Most experts are like family doctors, they get the correct diagnosis 70% of the time. And even if you juice them up real good, they will ALWAYS lose 5% to human error. The ML also hits the 70% mark, but it's a different 70%, so it'll fix 70% of the errors. Then you're batting at 0.91 instead of 0.70.
knodi123 5y ago
We did the same thing when I worked for a resume search/sort/share site. Built a big ML tool that could look at job listings and resumes and pick who was best for each job. Our training data set was millions of resumes, hundreds of thousands of jobs, and in most of those jobs, we could say which resumes got shortlisted and which resumes got hired. In the end, it gave basically the same results as keyword searching. But we marketed the shit out of it.

q-base 5y ago

That quote is seriously brilliant! Thanks for sharing.

DonHopkins 5y ago

Just don't build one solution to your problem with regular expressions: then you have two problems.

smichel17 5y ago

TFA ends with that quote.

bostonpete 5y ago

Well, the article does conclude with that exact tweet...

NumberCruncher 5y ago

The "the right tool for the right job" applies for ML topics too. If the job involves "looking smart and innovative" for whatever reasons, people tend to err on the side of overly complex solutions. On the other hand if the advice "let's just go with an SQL query built on a stack of egregiously oversimplifying assumptions" comes from someone, who doesn't know how SQL and linear regression / logistic regression with binning/bucketing / simple decision trees work, I would ask for a second opinion. Because a huge part of the retail banking, non-life insurance and marketing business is running on this simple stack. Obviously profitable. If the same advice comes from someone, who knows when to use deep learning instead of XGBoost and why, I would go with his/her advice. And I would try to keep him happy and on my team.

marcosdumay 5y ago

My workplace has got all kinds of attention for building a blockchain based data collection system that encompasses an entire sector of the economy. It's "almost done", so we are right now starting a simple set of REST services that write into a badly normalized transactional database just in case it stays "almost done" for too long.

arnaudsm 5y ago

This isn't ironic, I've actually done that multiple times in a large company. No one noticed, everyone went home happy.