6 ms·

Forcing Functions in Software Development

107 points by kiwiandroiddev 6y ago

dkersten 6y ago
> Try to add support for a completely different database. Details of your current database that have leaked into your data layer abstractions will soon become obvious Well, details of Postgres have leaked into my database queries and schemas, but it was a conscious decision to use Postgres and its features. Sure, it seems nice to be able to swap one database for another, but you lose out on a lot of what a database can do if you stick strictly for the lowest common denominator of features. I use Postgres partly because of its feature set, so I am going to use these features. This does mean that its unlikely I will ever run my software against a different SQL database, but I'm ok with this. I guess an important note is that you should be aware of it and it should be a conscious decision, rather than something that crept in over time. But I suppose that's a tangent and not the articles point. Forcing Functions is about unearthing the brittleness that has crept into a codebase over time and I absolutely agree that is a helpful and worthwhile exercise. I've seen plenty of codebases that were meant to be database agnostic, but porting them was still not painless.
- stingraycharles 6y ago
  Yeah I don’t get that obsession with “pluggable databases”. Abstractions have a real cost, and they typically complicate a lot: I would advice to just make sure you centralize the access to your database in a single module / class / whatever, but other than that, swapping out databases seems like a non-goal to me. How often do people really migrate to a different database? Even when doing so, migrating the existing data always is a lot more tedious than migrating the code, in my experience.
  - inopinatus 6y ago
    It's more a concern when shipping library/platform code. Applications can be as tightly coupled as they like.
    - dkersten 6y ago
      Should library code really be relying on a database? If its a database library, then ok, maybe it can stick to a subset of database features that works across all supported databases, but in that case, testing against these databases is no longer a "Forcing Function", but a core part of the libraries test suite to ensure correctness. The same goes for platform code, really. If you are advertising your platform to work with databases X, Y and Z, then you really need to be testing against those and not just as a Forcing Function to see if any brittleness crept in, but as a core part of ensuring your platform does as advertised.
      - inopinatus 6y ago
        > Should library code really be relying on a database Yes, many libraries need, or are enhanced by, persistence, whether that's a double-entry accounting module, a headless cms, or an e-commerce engine. > you really need to be testing against those Library authors need tools and guidance, not arbitrary constraints and high-watermark QA demands. Most of us have one or two DBs that we work with day-to-day but still want libraries we contribute to be broadly portable across any of the backends the framework they plug into supports. e.g. I can't maintain a test suite vs Oracle when I don't license it, but I still want to know that It'll Just Work if someone uses it in such combination, or that it's at least close enough they'll not have trouble making it work. More broadly I think the myth of testing every supported combination leads to a brittle mindset of saying "not supported, won't help", especially when systematised in a commercial environment, and to me it's the antithesis of good engineering.
        dkersten 6y ago
        Well, if your library can use the "lowest denominator" of supported databases, then there's nothing wrong with doing that. I'd still argue that you don't really support something you haven't tested against, but I guess this is pretty off topic. My overall point was more that sometimes being locked into a single database is a conscious decision because you want to make use of everything a database has to offer. In the context of the article, for "Forcing Functions", I absolutely agree that switching databases is a useful way to find weakness in your solution.
  - dkersten 6y ago
    > How often do people really migrate to a different database? Well, I have been part of a migration from MySQL to MariaDB, which was a lot more effort than one would expect given that they're meant to be more or less the same thing. It was a ton of effort and the abstracted ORM logic didn't actually help with this. So if it doesn't really help that much for a simple case like that, then it doesn't seem like there's much point in my opinion, as porting is going to be work either way (in a non-trivial application with non-trivial data access patterns, at least).
    - Kiro 6y ago
      I've done the same migration and I'm curious of what issues you had. For me, it was truly just plug'n'play with no additional work whatsoever.
      - dkersten 6y ago
        I don't remember, it was ~4 years ago and for a different company. I believe there were a small number of incompatibilities somewhere (composite indexes maybe? I remember doing stuff with them but don't remember why, could have just been for query optimisation) and a few areas where the performance was degraded (again, I think around indexes, but I could be mistaken). It wasn't a big deal over all, but it wasn't simple plug'n'play either and required some porting work.
  - pydry 6y ago
    I've been considering moving away from azure postgres to azure SQL server coz azure postgres runs at a snails pace.
    - speedgoose 6y ago
      Azure is ridiculously slow unless you ask for decent IOs performance but then it's ridiculously expensive.
      - pydry 6y ago
        Is that true for azure sql server too? I suspected it was faster but I haven't checked.
  - alzoid 6y ago
    I agree, I see those patterns in the wild a lot. You have a Microsoft shop using .NET and SQL Server yet the devs still abstract the data access layer. I think those patterns just became common 'just in case' they were needed. I find refactoring tools like "extract interface" take care of that so there is not need to write the abstraction until it's needed. On the other side, I worked on a web app that supported multiple db vendors, we did the classic DAO pattern which worked well. You still get to use custom SQL for each database if you need to. We tried an ORM at one point which worked out well. It was the same web app and we moved moved some DAO code to the Java Persistence API. We could then build the data access code and include it into our desktop (Mac Windows) and plug it in to a local DB (Derby). In that case, once JPA was working, the pluggable database was allowing us to save on development costs.
  - Chris_Newton 6y ago
    Yeah I don’t get that obsession with “pluggable databases”. Abstractions have a real cost, and they typically complicate a lot: I would advice to just make sure you centralize the access to your database in a single module / class / whatever, but other than that, swapping out databases seems like a non-goal to me. I tend to agree, as long as the database you’re using is of the free and open source type so it’s a low-risk dependency. The “pluggability principle” holds more strongly for higher-risk dependencies, IMHO. For example, if you’re working in an online environment and you can’t readily integrate a new payment processor or messaging service or whatever, your existing dependency starts to look like a single point of failure that is not under your control. Some careful abstraction of the essential features and avoiding dependencies on the peculiarities of any specific platform can be a valuable safety blanket if your external dependency catches fire one day.
    - jacques_chester 6y ago
      It's an example of what economists call "The Hold-up Problem", as part of "Incomplete Contracts". Basically, it's impossible to write a contract to cover all situations, so eventually any two parties will need to renegotiate. If one has leverage over the other, they can do stuff like raise prices or simply refuse service. One factor affecting that leverage is "asset specificity": given an asset that's covered by the contract, is it reusable in different contexts or is it too closely adapted to one purpose? The literal textbook example is a steel foundry and a railroad. The steel company needs a spur line to its foundry to receive raw materials and send out finished steel. The railroad company can run a spur from its main line to the foundry. In this case the asset (the spur line) is extremely specific. Without the spur line, the foundry is worthless. Without the foundry, the spur line is worthless. The balance of leverage then comes down to relative costs and gains. The railroad will hold the upper hand, because closing a spur line isn't existential, but it is for the steel foundry. In the software context it shows up as a "contract" between your software and a database. You can use an ORM to try to insulate yourself, reducing asset specificity, but that decision has costs (eg, specific DB features you must forego). You can choose to increase asset specificity by adapting to only one DB and fully utilising its strengths, but you may face a future hold-up. Note from the train example that hold-ups needn't be "pay me more". They can also be "I'm walking away". So a hold-up from Oracle will be "pay me more", whereas a hold-up from an obscure database with one developer could be that they simply quit, die or otherwise become incapable of working.
  - toong 6y ago
    If you are an enterprise software vendor (non-saas), orgs expect you use _their_ db. If they already have an Oracle license, or a MS-SQL DBA around, you either say goodbye to Postgres or give the contract to your competitor.
- MaxBarraclough 6y ago
  Agreed. It makes little sense to try to write your SQL queries to run fine on multiple different DBMSs. Different SQL DBMSs should be treated as different languages. Presumably the idea is based on an analogy to code portability: it can be good to ensure your C++ code compiles fine with multiple different compilers. Really though, it's more akin to writing code that compiles as both C# and Java; clearly madness.
  - deleted 6y ago
    [deleted]
- barrkel 6y ago
  I don't significantly disagree with you, but let me take the devil's advocate approach. Adding support for a different database doesn't mean restricting yourself to the lowest common denominator. It means using different techniques, more appropriate for the different database, that may optimize other parts of your data access and modification path, while pessimizing stuff your current database does. More importantly, it means extracting hard database dependencies like raw SQL or custom ORM fiddling from your business logic and entities, and pushing them behind a module or service boundary. Raw leverage of the database, if it's dispersed throughout your application, will limit your ability to change your schema (e.g. denormalize an attribute, split or join tables, convert a parent-child relationship to embedded JSON or vice versa) and address performance problems as you scale up. It'll also stop you having a single point of data access where you can partition or duplicate your data into different stores with different capabilities more suited to their access and modification patterns. These kinds of things become really important when the database becomes a bottleneck in your system.
  - dkersten 6y ago
    Just checking if I understood your point: for the purpose of Forcing Functions, running against a different database than the one designed for (and therefore where the tradeoffs may be different and things may run inefficiently) is still useful because it helps unearth design flaws, corner cases or brittleness? If so, then, sure , I agree with you. Not all reliance on a target database is actual features that don't have an easy or direct way to port.
    - barrkel 6y ago
      Almost. More that being forced to separate church and state, forced to keep business logic separate from database manipulation, and forced to go through an abstraction interface, then lets you leverage the interface later if and when the database becomes a bottleneck. It's a devil's advocate position. I actually think it's unrealistic and probably not worth it at the time. The forcing function is what's useful, not the database portability per se. And whether it's actually useful depends on being so successful that you have so much data that your regular RDBMS can't cope with some access or update patterns. That's a hefty bet early in anything, and probably doesn't make business sense, until it does, then you wish you did things differently. (Not coincidentally, it's a position we're in at my workplace; the database has hit its limits and we're needing to use hybrid approaches to hit latency NFRs.)
- LandR 6y ago
  I took this to mean be able to swap in and out your persistence layer. So my app can make calls which will persist data to a SQL database, or I can swap that layer out with another that persists to NoSql storage in the cloud. Possible because the persistence layer exposes an API or interface that is agnostic to the actual implementation of the layer ???
xupybd 6y ago
Who has time for this? I've always been on under resourced projects where every hour had to be signed off. Everything is a rush and quality is required but never budgeted for.
- auggierose 6y ago
  Time to change jobs then.
  - swyx 6y ago
    HN resorts to the "just quit your job lol" far too easily. people are in all sorts of different situations. you don't know if they are struggling to find work and this is the one shot they have been given. you don't know if they are in extreme demand and get paid well to fix bad situations like this. you just don't know. and telling people to blithely change jobs when they can't or don't want to isn't helpful
    - mjburgess 6y ago
      Indeed, I encounter this anti-pragmatic personality a lot in tech people. It's an extreme intolerance to imperfect circumstances: a preference for nothing at all over compromise. The issue is that tech attracts the mathemetically-minded who reasons from universal principles. Rather than the empirically-minded who start with cases, and abduce to provisional principles from those. To a aximoatic mind: when a universal principle is violated, the situation is declared Bad. To the case-base mind: when a tolerable situation seems to violate a principle, declare the principle Inapplicable. Of course both types of thought are helpful in different contexts, I suspect 'the management of one's life, day to day' however, should be a matter of case-base reasoning to rough principle.
      - couchand 6y ago
        Okay, that's a reasonable universal principle, but in this case it's Inapplicable. To quote the OP: > Everything is a rush and quality is required but never budgeted for. It doesn't really sound like this is a situation where the principle is wrong, it sounds like this is proving the principle correct: you get what you pay for.
tasogare 6y ago
> Delete the project from your development machine, clone the source code and set it up from scratch. Very good advice. This was obviously never done in my org, leading to newcomers (me included) wasting weeks to get started on some projects. Once it was fixed, a newcomer can start working in an hour.
- 2rsf 6y ago
  actually having a newcomer install everything and fix the documentation while doing it is even a better option, cloning has some of the dependencies but other might have been installed by the developer for example global packages for NPM or Python, setting of system wide environment variables etc.
  - nullsense 6y ago
    This happened at my org... 8 separate times. It was always slightly wrong each time, even after it was fixed. I think it's better now, but I never considered doing it myself as an established team member. Though I once inherited a large enterprise code base that I had to study and build a dev environment for pretty much all by myself. I had maybe 2 calls with the original team and a couple of emails but was mostly by myself just figuring it out. It was a pretty incredible experience and taught me a tonne about how the system worked and was put together. This helped immensely when we spun up a team to work on it. So, I get where this article is coming from due to that experience, but didn't think to do this kind of stuff on purpose.
    - OJFord 6y ago
      Unless you're hiring like crazy, it's never going to be and stay perfect, but hopefully with fixing whatever issues each time you stay relatively on top of it, and it's never too much work.
vagrantJin 6y ago
At an agency, we used to run our web apps on some crappy 08 model laptops running on a gig of memory with outdated browsers. If the webapp ran there without major hitches, it was considerd good enough. It made everyone on the team think hard about optimizing even before a single line of code was written. It really did force excessive simplicity and not jumping on new libs/frameworks just because we can.
mikewarot 6y ago
Long ago, in the mists of time, (1987) I wrote a program that worked with hand held computers, barcode scanners, and was meant to be run by folks working in a power plant. I wrote the code, and got it working in about 2 months. It did everything in the spec, and the customer (Russ) loved what we had, but it turned out (of course) there were many things missed in the spec. A week or two later, we worked out a deal for all of their power plants, provided I would work with Russ to make sure it did everything they wanted... it would take about a year, in the end. Russ taught me everything I needed to know about users and how they think, in a direct and very effective manner. He was the assistant plant manager, so he would pull a random person into the office, and say to them "I know you're not a computer person, I want to you to do X,Y,Z... but don't worry... if anything goes wrong, it's not your fault... it's Mike's fault (Russ points at Me). He then tells me to just watch... It took exactly 1 minute to learn the first lesson... there should always be "Press F1 for Help" somewhere on the screen. It was a very instructive and productive year. I've carried those lessons from his forcing function with me for decades. I love telling the story, thanks for listening.
- navaati 6y ago
  Thanks for telling. It's heartwarming to hear a story of competency, problems solved, and efficiency for once.
- tylerscott 6y ago
  That was great story. Thanks for sharing!
glenjamin 6y ago
If a tree falls in a forest with no-one around, does it make a sound? While the list of techniques here look like excellent ways to uncover unknown unknowns, be sure that it's actually valuable for you to resolve these issues. In almost all cases the goal isn't to create perfect software, it's to create effective software - and sometimes the ROI on these unusual cases doesn't stack up (or doesn't stack up _yet_).
gridlockd 6y ago
I don't buy the "bugs caught early are cheaper to fix" paradigm. Most bugs are caught early without special precautions, but special precautions can be quite expensive. Most software doesn't have truly catastrophic failure cases. If there's a bug slipping through, someone will run into it, report it, and it'll get fixed. If nobody runs into it, the bug doesn't cost anything. This is the most economic way to go about it, which is the reason why pretty much all successful software is kind of buggy. We all like to complain about it, but then we don't want to wait an extra year for the next version either. At the other end of the spectrum, if you need really reliable software, the solution is not to eliminate all the bugs, that's impossible. Even with perfect software, hardware can fail, bits can flip the wrong way. The solution is to make sure that errors can't bring down the airplane.
- cjfd 6y ago
  I don't think so. When one is fixing bugs locally before the thing goes into production one can use each and every debugging strategy under the sun to find the bug. If it runs 10 times slower because of generating massive logs that is acceptable when debugging locally. In production one is constrained to debugging techniques that do not disturb production. This constraint quite often makes debugging times longer by factors of 10 if not 100 while at the same time a customer is getting somewhat impatient. Also, the bug that would be discoverable locally but not in production is a bit of a strange animal. It does happen for cases where the developer has a better idea than the user what should be happening so the user will not even notice the bug. That sounds more like features that have been specified in too much detail than true bugs, though. More commonly everything that can go wrong locally will go wrong in production sooner or later. And in production you will, on top of that, hit all of the bugs that occur once every month and that only occur if the order of inputs is a bit strange while the database is under some load. If you have not done all possible debug/test work locally the application will hit production when there still is a lot of debug work to do and there will probably be so many problems that they even start interacting with one another and produce some really 'interesting' failures.
- dockd 6y ago
  This is just pushing testing to the customer. Why would the customer report a bug instead of switching to a less buggy product? Why do you think the customers are going to report bugs? I can't say that I've ever attempted to report a bug on an iPhone app, Windows, etc. It's like yelling at the wind.
hyperpape 6y ago
The reference to bugs being more cheaper to fix the earlier they're found is one of those claims that gets passed around without much attempt to look at the supposed sources. I believe the most common citation trail bottoms out in a study that no one has access to anymore, and the whole thing seems dodgy. One link: https://www.techwell.com/techwell-insights/2013/10/what-does-it-really-cost-fix-software-defect https://www.techwell.com/techwell-insights/2013/10/what-does....
- jasonpeacock 6y ago
  Checkout the book "Accelerate", it covers a lot of operation excellence practices supported by data, including shortening the iteration cycle (e.g. discovering bugs early): https://smile.amazon.com/Accelerate-Software-Performing-Technology-Organizations-ebook/dp/B07B9F83WM https://smile.amazon.com/Accelerate-Software-Performing-Tech...
- hyperpape 6y ago
  Here is more from Laurent Bossavit, showing the history of shoddy citations surrounding the 100x claim: https://gist.github.com/Morendil/f9c2e9f3f450d3a76de8aeee7cf2bd00 https://gist.github.com/Morendil/f9c2e9f3f450d3a76de8aeee7cf....

> Try to add support for a completely different database. Details of your current database that have leaked into your data layer abstractions will soon become obvious Well, details of Postgres have leaked into my database queries and schemas, but it was a conscious decision to use Postgres and its features. Sure, it seems nice to be able to swap one database for another, but you lose out on a lot of what a database can do if you stick strictly for the lowest common denominator of features. I use Postgres partly because of its feature set, so I am going to use these features. This does mean that its unlikely I will ever run my software against a different SQL database, but I'm ok with this. I guess an important note is that you should be aware of it and it should be a conscious decision, rather than something that crept in over time. But I suppose that's a tangent and not the articles point. Forcing Functions is about unearthing the brittleness that has crept into a codebase over time and I absolutely agree that is a helpful and worthwhile exercise. I've seen plenty of codebases that were meant to be database agnostic, but porting them was still not painless.

stingraycharles 6y ago
Yeah I don’t get that obsession with “pluggable databases”. Abstractions have a real cost, and they typically complicate a lot: I would advice to just make sure you centralize the access to your database in a single module / class / whatever, but other than that, swapping out databases seems like a non-goal to me. How often do people really migrate to a different database? Even when doing so, migrating the existing data always is a lot more tedious than migrating the code, in my experience.
- inopinatus 6y ago
  It's more a concern when shipping library/platform code. Applications can be as tightly coupled as they like.
  - dkersten 6y ago
    Should library code really be relying on a database? If its a database library, then ok, maybe it can stick to a subset of database features that works across all supported databases, but in that case, testing against these databases is no longer a "Forcing Function", but a core part of the libraries test suite to ensure correctness. The same goes for platform code, really. If you are advertising your platform to work with databases X, Y and Z, then you really need to be testing against those and not just as a Forcing Function to see if any brittleness crept in, but as a core part of ensuring your platform does as advertised.
    - inopinatus 6y ago
      > Should library code really be relying on a database Yes, many libraries need, or are enhanced by, persistence, whether that's a double-entry accounting module, a headless cms, or an e-commerce engine. > you really need to be testing against those Library authors need tools and guidance, not arbitrary constraints and high-watermark QA demands. Most of us have one or two DBs that we work with day-to-day but still want libraries we contribute to be broadly portable across any of the backends the framework they plug into supports. e.g. I can't maintain a test suite vs Oracle when I don't license it, but I still want to know that It'll Just Work if someone uses it in such combination, or that it's at least close enough they'll not have trouble making it work. More broadly I think the myth of testing every supported combination leads to a brittle mindset of saying "not supported, won't help", especially when systematised in a commercial environment, and to me it's the antithesis of good engineering.
      - dkersten 6y ago
        Well, if your library can use the "lowest denominator" of supported databases, then there's nothing wrong with doing that. I'd still argue that you don't really support something you haven't tested against, but I guess this is pretty off topic. My overall point was more that sometimes being locked into a single database is a conscious decision because you want to make use of everything a database has to offer. In the context of the article, for "Forcing Functions", I absolutely agree that switching databases is a useful way to find weakness in your solution.
- dkersten 6y ago
  > How often do people really migrate to a different database? Well, I have been part of a migration from MySQL to MariaDB, which was a lot more effort than one would expect given that they're meant to be more or less the same thing. It was a ton of effort and the abstracted ORM logic didn't actually help with this. So if it doesn't really help that much for a simple case like that, then it doesn't seem like there's much point in my opinion, as porting is going to be work either way (in a non-trivial application with non-trivial data access patterns, at least).
  - Kiro 6y ago
    I've done the same migration and I'm curious of what issues you had. For me, it was truly just plug'n'play with no additional work whatsoever.
    - dkersten 6y ago
      I don't remember, it was ~4 years ago and for a different company. I believe there were a small number of incompatibilities somewhere (composite indexes maybe? I remember doing stuff with them but don't remember why, could have just been for query optimisation) and a few areas where the performance was degraded (again, I think around indexes, but I could be mistaken). It wasn't a big deal over all, but it wasn't simple plug'n'play either and required some porting work.
- pydry 6y ago
  I've been considering moving away from azure postgres to azure SQL server coz azure postgres runs at a snails pace.
  - speedgoose 6y ago
    Azure is ridiculously slow unless you ask for decent IOs performance but then it's ridiculously expensive.
    - pydry 6y ago
      Is that true for azure sql server too? I suspected it was faster but I haven't checked.
- alzoid 6y ago
  I agree, I see those patterns in the wild a lot. You have a Microsoft shop using .NET and SQL Server yet the devs still abstract the data access layer. I think those patterns just became common 'just in case' they were needed. I find refactoring tools like "extract interface" take care of that so there is not need to write the abstraction until it's needed. On the other side, I worked on a web app that supported multiple db vendors, we did the classic DAO pattern which worked well. You still get to use custom SQL for each database if you need to. We tried an ORM at one point which worked out well. It was the same web app and we moved moved some DAO code to the Java Persistence API. We could then build the data access code and include it into our desktop (Mac Windows) and plug it in to a local DB (Derby). In that case, once JPA was working, the pluggable database was allowing us to save on development costs.
- Chris_Newton 6y ago
  Yeah I don’t get that obsession with “pluggable databases”. Abstractions have a real cost, and they typically complicate a lot: I would advice to just make sure you centralize the access to your database in a single module / class / whatever, but other than that, swapping out databases seems like a non-goal to me. I tend to agree, as long as the database you’re using is of the free and open source type so it’s a low-risk dependency. The “pluggability principle” holds more strongly for higher-risk dependencies, IMHO. For example, if you’re working in an online environment and you can’t readily integrate a new payment processor or messaging service or whatever, your existing dependency starts to look like a single point of failure that is not under your control. Some careful abstraction of the essential features and avoiding dependencies on the peculiarities of any specific platform can be a valuable safety blanket if your external dependency catches fire one day.
  - jacques_chester 6y ago
    It's an example of what economists call "The Hold-up Problem", as part of "Incomplete Contracts". Basically, it's impossible to write a contract to cover all situations, so eventually any two parties will need to renegotiate. If one has leverage over the other, they can do stuff like raise prices or simply refuse service. One factor affecting that leverage is "asset specificity": given an asset that's covered by the contract, is it reusable in different contexts or is it too closely adapted to one purpose? The literal textbook example is a steel foundry and a railroad. The steel company needs a spur line to its foundry to receive raw materials and send out finished steel. The railroad company can run a spur from its main line to the foundry. In this case the asset (the spur line) is extremely specific. Without the spur line, the foundry is worthless. Without the foundry, the spur line is worthless. The balance of leverage then comes down to relative costs and gains. The railroad will hold the upper hand, because closing a spur line isn't existential, but it is for the steel foundry. In the software context it shows up as a "contract" between your software and a database. You can use an ORM to try to insulate yourself, reducing asset specificity, but that decision has costs (eg, specific DB features you must forego). You can choose to increase asset specificity by adapting to only one DB and fully utilising its strengths, but you may face a future hold-up. Note from the train example that hold-ups needn't be "pay me more". They can also be "I'm walking away". So a hold-up from Oracle will be "pay me more", whereas a hold-up from an obscure database with one developer could be that they simply quit, die or otherwise become incapable of working.
- toong 6y ago
  If you are an enterprise software vendor (non-saas), orgs expect you use _their_ db. If they already have an Oracle license, or a MS-SQL DBA around, you either say goodbye to Postgres or give the contract to your competitor.
MaxBarraclough 6y ago
Agreed. It makes little sense to try to write your SQL queries to run fine on multiple different DBMSs. Different SQL DBMSs should be treated as different languages. Presumably the idea is based on an analogy to code portability: it can be good to ensure your C++ code compiles fine with multiple different compilers. Really though, it's more akin to writing code that compiles as both C# and Java; clearly madness.
- deleted 6y ago
  [deleted]
barrkel 6y ago
I don't significantly disagree with you, but let me take the devil's advocate approach. Adding support for a different database doesn't mean restricting yourself to the lowest common denominator. It means using different techniques, more appropriate for the different database, that may optimize other parts of your data access and modification path, while pessimizing stuff your current database does. More importantly, it means extracting hard database dependencies like raw SQL or custom ORM fiddling from your business logic and entities, and pushing them behind a module or service boundary. Raw leverage of the database, if it's dispersed throughout your application, will limit your ability to change your schema (e.g. denormalize an attribute, split or join tables, convert a parent-child relationship to embedded JSON or vice versa) and address performance problems as you scale up. It'll also stop you having a single point of data access where you can partition or duplicate your data into different stores with different capabilities more suited to their access and modification patterns. These kinds of things become really important when the database becomes a bottleneck in your system.
- dkersten 6y ago
  Just checking if I understood your point: for the purpose of Forcing Functions, running against a different database than the one designed for (and therefore where the tradeoffs may be different and things may run inefficiently) is still useful because it helps unearth design flaws, corner cases or brittleness? If so, then, sure , I agree with you. Not all reliance on a target database is actual features that don't have an easy or direct way to port.
  - barrkel 6y ago
    Almost. More that being forced to separate church and state, forced to keep business logic separate from database manipulation, and forced to go through an abstraction interface, then lets you leverage the interface later if and when the database becomes a bottleneck. It's a devil's advocate position. I actually think it's unrealistic and probably not worth it at the time. The forcing function is what's useful, not the database portability per se. And whether it's actually useful depends on being so successful that you have so much data that your regular RDBMS can't cope with some access or update patterns. That's a hefty bet early in anything, and probably doesn't make business sense, until it does, then you wish you did things differently. (Not coincidentally, it's a position we're in at my workplace; the database has hit its limits and we're needing to use hybrid approaches to hit latency NFRs.)
LandR 6y ago
I took this to mean be able to swap in and out your persistence layer. So my app can make calls which will persist data to a SQL database, or I can swap that layer out with another that persists to NoSql storage in the cloud. Possible because the persistence layer exposes an API or interface that is agnostic to the actual implementation of the layer ???

xupybd 6y ago

Who has time for this? I've always been on under resourced projects where every hour had to be signed off. Everything is a rush and quality is required but never budgeted for.

auggierose 6y ago
Time to change jobs then.
- swyx 6y ago
  HN resorts to the "just quit your job lol" far too easily. people are in all sorts of different situations. you don't know if they are struggling to find work and this is the one shot they have been given. you don't know if they are in extreme demand and get paid well to fix bad situations like this. you just don't know. and telling people to blithely change jobs when they can't or don't want to isn't helpful
  - mjburgess 6y ago
    Indeed, I encounter this anti-pragmatic personality a lot in tech people. It's an extreme intolerance to imperfect circumstances: a preference for nothing at all over compromise. The issue is that tech attracts the mathemetically-minded who reasons from universal principles. Rather than the empirically-minded who start with cases, and abduce to provisional principles from those. To a aximoatic mind: when a universal principle is violated, the situation is declared Bad. To the case-base mind: when a tolerable situation seems to violate a principle, declare the principle Inapplicable. Of course both types of thought are helpful in different contexts, I suspect 'the management of one's life, day to day' however, should be a matter of case-base reasoning to rough principle.
    - couchand 6y ago
      Okay, that's a reasonable universal principle, but in this case it's Inapplicable. To quote the OP: > Everything is a rush and quality is required but never budgeted for. It doesn't really sound like this is a situation where the principle is wrong, it sounds like this is proving the principle correct: you get what you pay for.

tasogare 6y ago

> Delete the project from your development machine, clone the source code and set it up from scratch. Very good advice. This was obviously never done in my org, leading to newcomers (me included) wasting weeks to get started on some projects. Once it was fixed, a newcomer can start working in an hour.

2rsf 6y ago
actually having a newcomer install everything and fix the documentation while doing it is even a better option, cloning has some of the dependencies but other might have been installed by the developer for example global packages for NPM or Python, setting of system wide environment variables etc.
- nullsense 6y ago
  This happened at my org... 8 separate times. It was always slightly wrong each time, even after it was fixed. I think it's better now, but I never considered doing it myself as an established team member. Though I once inherited a large enterprise code base that I had to study and build a dev environment for pretty much all by myself. I had maybe 2 calls with the original team and a couple of emails but was mostly by myself just figuring it out. It was a pretty incredible experience and taught me a tonne about how the system worked and was put together. This helped immensely when we spun up a team to work on it. So, I get where this article is coming from due to that experience, but didn't think to do this kind of stuff on purpose.
  - OJFord 6y ago
    Unless you're hiring like crazy, it's never going to be and stay perfect, but hopefully with fixing whatever issues each time you stay relatively on top of it, and it's never too much work.

vagrantJin 6y ago

At an agency, we used to run our web apps on some crappy 08 model laptops running on a gig of memory with outdated browsers. If the webapp ran there without major hitches, it was considerd good enough. It made everyone on the team think hard about optimizing even before a single line of code was written. It really did force excessive simplicity and not jumping on new libs/frameworks just because we can.

mikewarot 6y ago

Long ago, in the mists of time, (1987) I wrote a program that worked with hand held computers, barcode scanners, and was meant to be run by folks working in a power plant. I wrote the code, and got it working in about 2 months. It did everything in the spec, and the customer (Russ) loved what we had, but it turned out (of course) there were many things missed in the spec. A week or two later, we worked out a deal for all of their power plants, provided I would work with Russ to make sure it did everything they wanted... it would take about a year, in the end. Russ taught me everything I needed to know about users and how they think, in a direct and very effective manner. He was the assistant plant manager, so he would pull a random person into the office, and say to them "I know you're not a computer person, I want to you to do X,Y,Z... but don't worry... if anything goes wrong, it's not your fault... it's Mike's fault (Russ points at Me). He then tells me to just watch... It took exactly 1 minute to learn the first lesson... there should always be "Press F1 for Help" somewhere on the screen. It was a very instructive and productive year. I've carried those lessons from his forcing function with me for decades. I love telling the story, thanks for listening.

navaati 6y ago
Thanks for telling. It's heartwarming to hear a story of competency, problems solved, and efficiency for once.
tylerscott 6y ago
That was great story. Thanks for sharing!

glenjamin 6y ago

If a tree falls in a forest with no-one around, does it make a sound? While the list of techniques here look like excellent ways to uncover unknown unknowns, be sure that it's actually valuable for you to resolve these issues. In almost all cases the goal isn't to create perfect software, it's to create effective software - and sometimes the ROI on these unusual cases doesn't stack up (or doesn't stack up _yet_).

gridlockd 6y ago

I don't buy the "bugs caught early are cheaper to fix" paradigm. Most bugs are caught early without special precautions, but special precautions can be quite expensive. Most software doesn't have truly catastrophic failure cases. If there's a bug slipping through, someone will run into it, report it, and it'll get fixed. If nobody runs into it, the bug doesn't cost anything. This is the most economic way to go about it, which is the reason why pretty much all successful software is kind of buggy. We all like to complain about it, but then we don't want to wait an extra year for the next version either. At the other end of the spectrum, if you need really reliable software, the solution is not to eliminate all the bugs, that's impossible. Even with perfect software, hardware can fail, bits can flip the wrong way. The solution is to make sure that errors can't bring down the airplane.

cjfd 6y ago
I don't think so. When one is fixing bugs locally before the thing goes into production one can use each and every debugging strategy under the sun to find the bug. If it runs 10 times slower because of generating massive logs that is acceptable when debugging locally. In production one is constrained to debugging techniques that do not disturb production. This constraint quite often makes debugging times longer by factors of 10 if not 100 while at the same time a customer is getting somewhat impatient. Also, the bug that would be discoverable locally but not in production is a bit of a strange animal. It does happen for cases where the developer has a better idea than the user what should be happening so the user will not even notice the bug. That sounds more like features that have been specified in too much detail than true bugs, though. More commonly everything that can go wrong locally will go wrong in production sooner or later. And in production you will, on top of that, hit all of the bugs that occur once every month and that only occur if the order of inputs is a bit strange while the database is under some load. If you have not done all possible debug/test work locally the application will hit production when there still is a lot of debug work to do and there will probably be so many problems that they even start interacting with one another and produce some really 'interesting' failures.
dockd 6y ago
This is just pushing testing to the customer. Why would the customer report a bug instead of switching to a less buggy product? Why do you think the customers are going to report bugs? I can't say that I've ever attempted to report a bug on an iPhone app, Windows, etc. It's like yelling at the wind.

hyperpape 6y ago

The reference to bugs being more cheaper to fix the earlier they're found is one of those claims that gets passed around without much attempt to look at the supposed sources. I believe the most common citation trail bottoms out in a study that no one has access to anymore, and the whole thing seems dodgy. One link: https://www.techwell.com/techwell-insights/2013/10/what-does-it-really-cost-fix-software-defect https://www.techwell.com/techwell-insights/2013/10/what-does....

jasonpeacock 6y ago
Checkout the book "Accelerate", it covers a lot of operation excellence practices supported by data, including shortening the iteration cycle (e.g. discovering bugs early): https://smile.amazon.com/Accelerate-Software-Performing-Technology-Organizations-ebook/dp/B07B9F83WM https://smile.amazon.com/Accelerate-Software-Performing-Tech...
hyperpape 6y ago
Here is more from Laurent Bossavit, showing the history of shoddy citations surrounding the 100x claim: https://gist.github.com/Morendil/f9c2e9f3f450d3a76de8aeee7cf2bd00 https://gist.github.com/Morendil/f9c2e9f3f450d3a76de8aeee7cf....