5 ms·

It's cathartic to read other people who have to go through this. I'm fighting red tape for my team as we build out a dashboard. Outlook is packed with 1–2 hou

by anw 5y ago

It's cathartic to read other people who have to go through this. I'm fighting red tape for my team as we build out a dashboard. Outlook is packed with 1–2 hour meetings for the next 3 months where so far I'm: * being asked to load test our system to make sure it can handle the load (of 3 people?) * being asked to integrate with various analytics platforms so we can alert some poor schmuck at 3 AM in case the API goes down then (it's not a vital part of any platform) * told to have this run in k8s since everything runs in k8s * other pedantic tasks by Sys Ops who think everything is a nail and love to argue their points ad exhaustium (or worse, argue why their fav stack is the golden child) I understand the need for standards and making sure they're followed, but there really needs to be a human element of, "is this truly needed for what I'm trying to do?". So many engineering departments are all about automation, but don't truly think through how much automation is needed, rather than a 1 size fits all approach. I appreciate that this article comes to the conclusion that the more correct an answer will be, the more complicated it tends to be. I wish more people in decision making positions would understand this.

catlifeonmars 5y ago
The minor conclusion of this article was the more interesting (and perhaps more practical) of the two: Hide concessions to various leaders in the project roadmap. This isn’t just a “bureaucratic trick” as the OP suggested, it’s actually a way to convert unconditional advice into contingent advice, by encoding a priority.
- jerf 5y ago
  Yeah, that's not so much a "nifty bureaucracy hack" as a core skill to completing any project. It doesn't even have to be 20 unrelated people's feedback... it's my own priorities quite often that I mercilessly stuff on the backlog. YAGNI isn't just at the micro code level, it's a core project design skill. In fact I probably YAGNI my roadmap much harder than my code since I often have a good idea that I will in fact Need It at the microlevel after decades of experience and can save some time at that level, but at the project roadmap level anything you can trim is getting the product out generating value sooner. (Obviously one can go too far, blah blah blah. But just as with code, we have a much larger problem in practice grabbing too much from the project feature buffet than too little.)
  - AstralStorm 5y ago
    And then some to level priorities shift and you look at the backlog thinking "if only we have done x before that". Usually when your unfinished prototype ends up in production. That's the danger of reporting progress to people who think you can go to space on a paper glider. Probably half of more of start-ups end up failing like this as their quickly delivered prototype fails to capture the market due to not being actually better enough, or crumble under the initial success. Good for investors and managers who bail out early enough, very bad for users.
  - madeofpalk 5y ago
    > Yagni originally is an acronym that stands for "You Aren't Gonna Need It"
- daveevad 5y ago
  > Create an extended product roadmap and put those items at least a year off into the future “and as long as they don’t seem relevant, you can just keep pushing them into the future.” That actually seems to me like the root cause of all the calamity in the article, a culture of lying.
  - catlifeonmars 5y ago
    I don’t see it as lying in any meaningful way. Specifically in the article the problem was that there was technical feedback from many parties that have very little, if any stake in the matter. I’d be willing to bet that none of them even bothered to look at the product roadmap to check on the progress or status of their suggestions. Rather, the “cause of all the calamity” in the article seems to be the fact that the business has a culture of requiring feedback from random individuals who have very little stake in the project or product delivery.
    - daveevad 5y ago
      Upon further reflection, the root cause is poor communication, and the 'bureaucratic judo trick' is just a continuance, or perhaps even an escalation, of an organization's poor culture.
  - pjc50 5y ago
    This is absolutely not lying and I'm disappointed that anyone thinks it is. This isn't "not doing the thing and saying you did", it's just setting its delivery date into the future, an entirely routine operation for every software project that actually ships.
    - daveevad 5y ago
      From my perspective, the tactic misleads the stake holders about the real priorities. It's a deception and corrodes trust in the organization. The article even describes it as a 'bureaucratic judo trick.' It really seems to me as analogous to the micro-services guy or the architect guy insisting their way prevails.
- SamBam 5y ago
  > to convert unconditional advice into contingent advice, by encoding a priority This is one of the most important things I've learned as a developer, and one that I thought I invented myself, before I knew about agile, by keeping a whiteboard near my desk with yellow sticky notes ordered by property: "Yes, I get that it's a must-have feature, but where do you place it in relation to these other features?" The concept of prioritization of features, and of saying "if I stopped dead at some arbitrary point in this list, would you have been happy with your order?" seemed so eye-opening to people at the time.
  - anchpop 5y ago
    Sometimes, the features are really must-haves though. Let’s say it’s march 2020 and your boss wants you to design a mass-market covid vaccine. You have three requirements: it needs to be safe for human use, it needs to be effective at preventing covid, and it needs to be possible to manufacture. If any one of these is missing, your design is useless. I think a similar dynamic is visible in many software projects.
    - oroul 5y ago
      That's totally fine - but people need to also be aware that if something is really a must then they have to be willing to spend adequate time and resources on getting it done, instead of assuming whatever resources they have on hand will be sufficient. It's amazing the things that stop being a "must have" as soon as they have to spend more money.
      - hvidgaard 5y ago
        It's a never ending struggle to get people to create this ordered priority. I always tell my developers to say "if you do not give this an ordered priority, I will resolve items as I see fit. Should we need to stop for one reason or another, there is no guarantee of which have been resolved". Often times that is okay. I also tell them to always take the ones they're most uncertain about first. Better to front load hard problems and uncertainties.
    - 5y ago
- omoikane 5y ago
  This works especially well when you set and own those priorities, or if your management supports those priorities. Everyone who wants their feature will need to justify to you that their feature deserves a better placement on your roadmap. It does not work if you can not defend your priorities.
- wayoutthere 5y ago
  This is it. I do a lot of consulting work around this problem, and the roadmap is where the business and technology meet. It’s where you convert sprints into calendar boxes. It’s also the part most companies do poorly because nobody likes to spend money on good project/program managers (hint: hire product managers instead even though they’re ~25% more expensive because everything in 2021 is a product in some way). When you do it this way, you can decide well ahead of time if you need to bring in a contractor to build a must-have feature your team won’t have bandwidth for. It flips the narrative and puts the responsibility on the business side (which usually controls the budget anyway).
zdragnar 5y ago
Back when I was struggling to develop features in overengineered hell, I commented to my friends what a breath of fresh air updating a personal site with scp was. They all gave sighs and shudders of disgust, but then again, they had normal programming jobs, so I suppose it seemed quite backwards to them.
- hedora 5y ago
  Oh, but scp won’t update it atomically, so you should switch to a scheme that will. Then all you need to do is set cache policies correctly, coordinate with your CDN, and maybe do a staged rollout, just in case. /s Seriously though, rsync is your friend. :-)
  - zdragnar 5y ago
    You know, I think I might have switched to rsync at one point- I haven't had the site in a few years now, so my memory is a bit hazy. It was sufficiently small enough (no heavy media files) that I didn't mind if I left some unused files up there. Pretty much the only thing that I had to do was make a copy of the sqlite database each time just in case.
    - tonyedgecombe 5y ago
      --delete-after will delete files on the destination after everything else has synced so you can be sure you aren't linking to a non-existent asset.
  - zimpenfish 5y ago
    > Seriously though, rsync is your friend. :-) Even rsync might not be atomic enough for some situations[1] since it'll update files as it goes rather than in one huge transaction at the end. [1] I worked on the World Cup 2006 site for Yahoo! and we had this issue - solved with 'rsync --link-dest' and swapping symlinks.
    - fctorial 5y ago
runawaybottle 5y ago
I’ll add to the list: - Ceremonial unit tests for every little thing. The whole system is buggy as hell and we don’t have any confidence that the unit tests are truly covering critical parts of the app. But alas, test coverage, the god damn Pope that can never be bemoaned. - I’m not making this one up: A/B testing for an internal enterprise app.
- teddyh 5y ago
  Test coverage is almost the perfect illustration of Goodhart’s law. Good programming practices do result in high test coverage, coverage is very easy to measure, but very easy to fake with useless “tests”. So, when the coverage is measured, the coverage goes up, but stops being meaningful.
  - nickcox 5y ago
    High coverage isn't enough but, in my experience, it's a great place to start. I've written an depressingly high quantity of code in my career that blows up literally the first time it runs. I'd much rather that happen in a unit test than in production. Any test that exercises a given branch is better than nothing.
    - lmm 5y ago
      Coverage can tell you what you didn't test, but it can't tell you what you did test. > Any test that exercises a given branch is better than nothing. I disagree with this. If you have a test that doesn't actually test anything, you can't tell that you're not really testing that branch. No test is better than a bad test because it's easier to fix.
  - onorton 5y ago
    > but very easy to fake with useless “tests” While it doesn't alleviate the problems entirely, you can also run things like mutation tests that check that your unit tests actually test conditions, rather than just execute all the code.
- oxfordmale 5y ago
nxpnsv 5y ago
I feel your pain. Been there, done that, probably still have the t-shirt.
zimpenfish 5y ago
> * told to have this run in k8s since everything runs in k8s I've seen a production system handling one request (which takes a handful of ms) every 2 seconds (work hours only, mind) in k8s running 8 pods. It is quite breathtaking.
- devoutsalsa 5y ago
  How do they handle the load balancing with that much traffic?
onion2k 5y ago
being asked to load test our system to make sure it can handle the load (of 3 people?) The problematic load in a dashboard isn't users; it's querying the data sources to get up to date information. For example, if you're running a query to aggregate a bunch of things with lots of joins and that query takes 1.5s to run but your dashboard tries to run it every second so it can be 'real time' then you're in for a bad time even with just 1 user. You absolutely need to load test a dashboard application that's running against production data. being asked to integrate with various analytics platforms so we can alert some poor schmuck at 3 AM in case the API goes down then (it's not a vital part of any platform) It might not be vital right now, but if you make a dashboard for it then it'll quickly become vital. Putting metrics in front of people focuses them on those metrics...
- Sammi 5y ago
  I've downvoted your answer here as I think it is a ungenerous interpretation of the post you replied to. It's just as likely that OP did already know that what you are insisting on is not relevant to their use case. That might be why they stated it.
darkwater 5y ago
The other side of the coin you are not telling is: let's ship this small project to production without all those useless bells and whistles, and then fast-forward 12 months, suddenly everybody is using it and it starts failing spectacularly, and now all those teams that complained in the beginning have a fire to extinguish. I've been too many times on this other side of the coin.
GlennS 5y ago
Cathartic is certainly the word. The title in particular really hits the mark for me. There are a lot of people talking about computer programs, and telling us we should do things this way or that way. Even telling us that their way is certainly the best or only correct way. A great many of these people - perhaps the majority majority - are plain wrong. Some of them talk such nonsense that I suspect they don't have any actual ability to program at all! How can they be so sure of themselves?

by anw 5y ago

catlifeonmars 5y ago

The minor conclusion of this article was the more interesting (and perhaps more practical) of the two: Hide concessions to various leaders in the project roadmap. This isn’t just a “bureaucratic trick” as the OP suggested, it’s actually a way to convert unconditional advice into contingent advice, by encoding a priority.

jerf 5y ago
Yeah, that's not so much a "nifty bureaucracy hack" as a core skill to completing any project. It doesn't even have to be 20 unrelated people's feedback... it's my own priorities quite often that I mercilessly stuff on the backlog. YAGNI isn't just at the micro code level, it's a core project design skill. In fact I probably YAGNI my roadmap much harder than my code since I often have a good idea that I will in fact Need It at the microlevel after decades of experience and can save some time at that level, but at the project roadmap level anything you can trim is getting the product out generating value sooner. (Obviously one can go too far, blah blah blah. But just as with code, we have a much larger problem in practice grabbing too much from the project feature buffet than too little.)
- AstralStorm 5y ago
  And then some to level priorities shift and you look at the backlog thinking "if only we have done x before that". Usually when your unfinished prototype ends up in production. That's the danger of reporting progress to people who think you can go to space on a paper glider. Probably half of more of start-ups end up failing like this as their quickly delivered prototype fails to capture the market due to not being actually better enough, or crumble under the initial success. Good for investors and managers who bail out early enough, very bad for users.
- madeofpalk 5y ago
  > Yagni originally is an acronym that stands for "You Aren't Gonna Need It"
daveevad 5y ago
> Create an extended product roadmap and put those items at least a year off into the future “and as long as they don’t seem relevant, you can just keep pushing them into the future.” That actually seems to me like the root cause of all the calamity in the article, a culture of lying.
- catlifeonmars 5y ago
  I don’t see it as lying in any meaningful way. Specifically in the article the problem was that there was technical feedback from many parties that have very little, if any stake in the matter. I’d be willing to bet that none of them even bothered to look at the product roadmap to check on the progress or status of their suggestions. Rather, the “cause of all the calamity” in the article seems to be the fact that the business has a culture of requiring feedback from random individuals who have very little stake in the project or product delivery.
  - daveevad 5y ago
    Upon further reflection, the root cause is poor communication, and the 'bureaucratic judo trick' is just a continuance, or perhaps even an escalation, of an organization's poor culture.
- pjc50 5y ago
  This is absolutely not lying and I'm disappointed that anyone thinks it is. This isn't "not doing the thing and saying you did", it's just setting its delivery date into the future, an entirely routine operation for every software project that actually ships.
  - daveevad 5y ago
    From my perspective, the tactic misleads the stake holders about the real priorities. It's a deception and corrodes trust in the organization. The article even describes it as a 'bureaucratic judo trick.' It really seems to me as analogous to the micro-services guy or the architect guy insisting their way prevails.
SamBam 5y ago
> to convert unconditional advice into contingent advice, by encoding a priority This is one of the most important things I've learned as a developer, and one that I thought I invented myself, before I knew about agile, by keeping a whiteboard near my desk with yellow sticky notes ordered by property: "Yes, I get that it's a must-have feature, but where do you place it in relation to these other features?" The concept of prioritization of features, and of saying "if I stopped dead at some arbitrary point in this list, would you have been happy with your order?" seemed so eye-opening to people at the time.
- anchpop 5y ago
  Sometimes, the features are really must-haves though. Let’s say it’s march 2020 and your boss wants you to design a mass-market covid vaccine. You have three requirements: it needs to be safe for human use, it needs to be effective at preventing covid, and it needs to be possible to manufacture. If any one of these is missing, your design is useless. I think a similar dynamic is visible in many software projects.
  - oroul 5y ago
    That's totally fine - but people need to also be aware that if something is really a must then they have to be willing to spend adequate time and resources on getting it done, instead of assuming whatever resources they have on hand will be sufficient. It's amazing the things that stop being a "must have" as soon as they have to spend more money.
    - hvidgaard 5y ago
      It's a never ending struggle to get people to create this ordered priority. I always tell my developers to say "if you do not give this an ordered priority, I will resolve items as I see fit. Should we need to stop for one reason or another, there is no guarantee of which have been resolved". Often times that is okay. I also tell them to always take the ones they're most uncertain about first. Better to front load hard problems and uncertainties.
  - 5y ago
omoikane 5y ago
This works especially well when you set and own those priorities, or if your management supports those priorities. Everyone who wants their feature will need to justify to you that their feature deserves a better placement on your roadmap. It does not work if you can not defend your priorities.
wayoutthere 5y ago
This is it. I do a lot of consulting work around this problem, and the roadmap is where the business and technology meet. It’s where you convert sprints into calendar boxes. It’s also the part most companies do poorly because nobody likes to spend money on good project/program managers (hint: hire product managers instead even though they’re ~25% more expensive because everything in 2021 is a product in some way). When you do it this way, you can decide well ahead of time if you need to bring in a contractor to build a must-have feature your team won’t have bandwidth for. It flips the narrative and puts the responsibility on the business side (which usually controls the budget anyway).

zdragnar 5y ago

Back when I was struggling to develop features in overengineered hell, I commented to my friends what a breath of fresh air updating a personal site with scp was. They all gave sighs and shudders of disgust, but then again, they had normal programming jobs, so I suppose it seemed quite backwards to them.

hedora 5y ago
Oh, but scp won’t update it atomically, so you should switch to a scheme that will. Then all you need to do is set cache policies correctly, coordinate with your CDN, and maybe do a staged rollout, just in case. /s Seriously though, rsync is your friend. :-)
- zdragnar 5y ago
  You know, I think I might have switched to rsync at one point- I haven't had the site in a few years now, so my memory is a bit hazy. It was sufficiently small enough (no heavy media files) that I didn't mind if I left some unused files up there. Pretty much the only thing that I had to do was make a copy of the sqlite database each time just in case.
  - tonyedgecombe 5y ago
    --delete-after will delete files on the destination after everything else has synced so you can be sure you aren't linking to a non-existent asset.
- zimpenfish 5y ago
  > Seriously though, rsync is your friend. :-) Even rsync might not be atomic enough for some situations[1] since it'll update files as it goes rather than in one huge transaction at the end. [1] I worked on the World Cup 2006 site for Yahoo! and we had this issue - solved with 'rsync --link-dest' and swapping symlinks.
  - fctorial 5y ago

runawaybottle 5y ago

I’ll add to the list: - Ceremonial unit tests for every little thing. The whole system is buggy as hell and we don’t have any confidence that the unit tests are truly covering critical parts of the app. But alas, test coverage, the god damn Pope that can never be bemoaned. - I’m not making this one up: A/B testing for an internal enterprise app.

teddyh 5y ago
Test coverage is almost the perfect illustration of Goodhart’s law. Good programming practices do result in high test coverage, coverage is very easy to measure, but very easy to fake with useless “tests”. So, when the coverage is measured, the coverage goes up, but stops being meaningful.
- nickcox 5y ago
  High coverage isn't enough but, in my experience, it's a great place to start. I've written an depressingly high quantity of code in my career that blows up literally the first time it runs. I'd much rather that happen in a unit test than in production. Any test that exercises a given branch is better than nothing.
  - lmm 5y ago
    Coverage can tell you what you didn't test, but it can't tell you what you did test. > Any test that exercises a given branch is better than nothing. I disagree with this. If you have a test that doesn't actually test anything, you can't tell that you're not really testing that branch. No test is better than a bad test because it's easier to fix.
- onorton 5y ago
  > but very easy to fake with useless “tests” While it doesn't alleviate the problems entirely, you can also run things like mutation tests that check that your unit tests actually test conditions, rather than just execute all the code.
oxfordmale 5y ago

nxpnsv 5y ago

I feel your pain. Been there, done that, probably still have the t-shirt.

zimpenfish 5y ago

> * told to have this run in k8s since everything runs in k8s I've seen a production system handling one request (which takes a handful of ms) every 2 seconds (work hours only, mind) in k8s running 8 pods. It is quite breathtaking.

devoutsalsa 5y ago
How do they handle the load balancing with that much traffic?

onion2k 5y ago

being asked to load test our system to make sure it can handle the load (of 3 people?) The problematic load in a dashboard isn't users; it's querying the data sources to get up to date information. For example, if you're running a query to aggregate a bunch of things with lots of joins and that query takes 1.5s to run but your dashboard tries to run it every second so it can be 'real time' then you're in for a bad time even with just 1 user. You absolutely need to load test a dashboard application that's running against production data. being asked to integrate with various analytics platforms so we can alert some poor schmuck at 3 AM in case the API goes down then (it's not a vital part of any platform) It might not be vital right now, but if you make a dashboard for it then it'll quickly become vital. Putting metrics in front of people focuses them on those metrics...

Sammi 5y ago
I've downvoted your answer here as I think it is a ungenerous interpretation of the post you replied to. It's just as likely that OP did already know that what you are insisting on is not relevant to their use case. That might be why they stated it.

darkwater 5y ago

The other side of the coin you are not telling is: let's ship this small project to production without all those useless bells and whistles, and then fast-forward 12 months, suddenly everybody is using it and it starts failing spectacularly, and now all those teams that complained in the beginning have a fire to extinguish. I've been too many times on this other side of the coin.

GlennS 5y ago

Cathartic is certainly the word. The title in particular really hits the mark for me. There are a lot of people talking about computer programs, and telling us we should do things this way or that way. Even telling us that their way is certainly the best or only correct way. A great many of these people - perhaps the majority majority - are plain wrong. Some of them talk such nonsense that I suspect they don't have any actual ability to program at all! How can they be so sure of themselves?