10 ms·

There's No Such Thing as Software Productivity (2012)

130 points by pbrowne011 2y ago

akst 2y ago
In an economic sense there is absolutely software productivity, when it results in more outputs from the same economic inputs.
robwwilliams 2y ago
Ok, but ironically the article does explain by a comparison of two alternatives just how to measure productivity—-solving problems most efficiently (least amount of maintainable code).
rootedbox 2y ago
Sure there is. It's a ratio of inputs to outputs.. even in the example the inputs and outputs are measurable. The only thing this article gets at is that engineers may not know how to calculate their own productivity; but it doesn't means it's not calculable.
- halfcat 2y ago
  > It's a ratio of inputs to outputs.. even in the example the inputs and outputs are measurable. But reality is never that clear cut. How’s the ratio look when: - Peter goes to the park and the breakthrough doesn’t come? - Or it comes 3 weeks later? - Or he deletes 100 lines of code and introduces a new bug?
- pixl97 2y ago
  >It's a ratio of inputs to outputs.. even in the example the inputs and outputs are measurable. So more lines of code is better! Um, we know this doesn't work that way as a good measure. This is like comparing algorithms that do the same thing to algorithms that do different things. You're not going to get good valid comparisons. Metrics for one thing may not work at all for another.
  - cjensen 2y ago
    The output is measured with dollars, not lines of code. So are the inputs. It's a perfectly cromulent measure so long as we understand the limitations of the measure. For example, trying to measure the productivity of a day or a sprint? That's silly. Measure the output of a team which does not produce an entire product? Won't work because you'd have to figure out how to apportion the productivity.
    - fulafel 2y ago
      That's not the definition. Eg https://www.britannica.com/money/productivity https://www.britannica.com/money/productivity says: "productivity, in economics, the ratio of what is produced to what is required to produce it. Usually this ratio is in the form of an average, expressing the total output of some category of goods divided by the total input of, say, labour or raw materials."
      - cjensen 2y ago
        It's implied in the definition. Consider the units: a ratio should not have units. Lines of code per programmer per day would have weird units, for example, and could not be compared against number of windows installed per day for per car window installer. The only way for productivity to be useful is to normalize the inputs and outputs into money.
        deleted 2y ago
        [deleted]
    - lucianbr 2y ago
      It's a perfectly good measure except it does not help us at all. The whole reason for this discussion is situations like Microsoft having 200k employees and making $240B in a year. Which employees, teams, or even departments are more productive? They want to know. And even if it did not matter, likely the expense of this year influences the income over multiple future years, so you compare the dollars in / dollars out for which periods?
jes5199 2y ago
the problem is that software does not directly produce value, it is a way to place a bet on what will be valuable
- antupis 2y ago
  Pretty spot on and that is why devops stuff and automatic tests are so important. Those just let your bet faster.
voidhorse 2y ago
You can say this about any discipline. The root of the issue is that productivity for productivity's sake is meaningless, and it makes no sense to measure productivity as a general property when outputs vary. A tire factory has a distinct, singular goal: produce tires. It does this continually. Productivity is meaningful, but only in relation to a target that is typically specified by externalities (e.g. amount of demand) A software company is usually not in the business of producing consumable commodities so this kind of measurement does not make sense. It can make sense to measure productivity during a period for delivering a particular piece of software within a given time bound, but once it's delivered, productivity becomes meaningless. You always need to understand productivity in relation to some purpose and I don't know how these knuckleheads who think this abstract idea is basically like a concrete measurable essence, like mass, or liquid, got leadership positions.
- thuanao 2y ago
  Seems to me you’re talking about measuring software. Once we pick a measure though, we can calculate output/input (productivity). Imagine two people tasked with producing the same software, or software satisfying the same requirements or test suite. What would you call the person who produces it faster? More productive? Measuring software in financial terms or lines of code might not be the right measurement of software in all situations, but surely we can measure time and cost to produce software or software that satisfies equivalent requirements.
  - taberiand 2y ago
    Depends in the coming months and years which was more stable, maintainable and extensible.
  - hammock 2y ago
    You’re responding to the headline, not the article. The article has a plainly stated thesis: “Even if it could be measured, productivity in software does not approximate business value in any meaningful way”
    - 2y ago
csb6 2y ago
> I would argue that what good software developers do is remove problems. The opposite, in fact, of production But something is being produced - it is version 2.0 of the software. This is an artifact that is then shipped to users or deployed to a server. Peter’s solution fixed the issue and did not (seemingly) create further maintenance burden, which would have taken attention away from other tasks, i.e. reduced future productivity. I agree that metrics for programmer productivity are often useless (e.g. using lines of code is a bad idea for obvious reasons), but it seems silly to claim that the entire concept of productivity does not apply to the production of software.
- foobiekr 2y ago
  Silly and convenient. No one takes the claim that you can’t measure software productivity seriously and everyone simultaneously agrees that simple scalar metrics often fail to show the big picture in any and all disciplines. The rest is just usual software guy hubris and lack of awareness of the discipline.
  - musicale 2y ago
    > No one takes the claim that you can’t measure software productivity seriously At least three people do, and they are mentioned in the article: the author, his colleague, and Martin Fowler. I suspect that software productivity (like the productivity of scientists, artists, composers, etc.) on any non-trivial project may be measurable on a scale of years, but not necessarily months or calendar quarters. A larger issue is that productivity is often due to external factors as much as it is to individual effort, and overnight success is often the result of extended periods of limited progress or even repeated failure.
mprast 2y ago
I do appreciate the little coda at the end - nice that the author was self-aware enough to realize where the cruft was and cut it (and courteous to the reader!)
johnfn 2y ago
I'm not sure I follow. So you failed to measure software productivity in lines of code, therefore it follows that "There's No Such Thing as Software Productivity"? Don't you think that giving up after n=1 attempts at measuring software productivity might be a tad too fast to draw a generalized claim of impossibility? I might argue the real lesson learned is "Lines of Code are Not a Measure of Productivity in an Isolated, Toy Example". I suspect this sort of thing gets promulgated because it kind of massages our ego, like yes, they can measure other sorts of productivity, but not ours, oh no, we're too complex and intelligent, there's no way to measure the deep sorts of work that we do! Which, yes, OK, we're not exactly bricklayers, but surely, if you had to, you could do better.
- strulovich 2y ago
  Ok, but then what is a way to do it? The text gives an example to the core problem, and to argue differently requires thinking around it. In practice. I’ve seen many attempts at measuring productivity, but once you dig into them, you see they are just abstraction mechanisms above something that is similar to lines of code. I have yet to see an idea that sidesteps the core issue described in this post. Also, it applies to many types of work, and software is not unique in any way.
  - nemomarx 2y ago
    profit generated I think is the high level one, and then you want to dig from there into how much the software development contributed to this.
    - nomel 2y ago
      That's very tricky thing to quantify, especially with "unsung heroes". If my work is in preventing problems, the guy that fixes problems will be seen as the one that contributes more to profit, since impact is directly observed/measured. This is something that one of the orgs I worked for eventually realized. The people f'ing up, and then fixing their mistakes, were the ones getting promotions/bonuses/raises, because they were the ones interacting with all the execs.
jascha_eng 2y ago
You can very well at least count the problems that were solved (or deleted) you can also probably measure the value those solutions have in revenue or another metric. It's still true that measuring lines of code, time spent coding, commits or anything else is at best a proxy of productivity. It's also true that without any code changes problems more often than not don't get solved or we at least can't call the activity software development.
whoisthemachine 2y ago
Asking a software engineer to be more productive is akin to asking a mechanical engineer to be more productive. What does that end up looking like? More useless blueprints? It turns out when you ask software engineers to just crank out code, you just get lots of code.
fuzzfactor 2y ago
You could always look at things in a more abstract way ;) From one point of view, the users of the software are supposed to enjoy so much of a productivity increase that it's not supposed to matter if the coders are as productive as they could be or not. Give or take a few hundred percent at least. I realize that most people who've been to business school still aren't going to develop the needed acumen to handle a situation like this. Too many times the only training retained is a knee-jerk over-reaction to a fraction of a percent :\ Ever see one of these "leaders" have a cow and it was as stupid as it could have possibly been? I'm confident there are still some natural leaders that can thrive without worrying about every ounce of nose to the grindstone for their staff. Some things you just can't fix.
tbrownaw 2y ago
And yet somehow, software developers are doing something that's worth paying them a salary. And it's not always impossible to tell if someone's being held back at a lower salary band than they should be, or was promoted into a salary band that's higher than they're able to keep up with.
- liontwist 2y ago
  Yep it’s just nonsense. If it’s hard to define or measure I guess it doesn’t exist?
- Analemma_ 2y ago
  Yeah, these complaints are always so duplicitous. Somehow, it’s impossible to measure programmer productivity when it’s time for accountability or professional standards, but not when it’s time to assign annual bonuses.
timeforcomputer 2y ago
Yay Ben Rady. He has a great podcast with Matt Godbolt (who made Compiler Explorer) called Two's Complement.
irjustin 2y ago
I realized why this post rubs me the wrong way. It complains, but doesn't offer a solution. It simply criticizes and says "all engineers cannot and thusly should not be measured". The ironic thing is, the blog post is implicitly measuring by not explicitly measuring. The measurement is the bug ticket itself and whatever value attached to it. But to this end, I generally agree. There are qualitative and quantitative measurements. Quantitative is the value of the ticket commonly ascribed by the team (scrum? agile? whatever). Qualitative should come up in review. Qualitative is SO HARD. Top down? Team 360? Mixed? But it must be undertaken and refined by the team at each level of the org. Otherwise you will run into the exact situation described by the blog post and you won't know how to judge left from right, good from bad. Maybe the blog post's example isn't that great, too much information is missing to make a solid judgement, but you need to decide who to reward via promotion, annual raises and who to reprimand and who to not change. But still, all systems are terrible, but you must pick one less it be picked for you.
- fcantournet 2y ago
  Why do you need to evaluate people constantly and pit them against each other ? Why not give a raise to everyone and see what happens ? It's how we sent rockets to the moon, it seemed to work ok.
  - ebiester 2y ago
    Because the alternative is based on your manager's impressions. You get fired because "your manager doesn't like you." You get a promotion because "your manager likes you." Your coworker sits around doing very little and nobody notices for a few months. Your company and boss is sued because someone says their firing was based on discrimination because they can't prove it was for performance. The truth is that managers can suss out 90% of the problems without a number. However, we are asked to document the hell out of it if we want to do something about it. And twice in my career I've been wrong: I mistook someone who was quiet for not doing much until I dove into the work. I trust my gut, but I confirm with numbers.
  - irjustin 2y ago
    > It's how we sent rockets to the moon, it seemed to work ok. There are so many problems rooted in this statement. Government program, presidential mandate (i.e. unlimited budget), no competition. SpaceX is clearly better than NASA except maybe they don't push hard enough nor evaluate their engineers so you have a nice stable job if you do nothing.
revskill 2y ago
Productivity to me is about choosing the best tradeoff while minimizing the costs of going ahead.
mattmcknight 2y ago
So if you have a developer who writes no code and closes no tickets, who essentially does nothing except come to meetings, they are just as productive as a developer who writes some code and closes some tickets? Obviously there is a difference there. Negative code is still doing something as well. Back in the 90s we used Personal Software Process from the SEI, and we measured lines added, deleted, and changed. (as well as defects removed and added) It becomes clear that simple quantity of code and tickets is not enough- but it's also not nothing. Part of what is missing is quality and assessments of task complexity. Part of what is missing is the other parts of the jobs, like design and code reviews. I don't think it's hopeless, and can at least be used to look into why some people don't seem to make much at all.
danielmarkbruce 2y ago
And yet, Jeff Dean is significantly more productive than I am.
manmal 2y ago
Solving problems is productivity. That also means, solving them in a way that the solution doesn’t spawn new problems down the road, which is why we should follow the best practices. I can’t say whether deleting 100 lines is better than adding 1K lines, because before passing that judgement I would need to see what exactly those 100 and 1K lines are. OP is arguing too black and white IMO, and draws the conclusion before I‘m sold on the premise.
wpwpwpw 2y ago
Good productivity is about being able to do all tasks as planned. That includes the management, as its task is to attribute tasks. And whoever creates tasks. And whoever defines the objectives those tasks should answer to.
- musicale 2y ago
  Optimizing task completion is different from optimizing task benefit.
pjs_ 2y ago
In the scenario described in the article, Peter and Paul both achieve the same outcome in the same wall clock time. Obviously they are equivalently productive despite different working styles, by construction. But this doesn't account for the more realistic examples of Prakash, who completely fails to deliver a working solution, or delivers half a solution, and Percy, who gets it done two weeks late. I'm pretty sure you can define a shit_done/time_elapsed productivity metric for those two guys that is worse than that of Peter and Paul. Maybe I am a cynic but I suspect that some people are upvoting this because the framing makes them feel OK about getting paid six figures to work three hours a day...
dools 2y ago
I think this is just a semantic quibble based on a narrow interpretation of the word "produce". Because it also means: "cause (a particular result or situation) to happen or exist."
- default-kramer 2y ago
  Yeah, the comments on the post already said it all. First Isaac says > You seem to be playing a definition word game. [...] Obviously productivity does approximate business value in a very meaningful way when it is defined in terms of delivered business value. Then the author responds > Martin asserts "any true measure of software development productivity must be based on delivered business value". I agree, and I propose there is no such thing. We're better off dropping metaphors altogether and just talking about what programmers do: Solve problems. So I guess the author wants to replace the term "software productivity" with something like "problem-solving productivity, measured in terms of business value"...? Kind of silly IMO, but to each his own.
eacapeisfutuile 2y ago
This is fictious btw, or logic as follows is delete all code, deploy nothing, all bugs fixed, and achieve infinite productivity.
wcfrobert 2y ago
Wholeheartedly agree. SWEs are not factory workers. They don't punch in at 6am, put on their uniforms, make sure they meet their daily "lines of code" quota before clocking out at 10pm. We should not measure software productivity the same way we measure ball bearings production. Using lines of code to track productivity is absurd (do people really believe it or is it just a strawman at this point?). I'm reminded of that midwit meme where a junior has very few lines of code written because they don't know the code base well enough, the midwit writes up an whole framework, and the senior engineer has a net negative lines of code contribution. Theres also a fundamental difference between creating and maintaining code. Something like 10 guys wrote visicalc. Does that mean they were contributing millions of dollars in profit per hours? What about the maintainance to keep it going? Bug fixes? Patches? On call infra guys? What about opportunity cost of putting engineers on deadend projects? My point is tracking productivity in software dev - maybe all knowledge work for that matter - is complicated. Maybe that's why there's so much "busywork" (emails, slack, tickets, meetings, etc). Everyone wants to look productive but no one knows what it means
impure 2y ago
Let's say that you have two runners running the same marathon. The first one, Frank, sprints at full speed and eventually tires out and slows down. The second runner, Peter, takes a nap first and then finishes the marathon at the exact same time. Which of these two runners was faster in the race? The answer is: It doesn't matter. And therefore there is no such thing as running speed. There is such a thing as productivity in programming. If you could measure it it would likely be some combination of peer review and an analysis of the impact implemented features and fixes had. Some companies actually have programmers rate each other. I don't know how well it works and I think it can lead to perverse scenarios. But you can come up with metrics that are positively correlated with productivity.
- robertlagrant 2y ago
  The answer is neither is faster, because they finished at the same time.
0xbadcafebee 2y ago
"Put another way, productivity has no applicability as a metric in software. "How much did we create today?" is not a relevant question to ask. Even if it could be measured, productivity in software does not approximate business value in any meaningful way. This is because software development is not an activity that necessarily produces anything. This is ridiculous, irrelevant, and wrong. Of course software development produces things. It produces software. Which of these two developers was more "productive" today? The answer is: It doesn't matter. What matters the that Peter solved the problem, while simultaneously reducing long term maintenance costs for the team. Frank also solved the problem, but he increased maintenance costs by producing code, and so (all other things being equal) his solution is inferior. To call Peter more "productive" is to torture the metaphor beyond any possible point of utility. Ohhhhhhhhh. I get it. The author doesn't know what the word productivity means. Productivity does not mean "increases business value while decreasing maintenance costs [and having no net negative impact in any way]". It doesn't even mean "solving a problem". Productivity just means "to make something", or more specifically the rate at which something is made. That's all. You can make 10x more of something, and it can be garbage quality, but you did make it, and you did make more of it, so your productivity increased. If you produce 10x more grain than you did yesterday, you are more productive. The grain might now be full of heavy metals, pesticides and toxins. But you did in fact produce more grain. If you were trying to measure productivity of usable, healthy, high-quality grain, that is a different measurement than just productivity of grain. You may assume everybody knows what you mean when you say "productive", but you'd be wrong.
simpaticoder 2y ago
The difficulty of measuring productivity is particularly felt by senior developers. They save time and effort in ways that are non-obvious, which might be measured by dependencies that were not added, design patterns that were rejected, or processes that they pushed back on. Just like with living things, unchecked growth is unhealthy for an organism but the actions required are difficult to measure. One could start an attempt with a counter-factual narrative, but this does not map cleanly to KPIs.
jschrf 2y ago
Time spent on "incidental complexity" (new features, key fixes, performance) versus time spent on "accidental complexity" (anything and everything else). Easy metric to understand, easy metric to teach, just remember that it applies to teams and not individuals. See: Out of the Tar Pit.
deleted 2y ago
[deleted]
fulafel 2y ago
Productivity in economics refers to how many units of output you can generate with a given amount of input, it doesn't take into account quality, usefulness, etc. Complaining "lines-of-code is not a sensical productivity measure" is too kind to the concept of productivity. Using productivity as a metric leads to the same nonsequitur stuff in many many fields. It works somewhat for industrial output when you're working with commodities. Or it can work in some more fields as an ancillary measure if you pair it with some other quality, customer satisfaction, outcomes etc measure. But usually you don't want to maximize work while getting good outcomes, you just want the good outcomes.
vonnik 2y ago
And yet … there are some engineers who manage to solve hard problems to get something new working and debug it, however long it takes, and many others who don’t and/or can’t.
fermigier 2y ago
Strawman argument. It would have been (semi) interesting if the title was "Software productivity can't be measured by lines of code" (but already very eloquently stated in https://folklore.org/Negative_2000_Lines_Of_Code.html https://folklore.org/Negative_2000_Lines_Of_Code.html ).
lordnacho 2y ago
Isn't this whole debate a repeat of the old talking point about what is value? Labour theory of value, that kind of thing? This is simply never going to end if carried on along the lines of this article, or along the lines of most of the comments here. There's no way to objectively and reasonably put a value on something. All we have is a theory of subjective value, which does a bit of handwaving about utility and works out some ways where we can come to a price, regardless of the fact that the values in peoples' heads are subjective. Thus the distinction between knowledge work and "tangible work" like bricklaying is actually a moot point. Yes, you can measure "productivity" of bricklaying in metres per day, but ultimately you care about value, not amounts of wall. The arguments about one guy definitely being more productive than another are similar. One person values speed, another values maintenance costs downstream. It is subjective what ought to be more important.
sanitycheck 2y ago
Also, there's no such thing as sculpture productivity because all sculptors are doing is removing material.
lincpa 2y ago
[dead]
jFriedensreich 2y ago
Thats why bugs and chores have 0 points per default in pivotal tracker. Thats also why there was a push to force us say "as a user i..." at the beginning of story descriptions to make sure this is something creating user value. With these guards in place i don't follow the argument that productivity is not measurable, if a team builds features that solve an expressed user need and lets even say go through a final user acceptance check, this is a productive process and it is also very measurable!
focusbox 2y ago
[dead]
andrewqa 2y ago
[dead]
cushychicken 2y ago
Impact is the only useful metric. It is staggeringly hard to measure. Output is a weak proxy for impact. But it’s the one that makes intuitive sense to people. Doesn’t make it right or useful. I’m sure you all can envision a parable about your subfield of expertise that showcases how a seemingly light touch has a huge positive impact.
- virgilp 2y ago
  > It is staggeringly hard to measure. It's straight up impossible. Best we can do is observe and attempt to measure proxies. Take hypothetical situation: VPs A & B debate a business decision. Let's say A wins the argument and their solution leads to a revenue increase of $10M, and let's say we can confidently state that this is the outcome driven primarily by them winning the argument. Is the impact a net growth in business of $10M? In a sense, yes; but in some other sense, perhaps if they went with B's solution, the revenue growth would have been $15M? There's probably no way to know for sure unless you try both approaches, which is often impossible...
  - cushychicken 2y ago
    My theory is that impact is heavily context dependent. If you can solve a problem quickly, at the right time, with buyin from your org, that’s positive impact. That general case differs wildly when you descend to the particulars.
popcorncowboy 2y ago
Clicked for the bait, stayed for the background tile.
toss1 2y ago
YUP My first rule of software programming is: Work Hard to Avoid Writing Code — Code is habitat for bugs; more code equals more bugs, and interactions between separate parts of code can harbor even more "interesting" varieties. — Code takes time to run. Any NoOp is faster than even your best hand-tuned assembly module. Obviously, this is not absolute in any sense — it breaks down as soon as you need the system to actually do something, at which point you must write some code. But it should be the minimal amount to get the job done, and nothing more. Obligatory car analogy: While doing some amateur sportscar racing, a coach asked me "What are the things you do that slow the car down?". I thought for a moment and started saying "when I start into a corner, if I do a bit too much...", he interrupted saying "NO, no, what are the BIG things you do that slow the car down?". "Oh, like braking and turning and lifting off the throttle?". "Right. So what that means is that you should always avoid doing those things. Obviously, you will certainly have to so some of them as soon as you approach the end of the front straight after the start, but make sure you understand your car, the track, and your skills to the point where you do only the absolute minimum." Both the software and sportscar versions are deceptively simple — they take a LOT more thinking than it seems at first glance. And that thinking is totally worth it.
deleted 2y ago
[deleted]
lazyant 2y ago
"There is no such thing as software productivity." != "There is no such thing as a simple, objective software productivity metric".
daniel1492 2y ago
[dead]

130 points by pbrowne011 2y ago

akst 2y ago

In an economic sense there is absolutely software productivity, when it results in more outputs from the same economic inputs.

robwwilliams 2y ago

Ok, but ironically the article does explain by a comparison of two alternatives just how to measure productivity—-solving problems most efficiently (least amount of maintainable code).

rootedbox 2y ago

Sure there is. It's a ratio of inputs to outputs.. even in the example the inputs and outputs are measurable. The only thing this article gets at is that engineers may not know how to calculate their own productivity; but it doesn't means it's not calculable.

halfcat 2y ago
> It's a ratio of inputs to outputs.. even in the example the inputs and outputs are measurable. But reality is never that clear cut. How’s the ratio look when: - Peter goes to the park and the breakthrough doesn’t come? - Or it comes 3 weeks later? - Or he deletes 100 lines of code and introduces a new bug?
pixl97 2y ago
>It's a ratio of inputs to outputs.. even in the example the inputs and outputs are measurable. So more lines of code is better! Um, we know this doesn't work that way as a good measure. This is like comparing algorithms that do the same thing to algorithms that do different things. You're not going to get good valid comparisons. Metrics for one thing may not work at all for another.
- cjensen 2y ago
  The output is measured with dollars, not lines of code. So are the inputs. It's a perfectly cromulent measure so long as we understand the limitations of the measure. For example, trying to measure the productivity of a day or a sprint? That's silly. Measure the output of a team which does not produce an entire product? Won't work because you'd have to figure out how to apportion the productivity.
  - fulafel 2y ago
    That's not the definition. Eg https://www.britannica.com/money/productivity https://www.britannica.com/money/productivity says: "productivity, in economics, the ratio of what is produced to what is required to produce it. Usually this ratio is in the form of an average, expressing the total output of some category of goods divided by the total input of, say, labour or raw materials."
    - cjensen 2y ago
      It's implied in the definition. Consider the units: a ratio should not have units. Lines of code per programmer per day would have weird units, for example, and could not be compared against number of windows installed per day for per car window installer. The only way for productivity to be useful is to normalize the inputs and outputs into money.
      - deleted 2y ago
        [deleted]
  - lucianbr 2y ago
    It's a perfectly good measure except it does not help us at all. The whole reason for this discussion is situations like Microsoft having 200k employees and making $240B in a year. Which employees, teams, or even departments are more productive? They want to know. And even if it did not matter, likely the expense of this year influences the income over multiple future years, so you compare the dollars in / dollars out for which periods?

jes5199 2y ago

the problem is that software does not directly produce value, it is a way to place a bet on what will be valuable

antupis 2y ago
Pretty spot on and that is why devops stuff and automatic tests are so important. Those just let your bet faster.

voidhorse 2y ago

You can say this about any discipline. The root of the issue is that productivity for productivity's sake is meaningless, and it makes no sense to measure productivity as a general property when outputs vary. A tire factory has a distinct, singular goal: produce tires. It does this continually. Productivity is meaningful, but only in relation to a target that is typically specified by externalities (e.g. amount of demand) A software company is usually not in the business of producing consumable commodities so this kind of measurement does not make sense. It can make sense to measure productivity during a period for delivering a particular piece of software within a given time bound, but once it's delivered, productivity becomes meaningless. You always need to understand productivity in relation to some purpose and I don't know how these knuckleheads who think this abstract idea is basically like a concrete measurable essence, like mass, or liquid, got leadership positions.

thuanao 2y ago
Seems to me you’re talking about measuring software. Once we pick a measure though, we can calculate output/input (productivity). Imagine two people tasked with producing the same software, or software satisfying the same requirements or test suite. What would you call the person who produces it faster? More productive? Measuring software in financial terms or lines of code might not be the right measurement of software in all situations, but surely we can measure time and cost to produce software or software that satisfies equivalent requirements.
- taberiand 2y ago
  Depends in the coming months and years which was more stable, maintainable and extensible.
- hammock 2y ago
  You’re responding to the headline, not the article. The article has a plainly stated thesis: “Even if it could be measured, productivity in software does not approximate business value in any meaningful way”
  - 2y ago

csb6 2y ago

> I would argue that what good software developers do is remove problems. The opposite, in fact, of production But something is being produced - it is version 2.0 of the software. This is an artifact that is then shipped to users or deployed to a server. Peter’s solution fixed the issue and did not (seemingly) create further maintenance burden, which would have taken attention away from other tasks, i.e. reduced future productivity. I agree that metrics for programmer productivity are often useless (e.g. using lines of code is a bad idea for obvious reasons), but it seems silly to claim that the entire concept of productivity does not apply to the production of software.

foobiekr 2y ago
Silly and convenient. No one takes the claim that you can’t measure software productivity seriously and everyone simultaneously agrees that simple scalar metrics often fail to show the big picture in any and all disciplines. The rest is just usual software guy hubris and lack of awareness of the discipline.
- musicale 2y ago
  > No one takes the claim that you can’t measure software productivity seriously At least three people do, and they are mentioned in the article: the author, his colleague, and Martin Fowler. I suspect that software productivity (like the productivity of scientists, artists, composers, etc.) on any non-trivial project may be measurable on a scale of years, but not necessarily months or calendar quarters. A larger issue is that productivity is often due to external factors as much as it is to individual effort, and overnight success is often the result of extended periods of limited progress or even repeated failure.

mprast 2y ago

I do appreciate the little coda at the end - nice that the author was self-aware enough to realize where the cruft was and cut it (and courteous to the reader!)

johnfn 2y ago

I'm not sure I follow. So you failed to measure software productivity in lines of code, therefore it follows that "There's No Such Thing as Software Productivity"? Don't you think that giving up after n=1 attempts at measuring software productivity might be a tad too fast to draw a generalized claim of impossibility? I might argue the real lesson learned is "Lines of Code are Not a Measure of Productivity in an Isolated, Toy Example". I suspect this sort of thing gets promulgated because it kind of massages our ego, like yes, they can measure other sorts of productivity, but not ours, oh no, we're too complex and intelligent, there's no way to measure the deep sorts of work that we do! Which, yes, OK, we're not exactly bricklayers, but surely, if you had to, you could do better.

strulovich 2y ago
Ok, but then what is a way to do it? The text gives an example to the core problem, and to argue differently requires thinking around it. In practice. I’ve seen many attempts at measuring productivity, but once you dig into them, you see they are just abstraction mechanisms above something that is similar to lines of code. I have yet to see an idea that sidesteps the core issue described in this post. Also, it applies to many types of work, and software is not unique in any way.
- nemomarx 2y ago
  profit generated I think is the high level one, and then you want to dig from there into how much the software development contributed to this.
  - nomel 2y ago
    That's very tricky thing to quantify, especially with "unsung heroes". If my work is in preventing problems, the guy that fixes problems will be seen as the one that contributes more to profit, since impact is directly observed/measured. This is something that one of the orgs I worked for eventually realized. The people f'ing up, and then fixing their mistakes, were the ones getting promotions/bonuses/raises, because they were the ones interacting with all the execs.

jascha_eng 2y ago

You can very well at least count the problems that were solved (or deleted) you can also probably measure the value those solutions have in revenue or another metric. It's still true that measuring lines of code, time spent coding, commits or anything else is at best a proxy of productivity. It's also true that without any code changes problems more often than not don't get solved or we at least can't call the activity software development.

whoisthemachine 2y ago

Asking a software engineer to be more productive is akin to asking a mechanical engineer to be more productive. What does that end up looking like? More useless blueprints? It turns out when you ask software engineers to just crank out code, you just get lots of code.

fuzzfactor 2y ago

You could always look at things in a more abstract way ;) From one point of view, the users of the software are supposed to enjoy so much of a productivity increase that it's not supposed to matter if the coders are as productive as they could be or not. Give or take a few hundred percent at least. I realize that most people who've been to business school still aren't going to develop the needed acumen to handle a situation like this. Too many times the only training retained is a knee-jerk over-reaction to a fraction of a percent :\ Ever see one of these "leaders" have a cow and it was as stupid as it could have possibly been? I'm confident there are still some natural leaders that can thrive without worrying about every ounce of nose to the grindstone for their staff. Some things you just can't fix.

tbrownaw 2y ago

And yet somehow, software developers are doing something that's worth paying them a salary. And it's not always impossible to tell if someone's being held back at a lower salary band than they should be, or was promoted into a salary band that's higher than they're able to keep up with.

liontwist 2y ago
Yep it’s just nonsense. If it’s hard to define or measure I guess it doesn’t exist?
Analemma_ 2y ago
Yeah, these complaints are always so duplicitous. Somehow, it’s impossible to measure programmer productivity when it’s time for accountability or professional standards, but not when it’s time to assign annual bonuses.

timeforcomputer 2y ago

Yay Ben Rady. He has a great podcast with Matt Godbolt (who made Compiler Explorer) called Two's Complement.

irjustin 2y ago

I realized why this post rubs me the wrong way. It complains, but doesn't offer a solution. It simply criticizes and says "all engineers cannot and thusly should not be measured". The ironic thing is, the blog post is implicitly measuring by not explicitly measuring. The measurement is the bug ticket itself and whatever value attached to it. But to this end, I generally agree. There are qualitative and quantitative measurements. Quantitative is the value of the ticket commonly ascribed by the team (scrum? agile? whatever). Qualitative should come up in review. Qualitative is SO HARD. Top down? Team 360? Mixed? But it must be undertaken and refined by the team at each level of the org. Otherwise you will run into the exact situation described by the blog post and you won't know how to judge left from right, good from bad. Maybe the blog post's example isn't that great, too much information is missing to make a solid judgement, but you need to decide who to reward via promotion, annual raises and who to reprimand and who to not change. But still, all systems are terrible, but you must pick one less it be picked for you.

fcantournet 2y ago
Why do you need to evaluate people constantly and pit them against each other ? Why not give a raise to everyone and see what happens ? It's how we sent rockets to the moon, it seemed to work ok.
- ebiester 2y ago
  Because the alternative is based on your manager's impressions. You get fired because "your manager doesn't like you." You get a promotion because "your manager likes you." Your coworker sits around doing very little and nobody notices for a few months. Your company and boss is sued because someone says their firing was based on discrimination because they can't prove it was for performance. The truth is that managers can suss out 90% of the problems without a number. However, we are asked to document the hell out of it if we want to do something about it. And twice in my career I've been wrong: I mistook someone who was quiet for not doing much until I dove into the work. I trust my gut, but I confirm with numbers.
- irjustin 2y ago
  > It's how we sent rockets to the moon, it seemed to work ok. There are so many problems rooted in this statement. Government program, presidential mandate (i.e. unlimited budget), no competition. SpaceX is clearly better than NASA except maybe they don't push hard enough nor evaluate their engineers so you have a nice stable job if you do nothing.

revskill 2y ago

Productivity to me is about choosing the best tradeoff while minimizing the costs of going ahead.

mattmcknight 2y ago

So if you have a developer who writes no code and closes no tickets, who essentially does nothing except come to meetings, they are just as productive as a developer who writes some code and closes some tickets? Obviously there is a difference there. Negative code is still doing something as well. Back in the 90s we used Personal Software Process from the SEI, and we measured lines added, deleted, and changed. (as well as defects removed and added) It becomes clear that simple quantity of code and tickets is not enough- but it's also not nothing. Part of what is missing is quality and assessments of task complexity. Part of what is missing is the other parts of the jobs, like design and code reviews. I don't think it's hopeless, and can at least be used to look into why some people don't seem to make much at all.

danielmarkbruce 2y ago

And yet, Jeff Dean is significantly more productive than I am.

manmal 2y ago

Solving problems is productivity. That also means, solving them in a way that the solution doesn’t spawn new problems down the road, which is why we should follow the best practices. I can’t say whether deleting 100 lines is better than adding 1K lines, because before passing that judgement I would need to see what exactly those 100 and 1K lines are. OP is arguing too black and white IMO, and draws the conclusion before I‘m sold on the premise.

wpwpwpw 2y ago

Good productivity is about being able to do all tasks as planned. That includes the management, as its task is to attribute tasks. And whoever creates tasks. And whoever defines the objectives those tasks should answer to.

musicale 2y ago
Optimizing task completion is different from optimizing task benefit.

pjs_ 2y ago

In the scenario described in the article, Peter and Paul both achieve the same outcome in the same wall clock time. Obviously they are equivalently productive despite different working styles, by construction. But this doesn't account for the more realistic examples of Prakash, who completely fails to deliver a working solution, or delivers half a solution, and Percy, who gets it done two weeks late. I'm pretty sure you can define a shit_done/time_elapsed productivity metric for those two guys that is worse than that of Peter and Paul. Maybe I am a cynic but I suspect that some people are upvoting this because the framing makes them feel OK about getting paid six figures to work three hours a day...

dools 2y ago

I think this is just a semantic quibble based on a narrow interpretation of the word "produce". Because it also means: "cause (a particular result or situation) to happen or exist."

default-kramer 2y ago
Yeah, the comments on the post already said it all. First Isaac says > You seem to be playing a definition word game. [...] Obviously productivity does approximate business value in a very meaningful way when it is defined in terms of delivered business value. Then the author responds > Martin asserts "any true measure of software development productivity must be based on delivered business value". I agree, and I propose there is no such thing. We're better off dropping metaphors altogether and just talking about what programmers do: Solve problems. So I guess the author wants to replace the term "software productivity" with something like "problem-solving productivity, measured in terms of business value"...? Kind of silly IMO, but to each his own.

eacapeisfutuile 2y ago

This is fictious btw, or logic as follows is delete all code, deploy nothing, all bugs fixed, and achieve infinite productivity.

wcfrobert 2y ago

Wholeheartedly agree. SWEs are not factory workers. They don't punch in at 6am, put on their uniforms, make sure they meet their daily "lines of code" quota before clocking out at 10pm. We should not measure software productivity the same way we measure ball bearings production. Using lines of code to track productivity is absurd (do people really believe it or is it just a strawman at this point?). I'm reminded of that midwit meme where a junior has very few lines of code written because they don't know the code base well enough, the midwit writes up an whole framework, and the senior engineer has a net negative lines of code contribution. Theres also a fundamental difference between creating and maintaining code. Something like 10 guys wrote visicalc. Does that mean they were contributing millions of dollars in profit per hours? What about the maintainance to keep it going? Bug fixes? Patches? On call infra guys? What about opportunity cost of putting engineers on deadend projects? My point is tracking productivity in software dev - maybe all knowledge work for that matter - is complicated. Maybe that's why there's so much "busywork" (emails, slack, tickets, meetings, etc). Everyone wants to look productive but no one knows what it means

impure 2y ago

Let's say that you have two runners running the same marathon. The first one, Frank, sprints at full speed and eventually tires out and slows down. The second runner, Peter, takes a nap first and then finishes the marathon at the exact same time. Which of these two runners was faster in the race? The answer is: It doesn't matter. And therefore there is no such thing as running speed. There is such a thing as productivity in programming. If you could measure it it would likely be some combination of peer review and an analysis of the impact implemented features and fixes had. Some companies actually have programmers rate each other. I don't know how well it works and I think it can lead to perverse scenarios. But you can come up with metrics that are positively correlated with productivity.

robertlagrant 2y ago
The answer is neither is faster, because they finished at the same time.

0xbadcafebee 2y ago

"Put another way, productivity has no applicability as a metric in software. "How much did we create today?" is not a relevant question to ask. Even if it could be measured, productivity in software does not approximate business value in any meaningful way. This is because software development is not an activity that necessarily produces anything. This is ridiculous, irrelevant, and wrong. Of course software development produces things. It produces software. Which of these two developers was more "productive" today? The answer is: It doesn't matter. What matters the that Peter solved the problem, while simultaneously reducing long term maintenance costs for the team. Frank also solved the problem, but he increased maintenance costs by producing code, and so (all other things being equal) his solution is inferior. To call Peter more "productive" is to torture the metaphor beyond any possible point of utility. Ohhhhhhhhh. I get it. The author doesn't know what the word productivity means. Productivity does not mean "increases business value while decreasing maintenance costs [and having no net negative impact in any way]". It doesn't even mean "solving a problem". Productivity just means "to make something", or more specifically the rate at which something is made. That's all. You can make 10x more of something, and it can be garbage quality, but you did make it, and you did make more of it, so your productivity increased. If you produce 10x more grain than you did yesterday, you are more productive. The grain might now be full of heavy metals, pesticides and toxins. But you did in fact produce more grain. If you were trying to measure productivity of usable, healthy, high-quality grain, that is a different measurement than just productivity of grain. You may assume everybody knows what you mean when you say "productive", but you'd be wrong.

simpaticoder 2y ago

The difficulty of measuring productivity is particularly felt by senior developers. They save time and effort in ways that are non-obvious, which might be measured by dependencies that were not added, design patterns that were rejected, or processes that they pushed back on. Just like with living things, unchecked growth is unhealthy for an organism but the actions required are difficult to measure. One could start an attempt with a counter-factual narrative, but this does not map cleanly to KPIs.

jschrf 2y ago

Time spent on "incidental complexity" (new features, key fixes, performance) versus time spent on "accidental complexity" (anything and everything else). Easy metric to understand, easy metric to teach, just remember that it applies to teams and not individuals. See: Out of the Tar Pit.

deleted 2y ago

[deleted]

fulafel 2y ago

Productivity in economics refers to how many units of output you can generate with a given amount of input, it doesn't take into account quality, usefulness, etc. Complaining "lines-of-code is not a sensical productivity measure" is too kind to the concept of productivity. Using productivity as a metric leads to the same nonsequitur stuff in many many fields. It works somewhat for industrial output when you're working with commodities. Or it can work in some more fields as an ancillary measure if you pair it with some other quality, customer satisfaction, outcomes etc measure. But usually you don't want to maximize work while getting good outcomes, you just want the good outcomes.

vonnik 2y ago

And yet … there are some engineers who manage to solve hard problems to get something new working and debug it, however long it takes, and many others who don’t and/or can’t.

fermigier 2y ago

Strawman argument. It would have been (semi) interesting if the title was "Software productivity can't be measured by lines of code" (but already very eloquently stated in https://folklore.org/Negative_2000_Lines_Of_Code.html https://folklore.org/Negative_2000_Lines_Of_Code.html ).

lordnacho 2y ago

Isn't this whole debate a repeat of the old talking point about what is value? Labour theory of value, that kind of thing? This is simply never going to end if carried on along the lines of this article, or along the lines of most of the comments here. There's no way to objectively and reasonably put a value on something. All we have is a theory of subjective value, which does a bit of handwaving about utility and works out some ways where we can come to a price, regardless of the fact that the values in peoples' heads are subjective. Thus the distinction between knowledge work and "tangible work" like bricklaying is actually a moot point. Yes, you can measure "productivity" of bricklaying in metres per day, but ultimately you care about value, not amounts of wall. The arguments about one guy definitely being more productive than another are similar. One person values speed, another values maintenance costs downstream. It is subjective what ought to be more important.

sanitycheck 2y ago

Also, there's no such thing as sculpture productivity because all sculptors are doing is removing material.

lincpa 2y ago

[dead]

jFriedensreich 2y ago

Thats why bugs and chores have 0 points per default in pivotal tracker. Thats also why there was a push to force us say "as a user i..." at the beginning of story descriptions to make sure this is something creating user value. With these guards in place i don't follow the argument that productivity is not measurable, if a team builds features that solve an expressed user need and lets even say go through a final user acceptance check, this is a productive process and it is also very measurable!

focusbox 2y ago

[dead]

andrewqa 2y ago

[dead]

cushychicken 2y ago

Impact is the only useful metric. It is staggeringly hard to measure. Output is a weak proxy for impact. But it’s the one that makes intuitive sense to people. Doesn’t make it right or useful. I’m sure you all can envision a parable about your subfield of expertise that showcases how a seemingly light touch has a huge positive impact.

virgilp 2y ago
> It is staggeringly hard to measure. It's straight up impossible. Best we can do is observe and attempt to measure proxies. Take hypothetical situation: VPs A & B debate a business decision. Let's say A wins the argument and their solution leads to a revenue increase of $10M, and let's say we can confidently state that this is the outcome driven primarily by them winning the argument. Is the impact a net growth in business of $10M? In a sense, yes; but in some other sense, perhaps if they went with B's solution, the revenue growth would have been $15M? There's probably no way to know for sure unless you try both approaches, which is often impossible...
- cushychicken 2y ago
  My theory is that impact is heavily context dependent. If you can solve a problem quickly, at the right time, with buyin from your org, that’s positive impact. That general case differs wildly when you descend to the particulars.

popcorncowboy 2y ago

Clicked for the bait, stayed for the background tile.

toss1 2y ago

YUP My first rule of software programming is: Work Hard to Avoid Writing Code — Code is habitat for bugs; more code equals more bugs, and interactions between separate parts of code can harbor even more "interesting" varieties. — Code takes time to run. Any NoOp is faster than even your best hand-tuned assembly module. Obviously, this is not absolute in any sense — it breaks down as soon as you need the system to actually do something, at which point you must write some code. But it should be the minimal amount to get the job done, and nothing more. Obligatory car analogy: While doing some amateur sportscar racing, a coach asked me "What are the things you do that slow the car down?". I thought for a moment and started saying "when I start into a corner, if I do a bit too much...", he interrupted saying "NO, no, what are the BIG things you do that slow the car down?". "Oh, like braking and turning and lifting off the throttle?". "Right. So what that means is that you should always avoid doing those things. Obviously, you will certainly have to so some of them as soon as you approach the end of the front straight after the start, but make sure you understand your car, the track, and your skills to the point where you do only the absolute minimum." Both the software and sportscar versions are deceptively simple — they take a LOT more thinking than it seems at first glance. And that thinking is totally worth it.

deleted 2y ago

[deleted]

lazyant 2y ago

"There is no such thing as software productivity." != "There is no such thing as a simple, objective software productivity metric".

daniel1492 2y ago

[dead]