8 ms·

People say similar things about automation of software engineering. Different, but similar. I'm deeply suspicious. I do not yet have a concise statement for wh

by fultonn 28d ago

People say similar things about automation of software engineering. Different, but similar. I'm deeply suspicious. I do not yet have a concise statement for why, but a lot of literature on the sociology of knowledge work sort of points at my thoughts. Section 5 of the Thurston article cited by Tao touches the elephant. Raduchel's article on the economics of software [2] also touches it. I've tried to put words to this for a few years. I think I'm just going to start writing versions of it as see if that helps me shape the thought into something more concise. So, in the spirit of this article's style, here are some postulates: 1. There is a sociological process happening in the production function during knowledge work. 2. That production function and the associated sociological process spans years or even decades, and must outlast many of the artifacts that are produced during the early years of the function. 3. You cannot get the right lines of code or the right theorems proved without running that sociological process alongside the artifact production process. 4. It is impossible to completely separate the sociological process from the artifact construction process. If you just iterate on artifacts then too much of the required hidden state is lost to make progress in the right direction. This is true even if you include distilled artifacts capturing pieces of the sociological process (eg meeting notes, documentation, commit logs, prompts). 5. So you need that sociological process, or something like it, to still happen. 6. For a lot of knowledge work that process plays out in extremely high-fidelity social interactions [3] that we have not yet captured in the datasets that would be required to reproduce those dynamics. 7. And even if we do collect that data, our current architectures and training algorithms and hardware would be useless given the size of the datasets. So: the technology today gives us the ability to iterate on the production of artifacts. But it does not sufficiently simulate the social process which gives rise to the Right artifacts. This isn't exactly what I actually think, but it's a version of the thing that I intuit when I watch heavy use of AI in both software projects and formalization projects. And simulating that process feels way harder than people are currently assuming. [1] https://arxiv.org/pdf/math/9404236 https://arxiv.org/pdf/math/9404236 Section 5. [2] https://www.nationalacademies.org/read/11587/chapter/11 https://www.nationalacademies.org/read/11587/chapter/11 pp 166-168. [3] there is a reason we still gather in-person around white boards, and why doing so is more crucial for some types of work than others.

energy123 28d ago
Many eminent mathematicians did their best work while not talking about it with anyone, sometimes in isolation. Newton's calculus, Perelman's poincare, much of Grothendiek's work, Wiles's fermat, Ramanujan's earlier days. That's not all top mathematicians as you can look at Von Neumann as a sociable counter-example. But it shows that discussion of your current ideas is not a requirement. Grothendiek goes so far as to say it is a net negative for mathematical creativity because it is difficult to resist thinking like the herd without some level of seclusion.
- fultonn 27d ago
  > eminent I would use a different adjective: exceptional. There are not not times where lone geniuses produce amazing output. But most of humanity's progress over the last few thousand years (or at least certainly the last few hundred) resulting from a different type of work.
  - energy123 27d ago
    That hypothesis requires substantiation. Just because 10 people in a room came up with a breakthrough doesn't mean a sociological process contributed positively to that breakthrough. People generally like socializing, so the base rate is going to be high by default. The breakthroughs might have been more numerous if they had each followed Grothendieck's advice (who is a contender for the world's most talented theory builder) and intentionally decorrelated from one another to free themselves from convention. Referring to my list of examples as exceptional in the sense of being rare is rather unfair, given how numerous these examples are relative to the body of mathematical work we would consider incredible and especially relative to the desire of the average human to socialize. The fact that such a large % of that body of work occurred while the individual was in relative isolation is something we should pay attention to.
    - fultonn 23d ago
      > People generally like socializing If people don't like socializing then your argument not only falls apart, but actually leads us to something like the opposite of the conclusion you found. And people definitely don't like socializing! Because we're not talking about "socializing" in the sense of parties or even water cooler conversations. We're talking about socializing in the context of a group of professionals working together on under-specified problems/solutions. The English word for that type of socializing is: Meetings. Saying that "people like socializing" in the context of professional knowledge work is like saying "people like woodworking" in the context of stick framing cookie cutter developments in Georgia in the summer. I don't think I've ever worked with a productive scientist or mathematician who didn't loathe meetings. In fact, I think I can probably count on one hand the number of people I've met who seem to not loathe meetings. Also: there are pretty powerful social/professional forces -- in Mathematics and related disciplines in particular -- which incentivize people to play into the perception that they are a "lone genius" or at least the "main character". So the group of people we're talking about both hate meetings and have a strong incentive to be perceived as an independent genius/main contributor. And yet the vast majority of good science and mathematics involves large groups of people (who mostly complain about the resulting necessity of meetings, lol). I'll also submit that most of the "lone genius" instances are confounded by the strong social incentive to be perceived as either a lone genius or at least the main contributor. There are at least a few prominent historical examples of math/math-adjacent giants who "worked alone" but were in fact in deep conversation with both the literature and their peers. But, again, there's a lot of gray area, and that's what I meant in my original post when I said "This isn't exactly what I actually think". Two examples of gray area: 1. I'll certainly concede that there are some types of work that do not require that sort of social process to happen. That's the sort of work you seem to be pointing toward here. But opportunity to do that type of work is rare in general, and those opportunities are far more scarce today than it was in 1690 or even 1990. 2. There are also some types of work where the corresponding sociological process is pretty simple and is probably sufficiently captured. So I'm not bearish about what's possible, at least per se. But technology executives and analysts in particular have some really fundamental misunderstandings about how this type of labor works in most cases, and I see some of those same misunderstandings among lots of professional mathematicians.

by fultonn 28d ago

energy123 28d ago

Many eminent mathematicians did their best work while not talking about it with anyone, sometimes in isolation. Newton's calculus, Perelman's poincare, much of Grothendiek's work, Wiles's fermat, Ramanujan's earlier days. That's not all top mathematicians as you can look at Von Neumann as a sociable counter-example. But it shows that discussion of your current ideas is not a requirement. Grothendiek goes so far as to say it is a net negative for mathematical creativity because it is difficult to resist thinking like the herd without some level of seclusion.

fultonn 27d ago
> eminent I would use a different adjective: exceptional. There are not not times where lone geniuses produce amazing output. But most of humanity's progress over the last few thousand years (or at least certainly the last few hundred) resulting from a different type of work.
- energy123 27d ago
  That hypothesis requires substantiation. Just because 10 people in a room came up with a breakthrough doesn't mean a sociological process contributed positively to that breakthrough. People generally like socializing, so the base rate is going to be high by default. The breakthroughs might have been more numerous if they had each followed Grothendieck's advice (who is a contender for the world's most talented theory builder) and intentionally decorrelated from one another to free themselves from convention. Referring to my list of examples as exceptional in the sense of being rare is rather unfair, given how numerous these examples are relative to the body of mathematical work we would consider incredible and especially relative to the desire of the average human to socialize. The fact that such a large % of that body of work occurred while the individual was in relative isolation is something we should pay attention to.
  - fultonn 23d ago
    > People generally like socializing If people don't like socializing then your argument not only falls apart, but actually leads us to something like the opposite of the conclusion you found. And people definitely don't like socializing! Because we're not talking about "socializing" in the sense of parties or even water cooler conversations. We're talking about socializing in the context of a group of professionals working together on under-specified problems/solutions. The English word for that type of socializing is: Meetings. Saying that "people like socializing" in the context of professional knowledge work is like saying "people like woodworking" in the context of stick framing cookie cutter developments in Georgia in the summer. I don't think I've ever worked with a productive scientist or mathematician who didn't loathe meetings. In fact, I think I can probably count on one hand the number of people I've met who seem to not loathe meetings. Also: there are pretty powerful social/professional forces -- in Mathematics and related disciplines in particular -- which incentivize people to play into the perception that they are a "lone genius" or at least the "main character". So the group of people we're talking about both hate meetings and have a strong incentive to be perceived as an independent genius/main contributor. And yet the vast majority of good science and mathematics involves large groups of people (who mostly complain about the resulting necessity of meetings, lol). I'll also submit that most of the "lone genius" instances are confounded by the strong social incentive to be perceived as either a lone genius or at least the main contributor. There are at least a few prominent historical examples of math/math-adjacent giants who "worked alone" but were in fact in deep conversation with both the literature and their peers. But, again, there's a lot of gray area, and that's what I meant in my original post when I said "This isn't exactly what I actually think". Two examples of gray area: 1. I'll certainly concede that there are some types of work that do not require that sort of social process to happen. That's the sort of work you seem to be pointing toward here. But opportunity to do that type of work is rare in general, and those opportunities are far more scarce today than it was in 1690 or even 1990. 2. There are also some types of work where the corresponding sociological process is pretty simple and is probably sufficiently captured. So I'm not bearish about what's possible, at least per se. But technology executives and analysts in particular have some really fundamental misunderstandings about how this type of labor works in most cases, and I see some of those same misunderstandings among lots of professional mathematicians.