7 ms·

How These Things Work – A book about CS from first principles (2016)

688 points by falava 7y ago

Toenex 7y ago
Nice. I did an EE degree many years ago even though I wanted to work in software. The single best course I did was to design, simulate and have fabricated a 4 bit microprocessor. It completely solidified my understanding of how computations took place. Hopefully, texts like this will do the same for others.
- ci5er 7y ago
  I did a EE CS double major a LONG time ago. Senior year included a real summonavitch VLSI course, where in Q1 we had to design a 16- or 32- bit CPU. Over Thanksgiving break, TI manufactured them for us. Q2, we built a computer. Q3, we made it bootable (OS) and C compiler. I was interviewing for jobs in Japan (school was in Terre Haute, so travel overhead!) and I thought I was going to die. But it was the best thing that ever happened to me...
- JoeSmithson 7y ago
  In a similar vein, I highly recommend the MOOC "nand2tetris" which progresses smoothly from simple logic circuits right up to a high level Java-like programming language.
- thepete2 7y ago
  By "fabricate" do you mean that your design was produced in silicon? If so, that's super cool!
kragen 7y ago
Neat, and it looks like it's under a 3-clause BSD license, too: https://github.com/isovector/reasonablypolymorphic.com/blob/master/LICENSE https://github.com/isovector/reasonablypolymorphic.com/blob/... And it's tackling pretty advanced material — a bunch of category-theory stuff that I have no idea about. This is exciting! It looks like maybe it's unfinished: https://reasonablypolymorphic.com/book/tying-it-all-together.html https://reasonablypolymorphic.com/book/tying-it-all-together... ends, "Really, we’re just getting started," and then (the current version of) the book ends. What a cliffhanger ending! It doesn't seem to yet cover circuitry; the hardware it discusses seems to be a two-tape Turing machine, much like BF. The author seems to have been simulating the machine by hand to generate the included execution traces. I had a hard time finding the source at first: https://github.com/isovector/reasonablypolymorphic.com/blob/master/site/static/book/machine-diagrams.html https://github.com/isovector/reasonablypolymorphic.com/blob/... has a bunch of attribute-embedded &-escaped SVG (including XMLPIs!) that he almost certainly didn't type like that. That file is duplicated at https://github.com/isovector/reasonablypolymorphic.com/blob/master/docs/book/machine-diagrams.html https://github.com/isovector/reasonablypolymorphic.com/blob/... in the same format. As it turns out, the source for that post is in https://github.com/isovector/reasonablypolymorphic.com/blob/master/site/httw/2016-11-01-machine-diagrams.markdown https://github.com/isovector/reasonablypolymorphic.com/blob/..., with embedded Haskell to produce the SVGs. The build scripts looks like they might be in https://github.com/isovector/reasonablypolymorphic.com/tree/master/src https://github.com/isovector/reasonablypolymorphic.com/tree/... and https://github.com/isovector/reasonablypolymorphic.com/tree/master/scripts https://github.com/isovector/reasonablypolymorphic.com/tree/... but I can't tell where the code for generating SVG comes from. ("stack install" maybe? But then is it datetime, sitepipe, or strptime?) So I can't figure out how to fix the text in the SVGs to not crash into the diagram lines. Careful about cloning the repo. It's a quarter gig!
- javajosh 7y ago
  The author states that computer science has little to do with computers on the first page so I would not expect circuitry.
  - kragen 7y ago
    I don't disagree with him on that, but there's really quite a bit of stuff in there about quasi-circuitry-like things: "machines" and "latches" and things like > In the next chapter, we'll investigate how to make machines that change over time, which will be the basis for us to store information. From there it's just a short hop to actual computers! so I think that grounding abstract computation in something that can clearly be constructed in real life is actually very much a concern of the book, even if he's not planning to cover IEEE-754 denormals, the cost of TLB flushes, or strategies for reducing the bit error rate due to metastability when crossing clock domains.
koonsolo 7y ago
I never heard of "first principles" until Elon Musk used it. (Not a native English speaker) Now I see it everywhere. Must be the new overhyped term. "We start from first principles, just like Elon Musk". After looking at Google trends, I might be wrong, so nevermind ;) https://trends.google.com/trends/explore?date=all&q=First%20principles https://trends.google.com/trends/explore?date=all&q=First%20...
- eandre 7y ago
  That's the Baader-Meinhof phenomenon talking [1] :) [1] https://science.howstuffworks.com/life/inside-the-mind/human-brain/baader-meinhof-phenomenon.htm https://science.howstuffworks.com/life/inside-the-mind/human...
- deleted 7y ago
  [deleted]
- siddboots 7y ago
  It's a relatively common idiom here in Australia. It gets used quite a bit in STEM education, i.e. "prove such-and-such from first principles", but is also pretty common more colloquially. According to your link, it is also used a bit in South Africa (where Elon grew up), but less common in the US. Rather than being a new and overhyped term, perhaps it is a case of Elon using a term that is quite everyday to him, without realizing it is less familiar to the audience.
- chestervonwinch 7y ago
  It often occurs that technical jargon is swiped to be used as business speak by business leaders with technical backgrounds. For example I often hear “orthogonal” when “independent” is usually more appropriate.
- KineticLensman 7y ago
  UK here - I remember it back in college thirty+ years ago as in "build X from first principles", which could mean (e.g.) implement an algorithm without out using library calls.
forkeep 7y ago
a
matco11 7y ago
One of my favorite books on subject is The Elements of Computing Systems: Building a Modern Computer from First Principles by Noam Nisan https://www.amazon.com/Elements-Computing-Systems-Building-Principles/dp/0262640686 https://www.amazon.com/Elements-Computing-Systems-Building-P...
- djangovm 7y ago
  Would you say that this book is still relevant, now that it is 15 years old?
  - jambutters 7y ago
    yes, its structured in a way that's very hands on
  - dang 7y ago
    It's wonderfully relevant because it takes you through the basics and those are all still the same. Their simulator software was already old when I went through the book a few years ago, but I'm sure it still works fine.
- dbish 7y ago
  Agreed. I read through this on a recent vacation and even though i have a computer engineering degree, it was nice to get through the basics again and they have some fun exercises.
ceronman 7y ago
I love this kind of Books! In a much more general tone, there is "The Knowledge: How To Rebuild Our World After An Apocalypse" by Lewis Dartnell. This explains the first principles of a lot of things we take for granted in the modern world, from agriculture to food and clothing, medicine, chemistry and more. This book about CS principles is a great complement to that!
- 7thaccount 7y ago
  I've been looking for a book like that lol.
- kragen 7y ago
  As I posted the last time this book came up https://news.ycombinator.com/item?id=22294655 https://news.ycombinator.com/item?id=22294655 that book would be more accurately called The Misinformation. If you follow the instructions in it you will die. Better alternatives are listed in my linked comment.
  - jwhitlark 7y ago
    I agree. I was deeply disappointed by "The Knowledge..."
- fernly 7y ago
  Oh, puh-leeze! THE book on rebuilding civilization is How to Invent Everything: A Survival Guide for the Stranded Time-Traveler[1] by Ryan North. [1] https://www.amazon.com/How-Invent-Everything-Survival-Stranded-ebook/dp/B07B2LNVBY https://www.amazon.com/How-Invent-Everything-Survival-Strand...
hcarvalhoalves 7y ago
Half-way into it’s introducing monads and Maybe. Feels like teaching a stack machine after talking about the visitor pattern. There’s good information here but I’m not sure it covers the important fundamentals (such that I could give to a beginner).
- deleted 7y ago
  [deleted]
grumple 7y ago
I recommend Code: The Hidden Language of Computer Hardware and Software by Charles Petzold [1]. It is far more comprehensive than the OP, goes from pre-computer code, to electrical circuits, to an overview of assembly. No prior knowledge needed except how to read. 1. https://www.amazon.com/Code-Language-Computer-Hardware-Software/dp/0735611319 https://www.amazon.com/Code-Language-Computer-Hardware-Softw...
- throw1234651234 7y ago
  This is part of what allowed me to get into programming. The "no prior" knowledge part is absolutely true. I did start getting lost around the second half of the book.
- kragen 7y ago
  Code's good but it doesn't cover Kleisli categories and Kleisli composition, Peano arithmetic, parametric polymorphism, sum types, pattern matching, or any of numerous other things covered in Maguire's How These Things Work. So it's not accurate to say Code is "far more comprehensive"; Code mentions FORTRAN, ALGOL, COBOL, PL/1, and BASIC, but the example programs it contains are written in assembly, ALGOL, and BASIC. It doesn't contain any programs you can actually run except for an three-line BASIC program and some even simpler assembly programs.
  - grumple 7y ago
    You are correct, Code does not contain all of the knowledge relevant to computer science. In fact, no book does, as far as I'm aware. But it is far more comprehensive than the OP because it covers a greater breadth of subjects and in greater depth with more accessibility. You're comparing 50 pages of blog posts to 300 pages of book.
    - kragen 7y ago
      I think there's more computer science in those 50 pages, really. It's an easy win.
  - 7y ago
Anon84 7y ago
This looks really cool, thank you. Another book I particularly like in the same style are Feynman (lesser known) lectures on computation: https://amzn.to/2SSoJaR https://amzn.to/2SSoJaR where he takes you from single instructions all the way to quantum computing
oblosys 7y ago
For a do-it-yourself version of chapter 1 (about building complex circuits using only nands), I can recommend http://nandgame.com/ http://nandgame.com/
porknubbins 7y ago
A few years ago, I asked my engineer friend about how much of civilization he could rebuild singlehandedly, should he survive some hypothetical apocalyptic event. “All of it,” he replied. “Not all at once, but I know enough to be able to puzzle together the pieces I don’t know right this second.” While I admire the Connecticut-Yankee optimism of the engineer, as a non engineer I am seriously skeptical about how a single engineer could know enough about the chemistry, materials, physics, CS etc. I can explain what a battery, or transistor is supposed to do but wouldn't have the foggiest idea how to actually make one. In this scenario are we leaving the bunker to break into Bell Labs (or some research university library at least)?
- ClumsyPilot 7y ago
  The questions usually lack a starting point. It's one thing to be stuck I medieval time, but stone age is another thing entirely.
- augbog 7y ago
  Somewhat of a tangent but related, there is an anime called Dr.Stone where a brilliant genius scientist kid gets ported 3700 years in the future and people have reverted back to the stone ages. He teaches them how to build everything from scratch and makes some crazy stuff i.e. antibiotics, etc. Highly recommend
- latexr 7y ago
  I share your skepticism. Seems to me the engineer is falling prey to the Dunning-Kruger effect[1]. Rather than knowing enough to be able to puzzle together all the pieces they don’t know, I’d wager they don’t know enough to be able to discern what they won’t be able to figure out. [1]: https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
  - naasking 7y ago
    We studied a lot of detailed semiconductor physics in my engineering degree. Certainly loads of details were left out, but knowing something is possible and roughly along which lines it could be achieved is a huge hurdle in the innovation process.

688 points by falava 7y ago

Nice. I did an EE degree many years ago even though I wanted to work in software. The single best course I did was to design, simulate and have fabricated a 4 bit microprocessor. It completely solidified my understanding of how computations took place. Hopefully, texts like this will do the same for others.

ci5er 7y ago
I did a EE CS double major a LONG time ago. Senior year included a real summonavitch VLSI course, where in Q1 we had to design a 16- or 32- bit CPU. Over Thanksgiving break, TI manufactured them for us. Q2, we built a computer. Q3, we made it bootable (OS) and C compiler. I was interviewing for jobs in Japan (school was in Terre Haute, so travel overhead!) and I thought I was going to die. But it was the best thing that ever happened to me...
JoeSmithson 7y ago
In a similar vein, I highly recommend the MOOC "nand2tetris" which progresses smoothly from simple logic circuits right up to a high level Java-like programming language.
thepete2 7y ago
By "fabricate" do you mean that your design was produced in silicon? If so, that's super cool!

kragen 7y ago

Neat, and it looks like it's under a 3-clause BSD license, too: https://github.com/isovector/reasonablypolymorphic.com/blob/master/LICENSE https://github.com/isovector/reasonablypolymorphic.com/blob/... And it's tackling pretty advanced material — a bunch of category-theory stuff that I have no idea about. This is exciting! It looks like maybe it's unfinished: https://reasonablypolymorphic.com/book/tying-it-all-together.html https://reasonablypolymorphic.com/book/tying-it-all-together... ends, "Really, we’re just getting started," and then (the current version of) the book ends. What a cliffhanger ending! It doesn't seem to yet cover circuitry; the hardware it discusses seems to be a two-tape Turing machine, much like BF. The author seems to have been simulating the machine by hand to generate the included execution traces. I had a hard time finding the source at first: https://github.com/isovector/reasonablypolymorphic.com/blob/master/site/static/book/machine-diagrams.html https://github.com/isovector/reasonablypolymorphic.com/blob/... has a bunch of attribute-embedded &-escaped SVG (including XMLPIs!) that he almost certainly didn't type like that. That file is duplicated at https://github.com/isovector/reasonablypolymorphic.com/blob/master/docs/book/machine-diagrams.html https://github.com/isovector/reasonablypolymorphic.com/blob/... in the same format. As it turns out, the source for that post is in https://github.com/isovector/reasonablypolymorphic.com/blob/master/site/httw/2016-11-01-machine-diagrams.markdown https://github.com/isovector/reasonablypolymorphic.com/blob/..., with embedded Haskell to produce the SVGs. The build scripts looks like they might be in https://github.com/isovector/reasonablypolymorphic.com/tree/master/src https://github.com/isovector/reasonablypolymorphic.com/tree/... and https://github.com/isovector/reasonablypolymorphic.com/tree/master/scripts https://github.com/isovector/reasonablypolymorphic.com/tree/... but I can't tell where the code for generating SVG comes from. ("stack install" maybe? But then is it datetime, sitepipe, or strptime?) So I can't figure out how to fix the text in the SVGs to not crash into the diagram lines. Careful about cloning the repo. It's a quarter gig!

javajosh 7y ago
The author states that computer science has little to do with computers on the first page so I would not expect circuitry.
- kragen 7y ago
  I don't disagree with him on that, but there's really quite a bit of stuff in there about quasi-circuitry-like things: "machines" and "latches" and things like > In the next chapter, we'll investigate how to make machines that change over time, which will be the basis for us to store information. From there it's just a short hop to actual computers! so I think that grounding abstract computation in something that can clearly be constructed in real life is actually very much a concern of the book, even if he's not planning to cover IEEE-754 denormals, the cost of TLB flushes, or strategies for reducing the bit error rate due to metastability when crossing clock domains.

koonsolo 7y ago

I never heard of "first principles" until Elon Musk used it. (Not a native English speaker) Now I see it everywhere. Must be the new overhyped term. "We start from first principles, just like Elon Musk". After looking at Google trends, I might be wrong, so nevermind ;) https://trends.google.com/trends/explore?date=all&q=First%20principles https://trends.google.com/trends/explore?date=all&q=First%20...

eandre 7y ago
That's the Baader-Meinhof phenomenon talking [1] :) [1] https://science.howstuffworks.com/life/inside-the-mind/human-brain/baader-meinhof-phenomenon.htm https://science.howstuffworks.com/life/inside-the-mind/human...
deleted 7y ago
[deleted]
siddboots 7y ago
It's a relatively common idiom here in Australia. It gets used quite a bit in STEM education, i.e. "prove such-and-such from first principles", but is also pretty common more colloquially. According to your link, it is also used a bit in South Africa (where Elon grew up), but less common in the US. Rather than being a new and overhyped term, perhaps it is a case of Elon using a term that is quite everyday to him, without realizing it is less familiar to the audience.
chestervonwinch 7y ago
It often occurs that technical jargon is swiped to be used as business speak by business leaders with technical backgrounds. For example I often hear “orthogonal” when “independent” is usually more appropriate.
KineticLensman 7y ago
UK here - I remember it back in college thirty+ years ago as in "build X from first principles", which could mean (e.g.) implement an algorithm without out using library calls.

forkeep 7y ago

matco11 7y ago

One of my favorite books on subject is The Elements of Computing Systems: Building a Modern Computer from First Principles by Noam Nisan https://www.amazon.com/Elements-Computing-Systems-Building-Principles/dp/0262640686 https://www.amazon.com/Elements-Computing-Systems-Building-P...

djangovm 7y ago
Would you say that this book is still relevant, now that it is 15 years old?
- jambutters 7y ago
  yes, its structured in a way that's very hands on
- dang 7y ago
  It's wonderfully relevant because it takes you through the basics and those are all still the same. Their simulator software was already old when I went through the book a few years ago, but I'm sure it still works fine.
dbish 7y ago
Agreed. I read through this on a recent vacation and even though i have a computer engineering degree, it was nice to get through the basics again and they have some fun exercises.

ceronman 7y ago

I love this kind of Books! In a much more general tone, there is "The Knowledge: How To Rebuild Our World After An Apocalypse" by Lewis Dartnell. This explains the first principles of a lot of things we take for granted in the modern world, from agriculture to food and clothing, medicine, chemistry and more. This book about CS principles is a great complement to that!

7thaccount 7y ago
I've been looking for a book like that lol.
kragen 7y ago
As I posted the last time this book came up https://news.ycombinator.com/item?id=22294655 https://news.ycombinator.com/item?id=22294655 that book would be more accurately called The Misinformation. If you follow the instructions in it you will die. Better alternatives are listed in my linked comment.
- jwhitlark 7y ago
  I agree. I was deeply disappointed by "The Knowledge..."
fernly 7y ago
Oh, puh-leeze! THE book on rebuilding civilization is How to Invent Everything: A Survival Guide for the Stranded Time-Traveler[1] by Ryan North. [1] https://www.amazon.com/How-Invent-Everything-Survival-Stranded-ebook/dp/B07B2LNVBY https://www.amazon.com/How-Invent-Everything-Survival-Strand...

hcarvalhoalves 7y ago

Half-way into it’s introducing monads and Maybe. Feels like teaching a stack machine after talking about the visitor pattern. There’s good information here but I’m not sure it covers the important fundamentals (such that I could give to a beginner).

deleted 7y ago
[deleted]

grumple 7y ago

I recommend Code: The Hidden Language of Computer Hardware and Software by Charles Petzold [1]. It is far more comprehensive than the OP, goes from pre-computer code, to electrical circuits, to an overview of assembly. No prior knowledge needed except how to read. 1. https://www.amazon.com/Code-Language-Computer-Hardware-Software/dp/0735611319 https://www.amazon.com/Code-Language-Computer-Hardware-Softw...

throw1234651234 7y ago
This is part of what allowed me to get into programming. The "no prior" knowledge part is absolutely true. I did start getting lost around the second half of the book.
kragen 7y ago
Code's good but it doesn't cover Kleisli categories and Kleisli composition, Peano arithmetic, parametric polymorphism, sum types, pattern matching, or any of numerous other things covered in Maguire's How These Things Work. So it's not accurate to say Code is "far more comprehensive"; Code mentions FORTRAN, ALGOL, COBOL, PL/1, and BASIC, but the example programs it contains are written in assembly, ALGOL, and BASIC. It doesn't contain any programs you can actually run except for an three-line BASIC program and some even simpler assembly programs.
- grumple 7y ago
  You are correct, Code does not contain all of the knowledge relevant to computer science. In fact, no book does, as far as I'm aware. But it is far more comprehensive than the OP because it covers a greater breadth of subjects and in greater depth with more accessibility. You're comparing 50 pages of blog posts to 300 pages of book.
  - kragen 7y ago
    I think there's more computer science in those 50 pages, really. It's an easy win.
- 7y ago

Anon84 7y ago

This looks really cool, thank you. Another book I particularly like in the same style are Feynman (lesser known) lectures on computation: https://amzn.to/2SSoJaR https://amzn.to/2SSoJaR where he takes you from single instructions all the way to quantum computing

oblosys 7y ago

For a do-it-yourself version of chapter 1 (about building complex circuits using only nands), I can recommend http://nandgame.com/ http://nandgame.com/

porknubbins 7y ago

A few years ago, I asked my engineer friend about how much of civilization he could rebuild singlehandedly, should he survive some hypothetical apocalyptic event. “All of it,” he replied. “Not all at once, but I know enough to be able to puzzle together the pieces I don’t know right this second.” While I admire the Connecticut-Yankee optimism of the engineer, as a non engineer I am seriously skeptical about how a single engineer could know enough about the chemistry, materials, physics, CS etc. I can explain what a battery, or transistor is supposed to do but wouldn't have the foggiest idea how to actually make one. In this scenario are we leaving the bunker to break into Bell Labs (or some research university library at least)?

ClumsyPilot 7y ago
The questions usually lack a starting point. It's one thing to be stuck I medieval time, but stone age is another thing entirely.
augbog 7y ago
Somewhat of a tangent but related, there is an anime called Dr.Stone where a brilliant genius scientist kid gets ported 3700 years in the future and people have reverted back to the stone ages. He teaches them how to build everything from scratch and makes some crazy stuff i.e. antibiotics, etc. Highly recommend
latexr 7y ago
I share your skepticism. Seems to me the engineer is falling prey to the Dunning-Kruger effect[1]. Rather than knowing enough to be able to puzzle together all the pieces they don’t know, I’d wager they don’t know enough to be able to discern what they won’t be able to figure out. [1]: https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
- naasking 7y ago
  We studied a lot of detailed semiconductor physics in my engineering degree. Certainly loads of details were left out, but knowing something is possible and roughly along which lines it could be achieved is a huge hurdle in the innovation process.