8 ms·
I summarized my understanding of Linux systems
- smitty1e 3y agoI think it needs three areas, not two: 1. User space 2. Kernel 3. Hardware/network The kernel protects users from hardware, and hardware from users.
- t1tos 3y agothis is analagous to the fs hierarchy: root protects from the user
- topspin 3y agoThis is reasonable and correct. I would also have found places in that map for: dcache, block devices, character devices, scheduler, page cache and console/tty/pty. The first two replace "filesystem hierarchy". The second and third are ancient and fundamental classes of UNIX devices.
- cookiengineer 3y agoI can recommend taking a look at /proc and /sys, because that will clear up a lot of how things are intertwined and connected. procfs is what's used by pretty much all tools that do something with processes, like ps, top etc. The everything is a file philosophy becomes much more clear then. Even low level syscalls and their structs are offered by the kernel as file paths so you can interact with them by parsing those files as a struct, without actually needing to use kernel headers for compilation. eBPF and its bytecode VM are a little over the top, but are essential to known about in the upcoming years cause a lot of tools is moving towards using their own bpf modules.
- Cloudef 3y ago> The everything is a file philosophy becomes much more clear then. To be honest, everything is a file is kind of a lie in unix. /proc and /sys are pretty much plan9 inspiration.
- arghwhat 3y agoA more accurate term is that everything is a file descriptor. The main difference is that plan9 uses read and write for everything, whereas Linux and BSD uses ioctls on file descriptors for everything.
- vbezhenar 3y agoEverything is a descriptor. When I'm opening a TCP connection, there's no file, so calling it a file descriptor feels wrong. And at that point, the whole "everything is" turns into nonsense, because yes, everything is a pointer to something, so what.
- arghwhat 3y agoThere are named files (which have a file path) and anonymous files (which do not). You can see these in /proc/$PID/fd/$FD if you're curious - when the link doesn't start with '/', it's anonymous. Even process memory is just an anonymous file on Linux, and arguably a cleaner one as it operates on proper fds, instead of plan9 where a string "class name" (not a path) is used to access the magical '#g' filesystem. The difference to plan9 is not the files, but the way plan9 uses text protocols with read/write to ctl files. To open a TCP connection - if memory serves me right - you first have to write to a ctl file, which creates a new file for the connection. Then, you write the dial command to the ctl file of that connection, and after which you can open the connection file. On Linux, a syscall creates an anonymous file, and then everything after is operations on this anonymous file. There's some ideological benefits, but plan9 creates a mess of implicit text protocols, ugly string handling, syscall storms and memory inefficiencies. Their design is pretty much solely a limitation caused by the idea that all filesystems should exist through the 9p protocol, which as a networked protocol cannot share data (fds, structs), only copy (payloads to/from read, write). With the idea that all functionality had to be replaceable and mountable from remote machines, the only possible API became read/write for everything. I'd argue that fd-using syscalls and ioctls - basically per-file syscalls - is a superior approach to implement everything-as-a-file.
- 2y ago
- dfc 3y agoMy mental model of Linux does not have the CPU/Memory in user space. What am I missing?
- vbezhenar 3y agoUserspace program directly uses CPU and memory (unless you're using VM). In contrast to that, your userspace program does not directly access your network device or SSD, but uses kernel routines to access those indirectly.
- persnickety 3y agoIf you're using a hypervisor, then the userspace program inside the VM is also using the CPU and memory directly. You'd have to do full emulation to avoid that. Even with full emulation, I'd say memory is being accessed directly, unless you really go out of your way to make it weird.
- icedchai 3y agoWith full emulation, I'd argue memory access is not direct. Memory access from the emulated system will go through user space code in the emulator. That code may translate it to actual memory access, or perhaps an emulated, memory mapped I/O device like a frame buffer. Either way, there is something in the middle. You could argue that nothing is direct unless you're running on a bare metal system, no MMU, no page tables. How do you define "direct"?
- persnickety 3y agoFor the definition I used earlier, the bytes don't change shape in between. That makes MMU and interpreted access direct, and compression indirect.
- sophacles 3y agoIt doesn't directly access memory. The addresses in your userspace program are not the actual addresses of memory in the ram sticks - there is a table of mappings that the kernel sets up. When your process asks the kernel for memory, it says "i need 5KB, please put that at address XYZ". The kerenel goes and finds 5KB unused, probably at some other address ABC, and creates a mapping in the table that says XYZ translates to ABC. Then the kernel sets the MMU of the CPU to use the table for your process, and switches back to unpriveleged mode, letting your process run again. Your process in unpriveleged mode sends an instruction to write to the memory at XYZ, but the cpu will translate that instruction to ABC and write there instead. VMs (well not emulated vms, but if you're doing an x86 vm on x86 or an arm vm on arm) do something similar - an inaccurate (but useful for the concept) way to think of it is that the cpu does 2 layers of MMU for user processes in a vm. The kernel code isn't running directly when your program accesses memory, but it sets up the cpu so that the kernel still has control over your memory, and you only have access to what the kernel allows - its mediated by the kernel.
- richardwhiuk 3y agoI don't understand what the boxes on this diagram are meant to represent. It feels like an elaborate mechanism to draw something wrong in the hopes people will correct it.
- da39a3ee 3y ago[flagged]
- projektfu 3y agoFWIW, I also don't really understand what the boxes are supposed to represent, given that the arrows represent dependencies like PID <-- process. I thought a PID was an attribute of a process? To me, a block diagram might show [CPU Scheduler], [Virtual Device Manager], [VFS Manager], [Memory Manager], [Interrupt Handlers], etc... Of course, my knowledge of Linux internals is limited and perhaps it has a separation of the concept of PID and process where there is a literal dependency.
- sevagh 3y agoInterview prep.
- timeforcomputer 3y agoNice! I want to do something similar and map my understanding of Linux. I find some diagrams on Wikipedia fascinating (example: https://en.m.wikipedia.org/wiki/File:Linux_Graphics_Stack_2013.svg https://en.m.wikipedia.org/wiki/File:Linux_Graphics_Stack_20..., but more to do with the user library ecosystem rather than kernel and program runtime). These diagrams make me want to learn about each part and be able to comprehend in principle what is happening on my machine. Eventually...
- Jasper_ 3y agoAny diagram by ScotXW on Wikipedia is somewhere between misleading and completely wrong, and they're a constant pain on the Linux graphics community. If you're curious about the details in this case, ScotXW confuses EGL and OpenGL, the arrows aren't quite right, and the labels aren't quite right either (DRM is labeled "hardware-specific" but KMS isn't? The label for "hardware specific Userspace interface to hardware specific direct rendering manager" is quite weird), and some important boxes are flat out missing. It's nitpicking for sure, but when the diagram goes out of its way to add extremely weird details, it demands nitpicking. Nobody in the Linux graphics community would draw the diagram like this.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- hn_user82179 3y agoI remember when Wikipedia first became popular, there were a lot of warnings about how you couldn't trust the information because "anyone could edit it". I feel like, at least to my level of understanding whatever I'm reading about, it's been sufficient and I've never identified something wrong/inaccurate (except for perhaps recent news or recently debated political topics). This is the first time that I've seen that downside of Wikipedia, as I use it for understanding things like this and never would've known that the diagram I was learning from was wrong. Thanks for commenting this, it's good to know
- tmalsburg2 3y agoI learned a lot about this from the book "The design of the Unix operating system" by Maurice J. Bach.¹ It's an old book and many details deviate from actual present-day Linux, but it nonetheless gives a great overview of the key components and ideas. ¹ https://books.google.de/books/about/The_Design_of_the_UNIX_Operating_System.html?id=BxZpQgAACAAJ&source=kp_book_description&redir_esc=y https://books.google.de/books/about/The_Design_of_the_UNIX_O...
- guerrilla 3y agoThis is one of my favorite books. A true classic. There are follow-ups in that style for Linux and FreeBSD as well. I think Robert Love wrote the former.
- temeya 3y agoAnd Marshall Kirk McKusick wrote the latter, "The Design and Implementation of the FreeBSD Operating System"
- guerrilla 3y agoThanks, pretty sure that's the one I meant.
- deaddodo 3y agoI used this book as a primary reference for OS design (along with The Dinosaur Book) when designing my hobby OS. The FreeBSD kernel/world is almost exquisite in it's engineering simplicity. Compared to the sometimes chaotic world of GNU/Linux (in my experience).
- bigfatfrock 3y agoThank you! I was going to ask for the latest linux variant - are you speaking of Love's "Linux Kernel Development", or "Linux in a Nutshell"? I've been a primary linux user for a couple decades now but I'm not too keen on digging into kernel hacking but love details like the OPs post.
- thesuperbigfrog 3y ago"The Linux Programming Interface" by Michael Kerrisk is one of the best technical resources I have found and used to understand Linux: https://man7.org/tlpi/ https://man7.org/tlpi/ Description from the book's website: "The Linux Programming Interface (TLPI) describes system programming on Linux and UNIX. TLPI is both a guide and reference book for system programming: If you are new to system programming, you can read TLPI linearly as an introductory guide: each chapter builds on concepts presented in earlier chapters, with forward references kept to a minimum. Most chapters conclude with a set of exercises intended to consolidate the reader's understanding of the topics covered in the chapter. If you are an experienced system programmer, TLPI provides a comprehensive reference that you can consult for details of nearly the entire Linux and UNIX (i.e., POSIX) system programming interface. To support this use, the book is thoroughly cross referenced and has an extensive index."
- begueradj 3y agoWhy a UML book is listed as a reference ?
- deleted 3y ago[deleted]
- knorker 3y agoWhenever I've made notes like this, it's never been useful to my future self nor to anyone else. The only use I've had of this kind of documentation is that the process of writing it, made me understand it better. Basically write-only documentation. I would call myself a Linux expert, and while I can kinda see what you mean with this diagram, it would not have been useful to me back before I was an expert.
- persolb 3y agoIt almost resembles mind mapping. It is a useful ‘process’ to figure out what you think/know. And it might be a pretty picture. But it isn’t very useful as documentation.
- falserum 3y agoI found it useful. Allowed me to compare if I have similar idea to the author.
- codelobe 3y agoUsually I would agree. I typically make a "Crash-Course in $PLATFORM" document while keeping notes. These I very commonly reference in order to externalize my memory since it seems to be approaching capacity. I don't care about Ruby on Rails, but once I did, and I can reference my notes if I ever need to touch that platform again.
- pjmlp 3y agoUNIX IPC is kind of missing, streams, pipes, message boxes, shared memory. SUN RPC for NFS, yellow pages,...
- peter_d_sherman 3y agoA future simple linux-like (or unix-like) OS -- could theoretically be created with only 4 syscalls: open() read() write() close() Such a theoretical linux-like or unix-like OS would assume quite literally that "everything is a file" -- including the ability to perform all other syscall/API calls/functions via special system files, probably located in /proc and/or /sys and/or other special directories, as other posters have previously alluded to... Also, these 4 syscalls could theoretically be combined into a single syscall -- something like (I'll use a Pascal-like syntax here because it will read easier): FileHandleOrResult = OpenOrReadOrWriteOrClose(FileHandle: Integer; Mode: Integer; pData: Pointer; pDataLen: Integer); if Mode = 1 then open(); if Mode = 2 then read(); if Mode = 3 then write(); if Mode = 4 then close(); FileHandle is the handle for the file IF we have one; that's for read() write() and close() -- for open() it could be -1, or whatever... Mode is the mode, as previously enumerated. pData is a pointer to a pre-allocated data buffer, the data to read or write, or the full filename when opening... (And of course, the OS could overwrite it with text strings of any error messages that occur... if errors should occur...) pDataLen is the size of that buffer in bytes. When the Mode is open(), pData contains a string of the path and file to open. When Mode is read(), pData is read to, that is, overwritten. When Mode is write(), pData is used to write from. All in all, pretty simple, pretty straightforward... A "one syscall Linux or Unix (or Linux-like or Unix-like) operating system", if you will... for simplicity and understanding! (Andrew Tannenbaum would be pleased!) Related: "One-instruction set computer" (OISC): https://en.wikipedia.org/wiki/One-instruction_set_computer https://en.wikipedia.org/wiki/One-instruction_set_computer
- richardwhiuk 3y agoThat's already kind of how syscalls work - you shove the syscall number in a register, and then call an interrupt.
- samatman 3y agoYou've effectively reinvented 9p here. Which is good! There are some differences which may interest you: https://9fans.github.io/plan9port/man/man9/intro.html https://9fans.github.io/plan9port/man/man9/intro.html I think you may find that some of the additional complexity of 9p is necessary, but perhaps not all of it.
- deleted 3y ago[deleted]
- xwowsersx 3y agoCould someone suggest hands-on resources for learning about kernels, such as a book or series on writing your own kernel? I'd like to gain a deeper understanding of their workings and I think hands-on or project based learning would help.
- hnthrowaway0328 3y agoosdev would help. It's not a book but a website.
- xwowsersx 3y agoTaking a look, it has a ton of resources. Thanks!