13 ms·
The operating system: should there be one? (2013) [pdf]
- Causality1 5y agoTitle needs the [PDF] tag.
- UnlockedSecrets 5y agoThe PDF: Should there be one?
- aitchnyu 5y agoThere is a HTML-bundle alternative to PDF in my utopia. User agents can render documents optimized for any screen instead of hardcoded two column A4 pages. Nouns and verbs are highlighted. https://rewordify.com/index.php https://rewordify.com/index.php
- deleted 5y ago[deleted]
- p2t2p 5y agoOn one had - yes please, screw OSes, gimme something like Smalltalk environment (in it's Pharo incarnation preferably). On other hand - define OS. Isn't that Smalltalk environment an OS if it runs bare metal?
- ModernMech 5y agoYes, there is a very blurry line between operating systems, programming languages, and we can include databases as well. Some operating systems as you point out are PLs. Some PLs are implemented as databases. They’re all just really complex accounting systems by nature. Look at the implementation of a lot of OSes, PLs and DBs they are really just made up of tables keeping track of relationships all the way down.
- yjftsjthsd-h 5y agoAlso VMs; the JVM is targeted by multiple languages as if it were an OS
- infogulch 5y agoI would be soooo happy if OSes were just absorbed by databases. The file system is just a heirarchal key value store: genius! Let's make it untyped so you have to parse everything manually just to interpret it: not so great. Yes I'm saying the Unix file system is a worse mongodb. Please at least give me a postgres or something.
- LAC-Tech 5y agoI enjoyed this post, but I'm struggling to wrap my head around what relations you'd have in this OS. Sym links become foreign keys?
- p2t2p 5y agoJust a record pointing to another ID? I mean no need to enforce FK constraint for symlink. For hard links - yes, an FK like concept?
- infogulch 5y agoIt's not about relations, it's about schemas and types. My complaint is that none of the schemas and types are communicated by the OS in an explicit, uniform, programmatic way. Everything in 'special' mounted filesystems like procfs, sys, cgroupsv2, etc as a start. These all use implicit schemas of particularly named files with particular contents that you have to learn about out-of-band to just to do anything. Requirement 1: Everything has an inspectable schema. All the configuration files with their own home-grown format that you can only validate by restarting the program that relies on it and hoping that it doesn't crash your system is for the birds. Requirement 2: Files that must have a particular format must also have a well-defined Type that can be validated to be correct in a uniform way outside the program that uses it.
- pjmlp 5y ago
- travisjungroth 5y ago(Only read the abstract). I’m learning Emacs and it’s making me think that we missed on the correct abstraction level of the operating system. There are programs that are OS-like (Emacs, browsers, Electron, anything that takes plug-ins). Maybe operating systems could have been lower level, and we could have developed more/better options on the pseudoOS level on top. This seems somewhat inline with the Smalltalk vision.
- pjmlp 5y agoWell, have a look at http://www.softwarepreservation.org/projects/LISP/interlisp_family/ http://www.softwarepreservation.org/projects/LISP/interlisp_... Then follow up with "The Interlisp Programming Environment" https://www.computer.org/csdl/magazine/co/1981/04/01667317/13rRUyv53Ic https://www.computer.org/csdl/magazine/co/1981/04/01667317/1... "Graphical Programming in Interlisp-D" https://www.youtube.com/watch?v=wlN0hHLZL8c https://www.youtube.com/watch?v=wlN0hHLZL8c "Eric Bier Demonstrates Cedar" https://www.youtube.com/watch?v=z_dt7NG38V4 https://www.youtube.com/watch?v=z_dt7NG38V4 Cedar was the very first graphical workstation for something like .NET and Java would later kind of try to become.
- LargoLasskhyfv 5y agoWhile that seems to be like almost lost history, and a forgotten art, it is not! Because [1] https://interlisp.org/ https://interlisp.org/ & [2] https://github.com/interlisp https://github.com/interlisp
- travisjungroth 5y agoThanks!
- ducktective 5y agoSo...It Has Come To This... https://xkcd.com/1022/ https://xkcd.com/1022/
- overgard 5y agoI can't help but think about what happened with graphics APIs - the older ones (DX9, OpenGL) held your hand more, and the new ones are super bare metal and not written for the faint of heart. I think most would agree this is better though; for an application developer you can simply use a much higher level library built on top, and for the library/engine developer you have a lot more power and control. The old APIs were in an awkward middle ground of not being particularly easy to use, but also not being nearly low level enough. Nobody was happy. I think modern OS stacks are in a similar boat, at least Windows and macOS. True low level control is kind of a pain, but the OS libraries are not particularly pleasant or high level enough either. So largely things like Electron or Qt are acting as the defacto OS anyway to varying degrees of success but undeniable levels of resource waste. I wonder if modern languages and tooling and such are advanced enough that the OS should become more primitive and minimal. In a way it feels like we're moving there anyway with web tech becoming the defacto user space and toolkit. That's fine, but god if it wasnt all built on 50 layer of cruft our machines would be screaming fast. Smalltalk, if you get a chance to use it, really does act like its own OS in so many ways. It's great as a programmers OS really, although it doesn't really hit the right notes for average use. Still though, if someone had the urge to build it I could easily see a smalltalk based userland built on like a linux kernel (with no gnu user space) being a totally viable thing.
- tehsauce 5y agovery interesting take, thanks for sharing
- pjmlp 5y agoNah, Android, macOS and Windows are doing just fine, it is the classical UNIX desktop that cannot get their act together. The existing issues aren't technical, rather political, like the WinRT crusade that ended up bombing and now we have plenty of GUI toolkits to chose from on Windows. Swift, Java, Kotlin and the .NET languages are more than high level enough.
- otabdeveloper4 5y ago> the classical UNIX desktop CDE? That hasn't existed for decades.
- mjevans 5y agoUnix generally has the correct basis for the design. I agree that a filesystem as we know it in unix-like systems is a set of objects (directories) and files. Would it not be preferable to retain this simple interface and instead conceive of interaction between programs and threads not as a stream of distinct objects in memory, but rather as a data tree within each program's control which can be exposed as input or output for other programs to map against and write into / consume?
- overgard 5y agoI think the problem with the unix model is that there isn't a rich enough array of types -- everything is just a stream of bytes and the closest thing you have to lists and records are strings with spaces and newlines. It was revolutionary in the 70s but compared to what you can do with something like powershell (which has its own issues) it's just not a rich or easy to use model anymore.
- mjevans 5y agoSimplicity is a good baseline. In Unix systems, off the cuff scripting works really well for live human interaction. Complexity, particularly NOT-self-documenting complexity, becomes a nightmare. Imagine instead that each program controls a special area under /proc/(pid)/ which behave like tmpfs to that program: named interfaces are subdirectories, messages are files. Reflection of supported interfaces becomes easy. It integrates with existing software, and it's self documenting for both software and users. Complexity can be opt-in rather than the only way to operate.
- vbezhenar 5y ago> Imagine instead that each program controls a special area under /proc/(pid)/ which behave like tmpfs to that program: named interfaces are subdirectories, messages are files. That's API. Users will depend on it. So now you either have to freeze your code or break users flows.
- comex 5y ago
- sedev 5y agoTitle should probably have a [2013] marker.
- vbezhenar 5y agoI would love more collaboration between operating systems. There should be more layers with standard interfaces. For example BIOS used to be a driver layer back in the day, providing simple API to underlying OS. This concept should be expanded. I don't see no reason for every OS to rewrite those drivers again and again. There should be some standard for common device APIs, like disks, keyboards, mouses, GPU and so on. I should be able to buy an Nvidia GPU, download and install UEFI driver and it should provide Vulkan API for any underlying operating system, Windows, Linux, OpenBSD, ReactOS. Another example is file systems. All major operating systems are very similar when it comes to file systems, at least from the user PoV. Yes, there are Windows ACLs, Linux permissions, but it should be possible to abstract those away enough. So Windows could install its latest Windows 11 NTFS driver into UEFI and Linux should be able to just use it to access NTFS safely, without any gigantic efforts to reverse-engineer and reimplement everything from scratch. At one point I thought about virtualization being an answer to those demands. These days you can write completely virtualized operating system which needs to implement virtblk, virtfs, virtio-* drivers and run it on any host operating system with perfect driver support from manufacturer. GPU is not there, but I don't think that it's impossible to implement. Of course running a full-blown ad-ridden Windows 11 just as a hypervisor seems like not the most elegant solution. Unification will not play nicely with new progressive features like BTRFS, though. That's something to think about as well.
- klodolph 5y ago“BIOS” is really just a bunch of drivers in ROM chips somewhere. Thing is, most devices you actually don’t need special drivers because they are so standardized. Mice and keyboards are USB HID. There are a handful of ways you typically interact with storage, like using SCSI. Surprisingly, a lot of storage devices are actually SCSI, in some form or another. One of the big problems is that drivers are going to closely interact with other operating system services, like memory allocators or the scheduler. The interface has gotten a lot more complicated over the years. And the semantics that operating systems present to the user are sometimes different in significant ways. Back in the BIOS days, what you could do through the BIOS was rapidly falling behind what you could do with a proper driver. To pick on the filesystem example, there are some fairly major differences between Windows and Linux/macOS/BSD filesystems APIs from the developer perspective. Windows has mandatory locks and files are identified by name. On Linux/macOS/BSD, there's more of a separation between a file and its name—files do not even need to have names at all, and locks are advisory. For this and other reasons, there are some boundaries between files you use in WSL and files you use in Windows. I’m just very skeptical here that you could introduce another layer between devices and the OS, that would be thinner and more direct than using a hypervisor, more flexible than just using a standard protocol (like USB HID) which your OS already supports, and work with multiple operating systems.
- waynecochran 5y agoI see the paper uses the term "postmodern" multiple times and references a paper "Notes on Postmodern Programming" which begins with a Manifesto. What does this term mean in this context? I don't like what I smell.
- injidup 5y agoPostmodern in terms of programming refers to that body of thought that posits a replacement of the rigid classist structures with a new functional hermeneutics based not simply on truth or falseness but on an interpretation of access defined as a trajectory of internal and external states across the pattern. Most new PDM (post modern development) is actuated in a bias free programming language called "Chomsky" which is a rigid left recursive grammar untainted by Boolean logic.
- raxxorrax 5y agoYes, I heard postmodernists say that everyone should stop boolean them.
- TheOtherHobbes 5y agoThis is now evolving into Green software, where everything is a tree.
- jonathanstrange 5y agoI know you're joking, but just to clarify for others: Nothing about Chomsky's work is postmodern. On the contrary, Chomsky is a well-known and fierce critic of postmodernism, both in his political work and in linguistics. He's been openly criticizing "postmodern" thinkers and authors for decades. See for instance [1]. Moreover, his approach in linguistics has frequently been criticized as being too "scientistic" because he re-oriented linguistics towards the natural sciences. After all, he's the one who brought formal, recursive grammar formalisms and grammar hierarchies to the field. [1] https://www.youtube.com/watch?v=772WncdxCSw&t=1s https://www.youtube.com/watch?v=772WncdxCSw&t=1s
- stephenrkell 5y ago
- perbu 5y agoI've had some experience with running applications without an operating system. First and foremost I like the simplicity. Instead of having state distributed between the kernel and the application all the state is in your application. No longer having this dance where an application and the kernel interacts there is only one place where state is kept. This can lead to some spectacular results for things like TCP. Having the ability to serialize a TCP connection and sending it over the network means you're able to transfer a TCP connection from one host to another, even with SSL on top. Now, this can be done with a POSIX kernel as well, as long as you move the networking from the kernel into the application. But if you're doing this you're eroding away the benefits of the kernel, basically turning your OS into a bootloader. Same thing with scheduling. Another benefit is that, at least on compiled languages, code that isn't used isn't there. So the linker can strip away everything that is never called. This still feels like utopia. The average development environment is pretty complex and ripping out and replacing all the layers below your language runtime is too much effort.
- jiriro 5y agoIs there a demo showing how the Plan 9 is similar to Smalltalk regarding the “meta” stuff (programmabilty, message passing, ..)?
- jacquesm 5y agoLet's start off with defining what an Operating System really is because depending on where you draw that line the outcome will be substantially different. Ignoring mainframes and other large installations for the moment, strictly from a 'personal computer' point of view: If you see the operating system as the task-schedular + IPC clearinghouse then the system is simple, well defined and it can be expected to perform in a deterministic and verifiably correct way (assuming working hardware). If you define the operating system as 'everything and the kitchen sink' including media players, browsers and so on then there is no way to make any guarantees and determinism is going to be impossible to achieve. Personally I prefer the leaner-and-meaner operating systems that have a larger ring of userland software built on top of them to create as small as possible a core on which the actual end-user applications are run. Those systems tend to find good middle ground between reliability, speed and security because the amount of surface area exposed is going to be the minimum at every level. Something like Windows is on one end of that scale, something like MacOS is somewhere in the middle. Linux can be stripped (or more accurately: it could be stripped) to something a lot slimmer than the default distros, and something like IRIX (no longer used in production as far as I know) was roughly where MacOS sat (custom hardware, custom OS tied closely to that hardware). An ideal OS that limits itself to scheduling and IPC is currently not on the market for desktop usage, and the arrival of the mobile device market has blurred many of the lines that we thought were drawn pretty clearly in the past. The 'lets move everything into the kernel' philosophy is as far as I'm concerned fatally broken and likely things will have to get a lot worse before they will finally get better, and it would not surprise me if everything that we do today will be discarded once that happens.
- psyc 5y agoWhere do you rate the part of the OS that makes 100,000 types, brands, and models of devices all look about the same to our apps? I always saw hardware compatibility and abstraction as the greatest value proposition, not only for the user but for the PC industry as a whole.
- jacquesm 5y agoHardware abstraction layer. Can be done in userspace, with the exception of a timer. You may need to add some interrupt arbitration logic.
- deleted 5y ago[deleted]
- amelius 5y agoRelated question: do we need device drivers?
- qayxc 5y agoThat's a definite yes. I do NOT want to go back to the times when we had to write and ship individual drivers for each graphics card, sound card and peripheral we wanted to support in our software. Could you imagine having to select the physical port, IRQ and DMA channel of your sound device for each app that wants to output sound? Because that's what had to be done in DOS and screens like this https://flaterco.com/kb/audio/PCI/DS-XG_IRQ_error.png https://flaterco.com/kb/audio/PCI/DS-XG_IRQ_error.png were commonplace. Good enough if you had well-defined devices like SoundBlaster(Pro) or GUS that you could auto-detect without crashing the PC, but a living nightmare with clones that didn't quite behave the same way. So unless you only plan to support a single set of well-defined hardware, device drivers are a must.
- OrvalWintermute 5y ago> I do NOT want to go back to the times when we had to write and ship individual drivers for each graphics card, sound card and peripheral we wanted to support in our software. This 1000x - I remember this time and it was quite painful >Could you imagine having to select the physical port, IRQ and DMA channel of your sound device for each app that wants to output sound? Because that's what had to be done in DOS and screens like this https://flaterco.com/kb/audio/PCI/DS-XG_IRQ_error.png https://flaterco.com/kb/audio/PCI/DS-XG_IRQ_error.png were commonplace. Good enough if you had well-defined devices like SoundBlaster(Pro) or GUS that you could auto-detect without crashing the PC, but a living nightmare with clones that didn't quite behave the same way. I also remember the application working one second, unrelated hardware change (1), and back to the drawing board all over again
- Koshkin 5y agoIn the (post)modern world, every device should have its driver built-in (along with a CPU to run it on). The communication protocol (the command language) should be standardized for each given class of devices. The protocol should support the ability to upload custom “shaders,” whenever this makes sense, to tweak or extend the default behavior.
- unixhero 5y agoWhen was this written? I cannot find any meta information in the paper itself.
- arnaudsm 5y agoPardon the biological analogy, but diversity has 2 main benefits : resistance to epidemics (malware) and faster evolution (innovation). I don't want to converge towards a single OS, but welcome standards and bridges between them.
- kaba0 5y agoContinuing on your analogy, I would argue we have less diversity due to the all-encompassing three major OSes. Similarly to how you don’t really get competitors to eucaryotes and prokaryotes, the initial investment (device drivers) to create an OS (organism) that is actually usable is so high that one basically can’t overcome it when others are available.
- qPM9l3XJrF 5y ago"an operating system is a collection of things that don’t fit inside a language; there shouldn’t be one" I've wondered the same thing about databases. Just like an OS, a database has to manage data on disk and figure out when to cache it. The files & folders metaphor is arguably a relic of pre-computer times--instead of searching through folders, it is easier to search and sort based on particular attributes, something that databases are well-suited to. I think it'd be interesting to start with a database and try to build an OS around it, see how far you could go.
- wsc981 5y agoBut humans generally like to organise things hierarchically and I would think database are not well suited for this purpose. That is, unless there exists some kind of tree structured database that I don't know about ... Many years ago Microsoft tried to implement some kind of database oriented file system (it was a big dream of Bill Gates IIRC and probably meant for Longhorn), but this project never became part of an official Windows release AFAIK. It'd be interesting to know why Microsoft abandoned this initiative.
- qayxc 5y ago> It'd be interesting to know why Microsoft abandoned this initiative. There were technical issues. They simply never managed to get the system performant enough to be use as a FS replacement. There also were tons of issues with the concept itself, like proper tooling for visualising the stored objects to the user and a compatibility layer for legacy software. It seems as if parts of it was merged into MS SQL server products [0]. [0] https://docs.microsoft.com/en-us/archive/blogs/winfs/ https://docs.microsoft.com/en-us/archive/blogs/winfs/
- monocasa 5y agoYeah, it was based on Jet, so if I had to guess they're probably just talking about including some perf improvements in the core database engine that they made profiling the WinFS use case.
- 5y ago
- ilaksh 5y agoOSes do have shared abstractions such as file systems and process input and output. I think the trick is to make the common abstractions richer in a way that enough people agree on and are willing to adopt. Most of the shared OS features are basically grandfathered in. So a big part of this is that this stuff is not really a rational choice usually. It's just the way it's always been. So upgrading the capabilities and creating new common abstractions in an OS is not that hard. But almost impossible to get the majority to go along with it. Browsers demonstrate an additive way of getting a lot of that. In that we now have essentially two highly compatible widely deployed VMs -- Firefox and Chrome. Which can be programmed in a number of languages that compile to JavaScript or web assembly and share a rich API. Web assembly modules seem relevant. https://github.com/WebAssembly/proposal-type-imports/blob/master/proposals/type-imports/Overview.md https://github.com/WebAssembly/proposal-type-imports/blob/ma...
- gatestone 5y ago“Programs = data structures + algorithms” If you have more than one program, they need shared data and access procedures, like the “user” and “the hardware”. Who manages them: The OS? Maybe you want to share a bit more like some common higher level functionality. That becomes part of the OS, de jure (resident kernel) or at least de facto (loadable kernel modules).
- Kalanos 5y agoi just want protocols between workflows of serverless services
- mikewarot 5y agoIf you have one job to do, and it doesn't involve other computers, you don't need an OS. MS-DOS or equivalent program loader/file system API will work just fine. Windows, MacOS, and Linux work fine in a stand alone environment (unless an APT {advanced persistent threat, like the NSA} is after you) On the other hand, if you're even thinking about touching the internet, or any other computer, you need to have every single task limited ahead of time to a specified set of resources, like mainframes do. Doing anything less is foolish, and I certainly wouldn't build a civilization on such folly. Yes, in an era of mobile code, persistent internet, you need an operating system that doesn't give any default permissions to any executing task, ever. It must also safely and securely multiplex access to the hardware resources, all of the hardware resources, in all of the subsystems in the SoC, etc. There are no Operating Systems, as of yet. I weep for the future.
- lloydatkinson 5y agoIsn't what you just described Fuschia?
- marcus_holmes 5y agoI've been playing with Unikernels for Go server deployment - so far so good. I'm not sure if this counts as "no OS" - as I understand it (and I'm not sure I do) the Unikernel core provides the calls that the Go runtime is expecting. But the central idea that the server system can only run one program makes a lot of sense to me. Obviously it won't work for a desktop, but then the whole idea that desktops and servers run the same OS doesn't make much sense to me.
- OrvalWintermute 5y agoI've been expecting the unikernel approach to catch on more too - although not the "no OS" it can be pretty stripped down compared to a regular OS. Now that unikernels are starting to gain more traction in the DOD I expect this will continue.
- perbu 5y agoI've worked with Unikernels for a couple of years. They are great, but they are very, very pure and this can be limiting. Say I have an web API written in Go. For some reason I need to do some openssl stuff before I start up my main program. In a container I can just wrap my program in a little shell script which generates/signs a cert before starting the API. Such a simple task. It is gonna be a lot harder with a Go Unikernel as I could have to bake this functionality into the application itself. This Swiss Army knife that is Unix often has it's uses.
- pjmlp 5y agoYou can have best of both worlds with container OSes like Bottlerocket. In any case, in you have a web API written in Java then you can connect advanced monitoring tools like JDK Flight Recorder and directly affect the system via JMX beans. So it is more a matter of tooling maturity than anything else.
- marcus_holmes 5y agoI get that, but I rarely need to do stuff like this - I generally use the awesome ACME library to handle certs. But yes, if I did need to do that then a Unikernel wouldn't be useful. But the reduction in attack surface of not having all those Swiss Army blades hanging about - and not having to configure them just right so I don't accidentally damage stuff - is a real benefit.
- PaulHoule 5y agoIBM never had a plan for a mainframe operating system that made sense -- given the idea that the "360" was supposed to encompass the whole circle of applications. What happened was some academics figured out it was possible to make virtual machines on the 370 and that you could run a few different operating systems at the same time. If you were doing software dev on the 370, for instance, you would start up a VM running VM/CMS which was a single-user OS a lot like CP/M or MS-DOS, work there, possibly using services offered by another OS in another VM by some kind of message passing. This turned out to be the "master plan" in the long term, since they could have a few different mainframe OS, even some applications that run on "bare metal", but have it all run on one machine or one cluster of machines and be managed under the same pane of glass.
- mikewarot 5y agoThe reason it worked out was that each virtual machine was given a set of capabilities, and the hypervisor restricted access for that machine to only those resources. Eventually, this lesson will spread. I think about 5 more years of chaos until people finally catch on.
- kaba0 5y agoWe sort of have that with cloud computing, don’t we?
- PaulHoule 5y agoYep. Also there are many x86 and other systems where people use VMWare or something like that for "server consolidation" without going all the way out to the cloud. The difference today though is that the guest operating system today is a "full-service" operating system such as Linux or Windows. In many of these cases the partitioning of the service could be accomplished through user id's, processes, sessions, containers, and similar facilities offered by the guest OS.
- slackfan 5y agoPer law of headlines: Nope.
- schiem 5y agoMaybe this is a hot take, but it seems like browsers have essentially become what the operating system should have been, and it's the reason that everyone crams things into web apps / Electron instead of building native applications. Browsers have a set of APIs that are decided on by a standards committee, and you have to implement those standards if you want your browser to be compliant. As an application developer, all I need to do is check a page to see how widely a standard has been adopted and then use it - I can be sure that my code can ship to (nearly) every device and behave uniformly, with minimal tweaking. If the operating system had gone that route, we wouldn't have had to build more layers on top of it to abstract away the differences.
- kazinator 5y agoAs soon as two or more separate applications share some common body of code that helps them deal with The Computer, you have an operating system. It doesn't matter whether that common body is a table of software traps that dispatch machine language routines, or whether it is Smalltalk objects. Some game on a MS-DOS floppy disk that boots itself from the boot sector and uses BIOS calls has an operating system. The set of requirements which describe the host environment (the "virtual machine" in the classic sense) that forms the backdrop against with an application is developed is the operating system. The only way you don't have an operating system is when the application itself does everything it needs with regard to the outside world by itself poking at I/O registers, and handling every interrupt and so on. Anything independently organized for that purpose, given something resembling an API, is an operating system. There is, or used to be, a sensible version of "there should not be an operating system" I first heard some decades ago. Namely this: "there should not have be an operating system with a hardware-protected user/kernel boundary". The idea is that there doesn't have to be kernel which the hardware conceals using privilege levels and fences. The idea is supported by the notion that all security can be handled at the software level. The system, as such, is somehow verified to be correct. The only way to execute code is via compiling it via the system's trusted compiler, which generates only safe instruction sequences. Those sequences can call functions in the system, and those functions sandbox what can be done. All notion of trust and privilege is software-level without any privileged instructions having to fall down a trap into a privileged kernel which meticulously validates arguments. (All that rhetoric needs to be somehow adjusted for 2021, which finds us jaded by knowledge of side channel attacks, cache attacks and whatnot.)
- fouc 5y agoI think that there should be a humane operating system that operates for the human. Operating systems shouldn't get too hung up on supporting applications or browsers, instead they should focus more on serving humans. Common operations that humans need should be standardized across the operating system.