13 ms·

I'm just now starting to crack open some Lisp tutorials, coming from the OOP side of the fence, so please excuse the ignorance. For my learning purposes, would

I'm just now starting to crack open some Lisp tutorials, coming from the OOP side of the fence, so please excuse the ignorance. For my learning purposes, would you be open to explaining how symbolic expressions would circumvent the need for a file system?

derefr 8y ago
Instead of a file system (a system that consumes a block device and exposes a database mapping a hierarchy of tree-nodes with stringly-keyed names to seekable byte-buffers), picture something like Erlang’s DETS (a system that consumes a seekable byte-buffer and exposes a tuple store, where the tuples are regular in-memory objects that happen to live in an isolated heap which is memory-mapped from the file.) Now consider that anything that consumes a seekable byte-buffer as its backing store could just-as-well be modified to consume a block device as its backing store. Voila: your OS now has “durable memory” in place of a filesystem. (And, in fact, in modern systems you can skip the whole block-device layer and just sit your tuple store directly on top of NVMe.)
- nerdponx 8y ago
  consumes a block device and database mapping a hierarchy of tree-nodes with stringly-keyed names to seekable byte-buffers Sounds pretty much like a filesystem to me. Isn't this basically what Linux already does?
  - idle_zealot 8y ago
    OP is describing a file system in the text you quoted. So I should hope that it sounds like what Linux does now.
- skissane 8y ago
  I've often wondered, instead of a filesystem, why not a database? A filesystem is effectively a hierarchical model database with a very limited feature set (usually no transactions, minimal or no schema enforcement, very limited query language.) Why not add some of those features? Filesystem transactions: Windows supports them, but Microsoft has deprecated them; part of the reason, I think, is transactions required use of a different API to access files, which meant almost nobody used them; if they had been accessible through the same filesystem API as non-transactional operations, they might have seen more adoption. Schema enforcement: I could create a directory called "images", and then specify that all files in images have to be of MIME type image/*, and the FS will refuse to let me put a text file or executable in there. I could have a directory called "logs", and require all file names in that directory to have names containing a valid date/timestamp. I could enforce the rule that a .json file has to contain well-formed JSON, or a .xml file must contain well-formed XML. Querying: filesystem could support SQL queries over file extended attributes. (It doesn't have to be SQL, SQL syntax is pretty ugly.) "Find all files of MIME type image/png which are greater than 100KB in size?" "How many executable files are there? How much space do they consume?"
  - jcelerier 8y ago
    > I've often wondered, instead of a filesystem, why not a database? yeah, that's what a lot of people wondered in the 70s too and it failed spectacularly, and failed again spectacularly in the 2000s with microsoft's WinFS.
    - pjmlp 8y ago
      With that kind of remark I guess you never used a mainframe.
      - ori_b 8y ago
        In what way do mainframe file systems differ? As far as I'm aware, they largely use heirarchical file systems, like the confusingly named HFS or ZFS -- both of which are colliding acronyms from within IBM. HFS is the older Hierarchical File System, and ZFS being the newer z/Architecture File System. As far as I'm aware, the main difference is support for record oriented files, but the naming and lookup isn't so different.
        skissane 8y ago
        Well, I think even more than HFS or zFS, they use the MVS classic filesystem (datasets etc.) But that too is semi-hierarchical. (I say "semi" because to some extent the hierarchy is just a naming convention, but to some extent it's real – the initial qualifiers of a dataset name can be really hierarchical in that can select which catalog is used; and then of course PDS/PDSE members are an additional one-level hierarchy on the end.) But you are right, there is nothing especially database-oriented about file lookup and naming on IBM mainframes. Record-oriented files and key-sequenced VSAM files (and once upon a time ISAM too) are database-oriented features, but they relate to file contents not file naming/lookup/etc. I think the idea of catalogs is interesting, in that they permit a separation between the naming of datasets and the volumes they are stored upon. A dataset can be moved to another volume without changing the name used to access it. That is arguably more complex on Unix-like systems, since you need to muck around with symlinks or bind mounts to get the same effect.
frou_dh 8y ago
From the README of this specific project, it's clear that the line between where the language ends and the OS starts is blurred. If an OS is an exclusive environment for a single language, then that language's abstractions for handling data could be pushed much deeper than we're used to.
vbuwivbiu 8y ago
images
jonjacky 8y ago
Robert Strandh's LispOS proposal [1] has a single-level object store as an alternative to a conventional file system. See especially [2] and [3] [1] https://github.com/robert-strandh/LispOS https://github.com/robert-strandh/LispOS [2] https://github.com/robert-strandh/LispOS/blob/master/Documentation/chap-intro.tex https://github.com/robert-strandh/LispOS/blob/master/Documen... [3] https://github.com/robert-strandh/LispOS/blob/master/Documentation/chap-object-store.tex https://github.com/robert-strandh/LispOS/blob/master/Documen...

derefr 8y ago

Instead of a file system (a system that consumes a block device and exposes a database mapping a hierarchy of tree-nodes with stringly-keyed names to seekable byte-buffers), picture something like Erlang’s DETS (a system that consumes a seekable byte-buffer and exposes a tuple store, where the tuples are regular in-memory objects that happen to live in an isolated heap which is memory-mapped from the file.) Now consider that anything that consumes a seekable byte-buffer as its backing store could just-as-well be modified to consume a block device as its backing store. Voila: your OS now has “durable memory” in place of a filesystem. (And, in fact, in modern systems you can skip the whole block-device layer and just sit your tuple store directly on top of NVMe.)

nerdponx 8y ago
consumes a block device and database mapping a hierarchy of tree-nodes with stringly-keyed names to seekable byte-buffers Sounds pretty much like a filesystem to me. Isn't this basically what Linux already does?
- idle_zealot 8y ago
  OP is describing a file system in the text you quoted. So I should hope that it sounds like what Linux does now.
skissane 8y ago
I've often wondered, instead of a filesystem, why not a database? A filesystem is effectively a hierarchical model database with a very limited feature set (usually no transactions, minimal or no schema enforcement, very limited query language.) Why not add some of those features? Filesystem transactions: Windows supports them, but Microsoft has deprecated them; part of the reason, I think, is transactions required use of a different API to access files, which meant almost nobody used them; if they had been accessible through the same filesystem API as non-transactional operations, they might have seen more adoption. Schema enforcement: I could create a directory called "images", and then specify that all files in images have to be of MIME type image/*, and the FS will refuse to let me put a text file or executable in there. I could have a directory called "logs", and require all file names in that directory to have names containing a valid date/timestamp. I could enforce the rule that a .json file has to contain well-formed JSON, or a .xml file must contain well-formed XML. Querying: filesystem could support SQL queries over file extended attributes. (It doesn't have to be SQL, SQL syntax is pretty ugly.) "Find all files of MIME type image/png which are greater than 100KB in size?" "How many executable files are there? How much space do they consume?"
- jcelerier 8y ago
  > I've often wondered, instead of a filesystem, why not a database? yeah, that's what a lot of people wondered in the 70s too and it failed spectacularly, and failed again spectacularly in the 2000s with microsoft's WinFS.
  - pjmlp 8y ago
    With that kind of remark I guess you never used a mainframe.
    - ori_b 8y ago
      In what way do mainframe file systems differ? As far as I'm aware, they largely use heirarchical file systems, like the confusingly named HFS or ZFS -- both of which are colliding acronyms from within IBM. HFS is the older Hierarchical File System, and ZFS being the newer z/Architecture File System. As far as I'm aware, the main difference is support for record oriented files, but the naming and lookup isn't so different.
      - skissane 8y ago
        Well, I think even more than HFS or zFS, they use the MVS classic filesystem (datasets etc.) But that too is semi-hierarchical. (I say "semi" because to some extent the hierarchy is just a naming convention, but to some extent it's real – the initial qualifiers of a dataset name can be really hierarchical in that can select which catalog is used; and then of course PDS/PDSE members are an additional one-level hierarchy on the end.) But you are right, there is nothing especially database-oriented about file lookup and naming on IBM mainframes. Record-oriented files and key-sequenced VSAM files (and once upon a time ISAM too) are database-oriented features, but they relate to file contents not file naming/lookup/etc. I think the idea of catalogs is interesting, in that they permit a separation between the naming of datasets and the volumes they are stored upon. A dataset can be moved to another volume without changing the name used to access it. That is arguably more complex on Unix-like systems, since you need to muck around with symlinks or bind mounts to get the same effect.

frou_dh 8y ago

From the README of this specific project, it's clear that the line between where the language ends and the OS starts is blurred. If an OS is an exclusive environment for a single language, then that language's abstractions for handling data could be pushed much deeper than we're used to.

vbuwivbiu 8y ago

images

jonjacky 8y ago

Robert Strandh's LispOS proposal [1] has a single-level object store as an alternative to a conventional file system. See especially [2] and [3] [1] https://github.com/robert-strandh/LispOS https://github.com/robert-strandh/LispOS [2] https://github.com/robert-strandh/LispOS/blob/master/Documentation/chap-intro.tex https://github.com/robert-strandh/LispOS/blob/master/Documen... [3] https://github.com/robert-strandh/LispOS/blob/master/Documentation/chap-object-store.tex https://github.com/robert-strandh/LispOS/blob/master/Documen...