7 ms·

Rob Pike: the origin of dotfiles

470 points by keyist 14y ago

keyist 14y ago
Instead of putting a dotfile or dotdir in the user's home directory, do follow the XDG Base Directory specification: http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html http://standards.freedesktop.org/basedir-spec/basedir-spec-l... . It's easy to understand and requires only a marginal increase in effort/code.
- oofabz 14y ago
  I am always pleasantly surprised when programs follow XDG. One notable example is the fish shell. Most shells put dotfiles in $HOME, like .bash_profile or .zshrc. Fish puts everything in .config/fish/.
- moe 14y ago
  I see no benefit. It's not at all easy to understand, nor easy to implement, nor does it have any tangible advantages. It's one of these superfluous pseudo-standards that do nothing but add needless clutter. But gladly nobody seems to be using it anyway, I see only one directory in my ~/.local: vlc.
  - subway 14y ago
    Check out your .config and .cache directories.
  - keyist 14y ago
    I consider it good practice to allow users to customize where your program creates files. I don't like $HOME to be the default dumping ground for anything I run. A standard that uses environment variables means programs don't have to provide extra options for this customization (I've seen -f,--file , -c,--config and other variants). It allows for common code (libraries that implement the spec). If you poke around for feature requests for various open source programs, you'll find XDG basedir compliance come up occasionally (moreso for CLI utils). I wouldn't say "nobody seems to be using it"; quick scan of my folders includes Chromium, uzbl, htop. Git's next release will be compliant too.
    - tedunangst 14y ago
      I don't particularly like dumping everything in home, but XDG is worse. Now, if I want to start over with a clean profile, there's a whole list of locations I have to zap. Spraying shit in three directories is worse than one.
      - johnny22 14y ago
        my problem with that "one" directory is that i have to take care specifically to back it up. applications that at least use ~/.cache means i don't have to worry about backing up a bunch of useless stuff.
        yaix 14y ago
        Isn't that what /tmp is for? Why not put cache files into /tmp/<username> with 0700 permission?
        daxelrod 14y ago
        No, that's not necessarily what /tmp is for. Consider a browser cache. /tmp is not guaranteed to be preserved between program invocations[1]. /var/tmp [2] might be a better place. /var/cache [3] is intended for exactly this kind of thing. Unfortunately, it can be far more useful for an administrator in a multi-user setup to want user-specific caches in home directories. That way, you get all of the infrastructure for managing user data for free, like quotas. Also note: if you're considering putting data into /tmp or /var/tmp, please honor the TMPDIR environment variable, if it is set[4]. [1]: http://www.pathname.com/fhs/pub/fhs-2.3.html#TMPTEMPORARYFILES http://www.pathname.com/fhs/pub/fhs-2.3.html#TMPTEMPORARYFIL... [2]: http://www.pathname.com/fhs/pub/fhs-2.3.html#VARTMPTEMPORARYFILESPRESERVEDBETWEE http://www.pathname.com/fhs/pub/fhs-2.3.html#VARTMPTEMPORARY... [3]: http://www.pathname.com/fhs/pub/fhs-2.3.html#VARCACHEAPPLICATIONCACHEDATA http://www.pathname.com/fhs/pub/fhs-2.3.html#VARCACHEAPPLICA... [4]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_03 http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...
    - anamax 14y ago
  - sciurus 14y ago
    ~/.local isn't primarily where you should look; check ~/.config
  - darkarmani 14y ago
    It's only used by those few users that use SSH.
    - tedunangst 14y ago
      ? ssh keeps its files in ~/.ssh.
    - jlgreco 14y ago
      [jlgreco@local] ~ % find ~/.config ~/.local | grep ssh | wc -l 0 So.. no.
  - aroman 14y ago
    Actually, it is in considerably wide use for desktop linux applications. It definitely has tangible benefits as well -- it promotes the clear separation of app data and user configuration, and unclutters the home folder. And as pointed out by other replies, ~/.local is not the only XDG data dir. I'm curious, how do you find it not easy to implement?
  - davvid 14y ago
    I see no benefit. How about consistency?
    - lmm 14y ago
      When there are forty years of existing applications using ~/.appname?
  - dfc 14y ago
    You are running a modern linux distro with only one file/dir in .local?
    - tammer 14y ago
      % cd ~/.local cd: no such file or directory: /home/tammer/.local % uname -s -r Linux 3.4.7-1-ARCH
    - nucleardog 14y ago
      FreeBSD 7.1-RELEASE - No ~/.local, ~/.config FreeBSD 9.0-RELEASE - No ~/.local, ~/.config Linux 3.2.0-23 - No ~/.local, ~/.config Linux 3.0.0-16 - No ~/.local, ~/.config Linux 3.0.0-12 - No ~/.local, ~/.config Honestly, I'd never even heard of this scheme before today.
      - dfc 14y ago
        I guess I should have said "a modern linux desktop environment." Are these shell boxes or desktops? In my experience the majority of propgrams that use this spec are gtk/qt programs. PS Listing a kernel version does not really provide any information about your modern linux distro.
  - darkestkhan 14y ago
    Interesting... I have ~/.cache, ~/.local and ~/.config and all 3 of them have quite a lot of subdirs and dotfiles inside.
  - gioele 14y ago
    > I see no benefit. The first benefit is that it removes clutters from your $HOME. The second benefit is that you can now manage and backup your settings in a sane way. * ~/.config contains config files (should not be lost, but if lost you can recreate them); * ~/.local contains user data files (save them often, never lose them for they are not replaceable); * ~/.cache contains cached information (can be tmpfs mounted, you can delete it any time you want, no loss in functionality, you just lose some optimization); * ~/.run is for temporary system files (must be tmpfs mounted or it must be cleaned during at shutdown or power on). Luckily most of the apps used on Linux systems now use it, you are probably using Mac OS X.
    - onli 14y ago
      > The first benefit is that it removes clutters from your $HOME. Invisible clutter? That's a strange concept. But the rest of your point indeed makes sense. It still is easier and probably comment to backup the whole $HOME. But those points you can see as a benefit, though not obvious.
      - deleted 14y ago
        [deleted]
      - lubutu 14y ago
        As Rob mentions in his post, the more dotfiles there are in $HOME the slower path resolution for any subfiles becomes. How do we navigate to ./src? We open the directory and read all the entries until we find the one called "src". What happens if we encounter a morass of dotfiles beforehand? src takes a while to find. The clutter may be invisible to you, but it does gum up the works.
        duaneb 14y ago
        Unless you have many thousands of files, I can't imagine you would ever notice a slowdown.
        kbolino 14y ago
        For what it's worth, most modern file systems (JFS, XFS, ext4, reiserfs, btrfs, ...) have logarithmic (or better) directory lookup times. This is achieved using hashes and b-trees (or even hash tables).
  - vrotaru 14y ago
    find .local/ | wc -l 618 So I guess it depends on your Linux flavor. Mine is Ubuntu 12.04.
- malkia 14y ago
  Somehow .dotfiles work just fine under Windows :). As such they are somehow more "portable" Emacs saves them under %USERPROFILE% - I have in there - .alice, .android, .easyhg, .eclipse, .gstreamer-0.10, .lighttable, .m2, .matplotlib, .... .VirtualBox, .zenmap Also in "Application Data" - .emacs.d, .mc, .subversion My point is - this system works somehow even under non-unixy systems. Because it's simple.
  - Dylan16807 14y ago
    %APPDATA% is as simple as home but much better on the clutter front.
    - DeepDuh 14y ago
      True. I actually kinda like the Windows directory structure since Vista. If only the application devs would also follow the standard (hence the whole My Documents mess).
  - strictfp 14y ago
    It didn't use to work. I think it was added for XP SP2 or so, probably due to the popularity of manu *nix ports like Emacs.
    - pjmlp 14y ago
      It always did, at least in NTFS. What did not, and still does not, is to create them via Windows Explorer. You can create them just fine from the command line, or via Windows APIs.
      - eropple 14y ago
        You can create them in Windows Explorer with a little trick: type the name with a trailing dot. Entering ".vimrc." will save as ".vimrc", no trailing dot.
        pjmlp 14y ago
- joe_the_user 14y ago
  I notice the $XDG_DATA_HOME environment variable isn't set in my Intrepid install. Maybe its set in later Ubuntu version. But the value not being set for me means implies to me that it not being set is pretty common. Thus it seems like supporting this standard is going to involve supporting a fall-back of whatever you would do otherwise. So I don't see "easy" at all but rather extra BS. Sorry.
  - angusgr 14y ago
    From the spec: If $XDG_DATA_HOME is either not set or empty, a default equal to $HOME/.local/share should be used. Similarly there's a default (.config/) defined for XDG_CONFIG_HOME. So the fallback is well-defined rather than "whatever you would do otherwise". Also, for my 2c, you should consider updating your Intrepid install if you at all can. It hasn't been supported for over two years, so it hasn't seen any security updates in that time. The Ubuntu do-release-upgrade system is pretty easy and reliable.
    - ricardobeat 14y ago
      $HOME/.local/share doesn't exist on OSX either.
      - masklinn 14y ago
        If it does not exist, you've not used a soft using it. I have one.
        ricardobeat 14y ago
        Not much of a standard then - I have done hundreds of installs.
      - bkor 14y ago
        XDG is a freedesktop standard. Meaning: checking that standard on OS X is a bit strange.
- jeltz 14y ago
  I would have preferred if it did not rely on environment variables since almost nobody sets them which means there will be untested code in applications implementing the specification.
  - takluyver 14y ago
    Where possible, applications should rely on tested libraries to implement the specification, like PyXDG.
- yaix 14y ago
  On my system (Xubuntu), these environment variables are empty. Seems not to be much of a standard.
  - saljam 14y ago
    To be fair, that's how it works. They have default values when they're not set. These are for when you want to override them, e.g. to keep cache files local instead of over your net mounted home directory.
tyrmored 14y ago
This is a problem that libetc is supposed to help solve: http://ordiluc.net/fs/libetc/ http://ordiluc.net/fs/libetc/
- malkia 14y ago
  It also begs the question, why it was named "etc" :)
  - napum 14y ago
    I'm guessing it's because /etc contains configuration files and dot-files are just configuration files. Or do you mean, why was /etc named etc?
    - deno 14y ago
      I think they meant the latter. Why is it /etc instead of, perhaps more obvious, /config or /settings?
      - malkia 14y ago
        If it was meant for configuration - probably it would've been - /cfg /ini /set /opt /flg /arg /prm (params) As someone said - naming things is the hardest!
    - petercooper 14y ago
      For the latter, a lazy copy/paste from Wikipedia: "There has been controversy over the meaning of the name itself. In early versions of the UNIX Implementation Document from Bell labs, /etc is referred to as the etcetera directory as this directory historically held everything that did not belong elsewhere (however, the FHS restricts /etc to static configuration files and may not contain binaries)."
  - spitfire 14y ago
    You can ask this question for most of unix. Why /etc? Why /bin and /usr/bin? (Answer: At one time hard disks were very small and crashed a lot), why do we presume screens are black and white, etc, etc. Try to change any of it though, and a lot of luddites will come out screaming bloody murder. It's just not UNIX if it makes sense.
skeletonjelly 14y ago
If not dot files lazy programmers would have just found another way. Just look at the results of lazy programmers on any Android SD card.
- voltagex_ 14y ago
  Is there a preferred standard for using the SD/external storage on Android?
  - skeletonjelly 14y ago
    http://www.reddit.com/r/Android/comments/t3x0p/randroid_lets_start_project_clean_sd/c4jc41o http://www.reddit.com/r/Android/comments/t3x0p/randroid_lets... > I do find it odd that so many apps have this problem. It's trivially easy to get the proper location. Just call getExternalFilesDir() . Deletion at uninstall happens automatically. In fact, it's the ONLY way to make sure those files are deleted when you uninstall, because you can't run code at uninstallation.
    - barrkel 14y ago
      Probably because it came in fairly late (Froyo, I think).
- chj 14y ago
  Yay, I am one of them.
seanalltogether 14y ago
For those who object that dot files serve a purpose, I don't dispute that but counter that it's the files that serve the purpose, not the convention for their names. I would like to hear a good argument for why hidden files and folders are a good thing.
- frossie 14y ago
  They are a natural segregator of novice and advanced users. If you have a limited level of interaction with the unix shell (remember than in the old days everyone in a science academic department used a Unix machine, even the dusty professors you kept in the back of the supplies cabinet), setting up their account and then making sure that they couldnt get into any trouble because they didn't know about 'ls -a' was very much a feature, trust me. Moreover I personally much prefer them to global configuration directories because they are always local to the thing being configured and you can always override a global configuration by using them. In fact I think they are a very elegant way to handle "hidden options" - stuff you want to expose to the power users but not bother newbies with. tl;dr: I am not convinced they are a misfeature.
  - seanalltogether 14y ago
    I wonder if that is a good solution for separating novices and pros though? For me, hidden files and folders only help to reinforce the confusion people have about computers. It's one thing to provide a folder to a user with a lock or shield on it, it's another to completely hide it from them.
    - GauntletWizard 14y ago
      Negative. A lock or shield will convince a user that they're not to touch it, ever... Unless they are too dumb to understand the connotations, in which case they'll simply break it anyway. Hidden files are a good way to go - Out of sight, out of mind for most people, but not impossible to show them when they do need to touch.
    - brianpan 14y ago
dsrguru 14y ago
That's really interesting. I always assumed hiding dotfiles was a deliberate convention, but to semi-quote one of the commenters, Rob's got a point. Or two.
- adambratt 14y ago
  Heh, I remember when I first found out about dotfiles. I thought it was a genius idea... Little did I know.
emperorcezar 14y ago
If someone thinks hidden files are a misstep, then I don't want them designing a OS. Grandma really doesn't care about some conf file, or anything like that. She cares about the pictures of her grandkids. The opinion that there shouldn't be hidden files comes from a perspective of someone who is a "power user" and who can't step back and see that most users really don't care for some .config file. To them it's clutter that gets in the way of finding what they really care about. That said, dot files maybe the wrong way to do it. I like ~/Library in OSX. That's one good way. Edit: Note that I'm talking about a general trend in the post's comments and on here. Not the author's opinion.
- UK-AL 14y ago
  Did you read his post? He said exactly that. Put them in a config directory of some sort.
  - emperorcezar 14y ago
    Yes. I'm not talking about his post. I'm talking about a few other comments here and on G+.
- SCdF 14y ago
  Sounds like you agree with him, as he advocates a 'conf' directory at the end of his post.
rehack 14y ago
I think the case of hidden dot files is a good example of 'convention over configuration'[1]. Point is accepted, that it came into being due to a lazy programmer. But surely early people might have just liked the unintended consequence of some files (dot files) being hidden. Just like most of us, whenever we learnt unix thought that it is by design. If early users, had found the consequence a handicap, it would have been fixed long back. Its similar to the use of hash-tags on twitter or the @for addressing which got adopted by users first and so became features (although the paths to them being considered features are different). [1] http://en.wikipedia.org/wiki/Convention_over_configuration http://en.wikipedia.org/wiki/Convention_over_configuration Edit: Grammar
ilaksh 14y ago
The main takeaway I think is that the reason we are doing things the way we do them is because that is how we have been doing them, not because of some kind of genius design. And actually the whole thing can start out as a mistake, and then later on basically becomes a religion. Its very funny actually.
- mturmon 14y ago
  Yes, very organic. The original programming shortcut accidentally created a sort of ecological niche, in which all sorts of things started to take root. Now, nobody can bulldoze the lot because there's too much stuff growing there.
eliben 14y ago
I really like it that G+ is becoming a "lightweight blogging" platform. There are too many of these around, and folding them into a "social network" seems like a good idea. I wish they would add more features that would make this easier, though, since in general I think it's in everyone's interest and will pull more traffic to the site.
- 89a 14y ago
  I'd like it if I didn't need a Google account to read anything on it.
  - jkn 14y ago
    Well be happy then, you don't need a Google account, as can be seen by entering the url in an Incognito window.
    - duaneb 14y ago
      The problem arises if you are signed into google elsewhere but not on G+.
  - djeikyb 14y ago
    Yeah, it's pretty lame that if I'm signed into Google but don't want to accept G+'s real name policy, I can't read otherwise public posts.
- lmm 14y ago
  Adding a new one is not a good solution to "there are too many around". And I don't think consolidating everything into google is a good idea; these kind of posts belong on a different network to photos of my holidays.
- saljam 14y ago
  A bit off topic, but is it just me or is the quality of comments on public G+ posts is absolutely terrible? Within a few hours there has been more than 100 replies. Most are inane, rude, of the useless “great post!” type, or just plain old spam. I could only manage to glean about 8 or 9 insightful replies. Is this something intrinsic to G+? Is it a function of the author's popularity? The discussions here are normally much more sensible, but I would expect HN's readership to have a fairly similar demographic to that of Rob Pike's G+ subscribers.
caf 14y ago
The fact that "." and ".." were allocated actual directory entries and were returned when reading the directory, rather than just being handled by the kernel when parsing pathnames seems like the original sin of expediency here.
- jlgreco 14y ago
  Why? Seems like a brilliantly simple quick solution that would be rather easy to roll back in the future and lacks any real downsides besides being a tad weird.
  - caf 14y ago
    Because it was creating real directory entries that caused "." and ".." to be visible to userspace programs reading a directory, which then led to the hack in "ls" to hide them, which is where the article picks up. It also isn't that easy to roll back once userspace programs start to rely on it - for example, the assumption that the number of files in a directory is equal to st_nlink - 2 is now so widespread that it's part of the UNIX API.
    - jlgreco 14y ago
      shrug, it seems to have all worked out pretty well.
      - strictfp 14y ago
        But all programs must make exceptions for these entries. Imagine trying to add a third magic filename, '...', for the grandparent for example. You would have a lot of coding to do. Not really possible. It would have been better to mark the dirs as special somehow rather than having each program have the convention programmed in.
        caf 14y ago
        ...and of course even the kernel had to have exceptions for them anyway - it knew they were special because it had to stop userspace from unlink()ing or rename()ing them.
mseepgood 14y ago
He also writes why the dd command is so horrible: "dd is horrible on purpose. It's a joke about OS/360 JCL. But today it's an internationally standardized joke. I guess that says it all." http://en.wikipedia.org/wiki/Job_Control_Language http://en.wikipedia.org/wiki/Job_Control_Language
acabal 14y ago
I hate having dotfiles in ~/ for the same reason why I hate "My Documents" in Windows: because it's supposed to be my space that I organize, not a generic dumping ground for your config files, brand-named folders, or other nonessential garbage. I want my space to be mine. Keep your app's stuff out of there!
- chris_j 14y ago
  Where would you choose to put application config instead?
  - acabal 14y ago
    Literally anywhere but my personal space. Why not have /etc/<username>/? I could even live with a single ~/.config/ but very few apps use that.
    - moe 14y ago
      It's actually quite useful to have everything that belongs to a user under one directory. The canonical example is the backup. There's a strong case to be made for "tar czf /tmp/backup.tgz ~". Do you really want your backups to become as complicated as they are on OSX and Windows? Likewise, being able to mount home-directories from remote servers, and being able to easily delete/move/quota users are highly desirable features in multi-user systems.
- Foy 14y ago
  On Windows I've retreated to Desktop which remains fairly sane as long as I opt-out or remove shortcuts being created after an install. "My Documents" is a nightmare... I think only one folder (out of nearly a dozen) and only one file are actually of my making.
- zapu 14y ago
  Microsoft now encourages to use AppData/ which is much better.
- gnufs 14y ago
garfee 14y ago
omg!
89a 14y ago
Can't read without signing into Google… oh well nothing of value was lost.
- mproud 14y ago
  I'm with you.
antirez 14y ago
dotfiles are not perfect, but to have this very negative vision on a feature that also helped is a bit a revisionist attempt IMHO. Dotfiles provided a poor, but at least simple way to store program-specific-user-specific configuration, since another standard was missing. After all it's a simple and decentralized system that worked very well with the concept of unix user and ACL: you write something inside your home directory, and this changes the behavior of your program. Consider that this was invented many decades ago. Now it seems a lot better to have directories with sub directories. Maybe back then it was considered to be a waste of resources, inodes, and so forth. We can improve it, create a new standard, and have something better than dot files, but dot files are better than many other over-engineered solutions that I can imagine coming out of some kind of design commission to substitute them. Every time to passed your vim configuration to a friend you just copied a text file, sending it via email: you enjoyed one of the good points about dot files. Every time you did something like cat dotfile | grep option you enjoyed the positive effects of single-file plaintext configuration. Also it's worth saying that dot files are not just the concept of an hidden file with config inside. A lot of dot files also have a common simple format of multiple lines "<option> <value>", that's better than some XML or other hard to type format (IMHO JSON itself is not good for humans).
- ajanuary 14y ago
  How does any of those advantages apply only to dotfiles and not just plain files?
  - FuzzyDunlop 14y ago
    I think there's an aspect of convenience, too. Personally, I prefer that to, say, configuration stored in some unspecified location in ~/Library/Application Support/Application/* (on OS X).
leothekim 14y ago
"How many bugs and wasted CPU cycles and instances of human frustration (not to mention bad design) have resulted from that one small shortcut about 40 years ago?" Sigh, if only most of us had worked on a software system that has lasted as long as that.
specialist 14y ago
Tangent: I'm grumpy that Eclipse IDE uses the file names .project and .classpath. So they're hidden by default. Requiring special treatment. Their content is XML. What's wrong with project.xml and classpath.xml?
mcgwiz 14y ago
The author's gripe seems to be that the hiding of dotfiles was unintended, ergo dotfiles are Bad. Whether they were intended is irrelevant; their wide usage vindicates the practice. After all, traction = value. The problem of program state/configuration/metadata storage is adequately met by dotfiles. There are, no doubt, numerous unintended behaviors of programs. Most of these are simply ignored and certainly not leveraged the way the dot behavior is. People don't go out of their way to abuse an unintended system behavior; they simply leverage all capabilities of a system ("intended" or not) to meet their needs. Had dotfiles not gained traction, some other solution would have been designed (or "engineered") to meet the needs of program state/configuration/metadata storage. [Tangent: All of this reminds me of grammar freaks that harp on "correct" usage, completely oblivious to the fact that grammar changes, and "correct" is merely a lightweight pointer to the current norm.]

470 points by keyist 14y ago

Instead of putting a dotfile or dotdir in the user's home directory, do follow the XDG Base Directory specification: http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html http://standards.freedesktop.org/basedir-spec/basedir-spec-l... . It's easy to understand and requires only a marginal increase in effort/code.

oofabz 14y ago
I am always pleasantly surprised when programs follow XDG. One notable example is the fish shell. Most shells put dotfiles in $HOME, like .bash_profile or .zshrc. Fish puts everything in .config/fish/.
moe 14y ago
I see no benefit. It's not at all easy to understand, nor easy to implement, nor does it have any tangible advantages. It's one of these superfluous pseudo-standards that do nothing but add needless clutter. But gladly nobody seems to be using it anyway, I see only one directory in my ~/.local: vlc.
- subway 14y ago
  Check out your .config and .cache directories.
- keyist 14y ago
  I consider it good practice to allow users to customize where your program creates files. I don't like $HOME to be the default dumping ground for anything I run. A standard that uses environment variables means programs don't have to provide extra options for this customization (I've seen -f,--file , -c,--config and other variants). It allows for common code (libraries that implement the spec). If you poke around for feature requests for various open source programs, you'll find XDG basedir compliance come up occasionally (moreso for CLI utils). I wouldn't say "nobody seems to be using it"; quick scan of my folders includes Chromium, uzbl, htop. Git's next release will be compliant too.
  - tedunangst 14y ago
    I don't particularly like dumping everything in home, but XDG is worse. Now, if I want to start over with a clean profile, there's a whole list of locations I have to zap. Spraying shit in three directories is worse than one.
    - johnny22 14y ago
      my problem with that "one" directory is that i have to take care specifically to back it up. applications that at least use ~/.cache means i don't have to worry about backing up a bunch of useless stuff.
      - yaix 14y ago
        Isn't that what /tmp is for? Why not put cache files into /tmp/<username> with 0700 permission?
        daxelrod 14y ago
        No, that's not necessarily what /tmp is for. Consider a browser cache. /tmp is not guaranteed to be preserved between program invocations[1]. /var/tmp [2] might be a better place. /var/cache [3] is intended for exactly this kind of thing. Unfortunately, it can be far more useful for an administrator in a multi-user setup to want user-specific caches in home directories. That way, you get all of the infrastructure for managing user data for free, like quotas. Also note: if you're considering putting data into /tmp or /var/tmp, please honor the TMPDIR environment variable, if it is set[4]. [1]: http://www.pathname.com/fhs/pub/fhs-2.3.html#TMPTEMPORARYFILES http://www.pathname.com/fhs/pub/fhs-2.3.html#TMPTEMPORARYFIL... [2]: http://www.pathname.com/fhs/pub/fhs-2.3.html#VARTMPTEMPORARYFILESPRESERVEDBETWEE http://www.pathname.com/fhs/pub/fhs-2.3.html#VARTMPTEMPORARY... [3]: http://www.pathname.com/fhs/pub/fhs-2.3.html#VARCACHEAPPLICATIONCACHEDATA http://www.pathname.com/fhs/pub/fhs-2.3.html#VARCACHEAPPLICA... [4]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_03 http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_...
  - anamax 14y ago
- sciurus 14y ago
  ~/.local isn't primarily where you should look; check ~/.config
- darkarmani 14y ago
  It's only used by those few users that use SSH.
  - tedunangst 14y ago
    ? ssh keeps its files in ~/.ssh.
  - jlgreco 14y ago
    [jlgreco@local] ~ % find ~/.config ~/.local | grep ssh | wc -l 0 So.. no.
- aroman 14y ago
  Actually, it is in considerably wide use for desktop linux applications. It definitely has tangible benefits as well -- it promotes the clear separation of app data and user configuration, and unclutters the home folder. And as pointed out by other replies, ~/.local is not the only XDG data dir. I'm curious, how do you find it not easy to implement?
- davvid 14y ago
  I see no benefit. How about consistency?
  - lmm 14y ago
    When there are forty years of existing applications using ~/.appname?
- dfc 14y ago
  You are running a modern linux distro with only one file/dir in .local?
  - tammer 14y ago
    % cd ~/.local cd: no such file or directory: /home/tammer/.local % uname -s -r Linux 3.4.7-1-ARCH
  - nucleardog 14y ago
    FreeBSD 7.1-RELEASE - No ~/.local, ~/.config FreeBSD 9.0-RELEASE - No ~/.local, ~/.config Linux 3.2.0-23 - No ~/.local, ~/.config Linux 3.0.0-16 - No ~/.local, ~/.config Linux 3.0.0-12 - No ~/.local, ~/.config Honestly, I'd never even heard of this scheme before today.
    - dfc 14y ago
      I guess I should have said "a modern linux desktop environment." Are these shell boxes or desktops? In my experience the majority of propgrams that use this spec are gtk/qt programs. PS Listing a kernel version does not really provide any information about your modern linux distro.
- darkestkhan 14y ago
  Interesting... I have ~/.cache, ~/.local and ~/.config and all 3 of them have quite a lot of subdirs and dotfiles inside.
- gioele 14y ago
  > I see no benefit. The first benefit is that it removes clutters from your $HOME. The second benefit is that you can now manage and backup your settings in a sane way. * ~/.config contains config files (should not be lost, but if lost you can recreate them); * ~/.local contains user data files (save them often, never lose them for they are not replaceable); * ~/.cache contains cached information (can be tmpfs mounted, you can delete it any time you want, no loss in functionality, you just lose some optimization); * ~/.run is for temporary system files (must be tmpfs mounted or it must be cleaned during at shutdown or power on). Luckily most of the apps used on Linux systems now use it, you are probably using Mac OS X.
  - onli 14y ago
    > The first benefit is that it removes clutters from your $HOME. Invisible clutter? That's a strange concept. But the rest of your point indeed makes sense. It still is easier and probably comment to backup the whole $HOME. But those points you can see as a benefit, though not obvious.
    - deleted 14y ago
      [deleted]
    - lubutu 14y ago
      As Rob mentions in his post, the more dotfiles there are in $HOME the slower path resolution for any subfiles becomes. How do we navigate to ./src? We open the directory and read all the entries until we find the one called "src". What happens if we encounter a morass of dotfiles beforehand? src takes a while to find. The clutter may be invisible to you, but it does gum up the works.
      - duaneb 14y ago
        Unless you have many thousands of files, I can't imagine you would ever notice a slowdown.
      - kbolino 14y ago
        For what it's worth, most modern file systems (JFS, XFS, ext4, reiserfs, btrfs, ...) have logarithmic (or better) directory lookup times. This is achieved using hashes and b-trees (or even hash tables).
- vrotaru 14y ago
  find .local/ | wc -l 618 So I guess it depends on your Linux flavor. Mine is Ubuntu 12.04.
malkia 14y ago
Somehow .dotfiles work just fine under Windows :). As such they are somehow more "portable" Emacs saves them under %USERPROFILE% - I have in there - .alice, .android, .easyhg, .eclipse, .gstreamer-0.10, .lighttable, .m2, .matplotlib, .... .VirtualBox, .zenmap Also in "Application Data" - .emacs.d, .mc, .subversion My point is - this system works somehow even under non-unixy systems. Because it's simple.
- Dylan16807 14y ago
  %APPDATA% is as simple as home but much better on the clutter front.
  - DeepDuh 14y ago
    True. I actually kinda like the Windows directory structure since Vista. If only the application devs would also follow the standard (hence the whole My Documents mess).
- strictfp 14y ago
  It didn't use to work. I think it was added for XP SP2 or so, probably due to the popularity of manu *nix ports like Emacs.
  - pjmlp 14y ago
    It always did, at least in NTFS. What did not, and still does not, is to create them via Windows Explorer. You can create them just fine from the command line, or via Windows APIs.
    - eropple 14y ago
      You can create them in Windows Explorer with a little trick: type the name with a trailing dot. Entering ".vimrc." will save as ".vimrc", no trailing dot.
      - pjmlp 14y ago
joe_the_user 14y ago
I notice the $XDG_DATA_HOME environment variable isn't set in my Intrepid install. Maybe its set in later Ubuntu version. But the value not being set for me means implies to me that it not being set is pretty common. Thus it seems like supporting this standard is going to involve supporting a fall-back of whatever you would do otherwise. So I don't see "easy" at all but rather extra BS. Sorry.
- angusgr 14y ago
  From the spec: If $XDG_DATA_HOME is either not set or empty, a default equal to $HOME/.local/share should be used. Similarly there's a default (.config/) defined for XDG_CONFIG_HOME. So the fallback is well-defined rather than "whatever you would do otherwise". Also, for my 2c, you should consider updating your Intrepid install if you at all can. It hasn't been supported for over two years, so it hasn't seen any security updates in that time. The Ubuntu do-release-upgrade system is pretty easy and reliable.
  - ricardobeat 14y ago
    $HOME/.local/share doesn't exist on OSX either.
    - masklinn 14y ago
      If it does not exist, you've not used a soft using it. I have one.
      - ricardobeat 14y ago
        Not much of a standard then - I have done hundreds of installs.
    - bkor 14y ago
      XDG is a freedesktop standard. Meaning: checking that standard on OS X is a bit strange.
jeltz 14y ago
I would have preferred if it did not rely on environment variables since almost nobody sets them which means there will be untested code in applications implementing the specification.
- takluyver 14y ago
  Where possible, applications should rely on tested libraries to implement the specification, like PyXDG.
yaix 14y ago
On my system (Xubuntu), these environment variables are empty. Seems not to be much of a standard.
- saljam 14y ago
  To be fair, that's how it works. They have default values when they're not set. These are for when you want to override them, e.g. to keep cache files local instead of over your net mounted home directory.

tyrmored 14y ago

This is a problem that libetc is supposed to help solve: http://ordiluc.net/fs/libetc/ http://ordiluc.net/fs/libetc/

malkia 14y ago
It also begs the question, why it was named "etc" :)
- napum 14y ago
  I'm guessing it's because /etc contains configuration files and dot-files are just configuration files. Or do you mean, why was /etc named etc?
  - deno 14y ago
    I think they meant the latter. Why is it /etc instead of, perhaps more obvious, /config or /settings?
    - malkia 14y ago
      If it was meant for configuration - probably it would've been - /cfg /ini /set /opt /flg /arg /prm (params) As someone said - naming things is the hardest!
  - petercooper 14y ago
    For the latter, a lazy copy/paste from Wikipedia: "There has been controversy over the meaning of the name itself. In early versions of the UNIX Implementation Document from Bell labs, /etc is referred to as the etcetera directory as this directory historically held everything that did not belong elsewhere (however, the FHS restricts /etc to static configuration files and may not contain binaries)."
- spitfire 14y ago
  You can ask this question for most of unix. Why /etc? Why /bin and /usr/bin? (Answer: At one time hard disks were very small and crashed a lot), why do we presume screens are black and white, etc, etc. Try to change any of it though, and a lot of luddites will come out screaming bloody murder. It's just not UNIX if it makes sense.

skeletonjelly 14y ago

If not dot files lazy programmers would have just found another way. Just look at the results of lazy programmers on any Android SD card.

voltagex_ 14y ago
Is there a preferred standard for using the SD/external storage on Android?
- skeletonjelly 14y ago
  http://www.reddit.com/r/Android/comments/t3x0p/randroid_lets_start_project_clean_sd/c4jc41o http://www.reddit.com/r/Android/comments/t3x0p/randroid_lets... > I do find it odd that so many apps have this problem. It's trivially easy to get the proper location. Just call getExternalFilesDir() . Deletion at uninstall happens automatically. In fact, it's the ONLY way to make sure those files are deleted when you uninstall, because you can't run code at uninstallation.
  - barrkel 14y ago
    Probably because it came in fairly late (Froyo, I think).
chj 14y ago
Yay, I am one of them.

seanalltogether 14y ago

For those who object that dot files serve a purpose, I don't dispute that but counter that it's the files that serve the purpose, not the convention for their names. I would like to hear a good argument for why hidden files and folders are a good thing.

frossie 14y ago
They are a natural segregator of novice and advanced users. If you have a limited level of interaction with the unix shell (remember than in the old days everyone in a science academic department used a Unix machine, even the dusty professors you kept in the back of the supplies cabinet), setting up their account and then making sure that they couldnt get into any trouble because they didn't know about 'ls -a' was very much a feature, trust me. Moreover I personally much prefer them to global configuration directories because they are always local to the thing being configured and you can always override a global configuration by using them. In fact I think they are a very elegant way to handle "hidden options" - stuff you want to expose to the power users but not bother newbies with. tl;dr: I am not convinced they are a misfeature.
- seanalltogether 14y ago
  I wonder if that is a good solution for separating novices and pros though? For me, hidden files and folders only help to reinforce the confusion people have about computers. It's one thing to provide a folder to a user with a lock or shield on it, it's another to completely hide it from them.
  - GauntletWizard 14y ago
    Negative. A lock or shield will convince a user that they're not to touch it, ever... Unless they are too dumb to understand the connotations, in which case they'll simply break it anyway. Hidden files are a good way to go - Out of sight, out of mind for most people, but not impossible to show them when they do need to touch.
  - brianpan 14y ago

dsrguru 14y ago

That's really interesting. I always assumed hiding dotfiles was a deliberate convention, but to semi-quote one of the commenters, Rob's got a point. Or two.

adambratt 14y ago
Heh, I remember when I first found out about dotfiles. I thought it was a genius idea... Little did I know.

emperorcezar 14y ago

If someone thinks hidden files are a misstep, then I don't want them designing a OS. Grandma really doesn't care about some conf file, or anything like that. She cares about the pictures of her grandkids. The opinion that there shouldn't be hidden files comes from a perspective of someone who is a "power user" and who can't step back and see that most users really don't care for some .config file. To them it's clutter that gets in the way of finding what they really care about. That said, dot files maybe the wrong way to do it. I like ~/Library in OSX. That's one good way. Edit: Note that I'm talking about a general trend in the post's comments and on here. Not the author's opinion.

UK-AL 14y ago
Did you read his post? He said exactly that. Put them in a config directory of some sort.
- emperorcezar 14y ago
  Yes. I'm not talking about his post. I'm talking about a few other comments here and on G+.
SCdF 14y ago
Sounds like you agree with him, as he advocates a 'conf' directory at the end of his post.

rehack 14y ago

I think the case of hidden dot files is a good example of 'convention over configuration'[1]. Point is accepted, that it came into being due to a lazy programmer. But surely early people might have just liked the unintended consequence of some files (dot files) being hidden. Just like most of us, whenever we learnt unix thought that it is by design. If early users, had found the consequence a handicap, it would have been fixed long back. Its similar to the use of hash-tags on twitter or the @for addressing which got adopted by users first and so became features (although the paths to them being considered features are different). [1] http://en.wikipedia.org/wiki/Convention_over_configuration http://en.wikipedia.org/wiki/Convention_over_configuration Edit: Grammar

ilaksh 14y ago

The main takeaway I think is that the reason we are doing things the way we do them is because that is how we have been doing them, not because of some kind of genius design. And actually the whole thing can start out as a mistake, and then later on basically becomes a religion. Its very funny actually.

mturmon 14y ago
Yes, very organic. The original programming shortcut accidentally created a sort of ecological niche, in which all sorts of things started to take root. Now, nobody can bulldoze the lot because there's too much stuff growing there.

eliben 14y ago

I really like it that G+ is becoming a "lightweight blogging" platform. There are too many of these around, and folding them into a "social network" seems like a good idea. I wish they would add more features that would make this easier, though, since in general I think it's in everyone's interest and will pull more traffic to the site.

89a 14y ago
I'd like it if I didn't need a Google account to read anything on it.
- jkn 14y ago
  Well be happy then, you don't need a Google account, as can be seen by entering the url in an Incognito window.
  - duaneb 14y ago
    The problem arises if you are signed into google elsewhere but not on G+.
- djeikyb 14y ago
  Yeah, it's pretty lame that if I'm signed into Google but don't want to accept G+'s real name policy, I can't read otherwise public posts.
lmm 14y ago
Adding a new one is not a good solution to "there are too many around". And I don't think consolidating everything into google is a good idea; these kind of posts belong on a different network to photos of my holidays.
saljam 14y ago
A bit off topic, but is it just me or is the quality of comments on public G+ posts is absolutely terrible? Within a few hours there has been more than 100 replies. Most are inane, rude, of the useless “great post!” type, or just plain old spam. I could only manage to glean about 8 or 9 insightful replies. Is this something intrinsic to G+? Is it a function of the author's popularity? The discussions here are normally much more sensible, but I would expect HN's readership to have a fairly similar demographic to that of Rob Pike's G+ subscribers.

caf 14y ago

The fact that "." and ".." were allocated actual directory entries and were returned when reading the directory, rather than just being handled by the kernel when parsing pathnames seems like the original sin of expediency here.

jlgreco 14y ago
Why? Seems like a brilliantly simple quick solution that would be rather easy to roll back in the future and lacks any real downsides besides being a tad weird.
- caf 14y ago
  Because it was creating real directory entries that caused "." and ".." to be visible to userspace programs reading a directory, which then led to the hack in "ls" to hide them, which is where the article picks up. It also isn't that easy to roll back once userspace programs start to rely on it - for example, the assumption that the number of files in a directory is equal to st_nlink - 2 is now so widespread that it's part of the UNIX API.
  - jlgreco 14y ago
    shrug, it seems to have all worked out pretty well.
    - strictfp 14y ago
      But all programs must make exceptions for these entries. Imagine trying to add a third magic filename, '...', for the grandparent for example. You would have a lot of coding to do. Not really possible. It would have been better to mark the dirs as special somehow rather than having each program have the convention programmed in.
      - caf 14y ago
        ...and of course even the kernel had to have exceptions for them anyway - it knew they were special because it had to stop userspace from unlink()ing or rename()ing them.

mseepgood 14y ago

He also writes why the dd command is so horrible: "dd is horrible on purpose. It's a joke about OS/360 JCL. But today it's an internationally standardized joke. I guess that says it all." http://en.wikipedia.org/wiki/Job_Control_Language http://en.wikipedia.org/wiki/Job_Control_Language

acabal 14y ago

I hate having dotfiles in ~/ for the same reason why I hate "My Documents" in Windows: because it's supposed to be my space that I organize, not a generic dumping ground for your config files, brand-named folders, or other nonessential garbage. I want my space to be mine. Keep your app's stuff out of there!

chris_j 14y ago
Where would you choose to put application config instead?
- acabal 14y ago
  Literally anywhere but my personal space. Why not have /etc/<username>/? I could even live with a single ~/.config/ but very few apps use that.
  - moe 14y ago
    It's actually quite useful to have everything that belongs to a user under one directory. The canonical example is the backup. There's a strong case to be made for "tar czf /tmp/backup.tgz ~". Do you really want your backups to become as complicated as they are on OSX and Windows? Likewise, being able to mount home-directories from remote servers, and being able to easily delete/move/quota users are highly desirable features in multi-user systems.
Foy 14y ago
On Windows I've retreated to Desktop which remains fairly sane as long as I opt-out or remove shortcuts being created after an install. "My Documents" is a nightmare... I think only one folder (out of nearly a dozen) and only one file are actually of my making.
zapu 14y ago
Microsoft now encourages to use AppData/ which is much better.
gnufs 14y ago

garfee 14y ago

omg!

89a 14y ago

Can't read without signing into Google… oh well nothing of value was lost.

mproud 14y ago
I'm with you.

antirez 14y ago

dotfiles are not perfect, but to have this very negative vision on a feature that also helped is a bit a revisionist attempt IMHO. Dotfiles provided a poor, but at least simple way to store program-specific-user-specific configuration, since another standard was missing. After all it's a simple and decentralized system that worked very well with the concept of unix user and ACL: you write something inside your home directory, and this changes the behavior of your program. Consider that this was invented many decades ago. Now it seems a lot better to have directories with sub directories. Maybe back then it was considered to be a waste of resources, inodes, and so forth. We can improve it, create a new standard, and have something better than dot files, but dot files are better than many other over-engineered solutions that I can imagine coming out of some kind of design commission to substitute them. Every time to passed your vim configuration to a friend you just copied a text file, sending it via email: you enjoyed one of the good points about dot files. Every time you did something like cat dotfile | grep option you enjoyed the positive effects of single-file plaintext configuration. Also it's worth saying that dot files are not just the concept of an hidden file with config inside. A lot of dot files also have a common simple format of multiple lines "<option> <value>", that's better than some XML or other hard to type format (IMHO JSON itself is not good for humans).

ajanuary 14y ago
How does any of those advantages apply only to dotfiles and not just plain files?
- FuzzyDunlop 14y ago
  I think there's an aspect of convenience, too. Personally, I prefer that to, say, configuration stored in some unspecified location in ~/Library/Application Support/Application/* (on OS X).

leothekim 14y ago

"How many bugs and wasted CPU cycles and instances of human frustration (not to mention bad design) have resulted from that one small shortcut about 40 years ago?" Sigh, if only most of us had worked on a software system that has lasted as long as that.

specialist 14y ago

Tangent: I'm grumpy that Eclipse IDE uses the file names .project and .classpath. So they're hidden by default. Requiring special treatment. Their content is XML. What's wrong with project.xml and classpath.xml?

mcgwiz 14y ago

The author's gripe seems to be that the hiding of dotfiles was unintended, ergo dotfiles are Bad. Whether they were intended is irrelevant; their wide usage vindicates the practice. After all, traction = value. The problem of program state/configuration/metadata storage is adequately met by dotfiles. There are, no doubt, numerous unintended behaviors of programs. Most of these are simply ignored and certainly not leveraged the way the dot behavior is. People don't go out of their way to abuse an unintended system behavior; they simply leverage all capabilities of a system ("intended" or not) to meet their needs. Had dotfiles not gained traction, some other solution would have been designed (or "engineered") to meet the needs of program state/configuration/metadata storage. [Tangent: All of this reminds me of grammar freaks that harp on "correct" usage, completely oblivious to the fact that grammar changes, and "correct" is merely a lightweight pointer to the current norm.]