8 ms·

Choosing a unified format for configurations is an interesting task, because they all suck a lot (hehe). XML is too verbose to be nice to work with. Plain text

• mlvljr 12y ago
  Would be even cooler if "Python-ized" (i.e. if spacing mattered): that is they could look like this
  • robin_reala 12y ago
    That’s basically YAML (that they mentioned as a possibility). http://yaml.org/ http://yaml.org/
  • jacquesm 12y ago
    Significant whitespace in configuration files... what could possibly go wrong. Didn't we learn something from Makefiles here? Does that 'tab' look like 8 or 4 spaces to you?
    • tormeh 12y ago
      Same that goes wrong in Python: Nothing. As long as tabs are outlawed, anyway.
      • jacquesm 12y ago
        > As long as tabs are outlawed, anyway. When did that happen?
      • takeda 12y ago
        I see many people continuing holy wars between tabs and spaces, even to a point to dismiss specific technology because of it. If you simply enable option in your editor to make white spaces visible (i . e. you can tell the difference between tab and space), the problem becomes silly. I personally like to use tabs for indentation, because of huge flexibility being able to change tab size (which changes indentation) as needed, but that just me.
        • tormeh 12y ago
          It doesn't really matter which it is, as long as it's one of them. Spaces are more copy-paste friendly though.
    • Redoubts 12y ago
      Tabs look like a light grey arrow in any editor I care to use.
    • acdha 12y ago
      This is mentioned a lot by Python newcomers but rarely turns out to be a problem in practice because, unlike GNU Make, Python simply requires you to be consistent throughout the same file. That strikes me as an even more valuable property for configuration, which is more often accessed by multiple people and tools and more often as a distraction from other work rather than the primary focus.
      • josteink 12y ago
        > Python simply requires you to be consistent throughout the same file. You or the other team of developers/maintainers/sys-admins looking over or patching this particular file. Using whatever default editor they prefer with whatever config an tabifying/untabifying behaviour that has. Or maybe they are applying a patch produced on a different system with different tabulation? Or the other 2000 common ways for accuracy to go out the window. > That strikes me as an even more valuable property for configuration So you'd like to deliberately make the configuration system more brittle for absolutely no gain? Yeah. Count me up for anything else. I personally prefer s-expressions, but I would be willing to go the full mile with pretty much any format NOT based on whitespace. Whitespace is bullshit and depending on it even more so.
        • acdha 12y ago
          If your team can't pick a single style and stick with, it is exceedingly unlikely that whitespace will be your biggest source of problems.
          • josteink 12y ago
            So your argument is that because you should be doing $X anyway, let's make the entire OS's configuration system needlessly brittle, just because you're a python dev and like python's style. I don't think you understand the philosophy behind FreeBSD: it's about creating a predictable and reliable OS. This goes against those goals. Simple as that.
            • acdha 12y ago
• legulere 12y ago
  The YAML format mentioned in the slides actually has the least syntactic clutter, however the parsing is quite difficult
  • currysausage 12y ago
    Because it simply is too complex for a format that emphasizes easy readability. Some easy-to-grasp subset of YAML might be a good choice.
    • byuu 12y ago
      I came up with such a format (called BML) and for the same reasons. Here's an example that shows the entirety of the syntax: server path: /core/www/ host: example.com port: 80 service: true proxy host: proxy.example.com port: 8080 authentication: plain description :Primary web-facing server :Provides commerce-related functionality server ... proxy host="proxy.example.com" port="8080" authentication: plain Everything is a node, which can have a data value and zero or more child nodes. Nesting is determined by counting the number of indentions. It uses a counter so that if you want indents to be one tab, or two spaces, or four spaces, it will still work. (Being too rigid here makes the syntax very unfun for humans to write.) Once parsed, element-style (first proxy node) or attribute-style (second proxy node) become identical nodes and are treated the same (but with a flag in case you want to write out a modified file): they are fully interchangeable, so no attribute vs element debates, just use what works best for readability. (This really is critical. Some document types would be ten times as long without attribute-style nodes.) The syntax has no entities. foo="data" can capture any data that doesn't need quotes or newlines. foo: can capture any data that doesn't need a newline. foo\n:data\n:data can capture any data that doesn't need binary. foo\n:base64\n:base64 can capture absolutely anything. Node names must be [A-Za-z0-9-.]{1,}, and are case-sensitive. The same node name can appear multiple times at any level, even at the root level. Ambiguity is resolved by the order of appearance for each node. The data values of nodes are completely unparsed by the markup. The syntax knows no difference between strings, integers, floats, booleans, binary data, arrays, etc. The application parses the text however it wants. The library adds some convenience functions (.text() to strip surrounding whitespace, .integer() to get a number, .boolean() to decode true/yes/on vs false/no/off, etc.) File format is mandatory UTF-8 (no BOM.) Preferred line feeds are '\n', but '\r\n' is also permitted because Microsoft. The implementation in C is about 8KB. Since everything has a marker, allocations are not necessary for the node names/values (but you will need to allocate the tree structure, obviously.) There's an accompanying path query syntax (ala XPath) that's another 6KB of code or so. (All the edge cases are well-defined (mismatched indentation, mixing multi-line and child nodes, etc), but the post is getting a bit long.) This was the best I could do at minimalism. Removing any functionality it has results in ruling out many use cases.
• deleted 12y ago
  [deleted]
• progman 12y ago
  > Personally, I think the nicest and most expressive way is S-expressions. Not only that. S-expr could be created automatically and run on the fly at runtime, new configurations could be added without restart of services, and Lisp has a well known powerful macro system. That way also "Python-ized" configs could be translated into s-expr easily. Such Lisp based configs could be run in a C embedded Lisp interpreter. However, the FreeBSD team should at least seriously consider a new neutral logo. I wonder what BSD Unix has to do with the devil.
  • klez 12y ago
    > However, the FreeBSD team should at least seriously consider a new neutral logo. I wonder what BSD Unix has to do with the devil. It's not a devil, it's a Daemon. Like, Unix daemons (httpd, ftpd etc.)
    • macns 12y ago
      An evil daemon is worse than the devil (sarcasm) edit: paren'd edit 2: been using bsd and linux for servers for the past 10 years, love both of them for different reasons. This is the first time I've seen anyone comment on the FreeBSD logo like that. I thought it was hilarious
  • feld 12y ago
    You "BSD devil" trolls are really not putting in much effort.
  • ta0967 12y ago
    why, the goal and purpose of the FreeBSD Unix is to agitate 1600s-style cavemen of course.
• barosl 12y ago
  Regarding the configuration formats, I would recommend TOML. In fact, I hated it because it seemed to be just "another standard" that unnecessarily adds dependency. But after using Rust, whose package manager forces me to write the package configuration in TOML, I found the format is more like "JSON designed for configuration file." As you said, JSON is full of ':', '{' and '}'. And it is natural because it started from data interchange format, not for handwriting. TOML solves this problem very well IMHO. Also, unlike XML that requires an external structure to validate the types of the values, TOML values have types, just like JSON.
  • klez 12y ago
    Link for the lazy https://github.com/toml-lang/toml https://github.com/toml-lang/toml
  • boracay 12y ago
    It just seems that TOML (and similar formats) would be more of an interface rather than the configuration itself.
  • protomyth 12y ago
    Is there some reason for the commas in lists? I'm starting to like s-exprs more and more for configs and am starting to think it might be because of a lack of love for commas.
  • 23david 12y ago
    Can you give some examples of TOML-based Rust package configurations that you think are good? My experience using TOML for config files has been very different. IMO, using YAML for the config format is still the best option for most use-cases at this point. (Easy to support both json and YAML.) But I do think that it would be great to have a YAML successor that standardizes some of the cool things people are doing with custom YAML parses, and also removes outdated and unnecessarily complex parts of the original standard.
  • mst 12y ago
    There's also http://p3rl.org/JSONY http://p3rl.org/JSONY which is explicitly "JSON re-imagined for configuration files" by myself and ingy (the creator of YAML). The key win for us is that all JSON is valid JSONY, so you can use the human friendly syntax for hand editing but still just dump out a JSON object into the file if you're generating the entire thing from code.
• agumonkey 12y ago
  Take a look at GUIX (GNU Guile ~wrapper~ around Nix) http://www.gnu.org/software/guix/manual/guix.html#Programming-Interface http://www.gnu.org/software/guix/manual/guix.html#Programmin... That said s-exp can be cryptic depending on how you design your DSL, but overall it's a 80% great to read and easy to process, plus chopping s-exp in s-exp aware editors (not pointing at some very old very large one here)
• TheCondor 12y ago
  They were suggesting an api for it as well. Who cares what the format is if you access it programmatically?
  • Yhippa 12y ago
    As much as people make fun of Java for putting abstractions over implementations this is a situation where I would love to see something like this. We can always switch to the hot new data format with something like this.
• rsync 12y ago
  I've (purposely) ignored the config files and systemd wars. But that means I am truly ignorant - what, in fact, is the big problem with plain old unix-style config files ? Note that I am not asking about what is wrong with how they are scheduled and run - I get that controversy. But what is wrong with the files themselves ? I know I have absolutely hated every single interaction I have had with plists and launchd and whatever else on OSX ...
  • digi_owl 12y ago
    Best i can tell, they don't play well with GUI config tools...
  • vertex-four 12y ago
    > plain old unix-style config files Like what? /etc/X11/xorg.conf is a different format from /etc/openldap/ldap.conf is a different format from /etc/crontab is a different format from /etc/nsswitch.conf. Unifying configuration files under a single common format would allow for less complexity in configuration management tools - you could just create a data structure and serialise it to a file, rather than dealing with text-based templates with countless edge cases.
    • thwarted 12y ago
      And yet elsewhere, domain specific languages are preferred. I'd love to see the One Config File Format To End Config File Formats that supports xorg, ldap, crontab, nsswitch, apache, and anything else. Sure, use something like JSON, XML, INI, or YAML for this, for which the format is regular and parsable with an off-the-shelf parser, but then the domain specific stuff just moves into, for example, the key names. You've gained nothing in terms of maintenance or automatic editablity, but you have gained consistent use of whitespace and punctuation characters. "Creating a data structure and serializing it to a file" is doable text-based formats. Admittedly, templates are troublesome, but that's because you shouldn't be using templates anyway. Templates are an attempt to avoid understanding the domain specific language but still maintain the power of the domain specific language. I've never had good luck with this, which is why I mainly use configuration management tools, like puppet, to keep a fleet of machines in a consistent state by putting fully formed files in place. Templating takes significant care.
• TazeTSchnitzel 12y ago
  JSON is clunky, it's true, but it has advantages. It's very, very widely supported. There is no format versioning, there are no dialects, there is only One True JSON. It's incredibly minimal and supports no more, and no less, than it needs to. It's not perfect, but it's a very good choice.
  • tippytop 12y ago
    But JSON doesn't support comments which are pretty nice in a config file. You can do the ugly comment as property stuff, or strip them out, but you're getting away from the One True Format.
    • TazeTSchnitzel 12y ago
      That's true. You could standardise on JSON with comments removed by JSMin... The underlying data would still be standard.
      • darkandbrooding 12y ago
        Consider Sendmail as a cautionary tale. One more than one occasion I have seen a busy sysadmin manually edit a .cf file and forget to backport the edit to the .mc file. The next time someone regenerates the .cf from the .mc you have a vanishing edit. Yes, the sysadmin was clearly at fault. No, this is not a deal breaker, as seen by the longevity of Sendmail. But a config file that gets pushed through a preprocessor has a more complicated life cycle than a "static" equivalent. If your organization has comprehensively embraced ansible/chef/puppet/etc this may be a non-issue, because all of your configs might be generated. Just food for thought.
        • TazeTSchnitzel 12y ago
          > Yes, the sysadmin was clearly at fault. No, this is not a deal breaker, as seen by the longevity of Sendmail. But a config file that gets pushed through a preprocessor has a more complicated life cycle than a "static" equivalent. Er, only if the application doesn't run the preprocessor itself.
• tedunangst 12y ago
  OpenBSD solution has been to use "pf format" config files for as much as possible. They're all different, but immediately intelligible to anyone who's seen any of them before.
• cwp 12y ago
  NixOS got this right. Declarative configuration is great in theory but it doesn't scale well. See Amazon's CloudFront configuration files for what configuration in JSON looks like. (For example, https://s3-us-west-2.amazonaws.com/cloudformation-templates-us-west-2/CloudWatch_Logs.template https://s3-us-west-2.amazonaws.com/cloudformation-templates-...) It quickly gets hard to manage the syntax and balance all the quotes, parentheses, braces and brackets, with careful use of whitespace being critical for even a hope of understanding it. When you add in semantics, it gets even worse. Take the "Ref" construct in CloudFront to work around the fact that JSON doesn't have a way to name something and refer to it elsewhere. As the amount of configuration grows, you have to introduce abstractions to encapsulate common patterns, break out of deep nesting, name things etc. I guess it's possible to do that declaratively, but that leads to things like XSLT. (Or, open the CloudFormation JSON linked above and search for "Fn::". It makes me shudder). S-expressions would indeed be better than JSON, but they still wouldn't scale without macros. So either all programs would have to know lisp, or you need a dedicated program for executing the configuration. That's what NixOS has. It does all configuration via "nix expresssions" which are written in a lazy, functional language. The functional bit allows for the abstraction necessary to make large amounts of configuration readable and maintainable. The lazy bit makes executing large configurations efficient, given that you're usually only interested in certain parts of it at any one time. The result is that you can write a configuration that describes a cluster of machines from top to bottom: from how many app servers to put behind the load-balancer to which version of zlib to link node.js against. It takes a bit of effort to learn nix, and for many, the functional paradigm will be strange and scary. But's so worth it.
  • pmoriarty 12y ago
    Could you give an example of a NixOS config file that contains as much complexity as the CloudFront JSON config file, but does so in a more readable way?
• emmelaich 12y ago
  You can bypass the issue by using the amazing Augeas project; a universal configuration API. http://augeas.net/ http://augeas.net/ At the least a universal API provides a midpoint to moving to a universal format. It really deserves more attention and use! (One possible reason that it's not is that people might find it a little user-hostile at first.)