5 ms·

ClojureScript Builds, Rebooted

96 points by skrblr 12y ago

lynndylanhurley 12y ago
How will this work with tools like figwheel [1] and austin [2]? 1. https://github.com/bhauman/lein-figwheel https://github.com/bhauman/lein-figwheel 2. https://github.com/cemerick/austin https://github.com/cemerick/austin
- michaniskin 12y ago
  The post was a demonstration of exactly that: cljs incremental builds with live-reload and cljs browser repl.
weavejester 12y ago
Boot looks interesting, but replacing an immutable data structure with side-effectful functions feels like a step in the wrong direction.
- michaniskin 12y ago
  The main reason why build tools exist is to perform transformations on a file set. The vast majority of the things build tools do that are useful are side effects. These "side-effectful" functions are basically transducers; the reduction is performed on the file set instead of a sequence. They are mostly stateful transducers, but this does not fly in the face of functional programming. The opposite, in fact! They facilitate separation of concerns in a way that an immutable but global configuration map cannot.
  - weavejester 12y ago
    It's true that you need to interact with the filesystem eventually, but the longer that can be deferred, the more complexity can be avoided. Ideally I'd want to construct a functional data structure that describes my build process, and at the end pass it to a side-effectful function to produce a change to the filesystem. Boot appears to be side-effectful from the get-go, but perhaps I'm mistaken about how it operates?
    - michaniskin 12y ago
      All JVM build tools are side-effectful from the get-go, and they really have to be. Consider the :dependencies and :source-paths keys in a Leiningen project.clj. The purpose of these is to manipulate the mutable class path. To have a JVM build tool that doesn't revolve around the class path will require a complete reinvention of the JVM ecosystem and all of the existing tooling (like in our demo we use the Google Closure compiler, which mutates all kinds of things–that would have to go), which is, I'm sure, never going to happen.
      - weavejester 12y ago
        You need to eventually be side-effectful, but that doesn't mean you need to start side-effectful. The :dependencies in a Leiningen project map are just a data structure until they're passed to eval-in-project, which happens at the end of a chain functional operations. One of the core ideas of Clojure is that we should try to favour simple solutions over complex ones. Side-effectful functions are the some of the most complex tools we have, and while they are necessary eventually, it would be nice to have the majority of the code-base work with simple data structures, and push out the complexity of I/O to the edges of the application.
        brandonbloom 12y ago
        Side-effects aren't necessarily bad, it's composability that is good. Purity is a composable property, but there are plenty of composable side-effects. For example, let's ignore all other effects besides file IO (especially ignoring internet IO). Further, let's assume that our only IO operations are `(read path) => data` and `(write path data) => nil`. Let's also assume that both of these operations are atomic (ie you can't perceive a half-written file). If a build task attempts to read a file that doesn't exist, that task pauses itself. When a file is written, any task waiting on the file are resumed. If you re-write a file, it has to exactly match the already written file, or the build fails. To kick-off a build, you wait on one or more files and then start one or more tasks. Viewed this way, the file-system is a monotonic logic variable. Yes, the programming model is effectful, but there is a composable property: build repeatability. Just as you can compose arbitrary pure functions and get a pure function out, you can take any two arbitrary graphs of these constrained IO build tasks, compose them together, and the resulting larger graph will also be a repeatable build.
  - dustingetz 12y ago
    its not at all clear to me that you are correct. I'm glad that people are putting thought into the cljs build process though, to this day it is still a particularly un-fun part of clojurescript.
- brandonbloom 12y ago
  I think that the immutable data structure of lein is largely superficial. Once the final project map is built, all bets are off for the effectful behavior of downstream tasks/plugins. The only clear benefit from it is that you can capture a "complete" configuration with `lein pprint`. However, it does introduce significant cost in terms of the declarative configuration wack-a-mole, where you never quite know what magic ^:replace or :special-key to include to get what you want. If you're not going to attempt to fix this superficiality by pushing immutable data deeper in to the system, then it makes perfect sense to discard it completely.
  - michaniskin 12y ago
    Totally agree. Also, in boot we don't want to obtain a complete configuration before tasks run, because tasks can participate in the process. In boot a task can add dependencies or call other tasks, etc. This is why we need only one abstraction (tasks). Just like in a Clojure program you don't have a complete configuration of values in variables before the program runs, because it's the program that creates those values. Boot figures so much out on its own that `boot pprint` isn't even a thing you would want. Consider the hoops that needed to be jumped through to get the Maven wagons system implemented in Leiningen. In boot it's a non-issue–the environment is dynamic so you can just install wagon deps and then in the next expression install the deps and repositories that depend on that wagon. We didn't need to make any changes to boot to accommodate them.
    - brandonbloom 12y ago
      I still don't understand what the "task abstraction" is or what it provides. It seems to me that it's simply a Clojure function with a corresponding command line interface. Is that fair? Does it do something else too? If it's just a function, I don't know why command-line support is valuable. For interactive use, the Clojure REPL is just fine. For automated use, you only need a single shell utility like perl or awk to evaluate an expression or run a script.
      - michaniskin 12y ago

96 points by skrblr 12y ago

How will this work with tools like figwheel [1] and austin [2]? 1. https://github.com/bhauman/lein-figwheel https://github.com/bhauman/lein-figwheel 2. https://github.com/cemerick/austin https://github.com/cemerick/austin

michaniskin 12y ago
The post was a demonstration of exactly that: cljs incremental builds with live-reload and cljs browser repl.

weavejester 12y ago

Boot looks interesting, but replacing an immutable data structure with side-effectful functions feels like a step in the wrong direction.

michaniskin 12y ago
The main reason why build tools exist is to perform transformations on a file set. The vast majority of the things build tools do that are useful are side effects. These "side-effectful" functions are basically transducers; the reduction is performed on the file set instead of a sequence. They are mostly stateful transducers, but this does not fly in the face of functional programming. The opposite, in fact! They facilitate separation of concerns in a way that an immutable but global configuration map cannot.
- weavejester 12y ago
  It's true that you need to interact with the filesystem eventually, but the longer that can be deferred, the more complexity can be avoided. Ideally I'd want to construct a functional data structure that describes my build process, and at the end pass it to a side-effectful function to produce a change to the filesystem. Boot appears to be side-effectful from the get-go, but perhaps I'm mistaken about how it operates?
  - michaniskin 12y ago
    All JVM build tools are side-effectful from the get-go, and they really have to be. Consider the :dependencies and :source-paths keys in a Leiningen project.clj. The purpose of these is to manipulate the mutable class path. To have a JVM build tool that doesn't revolve around the class path will require a complete reinvention of the JVM ecosystem and all of the existing tooling (like in our demo we use the Google Closure compiler, which mutates all kinds of things–that would have to go), which is, I'm sure, never going to happen.
    - weavejester 12y ago
      You need to eventually be side-effectful, but that doesn't mean you need to start side-effectful. The :dependencies in a Leiningen project map are just a data structure until they're passed to eval-in-project, which happens at the end of a chain functional operations. One of the core ideas of Clojure is that we should try to favour simple solutions over complex ones. Side-effectful functions are the some of the most complex tools we have, and while they are necessary eventually, it would be nice to have the majority of the code-base work with simple data structures, and push out the complexity of I/O to the edges of the application.
      - brandonbloom 12y ago
        Side-effects aren't necessarily bad, it's composability that is good. Purity is a composable property, but there are plenty of composable side-effects. For example, let's ignore all other effects besides file IO (especially ignoring internet IO). Further, let's assume that our only IO operations are `(read path) => data` and `(write path data) => nil`. Let's also assume that both of these operations are atomic (ie you can't perceive a half-written file). If a build task attempts to read a file that doesn't exist, that task pauses itself. When a file is written, any task waiting on the file are resumed. If you re-write a file, it has to exactly match the already written file, or the build fails. To kick-off a build, you wait on one or more files and then start one or more tasks. Viewed this way, the file-system is a monotonic logic variable. Yes, the programming model is effectful, but there is a composable property: build repeatability. Just as you can compose arbitrary pure functions and get a pure function out, you can take any two arbitrary graphs of these constrained IO build tasks, compose them together, and the resulting larger graph will also be a repeatable build.
- dustingetz 12y ago
  its not at all clear to me that you are correct. I'm glad that people are putting thought into the cljs build process though, to this day it is still a particularly un-fun part of clojurescript.
brandonbloom 12y ago
I think that the immutable data structure of lein is largely superficial. Once the final project map is built, all bets are off for the effectful behavior of downstream tasks/plugins. The only clear benefit from it is that you can capture a "complete" configuration with `lein pprint`. However, it does introduce significant cost in terms of the declarative configuration wack-a-mole, where you never quite know what magic ^:replace or :special-key to include to get what you want. If you're not going to attempt to fix this superficiality by pushing immutable data deeper in to the system, then it makes perfect sense to discard it completely.
- michaniskin 12y ago
  Totally agree. Also, in boot we don't want to obtain a complete configuration before tasks run, because tasks can participate in the process. In boot a task can add dependencies or call other tasks, etc. This is why we need only one abstraction (tasks). Just like in a Clojure program you don't have a complete configuration of values in variables before the program runs, because it's the program that creates those values. Boot figures so much out on its own that `boot pprint` isn't even a thing you would want. Consider the hoops that needed to be jumped through to get the Maven wagons system implemented in Leiningen. In boot it's a non-issue–the environment is dynamic so you can just install wagon deps and then in the next expression install the deps and repositories that depend on that wagon. We didn't need to make any changes to boot to accommodate them.
  - brandonbloom 12y ago
    I still don't understand what the "task abstraction" is or what it provides. It seems to me that it's simply a Clojure function with a corresponding command line interface. Is that fair? Does it do something else too? If it's just a function, I don't know why command-line support is valuable. For interactive use, the Clojure REPL is just fine. For automated use, you only need a single shell utility like perl or awk to evaluate an expression or run a script.
    - michaniskin 12y ago