5 ms·

Is the compiler faster these days?

s_kilk 12y ago
Apparently there is a faster incremental complier in sbt these days, but I can't seem to find any info online about how the compiler performance has changed over time. I get the impression that compiler preformance is being worked on, and is improving over time.
- salex89 12y ago
  I didn't use scala in the older days, and started only recently, I used Java previously. And I'm quite ok with the times. Sometimes it does slow down a bit, but most of the time for most cases I think it's alright.
chrisloy 12y ago
Compilation speeds are largely dependent on the type of code you're writing. At London's Scala Exchange last year, Odersky claimed he can get 10k lines/second (10x slower than javac, but still not bad) out of scalac by writing simple imperative code, but libraries that make heavy use of the type system and implicit scope can be as slow as 1 line/minute. I consider it fair to expect the compiler to be slower when it's doing more heavy lifting, but it would definitely be nice to see the exponential blow-up on the tail curbed in future. edit: To clarify, the numbers above were what Odersky claimed; I haven't verified them. The 1 line/minute was claimed of trying to build Shapeless - I've never seen anything approaching that in the wild.
- pseudonom- 12y ago
  > 1 line/minute I get that that's an upper limit, but that's still pretty shocking. Is there an overarching reason for this type of performance? Examples of this style of code?
  - MrBuddyCasino 12y ago
    Here is a proof that the Scala type system is actually Turing complete: https://michid.wordpress.com/2010/01/29/scala-type-level-encoding-of-the-ski-calculus/ https://michid.wordpress.com/2010/01/29/scala-type-level-enc... I can't comment on the details, I'm more of of Clojure guy.
  - psuter 12y ago
    Code with many implicit parameters and higher-kinded types is hard to compile, as the compiler needs to solve essentially a Prolog-ish unification problem. Scalaz is a popular library written largely in this style (e.g. [1]). (FWIW I've never seen 1 line/minute.) [1] https://github.com/scalaz/scalaz/blob/series/7.2.x/core/src/main/scala/scalaz/Foldable.scala https://github.com/scalaz/scalaz/blob/series/7.2.x/core/src/...
    - zak_mc_kracken 12y ago
      OCaml and Haskell offer very similar functionalities and their compilation times are lightning fast. The main reason why Scala is slow is because the compiler is poorly written. Take a look at the articles from Paul Phillips, one of the Typesafe cofounders who left the company in frustration, he's probably the person who knows scalac the best and he says the code base is basically hopeless. Note that 1/3rd of Martin's presentation is about Dotty, and that's basically where he spends most of his time these days, he hardly contributes to Scala any more [1]. [1] https://twitter.com/odersky/status/574665768484339713 https://twitter.com/odersky/status/574665768484339713
      - frowaway001 12y ago
        > OCaml and Haskell offer very similar functionalities and their compilation times are lightning fast. It took only one sentence to show that you never used OCaml or Haskell, so I'm not sure how serious the rest of your claims should be taken.
        mercurial 12y ago
- modersky 12y ago
  I think you got that wrong by a factor of 10. I get between 500 and 1000 lines / sec on my laptop. It depends on the code style. The more straightforward the code the faster the compile. Still with incremental compilation it means I wait rarely more than a couple of seconds.
  - chrisloy 12y ago
    Thanks Martin, unfortunately it's past the point where I can edit my original comment or I'd go back and fix it. As I recall you said it off-mike in Bill Venners' talk, otherwise I would have fact-checked it! Anyway, details aside I thought it was a great illustration of the impact on compiler performance that code complexity can have.
eweise 12y ago
I do a lot of scala development and the compile times are fine since we have many projects but each project is fairly small (microservices). Incremental compilation is essential for reasonable compile times. We use maven with zinc.
lmm 12y ago
Faster? Yes. Fast? No.

Apparently there is a faster incremental complier in sbt these days, but I can't seem to find any info online about how the compiler performance has changed over time. I get the impression that compiler preformance is being worked on, and is improving over time.

salex89 12y ago
I didn't use scala in the older days, and started only recently, I used Java previously. And I'm quite ok with the times. Sometimes it does slow down a bit, but most of the time for most cases I think it's alright.

chrisloy 12y ago

Compilation speeds are largely dependent on the type of code you're writing. At London's Scala Exchange last year, Odersky claimed he can get 10k lines/second (10x slower than javac, but still not bad) out of scalac by writing simple imperative code, but libraries that make heavy use of the type system and implicit scope can be as slow as 1 line/minute. I consider it fair to expect the compiler to be slower when it's doing more heavy lifting, but it would definitely be nice to see the exponential blow-up on the tail curbed in future. edit: To clarify, the numbers above were what Odersky claimed; I haven't verified them. The 1 line/minute was claimed of trying to build Shapeless - I've never seen anything approaching that in the wild.

pseudonom- 12y ago
> 1 line/minute I get that that's an upper limit, but that's still pretty shocking. Is there an overarching reason for this type of performance? Examples of this style of code?
- MrBuddyCasino 12y ago
  Here is a proof that the Scala type system is actually Turing complete: https://michid.wordpress.com/2010/01/29/scala-type-level-encoding-of-the-ski-calculus/ https://michid.wordpress.com/2010/01/29/scala-type-level-enc... I can't comment on the details, I'm more of of Clojure guy.
- psuter 12y ago
  Code with many implicit parameters and higher-kinded types is hard to compile, as the compiler needs to solve essentially a Prolog-ish unification problem. Scalaz is a popular library written largely in this style (e.g. [1]). (FWIW I've never seen 1 line/minute.) [1] https://github.com/scalaz/scalaz/blob/series/7.2.x/core/src/main/scala/scalaz/Foldable.scala https://github.com/scalaz/scalaz/blob/series/7.2.x/core/src/...
  - zak_mc_kracken 12y ago
    OCaml and Haskell offer very similar functionalities and their compilation times are lightning fast. The main reason why Scala is slow is because the compiler is poorly written. Take a look at the articles from Paul Phillips, one of the Typesafe cofounders who left the company in frustration, he's probably the person who knows scalac the best and he says the code base is basically hopeless. Note that 1/3rd of Martin's presentation is about Dotty, and that's basically where he spends most of his time these days, he hardly contributes to Scala any more [1]. [1] https://twitter.com/odersky/status/574665768484339713 https://twitter.com/odersky/status/574665768484339713
    - frowaway001 12y ago
      > OCaml and Haskell offer very similar functionalities and their compilation times are lightning fast. It took only one sentence to show that you never used OCaml or Haskell, so I'm not sure how serious the rest of your claims should be taken.
      - mercurial 12y ago
modersky 12y ago
I think you got that wrong by a factor of 10. I get between 500 and 1000 lines / sec on my laptop. It depends on the code style. The more straightforward the code the faster the compile. Still with incremental compilation it means I wait rarely more than a couple of seconds.
- chrisloy 12y ago
  Thanks Martin, unfortunately it's past the point where I can edit my original comment or I'd go back and fix it. As I recall you said it off-mike in Bill Venners' talk, otherwise I would have fact-checked it! Anyway, details aside I thought it was a great illustration of the impact on compiler performance that code complexity can have.

eweise 12y ago

I do a lot of scala development and the compile times are fine since we have many projects but each project is fairly small (microservices). Incremental compilation is essential for reasonable compile times. We use maven with zinc.

lmm 12y ago

Faster? Yes. Fast? No.