6 ms·

Effect Systems vs. Print Debugging: A Pragmatic Solution

91 points by degurechaff 1y ago

svieira 1y ago
I wonder if this will wind up being a category of problems and the solution is a separate "system" set of effects (effectively `({user}, {system})`) or if this one-off extension is all that will be needed. Either way, extremely well explained both in motivate and implementation!
- epolanski 1y ago
  I think the solution is to not have side effects in pure functions really. At some point you're composing your pure functions and _have_ to call them by some effectful function (otherwise they're never executed or you're doing computations that aren't consumed by anything). The only sane alternative is to have a debug effectful variant where you turn off these checks. But then why would `stdout` debugging fine, and not say writing to a different stream or file?
- vilunov 1y ago
  > I wonder if this will wind up being a category of problems and the solution is a separate "system" set of effects (effectively `({user}, {system})`) or if this one-off extension is all that will be needed. It already is, kinda. In my practice very often you have global singleton values that are either defined as static variables, or passed as arguments to nearly all functions in a module. Since implicit presence of `Debug` effect is already a compilation parameter, it could be generalized to support any sets of implicit effects. Thus you might design a module that has implicit Logger and Database effects in all its functions.
  - epolanski 1y ago
    Singleton pattern isn't really used in functional-effect systems, dependency injection is generally used instead.
  - yccs27 1y ago
    > Thus you might design a module that has implicit Logger and Database effects in all its functions. Logging does seem like a very similar case to debugging, only that you expect to leave it on in production. On the other hand an implicit Database effect kind of defeat the point of an effect system. I think the key is that Debug and Logger effects don't really affect the rest of the code - if you remove all debug/log statements, the only thing that changes is the debug/log output (and slightly faster execution probably).
    - saghm 1y ago
      > Logging does seem like a very similar case to debugging, only that you expect to leave it on in production In a lot of logging systems, debug is one of the common levels of logging. I'm not even convinced that the term "debug" is unambiguous to clearly refer to something that's not a subset logging. Presumably the difference is intended to mean things printed that are unconditionally going to stdout rather than into some system that might change the output location and filter/annotate things, but I have to imagine that it might just make more sense not to have separate models for them at all
      - yccs27 1y ago
        I was referring to the specific Debug effect described in the post, which only works in debugging builds. This means that the production build cannot have unexpected output from a function with no declared effect.
        saghm 1y ago
        I understand. My point was that if you did want to extend further to have an effect for logging, it seems like having fully separate abstractions for them rather than finding a way to express them both as part of the same abstraction would make sense.
- jerf 1y ago
  The other major use case that leaps to mind is "observability"; I want to be able to poke a metric without the function becoming impure, since it is not uncommon for something deep down the call stack to poke a metric and I don't want it to propagate up the stack. You can also make a case for logging that isn't just debug logging. Propagating up a new effect just because some deep function needs to push a log is not very friendly or useful. Perhaps there is two or three more, but I do think this is a finite set that we can write a language around and just sort of consider them "ambient effects". While metrics and logging are nominally impure, in that they are certainly mutations, if you can't read the logs or the metrics without an effect, you still retain the really important aspect of purity, which is that the pure code can't cause a change that is observable by that or other code, with the very specific exception of the logging stream and metrics. I wouldn't be quite ready to put all my chips on this, but I think "the inability to create changes that can be witnessed" is actually the true goal, not "the inability to create changes" with no qualifications. Pure code already necessarily creates changes in a system, the key is that while the CPU registers may change and other parts of the system may mutate, the code can't witness those changes and conditionalize future execution on it. All effects-based systems have already agreed that there are things that are mutations in something real in the physical world they aren't going to consider effects, adding a couple more categories is not going from 0 to 1 but 12 to 15. It's not a strict purity question but a cost/benefits question. It occurs to me as I type this that a really ambitious language with a strong enough type system might even be able to turn this into a completely safe proposition, allowing users to declare effects with some sort of very safe "sink" associated with them that the type system checks can not escape out into the rest of the code in any visible way and constraining the visibility somewhere that is contained in the conventional effects system. All these things I'm talking about here and in the blog post are taking the form of values that simply disappear into the ether from the point of view of the creating function. I'd like the language to make it easy to query a function for which ambient effects it uses (as a development-time operation, not a run-time one), and I think that would clean up most of the rest of the practical problems.
  - ramses0 1y ago
    100% well said! Exactly how to handle log/print/metrics (as results/returns from functions) has been on my mental back burner for years. It's proper (but goofy) to pass in a `logger` object to every function, but the practicality of plumbing a "logger" everywhere is disgusting. There's a ton of value to be able to "capture" what was logged by a function, but as you mentioned with "metrics", the invasiveness of plumbing a metrics object "through" your code is really gross. Maybe a straw man syntax like: this.__classHooks__.logger = new AbstractLogger(...) this.__classHooks__.metrics = new AbstractMetrics(...) ...like a puzzle piece, if the container (parent object) wants to "attach" to the exported interface, it can kindof dependency injection, otherwise logs and metrics might fly off into the void.
throwawaymaths 1y ago
Honestly a pragmatic solution would be to have printing to stderr (debug print, logs, whatever) not be an "effect". then as an added benefit if its context is elided out in an optimization you know.
- jorkadeen 1y ago
  The challenge is that the compiler uses the type-and-effect system for many of its tasks, including whole-program optimization and code generation. If printing- or logging statements have no effect, the compiler might reorder them or even remove them.
- maleldil 1y ago
  This is similar to what Haskell has with trace[1]. It pretends to be a pure function (so it doesn't need the IO monad), but prints to stderr. [1] https://hackage.haskell.org/package/base-4.21.0.0/docs/Debug-Trace.html#v:trace https://hackage.haskell.org/package/base-4.21.0.0/docs/Debug...
  - JoelMcCracken 1y ago
    And, just like tfa alludes, it is awkward because of laziness; in non trivial scenarios it can be quite surprising how your trace isn’t tracing. This is bc of non strict eval in Haskell, but still the core is that the lang is designed to not eval stuff when it’s not necessary.
- naasking 1y ago
  It needs to be an effect so the compiler knows how optimizations are permitted to handle it. Otherwise your print statements might appear in totally the wrong order or even not at all. Imagine saying that certain memory stores are allowed to cross a memory barrier operation, you'd completely lose the ability to reason about concurrency around those stores.
  - throwawaymaths 1y ago
    debugging must be a nightmare if you can't get a good mental model of how the compiler is transforming your code
noreplydev 1y ago
As I understand, Flix implements a system to type not only a return type but an Effect too. If that's the case, It's not an overkill solution? and why?
- wk_end 1y ago
  It makes it easier to reason about a program. For an dev, you can immediately understand what global state, if any, can affect a function’s behaviour. And for the compiler, knowing about what effects an operation has can enable optimizations like dead code removal and autovectorization; the article goes into that.
draw_down 1y ago
[dead]
podgorniy 1y ago
Cool idea to "type" side effects I'm curious what other languages are trying to achieve such?
- jorkadeen 1y ago
  A few other languages are: - https://effekt-lang.org/ https://effekt-lang.org/ - https://koka-lang.github.io/ https://koka-lang.github.io/ - https://www.unison-lang.org/ https://www.unison-lang.org/ - https://antelang.org/ https://antelang.org/
  - ux266478 1y ago
    Oh I hadn't heard of Ante before. This looks very close to the language I wanted out of Rust. Haskell's module system, row polymorphism, linear types, no sepples glyph soup. That's an instant bookmark save. Will be watching that space very closely.
- ux266478 1y ago
  Koka[1] jumps to mind as a language built around effect types. Without having used Flex or Koka, I'm not sure how their effect types actually differ from the normal monad gauntlet you get in pure languages. [1] - https://koka-lang.github.io/koka/doc/index.html https://koka-lang.github.io/koka/doc/index.html
- davidatbu 1y ago
  Koka, Roc-lang come to mind.
- ookdatnog 1y ago
  AFAIK the first somewhat widely-known language to do this was Haskell (through libraries). I'm not 100% clear on the entire history, but I think it goes something like: 1. Initially there was no way to do effects in Haskell, everything was pure. 2. Then it was realized that IO can be modeled with monads, so the IO type and do notation were added. 3. Gradual realization that monads can be used to also constrain effects, ie you can construct a type of "stateful computations" that can read and write to a specific state, but not touch other states or write to disk or something. 4. Monad transformers are invented, which allow stacking monads on top of eachother to support multiple effects. Together with type classes, this gets us pretty close to extensible effects (the approach used in Flix, if I understand it correctly). So for example you can express that your function needs to write to a log and may exit early with an error message with the constraints `(MonadWriter w m, MonadError e m) => ... -> m resultType`, and you can then use monad transformers to build a stack that provides both of these effects. 5. Monad transformers have some issues though: they affect performance significantly and the interaction between effects is tricky to reason about. So an alternative is sought and found in extensible effects. The initial proposals were, iirc, based on free monads, but those aren't great for performance either, so ever since there has been a whole zoo of different effects and handlers implementations that all make different trade-offs and compromises, of which I think the `effectful` library is now the de facto default, and I think what it offers is quite similar to the Flix language's effect system (I'm not sure on what finer points it differs).
ducdetronquito 1y ago
Pleasant and interesting read !
vlovich123 1y ago
> Hence, when the compiler is run in production mode, we disable the lie that allows the implicit Debug effect. As a result, using dprintln in production mode causes a compilation error. The team can’t seem to make up its minds of the language is intended for high performance or not. They talk about the importance of purity for automatic optimizations but in the real world there’s all sorts of practical reasons for needing to debug production compiled code (eg imagine something like a browser and you’re trying to figure out some weird behavior that’s difficult to catch in a debugger but too slow to reproduce in a debug build or even not reproducible due to different timings resulting in different race conditions) Also blaming the users of your language for your language not being able to meet their needs isn’t a good look. It suggests the language is probably attracting the wrong users or positioning itself incorrectly in the market place.
- jorkadeen 1y ago
  We think that functional programmers should be able to write e.g. `List.count(x -> x > 5, l)` (or e.g. use pipelines with |>) and have it run as fast as an ordinary imperative loop with a mutable variable. The Flix compiler gives them that-- but it requires the program to undergo certain transformations that may require expressions to be moved around and eliminated. It is dangerous to perform such optimizations with incorrect assumptions about types or effects. How to support that together with print-debugging is the challenge. For systems in production, we have the `Logger` effect and associated handlers.
  - vlovich123 1y ago
    And yet modern optimizers don’t actually seem to have a problem with a transformation like that as you must know. Try list.iter().filter(|x| x>5).count() in Rust And yes, Rust doesn’t have an effect system yet, but others have mentioned Haskell and how it handles tracing and logging and the limitations of effect systems interplaying with such things.
    - jorkadeen 1y ago
      It was a simple example; whether a specific optimization applies is very tricky. We have to look at the details. When can Rust move or eliminate a binder? Does Rust support automatic parallelization? What happens if you use unsafe blocks to lie to their type and ownership system? I think many of the same issues will surface. To be me, the interesting question is: What happens when you lie to the type (and effect or ownership) system?
meisel 1y ago
How does Flix decide if it’s worth it to auto parallelize a map operation? Eg, if it’s a very trivial map operation, it may add overhead to parallelize it
meisel 1y ago
Is there a target use case(s) for this language, at least initially? It seems to focus on programming in a platform-independent sense, but where is it most likely to actually succeed? Are there particular OS’s or other languages that they try be particularly compatible with?
- wk_end 1y ago
  It runs on the JVM, so I suppose that's the "OS" it's compatible with and Java is the language. It's still mostly academic AFAICT, but I could see experimenting with it for backend stuff.
matheusmoreira 1y ago
To do this you made the Debug effect a special case in the compiler. Won't users want to create their own special effect types? Could these special effects be expressed within the language itself instead of being compiler builtins?
- jorkadeen 1y ago
  We can certainly extend it to a language feature-- if there are good use cases. Do you have some specific special effects in mind?
  - morcus 1y ago
    Disclaimer: I don't know much about this programming language or about Effects, so there may be a better way to do this already something I'd sometimes like to do when I'm profiling complex code will be to have an (essentially) global variable tracking the sum of how long a function took to execute over all invocations. I am guessing that mutating the global counter would count as an effect, and I wouldn't really want to add the effect all the way through the call graph. I think this is something where the handling ought to be similar to how you're handling Debug.
troupo 1y ago
That's why I don't like "purity above all costs" languages. It's not just print debugging. It's also logging. Oh, you want/have to add useful/required logging? Good luck refactoring your entire code to thread IO everywhere.
bstudios 1y ago
[dead]

91 points by degurechaff 1y ago

I wonder if this will wind up being a category of problems and the solution is a separate "system" set of effects (effectively `({user}, {system})`) or if this one-off extension is all that will be needed. Either way, extremely well explained both in motivate and implementation!

epolanski 1y ago
I think the solution is to not have side effects in pure functions really. At some point you're composing your pure functions and _have_ to call them by some effectful function (otherwise they're never executed or you're doing computations that aren't consumed by anything). The only sane alternative is to have a debug effectful variant where you turn off these checks. But then why would `stdout` debugging fine, and not say writing to a different stream or file?
vilunov 1y ago
> I wonder if this will wind up being a category of problems and the solution is a separate "system" set of effects (effectively `({user}, {system})`) or if this one-off extension is all that will be needed. It already is, kinda. In my practice very often you have global singleton values that are either defined as static variables, or passed as arguments to nearly all functions in a module. Since implicit presence of `Debug` effect is already a compilation parameter, it could be generalized to support any sets of implicit effects. Thus you might design a module that has implicit Logger and Database effects in all its functions.
- epolanski 1y ago
  Singleton pattern isn't really used in functional-effect systems, dependency injection is generally used instead.
- yccs27 1y ago
  > Thus you might design a module that has implicit Logger and Database effects in all its functions. Logging does seem like a very similar case to debugging, only that you expect to leave it on in production. On the other hand an implicit Database effect kind of defeat the point of an effect system. I think the key is that Debug and Logger effects don't really affect the rest of the code - if you remove all debug/log statements, the only thing that changes is the debug/log output (and slightly faster execution probably).
  - saghm 1y ago
    > Logging does seem like a very similar case to debugging, only that you expect to leave it on in production In a lot of logging systems, debug is one of the common levels of logging. I'm not even convinced that the term "debug" is unambiguous to clearly refer to something that's not a subset logging. Presumably the difference is intended to mean things printed that are unconditionally going to stdout rather than into some system that might change the output location and filter/annotate things, but I have to imagine that it might just make more sense not to have separate models for them at all
    - yccs27 1y ago
      I was referring to the specific Debug effect described in the post, which only works in debugging builds. This means that the production build cannot have unexpected output from a function with no declared effect.
      - saghm 1y ago
        I understand. My point was that if you did want to extend further to have an effect for logging, it seems like having fully separate abstractions for them rather than finding a way to express them both as part of the same abstraction would make sense.
jerf 1y ago
The other major use case that leaps to mind is "observability"; I want to be able to poke a metric without the function becoming impure, since it is not uncommon for something deep down the call stack to poke a metric and I don't want it to propagate up the stack. You can also make a case for logging that isn't just debug logging. Propagating up a new effect just because some deep function needs to push a log is not very friendly or useful. Perhaps there is two or three more, but I do think this is a finite set that we can write a language around and just sort of consider them "ambient effects". While metrics and logging are nominally impure, in that they are certainly mutations, if you can't read the logs or the metrics without an effect, you still retain the really important aspect of purity, which is that the pure code can't cause a change that is observable by that or other code, with the very specific exception of the logging stream and metrics. I wouldn't be quite ready to put all my chips on this, but I think "the inability to create changes that can be witnessed" is actually the true goal, not "the inability to create changes" with no qualifications. Pure code already necessarily creates changes in a system, the key is that while the CPU registers may change and other parts of the system may mutate, the code can't witness those changes and conditionalize future execution on it. All effects-based systems have already agreed that there are things that are mutations in something real in the physical world they aren't going to consider effects, adding a couple more categories is not going from 0 to 1 but 12 to 15. It's not a strict purity question but a cost/benefits question. It occurs to me as I type this that a really ambitious language with a strong enough type system might even be able to turn this into a completely safe proposition, allowing users to declare effects with some sort of very safe "sink" associated with them that the type system checks can not escape out into the rest of the code in any visible way and constraining the visibility somewhere that is contained in the conventional effects system. All these things I'm talking about here and in the blog post are taking the form of values that simply disappear into the ether from the point of view of the creating function. I'd like the language to make it easy to query a function for which ambient effects it uses (as a development-time operation, not a run-time one), and I think that would clean up most of the rest of the practical problems.
- ramses0 1y ago
  100% well said! Exactly how to handle log/print/metrics (as results/returns from functions) has been on my mental back burner for years. It's proper (but goofy) to pass in a `logger` object to every function, but the practicality of plumbing a "logger" everywhere is disgusting. There's a ton of value to be able to "capture" what was logged by a function, but as you mentioned with "metrics", the invasiveness of plumbing a metrics object "through" your code is really gross. Maybe a straw man syntax like: this.__classHooks__.logger = new AbstractLogger(...) this.__classHooks__.metrics = new AbstractMetrics(...) ...like a puzzle piece, if the container (parent object) wants to "attach" to the exported interface, it can kindof dependency injection, otherwise logs and metrics might fly off into the void.

throwawaymaths 1y ago

Honestly a pragmatic solution would be to have printing to stderr (debug print, logs, whatever) not be an "effect". then as an added benefit if its context is elided out in an optimization you know.

jorkadeen 1y ago
The challenge is that the compiler uses the type-and-effect system for many of its tasks, including whole-program optimization and code generation. If printing- or logging statements have no effect, the compiler might reorder them or even remove them.
maleldil 1y ago
This is similar to what Haskell has with trace[1]. It pretends to be a pure function (so it doesn't need the IO monad), but prints to stderr. [1] https://hackage.haskell.org/package/base-4.21.0.0/docs/Debug-Trace.html#v:trace https://hackage.haskell.org/package/base-4.21.0.0/docs/Debug...
- JoelMcCracken 1y ago
  And, just like tfa alludes, it is awkward because of laziness; in non trivial scenarios it can be quite surprising how your trace isn’t tracing. This is bc of non strict eval in Haskell, but still the core is that the lang is designed to not eval stuff when it’s not necessary.
naasking 1y ago
It needs to be an effect so the compiler knows how optimizations are permitted to handle it. Otherwise your print statements might appear in totally the wrong order or even not at all. Imagine saying that certain memory stores are allowed to cross a memory barrier operation, you'd completely lose the ability to reason about concurrency around those stores.
- throwawaymaths 1y ago
  debugging must be a nightmare if you can't get a good mental model of how the compiler is transforming your code

noreplydev 1y ago

As I understand, Flix implements a system to type not only a return type but an Effect too. If that's the case, It's not an overkill solution? and why?

wk_end 1y ago
It makes it easier to reason about a program. For an dev, you can immediately understand what global state, if any, can affect a function’s behaviour. And for the compiler, knowing about what effects an operation has can enable optimizations like dead code removal and autovectorization; the article goes into that.

draw_down 1y ago

[dead]

podgorniy 1y ago

Cool idea to "type" side effects I'm curious what other languages are trying to achieve such?

jorkadeen 1y ago
A few other languages are: - https://effekt-lang.org/ https://effekt-lang.org/ - https://koka-lang.github.io/ https://koka-lang.github.io/ - https://www.unison-lang.org/ https://www.unison-lang.org/ - https://antelang.org/ https://antelang.org/
- ux266478 1y ago
  Oh I hadn't heard of Ante before. This looks very close to the language I wanted out of Rust. Haskell's module system, row polymorphism, linear types, no sepples glyph soup. That's an instant bookmark save. Will be watching that space very closely.
ux266478 1y ago
Koka[1] jumps to mind as a language built around effect types. Without having used Flex or Koka, I'm not sure how their effect types actually differ from the normal monad gauntlet you get in pure languages. [1] - https://koka-lang.github.io/koka/doc/index.html https://koka-lang.github.io/koka/doc/index.html
davidatbu 1y ago
Koka, Roc-lang come to mind.
ookdatnog 1y ago
AFAIK the first somewhat widely-known language to do this was Haskell (through libraries). I'm not 100% clear on the entire history, but I think it goes something like: 1. Initially there was no way to do effects in Haskell, everything was pure. 2. Then it was realized that IO can be modeled with monads, so the IO type and do notation were added. 3. Gradual realization that monads can be used to also constrain effects, ie you can construct a type of "stateful computations" that can read and write to a specific state, but not touch other states or write to disk or something. 4. Monad transformers are invented, which allow stacking monads on top of eachother to support multiple effects. Together with type classes, this gets us pretty close to extensible effects (the approach used in Flix, if I understand it correctly). So for example you can express that your function needs to write to a log and may exit early with an error message with the constraints `(MonadWriter w m, MonadError e m) => ... -> m resultType`, and you can then use monad transformers to build a stack that provides both of these effects. 5. Monad transformers have some issues though: they affect performance significantly and the interaction between effects is tricky to reason about. So an alternative is sought and found in extensible effects. The initial proposals were, iirc, based on free monads, but those aren't great for performance either, so ever since there has been a whole zoo of different effects and handlers implementations that all make different trade-offs and compromises, of which I think the `effectful` library is now the de facto default, and I think what it offers is quite similar to the Flix language's effect system (I'm not sure on what finer points it differs).

ducdetronquito 1y ago

Pleasant and interesting read !

vlovich123 1y ago

> Hence, when the compiler is run in production mode, we disable the lie that allows the implicit Debug effect. As a result, using dprintln in production mode causes a compilation error. The team can’t seem to make up its minds of the language is intended for high performance or not. They talk about the importance of purity for automatic optimizations but in the real world there’s all sorts of practical reasons for needing to debug production compiled code (eg imagine something like a browser and you’re trying to figure out some weird behavior that’s difficult to catch in a debugger but too slow to reproduce in a debug build or even not reproducible due to different timings resulting in different race conditions) Also blaming the users of your language for your language not being able to meet their needs isn’t a good look. It suggests the language is probably attracting the wrong users or positioning itself incorrectly in the market place.

jorkadeen 1y ago
We think that functional programmers should be able to write e.g. `List.count(x -> x > 5, l)` (or e.g. use pipelines with |>) and have it run as fast as an ordinary imperative loop with a mutable variable. The Flix compiler gives them that-- but it requires the program to undergo certain transformations that may require expressions to be moved around and eliminated. It is dangerous to perform such optimizations with incorrect assumptions about types or effects. How to support that together with print-debugging is the challenge. For systems in production, we have the `Logger` effect and associated handlers.
- vlovich123 1y ago
  And yet modern optimizers don’t actually seem to have a problem with a transformation like that as you must know. Try list.iter().filter(|x| x>5).count() in Rust And yes, Rust doesn’t have an effect system yet, but others have mentioned Haskell and how it handles tracing and logging and the limitations of effect systems interplaying with such things.
  - jorkadeen 1y ago
    It was a simple example; whether a specific optimization applies is very tricky. We have to look at the details. When can Rust move or eliminate a binder? Does Rust support automatic parallelization? What happens if you use unsafe blocks to lie to their type and ownership system? I think many of the same issues will surface. To be me, the interesting question is: What happens when you lie to the type (and effect or ownership) system?

meisel 1y ago

How does Flix decide if it’s worth it to auto parallelize a map operation? Eg, if it’s a very trivial map operation, it may add overhead to parallelize it

meisel 1y ago

Is there a target use case(s) for this language, at least initially? It seems to focus on programming in a platform-independent sense, but where is it most likely to actually succeed? Are there particular OS’s or other languages that they try be particularly compatible with?

wk_end 1y ago
It runs on the JVM, so I suppose that's the "OS" it's compatible with and Java is the language. It's still mostly academic AFAICT, but I could see experimenting with it for backend stuff.

matheusmoreira 1y ago

To do this you made the Debug effect a special case in the compiler. Won't users want to create their own special effect types? Could these special effects be expressed within the language itself instead of being compiler builtins?

jorkadeen 1y ago
We can certainly extend it to a language feature-- if there are good use cases. Do you have some specific special effects in mind?
- morcus 1y ago
  Disclaimer: I don't know much about this programming language or about Effects, so there may be a better way to do this already something I'd sometimes like to do when I'm profiling complex code will be to have an (essentially) global variable tracking the sum of how long a function took to execute over all invocations. I am guessing that mutating the global counter would count as an effect, and I wouldn't really want to add the effect all the way through the call graph. I think this is something where the handling ought to be similar to how you're handling Debug.

troupo 1y ago

That's why I don't like "purity above all costs" languages. It's not just print debugging. It's also logging. Oh, you want/have to add useful/required logging? Good luck refactoring your entire code to thread IO everywhere.

bstudios 1y ago

[dead]