7 ms·

Structuring Clojure applications

198 points by nefreat 4y ago

TacticalCoder 4y ago
A similar post, also mentioning protocols and integrant, was posted here a few days ago and may also interest some: https://mccue.dev/pages/12-7-22-clojure-web-primer https://mccue.dev/pages/12-7-22-clojure-web-primer
dwohnitmok 4y ago
It's interesting to see a lot of FP communities independently arriving at the same architectural structures. See e.g. Haskell's "handle:" https://jaspervdj.be/posts/2018-03-08-handle-pattern.html https://jaspervdj.be/posts/2018-03-08-handle-pattern.html
- seanc 4y ago
  Convergent design is certainly a thing, but it's also quite possible that communities are cross-pollinating ideas. Either way, when you see those recurring patterns that's certainly a strong indicator that one should pay attention.
logistark 4y ago
For me, protocols i tend to not use it, because it makes it harder to understand the code and Cursive cannot find instances that implements the protocols. For testing purposes is easier to redef a function than implementing a full new test protocol.
- yogthos 4y ago
  I agree that in most cases with-redefs works fine, and I tend to use protocols sparingly myself for the same reasons. I thought it was worth mentioning that you can use them to encapsulate any effectful code since they can be used as a tool to help enforce a bit of discipline. If you're just calling functions that can cause side effects then it can get tricky to figure out what all the functions that need to be redefined are. You basically have to read through all the code to know what you need to mock. If you stick all the side effects in a protocol, then you're being very explicit about what needs to be mocked out.
haolez 4y ago
I'm an experienced developer and I'm getting the feeling that advanced languages are getting less relevant for most applications, since you usually just need a little glue code to glue together mainstream solutions or managed services. I don't need the power of Clojure to connect SQS to Lambda with some extra custom logic. But Clojure does look amazing :)
- ballpark 4y ago
  Interesting point. I've recently taken on a client who insists on using C# with their cloud solution. It's killing me. Though you're calling Clojure "advanced", what I'm missing is its simplicity.
  - haolez 4y ago
    Maybe a better classification would be "amazing languages" vs "good enough ubiquitous languages" :)
- yogthos 4y ago
  For me it's mostly a quality of life issue. I find Clojure workflow is far more pleasant than most languages because it's interactive, and I like Clojure as a language because it's small and focused. I find the main feature of Clojure lies in its simplicity as opposed to advanced features. Clojure code tends to be very direct where you're just passing data through a series of transformations, and you apply a few common patterns to solve a wide range of problems.
- austinbirch 4y ago
  I can see why Clojure is seen that way ("advanced", "powerful"), but a big piece of it’s design is Rich Hickey’s view that "we can make the same software we're making today with dramatically simpler stuff." Learning Clojure has made working in other languages much harder for me - pretty much all of my Clojure programming is functions plus transforming data structures (mostly maps). So few concepts to keep in your head! The majority of my work is in TypeScript these days, and I always write the most Clojure-y TypeScript I can manage.
dig1 4y ago
I am not fond of the multimethods because they can easily tangle the code and give you a false sense of scalability. For example, in a blog post, "handle-action" is nicely decoupled with 3 different actions, but let's imagine how that will look after someone adds 20 new actions. Good luck debugging that. Also, I saw numerous cases where people will copy/paste multimethod arguments without knowing what they are used for. I still find case/cond more readable and way more performant, especially since the author uses the same type for a multimethod dispatch, but YMMV.
- Folcon 4y ago
  case/cond is pretty readable, but I wouldn't say they provide a comparable api to multimethods. From my perspective, I reach for multimethods when I want to provide my caller with the ability to declare their own "actions" as it were. The goal is to offer an open interface. Without that need, yes, case or cond can be sufficient.
- escherize 4y ago
  Case and cond are nice, but they suffer from the expression problem. When you want to let other people add methods to your code without modifying the source you need to use multimethods (or protocols). If your method only needs 1 argument then why should the other ones matter? I don't see a problem here. clj-kondo will guide people to name them as _ or _thing anyway so you don't even need to think about it.
- fulafel 4y ago
  Multimethods also have some problems vs reloading that need working around.
twawaaay 4y ago
I love Clojure and also Common Lisp (basically, Lisp in general). But I also observed every single corporate Clojure project I had any connection with to fail spectacularly. Typically as a complete unmaintainable mess. Some as unmaintainable mess that is very slow and unreliable. My theory is that this is result of no guardrails on how to structure your application. Clojure to be productive must be used by people who absolutely know what they are doing when it comes to structuring your app. When it comes to Java you get hordes of devs still producing passable results. The structure is largely imposed by the frameworks (mostly by Spring/Spring Boot) and available help, literature. Even some antipatterns at the very least achieve some level of convention/predictability that is needed to be able to find your bearing around the codebase. I would say, if you have a normal-ish project, care about productivity but don't have really stellar and mature developers -- skip Clojure. Choose Clojure if you know how to use all that additional power, have need for it and understand what your added responsibilities are. If you don't know how to wield Clojure's power there is very little you can gain by choosing it but a lot to loose.
- agumonkey 4y ago
  Interesting take. How much of these architectural problems came from - process / workflow (everybody goes on hacking a topic for weeks and then integrate ?) - team size (I can foresee how clojure would make large team a problem, but small teams happier)
- dustingetz 4y ago
  i think you're measuring the economic climate of the last two decades and it's impact on management practices, hiring/recruiting etc. which is basically: scale scale scale money money scale, oh and fuck people. Clojure is not for those teams. Clojure is for those of us who reject that
- rockyj 4y ago
  Same problem exists for JS/TS projects btw. You can write anything, anywhere in any style suited to the developer. I guess that is why coporates like Java so much, it commoditizes developers, but can also rock the boat on the other side - cargo-cult programming with no original ideas or innovation.
nkh 4y ago
Any one tried Polylith with the multi-method approach mentioned in the article?
beders 4y ago
This approach is not simple. It complects a business process state with multi-methods. Don't do it. Model state where it belongs: in a database.
- yogthos 4y ago
  This approach is perfectly compatible with modelling state in the database. The problem this addresses is the data flow, which is a completely separate problem from managing the state itself.
rmuslimov 4y ago
This is very informative and beginner friendly write up which came be used as strategy for organizing apps in Clojure. I personally have something very similar which Redis storage used as persistent storage for storing jobs and tasks within workflow are potentially executed on diff hosts. I would recommend to extending this topic and share your thoughts about component/mount like abstraction to the code. For example notify-sender should have credentials to connect to the services. How they are delivered? as input to the :handle-action method or as as component/mount. Interesting to learn about your approach here. Thanks for sharing!
- yogthos 4y ago
  Glad to hear this was helpful, and regarding managing stuff like credentials, I like the pattern of initializing the client up front using component, mount. Then passing the client in the resource map. The advantage over simply passing credentials is that the initializer for the client can validate itself when it loads. If you just pass the credentials and assemble the client when you try to send the message then if some variables weren't set correctly you only find out when you try to use the client at as opposed to when application starts.
shaunparker 4y ago
I haven't looked at Clojure in a bit, but shouldn't the (nil? to-info) check in the transfer implementation of handle-action come first? It seems like that would never be reached in the current implementation.
- yogthos 4y ago
  Oh yeah you're totally right. :)
Jamesmoorez 4y ago
the-alchemist 4y ago
It seems like a lot of the anti-Clojure sentiment boils down to 1) lack of static typing, 2) poor IDE support. I'm wondering, though, doesn't the same apply to Ruby, Python, and Node projects? I've over-hauled 80k line Python projects, and the "lack of typing" there seemed to apply as well. Why don't Ruby, Python, and Node projects suffer from the same critique? Genuinely curious...
- fulafel 4y ago
  They do get the same themed critique from static language proponents, even more so since JS/Python don't have culture of using schema systems (ala spec and malli). But of course dynamic languages have a lot of upsides as well, it's just a tradeoff.

198 points by nefreat 4y ago

A similar post, also mentioning protocols and integrant, was posted here a few days ago and may also interest some: https://mccue.dev/pages/12-7-22-clojure-web-primer https://mccue.dev/pages/12-7-22-clojure-web-primer

dwohnitmok 4y ago

It's interesting to see a lot of FP communities independently arriving at the same architectural structures. See e.g. Haskell's "handle:" https://jaspervdj.be/posts/2018-03-08-handle-pattern.html https://jaspervdj.be/posts/2018-03-08-handle-pattern.html

seanc 4y ago
Convergent design is certainly a thing, but it's also quite possible that communities are cross-pollinating ideas. Either way, when you see those recurring patterns that's certainly a strong indicator that one should pay attention.

logistark 4y ago

For me, protocols i tend to not use it, because it makes it harder to understand the code and Cursive cannot find instances that implements the protocols. For testing purposes is easier to redef a function than implementing a full new test protocol.

yogthos 4y ago
I agree that in most cases with-redefs works fine, and I tend to use protocols sparingly myself for the same reasons. I thought it was worth mentioning that you can use them to encapsulate any effectful code since they can be used as a tool to help enforce a bit of discipline. If you're just calling functions that can cause side effects then it can get tricky to figure out what all the functions that need to be redefined are. You basically have to read through all the code to know what you need to mock. If you stick all the side effects in a protocol, then you're being very explicit about what needs to be mocked out.

haolez 4y ago

I'm an experienced developer and I'm getting the feeling that advanced languages are getting less relevant for most applications, since you usually just need a little glue code to glue together mainstream solutions or managed services. I don't need the power of Clojure to connect SQS to Lambda with some extra custom logic. But Clojure does look amazing :)

ballpark 4y ago
Interesting point. I've recently taken on a client who insists on using C# with their cloud solution. It's killing me. Though you're calling Clojure "advanced", what I'm missing is its simplicity.
- haolez 4y ago
  Maybe a better classification would be "amazing languages" vs "good enough ubiquitous languages" :)
yogthos 4y ago
For me it's mostly a quality of life issue. I find Clojure workflow is far more pleasant than most languages because it's interactive, and I like Clojure as a language because it's small and focused. I find the main feature of Clojure lies in its simplicity as opposed to advanced features. Clojure code tends to be very direct where you're just passing data through a series of transformations, and you apply a few common patterns to solve a wide range of problems.
austinbirch 4y ago
I can see why Clojure is seen that way ("advanced", "powerful"), but a big piece of it’s design is Rich Hickey’s view that "we can make the same software we're making today with dramatically simpler stuff." Learning Clojure has made working in other languages much harder for me - pretty much all of my Clojure programming is functions plus transforming data structures (mostly maps). So few concepts to keep in your head! The majority of my work is in TypeScript these days, and I always write the most Clojure-y TypeScript I can manage.

dig1 4y ago

I am not fond of the multimethods because they can easily tangle the code and give you a false sense of scalability. For example, in a blog post, "handle-action" is nicely decoupled with 3 different actions, but let's imagine how that will look after someone adds 20 new actions. Good luck debugging that. Also, I saw numerous cases where people will copy/paste multimethod arguments without knowing what they are used for. I still find case/cond more readable and way more performant, especially since the author uses the same type for a multimethod dispatch, but YMMV.

Folcon 4y ago
case/cond is pretty readable, but I wouldn't say they provide a comparable api to multimethods. From my perspective, I reach for multimethods when I want to provide my caller with the ability to declare their own "actions" as it were. The goal is to offer an open interface. Without that need, yes, case or cond can be sufficient.
escherize 4y ago
Case and cond are nice, but they suffer from the expression problem. When you want to let other people add methods to your code without modifying the source you need to use multimethods (or protocols). If your method only needs 1 argument then why should the other ones matter? I don't see a problem here. clj-kondo will guide people to name them as _ or _thing anyway so you don't even need to think about it.
fulafel 4y ago
Multimethods also have some problems vs reloading that need working around.

twawaaay 4y ago

I love Clojure and also Common Lisp (basically, Lisp in general). But I also observed every single corporate Clojure project I had any connection with to fail spectacularly. Typically as a complete unmaintainable mess. Some as unmaintainable mess that is very slow and unreliable. My theory is that this is result of no guardrails on how to structure your application. Clojure to be productive must be used by people who absolutely know what they are doing when it comes to structuring your app. When it comes to Java you get hordes of devs still producing passable results. The structure is largely imposed by the frameworks (mostly by Spring/Spring Boot) and available help, literature. Even some antipatterns at the very least achieve some level of convention/predictability that is needed to be able to find your bearing around the codebase. I would say, if you have a normal-ish project, care about productivity but don't have really stellar and mature developers -- skip Clojure. Choose Clojure if you know how to use all that additional power, have need for it and understand what your added responsibilities are. If you don't know how to wield Clojure's power there is very little you can gain by choosing it but a lot to loose.

agumonkey 4y ago
Interesting take. How much of these architectural problems came from - process / workflow (everybody goes on hacking a topic for weeks and then integrate ?) - team size (I can foresee how clojure would make large team a problem, but small teams happier)
dustingetz 4y ago
i think you're measuring the economic climate of the last two decades and it's impact on management practices, hiring/recruiting etc. which is basically: scale scale scale money money scale, oh and fuck people. Clojure is not for those teams. Clojure is for those of us who reject that
rockyj 4y ago
Same problem exists for JS/TS projects btw. You can write anything, anywhere in any style suited to the developer. I guess that is why coporates like Java so much, it commoditizes developers, but can also rock the boat on the other side - cargo-cult programming with no original ideas or innovation.

nkh 4y ago

Any one tried Polylith with the multi-method approach mentioned in the article?

beders 4y ago

This approach is not simple. It complects a business process state with multi-methods. Don't do it. Model state where it belongs: in a database.

yogthos 4y ago
This approach is perfectly compatible with modelling state in the database. The problem this addresses is the data flow, which is a completely separate problem from managing the state itself.

rmuslimov 4y ago

This is very informative and beginner friendly write up which came be used as strategy for organizing apps in Clojure. I personally have something very similar which Redis storage used as persistent storage for storing jobs and tasks within workflow are potentially executed on diff hosts. I would recommend to extending this topic and share your thoughts about component/mount like abstraction to the code. For example notify-sender should have credentials to connect to the services. How they are delivered? as input to the :handle-action method or as as component/mount. Interesting to learn about your approach here. Thanks for sharing!

yogthos 4y ago
Glad to hear this was helpful, and regarding managing stuff like credentials, I like the pattern of initializing the client up front using component, mount. Then passing the client in the resource map. The advantage over simply passing credentials is that the initializer for the client can validate itself when it loads. If you just pass the credentials and assemble the client when you try to send the message then if some variables weren't set correctly you only find out when you try to use the client at as opposed to when application starts.

shaunparker 4y ago

I haven't looked at Clojure in a bit, but shouldn't the (nil? to-info) check in the transfer implementation of handle-action come first? It seems like that would never be reached in the current implementation.

yogthos 4y ago
Oh yeah you're totally right. :)

Jamesmoorez 4y ago

the-alchemist 4y ago

It seems like a lot of the anti-Clojure sentiment boils down to 1) lack of static typing, 2) poor IDE support. I'm wondering, though, doesn't the same apply to Ruby, Python, and Node projects? I've over-hauled 80k line Python projects, and the "lack of typing" there seemed to apply as well. Why don't Ruby, Python, and Node projects suffer from the same critique? Genuinely curious...

fulafel 4y ago
They do get the same themed critique from static language proponents, even more so since JS/Python don't have culture of using schema systems (ala spec and malli). But of course dynamic languages have a lot of upsides as well, it's just a tradeoff.