9 ms·
Architecture Patterns with Python
- Pandabob 1y agoTruly one of the great python programming books. The one thing that I found missing was the lack of static typing in the code, but that was a deliberate decision by the authors.
- sevensor 1y agoHaven’t read the book, so I don’t know exactly what position they’re taking there, but type checking has done more to improve my Python than any amount of architectural advice. How hard it is to type hint your code is a very good gauge of how hard it will be to understand it later.
- pmg101 1y agoMy experience is that once people have static typing to lean on they focus much less on the things that in my view are more crucial to building clean, readable code: good, consistent naming and small chunks. Just the visual clutter of adding type annotations can make the code flow less immediately clear and then due to broken windows syndrome people naturally care less and less about visual clarity.
- natdempk 1y agoSo far off from what actually happens. The type annotations provide an easy scaffolding for understand what the code does in detail when reading making code flow and logic less ambiguous. Reading Python functions in isolation, you might not even know what data/structure you’re getting as input… if there’s something that muddles up immediate clarity it’s ambiguity about what data code is operating on.
- sevensor 1y agoExactly my experience. I call Python a surprise-typed language. You might write a function assuming its input is a list, but then somebody passes it a string, you can iterate over it so the function returns something, but not what you would have expected, and things get deeply weird somewhere else in your codebase as a result. Surprise! Type checking on the other hand makes duck typing awesome. All the flexibility, none of the surprises.
- zahlman 1y agoThis is because of Python's special handling of iteration and subscripting for strings (so as to avoid having a separate character type), not because of the duck typing. In ordinary circumstances (e.g. unless you need to be careful about a base case for recursion - but that would cause a local fault and not "deep weirdness at a distance"), the result is completely intuitive (e.g. you ask it to add each element of a sequence to some other container, and it does exactly that), and I've written code that used these properties very intentionally. If you passed a string expecting it to be treated as an atomic value rather than as a sequence (i.e. you made a mistake and want a type checker to catch it for you), there are many other things you can do to avoid creating that expectation in the first place.
- tayo42 1y agoType annotations are just like documentation though. Just because the annotation says int the function can still return a list.
- maleldil 1y agoAnnotations can and should be checked. If I change a parameter type, other code using the function will now show errors. That won't happen with just documentation.
- tayo42 1y agoIn some cases don't you need to actually execute the code to know what the type actually is. How does the type checker know then?
- maleldil 1y agoIt doesn't. There are cases where the type-checker can't know the type (e.g. json.load has to return Any), but there are tools in the language to reduce how much that happens. If you commit to a fully strictly-typed codebase, it doesn't happen often.
- sevensor 1y agoYou can actually annotate the return type of json.load better than that: JSON = float | bool | int | str | None | list[“JSON”] | dict[str, “JSON”]
- pansa2 1y ago> Annotations can and should be checked Unfortunately Python’s type system is unsound. It’s possible to pass all the checks and yet still have a function annotated `int` that returns a `list`.
- __MatrixMan__ 1y agoDo you mean that you're allowed to only use types where you want to, which means maybe the type checker can't check in cases where you haven't hinted enough, or is there some problem with the type system itself?
- zahlman 1y ago>So far off from what actually happens I disagree strongly, based on 20 years of using Python without annotations and ~5 years of seeing people ask questions about how to do advanced things with types. And based on reading Python code, and comparing that to how I feel when reading code in any manifest-typed language. >Reading Python functions in isolation, you might not even know what data/structure you’re getting as input I'm concerned with what capabilities the input offers, not the name given to one particular implementation of that set of capabilities. If I have to think about it in any more detail than "`ducks` is an iterable of Ducklike" (n.b.: a code definition for an ABC need not actually exist; it would be dead code that just complicates method resolution) I'm trying to do too much in that function. If I have to care about whether the iterable is a list or a string (given that length-1 strings satisfy the ABC), I'm either trying to do the wrong thing or using the wrong language. > if there’s something that muddles up immediate clarity it’s ambiguity about what data code is operating on. There is no ambiguity. There is just disregard for things that don't actually matter, and designing to make sure that they indeed don't matter.
- pansa2 1y ago> using the wrong language IMO this is the source of much of the demand for type hints in Python. People don't want to write idiomatic Python, they want to write Java - but they're stuck using Python because of library availability or an existing Python codebase. So, they write Java-style code in Python. Most of the time this means heavy use of type hints and an overuse of class hierarchies (e.g. introducing abstract classes just to satisfy the type checker) - which in my experience leads to code that's twice as long as it should be. But recently I heard more extreme advice - someone recommended "write every function as a member of a class" and "put every class in its own file".
- sevensor 1y agoI’d say I use type hints to write Python that looks more like Ocaml. Class hierarchies shallow to nonexistent. Abundant use of sum types. Whenever possible using Sequence, Mapping, and Set rather than list, dict, or set. (As these interfaces don’t include mutation, even if the collection itself is mutable.) Honestly if you’re heavily invested in object oriented modeling in Python, you’re doing it wrong. What a headache.
- globular-toast 1y agoYeah, people from statically typed languages sometimes can't understand how dynamically typed languages can even work. How can I do anything if I don't know what type to pass?! Because we write functions like "factorial(number)" instead of "int fac(int n)".
- xboxnolifes 1y agoI wish that's how python functions were written. What i usually see is `draw(**kwargs)`.
- Pandabob 1y agoYup! I'm also hopeful that the upcoming type-checker from Astral will be an improvement over Mypy. I've found that Mypy's error messages are sometimes hard to reason about. [0]: https://x.com/charliermarsh/status/1884651482009477368 https://x.com/charliermarsh/status/1884651482009477368
- dlisboa 1y ago> The one thing that I found missing was the lack of static typing in the code It has type hints, such as here: https://www.cosmicpython.com/book/chapter_08_events_and_message_bus.html https://www.cosmicpython.com/book/chapter_08_events_and_mess... Do you mean it's not strict enough? There are some parts of the book without them.
- zahlman 1y agoSome examples use dataclasses, which force type annotations. Python does not support static typing. Tooling based on type annotations doesn't affect the compilation process (unless you use metaprogramming, like dataclasses do) and cannot force Python to reject the code; it only offers diagnostics.
- seveibar 1y agoI’m a Typescript dev but this book is one of my favorite architecture books, I reference it all the time. My favorite pattern is the fake unit of work/service patterns for testing, I use this religiously in all my projects for faking (not mocking!!) third party services. It also helped me with dilemmas around naming, eg it recommends naming events in a very domain specific way rather than infrastructure or pattern specific way (eg CART_ITEM_BECAME_UNAVAILABLE is better than USER_NOTIFICATION). Some of these things are obvious but tedious to explain to teammates, so the fact that cosmic python is fully online makes it really easy to link to. Overall, a fantastic and formative resource for me!
- serial_dev 1y agoI haven't seen this book before, but I noticed that one of the authors, Harry J. W Percival, is the author of the TDD "goat" book. https://www.obeythetestinggoat.com/pages/book.html https://www.obeythetestinggoat.com/pages/book.html That book is in a similar place in my heart, I barely used Python in my professional life, yet it's a book I often come back to even if I'm using a different language. It's also great that book is available both online and in paper form. I'll definitely give this book a chance!
- cinntaile 1y agoI saw that a new, updated version of that book will be released this year.
- BerislavLopac 1y ago> faking (not mocking!!) You might like this: https://martinfowler.com/bliki/TestDouble.html https://martinfowler.com/bliki/TestDouble.html
- incangold 1y agoFakes over mocks every time
- GONE_KLOUT 1y agoUnfortunately https://www.cosmicpython.com/book/ https://www.cosmicpython.com/book/ does give a 404 - this is a very bad architectural choice for web applications. I hope their other tips are better.
- esafak 1y agohttps://www.cosmicpython.com/book/preface.html https://www.cosmicpython.com/book/preface.html
- globular-toast 1y agoI have this on my shelf. It's a small volume, similar to K&R, and like that book mine is showing visible signs of wear as I've thumbed through it a lot.
- barrenko 1y agoExcellent sequel to the goat book (TDD with Python), that got me to deploy my first real web application.
- SaturnIC 1y agoTDD is dysfunctional crap pushed by the lying scammer Robert Martin on inexperienced devs
- barrenko 1y agoNo argument on that from me in general, but the book in question is practical.
- wesselbindt 1y agoWhat's dysfunctional about it?
- shesstillamodel 1y agoEven though most people might think of web architectures when it comes to this book, we used this to design an architecture for an AI that optimises energy efficiency in a manufacturing factory. Great book!
- fastasucan 1y agoIs it easy to transpose to other types of architectures, or is it leaning heavily against web development? Thank you for sharing, your project sounds really interesting by the way! :)
- globular-toast 1y agoOne of the key points of the book (and DDD in general) is the web stuff is just a detail at the edge of an application. You should be able to replace the web bit (for which they use flask) with any other entry point. In fact, they do this by having an event subscriber entry point and IIRC a CLI entry point. The whole point is it all uses the same core code implementing the domain logic.
- chr1ss_code 1y agoThis was a great read and summary! About three years ago, I worked in a C#/.NET DDD environment, and now revisiting these concepts in Python really distills the essential parts. As I said, great read — highly recommend it if you're also into this kind of stuff.
- floppydiscen 1y agoGreat source for understanding how DDD works in a larger context. I love how concrete it is
- BerislavLopac 1y agoSome parts of this book are extremely useful, especially when it's talking about concepts that are more general than Python or any other specific language -- such as event-driven architecture, commands, CQRS etc. That being said, I have a number issues with other parts of it, and I have seen how dangerous it can be when inexperienced developers take it as a gospel and try to implement everything at once (which is a common problem with any collection of design patterns like this. For example, repository is a helpful pattern in general; but in many cases, including the examples in the book itself, it is a huge overkill that adds complexity with very little benefit. Even more so as they're using SQLAlchemy, which is a "repository" in its own right (or, more precisely, a relational database abstraction layer with an ORM added on top). Similarly, service layers and unit of work are useful when you have complex applications that cover multiple complex use cases; but in a system consisting of small services with narrow responsibilities they quickly become overly bloated using this pattern. And don't even get me started with dependency injection in Python. The essential thing about design patterns is that they're tools like any other, and the developers should understand when to use them, and even more importantly when not to use them. This book has some advice in that direction, but in my opinion it should be more prominent and placed upfront rather at the end of each chapter.
- kelafoja 1y agoCould you explain how repository pattern is a "huge overkill that adds complexity with very little benefit"? I find it a very light-weight pattern and would recommend to always use it when database access is needed, to clearly separate concerns. In the end, it's just making sure that all database access for a specific entity all goes through one point (the repository for that entity). Inside the repository, you can do whatever you want (run queries yourself, use ORM, etc). A lot of the stuff written in the article under the section Repository pattern has very little to do with the pattern, and much more to do with all sorts of Python, Django, and SQLAlchemy details.
- unculture 1y agoRepository pattern is useful if you really feel like you're going to need to switch out your database layer for something else at some point in the future, but I've literally never seen this happen in my career ever. Otherwise, it's just duplicate code you have to write.
- DanielVZ 1y agoWow this book is a goldmine for architecture patterns. I love how easy it is to get into a topic and quickly grasp it. Having said that, from a practical and experience standpoint, using some of these patterns can really spiral out into an increased complexity and performance issues in Python, specially when you use already opinionated frameworks like Django which already uses the ActiveRecord pattern. I’ve been in companies big and small using Python, both using and ignoring architectural patterns. Turns out all the big ones with strict architectural (n=3) pattern usage, although “clean”, the code is waaaay to complex and unnecessarily slow in tasks that at first glance should had been simple. Whereas the big companies that didn’t care for these although the code was REALLY ugly in some places (huge if-else files/functions, huge Django models with all business logic implemented in them), I was most productive because although the code was ugly I could read it, understand it, and modify the 1000 lines of if-else statements. Maybe this says something about me more than the code but I hate to admit I was more productive in the non clean code companies. And don’t get me started on the huge amount of discussions they avoided on what’s clean or not.
- porridgeraisin 1y agoMy experience matches this. It's so liberating as well. I find it easier to internalise such code in my head compared to abstraction-soup. As you can imagine, I like golang.
- exe34 1y agoMe three. I'm even happy to refactor code into a form where there's less repetition and perhaps more parametrised functions, etc. Finding my way around a soup of ultra abstracted Matryoshka ravioli is my least favourite part of programming. Instead of simplifying things, now I need to consult 12 different objects spread over as many files before I can create a FactoryFactory.
- lijok 1y agoStrict architectural pattern usage requires understanding the domain, and understanding the patterns. If you have both, navigating the codebase will be intuitive. If you don't, you'll find 1000 LOC functions easier to parse.
- eterps 1y agoI would have expected the book mentioning something about the concept of DTOs at some point. What could be the reason it doesn't?
- DeathArrow 1y agoI see Python at a nice glue language. I grew tired from the forced OOP mindset, where you have to enforce encapsulation and inheritance on everything, where you only have private fields which are set through methods. I grew tired of SOLID, clean coding, clean architecture, GoF patterns and Uncle Bob. I grew tired of the Kingdom of Nouns and of FizzBuzz Enterprise Editions. I now follow imperative or functional flows with least OOP as possible. In the rare cases I use Python (not because I don't want to, but because I mainly use .NET at work) I want the experience to be free of objects and patterns. I am not trying to say that this book doesn't have a value. It does. It's useful to learn some patterns. But don't try to fit everything in real life programming. Don't make everything about patterns, objects and SOLID.
- exe34 1y agomy favourite model is to write as many pure functions as possible, and then as many functions of 1-4 parameters that interact with the outside world, and only then create domain objects to wrap those - it keeps the unrelated complexity out of the domain and then I can also reuse those functions without having to create the entire object that I don't always need.
- DeathArrow 1y agoI am not convinced that domain driven design works. Objects doesn't model the real world well. Why we should think DDD model the real world or a business well? And why do we even need to model something? Computers are different than humans. I think we should be pragmatic and come with the best solution in terms of money/time/complexity. Not trying to mimick human thought using computers. After all a truck isn't mimicking horse and carriage. A plane isn't mimicking a bird.
- supriyo-biswas 1y agoAt its core, objects are just containers for properties, and exploiting that fact leads to easily understood systems than the one without. For example, at work I'm currently refactoring a test for parsing the CSV output of a system; as it stands it depends on hardcoded array indexes, which makes the thing a mess. Defining a few dataclasses here and there to model each entry of the CSV file, and then writing the test with said objects has made the test much more pleasant and easily understood.
- jjice 1y agoOh neat, I read the paper back of this book maybe two and a half or three years or so ago. I enjoyed it quite a bit. They do a good job at keeping tests a first class topic and consistently updating them with each addition. Some older architecture books don't treat testing as being as high up in their priorities. I've just found that having tests ready, easy to write, and easy to update, makes the development process more enjoyable for me since it's less manual for running the code to check for issue - tighter feedback look I guess. I will say that some of the event oriented parts of this book were very interesting, but didn't seem as practical to implement in my current work.
- localghost3000 1y agoI started writing python professionally a few years ago. Coming from Kotlin and TypeScript, I found the language approachable but I was struggling to build things in an idiomatic fashion that achieved the loose coupling and testability that I was used to. I bought this book after a colleague recommended it and read it cover to cover. It really helped me get my head around ways to manage complexity in non trivial Python codebases. I don’t follow every pattern it recommends, but it opened my eyes to what’s possible and how to apply my experience in other paradigms to Python without it becoming “Java guy does Python”. I cannot recommend it enough. Worth every penny.
- ctrlp 1y ago[dead]
- iLemming 1y agoNo mentioning of of https://polylith.gitbook.io/polylith https://polylith.gitbook.io/polylith? Is it related at all?
- slt2021 1y agoGreat stuff, thank you for sharing
- throw1222212121 1y agoHmm let's see we're going to - Reimplement SQLAlchemy models (we'll call it a "repository") - Reimplement SQLAlchemy sessions ("unit of work") - Add a "service layer" that doesn't even use the models -- we unroll all the model attributes into separate function parameters because that's less coupled somehow - Scatter everything across a message bus to remove any hope of debugging it - AND THIS IS JUST FOR WRITES! - For reads, we have a separate fucking denormalized table that we query using raw SQL. (Seriously, see Chapter 12) Hey, let's see how much traffic MADE.com serves. 500k total visits from desktop + mobile last month works out to... 12 views per MINUTE. Gee, I wish my job was cushy enough that I could spend all day writing about "DDD" with my thumb up my ass.
- florbo 1y agoI've made it through about 75% of the book and have never gotten the sense that they think everything discussed in the book is something you should always do. Each pattern discussed has a summary of pros and cons. While they may be a bit lacking, they clearly articulate the fact that you should be thinking whether or not the pattern matches the application's needs. I don't think there's many applications that will require everything in the book but there are certaintly many applications that could apply one or more patterns discussed.
- theoreticalmal 1y agoYikes
- globular-toast 1y agoOK so show us how to write software for a complex business properly. Oh, I see, it's a throwaway account. This is just drive-by negativity with zero value.
- moi2388 1y ago“ If you’re reading this book, we probably don’t need to convince you that Python is great” I actually do. It’s slow, buggy and not type safe. Everything good about Python is actually C, namely the good packages. They’re not written in Python, because Python is shit.
- throwawaysjskdk 1y agoI don’t understand the need for most of the patterns described in this book. Why abstract away SQLalchemy using a Repository when it is already an abstraction over the database? What’s the purpose of the unit of work? To me hand rolling SQL is much more maintainable than this abstraction soup