7 ms·

Yeah it because the Erlang VM have a preemptive scheduler and each process have a mailbox which gets a designated amount of time to access cpu. You can do a fo

Yeah it because the Erlang VM have a preemptive scheduler and each process have a mailbox which gets a designated amount of time to access cpu. You can do a for(1) for loop and it won't hog the cpu because it get premptive. The problem is when you have large amount of problematic mails in the mailbox....

lostcolony 6y ago
So having written production Erlang, I have literally never had an issue with large amounts of problematic mails in the mailbox. It's like, yeah, technically it's possible, since there's no forced constraints around what kinds of messages a given process can be sent (which is intentional; after all, the process may enter a later receive statement that knows how to handle those kinds of messages), but it's really easy to do correctly. And you can always program defensively if you want to limit the kinds of messages you respond to by having a wildcard pattern match at the end of a receive.
- filmor 6y ago
  Overfull mailboxes can be problematic even with a wildcard `receive` and can happen if some branch of the control flow is slower than expected. We have some places in our code that essentially do manual mailbox handling by merging similar messages and keeping them in the process state until it's ready to actually handle them.
  - lostcolony 6y ago
    Right, but I wouldn't characterize that as problematic messages...rather problematic handling, or systemic issues with load. I.e., my fix does not change the format of the messages. Semantics, perhaps, but a difference between "this design decision of Erlang's caused me problems" and "this architectural/design decision of mine caused me problems"
- toast0 6y ago
  If you're not running into overfull mailboxes, you're not trying hard enough! :D Usually, it's some process that's gotten itself into a bad state and isn't making any progress (which happens, we're human), so ideally you had a watchdog on it it, to kill it if it makes no progress, or flag for human intervention, or stop sending it work, cause it's not doing any of it. But sometimes, you get into a bad place where you've got much more work coming in than your process can handle, and it just grows and grows. This can get worse if some of the the processing involves send + receive to another process and the pattern doesn't trigger the selective receive optimization where the current mailbox is marked at make_ref() and receive only checks from the marker, instead of the whole thing. If you miss that optimization, the receives will check the whole mailbox, which takes a long time if the mailbox is large, and tends to make you get further behind, making it take longer, etc, until you lose all hope of catching up, and eventually run out of memory; possibly triggering process death if you've set max_heap_size on the process (although where I was using Erlang we didn't set that), or triggering OS process death of BEAM when it can't get more ram because allocation failed or OOM killer, or triggering OS death, if it has trouble when BEAM sucks in all the memory and can't muster its OOM killer in time and just gets stuck or OOM kills the wrong thing.
  - lostcolony 6y ago
    Yep; as I mentioned elsewhere, I'm not discounting mailboxes filling faster than you can handle them, but rather the idea that you have 'problematic mails in the mailbox' - I've never had problematic mail in the mailbox. I've certainly seen mailboxes grow, because they were receiving messages faster than I was processing them, and i had to rethink my design. But that isn't an issue with the mail going into the mailbox; that's an issue with my not scaling out my receivers to fit the load. As I mentioned elsewhere, that may seem like semantics, but to me it isn't; it means the language's working that way makes sense, and it's rather an issue that I created a bottleneck with no way to relieve the pressure (as compared to messages that can't be handled and just sit there causing receives to take longer and longer over time).
    - toast0 6y ago
      Oh, I think I see. I don't think there's such thing as a problematic mail for BEAM really, mails are just terms, and BEAM handles terms, no problem. A mail that contained a small slice of a very large binary, would of course keep the whole referenced binary hanging around, which could be bad, or a large tuple could be bad because if you tried to inspect the message queue through process_info in another process, that would be a big copy. But I think maybe the original poster just meant lots of bad mail in the mailbox to mean mail that would take a long time to process, because of how the receiving process will handle it. Or, possibly bad mail meaning (as you suggest, perhaps), mail that won't be matched by any receive, resulting in longer and longer receives when they need to start from the top.

lostcolony 6y ago

So having written production Erlang, I have literally never had an issue with large amounts of problematic mails in the mailbox. It's like, yeah, technically it's possible, since there's no forced constraints around what kinds of messages a given process can be sent (which is intentional; after all, the process may enter a later receive statement that knows how to handle those kinds of messages), but it's really easy to do correctly. And you can always program defensively if you want to limit the kinds of messages you respond to by having a wildcard pattern match at the end of a receive.

filmor 6y ago
Overfull mailboxes can be problematic even with a wildcard `receive` and can happen if some branch of the control flow is slower than expected. We have some places in our code that essentially do manual mailbox handling by merging similar messages and keeping them in the process state until it's ready to actually handle them.
- lostcolony 6y ago
  Right, but I wouldn't characterize that as problematic messages...rather problematic handling, or systemic issues with load. I.e., my fix does not change the format of the messages. Semantics, perhaps, but a difference between "this design decision of Erlang's caused me problems" and "this architectural/design decision of mine caused me problems"
toast0 6y ago
If you're not running into overfull mailboxes, you're not trying hard enough! :D Usually, it's some process that's gotten itself into a bad state and isn't making any progress (which happens, we're human), so ideally you had a watchdog on it it, to kill it if it makes no progress, or flag for human intervention, or stop sending it work, cause it's not doing any of it. But sometimes, you get into a bad place where you've got much more work coming in than your process can handle, and it just grows and grows. This can get worse if some of the the processing involves send + receive to another process and the pattern doesn't trigger the selective receive optimization where the current mailbox is marked at make_ref() and receive only checks from the marker, instead of the whole thing. If you miss that optimization, the receives will check the whole mailbox, which takes a long time if the mailbox is large, and tends to make you get further behind, making it take longer, etc, until you lose all hope of catching up, and eventually run out of memory; possibly triggering process death if you've set max_heap_size on the process (although where I was using Erlang we didn't set that), or triggering OS process death of BEAM when it can't get more ram because allocation failed or OOM killer, or triggering OS death, if it has trouble when BEAM sucks in all the memory and can't muster its OOM killer in time and just gets stuck or OOM kills the wrong thing.
- lostcolony 6y ago
  Yep; as I mentioned elsewhere, I'm not discounting mailboxes filling faster than you can handle them, but rather the idea that you have 'problematic mails in the mailbox' - I've never had problematic mail in the mailbox. I've certainly seen mailboxes grow, because they were receiving messages faster than I was processing them, and i had to rethink my design. But that isn't an issue with the mail going into the mailbox; that's an issue with my not scaling out my receivers to fit the load. As I mentioned elsewhere, that may seem like semantics, but to me it isn't; it means the language's working that way makes sense, and it's rather an issue that I created a bottleneck with no way to relieve the pressure (as compared to messages that can't be handled and just sit there causing receives to take longer and longer over time).
  - toast0 6y ago
    Oh, I think I see. I don't think there's such thing as a problematic mail for BEAM really, mails are just terms, and BEAM handles terms, no problem. A mail that contained a small slice of a very large binary, would of course keep the whole referenced binary hanging around, which could be bad, or a large tuple could be bad because if you tried to inspect the message queue through process_info in another process, that would be a big copy. But I think maybe the original poster just meant lots of bad mail in the mailbox to mean mail that would take a long time to process, because of how the receiving process will handle it. Or, possibly bad mail meaning (as you suggest, perhaps), mail that won't be matched by any receive, resulting in longer and longer receives when they need to start from the top.