10 ms·
This is why it’s almost always wrong for library functions to log anything, even on ”errors”. Pass the status up through return values or exceptions. As a libra
by Too 9mo ago
This is why it’s almost always wrong for library functions to log anything, even on ”errors”. Pass the status up through return values or exceptions. As a library author you have no clue as how an application might use it. Multi threading, retry loops and expected failures will turn what’s a significant event in one context into what’s not even worthy of a debug log in another. No rule without exceptions of course, one valid case could be for example truly slow operations where progress reports are expected. Modern tracing telemetry with sampling can be another solution for the paranoid.
- echelon 9mo agoYou need a tuple: (context, level) The application owner should be able to adjust the contexts up or down. This is the point of ownership and where responsibility over which logs matter is handled. A library author might have ideas and provide useful suggestions, but it's ultimately the application owner who decides. Some libraries have huge blast radius and their `error` might be your `error` too. In other contexts, it could just be a warning. Library authors should make a reasonable guess about who their customer is and try to provide semantic, granular, and controllable failure behavior. As an example, Rust's logging ecosystem provides nice facilities for fine-grained tamping down of errors by crate (library) or module name. Other languages and logging libraries let you do this as well. That capability just isn't adopted everywhere.
- Izkata 9mo agoPython's built-in logging is the same if used correctly, where the library gets a logger based on its module name (this part isn't enforced) and the application can add a handler to that logger to route the logs differently if needed.
- deleted 9mo ago[deleted]
- esrauch 9mo agoI think an example where libraries could sensibly log error is if you have a condition which is recoverable but may cause a significant slowdown, including a potential DoS issue, and the application owner can remediate. You don't want to throw because destroying someone's production isn't worth it. You don't want to silent continue in that state because realistically there's no way for application owner to understand what is happening and why.
- TZubiri 9mo agoWe call those warnings, and it's very common to downgrade errors to warnings by wrapping an exception and printing the trace as you would an exception.
- kgklxksnrb 9mo agoLogging warnings are cowardly, you just push the decision to the log consumer to decide if the error should be acted on. Warnings are just errors that no one wants to deal with.
- bluGill 9mo agoWarnings are for where you expect someplace else to know/log if it really is an error but it might also be normal. You might log why a file io operation failed: if the caller recovers somehow it isn't an errer, but if they can't they log an error and when investigating the warning gives the detail you need to figure it out.
- kgklxksnrb 9mo agoWho proactively investigates warnings?
- bluGill 9mo agostatistacs are someimes run and the most common investigated (normally shut up the noise) mostly though when you are on a known problem warnings should be a useful filter to find where in the logs the problem might have started, then you use that timestamp to find info logs in the same area
- makeitdouble 9mo agoWarning logs are usually polluted with stuff nobody wants to fix but try to wash their hands off with a log. Like deprecated calls or error logs that got demoted because it didn't matter in practice. Anything that has a measurable impact on production should be logged above that, except if your system ignores log levels in the first place, but that's another can of worms.
- MobiusHorizons 9mo agoWhat you are proposing sounds like a nightmare to debug. The high level perspective of the operation is of course valuable for determining if an investigation is necessary, but the low level perspective in the library code is almost always where the relevant details are hiding. Not logging these details means you are in the dark about anything your abstractions are hiding from higher level code (which is usually a lot)
- TZubiri 9mo agoYou can log your IO and as long as your functions are idempotent that should be enough info to replicate.
- dpark 9mo agoAssuming everything is idempotent is a tall order. There are a lot of libraries that haven non-idempotent actions. There are a lot of inputs that can be problematic to log, too.
- TZubiri 9mo agoSay like opening a file? I guess in those cases standard practice is for lib to return a detailed error yeah. As far as traces, trying to solve issues that depend on external systems is indeed a tall order for your code. Isn't it beyond the scope of the thing being programmed.
- dpark 9mo agoI don’t really understand what you mean about opening files. Is this just an example of an idempotent action or is there some specific significance here? Either way logging the input (file name) is notably not sufficient for debugging if the file can change between invocations. The action can be idempotent and still be affected by other changes in the system. > trying to solve issues that depend on external systems is indeed a tall order for your code. Isn't it beyond the scope of the thing being programmed. If my program is broken I need it fixed regardless of why it’s broken. The specific example here of a file changing is likely to manifest as flakiness that’s impossible to diagnose without detailed logs from within the library.
- Etherlord87 9mo agoThis seems like such an obvious answer to the problem, your program isn't truly modularized if logging is global. If an error is unexpected it should bubble all the way up, but if it's expected and dealt with, the error message should be suppressed or its type changed to a warning.
- dpark 9mo agoI’ve worked on systems with “modularized” logging. It’s never been pleasant because investigations involve stitching together a bunch of different log sources to understand erase actually happened. A global log dump with attribution (module/component/file/line) is far easier to work with.
- cogman10 9mo agoDepending on the language and logging framework, debug/trace logging can be acceptable in a library. But you have to be extra careful to make sure that it's ultimately a no-op. A common problem in Java is someone will drop a log that looks something like this `log.trace("Doing " + foo + " to " + bar);` The problem is, especially in a hot loop, that throw away string concatenation can ultimately be a performance problem. Especially if `foo` or `bar` have particularly expensive `toString` functions. The proper way to do something like this in java is either log.trace("Doing $1 to $2", foo, bar); or if (log.traceEnabled()) { log.trace("Doing " + foo + " to " + bar); }
- TZubiri 9mo agoHow about wrapping the log.trace param in a lambda and monkeypatching log.trace to take a function that returns a string, and of course pushing the conditional to the monkeypatched func.
- 01HNNWZ0MV43FF 9mo agoThat is why the popular `tracing` crate in Rust uses macros for logging instead of functions. If the log level is too low, it doesn't evaluate the body of the macro
- tsimionescu 9mo agoDoes that mean the log level is a compilation parameter? Ideally, log levels shouldn't even be startup parameters, they should be changeable on the fly, at least for any server side code. Having to restart if bad enough, having to recompile to get debug logs would be an extraordinary nightmare (not only do you need to get your customers to reproduce the issue with debug logs, you actually have to ship them new binaries, which likely implies export controls and security validations etc).
- bluGill 9mo agoI don't know how rust does it, but my internal C++ framework has a global static array so that we can lookup the current log level quickly, and change it at runtime as needed. It is very valuable to turn on specific debug logs at times, when someone has a problem and we want to know what some code is doing
- renewiltord 9mo agoConflicting goals for the predominant libraries is what causes this. Log4J2 has a rewrite appender that solves the problem. But if you want zero-copy etc I don’t think there’s such a solution.
- pca006132 9mo agoWonder if someone used effect handlers for error logging. Sounds like a natural and modular way of handling this problem.
- cyphar 9mo agoOn paper, USDT probes are the best way for libraries (and binaries) to provide information for debugging because they can be used programmatically and have no performance overhead until they are measured but unfortunately they are not widely used.
- paulddraper 9mo agoIt may be unwise to log errors at low layers but logging informational and debug messages are useful (at least, when the caller enables them).
- jeroenhd 9mo agoI very much appreciate libraries that provide optional logging. Tracing error causes in network protocol calls can be pretty near impossible without throwing a library/package/crate/whatever into TRACE mode. Of course they shouldn't just be dumping text to stdout/stderr, but as long as the library logging is optional (or only logs when the library has reached some kind of unrecoverable state with instructions to file a bug report), logging is often the right call. It's easier to have logs and turn them off at compile time/runtime than to not have logs and need them once deployed.