7 ms·
Interesting idea - I like seeing a list of pet-peeves followed by a proposal for a straightforward way to have a set of 'alternative defaults' that remains back
by sethev 2mo ago
Interesting idea - I like seeing a list of pet-peeves followed by a proposal for a straightforward way to have a set of 'alternative defaults' that remains backwards compatible. If you don't want to opt in, don't run the new PRAGMA edition = 2026.
Too often it's just a list of issues and a wish that everyone else will change.
In (mild) defense of SQLITE_BUSY - busy_timeout just tells sqlite to sleep and retry up to the timeout when it receives SQLITE_BUSY. It seems like a sensible default for a library to leave that up the calling code - which may have something else it could do while it waits. However, that logic often gets missed!
- groundzeros2015 2mo agoYep. The whole locking database thing is this persistent myth about SQLite. All databases lock on write, it’s a question of the granularity of the lock. Multiple writers simply take turns.
- DANmode 2mo agoThey don’t know by now, honestly they don’t want to know.
- afiori 2mo agoyes but no... most database allow for at least as many parallel and concurrent writes as there are tables at a minimum. The "lock on write" problem is that in MySQL i could run a OLAP pipeline for a few hours and have a fully functioning database with degraded perfomance, on SQLite the same pipeline would lock the database for the full hour. (there are surely ways to solve this (eg using the main db as read-only and a secondary db for writes or splitting the writes in incremental transaction), but it is not a "myth".
- groundzeros2015 2mo agoThe “myth” is you can’t use it for a website because if you have multiple requests that need to write they can’t do it concurrently. (They just take turns of course for the milliseconds of write time.) If you run a transaction with writes for an hour on any database, the data you update will literally be locked. So your example only works if results are independent of the data other programs want to use. Of course more granularity of locking is better and enables more designs that would not otherwise work. But somewhere you run into the same problem of writers taking turns.
- mamcx 2mo agoAfter read, it hit me that because sqlite is a DB, "editions" as-is not work. Because it not tied to the data but to the code. Instead, what I think should be is that the PRAGMAs become "data" that is always checked in full with "if manually set" and then on next "open" THEY GET APPLIED. That is. (and in the command line when open interactively they show up).
- aidenn0 2mo agoRE: SQLITE_BUSY: I would replace "often" with "nearly always." On top of that, it's often not fixed even when pointed out. "This software only has one writer, so we don't need to handle SQLITE_BUSY" translates to me sending SIGSTOP to a process any time I want to run some queries against its database.
- platz 2mo agobusy_timeout is often sidestepped (ignored) when a transaction attempts to upgrade from a read to a write producing SQLITE_BUSY. By default, SQLite transactions start in DEFERRED mode, acting as read transactions until an actual write operation occurs. If another connection begins writing to the database while your transaction is in this read state, an immediate SQLITE_BUSY error is triggered regardless of what you set busy_timeout to
- tzot 2mo agoExactly. In cases where I expect long-running parallel connections from separate processes to the same sqlite file, I make sure that all read transactions do `BEGIN DEFERRED` so `COMMIT` releases the read locks, and all write transactions do `BEGIN IMMEDIATE` so that `SQLITE_BUSY` timeout is not side-stepped. There was one case where all transactions were implemented using nested `SAVEPOINT bla` so `BEGIN IMMEDIATE` could not be used without more hassle, so this ended all “I know I'm going to write” transactions to instantly update a single-row table so that their lock would not begin as DEFERRED and eventually switch to `IMMEDIATE`; this way almost all `SQLITE_BUSY` side-steppings disappeared. (timeout was set to 30 seconds but all read/write transactions were instrumented to have less than 5 seconds duration).
- platz 2mo agolol, I briefly thought of such a technique, but in the end, found it simpler to just use a synchronization primitive at the application level to serialize db access on transactions where I knew I was going to write. it amounts to about the same honor code.
- tptacek 2mo agoThis isn't so much a list of pet peeves as it is the almost universal way people that work seriously with SQLite configure the database. It's reasonable to suggest that the alternative settings for each of these suggestions is probably the wrong default for 2026.
- doc_ick 2mo agoI’d say these are reasonable settings for most uses. Though do you know of surveys that back this up? I don’t mean to nit pick too much, I’d just like to see common uses and the data.
- tptacek 2mo agoSQLite is used in a lot of unconventional settings (for SQL databases) where these settings don't make as much sense. But that's what makes the "edition" useful; it captures the use case we all mean when we're thinking of the "database" lego in an application stack.
- mikepurvis 2mo agoWhile that's true, editions are more about leaving legacy decisions behind while keeping the backward compatibility promise. Even if you're in one of those unconventional settings (say, a bare-metal microcontroller or something), you'd probably still start from edition 2026 and mutate your settings accordingly, rather than using the defaults that are 26 years old.
- tialaramex 2mo agoYeah, that's important. Rust's 2015 edition is worse, not just different from what you'd write today with 2024 edition. There's a clear direction of travel.
- doc_ick 2mo agoSo you’re saying there’s no data to group these settings by editions.