13 ms·

Why Is SQLite Coded In C

357 points by plainOldText 11mo ago

peter_d_sherman 11mo ago
>"1.3. Low-Dependency Libraries written in C do not have a huge run-time dependency. In its minimum configuration, SQLite requires only the following routines from the standard C library: memcmp() memcpy() memmove() memset() strcmp() strlen() strncmp() In a more complete build, SQLite also uses library routines like malloc() and free() and operating system interfaces for opening, reading, writing, and closing files. But even then, the number of dependencies is very small. Other "modern" languages, in contrast, often require multi-megabyte runtimes loaded with thousands and thousands of interfaces." Very laudable! (I should also point out that SQLite could conceivably be compiled with small (in terms of lines of code) C compilers like Fabrice Bellard's Tiny C Compiler (TCC). Also SQLite's few required standard C library routines listed above could conceivably be coded inside of SQLite itself(!) (they are, after all, just additional lines of C code in a different place -- and those could conceivably be moved or copied) -- thus removing the dependency/requirement for any standard C library whatsoever!) Anyway, we love SQLite!
vitali2y 11mo ago
[dead]
deanebarker 11mo ago
It's hard to argue with success. SQLite's pervasiveness is kind of a royal flush.
mikece 11mo ago
The fact that a C library can easily be wrapped by just about any language is really useful. We're considering writing a library for generating a UUID (that contains a key and value) for reasons that make sense to us and I proposed writing this in C so we could simply wrap it as a library for all of the languages we use internally rather than having to re-implement it several times. Not sure if we'll actually build this library but if we do it will be in C (I did managed to get the "wrap it for each language" proposal pre-approved).
- 01HNNWZ0MV43FF 11mo ago
  It is. You can also write it in C++ or Rust and expose a C API+ABI, and then you're distributing a binary library that the OS sees as very similar to a C library. Occasionally when working in Lua I'd write something low-level in C++, wrap it in C, and then call the C wrapper from Lua. It's extra boilerplate but damn is it nice to have a REPL for your C++ code. Edit: Because someone else will say it - Rust binary artifacts _are_ kinda big by default. You can compile libstd from scratch on nightly (it's a couple flags) or you can amortize the cost by packing more functions into the same binary, but it is gonna have more fixed overhead than C or C++.
  - bsder 11mo ago
    > It is. You can also write it in C++ or Rust and expose a C API+ABI, and then you're distributing a binary library that the OS sees as very similar to a C library. If I want a "C Library", I want a "C Library" and not some weird abomination that has been surgically grafted to libstdc++ or similar (but be careful of which version as they're not compatible and the name mangling changes and ...). This isn't theoretical. It's such a pain that the C++ folks started resorting to header-only libraries just to sidestep the nightmare.
    - uecker 11mo ago
      Rust libraries also impose an - in my opinion - unacceptable burden to the open source ecosystem: https://www.debian.org/releases/trixie/release-notes/issues.html#limitations-in-security-support https://www.debian.org/releases/trixie/release-notes/issues.... This makes me less safe rather than more. Note that there is a substantial double standard here, we could never in the name of safety impose this level of burden from C tooling side because maintainers would rightfully be very upset (even toggling a warning in the default set causes discussions). For the same reason it should be unacceptable to use Rust before this is fixed, but somehow the memory safety absolutists convinced many people that this is more important than everything else. (I also think memory safety is important, but I can't help but thinking that pushing for Rust is more harmful to me than good. )
deleted 11mo ago
[deleted]
plainOldText 11mo ago
I’d be curious to know what the creators of SQLite would have to say about Zig. Zig gives the programmer more control than Rust. I think this is one of the reasons why TigerBeetle is written in Zig.
- Jtsummers 11mo ago
  > Nearly all systems have the ability to call libraries written in C. This is not true of other implementation languages. From section "1.2 Compatibility". How easy is it to embed a library written in Zig in, say, a small embedded system where you may not be using Zig for the rest of the work? Also, since you're the submitter, why did you change the title? It's just "Why is SQLite Coded in C", you added the "and not Rust" part.
  - plainOldText 11mo ago
    The article allocates the last section to explaining why Rust is not a good fit (yet) so I wanted the title to cover that part of the conversation since I believe it is meaningful. It illustrates the tradeoffs in software engineering.
    - Jtsummers 11mo ago
      > Otherwise please use the original title, unless it is misleading or linkbait; don't editorialize. From the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- metaltyphoon 11mo ago
  > Zig gives the programmer more control than Rust More control over what exactly? Allocations? There is nothing Zig can do that Rust can’t.
  - Cloudef 11mo ago
    I think zig generally composes better than rust. With rust you pretty much have to start over if you want reusable / composable code, that is not use the default std. Rust has small crates for every little thing because it doesn't compose well, as well to improve compile times. libc in the default std also is major L.
pm2222 11mo ago
These points strike me: Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Rust needs to mature a little more, stop changing so fast, and move further toward being old and boring. Rust needs to demonstrate that it can do the kinds of work that C does in SQLite without a significant speed penalty.
- rstuart4133 11mo ago
  > Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. This is annoying in Rust. To me array accesses aren't the most annoying, it's match{} branches that will never been invoked. There is unreachable!() for such situations, and you would hope that: if array_access_out_of_bounds { unreachable!(); } is recognised by the Rust tooling and just ignored. That's effectively the same as SQLite is doing now by not doing the check. But it isn't ignored by the tooling: unreachable!() is reported as a missed line. Then there is the test code coverage including the standard output by default, and you have to use regex's on path names to remove it.
  - steveklabnik 11mo ago
    A more direct translation of the sqlite strategy here is to use get_unchecked instead of [], and then you get the same behaviors. Your example does what [] does already, it’s just a more verbose way of writing the same thing. It’s not the same behavior as sqlite.
    - rstuart4133 11mo ago
      > A more direct translation of the sqlite strategy here is to use get_unchecked instead of [], and then you get the same behaviors. Array access was just an example. My point was that Rust makes 100% code coverage for unit tests well neigh impossible. For those of us who like 100% test coverage, that is a major annoyance. For way I use Rust that could be fixed by the tooling simply not counting unreachable!() lines as unreached. For Sqlite, who does branch coverage testing on the compiled binary you would have to go a step further, and provide a compile time option that elides all paths that lead to unreachable!() from the binary. I recall Sqlite saying when they changed to 100% branch coverage, their bug reports dropped by a factor of 7. I hope I remember that correctly. If I do, I'm pretty sure they won't be looking at Rust until they can achieve the same outcome. They can't come close now. Replacing every array access with get_unchecked() and the consequent explosion of unsafe area's wouldn't fly with anyone I worked with. You did say to me in another thread unsafe is perfectly fine in Rust programs, but you are literally the only person holding that opinion I come across.
Jtsummers 11mo ago
Two previous, and substantial, discussions on this page: https://news.ycombinator.com/item?id=28278859 https://news.ycombinator.com/item?id=28278859 - August 2021 https://news.ycombinator.com/item?id=16585120 https://news.ycombinator.com/item?id=16585120 - March 2018
- bravura 11mo ago
  I'm curious about tptacek's comment (https://news.ycombinator.com/item?id=28279426 https://news.ycombinator.com/item?id=28279426). 'the "security" paragraphs in this page do the rest of the argument a disservice. The fact is, C is a demonstrable security liability for sqlite.' The current doc no longer has any paragraphs about security, or even the word security once. The 2021 edition of the doc contained this text which no longer appears: 'Safe languages are often touted for helping to prevent security vulnerabilities. True enough, but SQLite is not a particularly security-sensitive library. If an application is running untrusted and unverified SQL, then it already has much bigger security issues (SQL injection) that no "safe" language will fix. It is true that applications sometimes import complete binary SQLite database files from untrusted sources, and such imports could present a possible attack vector. However, those code paths in SQLite are limited and are extremely well tested. And pre-validation routines are available to applications that want to read untrusted databases that can help detect possible attacks prior to use.' https://web.archive.org/web/20210825025834/https%3A//www.sqlite.org/whyc.html https://web.archive.org/web/20210825025834/https%3A//www.sql...
bfkwlfkjf 11mo ago
> Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Huh it's not everyday that I hear a genuinely new argument. Thanks for sharing.
- beached_whale 11mo ago
  I think those branches are often not there because it's provably never going out of bounds. There are ways to ensure the compiler knows the bounds cannot be broken.
- ChadNauseam 11mo ago
  I wonder if this problem could be mitigated by not requiring coverage of branches that unconditionally lead to panics. or if there could be some kind of marking on those branches that indicate that they should never occur in correct code
  - accelbred 11mo ago
    You'd want to statically prove that any panic is unreachable
- jonahx 11mo ago
  So is the argument that safe langs produce stuff like: // pseudocode if (i >= array_length) panic("index out of bounds") that are never actually run if the code is correct? But (if I understand correctly) these are checks implicitly added by the compiler. So the objection amounts to questioning the correctness of this auto-generated code, and is predicated upon mistrusting the correctness of the compiler? But presumably the Rust compiler itself would have thorough tests that these kinds of checks work? Someone please correct me if I'm misunderstanding the argument.
  - oconnor663 11mo ago
    > questioning the correctness of this auto-generated code I wouldn't put it that way. Usually when we say the compiler is "incorrect", we mean that it's generating code that breaks the observable behavior of some program. In that sense, adding extra checks that can't actually fail isn't a correctness issue; it's just an efficiency issue. I'd usually say the compiler is being "conservative" or "defensive". However, the "100% branch testing" strategy that we're talking about makes this more complicated, because this branch-that's-never-taken actually is observable, not to the program itself but to its test suite.
firesteelrain 11mo ago
One thing I found especially interesting is the section at the end about why Rust isn’t used. It leaves open the door and at least is constructive feedback to the Rust community
DarkNova6 11mo ago
> All that said, it is possible that SQLite might one day be recoded in Rust. Recoding SQLite in Go is unlikely since Go hates assert(). But Rust is a possibility. Some preconditions that must occur before SQLite is recoded in Rust include: - Rust needs to mature a little more, stop changing so fast, and move further toward being old and boring. - Rust needs to demonstrate that it can be used to create general-purpose libraries that are callable from all other programming languages. - Rust needs to demonstrate that it can produce object code that works on obscure embedded devices, including devices that lack an operating system. - Rust needs to pick up the necessary tooling that enables one to do 100% branch coverage testing of the compiled binaries. - Rust needs a mechanism to recover gracefully from OOM errors. - Rust needs to demonstrate that it can do the kinds of work that C does in SQLite without a significant speed penalty.
- steveklabnik 11mo ago
  1. Rust has had ten years since 1.0. It changes in backward compatible ways. For some people, they want no changes at all, so it’s important to nail down which sense is meant. 2. This has been demonstrated. 3. This one hinges on your definition of “obscure,” but the “without an operating system” bit is unambiguously demonstrated. 4. I am not an expert here, but given that you’re testing binaries, I’m not sure what is Rust specific. I know the Ferrocene folks have done some of this work, but I don’t know the current state of things. 5. Rust as a language does no allocation. This OOM behavior is the standard library, of which you’re not using in these embedded cases anyway. There, you’re free to do whatever you’d like, as it’s all just library code. 6. This also hinges on a lot of definitions, so it could be argued either way.
  - wrs 11mo ago
    For a little more color on 5, as a user of no_std Rust on embedded processors I use crates like heapless or trybox that provide Vec, String, etc. APIs like the std ones, but fallible. Of course, two libraries that choose different no_std collection types can't communicate...but hey, we're comparing to C here.
    - 11mo ago
dgfitz 11mo ago
> Rust needs to mature a little more, stop changing so fast, and move further toward being old and boring. Talking about C99, or C++11, and then “oh you need the nightly build of rust” were juxtaposed in such a way that I never felt comfortable banging out “yum install rust” and giving it a go.
- steveklabnik 11mo ago
  Other than some operating systems projects, I haven’t run into a “requires nightly” in the wild for years. Most users use the stable releases. (There are some decent reasons to use the nightly toolchain in development even if you don’t rely on any unfinished features in your codebase, but that means they build on stable anyway just fine if you prefer.)
  - dgfitz 11mo ago
    Good to know, maybe I’ll give it a whirl. I’d been under the (mistaken, apparently) impression that if one didn’t update monthly they were going to have a bad time.
    - steveklabnik 11mo ago
      You may be running into forwards compatibility issues, not backwards compatibility issues, which is what nightly is about. The Rust Project releases a new stable compiler every six weeks. Because it is backwards compatible, most people update fairly quickly, as it is virtually always painless. So this may mean, if you don’t update your compiler, you may try out a new package version and it may use features or standard library calls that don’t exist in the version you’re using, because the authors updated regularly. There’s been some developments in Cargo to try and mitigate some of this, but since it’s not what the majority of users do, it’s taken a while and those features landed relatively recently, so they’re not widely adopted yet. Nightly features are ones that aren’t properly accepted into the language yet, and so are allowed to break in backwards incompatible ways at any time.
      - 11mo ago
jasonthorsness 11mo ago
“None of the safe programming languages existed for the first 10 years of SQLite's existence. SQLite could be recoded in Go or Rust, but doing so would probably introduce far more bugs than would be fixed, and it may also result in slower code.” Modern languages might do more than C to prevent programmers from writing buggy code, but if you already have bug-free code due to massive time, attention, and testing, and the rate of change is low (or zero), it doesn’t really matter what the language is. SQLIte could be assembly language for all it would matter.
- oconnor663 11mo ago
  > and the rate of change is low (or zero) This jives with a point that the Google Security Blog made last year: "The [memory safety] problem is overwhelmingly with new code...Code matures and gets safer with time." https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html https://security.googleblog.com/2024/09/eliminating-memory-s...
  - miohtama 11mo ago
    You can find historical SQLite CVEs here https://www.sqlite.org/cves.html https://www.sqlite.org/cves.html Note that although code matures the chances of C Human error bugs will never go to zero. We have some bad incidents like Heartbleed to show this.
    - ziotom78 11mo ago
      Right, but I believe nobody can claim that Human error bugs go to zero for Rust code.
      - john_the_writer 11mo ago
        Agreed. I rather dislike the idea of "safe" coding languages. Fighting with a memory leak in an elixir app, for the past week. I never viewed c or c++ as unsafe. Writing code is hard, always has been, always will be. It is never safe.
pizlonator 11mo ago
SQLite works great in Fil-C with minimal changes. So, the argument for keeping SQLite written in C is that it gives the user the choice to either: - Build SQLite with Yolo-C, in which case you get excellent performance and lots of tooling. And it's boring in the way that SQLite devs like. But it's not "safe" in the sense of memory safe languages. - Build SQLite with Fil-C, in which case you get worse (but still quite good) performance and memory safety that exceeds what you'd get with a Rust/Go/Java/whatever rewrite. Recompiling with Fil-C is safer than a rewrite into other memory safe languages because Fil-C is safe through all dependencies, including the syscall layer. Like, making a syscall in Rust means writing some unsafe code where you could screw up buffer sizes or whatnot, while making a syscall in Fil-C means going through the Fil-C runtime.
wodenokoto 11mo ago
I think it’s more interesting that DuckDB is written in C++ and not rust than SQLite. SQLite is old, huge and known for its gigantic test coverage. There’s just so much to rewrite. DuckDB is from 2019, so new enough to jump on the “rust is safe and fast”
- tonyhart7 11mo ago
  if they write it on modern C++ then its alright tbh
- jandrewrogers 11mo ago
  If maximum performance is a top objective, it is probably because C++ produces faster binaries with less code. Modern C++ specifically also has a lot of nice compile-time safety features, especially for database-like code.
  - wodenokoto 11mo ago
    I can’t verify those claims one way or another, but I’m interested to hear why they were downvoted.
    - jandrewrogers 11mo ago
      I've worked on a couple different projects that did substantial parallel development in C++20 and Rust, which created interesting opportunities for concrete comparison. It was performance-engineered code and we needed to validate their equivalence by testing them against each other. The practical differences are larger than the theoretical differences, so I would expect the gap to diminish over time. Rust reminded me of when I used to write database engines in Java. It required a lot more code, which has its own costs, but never really delivered on claims of comparable performance. The "more code" part largely comes down to the more limited ability to build good abstractions compared to C++20 and more limited composability. The "slower binaries" part comes down to worse codegen, which you can't blame on Rust per se, and a lot of extra overhead introduced in the code to satisfy the Rust safety model that would simply not be required in other systems languages. Safety is a mixed bag. Rust can check several things at compile-time that C++20 cannot. C++20 can check several things at compile-time that Rust cannot. For high-performance database-y code, memory is allocated at startup and is accessed via managed index handles. Rust does the same thing. In these types of memory models, i.e. no dynamic allocation and no raw pointers, both Rust and C++20 offer similar memory safety guarantees. Most high-performance software is thread-per-core that is almost purely single-threaded, so thread-safety concerns are limited. That said, stripping away all of the above, the only real advantage that C++20 has its much more powerful toolset for building abstractions. Its performance and unique safety elements are based almost entirely on the ability to build concise, contextual, and highly composable abstractions as needed. This is not a feature that should be downplayed, I immediately miss it when I use most other languages.
tonyhart7 11mo ago
because Rust isnt out yet back then????
vincent-manis 11mo ago
The point about bounds checking in `safe' languages is well taken, it does prevent 100% test coverage. As we all agree, SQLite has been exhaustively tested, and arguments for bounds checking in it are therefore weakened. Still, that's not an argument for replicating this practice elsewhere, not unless you are Dr Hipp and willing to work very hard at testing. C.A.R. Hoare's comment on eliminating runtime checks in release builds is well-taken here: “What would we think of a sailing enthusiast who wears his life-jacket when training on dry land but takes it off as soon as he goes to sea?” I am not Dr Hipp, and therefore I like run-time checks.
slashdev 11mo ago
This is ignoring the elephant in the room: SQLite is being rewritten in Rust and it's going quite well. https://github.com/tursodatabase/turso https://github.com/tursodatabase/turso It has async I/O support on Linux with io_uring, vector support, BEGIN CONCURRENT for improved write throughput using multi-version concurrency control (MVCC), Encryption at rest, incremental computation using DBSP for incremental view maintenance and query subscriptions. Time will tell, but this may well be the future of SQLite.
- zvmaz 11mo ago
  In the link you provided, this is what I read: "An in-process SQL database, compatible with SQLite." Compatible with SQLite. So it's another database?
  - simonw 11mo ago
    Yeah, I don't think it even counts as a fork - it's a ground-up re-implementation which is already adding features that go beyond the original.
  - ForHackernews 11mo ago
    It's a fork and a rewrite.
- tonyhart7 11mo ago
  so its sqlite++ since they added bunch of things on top of that
- metaltyphoon 11mo ago
  The moment turso becomes stable , SQLite will inevitably fade away with time if they don’t rethink how contributions should be taken. I honestly believe the Linux philosophy of software development will be what catapults turso forward.
- assimpleaspossi 11mo ago
  >>SQLite is being rewritten in Rust SQLite is NOT being rewritten in Rust! >>Turso Database is an in-process SQL database written in Rust, compatible with SQLite.
jokoon 11mo ago
I wonder if the hype helps rust being a better language At this point I wish the creators of the language could talk about what rust is bad at.
- steveklabnik 11mo ago
  Folks involved often do! Talking about what’s not great is the only path towards getting better, because you have to identify pain points in order to fix them.
  - estebank 11mo ago
    I would go as far as saying that 90% of managing the project is properly communicating, discussing and addressing the ways in which Rust sucks. The all-hands in NL earlier this year was wall to wall meetings about how much things suck and what to do about them! I mean this in the best possible way. ^_^
matt3210 11mo ago
I can compile c anywhere and for any processor, which can’t be said for rust
saalweachter 11mo ago
I think beyond the historical reasons why C was the best choice when SQLite was being developed, or the advantages it has today, there's also just no reason to rewrite SQLite in another language. We don't have to have one implementation of a lightweight SQL database. You can go out right now and start your own implementation in Rust or C++ or Go or Lisp or whatever you like! You can even make compatible APIs for it so that it can be a drop-in replacement for SQLite! No one can stop you! You don't need permission! But why would we want to throw away the perfectly good C implementation, and why would we expect the C experts who have been carefully maintaining SQLite for a quarter century to be the ones to learn a new language and start over?
- turtletontine 11mo ago
  Thanks for this, I fully agree. One frustration I have with the modern moment is the tendency to view anything more than five years old with disdain, as utterly irrelevant and obsolete. Maybe I’m just going old, but I like my technology dependable and boring, especially software. Glad to see someone express respect for the decades of expertise that have gone into things we take for granted.
  - friendly_wizard 11mo ago
    I think we owe an equally proportionate measure of respect to the authors of boring old sqlite who, given the depth of their experience, recognize that there may in fact be benefits to be gained from the rewrite and are open to exploring the possibility. The blockers as stated, I have no doubt, were carefully considered with a level of insight few of us could match. If and when the time is right, if they choose to undertake the effort I'm sure the juice will be worth the squeeze. The fact that they're not jumping in blindly today is telling. Even more telling will be if they do eventually go that route.
- AdamJacobMuller 11mo ago
  One good reason is that people have written golang adapters, so that you can use sqlite databases without cgo. I agree to what I think you're saying which is that "sqlite" has, to some degree, become so ubiquitous that it's evolved beyond a single implementation. We, of course, have sqlite the C library but there is also sqlite the database file format and there is no reason we can't have an sqlite implementation in golang (we already do) and one in pure rust too. I imagine that in the future that will happen (pure rust implementation) and that perhaps at some point much further in the future, that may even become the dominant implementation.
binary132 11mo ago
I love him so much.
sema4hacker 11mo ago
"Why SQLite is coded in C..." is an explanation, as documented at sqlite.org. "Why is SQLite coded in C and not Rust?" is a question, which immediately makes me want to ask "Why do you need SQLite coded in Rust?".
- lifthrasiir 11mo ago
  Because the title has been editorialized.
- urbandw311er 11mo ago
  Indeed. Why is SQLite coded in C and not BASIC?
- t14n 11mo ago
  fwiw there's a project doing just that: https://github.com/tursodatabase/turso https://github.com/tursodatabase/turso they have a blog hinting at some answers as to "why": https://turso.tech/blog/introducing-limbo-a-complete-rewrite-of-sqlite-in-rust https://turso.tech/blog/introducing-limbo-a-complete-rewrite...
rednafi 11mo ago
Also, Rust needs a better stdlib. A crate for every little thing is kinda nuts. One reason I enjoy Go is because of the pragmatic stdlib. On most cases, I can get away without pulling in any 3p deps. Now of course Go doesn’t work where you can’t tolerate GC pauses and need some sort of FFI. But because of the stdlib and faster compilation, Go somehow feels lighter than Rust.
- firesteelrain 11mo ago
  Rust doesn’t really need a better stdlib as much as a broader one, since it is intentionally narrow. Go’s stdlib includes opinions like net/http and templates that Rust leaves to crates. The trade-off is Rust favors stability and portability at the core, while Go favors out-of-the-box ergonomics. Both approaches work, just for different teams.
- tonyhart7 11mo ago
  me when I dont know ball:
- afdbcreid 11mo ago
  Is Rust's stdlib worse than C's? It's not an argument here.
globalnode 11mo ago
[flagged]
coolThingsFirst 11mo ago
I don't want to sound cynical but a lot of it has to deal with the simplicity of the language. It's much harder to find a good Rust engineer than a C one. When all you have is pointers and structs it's much easier to meet the requirements for the role.
system2 11mo ago
What's up with SQLite news lately? I feel like I see at least 1-2 posts about it per day.
Havoc 11mo ago
For a project that is functionally “done” switching doesn’t make sense. Something like kernel code where you know it’ll continue to evolve - there going through the pain may be worth it
daxfohl 11mo ago
It sounds like the core doesn't even allocate, and presumably the extended library allocates in limited places using safe patterns. So there wouldn't be much benefit from Rust anyway, I'd think. Had SQLite ever had a memory leak or use-after-delete bug on a production release? If so, that answers the question. But I've never heard of one. Also, does it use doubly linked lists or graphs at all? Those can, in a way, be safer in C since Rust makes you roll your own virtual pointer arena.
- steveklabnik 11mo ago
  Rust’s memory safety guarantees aren’t exclusive to hep allocation. In fact, the language doesn’t heap allocate at all. You can write a linked list the same way you would in C if you wish.
- thinkharderdev 11mo ago
  > Also, does it use doubly linked lists or graphs at all? Those can, in a way, be safer in C since Rust makes you roll your own virtual pointer arena. You can implement a linked list in Rust the same as you would in C using raw pointers and some unsafe code. In fact there is one in the standard library.
- dathinab 11mo ago
  > Had SQLite ever had a memory leak or use-after-delete bug on a production release? sure, it's an old library they had pretty much anything (not because they don't know what they are doing but because shit happens) lets check CVEs of the last few years: - CVE-2025-29088 type confusion - CVE-2025-29087 out of bounds write - CVE-2025-7458 integer overflow, possible in optimized rust but test builds check for it - CVE-2025-6965 memory corruption, rust might not have helped - CVE-2025-3277 integer overflow, rust might have helped - CVE-2024-0232 use after free - CVE-2023-36191 segmentation violation, unclear if rust would have helped - CVE-2023-7104 buffer overflow - CVE-2022-46908 validation logic error - CVE-2022-35737 array bounds overflow - CVE-2021-45346 memory leak ... as you can see the majority of CVEs of sqlite are much less likely in rust (but a rust sqlite impl. likely would use unsafe, so not impossible) as a side note there being so many CVEs in 2025 seem to be related to better some companies (e.g. Google) having done quite a bit of fuzz testing of SQLite other takeaways: - 100% branch coverage is nice, but doesn't guarantee memory soundness in C - given how deeply people look for CVEs in SQLite the number of CVEs found is not at all as bad as it might look but also one final question: SQLite uses some of the best C programmers out there, only they merge anything to the code, it had very limited degree of change compared to a typical company project. And we still have memory vulnerabilities. How is anyone still arguing for C for new projects?
kazinator 11mo ago
> The C language is old and boring. It is a well-known and well-understood language. So you might think, but there is a committee actively undermining this, not to mention compiler people keeping things exciting also. There is a dogged adherence to backward compatibility, so that you can't pretend C has not gone anywhere in thirty-five years, if you like --- provided you aren't invoking too much undefined behavior. (You can't as easily pretend that your compiler has not gone anywhere in 35 years with regard to things you are doing out of spec.)
next_xibalba 11mo ago
Aren't SQLite’s bottlenecks primarily io-bound (not CPU)? If so, fopen, fread, or syscalls are the most important to performance and pure language efficiency wouldn't be limiter.
morshu9001 11mo ago
This is what I expected. Rust is the first thing that has been worth considering as a C replacement. C++ wasn't.
dathinab 11mo ago
It's kinda sad to read as most of their arguments might seem right at first but if put under scrutiny really fall apart. Like why defend C in 2025 when you only have to defend C in 2000 and then argue you have a old, stable, deeply tested, C code base which has no problem with anything like "commonly having memory safety issues" and is maintained by a small group of people very highly skilled in C. Like that argument alone is all you need, a win, simple straight forward, hard to contest. But most of the other arguments they list can be picked apart and are only half true.
- Deanoumean 11mo ago
  The argument you propose only works for justifying a maintenance mode for and old codebase. If you want to take the chance to turn away new developers from complex abominations like C++ and Rust and garbage collected sloths like Java and get them to consider a comparatively simple but ubiquitous language that is C, you have to offer more.
  - dangus 11mo ago
    Is SQLite looking for new developers? Will they ever need a large amount of developers like a mega-corp that needs to hire 100 React engineers?
    - metaltyphoon 11mo ago
      No, but as morbid as this sounds, the three(?) devs one day will pass away so now what?
      - hoppp 11mo ago
        Then the rights will be sold to a FAANG or an open souce fork like libSQL will live on.
        colejohnson66 11mo ago
        SQLite is public domain (as much as is legally possible). So there's no "rights" to "sell" except the trademark.
ternaryoperator 11mo ago
> Recoding SQLite in Go is unlikely since Go hates assert() Any idea what this refers to? assert is a macro in C. Is the implication that OP wants the capability of testing conditions and then turning off the tests in a production release? If so, then I think the argument is more that go hates the idea of a preprocessor. Or have I misunderstood the point being made?
- steveklabnik 11mo ago
  https://go.dev/doc/faq#assertions https://go.dev/doc/faq#assertions
  - ternaryoperator 11mo ago
    Steve, thanks for taking the time to point me to this on-point passage.
psyclobe 11mo ago
SQLite is a true landmark, c not withstanding it just happened to be the right tool at the right time and by now anything else is well not as interesting as what they have going on now; totally bucks the trend of throw away software.
unsungNovelty 11mo ago
As I write more code, use more software and read about rewrites... The biggest gripe I have with a rewrite is... A lof of the time we rewrite for feature parity. Not the exact same thing. So you are kind ignoring/missing/forgetting all those edge cases and patches that were added along the way for so many niche or otherwise reasons. This means broken software. Something which used to work before but not anymore. They'll have to encounter all of them again in the wild and fix it again. Obviously if we are to rewrite an important piece of software like this, you'd emphasise more on all of these. But it's hard for me to comprehend whether it will be 100%. But other than sqlite, think SDL. If it is to be rewritten. It's really hard for me to comprehend that it's negligible in effect. Am guessing horrible releases before it gets better. Users complaining for things that used work. C is going to be there long after the next Rust is where my money is. And even if Rust is still present, there would be a new Rust then. So why rewrite? Rewrites shouldn't be the default thinking no?
a-saleh 11mo ago
Ok, I didn't expect such a high praise for rust. I am not joking.
deleted 11mo ago
[deleted]
steeleduncan 11mo ago
> SQLite could be recoded in Go Sqlite has been recoded (automatically) in Go a while ago [1], and it is widely deployed > would probably introduce far more bugs than would be fixed It runs against the same test suite with no issues > and it may also result in slower code It is quite a lot slower, but it is still widely used as it turns out that the convenience of a native port outweighs the performance penalty in most cases. I don't think SQLite should be rewritten in Go, Rust, Zig, Nim, Swift ... but ANSI C is a subset of the feature set of most modern programming languages. Projects such as this could be written and maintained in C indefinitely, and be automatically translated to other languages for the convenience of users in those languages [1] https://pkg.go.dev/modernc.org/sqlite https://pkg.go.dev/modernc.org/sqlite
- sim7c00 11mo ago
  > would probably introduce far more bugs than would be fixed It runs against the same test suite with no issues - that proves nothing about bugs existing or not.
- ChrisRR 11mo ago
  > It runs against the same test suite with no issues That doesn't guarantee no bugs. It just means that the existing behaviour covered by the tests is still the same. It may introduce new issues in untested edge cases or performance issues
- sgbeal 11mo ago
  > It runs against the same test suite with no issues It runs against the same public test suite. The proprietary test suite is much more intensive.
skeester 11mo ago
[dead]
negrel 11mo ago
[dead]
6r17 11mo ago
If I remember correctly most of SQLite "closed-source" leverage comes from the test-suite - which probably cannot transpose to another language as easily. Ultimately there are already other solutions coming up re-writing it in rust or go.
belter 11mo ago
Some of the most interesting comments are out of: "3. Why Isn't SQLite Coded In A "Safe" Language?" "....Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy..." "...Safe languages usually want to abort if they encounter an out-of-memory (OOM) situation. SQLite is designed to recover gracefully from an OOM. It is unclear how this could be accomplished in the current crop of safe languages..."
dusted 11mo ago
In my opinion, you don't get to ask "why is X done by Y" before you've done X yourself by something not Y and not Y by proxy either.
BiraIgnacio 11mo ago
"1. C Is Best"
MomsAVoxell 11mo ago
Back in the good ol'/bad ol' days of the very early web/Internet, I had the fortune of working with someone who, lets say, has kind of a background in certain operating systems circles. Not only had this fellow built a functional ISP in one of the toughest markets (at that time), in the world - but he'd also managed to build the database engine and quite a few of the other tools that ran that ISP, and was in danger of setting a few standards for a few things which, since then, have long since settled out, but .. nevertheless .. it could've been. Anyway, this fellow wrote everything in C. His web page, his TODO.h for the day .. he had C-based tools for managing his docs, for doing syncs between various systems under his command (often in very far-away locations, and even under water a couple times) .. everything, in C. The database system he wrote in pure C was, at the time, quite a delight. It gave a few folks further up the road a bit of a tight neck. He went on to do an OS, because of course he did. Just sayin', SQLite devs aren't the only ones who got this right. ;)
1vuio0pswjnm7 11mo ago
Why doesn't ON CONFLICT(column_name) accept multiple arguments, i.e., multiple columns One stupid workaround is combining multiple columns into one, with values separated by a space, for example. This works when each column value is always a string containing no spaces Another stupid workaround, probably slower, might be to hash the multiple columns into a new column and use ON CONFLICT(newcolumn_name)
netrap 11mo ago
Why not just say "because I don't want to change it"? rustaceans will always say your argument isn't valid for some reason or another. IDGAF, it's written in C because it is -- that's it!
zenxyzzy 11mo ago
I'm really getting tired of resume driven development. Choosing technology X because the coder wants it on their resume is a truly shitty reason. Rust is just the lastest trendy bullshit that will make meh programmers into superstars.
deleted 11mo ago
[deleted]

357 points by plainOldText 11mo ago

>"1.3. Low-Dependency Libraries written in C do not have a huge run-time dependency. In its minimum configuration, SQLite requires only the following routines from the standard C library: memcmp() memcpy() memmove() memset() strcmp() strlen() strncmp() In a more complete build, SQLite also uses library routines like malloc() and free() and operating system interfaces for opening, reading, writing, and closing files. But even then, the number of dependencies is very small. Other "modern" languages, in contrast, often require multi-megabyte runtimes loaded with thousands and thousands of interfaces." Very laudable! (I should also point out that SQLite could conceivably be compiled with small (in terms of lines of code) C compilers like Fabrice Bellard's Tiny C Compiler (TCC). Also SQLite's few required standard C library routines listed above could conceivably be coded inside of SQLite itself(!) (they are, after all, just additional lines of C code in a different place -- and those could conceivably be moved or copied) -- thus removing the dependency/requirement for any standard C library whatsoever!) Anyway, we love SQLite!

vitali2y 11mo ago

[dead]

deanebarker 11mo ago

It's hard to argue with success. SQLite's pervasiveness is kind of a royal flush.

mikece 11mo ago

The fact that a C library can easily be wrapped by just about any language is really useful. We're considering writing a library for generating a UUID (that contains a key and value) for reasons that make sense to us and I proposed writing this in C so we could simply wrap it as a library for all of the languages we use internally rather than having to re-implement it several times. Not sure if we'll actually build this library but if we do it will be in C (I did managed to get the "wrap it for each language" proposal pre-approved).

01HNNWZ0MV43FF 11mo ago
It is. You can also write it in C++ or Rust and expose a C API+ABI, and then you're distributing a binary library that the OS sees as very similar to a C library. Occasionally when working in Lua I'd write something low-level in C++, wrap it in C, and then call the C wrapper from Lua. It's extra boilerplate but damn is it nice to have a REPL for your C++ code. Edit: Because someone else will say it - Rust binary artifacts _are_ kinda big by default. You can compile libstd from scratch on nightly (it's a couple flags) or you can amortize the cost by packing more functions into the same binary, but it is gonna have more fixed overhead than C or C++.
- bsder 11mo ago
  > It is. You can also write it in C++ or Rust and expose a C API+ABI, and then you're distributing a binary library that the OS sees as very similar to a C library. If I want a "C Library", I want a "C Library" and not some weird abomination that has been surgically grafted to libstdc++ or similar (but be careful of which version as they're not compatible and the name mangling changes and ...). This isn't theoretical. It's such a pain that the C++ folks started resorting to header-only libraries just to sidestep the nightmare.
  - uecker 11mo ago
    Rust libraries also impose an - in my opinion - unacceptable burden to the open source ecosystem: https://www.debian.org/releases/trixie/release-notes/issues.html#limitations-in-security-support https://www.debian.org/releases/trixie/release-notes/issues.... This makes me less safe rather than more. Note that there is a substantial double standard here, we could never in the name of safety impose this level of burden from C tooling side because maintainers would rightfully be very upset (even toggling a warning in the default set causes discussions). For the same reason it should be unacceptable to use Rust before this is fixed, but somehow the memory safety absolutists convinced many people that this is more important than everything else. (I also think memory safety is important, but I can't help but thinking that pushing for Rust is more harmful to me than good. )

deleted 11mo ago

[deleted]

plainOldText 11mo ago

I’d be curious to know what the creators of SQLite would have to say about Zig. Zig gives the programmer more control than Rust. I think this is one of the reasons why TigerBeetle is written in Zig.

Jtsummers 11mo ago
> Nearly all systems have the ability to call libraries written in C. This is not true of other implementation languages. From section "1.2 Compatibility". How easy is it to embed a library written in Zig in, say, a small embedded system where you may not be using Zig for the rest of the work? Also, since you're the submitter, why did you change the title? It's just "Why is SQLite Coded in C", you added the "and not Rust" part.
- plainOldText 11mo ago
  The article allocates the last section to explaining why Rust is not a good fit (yet) so I wanted the title to cover that part of the conversation since I believe it is meaningful. It illustrates the tradeoffs in software engineering.
  - Jtsummers 11mo ago
    > Otherwise please use the original title, unless it is misleading or linkbait; don't editorialize. From the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
metaltyphoon 11mo ago
> Zig gives the programmer more control than Rust More control over what exactly? Allocations? There is nothing Zig can do that Rust can’t.
- Cloudef 11mo ago
  I think zig generally composes better than rust. With rust you pretty much have to start over if you want reusable / composable code, that is not use the default std. Rust has small crates for every little thing because it doesn't compose well, as well to improve compile times. libc in the default std also is major L.

pm2222 11mo ago

These points strike me: Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Rust needs to mature a little more, stop changing so fast, and move further toward being old and boring. Rust needs to demonstrate that it can do the kinds of work that C does in SQLite without a significant speed penalty.

rstuart4133 11mo ago
> Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. This is annoying in Rust. To me array accesses aren't the most annoying, it's match{} branches that will never been invoked. There is unreachable!() for such situations, and you would hope that: if array_access_out_of_bounds { unreachable!(); } is recognised by the Rust tooling and just ignored. That's effectively the same as SQLite is doing now by not doing the check. But it isn't ignored by the tooling: unreachable!() is reported as a missed line. Then there is the test code coverage including the standard output by default, and you have to use regex's on path names to remove it.
- steveklabnik 11mo ago
  A more direct translation of the sqlite strategy here is to use get_unchecked instead of [], and then you get the same behaviors. Your example does what [] does already, it’s just a more verbose way of writing the same thing. It’s not the same behavior as sqlite.
  - rstuart4133 11mo ago
    > A more direct translation of the sqlite strategy here is to use get_unchecked instead of [], and then you get the same behaviors. Array access was just an example. My point was that Rust makes 100% code coverage for unit tests well neigh impossible. For those of us who like 100% test coverage, that is a major annoyance. For way I use Rust that could be fixed by the tooling simply not counting unreachable!() lines as unreached. For Sqlite, who does branch coverage testing on the compiled binary you would have to go a step further, and provide a compile time option that elides all paths that lead to unreachable!() from the binary. I recall Sqlite saying when they changed to 100% branch coverage, their bug reports dropped by a factor of 7. I hope I remember that correctly. If I do, I'm pretty sure they won't be looking at Rust until they can achieve the same outcome. They can't come close now. Replacing every array access with get_unchecked() and the consequent explosion of unsafe area's wouldn't fly with anyone I worked with. You did say to me in another thread unsafe is perfectly fine in Rust programs, but you are literally the only person holding that opinion I come across.

Jtsummers 11mo ago

Two previous, and substantial, discussions on this page: https://news.ycombinator.com/item?id=28278859 https://news.ycombinator.com/item?id=28278859 - August 2021 https://news.ycombinator.com/item?id=16585120 https://news.ycombinator.com/item?id=16585120 - March 2018

bravura 11mo ago
I'm curious about tptacek's comment (https://news.ycombinator.com/item?id=28279426 https://news.ycombinator.com/item?id=28279426). 'the "security" paragraphs in this page do the rest of the argument a disservice. The fact is, C is a demonstrable security liability for sqlite.' The current doc no longer has any paragraphs about security, or even the word security once. The 2021 edition of the doc contained this text which no longer appears: 'Safe languages are often touted for helping to prevent security vulnerabilities. True enough, but SQLite is not a particularly security-sensitive library. If an application is running untrusted and unverified SQL, then it already has much bigger security issues (SQL injection) that no "safe" language will fix. It is true that applications sometimes import complete binary SQLite database files from untrusted sources, and such imports could present a possible attack vector. However, those code paths in SQLite are limited and are extremely well tested. And pre-validation routines are available to applications that want to read untrusted databases that can help detect possible attacks prior to use.' https://web.archive.org/web/20210825025834/https%3A//www.sqlite.org/whyc.html https://web.archive.org/web/20210825025834/https%3A//www.sql...

bfkwlfkjf 11mo ago

> Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy. Huh it's not everyday that I hear a genuinely new argument. Thanks for sharing.

beached_whale 11mo ago
I think those branches are often not there because it's provably never going out of bounds. There are ways to ensure the compiler knows the bounds cannot be broken.
ChadNauseam 11mo ago
I wonder if this problem could be mitigated by not requiring coverage of branches that unconditionally lead to panics. or if there could be some kind of marking on those branches that indicate that they should never occur in correct code
- accelbred 11mo ago
  You'd want to statically prove that any panic is unreachable
jonahx 11mo ago
So is the argument that safe langs produce stuff like: // pseudocode if (i >= array_length) panic("index out of bounds") that are never actually run if the code is correct? But (if I understand correctly) these are checks implicitly added by the compiler. So the objection amounts to questioning the correctness of this auto-generated code, and is predicated upon mistrusting the correctness of the compiler? But presumably the Rust compiler itself would have thorough tests that these kinds of checks work? Someone please correct me if I'm misunderstanding the argument.
- oconnor663 11mo ago
  > questioning the correctness of this auto-generated code I wouldn't put it that way. Usually when we say the compiler is "incorrect", we mean that it's generating code that breaks the observable behavior of some program. In that sense, adding extra checks that can't actually fail isn't a correctness issue; it's just an efficiency issue. I'd usually say the compiler is being "conservative" or "defensive". However, the "100% branch testing" strategy that we're talking about makes this more complicated, because this branch-that's-never-taken actually is observable, not to the program itself but to its test suite.

firesteelrain 11mo ago

One thing I found especially interesting is the section at the end about why Rust isn’t used. It leaves open the door and at least is constructive feedback to the Rust community

DarkNova6 11mo ago

> All that said, it is possible that SQLite might one day be recoded in Rust. Recoding SQLite in Go is unlikely since Go hates assert(). But Rust is a possibility. Some preconditions that must occur before SQLite is recoded in Rust include: - Rust needs to mature a little more, stop changing so fast, and move further toward being old and boring. - Rust needs to demonstrate that it can be used to create general-purpose libraries that are callable from all other programming languages. - Rust needs to demonstrate that it can produce object code that works on obscure embedded devices, including devices that lack an operating system. - Rust needs to pick up the necessary tooling that enables one to do 100% branch coverage testing of the compiled binaries. - Rust needs a mechanism to recover gracefully from OOM errors. - Rust needs to demonstrate that it can do the kinds of work that C does in SQLite without a significant speed penalty.

steveklabnik 11mo ago
1. Rust has had ten years since 1.0. It changes in backward compatible ways. For some people, they want no changes at all, so it’s important to nail down which sense is meant. 2. This has been demonstrated. 3. This one hinges on your definition of “obscure,” but the “without an operating system” bit is unambiguously demonstrated. 4. I am not an expert here, but given that you’re testing binaries, I’m not sure what is Rust specific. I know the Ferrocene folks have done some of this work, but I don’t know the current state of things. 5. Rust as a language does no allocation. This OOM behavior is the standard library, of which you’re not using in these embedded cases anyway. There, you’re free to do whatever you’d like, as it’s all just library code. 6. This also hinges on a lot of definitions, so it could be argued either way.
- wrs 11mo ago
  For a little more color on 5, as a user of no_std Rust on embedded processors I use crates like heapless or trybox that provide Vec, String, etc. APIs like the std ones, but fallible. Of course, two libraries that choose different no_std collection types can't communicate...but hey, we're comparing to C here.
  - 11mo ago

dgfitz 11mo ago

> Rust needs to mature a little more, stop changing so fast, and move further toward being old and boring. Talking about C99, or C++11, and then “oh you need the nightly build of rust” were juxtaposed in such a way that I never felt comfortable banging out “yum install rust” and giving it a go.

steveklabnik 11mo ago
Other than some operating systems projects, I haven’t run into a “requires nightly” in the wild for years. Most users use the stable releases. (There are some decent reasons to use the nightly toolchain in development even if you don’t rely on any unfinished features in your codebase, but that means they build on stable anyway just fine if you prefer.)
- dgfitz 11mo ago
  Good to know, maybe I’ll give it a whirl. I’d been under the (mistaken, apparently) impression that if one didn’t update monthly they were going to have a bad time.
  - steveklabnik 11mo ago
    You may be running into forwards compatibility issues, not backwards compatibility issues, which is what nightly is about. The Rust Project releases a new stable compiler every six weeks. Because it is backwards compatible, most people update fairly quickly, as it is virtually always painless. So this may mean, if you don’t update your compiler, you may try out a new package version and it may use features or standard library calls that don’t exist in the version you’re using, because the authors updated regularly. There’s been some developments in Cargo to try and mitigate some of this, but since it’s not what the majority of users do, it’s taken a while and those features landed relatively recently, so they’re not widely adopted yet. Nightly features are ones that aren’t properly accepted into the language yet, and so are allowed to break in backwards incompatible ways at any time.
    - 11mo ago

jasonthorsness 11mo ago

“None of the safe programming languages existed for the first 10 years of SQLite's existence. SQLite could be recoded in Go or Rust, but doing so would probably introduce far more bugs than would be fixed, and it may also result in slower code.” Modern languages might do more than C to prevent programmers from writing buggy code, but if you already have bug-free code due to massive time, attention, and testing, and the rate of change is low (or zero), it doesn’t really matter what the language is. SQLIte could be assembly language for all it would matter.

oconnor663 11mo ago
> and the rate of change is low (or zero) This jives with a point that the Google Security Blog made last year: "The [memory safety] problem is overwhelmingly with new code...Code matures and gets safer with time." https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html https://security.googleblog.com/2024/09/eliminating-memory-s...
- miohtama 11mo ago
  You can find historical SQLite CVEs here https://www.sqlite.org/cves.html https://www.sqlite.org/cves.html Note that although code matures the chances of C Human error bugs will never go to zero. We have some bad incidents like Heartbleed to show this.
  - ziotom78 11mo ago
    Right, but I believe nobody can claim that Human error bugs go to zero for Rust code.
    - john_the_writer 11mo ago
      Agreed. I rather dislike the idea of "safe" coding languages. Fighting with a memory leak in an elixir app, for the past week. I never viewed c or c++ as unsafe. Writing code is hard, always has been, always will be. It is never safe.

pizlonator 11mo ago

SQLite works great in Fil-C with minimal changes. So, the argument for keeping SQLite written in C is that it gives the user the choice to either: - Build SQLite with Yolo-C, in which case you get excellent performance and lots of tooling. And it's boring in the way that SQLite devs like. But it's not "safe" in the sense of memory safe languages. - Build SQLite with Fil-C, in which case you get worse (but still quite good) performance and memory safety that exceeds what you'd get with a Rust/Go/Java/whatever rewrite. Recompiling with Fil-C is safer than a rewrite into other memory safe languages because Fil-C is safe through all dependencies, including the syscall layer. Like, making a syscall in Rust means writing some unsafe code where you could screw up buffer sizes or whatnot, while making a syscall in Fil-C means going through the Fil-C runtime.

wodenokoto 11mo ago

I think it’s more interesting that DuckDB is written in C++ and not rust than SQLite. SQLite is old, huge and known for its gigantic test coverage. There’s just so much to rewrite. DuckDB is from 2019, so new enough to jump on the “rust is safe and fast”

tonyhart7 11mo ago
if they write it on modern C++ then its alright tbh
jandrewrogers 11mo ago
If maximum performance is a top objective, it is probably because C++ produces faster binaries with less code. Modern C++ specifically also has a lot of nice compile-time safety features, especially for database-like code.
- wodenokoto 11mo ago
  I can’t verify those claims one way or another, but I’m interested to hear why they were downvoted.
  - jandrewrogers 11mo ago
    I've worked on a couple different projects that did substantial parallel development in C++20 and Rust, which created interesting opportunities for concrete comparison. It was performance-engineered code and we needed to validate their equivalence by testing them against each other. The practical differences are larger than the theoretical differences, so I would expect the gap to diminish over time. Rust reminded me of when I used to write database engines in Java. It required a lot more code, which has its own costs, but never really delivered on claims of comparable performance. The "more code" part largely comes down to the more limited ability to build good abstractions compared to C++20 and more limited composability. The "slower binaries" part comes down to worse codegen, which you can't blame on Rust per se, and a lot of extra overhead introduced in the code to satisfy the Rust safety model that would simply not be required in other systems languages. Safety is a mixed bag. Rust can check several things at compile-time that C++20 cannot. C++20 can check several things at compile-time that Rust cannot. For high-performance database-y code, memory is allocated at startup and is accessed via managed index handles. Rust does the same thing. In these types of memory models, i.e. no dynamic allocation and no raw pointers, both Rust and C++20 offer similar memory safety guarantees. Most high-performance software is thread-per-core that is almost purely single-threaded, so thread-safety concerns are limited. That said, stripping away all of the above, the only real advantage that C++20 has its much more powerful toolset for building abstractions. Its performance and unique safety elements are based almost entirely on the ability to build concise, contextual, and highly composable abstractions as needed. This is not a feature that should be downplayed, I immediately miss it when I use most other languages.

tonyhart7 11mo ago

because Rust isnt out yet back then????

vincent-manis 11mo ago

The point about bounds checking in `safe' languages is well taken, it does prevent 100% test coverage. As we all agree, SQLite has been exhaustively tested, and arguments for bounds checking in it are therefore weakened. Still, that's not an argument for replicating this practice elsewhere, not unless you are Dr Hipp and willing to work very hard at testing. C.A.R. Hoare's comment on eliminating runtime checks in release builds is well-taken here: “What would we think of a sailing enthusiast who wears his life-jacket when training on dry land but takes it off as soon as he goes to sea?” I am not Dr Hipp, and therefore I like run-time checks.

slashdev 11mo ago

This is ignoring the elephant in the room: SQLite is being rewritten in Rust and it's going quite well. https://github.com/tursodatabase/turso https://github.com/tursodatabase/turso It has async I/O support on Linux with io_uring, vector support, BEGIN CONCURRENT for improved write throughput using multi-version concurrency control (MVCC), Encryption at rest, incremental computation using DBSP for incremental view maintenance and query subscriptions. Time will tell, but this may well be the future of SQLite.

zvmaz 11mo ago
In the link you provided, this is what I read: "An in-process SQL database, compatible with SQLite." Compatible with SQLite. So it's another database?
- simonw 11mo ago
  Yeah, I don't think it even counts as a fork - it's a ground-up re-implementation which is already adding features that go beyond the original.
- ForHackernews 11mo ago
  It's a fork and a rewrite.
tonyhart7 11mo ago
so its sqlite++ since they added bunch of things on top of that
metaltyphoon 11mo ago
The moment turso becomes stable , SQLite will inevitably fade away with time if they don’t rethink how contributions should be taken. I honestly believe the Linux philosophy of software development will be what catapults turso forward.
assimpleaspossi 11mo ago
>>SQLite is being rewritten in Rust SQLite is NOT being rewritten in Rust! >>Turso Database is an in-process SQL database written in Rust, compatible with SQLite.

jokoon 11mo ago

I wonder if the hype helps rust being a better language At this point I wish the creators of the language could talk about what rust is bad at.

steveklabnik 11mo ago
Folks involved often do! Talking about what’s not great is the only path towards getting better, because you have to identify pain points in order to fix them.
- estebank 11mo ago
  I would go as far as saying that 90% of managing the project is properly communicating, discussing and addressing the ways in which Rust sucks. The all-hands in NL earlier this year was wall to wall meetings about how much things suck and what to do about them! I mean this in the best possible way. ^_^

matt3210 11mo ago

I can compile c anywhere and for any processor, which can’t be said for rust

saalweachter 11mo ago

I think beyond the historical reasons why C was the best choice when SQLite was being developed, or the advantages it has today, there's also just no reason to rewrite SQLite in another language. We don't have to have one implementation of a lightweight SQL database. You can go out right now and start your own implementation in Rust or C++ or Go or Lisp or whatever you like! You can even make compatible APIs for it so that it can be a drop-in replacement for SQLite! No one can stop you! You don't need permission! But why would we want to throw away the perfectly good C implementation, and why would we expect the C experts who have been carefully maintaining SQLite for a quarter century to be the ones to learn a new language and start over?

turtletontine 11mo ago
Thanks for this, I fully agree. One frustration I have with the modern moment is the tendency to view anything more than five years old with disdain, as utterly irrelevant and obsolete. Maybe I’m just going old, but I like my technology dependable and boring, especially software. Glad to see someone express respect for the decades of expertise that have gone into things we take for granted.
- friendly_wizard 11mo ago
  I think we owe an equally proportionate measure of respect to the authors of boring old sqlite who, given the depth of their experience, recognize that there may in fact be benefits to be gained from the rewrite and are open to exploring the possibility. The blockers as stated, I have no doubt, were carefully considered with a level of insight few of us could match. If and when the time is right, if they choose to undertake the effort I'm sure the juice will be worth the squeeze. The fact that they're not jumping in blindly today is telling. Even more telling will be if they do eventually go that route.
AdamJacobMuller 11mo ago
One good reason is that people have written golang adapters, so that you can use sqlite databases without cgo. I agree to what I think you're saying which is that "sqlite" has, to some degree, become so ubiquitous that it's evolved beyond a single implementation. We, of course, have sqlite the C library but there is also sqlite the database file format and there is no reason we can't have an sqlite implementation in golang (we already do) and one in pure rust too. I imagine that in the future that will happen (pure rust implementation) and that perhaps at some point much further in the future, that may even become the dominant implementation.

binary132 11mo ago

I love him so much.

sema4hacker 11mo ago

"Why SQLite is coded in C..." is an explanation, as documented at sqlite.org. "Why is SQLite coded in C and not Rust?" is a question, which immediately makes me want to ask "Why do you need SQLite coded in Rust?".

lifthrasiir 11mo ago
Because the title has been editorialized.
urbandw311er 11mo ago
Indeed. Why is SQLite coded in C and not BASIC?
t14n 11mo ago
fwiw there's a project doing just that: https://github.com/tursodatabase/turso https://github.com/tursodatabase/turso they have a blog hinting at some answers as to "why": https://turso.tech/blog/introducing-limbo-a-complete-rewrite-of-sqlite-in-rust https://turso.tech/blog/introducing-limbo-a-complete-rewrite...

rednafi 11mo ago

Also, Rust needs a better stdlib. A crate for every little thing is kinda nuts. One reason I enjoy Go is because of the pragmatic stdlib. On most cases, I can get away without pulling in any 3p deps. Now of course Go doesn’t work where you can’t tolerate GC pauses and need some sort of FFI. But because of the stdlib and faster compilation, Go somehow feels lighter than Rust.

firesteelrain 11mo ago
Rust doesn’t really need a better stdlib as much as a broader one, since it is intentionally narrow. Go’s stdlib includes opinions like net/http and templates that Rust leaves to crates. The trade-off is Rust favors stability and portability at the core, while Go favors out-of-the-box ergonomics. Both approaches work, just for different teams.
tonyhart7 11mo ago
me when I dont know ball:
afdbcreid 11mo ago
Is Rust's stdlib worse than C's? It's not an argument here.

globalnode 11mo ago

[flagged]

coolThingsFirst 11mo ago

I don't want to sound cynical but a lot of it has to deal with the simplicity of the language. It's much harder to find a good Rust engineer than a C one. When all you have is pointers and structs it's much easier to meet the requirements for the role.

system2 11mo ago

What's up with SQLite news lately? I feel like I see at least 1-2 posts about it per day.

Havoc 11mo ago

For a project that is functionally “done” switching doesn’t make sense. Something like kernel code where you know it’ll continue to evolve - there going through the pain may be worth it

daxfohl 11mo ago

It sounds like the core doesn't even allocate, and presumably the extended library allocates in limited places using safe patterns. So there wouldn't be much benefit from Rust anyway, I'd think. Had SQLite ever had a memory leak or use-after-delete bug on a production release? If so, that answers the question. But I've never heard of one. Also, does it use doubly linked lists or graphs at all? Those can, in a way, be safer in C since Rust makes you roll your own virtual pointer arena.

steveklabnik 11mo ago
Rust’s memory safety guarantees aren’t exclusive to hep allocation. In fact, the language doesn’t heap allocate at all. You can write a linked list the same way you would in C if you wish.
thinkharderdev 11mo ago
> Also, does it use doubly linked lists or graphs at all? Those can, in a way, be safer in C since Rust makes you roll your own virtual pointer arena. You can implement a linked list in Rust the same as you would in C using raw pointers and some unsafe code. In fact there is one in the standard library.
dathinab 11mo ago
> Had SQLite ever had a memory leak or use-after-delete bug on a production release? sure, it's an old library they had pretty much anything (not because they don't know what they are doing but because shit happens) lets check CVEs of the last few years: - CVE-2025-29088 type confusion - CVE-2025-29087 out of bounds write - CVE-2025-7458 integer overflow, possible in optimized rust but test builds check for it - CVE-2025-6965 memory corruption, rust might not have helped - CVE-2025-3277 integer overflow, rust might have helped - CVE-2024-0232 use after free - CVE-2023-36191 segmentation violation, unclear if rust would have helped - CVE-2023-7104 buffer overflow - CVE-2022-46908 validation logic error - CVE-2022-35737 array bounds overflow - CVE-2021-45346 memory leak ... as you can see the majority of CVEs of sqlite are much less likely in rust (but a rust sqlite impl. likely would use unsafe, so not impossible) as a side note there being so many CVEs in 2025 seem to be related to better some companies (e.g. Google) having done quite a bit of fuzz testing of SQLite other takeaways: - 100% branch coverage is nice, but doesn't guarantee memory soundness in C - given how deeply people look for CVEs in SQLite the number of CVEs found is not at all as bad as it might look but also one final question: SQLite uses some of the best C programmers out there, only they merge anything to the code, it had very limited degree of change compared to a typical company project. And we still have memory vulnerabilities. How is anyone still arguing for C for new projects?

kazinator 11mo ago

> The C language is old and boring. It is a well-known and well-understood language. So you might think, but there is a committee actively undermining this, not to mention compiler people keeping things exciting also. There is a dogged adherence to backward compatibility, so that you can't pretend C has not gone anywhere in thirty-five years, if you like --- provided you aren't invoking too much undefined behavior. (You can't as easily pretend that your compiler has not gone anywhere in 35 years with regard to things you are doing out of spec.)

next_xibalba 11mo ago

Aren't SQLite’s bottlenecks primarily io-bound (not CPU)? If so, fopen, fread, or syscalls are the most important to performance and pure language efficiency wouldn't be limiter.

morshu9001 11mo ago

This is what I expected. Rust is the first thing that has been worth considering as a C replacement. C++ wasn't.

dathinab 11mo ago

It's kinda sad to read as most of their arguments might seem right at first but if put under scrutiny really fall apart. Like why defend C in 2025 when you only have to defend C in 2000 and then argue you have a old, stable, deeply tested, C code base which has no problem with anything like "commonly having memory safety issues" and is maintained by a small group of people very highly skilled in C. Like that argument alone is all you need, a win, simple straight forward, hard to contest. But most of the other arguments they list can be picked apart and are only half true.

Deanoumean 11mo ago
The argument you propose only works for justifying a maintenance mode for and old codebase. If you want to take the chance to turn away new developers from complex abominations like C++ and Rust and garbage collected sloths like Java and get them to consider a comparatively simple but ubiquitous language that is C, you have to offer more.
- dangus 11mo ago
  Is SQLite looking for new developers? Will they ever need a large amount of developers like a mega-corp that needs to hire 100 React engineers?
  - metaltyphoon 11mo ago
    No, but as morbid as this sounds, the three(?) devs one day will pass away so now what?
    - hoppp 11mo ago
      Then the rights will be sold to a FAANG or an open souce fork like libSQL will live on.
      - colejohnson66 11mo ago
        SQLite is public domain (as much as is legally possible). So there's no "rights" to "sell" except the trademark.

ternaryoperator 11mo ago

> Recoding SQLite in Go is unlikely since Go hates assert() Any idea what this refers to? assert is a macro in C. Is the implication that OP wants the capability of testing conditions and then turning off the tests in a production release? If so, then I think the argument is more that go hates the idea of a preprocessor. Or have I misunderstood the point being made?

steveklabnik 11mo ago
https://go.dev/doc/faq#assertions https://go.dev/doc/faq#assertions
- ternaryoperator 11mo ago
  Steve, thanks for taking the time to point me to this on-point passage.

psyclobe 11mo ago

SQLite is a true landmark, c not withstanding it just happened to be the right tool at the right time and by now anything else is well not as interesting as what they have going on now; totally bucks the trend of throw away software.

unsungNovelty 11mo ago

As I write more code, use more software and read about rewrites... The biggest gripe I have with a rewrite is... A lof of the time we rewrite for feature parity. Not the exact same thing. So you are kind ignoring/missing/forgetting all those edge cases and patches that were added along the way for so many niche or otherwise reasons. This means broken software. Something which used to work before but not anymore. They'll have to encounter all of them again in the wild and fix it again. Obviously if we are to rewrite an important piece of software like this, you'd emphasise more on all of these. But it's hard for me to comprehend whether it will be 100%. But other than sqlite, think SDL. If it is to be rewritten. It's really hard for me to comprehend that it's negligible in effect. Am guessing horrible releases before it gets better. Users complaining for things that used work. C is going to be there long after the next Rust is where my money is. And even if Rust is still present, there would be a new Rust then. So why rewrite? Rewrites shouldn't be the default thinking no?

a-saleh 11mo ago

Ok, I didn't expect such a high praise for rust. I am not joking.

deleted 11mo ago

[deleted]

steeleduncan 11mo ago

> SQLite could be recoded in Go Sqlite has been recoded (automatically) in Go a while ago [1], and it is widely deployed > would probably introduce far more bugs than would be fixed It runs against the same test suite with no issues > and it may also result in slower code It is quite a lot slower, but it is still widely used as it turns out that the convenience of a native port outweighs the performance penalty in most cases. I don't think SQLite should be rewritten in Go, Rust, Zig, Nim, Swift ... but ANSI C is a subset of the feature set of most modern programming languages. Projects such as this could be written and maintained in C indefinitely, and be automatically translated to other languages for the convenience of users in those languages [1] https://pkg.go.dev/modernc.org/sqlite https://pkg.go.dev/modernc.org/sqlite

sim7c00 11mo ago
> would probably introduce far more bugs than would be fixed It runs against the same test suite with no issues - that proves nothing about bugs existing or not.
ChrisRR 11mo ago
> It runs against the same test suite with no issues That doesn't guarantee no bugs. It just means that the existing behaviour covered by the tests is still the same. It may introduce new issues in untested edge cases or performance issues
sgbeal 11mo ago
> It runs against the same test suite with no issues It runs against the same public test suite. The proprietary test suite is much more intensive.

skeester 11mo ago

[dead]

negrel 11mo ago

[dead]

6r17 11mo ago

If I remember correctly most of SQLite "closed-source" leverage comes from the test-suite - which probably cannot transpose to another language as easily. Ultimately there are already other solutions coming up re-writing it in rust or go.

belter 11mo ago

Some of the most interesting comments are out of: "3. Why Isn't SQLite Coded In A "Safe" Language?" "....Safe languages insert additional machine branches to do things like verify that array accesses are in-bounds. In correct code, those branches are never taken. That means that the machine code cannot be 100% branch tested, which is an important component of SQLite's quality strategy..." "...Safe languages usually want to abort if they encounter an out-of-memory (OOM) situation. SQLite is designed to recover gracefully from an OOM. It is unclear how this could be accomplished in the current crop of safe languages..."

dusted 11mo ago

In my opinion, you don't get to ask "why is X done by Y" before you've done X yourself by something not Y and not Y by proxy either.

BiraIgnacio 11mo ago

"1. C Is Best"

MomsAVoxell 11mo ago

Back in the good ol'/bad ol' days of the very early web/Internet, I had the fortune of working with someone who, lets say, has kind of a background in certain operating systems circles. Not only had this fellow built a functional ISP in one of the toughest markets (at that time), in the world - but he'd also managed to build the database engine and quite a few of the other tools that ran that ISP, and was in danger of setting a few standards for a few things which, since then, have long since settled out, but .. nevertheless .. it could've been. Anyway, this fellow wrote everything in C. His web page, his TODO.h for the day .. he had C-based tools for managing his docs, for doing syncs between various systems under his command (often in very far-away locations, and even under water a couple times) .. everything, in C. The database system he wrote in pure C was, at the time, quite a delight. It gave a few folks further up the road a bit of a tight neck. He went on to do an OS, because of course he did. Just sayin', SQLite devs aren't the only ones who got this right. ;)

1vuio0pswjnm7 11mo ago

Why doesn't ON CONFLICT(column_name) accept multiple arguments, i.e., multiple columns One stupid workaround is combining multiple columns into one, with values separated by a space, for example. This works when each column value is always a string containing no spaces Another stupid workaround, probably slower, might be to hash the multiple columns into a new column and use ON CONFLICT(newcolumn_name)

netrap 11mo ago

Why not just say "because I don't want to change it"? rustaceans will always say your argument isn't valid for some reason or another. IDGAF, it's written in C because it is -- that's it!

zenxyzzy 11mo ago

I'm really getting tired of resume driven development. Choosing technology X because the coder wants it on their resume is a truly shitty reason. Rust is just the lastest trendy bullshit that will make meh programmers into superstars.

deleted 11mo ago

[deleted]