8 ms·

I truly appreciate your in-depth articulation. Indeed, the Yjs state update can be problematic due to its growing size and constant updates. I will have a loo

by Pi9h 2y ago

I truly appreciate your in-depth articulation. Indeed, the Yjs state update can be problematic due to its growing size and constant updates. I will have a look at MyRocks. Reference pointers sound more plausible. I spent time analyzing and deciding my usage of uuid7 from different perspectives. From the git logs, you can see it came at the last minute.

benpacker 2y ago
As someone who: - would love to self host, and would much prefer keeping it to Postgres / Redis for which there is widespread knowledge on how to host - would like you to keep your development velocity up I'd encourage you not to switch databases, or at least to defer this for a while. I can't imagine you'll have issues with the amount of WAL written for quite a while, and by that time, the world could be quite different (OrioleDB might be mature! https://github.com/orioledb/orioledb https://github.com/orioledb/orioledb)
- sgarland 2y ago
  MyRocks speaks MySQL. It’s just a different storage engine behind MySQL, replacing InnoDB. The major change I can think of would be replacing the `uuid` type with `bin(16)`, and needing to do the encoding and decoding in the app. Re: development velocity, IMO there’s a solid base product now. I feel like it’s a great time to make a solid change to help future velocity, but I’m not OP.
- Pi9h 2y ago
  I should have been more explicit. I do not intend to change the Postgres database or introduce a new one. I’m sure this won’t be an issue for the majority use-case. However, I am open to learning more about alternate ways to efficiently handle Yjs state updates, which may be useful for a cloud version that would run at scale. If I were to go that way, it would not affect self-hosted users and would probably be via a reference pointer and not a database switch. This is absolutely not an issue at the moment. Nothing to worry about.
sgarland 2y ago
Agreed that if you want to keep the relational part, using the DB to store pointers is probably the easiest, though it might require rethinking search. I appreciate you at least considering the various options. If you do nothing else, tuning Postgres parameters, optimizing column ordering, and removing duplicated indices will be a great step forward that is completely backwards-compatible.

by Pi9h 2y ago

benpacker 2y ago

As someone who: - would love to self host, and would much prefer keeping it to Postgres / Redis for which there is widespread knowledge on how to host - would like you to keep your development velocity up I'd encourage you not to switch databases, or at least to defer this for a while. I can't imagine you'll have issues with the amount of WAL written for quite a while, and by that time, the world could be quite different (OrioleDB might be mature! https://github.com/orioledb/orioledb https://github.com/orioledb/orioledb)

sgarland 2y ago
MyRocks speaks MySQL. It’s just a different storage engine behind MySQL, replacing InnoDB. The major change I can think of would be replacing the `uuid` type with `bin(16)`, and needing to do the encoding and decoding in the app. Re: development velocity, IMO there’s a solid base product now. I feel like it’s a great time to make a solid change to help future velocity, but I’m not OP.
Pi9h 2y ago
I should have been more explicit. I do not intend to change the Postgres database or introduce a new one. I’m sure this won’t be an issue for the majority use-case. However, I am open to learning more about alternate ways to efficiently handle Yjs state updates, which may be useful for a cloud version that would run at scale. If I were to go that way, it would not affect self-hosted users and would probably be via a reference pointer and not a database switch. This is absolutely not an issue at the moment. Nothing to worry about.

sgarland 2y ago

Agreed that if you want to keep the relational part, using the DB to store pointers is probably the easiest, though it might require rethinking search. I appreciate you at least considering the various options. If you do nothing else, tuning Postgres parameters, optimizing column ordering, and removing duplicated indices will be a great step forward that is completely backwards-compatible.