5 ms·

Sure the data is lost. A session commonly holds arbitrary state, and even if it’s just the login information. This is ridiculous.

by 9dev 4mo ago

trumpdong 4mo ago
If you consider it important, you have to store it in a real database. No buts. If you don't consider it important, sharded redis works fine.
- 9dev 4mo ago
  Redis is a real database. If I wasn’t convinced it could retain data I hand it, I wouldn’t use it in the first place. Just because it works for your use case right now doesn’t mean there isn’t room for improvements to support others too.
  - amtamt 3mo ago
    > Redis is a real database No, redis is a memory cache, with some ACID like features bolt on. Even real databases have hard time maintaining consistency across nodes. CAP is a real constraint.
  - trumpdong 4mo ago
    > Redis is a real database. Oh good, then you don't need to do any of the stuff that you suggested to do
    - 9dev 4mo ago
      These two concerns are not mutually exclusive, the kind of database or data stored within it doesn't give any availability guarantees on its own. Even a single Postgres instance, which I suppose fits your understanding of a real database, is a single point of failure and not a highly available setup: If your database server goes down, clients get errors and the database is thus unavailable.
tossandthrow 4mo ago
Obviously these are application decisions. You, obviously, don't commit important data only to a session that you can loose, if the application does not allow it. We use redis as infrastructure. To route events and as a cache. For us redis could go down and we would merely see a degradation of our service with no data loss. I recommend using redis like that. And then use a database that supports transactions for real data problems. But we are different. And that's OK.
- 9dev 4mo ago
  This discussion is a bit weird. We started off from, Redis should have better availability guarantees. Specifically to avoid the degradation of service you described. But that requires running on multiple instances, which in turn requires to share the data across all replicas.
amtamt 3mo ago
No two processes can guarantee data consistency unless using shared memory with some kind of locking on update. And given two servers don't share memory, two processes running on these servers can not guarantee consistency either. To put the simple terms... App writes to node-A, node-A (/process on node-A) crashes before change is synced from node-A to node-B, data is lost. This is true for redis and true for postgresql/ mysql or any similar database. Difference between redis and a "database" is that database protects against this problem by writing change to durable storage before telling app that write is successful. Redis So if one wants to have a consistent session storage, one should use a proper database or use AOF redis persistense with single node (https://chatgpt.com/s/t_6a24ab818e2881918db959cec8d8cc2d https://chatgpt.com/s/t_6a24ab818e2881918db959cec8d8cc2d)
- 9dev 3mo ago
  First up, if I wanted to talk to a machine, I would've asked one myself. Then, I don't understand your point really: Yes, the CAP theorem is a thing. There are compromise solutions available however to enable highly available data storage. Some of them for Redis too, but they are more complicated than those for other database engines. Which is the point of this discussion.
  - amtamt 3mo ago
    Point is... with AOF and RDB enabled, and wait command used in sane manner, one can get reasonable consistency with a significant speed tradeoff and increase in application complexity. So if consistent cache is needed, one can have that with some compromises, but then probably one could use a database straight away.
    - 9dev 3mo ago
      Again: Redis is a database, not just a cache - it doesn’t care if you store ephemeral cache artifacts or customer records within it. Redis doesn’t pose any constraints on the type of data. Inversely, you can use Postgres as a semi or fully consistent cache. And yes, what you’re saying is technically correct, even a well-tuned single node doesn’t solve the availability problem: if it goes down, you have an outage. To avoid that, you need multiple instances to provide the same data, avoiding downtime if one of the nodes breaks eventually.

Sure the data is lost. A session commonly holds arbitrary state, and even if it’s just the login information. This is ridiculous.

by 9dev 4mo ago

Sure the data is lost. A session commonly holds arbitrary state, and even if it’s just the login information. This is ridiculous.

trumpdong 4mo ago

If you consider it important, you have to store it in a real database. No buts. If you don't consider it important, sharded redis works fine.

9dev 4mo ago
Redis is a real database. If I wasn’t convinced it could retain data I hand it, I wouldn’t use it in the first place. Just because it works for your use case right now doesn’t mean there isn’t room for improvements to support others too.
- amtamt 3mo ago
  > Redis is a real database No, redis is a memory cache, with some ACID like features bolt on. Even real databases have hard time maintaining consistency across nodes. CAP is a real constraint.
- trumpdong 4mo ago
  > Redis is a real database. Oh good, then you don't need to do any of the stuff that you suggested to do
  - 9dev 4mo ago
    These two concerns are not mutually exclusive, the kind of database or data stored within it doesn't give any availability guarantees on its own. Even a single Postgres instance, which I suppose fits your understanding of a real database, is a single point of failure and not a highly available setup: If your database server goes down, clients get errors and the database is thus unavailable.

tossandthrow 4mo ago

Obviously these are application decisions. You, obviously, don't commit important data only to a session that you can loose, if the application does not allow it. We use redis as infrastructure. To route events and as a cache. For us redis could go down and we would merely see a degradation of our service with no data loss. I recommend using redis like that. And then use a database that supports transactions for real data problems. But we are different. And that's OK.

9dev 4mo ago
This discussion is a bit weird. We started off from, Redis should have better availability guarantees. Specifically to avoid the degradation of service you described. But that requires running on multiple instances, which in turn requires to share the data across all replicas.

amtamt 3mo ago

No two processes can guarantee data consistency unless using shared memory with some kind of locking on update. And given two servers don't share memory, two processes running on these servers can not guarantee consistency either. To put the simple terms... App writes to node-A, node-A (/process on node-A) crashes before change is synced from node-A to node-B, data is lost. This is true for redis and true for postgresql/ mysql or any similar database. Difference between redis and a "database" is that database protects against this problem by writing change to durable storage before telling app that write is successful. Redis So if one wants to have a consistent session storage, one should use a proper database or use AOF redis persistense with single node (https://chatgpt.com/s/t_6a24ab818e2881918db959cec8d8cc2d https://chatgpt.com/s/t_6a24ab818e2881918db959cec8d8cc2d)

9dev 3mo ago
First up, if I wanted to talk to a machine, I would've asked one myself. Then, I don't understand your point really: Yes, the CAP theorem is a thing. There are compromise solutions available however to enable highly available data storage. Some of them for Redis too, but they are more complicated than those for other database engines. Which is the point of this discussion.
- amtamt 3mo ago
  Point is... with AOF and RDB enabled, and wait command used in sane manner, one can get reasonable consistency with a significant speed tradeoff and increase in application complexity. So if consistent cache is needed, one can have that with some compromises, but then probably one could use a database straight away.
  - 9dev 3mo ago
    Again: Redis is a database, not just a cache - it doesn’t care if you store ephemeral cache artifacts or customer records within it. Redis doesn’t pose any constraints on the type of data. Inversely, you can use Postgres as a semi or fully consistent cache. And yes, what you’re saying is technically correct, even a well-tuned single node doesn’t solve the availability problem: if it goes down, you have an outage. To avoid that, you need multiple instances to provide the same data, avoiding downtime if one of the nodes breaks eventually.