5 ms·

You can now directly sync Postgres with Redis

22 points by vijaymohan1979 1y ago

deleted 1y ago
[deleted]
yohannparis 1y ago
No Pricing information nor an open-source project. I don't know what to do with this product.
- jasonlotito 1y ago
  Assuming the link to the repo is correct: it's open source. https://github.com/redfly-ai-org/redfly.ai/blob/main/LICENSE https://github.com/redfly-ai-org/redfly.ai/blob/main/LICENSE
  - bradleyjkemp 1y ago
    I believe this is another case of abusing GitHub for visibility, not actually doing anything meaningful open source The code in the repo just seems to be connection code ("provides a way for anybody to test our Redis synchronization service on demand") rather than the sync service itself
    - deleted 1y ago
      [deleted]
- vijaymohan1979 1y ago
  https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A
maxmcd 1y ago
Since db cache consistency is a complex research space with things like https://readyset.io/ https://readyset.io/ and https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final170_update.pdf https://www.usenix.org/system/files/conference/nsdi13/nsdi13... it would be great to hear more specifics about what this is giving up in order to sidestep all that complexity.
- _1tem 1y ago
  There’s also Polyscale as a service.
  - vijaymohan1979 1y ago
    https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A
- vijaymohan1979 1y ago
  https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A
henning 1y ago
It'd be nice to see documentation of how it works. Is it using a replication channel for Postgres? The write-ahead log? It looks like it does schema introspection and then works off that. How does it handle schema changes? How does this work in scenarios where you have multi-master replication? What's the compatibility with popular Postgres extensions like PostGIS? What happens if the database is rebooted or crashes? Yes I realize the source is there.
- porridgeraisin 1y ago
  > Yes, I realize the source is there It isn't actually. The codein the repo just seems to be some scaffolding.
- vijaymohan1979 1y ago
  https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A
hangonhn 1y ago
Did you really mean to leave your AES key in the code in RedflyEncryptionKeys? > public const string AesKey = ... Also, the way AES is used in the code is not good practice. It seems to be using plain AES ( https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.aes?view=net-9.0 https://learn.microsoft.com/en-us/dotnet/api/system.security... ), which isn't meant to be used bare like that. It needs to be coupled with a digest algorithm to protect the ciphertext. Maybe use AesGcm instead?
- foxyv 1y ago
  The reason for this is that AES is not an authenticated algorithm so there is no way to determine if the ciphertext has been modified since encryption. The ciphertext could be modified/corrupted and you wouldn't know. Also, AES is deterministic and will encrypt the same data the same way every time. This means if you are encrypting a lot of fields you will be able to do statistical attacks. Using an initialization vector with AES GCM is similar to salting a hash. This way there is no statistical method to determine the contents of the ciphertext.
- vijaymohan1979 1y ago
  https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A
deleted 1y ago
[deleted]
solatic 1y ago
Can someone ELI5 where this is going to be valuable? If you set up Postgres read replicas, make sure your queries are covered by appropriate indexes, and your hot queries are handled by Postgres from RAM anyway - do you really need a separate Redis cache? I feel like anytime I'm going to pull in Redis for a cache, it's going to be for data that I'm not interested in having guaranteed persistency for, like session tokens, and so it'll be for data that I wouldn't be storing in Postgres anyway?
- lotharcable 1y ago
  Probably useful for sharing stuff with services that already have access to redis, but you don't want to add SQL client dependencies for and/or don't want to give access to your database. But you want to be able to store/have single source of authority and update them using a SQL database. This would then eliminate the need to maintain a separate lambda or cron job or something like that to keep them in sync.
- vijaymohan1979 1y ago
  https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A

deleted 1y ago

[deleted]

yohannparis 1y ago

No Pricing information nor an open-source project. I don't know what to do with this product.

jasonlotito 1y ago
Assuming the link to the repo is correct: it's open source. https://github.com/redfly-ai-org/redfly.ai/blob/main/LICENSE https://github.com/redfly-ai-org/redfly.ai/blob/main/LICENSE
- bradleyjkemp 1y ago
  I believe this is another case of abusing GitHub for visibility, not actually doing anything meaningful open source The code in the repo just seems to be connection code ("provides a way for anybody to test our Redis synchronization service on demand") rather than the sync service itself
  - deleted 1y ago
    [deleted]
vijaymohan1979 1y ago
https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A

maxmcd 1y ago

Since db cache consistency is a complex research space with things like https://readyset.io/ https://readyset.io/ and https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final170_update.pdf https://www.usenix.org/system/files/conference/nsdi13/nsdi13... it would be great to hear more specifics about what this is giving up in order to sidestep all that complexity.

_1tem 1y ago
There’s also Polyscale as a service.
- vijaymohan1979 1y ago
  https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A
vijaymohan1979 1y ago
https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A

henning 1y ago

It'd be nice to see documentation of how it works. Is it using a replication channel for Postgres? The write-ahead log? It looks like it does schema introspection and then works off that. How does it handle schema changes? How does this work in scenarios where you have multi-master replication? What's the compatibility with popular Postgres extensions like PostGIS? What happens if the database is rebooted or crashes? Yes I realize the source is there.

porridgeraisin 1y ago
> Yes, I realize the source is there It isn't actually. The codein the repo just seems to be some scaffolding.
vijaymohan1979 1y ago
https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A

hangonhn 1y ago

Did you really mean to leave your AES key in the code in RedflyEncryptionKeys? > public const string AesKey = ... Also, the way AES is used in the code is not good practice. It seems to be using plain AES ( https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.aes?view=net-9.0 https://learn.microsoft.com/en-us/dotnet/api/system.security... ), which isn't meant to be used bare like that. It needs to be coupled with a digest algorithm to protect the ciphertext. Maybe use AesGcm instead?

foxyv 1y ago
The reason for this is that AES is not an authenticated algorithm so there is no way to determine if the ciphertext has been modified since encryption. The ciphertext could be modified/corrupted and you wouldn't know. Also, AES is deterministic and will encrypt the same data the same way every time. This means if you are encrypting a lot of fields you will be able to do statistical attacks. Using an initialization vector with AES GCM is similar to salting a hash. This way there is no statistical method to determine the contents of the ciphertext.
vijaymohan1979 1y ago
https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A

deleted 1y ago

[deleted]

solatic 1y ago

Can someone ELI5 where this is going to be valuable? If you set up Postgres read replicas, make sure your queries are covered by appropriate indexes, and your hot queries are handled by Postgres from RAM anyway - do you really need a separate Redis cache? I feel like anytime I'm going to pull in Redis for a cache, it's going to be for data that I'm not interested in having guaranteed persistency for, like session tokens, and so it'll be for data that I wouldn't be storing in Postgres anyway?

lotharcable 1y ago
Probably useful for sharing stuff with services that already have access to redis, but you don't want to add SQL client dependencies for and/or don't want to give access to your database. But you want to be able to store/have single source of authority and update them using a SQL database. This would then eliminate the need to maintain a separate lambda or cron job or something like that to keep them in sync.
vijaymohan1979 1y ago
https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A