5 ms·

My biggest frustration is some of the questions came close to forcing some real answers, the biggest being data deletion. Zuckerberg was always extremely carefu

by alva 8y ago

My biggest frustration is some of the questions came close to forcing some real answers, the biggest being data deletion. Zuckerberg was always extremely careful when it came to questions of account deletion. One question asked "when a user deletes their account do you erase the data", Zuckerberg visibly tripped up and referred back to "data is deleted". There is quite a bit of discussion online on whether FB simply marks the data in their database as deleted or whether it is actually removed.

nemothekid 8y ago
>There is quite a bit of discussion online on whether FB simply marks the data in their database as deleted or whether it is actually removed. This shouldn't be a discussion. I can't think of a single non-toy database that actually deletes data when you press DELETE. All the LSM databases (pretty much anything using Facebook's RocksDB) doesn't actually remove data until a compaction event - and MySql/Postgres don't actually delete data until you a VACUUM occurs. (then there's whole minefield of did the OS actually remove the data, or did the filesystem just mark the affected area as deleted, but didn't scrub the bits). Then theres the backups - FB may have a ton of backups in cold storage that may have your data that wasn't removed. The conversation about "FB simply marks the data in their database as deleted or whether it is actually removed." is unfruitful. The answer is "no" for almost every company on the planet, not just FB.
- anarazel 8y ago
  > Postgres don't actually delete data until you a VACUUM occurs. FWIW, a large portion of cleanup happens on access. If the entire page will not be accessed until the next VACUUM that doesn't change anything, but in OLTP workloads that space will often be reclaimed much earlier. Doesn't change anything about your larger point though.
- IAmEveryone 8y ago
  There is obviously a certain leeway in time allowed for a deletion request to actually be fulfilled. But other than that: if technology gets in the way what people and the law demand, it's technology that needs to change, not people.
- cozicoolmail 8y ago
  Exactly - and for all intents and purposes, that should be good enough. If the data is marked "deleted" and then written over within some reasonable amount of time (whatever their VACCUM cycle is to rewrite the immutable partition) then that should be good enough. Now as for backups, this is much harder. I don't know how reasonable it is to ask them to discard backups in cold storage - seems like a compliance nightmare that would absolutely punish smaller players that can't build infrastructure to do that.
  - phn 8y ago
    Regarding backups, I think the answer might be: "keep the backups only as long as you need". Is it really reasonable for an active site to keep backups older than a couple of months? A year?
- anarazel 8y ago
  I don't think that's a meaningful comparison. There's quite the difference between "The data can be readily returned from the database if you omit the NOT deleted clause." and "You can use forensic tools to get data back. Maybe."
seagull 8y ago
(Disclaimer: I left FB several years back, this may not be completely accurate today. However, I would be surprised if this in particular has changed.) I worked on this system. Account deletion is a completely separate process from content deletion. The latter can in some cases just mark data as deleted or ready to be garbage collected, but the former actually deletes everything associated with an account. There are some nuances that are hard to explain well in front of a bunch of Senators though: your messages to other people will still stay in their inbox, it's not feasible to purge backups so there are tapes with your data around for several months. Also, derived data (ML models, etc) trained with the account data sticks around, although this isn't really identifiable info.

by alva 8y ago

nemothekid 8y ago

>There is quite a bit of discussion online on whether FB simply marks the data in their database as deleted or whether it is actually removed. This shouldn't be a discussion. I can't think of a single non-toy database that actually deletes data when you press DELETE. All the LSM databases (pretty much anything using Facebook's RocksDB) doesn't actually remove data until a compaction event - and MySql/Postgres don't actually delete data until you a VACUUM occurs. (then there's whole minefield of did the OS actually remove the data, or did the filesystem just mark the affected area as deleted, but didn't scrub the bits). Then theres the backups - FB may have a ton of backups in cold storage that may have your data that wasn't removed. The conversation about "FB simply marks the data in their database as deleted or whether it is actually removed." is unfruitful. The answer is "no" for almost every company on the planet, not just FB.

anarazel 8y ago
> Postgres don't actually delete data until you a VACUUM occurs. FWIW, a large portion of cleanup happens on access. If the entire page will not be accessed until the next VACUUM that doesn't change anything, but in OLTP workloads that space will often be reclaimed much earlier. Doesn't change anything about your larger point though.
IAmEveryone 8y ago
There is obviously a certain leeway in time allowed for a deletion request to actually be fulfilled. But other than that: if technology gets in the way what people and the law demand, it's technology that needs to change, not people.
cozicoolmail 8y ago
Exactly - and for all intents and purposes, that should be good enough. If the data is marked "deleted" and then written over within some reasonable amount of time (whatever their VACCUM cycle is to rewrite the immutable partition) then that should be good enough. Now as for backups, this is much harder. I don't know how reasonable it is to ask them to discard backups in cold storage - seems like a compliance nightmare that would absolutely punish smaller players that can't build infrastructure to do that.
- phn 8y ago
  Regarding backups, I think the answer might be: "keep the backups only as long as you need". Is it really reasonable for an active site to keep backups older than a couple of months? A year?
anarazel 8y ago
I don't think that's a meaningful comparison. There's quite the difference between "The data can be readily returned from the database if you omit the NOT deleted clause." and "You can use forensic tools to get data back. Maybe."

seagull 8y ago

(Disclaimer: I left FB several years back, this may not be completely accurate today. However, I would be surprised if this in particular has changed.) I worked on this system. Account deletion is a completely separate process from content deletion. The latter can in some cases just mark data as deleted or ready to be garbage collected, but the former actually deletes everything associated with an account. There are some nuances that are hard to explain well in front of a bunch of Senators though: your messages to other people will still stay in their inbox, it's not feasible to purge backups so there are tapes with your data around for several months. Also, derived data (ML models, etc) trained with the account data sticks around, although this isn't really identifiable info.