14 ms·

Personal data storage is an idea whose time has come

407 points by erlend_sh 1y ago

esafak 1y ago
Isn't this what web3 was about? Was it the wrong approach?
Al-Khwarizmi 1y ago
Glad to see a mention to Opera Unite. I found it to be a really revolutionary idea, anyone could have a simple static website running in their browser with zero tech knowledge needed. I think the world would have been better if that idea succeeded as a way for people to share their content, rather than the highly monetized and manipulative social networks.
pydry 1y ago
The problem isnt technical feasibility it is market incentives. Most companies have no incentive to let you hold your data when they can just hold it for you. If they do this they can mine it for data to improve their product as well as sell or otherwise indirectly profit from it. And, it's easier. Also, while the market for privacy focused products isnt nothing, the number of people willing to pay a lot extra to compensate for the missed opportunities companies get by collecting your data is, i think, smaller than many people imagine. Which is sad. I think the only way it will grow to an appreciable size is by seeing up close and personal what a really vicious stasi-like secret police does with dragnet surveillance and come out the other side, with scars. I believe we've only seen a small taste of this.
- fidotron 1y ago
  > The problem isnt technical feasibility it is market incentives. This is understating it honestly. The software industry has become completely reliant on renting data access back to users to maintain subscription revenue. One effect of this is it has devalued the actual software in the eyes of users to such a degree that virtually no one will pay for alternatives, certainly not enough to compensate the development cost.
- dist-epoch 1y ago
  You got the market incentives wrong. Most people have no incentive of owning their data. Otherwise the companies which don't give you that would die out because people wouldn't use them if they cared. Same fallacy as believing smartphones are giant and with non-user swappable batteries because somehow smartphone making companies are forcing this on the market, instead of the real reason which is that it's what consumers want.
  - kalaksi 1y ago
    I don't think it's so black-and-white. There are multiple forces at play simultaneously. I agree that people don't care enough about owning their data for it to matter more than what the companies want to push, which is of course monetizing the data and maximizing user lock-in. Similarly, I think it's in the companies' interests to use non-swappable batteries: simpler and cheaper to manufacture (I think this is the main reason) and the device is made obsolete earlier which is an added bonus. Maybe small improvements in size etc., but that's a very small difference. Modern phones are already larger even with non-swappable batteries so I'm not sure it mattered. But again, having a non-swappable battery has to be weighed against other features, and availability of alternatives. In the end, people just care more about the other features, even though swappable battery would be a good thing. Just to conclude: I don't believe markets work to fully cater to what customers actually want. It's more like customers (and other parties) get a compromise between what different parties in the market want.
  - btbuildem 1y ago
    > the real reason which is that it's what consumers want Consumers want what they're told to want by a constant barrage of commercial propaganda. Devices are large and non-serviceable because this way they can be sold with a higher profit margin. Side effect being that the larger screens make the embedded commercial propaganda more effective and easy to deliver.
  - pydry 1y ago
    I get what you're saying. People want vendor lock in...otherwise they wouldnt pay for it. People want bait and switch sales tactics...otherwise they wouldnt work. People are perfectly fine with high rents...if they didnt, they would not pay them. People want their smartphones to be deliberately slowed down when they get old...otherwise theyd vote against it with their wallet.
- theshrike79 1y ago
  Of all the big name corporations Apple is the only one I can see doing this. I'm still hoping they release an Apple TV Pro with fully local LLM capability that's shared with everyone in the family - adding a few TB of disk space to it for local data storage and backups wouldn't be a massive thing.
tjpnz 1y ago
If this takes off I fear big tech very quickly finding friends among those pushing for things like chat control, while potentially reevaluating some of its more consumer friendly "views" towards privacy. Very easy to undermine something when you start speaking of its potential to facilitate CSAM.
- outime 1y ago
  This guy has eyes and eyes can be used to visualize CSAM! What if...
- anonbuddy 1y ago
  that is exactly what is going to happen, as more people become aware. that's why we all need to exercise our rights and freedoms. I'm scared that if we fail to do this in next few years. And let the AI be used in similar ways like it has been used to create social media algorithms. Then we are all fucked! Whoever owns your AI owns you, so it better be you who owns it!
seu 1y ago
The fact that the AT Protocol relies on everyone having a domain name, which is a centralized system over which few people have control, and about whose workings most people have no clue about, is problematic. Also impractical, once we consider that - as far as I can understand - 8 billion people should have their own domain name.
- deleted 1y ago
  [deleted]
- diggan 1y ago
  > The fact that the AT Protocol relies on everyone having a domain name Well, either that or someone else hosting their identity (see did:plc), which seems to be the part you say should exist? Probably DNS is the most decentralized centralized system we have available today that most people can actually use, unless I'm missing some obviously better way of doing the same thing?
  - dist-epoch 1y ago
    > Well, either that or someone else hosting their identity (see did:plc) Wouldn't that turn into did:plc:facebook all over again?
    - diggan 1y ago
      If there was no way of moving away from it, probably yeah. But since you can migrate from a did:plc to did:web, I don't feel like they're very similar situations at all.
  - nsndndkddk 1y ago
    The thing your missing is ICANN is headquartered in the US. The US political situation is dire and I think this could be a real danger for the internet at large. We might end up with disagreeing DNS worldwide at some point. E.g. if you hold a domain and have a non-authorized viewpoint so your DNS entry gets snuffed. But from a practical point of view a decentralised system should not rely on domain name ownership. Any computer can generate a private/public key pair, which is all you need for identify.
dist-epoch 1y ago
How do I post a message on Discord/Twitter/Instagram from my personal data storage? If this is not supported, this idea is born-dead. Very few will use it, for the regular person the conversation goes like this: - Who can see my personal data storage posts? Can someone with Twitter see them? - No, but you'll own your data - Bye So maybe start with something which backs-up what you post on Twitter/Instagram/Discord to your personal data storage through APIs/data export.... This has no downside if it's easy to "activate"
- BoredPositron 1y ago
  The creator/consumer divide is still 90/10. Your example just doesn't matter.
  - dist-epoch 1y ago
    If I don't create anything, and just consume creators, what do I need a personal data store for?
    - obk0943t 1y ago
      Just your existence itself already create a lot of data ;)
    - anonbuddy 1y ago
      you just created a comment here. also your government, your service providers and many other entities are creating data on your behalf
  - dotancohen 1y ago
    I think you got the ratio backwards, but assuming that then your argument serves to bolster GP's position.
- CuriouslyC 1y ago
  At this point distributed protocols are getting good enough that for a large class of social applications, network effects are the only thing keeping the incumbents in place. The irony of ad supported free services is that if you just let the advertisers pay you directly for eyeball time then paid for your services, it'd be better for you financially while keeping the web pure outside of the "paid to consume ads" app.
akoboldfrying 1y ago
Who has an incentive to provide a Solid server? Not big social media companies, who want the personal information that Solid attempts to withhold. I don't think anyone is prepared to offer a convenient, high quality Solid-based social media experience to everyone for free, because that costs a lot of money. And if you know anything about human nature, it will have to be convenient and completely free in order to have a chance of capturing any mindshare outside of weird tech nerd circles. > the platforms should be asking us what kinds of data they may copy from our servers, and only with strictly temporary allowances. Until practical homomorphic encryption arrives, I don't see how this temporariness can be enforced. If we rely on promises or regulation instead of the technical ability to enforce this, how is that any better than today's social media companies promising not to do anything bad with the data they have on us?
- erlend_sh 1y ago
  See this response: https://news.ycombinator.com/item?id=45480884 https://news.ycombinator.com/item?id=45480884 Aka: I agree it can’t be dine with technology; it has to be done with regulation, and the EU example already models a lot of it.
- anonbuddy 1y ago
  'that costs a lot of money' price of intelligence is dropping day by day like it or not, sooner or later price incentives for someone to host such social media experience could become financially viable
Khaine 1y ago
It was an idea that never went away. Many people have wanted to self host everything. Sadly companies have found it easier to centralise, and then as a bonus can monetise that data.
- 9dev 1y ago
  It wasn’t the companies but the users that found it easier. There’s a reason why everyone’s on Facebook, instagram, and gmail instead of running their own hosts—because it’s vastly easier for the majority of people to do so, and because everyone else is there. We have not solved decentralisation in an accessible and useful way yet, and the incentives won’t change until we do. If ever.
  - nubinetwork 1y ago
    God forbid that people actually have to learn and do something instead of sitting around being a doomscrolling tiktok zombie... /s
    - bluebarbet 1y ago
      Slightly offtopic, but the sheer scale of the phenomenon you allude to - of screen-addled zombification - is really turbo-charging my own misanthropy. People staggering around, necks hunched, eyes down, all but glued to their miserable little toys. Everywhere, everyone, all the time. It's just pathetic. I guess I had hoped humans would have more self-control than this.
      - nkrisc 1y ago
        Stop viewing them in isolation and view them as a product of their environment. They weren't born with a phone in hand, someone gave it to them and someone created Tik Tok for them.
        bluebarbet 1y ago
        That's a fair argument. It's also unfalsifiable and based on an underlying personal worldview. Specifically (I would venture) an "us and them" view of things where history is determined by groups and power - a left-wing outlook, basically! I'm a bit of a liberal individualist by nature, I see personal responsibility and autonomy as a thing. I'm not sure how I'd go about deprogramming myself of this even if I wanted to. But it would help with the misanthropy, for sure.
keepamovin 1y ago
I’m continuing to explore ideas like this in my DN project (short for DownloadNet or Discernet). The core concept: a browser controller / instrumentation harness that, by default, saves everything you browse to disk, and makes it available via full-text search or a browsable alphabetical index. The browser controller actually runs its own local server that handles indexing and archiving on your disk, while the front end lives inside your browser as a dashboard or control pane. So it’s both a locally hosted app and a browser extension of sorts. This is still a work in progress, but one direction I want to push further is allowing users to publish curated collections or search indexes of their browsing history. More likely, though, you’d create a separate archive centered on a topic you care about, and as you browse you selectively add pages to that topic. Over time, you end up with a niche search engine tied to your expertise. If that archive is good, others might find it valuable—and you might choose to publish it from your own machine. With tunneling tech (Cloudflare, Tor, etc.), you can expose your local box to the public internet. The vision is: user-sovereign data, but still shareable. You could even federate groups of topic-based archives into a shared search ecosystem, useful for domains like biotech or other specialized fields. Another crucial point: DownloadNet archives your browsing in real time. It doesn’t crawl externally; it captures exactly what you see, including sites you access via institutional credentials (e.g. research journals behind paywalls). Then you can optionally share those archives with a trusted group. I’m also exploring a web-document bundle format: package an interactive set of web pages (not just one) into a self-contained snapshot you can send (e.g. via email). The recipient can browse that snapshot locally, with all internal links intact, as of a particular moment in time. It’s a simple but powerful idea, and I think it has real growth potential in the data-sovereignty space. I started this as a passion project, and I believe many others care deeply about these ideas too. If you’re interested or want to get involved, head to the repository. One way my vision differs from something like Solid is the philosophy of adoption: rather than launching with a full-blown protocol, you start with a simple tool that users adopt, extend, and share. Over time, emergent use cases and community practices shape the system. It’s bottom-up rather than top-down. I’m not dissing Solid — I understand its aims and don’t see this as strictly competitive or exclusive. But I feel the incremental, user-led route is likelier to produce something sustainable. You grow it in the wild, learn what users actually need, and adapt. Instead of trying to design for all cases in advance, you let real-world use teach you what matters. Anyway, that’s the gist of my vision—and how it diverges from other approaches like the one in the article you referenced. While it may seem as a condemnation of other ideas, it's not. So please don't take it that way. If this is something you could get into, I encourage you come on over to the repo and share your contribution. I also riff more on Solid, this article and the approach of DN if you're interested, here: https://github.com/DO-SAY-GO/dn/wiki/What-is-DiskerNet-and-how-does-it-compare-to-other-approaches-like-SOLID%3F https://github.com/DO-SAY-GO/dn/wiki/What-is-DiskerNet-and-h...
crazygringo 1y ago
> Rather than being in countless separate places on the internet in the hands of whomever it had been resold to, your data is in one place, controlled by you. I don't see how this follows. The moment you create/share data with a site, what's to prevent them from reselling it? The only thing this seems to attempt to solve is portability/interop (and moving control of and responsibility for blocking/moderation/spam to users rather than sites). I don't see how it helps at all with privacy or you "controlling" who gets your data. If you give it to site A but not data collector B, what's preventing A from selling it to B? As far as I can tell, the situation will remain identical to how it is today. Your data will never be in one place unless you never share it. The moment you use it with other sites or services, it is stored there too, out of your control.
- erlend_sh 1y ago
  > The moment you create/share data with a site, what's to prevent them from reselling it? If I can clearly assert origin and personal ownership of my data, I can forbid further reselling of it. EU legislation shows that we can actually have the right to demand that a company forgets about us. Asserting such rights become easier the more accurately we define what data is ours.
  - crazygringo 1y ago
    > If I can clearly assert origin and personal ownership of my data, I can forbid further reselling of it. Can you? A site's TOS will say that by sharing your data, you grant them the right to display, reuse and redistribute it, the same as you do now. And that would take precedence because your host provided the data. They requested and you provided. The only thing that would change that is actual legislation. But then the legislation is orthogonal to personal data storage. If you want legislation for that, pursue legislation for that. Personal data storage is completely separate, and the two shouldn't be confused with each other.
    - layer8 1y ago
      The right granted by the TOS elapses when you cancel the respective service, or when you revoke your consent (in which case the service provider may possibly cancel the service). (Some TOS are also simply illegal to begin with.) That’s what the GP is referring to.
rob_c 1y ago
Aka, more dunking on "the cloud". Now it's cool to be able to do so. How about we go back 20yr and train a generation of unix sysadmins and self host at companies and at home.
mactavish88 1y ago
For those of us who've been around for some time and still value privacy, this sort of paradigm is obvious. The trouble isn't a lack of the right technologies - I'd argue it's a problem in the go-to-market strategy of those building these products/technologies. Ideas flow along lines carved out by power/influence. Facebook's early strategy was to start with restricting its usage to people at Harvard University - arguably a highly influential institution - and then expand outwards to other highly influential institutions. Only once the "who's who" from those institutions were already onboard did they let down the walls to allow us plebs in, and we all rushed in head-first. X's current strategy leverages Musk's visibility and influence (for better or worse). Get the most prominent influencers onboard with your decentralized social network, and others will follow (dramatically easier said than done, of course). But without a significant contingent of influencers/powerful people, your network's DoA.
- btbuildem 1y ago
  > prominent influencers onboard with your decentralized social network That's sort of a contradiction, no? Or at least it assumes transplanting the same mechanisms into a new milieu -- which I argue is something to leave behind, because it's those very mechanisms that have ruined the current internet. I think instead of tapping into the same addictive attention economy schemes, the distributed / decentralized socials could onboard people en-masse by providing what's missing there, and filling a real need.
  - mactavish88 1y ago
    Even if they fill a real need, their go-to-market strategy will determine whether the masses even know about them, or give a damn about trying them out in the first place.
InMice 1y ago
Among the first page and 2nd page (top 60) there is always atleast 1 post about how we're gonnna "take back the web" or make it back into some form of our 90s millenial nostalgia memories, self hosting, federated this or that, etc etc. Meanwhile - Nothing changes, everything generally gets worse and younger generations come into the world with no memories of the 90s internet or the world before mobile devices or surveillence everywhere. Applying for a job or apartment or anything today means creating endless pointless copies of your pesonal information in databases across the world that will eventually be neglected, hacked, exploited, sold off etc I dont know the way out if there is one, I guess we can keep fantasizing and thinking about it. It just feels like it would be easier to get the earth to start spinning the other way sometimes.
- erlend_sh 1y ago
  This is demonstrably not fantasy as the example case is a fully productionized network (Bluesky and the rest of AT-net) that’s having real-world impact to the point where it’s under threat from several authoritarian states.
  - ffsm8 1y ago
    It has? Don't get me wrong, I'm in the tech industry and generally more online then likely 95% of the population, but ime ... Nobody even knows what bluesky is? (They also don't know what X is, though they DO know what Twitter is) And even more niche products like mostodon, the fediverse altogether etc are entirely unknown to most of the tech industry too.
    - tomrod 1y ago
      Sometimes tech leads the world, however unwillingly, to better outcomes.
      - oceanplexian 1y ago
        Tech is downstream of culture. Seems that smart people keep getting duped by this idea. For example Twitter and Facebook didn’t result in a bunch of Democracies springing up after the Arab Spring, it resulted in the complete opposite. Tech simply amplifies the culture that was already there.
gibsonf1 1y ago
Systems Twin Intelligence, where a Pod represents the full space-time information for part of the world, using Solid Protocol: https://graphmetrix.com/trinpod-server https://graphmetrix.com/trinpod-server The W3C Linked Web Storage (LWS) working group is transforming Solid into a web standard: https://www.w3.org/groups/wg/lws/ https://www.w3.org/groups/wg/lws/
zeroCalories 1y ago
I find the ideas of data coops to be very appealing. I don't want to depend on faceless mega-corps like Google to host stuff like my email, but I also don't find the idea self-hosting to be realistic. I wouldn't mind paying for the security since losing access to certain accounts would be a disaster, but I'm already locked in, and the benefits of existing services would be marginal compared to the cost of moving.
- anonbuddy 1y ago
  ideally you should be able in a simple way to host your stuff, in this case in a POD. That service should be provided by a utility company, same way we have internet providers now. They will be well regulated and it would be in their interest to safely hold your data because if not, they would face legal and financial consequences. All other services would read/write from your Pod.
gcanyon 1y ago
Both of these proposals (as far as I've read them, YMMV) fail the evolutionary test. At the scale we're talking about, ideas must proceed as evolution does: not with a far-away goal in mind, but with incremental changes, each of which individually must be an improvement over the status quo. We are at (near) a significant local maximum, and (again, as far as I've read, which is not all of it for sure) the people pitching this form of information control have given no set of steps from here to there without significant cost/effort. Of course they don't have to have the whole path in mind. By definition they just need the first step or two. But they must be steps up. You don't get wings by wanting to fly; first you need feathers to keep warm (I am not an evolutionary biologist, I don't know if that's a valid theory).
- jauntywundrkind 1y ago
  99.9% of BlueSky users use only Bluesky services. But BlueSky has a Personal Data Service for each. That means: Those users have credible exit to take their data off BlueSky's hosting to someplace else (and as of a week or two ago to move back to BlueSky if they want). Those users can put whatever kind of data they want in their PDS. They can host their git data via https://tangled.org https://tangled.org . They can store their music listening scrobbles with https://teal.fm https://teal.fm . They can blog on https://leaflet.pub https://leaflet.pub . And there's been rapidly advancing host it yourself options. Plenty of folk individually or collectively host PDS. There are alternate relays that collect &n syndicate out everyone's PDS data as that changes. Hosting the aggregation layer is significantly harder especially if you are trying to fully connect the network but there are a couple & progress is good. it feels like a huge improvement over the status quo, and there's extremely visible developer energy building forward & rolling with the concepts. The breakdown on architecture allows for wins and work in various areas. The base seems solid, the core seems coherent & well built, built to scale not as one big thing but coherent layers. I think it's doing what you are asking for, and the signs of advancement & uptake warm my heart to see.
  - senordevnyc 1y ago
    99.9% of BlueSky users use only Bluesky services. I highly, highly doubt this, even in the narrowest sense of how many BlueSky users still actively post on X.
vuldin 1y ago
IPFS and Filecoin exist to solve this problem. https://ipfs.tech https://ipfs.tech https://filecoin.io https://filecoin.io
- robinkunz 1y ago
  thought the same.
- attila-lendvai 1y ago
  and https://www.ethswarm.org/ https://www.ethswarm.org/
lerp-io 1y ago
you store ur photos on fb same way you store your money at the bank and your code on github, its delegation of concerns, you can make same argument for literally anything....not using your own silicon, growing your own food, financing your own venture, owning your own land, etc etc.... maybe its more "secure" vs "less efficient" or some other tradeoff. and you have to get the right balance or take risks for optimal efficiency / profit/whatever your values are
dd_xplore 1y ago
When I was a kid, a 4GB pendrive was a huge thing for me. I used to think my 40GB HDD would never fill up, but then Internet started to grow. Today it doesn’t even matter how muc storage you have it’ll always fill up. I have started to self host quite a lot of stuff but eve then every storage solution has a life of 5-6 years in which atleast one of the components would fail. We click enormous amounts of photos but they do not have any impact like printed photo albums. With ever growing storage costs (both cloud based and self hosted) I’m thinking of going back to keep only important stuff that too in print format.
- ivanjermakov 1y ago
  In the age of abundance, smart prioritization is needed.
- Jaxan 1y ago
  We still print photo albums. I can strongly recommend this!
  - theshrike79 1y ago
    I bought a Canon SELPHY photo printer on a Black Friday sale last year. It prints archive quality photos we can put in an album to save forever. It's kind of fun to go through the thousands of photos in our digital photo libraries and pick the best and most impactful ones to print and save "forever".
- AdrianB1 1y ago
  I run a NAS, in various forms, for almost 20 years. The lifetime is quite longer, I still have ~ 10 year old drives in the backup NAS built on a Ryzen 1600 (8 years) and the average power supply works for me 10-12 years. The primary NAS is still on hardware that is more than 5 years old, except the drives that I just replaced with higher capacity. As I find the size of current drives bigger than my yearly additions (personal pictures and movies), I am quite happy with a 10 year lifetime at low usage. I would love some reliable and affordable long term offline storage, but backup tapes and a reader are not affordable and not in common use for end users. Otherwise I would build a tiered storage system with more reliability and even performance (nvme hot tier? maybe).
dangus 1y ago
This article seems pretty far detached from the problems that people experience using technology. It’s the kind of thing that only deeply technical people consider. When someone uses a service like Dropbox or iCloud Drive or Google Drive, they really aren’t experiencing any kind of problem where their data “isn’t theirs” or is “trapped.” It’s not that hard to migrate to something else and the services themselves are reasonably low-friction. In terms of social data, users don’t really have a major issue with the status quo, and those who do have already developed relatively popular solutions like Mastodon and BlueSky. Even “proprietary” photos applications like Apple Photos and Google Photos have very easy migration paths to other services. So what exactly is the problem we’re trying to solve here? Giving me an @Bob handle? Did I want that or need that?
- crazygringo 1y ago
  > In terms of social data, users don’t really have a major issue with the status quo That's exactly it. And with social media (unlike files and photo storage) migration isn't really something people care about, because it's about the present not the past. If you move from Twitter to Bluesky, does anyone care about moving their tweet history? They just want their list of followers to migrate over as much as possible, which happens relatively organically anyways.
skybrian 1y ago
Bluesky’s PDS is currently fairly limited due to the lack of support for private data and inadequate permissions [1]. Hopefully they’ll fix that soon. [1] https://bsky.app/profile/byarielm.fyi/post/3lz4vzzhybk2b https://bsky.app/profile/byarielm.fyi/post/3lz4vzzhybk2b
xenodium 1y ago
> Meanwhile - Nothing changes, everything generally gets worse https://LMNO.lol https://LMNO.lol is my grain of sand. I wasn't happy the state of blogging (tracking, bloat, ads, paywalls...), so I built https://LMNO.lol https://LMNO.lol. It's offline first and you can browse blogs from anywhere (even terminal). Your blog is a single Markdown file. Drag and drop it to the browser and your entire blog is generated. Custom domains are welcome. My blog is running off LMNO.lol that https://xenodium.com https://xenodium.com
lukeschlather 1y ago
I love the idea of personal data storage and I want it to be the default, but I think there are some possibly insurmountable technical problems. This article doesn't mention schema once, and schemas make seamless data portability virtually impossible. I've spent a week making sure a simple CRUD app could change a string field to a UUID field without causing any outage or bugs. You can export your data from Google or Facebook today, but then you need to write a copy of the source UI that faithfully replicates the way all those data fields are supposed to display. And tomorrow the source makes a change so what used to be one field is now two fields, oh and they also removed another field entirely so that data is just gone. Well, in future dumps anyway. Are you going to use the old schema or the new schema for your display? Is it possible to do both? When everything is in data silos, you can freely and safely change data format, which is something that needs to happen a lot as applications evolve. Even in a data silo, doing this is pretty tricky and bugs and data loss are significant risks. If you're trying to sync between an unbounded number of data repositories where each repository has potentially conflicting relationships with the data schema, data loss is practically assured. Another big problem is schema permissions and identity. I might have some piece of data that says "person A is allowed to see this set of fields" and another piece that says "person A is blocked from seeing this other set of fields." This gets synced to 3 different servers, one of those servers has no idea that userA is in fact person A. So you fail closed, but then the data on that server practically does not exist if the goal of this data repository is sharing some data with person A. You really can't do any sort of fine-grained access controls in a system where trust/identity/auditing is decentralized.
back2dafucha 1y ago
[dead]
impure-aqua 1y ago
I don't see what advantage any company gets from choosing to build products that enable personal data ownership. I say this as someone working on a venture with these sorts of design aims, it feels like pushing a boulder uphill often. The business model of cloud service providers makes a lot of sense- we have a system which stores and operates on your data, you pay some rental fee for us to store it and operate on it, easy peasy. The cost is related to both the utility of the operations the operator performs (to both the operator and the user) and the amount of data the user stores. Fundamentally this is how everything from Dropbox to Facebook is governed- Dropbox does not devise much utility per GB and users store a lot, so you rent per GB, but at Facebook, they don't store lots of your stuff, and on the data side maybe you don't get much value from it as it's a cesspit, but the data is valuable to Facebook to sell ads, etc, so they can provide the service for free. Importantly, you don't need to improve the product to continue extracting this rent, because the product you are selling is not Dropbox v4, Facebook v2.3, rather you are selling ongoing access to the rental. As soon as you introduce even simply a federated system where a few corporate operators are involved, it becomes very hard to justify extracting rent there as the network designer, as the operators are taking on the cost of actually storing the data. You have to really be iterating on the core product to use a SaaS business model here. Some things simply don't need a v4, does Dropbox really need that much iteration? Meanwhile as the system designer, life has become a lot more complex for you. Suddenly you cannot push unilateral sweeping changes to APIs, you need to version things in a way that is compatible between, say, one university updating their system but not the other. Since your users are a few large operators rather than millions of individuals, you lose the network effect advantage of being able to screw over a few users for the "greater good", since if you irritate one corporate client, you lose a lot of your install base. Why would you voluntarily choose this harder path as a company? Things get even worse as you increase the level of decentralization. The reality is users expect the polished experience that the rental companies can give you; they want their data always accessible so that their friend can see the pic they shared without needing to keep their own computers running, they want the "like counter" to go up without their personal node subscribing to messages from other nodes, etc. The only users that will accept a worse experience are people who have are motivated by their philosophy re: personal data ownership, and this crowd will want a FOSS solution, so you can say goodbye to charging them for Dropbox v4, they are simply not interested if you're not giving them the source code for free. (I suspect this is where the author sits, but fundamentally I don't think it will get mass appeal, most people simply do not care about data ownership above something that "just works".) So now you are dealing with problems like dynamic generation of redundant data and fault- and Byzantine-tolerant consensus algorithms so that your system can maintain function even when the user turns their computer off, and you have to deal with wrapped-key cryptography so that the redundant data can be split across all these user nodes without you worrying that an unauthorized user can read it, and then you have issues like how do you deal with nodes that are too slow to process updates (perhaps some user data needs to be stored in this conflict-free replicated datatype you devise), and eventually you go through all of this to... create a system that is less monetizable than the rental model, because you can't extract that rent for ongoing data storage, and we know users are not interested in actually paying for software.
ksec 1y ago
In terms of NAS, I have long wonder if there is a market for a combination of both online and offline. We will need at least 2 HDD for redundancy and to prevent bit riot. And the NAS will be sold as a whole package and subscription, with an encrypted backup services included for first 2 years and requires the backup subscription to work there after. The profit margin is first on the hardware and then on long tail backup which is charged like iCloud and Google storage per tier. Where your 1.5TB storage will be charged at 2TB storage. Before 2014 I would have thought Apple to potentially take this route for Time Capsule. Instead they doubled down on iCloud. Google will never take this route. Microsoft is not interested. Amazon should have done this and bundled with cold storage back up but their track record are not good enough. I doubt people trust Meta enough even if the solution was perfect. In pre 2012 you could at least bet on Apple to be somewhat customer centric. May be UniFi will do it. They just announced their 2 Bay UNAS and I only just discovered, they are a 40B market cap company. ( I thought they were much smaller )
- phkahler 1y ago
  >> And the NAS will be sold as a whole package and subscription... Misses the point entirely.
  - ksec 1y ago
    Data will need Backup to be safe. You could tell everyday customer to get NAS and they wouldn't know what is Bit Riot until they saw their Image and Video with errors or broken. They also wouldn't do off site backup. Company wants long subscription model. Right now everyone is only talking about options that are extreme in both ends.
- detaro 1y ago
  Synology sells cloud backup services for their NASes. And a bunch of other brands at least can easily connect to other services.
- Larrikin 1y ago
  >with an encrypted backup services included for first 2 years and requires the backup subscription to work there after. Its confusing if you mean the NAS will stop working if you stop paying for the subscription or not. If you can no longer access your data on the NAS without a subscription, then the NAS just becomes the cloud with an extra up front cost plus the cost of your own electricity. Personally I have started moving as much of my data out of the cloud as possible. I've got a Synology and a few single board computers running various services with a Synology in my parent's home for their photos. Their photos back up to my NAS and my data to their Synology. Its a shame Synology decided to enshitify this year for all products going forward, but UGreen looks like a suitable replacement when I outgrow my current NAS.
jauntywundrkind 1y ago
> Another spiritually similar idea being championed at the time came from the Opera browser folks who wanted to put "a web server in your browser". Opera Unite was such an awesome idea. https://arstechnica.com/information-technology/2009/06/opera-hoping-to-reinvent-the-web-by-making-browser-a-server/ https://arstechnica.com/information-technology/2009/06/opera... There was a neat idea a bit back to allow Service Workers to work across origin: foreign fetch. It wasn't on the internet, was only in the scope of your browser, but I thought it was such a neat advancement. Would have done so much to allow the offline web to weave itself. Alas, deprecated. https://developer.chrome.com/blog/foreign-fetch https://developer.chrome.com/blog/foreign-fetch
brendoncarroll 1y ago
I work on a FOSS project in this space, Blobcache. https://github.com/blobcache/blobcache https://github.com/blobcache/blobcache Trusting a server to store an application's state is a different thing from trusting it to author changes or to read the data. Servers should become dumber, and clients should become smarter. When I use an app, I want the app to load E2E encrypted state from storage (possibly on another machine, possibly not owned by me) make whatever changes and produce new encrypted data to send back to the server. The server should just be trusted for durability, and to prevent unauthorized access, but not to tell the truth about doing either of those things. Blobcache provides an API to facilitate transactions on E2EE state between a dumb storage server and any smart client. Blobcache can be installed on old hardware along with a VPN like Tailscale and then loaded up with data from other devices. Configuration is like SSH, drop a key in a configuration file to grant access. It removes most of the friction associated with consuming and producing storage as a resource. I'm using it to build E2EE version control like Git, but for your whole home directory. https://github.com/gotvc/got https://github.com/gotvc/got
- ianopolous 1y ago
  We should talk. This very similar to how apps use E2EE data in Peergos. Maybe we can join forces. https://peergos.org/posts/a-better-web https://peergos.org/posts/a-better-web
  - brendoncarroll 1y ago
    I couldn't find an email in your bio. You can reach me via the email at the bottom of my website (in my HN bio). Looking through the docs on Peergos, it looks like it's built on top of IPFS. I've been meaning to write some documentation for Blobcache comparing it to IPFS. I can give a quick gist here. Blobcache Volumes are similar to an IPNS name, and the set of IPFS blocks that can be transitively reached from it. A significant difference is that Blobcache Volumes expose a transaction API with serializable isolation semantics. IPFS provides distributed, available-but-inconsistent, cryptographically signed cells. IPFS chooses availability, and Blobcache chooses consistency. A Blobcache Volume corresponds to a specific entity maintained and controlled by a specific Node. An IPFS name exists as a distributed entity on the network. Most applications need some sort of consistent transactional cell (even if they don't realize it), but in order to be useful, inconsistent-but-available cells have to be used carefully in an application specific way. I blame this required application-specific care for the lack of adoption of CRDTs. There's a long tail of other differences too. IPFS was pretty badly behaved the last time I used it, trying to configure my router, and creating lots of connections to other nodes. Blobcache is more like a web browser; it creates transient connections in immediate response to user actions. That whole ecosystem is filled with complicated abstractions. Just as an example, the Multihash format is pervasive. It amounts to a tag for the algorithm used to create a hash, and then the hash output. I'd rather not have that indirection. All the hashes in Blobcache are 256 bits, and you set the algorithm per Volume. In Go that means the hashes can just be `[32]byte` instead of a slice and a tag and a table of algorithms. I haven't used IPFS in a while, but I became pretty familiar with it awhile ago. Had I been able to build any of the stuff I was interested in on top of it, I probably wouldn't have written Blobcache.
deleted 1y ago
[deleted]
didip 1y ago
As in self hosting? I love self hosting idea for myself out of principles. But unforunately it will never take off in a huge way because convenience is king. Average Joe and Jane want to install things with as little efforts as possible.
- AdrianB1 1y ago
  You can self host, but in order to be reachable you need to be discoverable. If the discovery is based on a mechanism that is controlled by someone else that can become an evil party, self-hosting in isolation is not too useful.
browningstreet 1y ago
Ideas like the Solid protocol have a limited timeframe to make it or go away. Not sure why anyone is still talking about it. TBL is rightfully a legend but this is now just a windmill. Next, please.
- righthand 1y ago
  This comment has inspired me to target SOLID and “things I can do to help” on my Sunday afternoon research block. This type of commentary is rife in this article thread and is now just a windmill. Next, please.
  - browningstreet 1y ago
    If Schneier can’t get more than 13 comments on a solid protocol crypto wallet, I personally don’t think that anyone will ever care about a solid protocol app of any kind. And I’m all for it, just calling it as I personally see it. Some things are fire, some things are warm, and some things are DOA. And I’m typing this on my Linux desktop (f’real). https://www.schneier.com/blog/archives/2024/07/data-wallets-using-the-solid-protocol.html https://www.schneier.com/blog/archives/2024/07/data-wallets-...
    - righthand 1y ago
      A Solid protocol cryptowallet. Arcane on top of arcane. I think it’s entirely unfair to dismiss technology because it hasn’t demanded immediate adoption by society. Solid is attempting to help define a better data future. We have working mechanisms in place but everyone is at a disadvantage except the people loyal to these giant corps. Attempting to give people the power to organize their data as they wish and to be used as they wish is worth it. Even if it doesn’t bring a renaissance.
      - browningstreet 1y ago
        Crypto wallets are not nearly as arcane as Solid. How many people have Binance accounts? Market share matters, critical mass matters, adoption matters. I'm suggesting that mindshare goes negative over time if these things aren't achieved, and when you have long-tail blog posts trying to pump life into it, it's pivot time. Righteousness alone doesn't win any of those things. It's been a very long time since Solid was released and it's like a whisper in the wind.
system7rocks 1y ago
I love this idea, and I imagine with years of successful lobbying efforts we could potentially get some laws passed to provide rights and clarity around our own data that could move us into this direction. But until then, while BlueSky is solid, I'll wait and see.
righthand 1y ago
> Whether these providers are strictly cooperatives in the formal sense isn't what's most important here though; I think the context of “encouraging people to switch” to a pds/solid/data coop, how they operate IS important. For two reasons: - data coop and controlling data opens the door to a new market if we’re going to join data coops, then we may as well try to share the profits from said coop fairly. Otherwise Facebook can step in as a “data-coop” and keep-on-keeping-on - a secondary effect is that now there is an incentive to move off facebook. If I can join my local Nowheresville.USA.town data coop and benefit directly to my community by storing data together then I am encouraged to switch to this new paradigm That is the major undiscussed shift to me. I believe the only way out of the Big Tech dystopia is to incentivize the switch. Even if the reward is pennies. Invest in the community oil well.
purpleKiwi 1y ago
How do I, as a complete noob, use the powers of atproto and the fact I own a domain?
dzonga 1y ago
I like the convenience of the cloud. but don't know whether its due to declining literacy rates / awareness etc. the cloud is nice and e.g google storage, iCloud but now with fast microsd's you can buy 1TB for $100. have a few copies then boom, you own your own data. but now phones don't allow you to have microsd's so here we are. likewise things like email etc instead of all of us being on gmail we could have community email servers etc.
- Larrikin 1y ago
  Sony phones continue to have MicroSD slots, headphone jacks, AND remain water resistant. They have been that way for at least a decade.
- layer8 1y ago
  I use Dropbox, but with an encryption overlay that also integrates into the iOS Files app for ease of use on mobile. So it’s possible to use cloud storage and still keep your data private.
AlienRobot 1y ago
When I read the title I couldn't help but think "did everyone forgot about hard disks?" I'm sure Tim Berners-Lee is much smarter than me, but I kind of feel there are some parallels between the idea of "owning" posts you made in a platform and the ludicrous idea of "owning" game items as NFTs in a blockchain. The latter promises interoperability that games would never deliver. I wonder about the former. At least I feel the major dealbreaker with this technology is just that it's not worth it for both parties involved. Right now, Facebook hosts all the posts and monetizes them with ads. So long as they are making money with ads, they have no reason to delete the posts they're hosting, as the posts are their money maker. But what happens if Facebook no longer "owns" the posts? So now your posts are in your "personal cloud", which means that unless they are encrypted any website or local app can display them, even without any ads. This means Facebook is no longer making money off the posts. Why would they accept this? On the flip side, who is paying for the hosting? Facebook? It's no longer their servers hosting the content, so I don't think so? Is Facebook supposed to pay the cloud service for metered API access? Can a cloud service offer different rates to different companies? Is the user supposed to pay for their cloud storage? So you're going to make users pay money to use facebook? What happens if a post violates the ToS? Can facebook delete my post in my cloud storage against my will? What happens if content that is legal where facebook operates is illegal where the cloud servers operate? Can I manually edit the data in my cloud storage like I'd be able with a file and then facebook has to treat every post as if it were untrusted input? What happens if my cloud storage closes my account? I just lose everything? Will I be able to back up my cloud to my hard disk and reupload it to another cloud so facebook can access it? How is facebook going to handle a single user with 2 clouds that have different content? I feel like this is a very complex thing and there are infinite questions that we can have about how this would be implemented in practice, while it's presented as simply "you own your data."
bawolff 1y ago
This is never going to happen. The incentives do not make sense. Any utopian future that requires a party to put in a lot of effort to change something in a way that would be a net negative for them, is just not going to happen. People do not spend money to change the world in a way that would be worse for them but better for other people.
- JumpCrisscross 1y ago
  > The incentives do not make sense Commercial incentives, no. If this preference exists, it would need to be pursued civically.
  - bawolff 1y ago
    I don't think the average citizen cares enough or even understands the benefits But lets say you get them on board and pass some law. Unless its a huge market like the EU or USA, probably what immediately happens is everyone pulls out of that market. Not out of malice but because they suddenly have to rewrite their app and that's probably quite expensive.
herf 1y ago
Vertically integrated apps are much cheaper to run - Instagram stores only a small fraction of your photos and makes a lot of money from them. It is somewhat harder to explain why we pay for things like iCloud, which mostly has no web API, only APIs for Apple devices. (Plenty of value there because it keeps you from having to buy a bigger iPhone.) But there are lots of these "almost general purpose" solutions, paying to upload files and store them, but where you cannot use them as you like. Why not dozens of apps running over the "web filesystem" like happens on the desktop? Two reasons: 1. Amazon pricing for transit/bandwidth is way higher than storage, and so it makes accessing your own data quite expensive if it is not in the same datacenter. 2. And there is a huge security and usability gap between "pick one photo" vs "give me [scoped] access to your Dropbox" Often the general-purpose mode does not work that well, is quite slow, or just costs a lot in bandwidth, a thing nobody wants to pay extra for when they're already paying for storage.
nayuki 1y ago
> Data Ownership as a conversation changes when data resides primarily with people-governed institutions rather than corporations. This is a false contrast. Corporations are institutions governed by people - specifically a board of directors, elected by shareholders. They aren't governed by aliens nor are they self-sentient. https://en.wikipedia.org/wiki/Institution#Examples https://en.wikipedia.org/wiki/Institution#Examples , https://en.wikipedia.org/wiki/Institution#Examples https://en.wikipedia.org/wiki/Institution#Examples Perhaps you meant that you are against for-profit corporations where the customer (who stores data) has no vote in the operation of the corporation? If so, then say that and don't imply it. People often use "corporation" as a pejorative, often in contrast to individual people. But they forget that a corporation is composed of people and ultimately owned by (some) people - but the kind of people that the writer does not like (shareholders, profit-makers, etc.). > Notice that Alice’s handle is now @alice.com. It's funny you're using .com as the example, because: > The domain com is a top-level domain (TLD) in the Domain Name System (DNS) of the Internet. Created in the first group of Internet domains in March of 1985, its name is derived from the word commercial, indicating its original intended purpose for subdomains registered by commercial organizations. Later, the domain opened for general purposes. -- https://en.wikipedia.org/wiki/.com https://en.wikipedia.org/wiki/.com Even when you're arguing against commercial organizations for storing personal data. Now you're just naming individual people as if they were companies.
- HenriTEL 1y ago
  To be fair nowadays .com refer much more to the default, main or official domain of an entity. Say you know the name of a non corporate website, are going to try .com first of something else?
  - XorNot 1y ago
    Yeah it strikes me that basically .com will eventually get canonically termed to mean "common" since that's how it's actually used.
est 1y ago
PDS is a cool idea, I hope the community addresses problems like content farm, spam and original attribution as a higher priority. Or I see malicious actors would wreck the federation mechanism. This is already the case with Email SMTPs
BrenBarn 1y ago
There are good ideas here. They won't come to fruition without some form of force. I'm not sure if TBL doesn't realize, is unwilling to accept, or just wants to avoid saying out loud that the only reason it worked for him to create the web as an open protocol is that no one was prepared for it so no one was in a position to co-opt it, commercialize it, and enshittify it. Now corporations are prepared. They will co-opt, commercialize, and enshittify whatever system you come up with unless it is accompanied by a giant hammer that will brutally destroy them if they don't change their wicked ways.
gatestone 1y ago
No one mentioned Upspin? A global file namespace (URL, but better...) and protocol to isolate public data users from private governance and storage, by gurus like Rob Pike. https://github.com/upspin/upspin https://github.com/upspin/upspin
Lumoscore 1y ago
It’s completely true that the system we use today—where a few big companies hold all of our private information in one place—is a bad model. It’s risky for security, and it means you have no real power or ownership over your own data. The good news is that we don’t have to wonder if a better way is possible. The technology is already here! Projects like Solid (Pods) and AT Protocol (PDS) have proven we can separate your information from the applications you use. You can put your data into your own secure digital "locker" or vault. The difficulty now is not the technology, but getting people to actually use it: 1- It’s Too Hard to Use: Setting up and managing your personal data locker is currently as complicated as managing a super-secret password for a crypto account. For everyone to adopt it, it needs to be way simpler than just clicking "Log in with Google." If it’s too much work for regular people, it will fail. 2- Big Companies Don't Want to Change (The Incentive Problem): The biggest tech companies make billions by collecting and using your data. They have no reason to switch to a system where they have to ask permission to use data they don't own, unless a major law forces them to, or a new competitor steals their users. 3- Privacy Isn't Enough (The Benefit Problem): Most people won't switch just for "privacy." The new system must offer clear, positive benefits, like letting you move all your friends to a new social app instantly, or securely filling out long forms with a single click from your data locker. The key to success is building user-friendly tools that hide all the complexity and make this new, secure way of managing data simple for everyone.
selinkocalar 1y ago
The concept makes sense but the execution is always where these things fall apart. Most people don't want to manage their own data infrastructure. The bigger issue is interoperability. Your personal data store is only useful if apps actually integrate with it, and getting developers to adopt new standards is tough.

407 points by erlend_sh 1y ago

Isn't this what web3 was about? Was it the wrong approach?

Glad to see a mention to Opera Unite. I found it to be a really revolutionary idea, anyone could have a simple static website running in their browser with zero tech knowledge needed. I think the world would have been better if that idea succeeded as a way for people to share their content, rather than the highly monetized and manipulative social networks.

pydry 1y ago

The problem isnt technical feasibility it is market incentives. Most companies have no incentive to let you hold your data when they can just hold it for you. If they do this they can mine it for data to improve their product as well as sell or otherwise indirectly profit from it. And, it's easier. Also, while the market for privacy focused products isnt nothing, the number of people willing to pay a lot extra to compensate for the missed opportunities companies get by collecting your data is, i think, smaller than many people imagine. Which is sad. I think the only way it will grow to an appreciable size is by seeing up close and personal what a really vicious stasi-like secret police does with dragnet surveillance and come out the other side, with scars. I believe we've only seen a small taste of this.

fidotron 1y ago
> The problem isnt technical feasibility it is market incentives. This is understating it honestly. The software industry has become completely reliant on renting data access back to users to maintain subscription revenue. One effect of this is it has devalued the actual software in the eyes of users to such a degree that virtually no one will pay for alternatives, certainly not enough to compensate the development cost.
dist-epoch 1y ago
You got the market incentives wrong. Most people have no incentive of owning their data. Otherwise the companies which don't give you that would die out because people wouldn't use them if they cared. Same fallacy as believing smartphones are giant and with non-user swappable batteries because somehow smartphone making companies are forcing this on the market, instead of the real reason which is that it's what consumers want.
- kalaksi 1y ago
  I don't think it's so black-and-white. There are multiple forces at play simultaneously. I agree that people don't care enough about owning their data for it to matter more than what the companies want to push, which is of course monetizing the data and maximizing user lock-in. Similarly, I think it's in the companies' interests to use non-swappable batteries: simpler and cheaper to manufacture (I think this is the main reason) and the device is made obsolete earlier which is an added bonus. Maybe small improvements in size etc., but that's a very small difference. Modern phones are already larger even with non-swappable batteries so I'm not sure it mattered. But again, having a non-swappable battery has to be weighed against other features, and availability of alternatives. In the end, people just care more about the other features, even though swappable battery would be a good thing. Just to conclude: I don't believe markets work to fully cater to what customers actually want. It's more like customers (and other parties) get a compromise between what different parties in the market want.
- btbuildem 1y ago
  > the real reason which is that it's what consumers want Consumers want what they're told to want by a constant barrage of commercial propaganda. Devices are large and non-serviceable because this way they can be sold with a higher profit margin. Side effect being that the larger screens make the embedded commercial propaganda more effective and easy to deliver.
- pydry 1y ago
  I get what you're saying. People want vendor lock in...otherwise they wouldnt pay for it. People want bait and switch sales tactics...otherwise they wouldnt work. People are perfectly fine with high rents...if they didnt, they would not pay them. People want their smartphones to be deliberately slowed down when they get old...otherwise theyd vote against it with their wallet.
theshrike79 1y ago
Of all the big name corporations Apple is the only one I can see doing this. I'm still hoping they release an Apple TV Pro with fully local LLM capability that's shared with everyone in the family - adding a few TB of disk space to it for local data storage and backups wouldn't be a massive thing.

tjpnz 1y ago

If this takes off I fear big tech very quickly finding friends among those pushing for things like chat control, while potentially reevaluating some of its more consumer friendly "views" towards privacy. Very easy to undermine something when you start speaking of its potential to facilitate CSAM.

outime 1y ago
This guy has eyes and eyes can be used to visualize CSAM! What if...
anonbuddy 1y ago
that is exactly what is going to happen, as more people become aware. that's why we all need to exercise our rights and freedoms. I'm scared that if we fail to do this in next few years. And let the AI be used in similar ways like it has been used to create social media algorithms. Then we are all fucked! Whoever owns your AI owns you, so it better be you who owns it!

seu 1y ago

The fact that the AT Protocol relies on everyone having a domain name, which is a centralized system over which few people have control, and about whose workings most people have no clue about, is problematic. Also impractical, once we consider that - as far as I can understand - 8 billion people should have their own domain name.

deleted 1y ago
[deleted]
diggan 1y ago
> The fact that the AT Protocol relies on everyone having a domain name Well, either that or someone else hosting their identity (see did:plc), which seems to be the part you say should exist? Probably DNS is the most decentralized centralized system we have available today that most people can actually use, unless I'm missing some obviously better way of doing the same thing?
- dist-epoch 1y ago
  > Well, either that or someone else hosting their identity (see did:plc) Wouldn't that turn into did:plc:facebook all over again?
  - diggan 1y ago
    If there was no way of moving away from it, probably yeah. But since you can migrate from a did:plc to did:web, I don't feel like they're very similar situations at all.
- nsndndkddk 1y ago
  The thing your missing is ICANN is headquartered in the US. The US political situation is dire and I think this could be a real danger for the internet at large. We might end up with disagreeing DNS worldwide at some point. E.g. if you hold a domain and have a non-authorized viewpoint so your DNS entry gets snuffed. But from a practical point of view a decentralised system should not rely on domain name ownership. Any computer can generate a private/public key pair, which is all you need for identify.

dist-epoch 1y ago

How do I post a message on Discord/Twitter/Instagram from my personal data storage? If this is not supported, this idea is born-dead. Very few will use it, for the regular person the conversation goes like this: - Who can see my personal data storage posts? Can someone with Twitter see them? - No, but you'll own your data - Bye So maybe start with something which backs-up what you post on Twitter/Instagram/Discord to your personal data storage through APIs/data export.... This has no downside if it's easy to "activate"

BoredPositron 1y ago
The creator/consumer divide is still 90/10. Your example just doesn't matter.
- dist-epoch 1y ago
  If I don't create anything, and just consume creators, what do I need a personal data store for?
  - obk0943t 1y ago
    Just your existence itself already create a lot of data ;)
  - anonbuddy 1y ago
    you just created a comment here. also your government, your service providers and many other entities are creating data on your behalf
- dotancohen 1y ago
  I think you got the ratio backwards, but assuming that then your argument serves to bolster GP's position.
CuriouslyC 1y ago
At this point distributed protocols are getting good enough that for a large class of social applications, network effects are the only thing keeping the incumbents in place. The irony of ad supported free services is that if you just let the advertisers pay you directly for eyeball time then paid for your services, it'd be better for you financially while keeping the web pure outside of the "paid to consume ads" app.

akoboldfrying 1y ago

Who has an incentive to provide a Solid server? Not big social media companies, who want the personal information that Solid attempts to withhold. I don't think anyone is prepared to offer a convenient, high quality Solid-based social media experience to everyone for free, because that costs a lot of money. And if you know anything about human nature, it will have to be convenient and completely free in order to have a chance of capturing any mindshare outside of weird tech nerd circles. > the platforms should be asking us what kinds of data they may copy from our servers, and only with strictly temporary allowances. Until practical homomorphic encryption arrives, I don't see how this temporariness can be enforced. If we rely on promises or regulation instead of the technical ability to enforce this, how is that any better than today's social media companies promising not to do anything bad with the data they have on us?

erlend_sh 1y ago
See this response: https://news.ycombinator.com/item?id=45480884 https://news.ycombinator.com/item?id=45480884 Aka: I agree it can’t be dine with technology; it has to be done with regulation, and the EU example already models a lot of it.
anonbuddy 1y ago
'that costs a lot of money' price of intelligence is dropping day by day like it or not, sooner or later price incentives for someone to host such social media experience could become financially viable

Khaine 1y ago

It was an idea that never went away. Many people have wanted to self host everything. Sadly companies have found it easier to centralise, and then as a bonus can monetise that data.

9dev 1y ago
It wasn’t the companies but the users that found it easier. There’s a reason why everyone’s on Facebook, instagram, and gmail instead of running their own hosts—because it’s vastly easier for the majority of people to do so, and because everyone else is there. We have not solved decentralisation in an accessible and useful way yet, and the incentives won’t change until we do. If ever.
- nubinetwork 1y ago
  God forbid that people actually have to learn and do something instead of sitting around being a doomscrolling tiktok zombie... /s
  - bluebarbet 1y ago
    Slightly offtopic, but the sheer scale of the phenomenon you allude to - of screen-addled zombification - is really turbo-charging my own misanthropy. People staggering around, necks hunched, eyes down, all but glued to their miserable little toys. Everywhere, everyone, all the time. It's just pathetic. I guess I had hoped humans would have more self-control than this.
    - nkrisc 1y ago
      Stop viewing them in isolation and view them as a product of their environment. They weren't born with a phone in hand, someone gave it to them and someone created Tik Tok for them.
      - bluebarbet 1y ago
        That's a fair argument. It's also unfalsifiable and based on an underlying personal worldview. Specifically (I would venture) an "us and them" view of things where history is determined by groups and power - a left-wing outlook, basically! I'm a bit of a liberal individualist by nature, I see personal responsibility and autonomy as a thing. I'm not sure how I'd go about deprogramming myself of this even if I wanted to. But it would help with the misanthropy, for sure.

keepamovin 1y ago

I’m continuing to explore ideas like this in my DN project (short for DownloadNet or Discernet). The core concept: a browser controller / instrumentation harness that, by default, saves everything you browse to disk, and makes it available via full-text search or a browsable alphabetical index. The browser controller actually runs its own local server that handles indexing and archiving on your disk, while the front end lives inside your browser as a dashboard or control pane. So it’s both a locally hosted app and a browser extension of sorts. This is still a work in progress, but one direction I want to push further is allowing users to publish curated collections or search indexes of their browsing history. More likely, though, you’d create a separate archive centered on a topic you care about, and as you browse you selectively add pages to that topic. Over time, you end up with a niche search engine tied to your expertise. If that archive is good, others might find it valuable—and you might choose to publish it from your own machine. With tunneling tech (Cloudflare, Tor, etc.), you can expose your local box to the public internet. The vision is: user-sovereign data, but still shareable. You could even federate groups of topic-based archives into a shared search ecosystem, useful for domains like biotech or other specialized fields. Another crucial point: DownloadNet archives your browsing in real time. It doesn’t crawl externally; it captures exactly what you see, including sites you access via institutional credentials (e.g. research journals behind paywalls). Then you can optionally share those archives with a trusted group. I’m also exploring a web-document bundle format: package an interactive set of web pages (not just one) into a self-contained snapshot you can send (e.g. via email). The recipient can browse that snapshot locally, with all internal links intact, as of a particular moment in time. It’s a simple but powerful idea, and I think it has real growth potential in the data-sovereignty space. I started this as a passion project, and I believe many others care deeply about these ideas too. If you’re interested or want to get involved, head to the repository. One way my vision differs from something like Solid is the philosophy of adoption: rather than launching with a full-blown protocol, you start with a simple tool that users adopt, extend, and share. Over time, emergent use cases and community practices shape the system. It’s bottom-up rather than top-down. I’m not dissing Solid — I understand its aims and don’t see this as strictly competitive or exclusive. But I feel the incremental, user-led route is likelier to produce something sustainable. You grow it in the wild, learn what users actually need, and adapt. Instead of trying to design for all cases in advance, you let real-world use teach you what matters. Anyway, that’s the gist of my vision—and how it diverges from other approaches like the one in the article you referenced. While it may seem as a condemnation of other ideas, it's not. So please don't take it that way. If this is something you could get into, I encourage you come on over to the repo and share your contribution. I also riff more on Solid, this article and the approach of DN if you're interested, here: https://github.com/DO-SAY-GO/dn/wiki/What-is-DiskerNet-and-how-does-it-compare-to-other-approaches-like-SOLID%3F https://github.com/DO-SAY-GO/dn/wiki/What-is-DiskerNet-and-h...

crazygringo 1y ago

> Rather than being in countless separate places on the internet in the hands of whomever it had been resold to, your data is in one place, controlled by you. I don't see how this follows. The moment you create/share data with a site, what's to prevent them from reselling it? The only thing this seems to attempt to solve is portability/interop (and moving control of and responsibility for blocking/moderation/spam to users rather than sites). I don't see how it helps at all with privacy or you "controlling" who gets your data. If you give it to site A but not data collector B, what's preventing A from selling it to B? As far as I can tell, the situation will remain identical to how it is today. Your data will never be in one place unless you never share it. The moment you use it with other sites or services, it is stored there too, out of your control.

erlend_sh 1y ago
> The moment you create/share data with a site, what's to prevent them from reselling it? If I can clearly assert origin and personal ownership of my data, I can forbid further reselling of it. EU legislation shows that we can actually have the right to demand that a company forgets about us. Asserting such rights become easier the more accurately we define what data is ours.
- crazygringo 1y ago
  > If I can clearly assert origin and personal ownership of my data, I can forbid further reselling of it. Can you? A site's TOS will say that by sharing your data, you grant them the right to display, reuse and redistribute it, the same as you do now. And that would take precedence because your host provided the data. They requested and you provided. The only thing that would change that is actual legislation. But then the legislation is orthogonal to personal data storage. If you want legislation for that, pursue legislation for that. Personal data storage is completely separate, and the two shouldn't be confused with each other.
  - layer8 1y ago
    The right granted by the TOS elapses when you cancel the respective service, or when you revoke your consent (in which case the service provider may possibly cancel the service). (Some TOS are also simply illegal to begin with.) That’s what the GP is referring to.

rob_c 1y ago

Aka, more dunking on "the cloud". Now it's cool to be able to do so. How about we go back 20yr and train a generation of unix sysadmins and self host at companies and at home.

mactavish88 1y ago

For those of us who've been around for some time and still value privacy, this sort of paradigm is obvious. The trouble isn't a lack of the right technologies - I'd argue it's a problem in the go-to-market strategy of those building these products/technologies. Ideas flow along lines carved out by power/influence. Facebook's early strategy was to start with restricting its usage to people at Harvard University - arguably a highly influential institution - and then expand outwards to other highly influential institutions. Only once the "who's who" from those institutions were already onboard did they let down the walls to allow us plebs in, and we all rushed in head-first. X's current strategy leverages Musk's visibility and influence (for better or worse). Get the most prominent influencers onboard with your decentralized social network, and others will follow (dramatically easier said than done, of course). But without a significant contingent of influencers/powerful people, your network's DoA.

btbuildem 1y ago
> prominent influencers onboard with your decentralized social network That's sort of a contradiction, no? Or at least it assumes transplanting the same mechanisms into a new milieu -- which I argue is something to leave behind, because it's those very mechanisms that have ruined the current internet. I think instead of tapping into the same addictive attention economy schemes, the distributed / decentralized socials could onboard people en-masse by providing what's missing there, and filling a real need.
- mactavish88 1y ago
  Even if they fill a real need, their go-to-market strategy will determine whether the masses even know about them, or give a damn about trying them out in the first place.

InMice 1y ago

Among the first page and 2nd page (top 60) there is always atleast 1 post about how we're gonnna "take back the web" or make it back into some form of our 90s millenial nostalgia memories, self hosting, federated this or that, etc etc. Meanwhile - Nothing changes, everything generally gets worse and younger generations come into the world with no memories of the 90s internet or the world before mobile devices or surveillence everywhere. Applying for a job or apartment or anything today means creating endless pointless copies of your pesonal information in databases across the world that will eventually be neglected, hacked, exploited, sold off etc I dont know the way out if there is one, I guess we can keep fantasizing and thinking about it. It just feels like it would be easier to get the earth to start spinning the other way sometimes.

erlend_sh 1y ago
This is demonstrably not fantasy as the example case is a fully productionized network (Bluesky and the rest of AT-net) that’s having real-world impact to the point where it’s under threat from several authoritarian states.
- ffsm8 1y ago
  It has? Don't get me wrong, I'm in the tech industry and generally more online then likely 95% of the population, but ime ... Nobody even knows what bluesky is? (They also don't know what X is, though they DO know what Twitter is) And even more niche products like mostodon, the fediverse altogether etc are entirely unknown to most of the tech industry too.
  - tomrod 1y ago
    Sometimes tech leads the world, however unwillingly, to better outcomes.
    - oceanplexian 1y ago
      Tech is downstream of culture. Seems that smart people keep getting duped by this idea. For example Twitter and Facebook didn’t result in a bunch of Democracies springing up after the Arab Spring, it resulted in the complete opposite. Tech simply amplifies the culture that was already there.

gibsonf1 1y ago

Systems Twin Intelligence, where a Pod represents the full space-time information for part of the world, using Solid Protocol: https://graphmetrix.com/trinpod-server https://graphmetrix.com/trinpod-server The W3C Linked Web Storage (LWS) working group is transforming Solid into a web standard: https://www.w3.org/groups/wg/lws/ https://www.w3.org/groups/wg/lws/

zeroCalories 1y ago

I find the ideas of data coops to be very appealing. I don't want to depend on faceless mega-corps like Google to host stuff like my email, but I also don't find the idea self-hosting to be realistic. I wouldn't mind paying for the security since losing access to certain accounts would be a disaster, but I'm already locked in, and the benefits of existing services would be marginal compared to the cost of moving.

anonbuddy 1y ago
ideally you should be able in a simple way to host your stuff, in this case in a POD. That service should be provided by a utility company, same way we have internet providers now. They will be well regulated and it would be in their interest to safely hold your data because if not, they would face legal and financial consequences. All other services would read/write from your Pod.

gcanyon 1y ago

Both of these proposals (as far as I've read them, YMMV) fail the evolutionary test. At the scale we're talking about, ideas must proceed as evolution does: not with a far-away goal in mind, but with incremental changes, each of which individually must be an improvement over the status quo. We are at (near) a significant local maximum, and (again, as far as I've read, which is not all of it for sure) the people pitching this form of information control have given no set of steps from here to there without significant cost/effort. Of course they don't have to have the whole path in mind. By definition they just need the first step or two. But they must be steps up. You don't get wings by wanting to fly; first you need feathers to keep warm (I am not an evolutionary biologist, I don't know if that's a valid theory).

jauntywundrkind 1y ago
99.9% of BlueSky users use only Bluesky services. But BlueSky has a Personal Data Service for each. That means: Those users have credible exit to take their data off BlueSky's hosting to someplace else (and as of a week or two ago to move back to BlueSky if they want). Those users can put whatever kind of data they want in their PDS. They can host their git data via https://tangled.org https://tangled.org . They can store their music listening scrobbles with https://teal.fm https://teal.fm . They can blog on https://leaflet.pub https://leaflet.pub . And there's been rapidly advancing host it yourself options. Plenty of folk individually or collectively host PDS. There are alternate relays that collect &n syndicate out everyone's PDS data as that changes. Hosting the aggregation layer is significantly harder especially if you are trying to fully connect the network but there are a couple & progress is good. it feels like a huge improvement over the status quo, and there's extremely visible developer energy building forward & rolling with the concepts. The breakdown on architecture allows for wins and work in various areas. The base seems solid, the core seems coherent & well built, built to scale not as one big thing but coherent layers. I think it's doing what you are asking for, and the signs of advancement & uptake warm my heart to see.
- senordevnyc 1y ago
  99.9% of BlueSky users use only Bluesky services. I highly, highly doubt this, even in the narrowest sense of how many BlueSky users still actively post on X.

vuldin 1y ago

IPFS and Filecoin exist to solve this problem. https://ipfs.tech https://ipfs.tech https://filecoin.io https://filecoin.io

robinkunz 1y ago
thought the same.
attila-lendvai 1y ago
and https://www.ethswarm.org/ https://www.ethswarm.org/

lerp-io 1y ago

you store ur photos on fb same way you store your money at the bank and your code on github, its delegation of concerns, you can make same argument for literally anything....not using your own silicon, growing your own food, financing your own venture, owning your own land, etc etc.... maybe its more "secure" vs "less efficient" or some other tradeoff. and you have to get the right balance or take risks for optimal efficiency / profit/whatever your values are

dd_xplore 1y ago

When I was a kid, a 4GB pendrive was a huge thing for me. I used to think my 40GB HDD would never fill up, but then Internet started to grow. Today it doesn’t even matter how muc storage you have it’ll always fill up. I have started to self host quite a lot of stuff but eve then every storage solution has a life of 5-6 years in which atleast one of the components would fail. We click enormous amounts of photos but they do not have any impact like printed photo albums. With ever growing storage costs (both cloud based and self hosted) I’m thinking of going back to keep only important stuff that too in print format.

ivanjermakov 1y ago
In the age of abundance, smart prioritization is needed.
Jaxan 1y ago
We still print photo albums. I can strongly recommend this!
- theshrike79 1y ago
  I bought a Canon SELPHY photo printer on a Black Friday sale last year. It prints archive quality photos we can put in an album to save forever. It's kind of fun to go through the thousands of photos in our digital photo libraries and pick the best and most impactful ones to print and save "forever".
AdrianB1 1y ago
I run a NAS, in various forms, for almost 20 years. The lifetime is quite longer, I still have ~ 10 year old drives in the backup NAS built on a Ryzen 1600 (8 years) and the average power supply works for me 10-12 years. The primary NAS is still on hardware that is more than 5 years old, except the drives that I just replaced with higher capacity. As I find the size of current drives bigger than my yearly additions (personal pictures and movies), I am quite happy with a 10 year lifetime at low usage. I would love some reliable and affordable long term offline storage, but backup tapes and a reader are not affordable and not in common use for end users. Otherwise I would build a tiered storage system with more reliability and even performance (nvme hot tier? maybe).

dangus 1y ago

This article seems pretty far detached from the problems that people experience using technology. It’s the kind of thing that only deeply technical people consider. When someone uses a service like Dropbox or iCloud Drive or Google Drive, they really aren’t experiencing any kind of problem where their data “isn’t theirs” or is “trapped.” It’s not that hard to migrate to something else and the services themselves are reasonably low-friction. In terms of social data, users don’t really have a major issue with the status quo, and those who do have already developed relatively popular solutions like Mastodon and BlueSky. Even “proprietary” photos applications like Apple Photos and Google Photos have very easy migration paths to other services. So what exactly is the problem we’re trying to solve here? Giving me an @Bob handle? Did I want that or need that?

crazygringo 1y ago
> In terms of social data, users don’t really have a major issue with the status quo That's exactly it. And with social media (unlike files and photo storage) migration isn't really something people care about, because it's about the present not the past. If you move from Twitter to Bluesky, does anyone care about moving their tweet history? They just want their list of followers to migrate over as much as possible, which happens relatively organically anyways.

skybrian 1y ago

Bluesky’s PDS is currently fairly limited due to the lack of support for private data and inadequate permissions [1]. Hopefully they’ll fix that soon. [1] https://bsky.app/profile/byarielm.fyi/post/3lz4vzzhybk2b https://bsky.app/profile/byarielm.fyi/post/3lz4vzzhybk2b

xenodium 1y ago

> Meanwhile - Nothing changes, everything generally gets worse https://LMNO.lol https://LMNO.lol is my grain of sand. I wasn't happy the state of blogging (tracking, bloat, ads, paywalls...), so I built https://LMNO.lol https://LMNO.lol. It's offline first and you can browse blogs from anywhere (even terminal). Your blog is a single Markdown file. Drag and drop it to the browser and your entire blog is generated. Custom domains are welcome. My blog is running off LMNO.lol that https://xenodium.com https://xenodium.com

lukeschlather 1y ago

I love the idea of personal data storage and I want it to be the default, but I think there are some possibly insurmountable technical problems. This article doesn't mention schema once, and schemas make seamless data portability virtually impossible. I've spent a week making sure a simple CRUD app could change a string field to a UUID field without causing any outage or bugs. You can export your data from Google or Facebook today, but then you need to write a copy of the source UI that faithfully replicates the way all those data fields are supposed to display. And tomorrow the source makes a change so what used to be one field is now two fields, oh and they also removed another field entirely so that data is just gone. Well, in future dumps anyway. Are you going to use the old schema or the new schema for your display? Is it possible to do both? When everything is in data silos, you can freely and safely change data format, which is something that needs to happen a lot as applications evolve. Even in a data silo, doing this is pretty tricky and bugs and data loss are significant risks. If you're trying to sync between an unbounded number of data repositories where each repository has potentially conflicting relationships with the data schema, data loss is practically assured. Another big problem is schema permissions and identity. I might have some piece of data that says "person A is allowed to see this set of fields" and another piece that says "person A is blocked from seeing this other set of fields." This gets synced to 3 different servers, one of those servers has no idea that userA is in fact person A. So you fail closed, but then the data on that server practically does not exist if the goal of this data repository is sharing some data with person A. You really can't do any sort of fine-grained access controls in a system where trust/identity/auditing is decentralized.

back2dafucha 1y ago

[dead]

impure-aqua 1y ago

I don't see what advantage any company gets from choosing to build products that enable personal data ownership. I say this as someone working on a venture with these sorts of design aims, it feels like pushing a boulder uphill often. The business model of cloud service providers makes a lot of sense- we have a system which stores and operates on your data, you pay some rental fee for us to store it and operate on it, easy peasy. The cost is related to both the utility of the operations the operator performs (to both the operator and the user) and the amount of data the user stores. Fundamentally this is how everything from Dropbox to Facebook is governed- Dropbox does not devise much utility per GB and users store a lot, so you rent per GB, but at Facebook, they don't store lots of your stuff, and on the data side maybe you don't get much value from it as it's a cesspit, but the data is valuable to Facebook to sell ads, etc, so they can provide the service for free. Importantly, you don't need to improve the product to continue extracting this rent, because the product you are selling is not Dropbox v4, Facebook v2.3, rather you are selling ongoing access to the rental. As soon as you introduce even simply a federated system where a few corporate operators are involved, it becomes very hard to justify extracting rent there as the network designer, as the operators are taking on the cost of actually storing the data. You have to really be iterating on the core product to use a SaaS business model here. Some things simply don't need a v4, does Dropbox really need that much iteration? Meanwhile as the system designer, life has become a lot more complex for you. Suddenly you cannot push unilateral sweeping changes to APIs, you need to version things in a way that is compatible between, say, one university updating their system but not the other. Since your users are a few large operators rather than millions of individuals, you lose the network effect advantage of being able to screw over a few users for the "greater good", since if you irritate one corporate client, you lose a lot of your install base. Why would you voluntarily choose this harder path as a company? Things get even worse as you increase the level of decentralization. The reality is users expect the polished experience that the rental companies can give you; they want their data always accessible so that their friend can see the pic they shared without needing to keep their own computers running, they want the "like counter" to go up without their personal node subscribing to messages from other nodes, etc. The only users that will accept a worse experience are people who have are motivated by their philosophy re: personal data ownership, and this crowd will want a FOSS solution, so you can say goodbye to charging them for Dropbox v4, they are simply not interested if you're not giving them the source code for free. (I suspect this is where the author sits, but fundamentally I don't think it will get mass appeal, most people simply do not care about data ownership above something that "just works".) So now you are dealing with problems like dynamic generation of redundant data and fault- and Byzantine-tolerant consensus algorithms so that your system can maintain function even when the user turns their computer off, and you have to deal with wrapped-key cryptography so that the redundant data can be split across all these user nodes without you worrying that an unauthorized user can read it, and then you have issues like how do you deal with nodes that are too slow to process updates (perhaps some user data needs to be stored in this conflict-free replicated datatype you devise), and eventually you go through all of this to... create a system that is less monetizable than the rental model, because you can't extract that rent for ongoing data storage, and we know users are not interested in actually paying for software.

ksec 1y ago

In terms of NAS, I have long wonder if there is a market for a combination of both online and offline. We will need at least 2 HDD for redundancy and to prevent bit riot. And the NAS will be sold as a whole package and subscription, with an encrypted backup services included for first 2 years and requires the backup subscription to work there after. The profit margin is first on the hardware and then on long tail backup which is charged like iCloud and Google storage per tier. Where your 1.5TB storage will be charged at 2TB storage. Before 2014 I would have thought Apple to potentially take this route for Time Capsule. Instead they doubled down on iCloud. Google will never take this route. Microsoft is not interested. Amazon should have done this and bundled with cold storage back up but their track record are not good enough. I doubt people trust Meta enough even if the solution was perfect. In pre 2012 you could at least bet on Apple to be somewhat customer centric. May be UniFi will do it. They just announced their 2 Bay UNAS and I only just discovered, they are a 40B market cap company. ( I thought they were much smaller )

phkahler 1y ago
>> And the NAS will be sold as a whole package and subscription... Misses the point entirely.
- ksec 1y ago
  Data will need Backup to be safe. You could tell everyday customer to get NAS and they wouldn't know what is Bit Riot until they saw their Image and Video with errors or broken. They also wouldn't do off site backup. Company wants long subscription model. Right now everyone is only talking about options that are extreme in both ends.
detaro 1y ago
Synology sells cloud backup services for their NASes. And a bunch of other brands at least can easily connect to other services.
Larrikin 1y ago
>with an encrypted backup services included for first 2 years and requires the backup subscription to work there after. Its confusing if you mean the NAS will stop working if you stop paying for the subscription or not. If you can no longer access your data on the NAS without a subscription, then the NAS just becomes the cloud with an extra up front cost plus the cost of your own electricity. Personally I have started moving as much of my data out of the cloud as possible. I've got a Synology and a few single board computers running various services with a Synology in my parent's home for their photos. Their photos back up to my NAS and my data to their Synology. Its a shame Synology decided to enshitify this year for all products going forward, but UGreen looks like a suitable replacement when I outgrow my current NAS.

jauntywundrkind 1y ago

> Another spiritually similar idea being championed at the time came from the Opera browser folks who wanted to put "a web server in your browser". Opera Unite was such an awesome idea. https://arstechnica.com/information-technology/2009/06/opera-hoping-to-reinvent-the-web-by-making-browser-a-server/ https://arstechnica.com/information-technology/2009/06/opera... There was a neat idea a bit back to allow Service Workers to work across origin: foreign fetch. It wasn't on the internet, was only in the scope of your browser, but I thought it was such a neat advancement. Would have done so much to allow the offline web to weave itself. Alas, deprecated. https://developer.chrome.com/blog/foreign-fetch https://developer.chrome.com/blog/foreign-fetch

brendoncarroll 1y ago

I work on a FOSS project in this space, Blobcache. https://github.com/blobcache/blobcache https://github.com/blobcache/blobcache Trusting a server to store an application's state is a different thing from trusting it to author changes or to read the data. Servers should become dumber, and clients should become smarter. When I use an app, I want the app to load E2E encrypted state from storage (possibly on another machine, possibly not owned by me) make whatever changes and produce new encrypted data to send back to the server. The server should just be trusted for durability, and to prevent unauthorized access, but not to tell the truth about doing either of those things. Blobcache provides an API to facilitate transactions on E2EE state between a dumb storage server and any smart client. Blobcache can be installed on old hardware along with a VPN like Tailscale and then loaded up with data from other devices. Configuration is like SSH, drop a key in a configuration file to grant access. It removes most of the friction associated with consuming and producing storage as a resource. I'm using it to build E2EE version control like Git, but for your whole home directory. https://github.com/gotvc/got https://github.com/gotvc/got

ianopolous 1y ago
We should talk. This very similar to how apps use E2EE data in Peergos. Maybe we can join forces. https://peergos.org/posts/a-better-web https://peergos.org/posts/a-better-web
- brendoncarroll 1y ago
  I couldn't find an email in your bio. You can reach me via the email at the bottom of my website (in my HN bio). Looking through the docs on Peergos, it looks like it's built on top of IPFS. I've been meaning to write some documentation for Blobcache comparing it to IPFS. I can give a quick gist here. Blobcache Volumes are similar to an IPNS name, and the set of IPFS blocks that can be transitively reached from it. A significant difference is that Blobcache Volumes expose a transaction API with serializable isolation semantics. IPFS provides distributed, available-but-inconsistent, cryptographically signed cells. IPFS chooses availability, and Blobcache chooses consistency. A Blobcache Volume corresponds to a specific entity maintained and controlled by a specific Node. An IPFS name exists as a distributed entity on the network. Most applications need some sort of consistent transactional cell (even if they don't realize it), but in order to be useful, inconsistent-but-available cells have to be used carefully in an application specific way. I blame this required application-specific care for the lack of adoption of CRDTs. There's a long tail of other differences too. IPFS was pretty badly behaved the last time I used it, trying to configure my router, and creating lots of connections to other nodes. Blobcache is more like a web browser; it creates transient connections in immediate response to user actions. That whole ecosystem is filled with complicated abstractions. Just as an example, the Multihash format is pervasive. It amounts to a tag for the algorithm used to create a hash, and then the hash output. I'd rather not have that indirection. All the hashes in Blobcache are 256 bits, and you set the algorithm per Volume. In Go that means the hashes can just be `[32]byte` instead of a slice and a tag and a table of algorithms. I haven't used IPFS in a while, but I became pretty familiar with it awhile ago. Had I been able to build any of the stuff I was interested in on top of it, I probably wouldn't have written Blobcache.

deleted 1y ago

[deleted]

didip 1y ago

As in self hosting? I love self hosting idea for myself out of principles. But unforunately it will never take off in a huge way because convenience is king. Average Joe and Jane want to install things with as little efforts as possible.

AdrianB1 1y ago
You can self host, but in order to be reachable you need to be discoverable. If the discovery is based on a mechanism that is controlled by someone else that can become an evil party, self-hosting in isolation is not too useful.

browningstreet 1y ago

Ideas like the Solid protocol have a limited timeframe to make it or go away. Not sure why anyone is still talking about it. TBL is rightfully a legend but this is now just a windmill. Next, please.

righthand 1y ago
This comment has inspired me to target SOLID and “things I can do to help” on my Sunday afternoon research block. This type of commentary is rife in this article thread and is now just a windmill. Next, please.
- browningstreet 1y ago
  If Schneier can’t get more than 13 comments on a solid protocol crypto wallet, I personally don’t think that anyone will ever care about a solid protocol app of any kind. And I’m all for it, just calling it as I personally see it. Some things are fire, some things are warm, and some things are DOA. And I’m typing this on my Linux desktop (f’real). https://www.schneier.com/blog/archives/2024/07/data-wallets-using-the-solid-protocol.html https://www.schneier.com/blog/archives/2024/07/data-wallets-...
  - righthand 1y ago
    A Solid protocol cryptowallet. Arcane on top of arcane. I think it’s entirely unfair to dismiss technology because it hasn’t demanded immediate adoption by society. Solid is attempting to help define a better data future. We have working mechanisms in place but everyone is at a disadvantage except the people loyal to these giant corps. Attempting to give people the power to organize their data as they wish and to be used as they wish is worth it. Even if it doesn’t bring a renaissance.
    - browningstreet 1y ago
      Crypto wallets are not nearly as arcane as Solid. How many people have Binance accounts? Market share matters, critical mass matters, adoption matters. I'm suggesting that mindshare goes negative over time if these things aren't achieved, and when you have long-tail blog posts trying to pump life into it, it's pivot time. Righteousness alone doesn't win any of those things. It's been a very long time since Solid was released and it's like a whisper in the wind.

system7rocks 1y ago

I love this idea, and I imagine with years of successful lobbying efforts we could potentially get some laws passed to provide rights and clarity around our own data that could move us into this direction. But until then, while BlueSky is solid, I'll wait and see.

righthand 1y ago

> Whether these providers are strictly cooperatives in the formal sense isn't what's most important here though; I think the context of “encouraging people to switch” to a pds/solid/data coop, how they operate IS important. For two reasons: - data coop and controlling data opens the door to a new market if we’re going to join data coops, then we may as well try to share the profits from said coop fairly. Otherwise Facebook can step in as a “data-coop” and keep-on-keeping-on - a secondary effect is that now there is an incentive to move off facebook. If I can join my local Nowheresville.USA.town data coop and benefit directly to my community by storing data together then I am encouraged to switch to this new paradigm That is the major undiscussed shift to me. I believe the only way out of the Big Tech dystopia is to incentivize the switch. Even if the reward is pennies. Invest in the community oil well.

purpleKiwi 1y ago

How do I, as a complete noob, use the powers of atproto and the fact I own a domain?

dzonga 1y ago

I like the convenience of the cloud. but don't know whether its due to declining literacy rates / awareness etc. the cloud is nice and e.g google storage, iCloud but now with fast microsd's you can buy 1TB for $100. have a few copies then boom, you own your own data. but now phones don't allow you to have microsd's so here we are. likewise things like email etc instead of all of us being on gmail we could have community email servers etc.

Larrikin 1y ago
Sony phones continue to have MicroSD slots, headphone jacks, AND remain water resistant. They have been that way for at least a decade.
layer8 1y ago
I use Dropbox, but with an encryption overlay that also integrates into the iOS Files app for ease of use on mobile. So it’s possible to use cloud storage and still keep your data private.

AlienRobot 1y ago

When I read the title I couldn't help but think "did everyone forgot about hard disks?" I'm sure Tim Berners-Lee is much smarter than me, but I kind of feel there are some parallels between the idea of "owning" posts you made in a platform and the ludicrous idea of "owning" game items as NFTs in a blockchain. The latter promises interoperability that games would never deliver. I wonder about the former. At least I feel the major dealbreaker with this technology is just that it's not worth it for both parties involved. Right now, Facebook hosts all the posts and monetizes them with ads. So long as they are making money with ads, they have no reason to delete the posts they're hosting, as the posts are their money maker. But what happens if Facebook no longer "owns" the posts? So now your posts are in your "personal cloud", which means that unless they are encrypted any website or local app can display them, even without any ads. This means Facebook is no longer making money off the posts. Why would they accept this? On the flip side, who is paying for the hosting? Facebook? It's no longer their servers hosting the content, so I don't think so? Is Facebook supposed to pay the cloud service for metered API access? Can a cloud service offer different rates to different companies? Is the user supposed to pay for their cloud storage? So you're going to make users pay money to use facebook? What happens if a post violates the ToS? Can facebook delete my post in my cloud storage against my will? What happens if content that is legal where facebook operates is illegal where the cloud servers operate? Can I manually edit the data in my cloud storage like I'd be able with a file and then facebook has to treat every post as if it were untrusted input? What happens if my cloud storage closes my account? I just lose everything? Will I be able to back up my cloud to my hard disk and reupload it to another cloud so facebook can access it? How is facebook going to handle a single user with 2 clouds that have different content? I feel like this is a very complex thing and there are infinite questions that we can have about how this would be implemented in practice, while it's presented as simply "you own your data."

bawolff 1y ago

This is never going to happen. The incentives do not make sense. Any utopian future that requires a party to put in a lot of effort to change something in a way that would be a net negative for them, is just not going to happen. People do not spend money to change the world in a way that would be worse for them but better for other people.

JumpCrisscross 1y ago
> The incentives do not make sense Commercial incentives, no. If this preference exists, it would need to be pursued civically.
- bawolff 1y ago
  I don't think the average citizen cares enough or even understands the benefits But lets say you get them on board and pass some law. Unless its a huge market like the EU or USA, probably what immediately happens is everyone pulls out of that market. Not out of malice but because they suddenly have to rewrite their app and that's probably quite expensive.

herf 1y ago

Vertically integrated apps are much cheaper to run - Instagram stores only a small fraction of your photos and makes a lot of money from them. It is somewhat harder to explain why we pay for things like iCloud, which mostly has no web API, only APIs for Apple devices. (Plenty of value there because it keeps you from having to buy a bigger iPhone.) But there are lots of these "almost general purpose" solutions, paying to upload files and store them, but where you cannot use them as you like. Why not dozens of apps running over the "web filesystem" like happens on the desktop? Two reasons: 1. Amazon pricing for transit/bandwidth is way higher than storage, and so it makes accessing your own data quite expensive if it is not in the same datacenter. 2. And there is a huge security and usability gap between "pick one photo" vs "give me [scoped] access to your Dropbox" Often the general-purpose mode does not work that well, is quite slow, or just costs a lot in bandwidth, a thing nobody wants to pay extra for when they're already paying for storage.

nayuki 1y ago

> Data Ownership as a conversation changes when data resides primarily with people-governed institutions rather than corporations. This is a false contrast. Corporations are institutions governed by people - specifically a board of directors, elected by shareholders. They aren't governed by aliens nor are they self-sentient. https://en.wikipedia.org/wiki/Institution#Examples https://en.wikipedia.org/wiki/Institution#Examples , https://en.wikipedia.org/wiki/Institution#Examples https://en.wikipedia.org/wiki/Institution#Examples Perhaps you meant that you are against for-profit corporations where the customer (who stores data) has no vote in the operation of the corporation? If so, then say that and don't imply it. People often use "corporation" as a pejorative, often in contrast to individual people. But they forget that a corporation is composed of people and ultimately owned by (some) people - but the kind of people that the writer does not like (shareholders, profit-makers, etc.). > Notice that Alice’s handle is now @alice.com. It's funny you're using .com as the example, because: > The domain com is a top-level domain (TLD) in the Domain Name System (DNS) of the Internet. Created in the first group of Internet domains in March of 1985, its name is derived from the word commercial, indicating its original intended purpose for subdomains registered by commercial organizations. Later, the domain opened for general purposes. -- https://en.wikipedia.org/wiki/.com https://en.wikipedia.org/wiki/.com Even when you're arguing against commercial organizations for storing personal data. Now you're just naming individual people as if they were companies.

HenriTEL 1y ago
To be fair nowadays .com refer much more to the default, main or official domain of an entity. Say you know the name of a non corporate website, are going to try .com first of something else?
- XorNot 1y ago
  Yeah it strikes me that basically .com will eventually get canonically termed to mean "common" since that's how it's actually used.

est 1y ago

PDS is a cool idea, I hope the community addresses problems like content farm, spam and original attribution as a higher priority. Or I see malicious actors would wreck the federation mechanism. This is already the case with Email SMTPs

BrenBarn 1y ago

There are good ideas here. They won't come to fruition without some form of force. I'm not sure if TBL doesn't realize, is unwilling to accept, or just wants to avoid saying out loud that the only reason it worked for him to create the web as an open protocol is that no one was prepared for it so no one was in a position to co-opt it, commercialize it, and enshittify it. Now corporations are prepared. They will co-opt, commercialize, and enshittify whatever system you come up with unless it is accompanied by a giant hammer that will brutally destroy them if they don't change their wicked ways.

gatestone 1y ago

No one mentioned Upspin? A global file namespace (URL, but better...) and protocol to isolate public data users from private governance and storage, by gurus like Rob Pike. https://github.com/upspin/upspin https://github.com/upspin/upspin

Lumoscore 1y ago

It’s completely true that the system we use today—where a few big companies hold all of our private information in one place—is a bad model. It’s risky for security, and it means you have no real power or ownership over your own data. The good news is that we don’t have to wonder if a better way is possible. The technology is already here! Projects like Solid (Pods) and AT Protocol (PDS) have proven we can separate your information from the applications you use. You can put your data into your own secure digital "locker" or vault. The difficulty now is not the technology, but getting people to actually use it: 1- It’s Too Hard to Use: Setting up and managing your personal data locker is currently as complicated as managing a super-secret password for a crypto account. For everyone to adopt it, it needs to be way simpler than just clicking "Log in with Google." If it’s too much work for regular people, it will fail. 2- Big Companies Don't Want to Change (The Incentive Problem): The biggest tech companies make billions by collecting and using your data. They have no reason to switch to a system where they have to ask permission to use data they don't own, unless a major law forces them to, or a new competitor steals their users. 3- Privacy Isn't Enough (The Benefit Problem): Most people won't switch just for "privacy." The new system must offer clear, positive benefits, like letting you move all your friends to a new social app instantly, or securely filling out long forms with a single click from your data locker. The key to success is building user-friendly tools that hide all the complexity and make this new, secure way of managing data simple for everyone.

selinkocalar 1y ago

The concept makes sense but the execution is always where these things fall apart. Most people don't want to manage their own data infrastructure. The bigger issue is interoperability. Your personal data store is only useful if apps actually integrate with it, and getting developers to adopt new standards is tough.