7 ms·
Open Source Implementation of Apple's Private Compute Cloud
- kiwicopple 11mo agoimpressive work jmo - thanks for open sourcing this (and OSI-compliant) we are working on a challenge which is somewhat like a homomorphic encryption problem - I'm wondering if OpenPCC could help in some way? : When developing websites/apps, developers generally use logs to debug production issues. However with wearables, logs can be privacy issue: imagine some AR glasses logging visual data (like someone's face). Would OpenPCC help to extract/clean/anonymize this sort of data for developers to help with their debugging?
- jmort 11mo agoYep, you could run an anonymization workload inside the OpenPCC compute node. We target inference as the "workload" but it's really just attested HTTP server where you can't see inside. So, in this case your client (the wearable) would send its data first through OpenPCC to a server that runs some anonymization process. If it's possible to anonymize on the wearable, that would be simpler. The challenge is what does the anonymizer "do" to be perfect? As an aside, IMO homomorphic encryption (still) isn't ready...
- pjmlp 11mo ago[flagged]
- mlnj 11mo agoIt is an implementation. As long has it behaves the same...
- pjmlp 11mo ago[flagged]
- parting0163 11mo agothe point here wasn't to be a complete clone of Apple's PCC.
- deleted 11mo ago[deleted]
- pjmlp 11mo agoTitle says otherwise.
- Robin_Message 11mo agoIt's not a drop-in replacement; rather it is an implementation of the same ideas (+ some extra ones) but open source so it can be used for things other than Apple devices.
- pjmlp 11mo agoWhich isn't the same as the title suggests.
- rrdharan 11mo agoI think the parent has a valid point. The actual README says "inspired by Apple’s Private Cloud Compute". I think it's more fair to say it implements the same idea but it is not an opensource implementation of Apple's Private Compute Cloud the way e.g. minio is an implementation of S3, so the HN title is misleading.
- kreetx 11mo agoI read this and your reply to the sibling, you seem to have reputation to be sensible - what are you trying to say? If someone re-implements or reverses a service then it doesn't need to be in the same language.
- almostgotcaught 11mo agoThis dude stays commenting on things he doesn't actually understand anything about. I have run into him multiple times in threads on what I do (compilers) and he's clueless but insistent.
- pjmlp 11mo agoThakfully that the Internet is full of analists with such deep understanding of others personality, otherwise we would all be lost. As for the matter at hand, I assume that such implementation is validated against which specification exactly? Gladly to be educated in my clueless.
- almostgotcaught 11mo agoI guess English isn't your first language so nbd but I have no idea what you're asking.
- pjmlp 11mo agoTo be educated on the public specification of Apple's Private Compute Cloud so that I become less clueless, according to you.
- almostgotcaught 11mo agoThis question makes zero sense - PCC is a (proprietary) system not an interface. There is no spec just like there's no spec for how you have the furniture arranged in your own house.
- ryanMVP 11mo agoReading the whitepaper, the inference provider still has the ability to access the prompt and response plaintext. This scheme does seem to guarantee that plaintext cannot be read for all other parties (e.g. the API router), and that the client's identity is hidden and cannot be associated with their request. Perhaps the precise privacy guarantees and allowances should be summarized in the readme. With that in mind, does this scheme offer any advantage over the much simpler setup of a user sending an inference request: - directly to an inference provider (no API router middleman) - that accepts anonymous crypto payments (I believe such things exist) - using a VPN to mask their IP?
- Terretta 11mo ago> the inference provider still has the ability to access the prompt and response plaintext Folks may underestimate the difficulty of providing compute that the provider “cannot”* access to reveal even at gunpoint. BYOK does cover most of it, but oh look, you brought me and my code your key, thanks… Apple's approach, and certain other systems such as AWS's Nitro Enclaves, aim at this last step of the problem: - https://security.apple.com/documentation/private-cloud-compute https://security.apple.com/documentation/private-cloud-compu... - https://aws.amazon.com/confidential-computing/ https://aws.amazon.com/confidential-computing/ NCC Group verified AWS's approach and found: 1. There is no mechanism for a cloud service provider employee to log in to the underlying host. 2. No administrative API can access customer content on the underlying host. 3. There is no mechanism for a cloud service provider employee to access customer content stored on instance storage and encrypted EBS volumes. 4. There is no mechanism for a cloud service provider employee to access encrypted data transmitted over the network. 5. Access to administrative APIs always requires authentication and authorization. 6. Access to administrative APIs is always logged. 7. Hosts can only run tested and signed software that is deployed by an authenticated and authorized deployment service. No cloud service provider employee can deploy code directly onto hosts. - https://aws.amazon.com/blogs/compute/aws-nitro-system-gets-independent-affirmation-of-its-confidential-compute-capabilities/ https://aws.amazon.com/blogs/compute/aws-nitro-system-gets-i... Points 1 and 2 are more unusual than 3 - 7. Folks who enjoy taking things apart to understand them can hack at Apple's here: https://security.apple.com/blog/pcc-security-research/ https://security.apple.com/blog/pcc-security-research/ * Except by, say, withdrawing the system (see Apple in UK) so users have to use something less secure, observably changing the system, or other transparency trippers.
- derpsteb 11mo agoI was part of a team that does the same thing. Arguably as a paid service, but source availability and meaningful attestation. Service: https://www.privatemode.ai/ https://www.privatemode.ai/ Code: https://github.com/edgelesssys/privatemode-public https://github.com/edgelesssys/privatemode-public
- m1ghtym0 11mo agoExactly, attestation is what matters. Excluding the inference provider from the prompt is the USP here. Privatemode can do that via an attestation chain (source code -> reproducible build -> TEE attestation report) + code/stack that ensures isolation (Kata/CoCo, runtime policy).
- jmort 11mo agoOpenPCC is Apache 2.0 without a CLA to prevent rugpulls whereas edgeless is BSL
- jiveturkey 11mo ago<3
- jabedude 11mo agoWhere is the compute node source code?
- saurik 11mo agoYes: "provably" private... unless you have $1000 for a logic analyzer and a steady hand to solder together a fake DDR module. https://news.ycombinator.com/item?id=45746753 https://news.ycombinator.com/item?id=45746753
- Lord-Jobo 11mo agowell, also indefinite time and physical access.
- saurik 11mo agoWhich is what the provider themselves have, by definition. The people who run these services are literally sitting next to the box day in and day out... this isn't "provably" anything. You can trust them not to take advantage of the fact that they own the hardware, and you can even claim it makes it ever so slightly harder for them to do so, but this isn't something where the word "provably" is anything other than a lie.
- anon721656321 11mo agoyeah, for a moment I was reading it as being a holomorphic encryption type setup, which I think is the only case where you can say 'provably private'. It's better than nothing, I guess... But if you placed the server at the NSA, and said "there is something on here that you really want, it's currently powered on and connected to the network, and the user is accessing it via ssh", it seems relatively straightforward for them to intercept and access.
- anon5739483 11mo ago[dead]
- sublimefire 11mo agoIf you trust the provider then it does not make it much better to use such architecture. If you do not then at least the execution should be inside a confidential system so that even soldering would not get you to data
- okelahbos28 11mo ago[flagged]
- utopiah 11mo agoThat's nice... in theory. Like it could be cool, and useful... but like what would I actually run on it if I'm not a spammer? Edit : reminds me of federated learning and FlowerLLM (training only AFAIR, not inference), like... yes, nice, I ALWAYS applaud any way to disentangle from proprieaty software and wall gardens... but like what for? What actual usage?
- utopiah 11mo agoGimme an actual example instead of downvoting, help me learn. Edit on that too : makes me think of OpenAI Whisper as a service via /e/OS and supposedly anonymous proxying (by mixing), namely running STT remotely. That would be an actual potential usage... but IMHO that's low end enough to be run locally. So I'm still looking for an application here.
- wat10000 11mo agoAre you looking for a general application of LLMs too large to run locally? Because anything you might use remote inference for, you might want to use privately.
- utopiah 11mo agoSure that'd do, what - useful thing (according to someone specific requirements, maybe hallucinations are OK, maybe not) that - needs privacy (for example generating code that will be open source probably does not need that) - can't be run locally - can be trusted to actually process as said it does
- wat10000 11mo agoI've found LLMs to be extremely useful for writing helper scripts (debugger enhancements, analyzing large disassembly dumps, that sort of thing) and as a next-level source code search. Take a large code base and you get an error in component A involving a type in component B and it's not immediately obvious how the two are connected. I've had great success in giving an LLM the error message and access to the code and asking it how B got to A and why that's an error. This is something I could certainly do myself, but there are times when it would take 100x longer. The key is that these are all things I can verify without much difficulty: read over the script, spot-check the analysis, look at the claimed connection between A and B and see if it's real. And I don't really care about style, quality, maintainability. You certainly can run this locally, but anything that will fit into reasonable local hardware won't be as good. I don't need to trust it to process as it says it does, because I'm verifying the output. And as far as I'm concerned, "needs privacy" is always true. I don't care if the code will be open source. I don't care if it's analyzing existing code that's already open source. Other people have no business seeing what I'm doing unless I explicitly allow it. In any case, I work on a lot of proprietary code as well, and my employer would be most displeased if I were exposing it to others.
- MangoToupe 11mo ago@dang can we modify the title to acknowledge that it's specific to chatbots? The title reads like this is about generic compute, and the content is emphatically not about generic compute. I realize this is just bad branding by apple but it's still hella confusing.
- jmort 11mo agoIt does work generically. Like Apple, we initially targeted inference, but it under the hood just an anonymous, attested HTTP server wrapper. The ComputeNode can run an arbitrary workload.
- MangoToupe 11mo agoInteresting!
- wferrell 11mo agoReally nice release. Excited to see this out in the wild and hopeful more companies leverage this for better end user privacy.
- DeveloperOne 11mo agoGlad to see Golang here. Go will surpass Python in the AI field, mark my words.
- mr_windfrog 11mo ago[dead]
- nixpulvis 11mo agoThought this was going to be about Orchard from the title.
- sublimefire 11mo agoQuite similar to what Azure with conf ai inference did [1]. [1] https://techcommunity.microsoft.com/blog/azureconfidentialcomputingblog/azure-ai-confidential-inferencing-technical-deep-dive/4253150 https://techcommunity.microsoft.com/blog/azureconfidentialco...
- jmort 11mo agoI haven’t been able to find their source code. Pretty important for the transparency side of it. Have you seen it?
- deleted 11mo ago[deleted]