13 ms·
Keybase team member here. Interesting fact: git doesn't check the validity of sha-1 hashes in your commit history. Meaning if someone compromises your hosted o
by malgorithms 9y ago
Keybase team member here. Interesting fact: git doesn't check the validity of sha-1 hashes in your commit history. Meaning if someone compromises your hosted origin, they can quietly compromise your history. So even the fears about data leaks aside, this is a big win for safety.
From an entrepreneurial perspective, this is my favorite thing we've done at Keybase. It pushes all the buttons: (1) it's relatively simple, (2) it's filling a void, (3) it's powered by all our existing tech, and (4) it doesn't complicate our product. What I mean by point 4 is that it adds very little extra UX and doesn't change any of the rest of the app. If you don't use git, cool. If you do, it's there for you.
What void does this fill? Previously, I managed some solo repositories of private data in a closet in my apartment. Who does that? It required a mess: uptime of a computer, a good link, and dynamic dns. And even then, I never could break over the hurdle of setting up team repositories with safe credential management...like for any kind of collaboration. With this simple screen, you can grab 5 friends, make a repo in a minute, and all start working on it. With much better data safety than most people can achieve on their own.
- drudru11 9y agoThanks for making this - this is the first keybase product that I've gotten really excited about!
- Foxboron 9y agoWhile talking about git and security: Signing tags are not as affective as you'd think. refs are never actually signed, it's the objects they are pointing at that are signed. This opens up to interesting attacks where you can move refs around to previous vulnerable versions. Git also never checks if the metadata the tag points at is correct! Interesting paper: https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_torres-arias.pdf https://www.usenix.org/system/files/conference/usenixsecurit...
- im_down_w_otp 9y agoYeah, we implemented this paper's proposal (their version has some bugs, gaps, and infinite loop issues) where I work to be able to have higher assurance on the validity of our source repositories. First version in shell with a fairly robust test suite, and the next version in Rust. Originally started to do it in Rust, but libgit2 was sufficiently obtuse that we opted for getting to a complete, working thing first.
- ianleeclark 9y agoFor your first point, can't you verify the signature for the commit? In order to to compromise the origin, they must also compromise the secret key of whoever is signing commits. I say that in full realization that 99% of people probably don't even know that you can sign commits, but the first point doesn't seem valid, as you can ensure integrity of commit history.
- gaxun 9y ago> hurdle of setting up team repositories with safe credential management...like for any kind of collaboration Identity continues to be the key selling point of keybase. I'm excited by this. I can keep clones of my private repositories here. Things like dotfiles and configurations. That sounds like a good start. And I can also easily share code to people who need to see it.
- mikeash 9y agoThis looks sweet. I bounce between using Bitbucket or Dropbox for private repos depending on my needs. Bitbucket has lots of features but is a little annoying to set up a new project. Dropbox is really easy but doesn't always work well (e.g. git push ends up being effectively async). Your version of it looks to be just as easy as Dropbox, maybe even easier, but without any of the downsides. And it's encrypted!
- eropple 9y agoFor those reading, you should under no circumstances use Dropbox as git hosting unless you're using git-remote-dropbox: https://github.com/anishathalye/git-remote-dropbox https://github.com/anishathalye/git-remote-dropbox Hosing your repo is way too easy otherwise.
- mikeash 9y agoDoes it matter much? If I hose my repo (which I don't think is that easy, since I've been doing this for years and never had an issue) then I can delete it and clone a new one from my local copy. Especially when it's just me, and I'm only pushing to the repo from one machine at a time.
- eropple 9y agoIt can hose your local, too. And it can happen easier than you think--I've seen it happen because a laptop that pushed to Dropbox went to sleep mid-sync and a desktop synced after. Fighting the Dropbox API to unwind it is a huge pain. git-remote-dropbox works as you would expect a Git remote to work; it's API-driven and actively discourages even syncing the remote repository down to your machine. I would so, so strongly suggest you switch to it if you want to use Dropbox as a store. Bare-git-repo-on-KBFS is inadvisable for a similar reason, which is why I'm so excited to see what they're doing here.
- mikeash 9y agoHow would it hose my local? I thought git's design meant that it might possibly pull down new corrupted refs, but whatever I currently had would remain intact, so it's just a matter of reverting. Not so?
- eropple 9y agoSo I love Keybase unconditionally and if you guys weren't rolling in physical offices (and not one in Boston) I'd have been beating down your door to come work there--I think what Keybase is doing is important and it's something I'd love to work on. But I have a serious question that maybe you can answer, and it's something everybody who I've showed this to has asked me: How is Keybase gonna make money? How am I assured that this, and everything else in my Keybase storage, is going to be there in six months? Like, I still have a private server in a closet in my apartment that syncs all the stuff I trust Keybase with because I don't know what the business-side failure case is. You guys should be taking my money, is what I'm saying. Also probably hiring me. But definitely taking my money.
- Osmium 9y ago> You guys should be taking my money, is what I'm saying. Completely agreed. The reason I don't use Keybase more than I do is because I half expect them to be acquired/something else to happen. Would gladly give them my $10/mo. for a 1TB instead of Dropbox. With that said, I completely understand why they aren't right now -- maybe they're not going after the consumer market, maybe they don't want to box themselves in with customer support obligations, etc. But I really would like to use them.
- eropple 9y agoFor sure. I would give them my ten bucks a month for the 100GB I get for free.
- sandGorgon 9y agoI will pay a LOT of money if you can slap a half decent web interface on it. Surprisingly, you guys look like a direct clone of the new Bitbucket interface. Its not my favorite (I like github so much better) - but Bitbucket with its inbuilt Pipelines integrations is so much better than Github.
- zeroping 9y agoBut how will their web server present that data? They can't read the data.
- falsedan 9y ago> Interesting fact: git doesn't check the validity of sha-1 hashes in your commit history. You mean, you have to run git fsck after pulling, since git only checks that you got what you asked for?
- u801e 9y ago> Interesting fact: git doesn't check the validity of sha-1 hashes in your commit history. Isn't the commit sha1 determined, in part, by the sha1 values of the tree it refers to as well as the sha1 of the parent commit? If you fetch a branch from a compromised remote, all the sha1 values of the commits that were compromised would be different.
- azag0 9y agoCorrect, but git doesn't recompute the hashes locally, so it wouldn't know they are wrong.
- u801e 9y agoAh, so if I were to manually craft a commit in a text editor in the format: tree sha1 parent sha1 of parent I want to attach it to author some string committer some string The commit message I could add this to the git object store manually under the same sha1 file and a client could just fetch it? Would the client try to fetch the faked objects when it already has the real objects in its copy of the object store? That is, would it think it has the commit because the sha1 hasn't changed, but the tree sha1 has been updated and it would presumably refer to blobs that the client doesn't already have and try to fetch them. Or would it not proceed because it already has the commit?
- simonbyrne 9y agoIt doesn't seem to verify hashes of objects on checkout, but it does when receiving packfiles. So it's difficult to see how this could be an exploit unless the attacker has access to your local .git directory.
- eridius 9y agoIf you're syncing the repository itself (e.g. over Dropbox) instead of using git remotes, then it could be exploited.
- 9y ago
- tareqak 9y agoThank you and the rest of the Keybase team for your hard work! Is it possible to use this on the Keybase mobile app for like note-taking?
- hannob 9y ago> Keybase team member here. Interesting fact: git doesn't check the validity of sha-1 hashes in your commit history. I heard this a couple of times and tried to confirm it a while ago, but was unable to. I wasn't able to forge a repository with faulty hashes in it. I also heard plenty of people tell me that there exist public repositories with wrong hashes in them, but when I asked them they never could come up with concrete examples in the wild. I'm seriously curious about this, can you provide any clonable proof of concept repository with wrong hashes?
- simonbyrne 9y agoLikewise, I would be keen to see an example of this.
- lsiebert 9y agoI may be wrong, but here's my current understanding. I believe Git CAN check the validity of sha1 hashes (I read the source a few years ago and have a very tiny git commit) using git fsck, which I believe kernel.org does nightly. It just doesn't do so automatically with every commit or whatever. But you can set up a test in your server, I believe, if that's important to you, either watching the files, or checking pushes which I believe github does. So that's not the issue. It's sha-1 collision attacks that are a theoretical issue. My understanding of the currently known SHA-1 attack is that it requires binary data (hence PDF files for the example) and requires you to control both the original file and the subsequent file. So an attack would have to generate an apparently innocent file and a malicious file both of which have a binary block, insert the innocent file into the repo, and then somehow, most likely outside of a git push given mitigations like github's, replace that innocent file with the malicious file. Now to your question, checking in the PDF files from the proof of the attack in git doesn't work, because git also adds header info. And generating the files requires ~ $100,000 dollars worth of ec2 time, or the equivalent, so nobody has gone through the trouble of generating files that allow this specifically to prove it for git. Bit it's definitely possible, and cheap enough for a criminal organization or a state agency to do. Just because someone hasn't done it for git specifically shouldn't mean that the attack isn't possible, just that security researchers don't have unlimited funds, and the existing proof, while not specific to git shows the issue generally applies. Last I saw, the git mailing list was debating sha3-256 and BLAKE vs SHA-256. There's some indication that SHA-256 may get intel HW support, and that may be useful for speed with really really big git repos (like microsoft's apparently). SHA-256 doesn't have an attack on it that's known but unlike ShA3-256 (and I believe BLAKE since it's a stream cipher) SHA-256 is a block cipher, so it's not stateful. That means, while no known attack exists, theoretically if an attack existed you could corrupt a specific block in a similar manner to SHA-1. But SHA-256 has been much more extensively tested for issues while SHA3-256 is newer... it was created ostensibly as a backup in case the current known safe standard of crypto like SHA-256 is attackable. There are some issues with SHA-256 being used in repos that have signed SHA-1 hashes already, in terms of mapping SHA-256 to SHA-1 hashes without borking the signing. Obviously if you change the underlying structure of signed stuff to store a new hash, it changes the hash. My personal thought would be to implement SHA-256 and SHA3-256 as options simultaneously, as they are both NIST standards, make SHA-256 the standard so big repos can be as fast as possible. I am not a crypto expert, or a git expert though, so if I'm wrong, please correct me. Being wrong means I get to learn stuff and that's great!
- WorldMaker 9y agoHow likely/easy would it be to add "know nothing" mirrors of these encrypted repositories? Say that I trust the keybase app (or something that speaks its protocols) possibly indefinitely, but maybe I'm not keen on a single cloud storage backend and want additional secure backup options. (Maybe I'm even unconvinced about the long term guarantee of keybase's storage space offerings due to possibly changing cost/business model factors, as others have pointed out here.) It would be nice if I could have an encrypted copy in S3 or Dropbox or somewhere, that presumably maybe git couldn't directly make use of, and would be encrypted and those services couldn't touch either, but that the app could still push/pull changes to. Certainly, I'd still have an unencrypted view of the contents in any local clones of the repository I may have in the case that I couldn't access keybase storage, but it still seems like there may be useful cases where an encrypted backup is somewhere else in the cloud as well, as a safe failover just in case.
- aeorgnoieang 9y agoYou can do this with a Git helper like this one: - [spwhitton/git-remote-gcrypt: PGP-encrypted git remotes](https://github.com/spwhitton/git-remote-gcrypt https://github.com/spwhitton/git-remote-gcrypt) I use [Pass](https://www.passwordstore.org/ https://www.passwordstore.org/), a password manager, which uses GPG and Git, and I keep an encrypted copy of my Pass Git repo in Dropbox and have that repo copy setup as a remote in all of the local copies of my password repo. So, the contents of the local repos are encrypted, but in the encrypted copy all of the Git data is encrypted too.
- cs702 9y agoHow does this compare to git-gpg, mentioned below in this thread? https://news.ycombinator.com/item?id=15403360 https://news.ycombinator.com/item?id=15403360
- ptspts 9y agoSee my comparison of Keybase Git, git-remote-gcrypt and git-gpg here: http://ptspts.blogspot.com/2017/10/comparison-of-encrypted-git-remote.html http://ptspts.blogspot.com/2017/10/comparison-of-encrypted-g...
- RasputinsBro 9y agoThanks for that. Is Keybase Git all hype then? Because the alternatives seem a lot better. To be honest, I'm not even sure I understand what Keybase _is_.
- kazinator 9y ago> git doesn't check the validity of sha-1 hashes in your commit history. What, like never? Or just not under specific circumstances? I sure wouldn't want git to be doing that in every darn operation that traverses the history, like git log. When receiving packets from another repo though, it would be useful.
- dheelus 9y agoWhat would I need to do to permit someone read-only, clear-text, non-public access to an encrypted repo? Can a combination of existing GIT / GitHub privileges and the Keybase solution help? If yes, and if you can add 2FA and we might be interested in becoming a customer.
- Natanael_L 9y agoToday your usecase can be solved ad-hoc by additionally manually signing what you push to keybase git, shared with the people you want to have read access. If you want an encrypted storage solution with integrated read only access capabilities, I recommend using Tahoe-LAFS. You can probably store a git repository in it just fine.
- eridius 9y agoThis looks fantastic! I have a couple of questions not answered in the FAQ though: 1. Is there (or will there be) any way to create an encrypted git repo shared between a few users that aren't part of a team? e.g. could I create a repo that belongs to eridius,chris and have us both access it? 2. Can I create a repo that belongs to a subteam? And on a different note, I want to create a team but the name is currently taken by a user. The user has zero activity (no devices, no proofs, chain is completely empty, literally nothing). Is there any way to recover a name that's being squatted on?
- strib 9y ago> 1. Is there (or will there be) any way to create an encrypted git repo shared between a few users that aren't part of a team? e.g. could I create a repo that belongs to eridius,chris and have us both access it? Yep, though it's undocumented and it won't show up in the GUI right now (maybe ever). You can just push/pull directly to repos like "keybase://private/u1,u2,u3/foo" and it will create it on the fly. But we warned, there's currently no way to delete those, and typos in the git URL can cause unintended repos to pop up. > Can I create a repo that belongs to a subteam? Yep, should be the same as a regular team.
- rgbrenner 9y agoAnd even then, I never could break over the hurdle of setting up team repositories with safe credential management...like for any kind of collaboration. With this simple screen, you can grab 5 friends, make a repo in a minute, and all start working on it. You can already do that with Gogs.. It's a single binary, uses git, supports accounts, 2 factor, etc. https://gogs.io/ https://gogs.io/ Really useful for small teams that don't want to use github or gitlab.
- _spoonman 9y agoCongratulations on the launch. I'm a Keybase user myself and I think you all have done a fantastic job. When the SHA-1 collision was calculated earlier this year, Linus commented on git and SHA-1. No further questions, just sharing it here if you happened not to see it: https://marc.info/?l=git&m=148787047422954 https://marc.info/?l=git&m=148787047422954 Again, thanks for all the hard work. Best of luck.
- tosstossy 9y agotransfer.fsckObjects=true There's been talk of making this the default, but it's trivial enough to stick in your .git
- rolandog 9y agoGreat work! I just happened to stumble into Mike Gerwitz's (2012) Horror Story [1] today about trust in git. This is great for safety. [1] https://mikegerwitz.com/papers/git-horror-story https://mikegerwitz.com/papers/git-horror-story
- rlpb 9y ago> git doesn't check the validity of sha-1 hashes in your commit history. By default. But set "git config --global transfer.fsckObjects true" and it will. No need to install anything else just for that.
- IshKebab 9y agoWho actually does that though? I'm guessing less than 1% of users. Any idea why it isn't on by default?
- aey 9y agoIt looks like you guys use react for a lot of your development. How do I know that you won’t push compromised code behind the scenes? Even unknowingly. Btw your product is awesome! Multi platform encrypted team chat that doesn’t even need 4gb of ram :)