7 ms·

Thought it was bait, but I can confirm I can find private repos in the search results. What the heck?

tailspin2019 2y ago
Were those repos always private?
simonw 2y ago
Which private repos? Any that you're willing to share (I get that sharing names of private repos goes against the whole idea of them being private!)
rfoo 2y ago
Since your repo's name is public now anyway, could you please help us and post it here? I'm really curious about what happened. Since public GitHub activities were archived [1], if you post it here we can check if it were ever public or it's truly private at all time. [1] https://www.gharchive.org/ https://www.gharchive.org/
- johncoatesdev 2y ago
  How do I check this? I found a repo I'm pretty sure was always private on there that I deleted a while back. https://github.com/johncoates/JCBootstrap https://github.com/johncoates/JCBootstrap There's no archived version on archive.org at least.
  - simonw 2y ago
    https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/johncoates/JCBootstrap https://archive.softwareheritage.org/browse/origin/directory... shows a "snapshot date" of "11 August 2015, 07:28:00 UTC" - any chance it was public on that date such that the crawler could have accessed it?
    - sdesol 2y ago
      I checked my GitHub archive (https://www.gharchive.org/ https://www.gharchive.org/) indexed data and the only repo that I saw for johncoates was LanscapeVideos, which has a last event time of 2015-06-09 07:09:52+02 It is important to note that GitHub archive is not 100% accurate and there is over 319 missing hours.
    - johncoatesdev 2y ago
      I can't find any reason why I would have made it public. I made the repo in 2014 for internal use and don't like to share projects like that. I'm pretty careful when releasing any code publicly. It's some code that other private projects depend on. I searched for any references in public code and there are none, so there should have been no reason to make it public. Interestingly my public code with thousands of stars isn't in "The Stack".
      - bredren 2y ago
        This shouldn't be something where we're relying on recollection. Presumably github repo privacy state has an audit trail. This would allow GH to prove / disprove claims on any given repo easily. I hope a rep steps in to do so.
        johncoatesdev 2y ago
        Yeah I agree. Tried https://news.ycombinator.com/item?id=39771541 https://news.ycombinator.com/item?id=39771541 but there's nothing related to this repo. Does GitHub send an email out when you make something public? I don't have any emails related to this repo.
        simonw 2y ago
        I just upgraded the tool at https://observablehq.com/@simonw/github-public-repo-history https://observablehq.com/@simonw/github-public-repo-history to use lowercase comparisons (previously it was case sensitive) so it's worth having another look.
- yreg 2y ago
  It seems the original ones (from the mastodon post) are - https://github.com/emenel/dust https://github.com/emenel/dust - https://github.com/emenel/portfolio https://github.com/emenel/portfolio (based on https://archive.softwareheritage.org/browse/search/?q=emenel&with_visit=true&with_content=true https://archive.softwareheritage.org/browse/search/?q=emenel...) Care to check them on gharchive? I bet they used to be public.
  - sdesol 2y ago
    I checked my GitHub archive data and emenel/dust is not there, but emenel/portfolio is. Note, GitHub archive is not 100% accurate and it is missing 319 hours.
  - latexr 2y ago
    > based on The data for The Stack’s dataset is sourced from the Software Heritage Archive, so checking that is redundant. We need different sources.
    - mkishi 2y ago
      Even still, both repos had READMEs [1][2] clearly meant to be read by the public. The archival was only successful years ago, with a failed snapshot as far back as 2021 [3]. This really seems like they forgot it was ever public. Now, this is only about it being a GitHub breach. Whether unlicensed (emenel/portfolio) or GPL (emenel/dust) code should be allowed in such datasets is a different matter. [1] https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/emenel/portfolio https://archive.softwareheritage.org/browse/origin/directory... [2] https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/emenel/dust https://archive.softwareheritage.org/browse/origin/directory... [3] https://archive.softwareheritage.org/browse/origin/visits/?origin_url=https://github.com/emenel/portfolio https://archive.softwareheritage.org/browse/origin/visits/?o...
    - yreg 2y ago
      I used SW Heritage to identify the repos that were used for training, since op did not post the repo names. The “different source” is supposed to be ghactions.

tailspin2019 2y ago

Were those repos always private?

Which private repos? Any that you're willing to share (I get that sharing names of private repos goes against the whole idea of them being private!)

rfoo 2y ago

Since your repo's name is public now anyway, could you please help us and post it here? I'm really curious about what happened. Since public GitHub activities were archived [1], if you post it here we can check if it were ever public or it's truly private at all time. [1] https://www.gharchive.org/ https://www.gharchive.org/

johncoatesdev 2y ago
How do I check this? I found a repo I'm pretty sure was always private on there that I deleted a while back. https://github.com/johncoates/JCBootstrap https://github.com/johncoates/JCBootstrap There's no archived version on archive.org at least.
- simonw 2y ago
  https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/johncoates/JCBootstrap https://archive.softwareheritage.org/browse/origin/directory... shows a "snapshot date" of "11 August 2015, 07:28:00 UTC" - any chance it was public on that date such that the crawler could have accessed it?
  - sdesol 2y ago
    I checked my GitHub archive (https://www.gharchive.org/ https://www.gharchive.org/) indexed data and the only repo that I saw for johncoates was LanscapeVideos, which has a last event time of 2015-06-09 07:09:52+02 It is important to note that GitHub archive is not 100% accurate and there is over 319 missing hours.
  - johncoatesdev 2y ago
    I can't find any reason why I would have made it public. I made the repo in 2014 for internal use and don't like to share projects like that. I'm pretty careful when releasing any code publicly. It's some code that other private projects depend on. I searched for any references in public code and there are none, so there should have been no reason to make it public. Interestingly my public code with thousands of stars isn't in "The Stack".
    - bredren 2y ago
      This shouldn't be something where we're relying on recollection. Presumably github repo privacy state has an audit trail. This would allow GH to prove / disprove claims on any given repo easily. I hope a rep steps in to do so.
      - johncoatesdev 2y ago
        Yeah I agree. Tried https://news.ycombinator.com/item?id=39771541 https://news.ycombinator.com/item?id=39771541 but there's nothing related to this repo. Does GitHub send an email out when you make something public? I don't have any emails related to this repo.
        simonw 2y ago
        I just upgraded the tool at https://observablehq.com/@simonw/github-public-repo-history https://observablehq.com/@simonw/github-public-repo-history to use lowercase comparisons (previously it was case sensitive) so it's worth having another look.
yreg 2y ago
It seems the original ones (from the mastodon post) are - https://github.com/emenel/dust https://github.com/emenel/dust - https://github.com/emenel/portfolio https://github.com/emenel/portfolio (based on https://archive.softwareheritage.org/browse/search/?q=emenel&with_visit=true&with_content=true https://archive.softwareheritage.org/browse/search/?q=emenel...) Care to check them on gharchive? I bet they used to be public.
- sdesol 2y ago
  I checked my GitHub archive data and emenel/dust is not there, but emenel/portfolio is. Note, GitHub archive is not 100% accurate and it is missing 319 hours.
- latexr 2y ago
  > based on The data for The Stack’s dataset is sourced from the Software Heritage Archive, so checking that is redundant. We need different sources.
  - mkishi 2y ago
    Even still, both repos had READMEs [1][2] clearly meant to be read by the public. The archival was only successful years ago, with a failed snapshot as far back as 2021 [3]. This really seems like they forgot it was ever public. Now, this is only about it being a GitHub breach. Whether unlicensed (emenel/portfolio) or GPL (emenel/dust) code should be allowed in such datasets is a different matter. [1] https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/emenel/portfolio https://archive.softwareheritage.org/browse/origin/directory... [2] https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/emenel/dust https://archive.softwareheritage.org/browse/origin/directory... [3] https://archive.softwareheritage.org/browse/origin/visits/?origin_url=https://github.com/emenel/portfolio https://archive.softwareheritage.org/browse/origin/visits/?o...
  - yreg 2y ago
    I used SW Heritage to identify the repos that were used for training, since op did not post the repo names. The “different source” is supposed to be ghactions.