8 ms·

Maintainers: 2 months of inactivity on a FOSS project Redditors: Yep, this is the end.

pwdisswordfish8 5y ago
I mean, just look at <https://github.com/ytdl-org/youtube-dl/issues/29809 https://github.com/ytdl-org/youtube-dl/issues/29809>. They don’t even bother removing spam.
_joel 5y ago
The amount youtube and the other sites it supports changes to subvert the use of this very tool, I can see why people are wondering what's happening
deleted 5y ago
[deleted]
dTal 5y ago
2 months is a long time in youtube-dl world. It's not really a "software project" in the traditional sense, where you can stop working on it once it's "done". It's more of a "social project", a focal point for the required ongoing activity necessary to keep sites working. Youtube-dl without daily commits is useless.
- bittercynic 5y ago
  I'm currently using it to download a youtube video. If it still works for it's main function, maybe it's just not a high priority project until something breaks?
  - banana_giraffe 5y ago
    Primary function, perhaps, but it's used for lots of websites that aren't YouTube, and some of those websites have broken. There are open pull requests and/or bugs for many websites that aren't being approved. This is rather unusual for youtube-dl, it normally had a release ever two weeks or so.
  - klodolph 5y ago
    It may say “YouTube” in the name, but it’s a tool designed to work with many sites.
  - d0lphin 5y ago
    It's actually already broken. If you try to download more than 3-16 videos (the limit is not clear), you start to be rate limited to 300 kbps or so. According to Reddit this is fixed in a fork called yt-dlp
    - geerlingguy 5y ago
      One other workaround is to use '-F' to get a list of formats, and choose one of the non-default formats to download. This seems to work for me.
    - ketralnis 5y ago
      To be honest that sounds fine. You're presumably downloading it to watch it offline, the download speed isn't really material as long as it finishes eventually
      - majewsky 5y ago
        youtube-dl is also used by video players like mpv for live streaming. In this case, 70 kbps is completely unusable for the vast majority of videos.
ugjka 5y ago
For a project like youtube-dl it is a long time, because they use unofficial APIs (fancy word for scraping) of video sites that can shift even on daily basis. If you look at their Github issues it is just people endlessly complaining that some websites are broken again
- OJFord 5y ago
  > unofficial APIs (fancy word for scraping) Using a non-public API is not at all the same as scraping, which refers to parsing a rendered HTML page for the content you want. Both have this maintenance problem, but one's not a fancy word for the other.
  - simonw 5y ago
    That used to be true, but today, with so many websites operating as SPAs against undocumented APIs, I think it's reasonable to redefine "scraping" to mean extracting data from unofficial APIs in addition to extracting it by parsing HTML. After all, what is a scrapeable HTML page if not a grotesquely convoluted undocumented API with an unstable output format?
    - addingnumbers 5y ago
      Scraping refers specifically to extracting data from a format designed to be read by humans instead of machines. The gross inefficiency and low data-to-layout ratio are the key things being expressed through connotations of the word "scrape". To scrape is to extract a small amount of something from a much larger substrate. To call every query a scrape is to diminish the specificity and utility of the term.
      - ugjka 5y ago
        So you get a video feed in the end, that is viewed by human presumably?
        addingnumbers 5y ago
kbenson 5y ago
That's not an accurate assessment of what's going on. The post linked states "no contact with the maintainers" not just inactivity, and if you follow the github issues linked, it's about people wondering what's going on and if anyone else can approve pull requests because there's lots of pull requests waiting. There's 843 pull requests at this time, and I just looked and over 50 are from just the last month. It's not that there's repo inactivity, it seems to be that this is an extremely active repo which saw everything grind to a halt when the admins went dark. That's quite a bit different then just "inactivity".
- luovatek 5y ago
  What if the maintainers just got a bad case of covid? I hear there's this awful pandemic, but I guess they might only be baseless rumors...
  - kbenson 5y ago
    Then the repo might be dead because of that. I don't think people are (or should be) necessarily making assessments as to the reasoning of the admins without evidence, and there's a bunch of reasons why they might not be able to participate for a few months or even ever again, but if they are unavailable for a few months and nobody can even explain why, the end result is functionally the same if they're the only ones with the keys to the kingdom, which is that it's dead until something changes.
    - aabbcc1241 5y ago
      Is it a way to work on FOSS in decentralized or even distributed manner? Without relaying on the Github index as the critical point.
      - kbenson 5y ago
        Sure, fork it and have alternate implementations and releases. This will necessarily fragment the namespace for it though, and the only way around that is to have some shared responsibility for making releases, and at that point, you might as well have enough shared admin responsibilities on the one repo, and the problem is also solved. Another way to look at it is that this is not a problem of lack of solutions, it a problem that the person or people at the bottleneck (merging pull requests, testing and releasing) haven't shared that responsibility with enough additional people so this isn't a problem. That, or there's some legal case in the background and they are under advisement to not work on or respond to anything to do with this repository. That seems just as likely in my eyes. Doesn't change the situation though, this repo is effectively dead right now (but someone could fork and start applying an releasing if they wanted AFAICT).
umvi 5y ago
I guess it depends on if sub-2-month feature development is needed to keep up with YouTube's changes or not. I maintain a GCC code coverage tool on GitHub, and since GCC doesn't change very often and the feature set of the tool is fairly complete, I sometimes go 6+ months without commits. Usually I don't touch it unless someone opens an issue.
shp0ngle 5y ago
As other mentioned, it’s different with YouTube, as the site changes constantly
- dspillett 5y ago
  Also, in common with other massive online properties for instance Amazon, not consistently: changes sometimes roll out a bit at a time so users in different areas get different versions, either due to global roll-out being a staged process or because a UI experiment is being performed. It usually doesn't matter to a well written scraper as the core data is still accessible in the same way despite the UI sugar coat having changed, but sometimes there are significant enough changes under the hood too that the scraper needs to deal with while still supporting the older format(s).
tomaszs 5y ago
Exactly. It is ok to compete but announcing your competitor project is dead to promote your own, it is a little sketchy to say the least.
devrand 5y ago
Having contributed to youtube-dl in the past, long turnaround times from the maintainers was pretty normal. I've had (and still have some) open PRs that have been ready to merge for going on a year. The two months is really not that big of deal. That being said, the project probably could use some reorganization. It requires a lot of community contributions to keep all the extractors maintained so long turnaround times for reviews isn't ideal.
hatsunearu 5y ago
> Considering there's been weekly releases for years, two months is odd and worry some. https://old.reddit.com/r/DataHoarder/comments/p9riey/youtubedl_is_possibly_dead/h9zvalr/ https://old.reddit.com/r/DataHoarder/comments/p9riey/youtube... Sounds more like maybe the person is sick or something.
jms703 5y ago
It's August. Could the maintainers be on holiday?
momothereal 5y ago
Seriously. I've seen core/lead maintainers on projects I worked on go awol for months and years at a time, and come back guns blazing.
ChrisArchitect 5y ago
dark thought but I've had multiple instances in last year+ where someone hasn't posted in awhile, tweeted etc and I wonder for a second, maybe they died? The pandemic is real. And random ppl disappearing unexpectedly is part of it. I hope all is well.

I mean, just look at <https://github.com/ytdl-org/youtube-dl/issues/29809 https://github.com/ytdl-org/youtube-dl/issues/29809>. They don’t even bother removing spam.

_joel 5y ago

The amount youtube and the other sites it supports changes to subvert the use of this very tool, I can see why people are wondering what's happening

deleted 5y ago

[deleted]

dTal 5y ago

2 months is a long time in youtube-dl world. It's not really a "software project" in the traditional sense, where you can stop working on it once it's "done". It's more of a "social project", a focal point for the required ongoing activity necessary to keep sites working. Youtube-dl without daily commits is useless.

bittercynic 5y ago
I'm currently using it to download a youtube video. If it still works for it's main function, maybe it's just not a high priority project until something breaks?
- banana_giraffe 5y ago
  Primary function, perhaps, but it's used for lots of websites that aren't YouTube, and some of those websites have broken. There are open pull requests and/or bugs for many websites that aren't being approved. This is rather unusual for youtube-dl, it normally had a release ever two weeks or so.
- klodolph 5y ago
  It may say “YouTube” in the name, but it’s a tool designed to work with many sites.
- d0lphin 5y ago
  It's actually already broken. If you try to download more than 3-16 videos (the limit is not clear), you start to be rate limited to 300 kbps or so. According to Reddit this is fixed in a fork called yt-dlp
  - geerlingguy 5y ago
    One other workaround is to use '-F' to get a list of formats, and choose one of the non-default formats to download. This seems to work for me.
  - ketralnis 5y ago
    To be honest that sounds fine. You're presumably downloading it to watch it offline, the download speed isn't really material as long as it finishes eventually
    - majewsky 5y ago
      youtube-dl is also used by video players like mpv for live streaming. In this case, 70 kbps is completely unusable for the vast majority of videos.

ugjka 5y ago

For a project like youtube-dl it is a long time, because they use unofficial APIs (fancy word for scraping) of video sites that can shift even on daily basis. If you look at their Github issues it is just people endlessly complaining that some websites are broken again

OJFord 5y ago
> unofficial APIs (fancy word for scraping) Using a non-public API is not at all the same as scraping, which refers to parsing a rendered HTML page for the content you want. Both have this maintenance problem, but one's not a fancy word for the other.
- simonw 5y ago
  That used to be true, but today, with so many websites operating as SPAs against undocumented APIs, I think it's reasonable to redefine "scraping" to mean extracting data from unofficial APIs in addition to extracting it by parsing HTML. After all, what is a scrapeable HTML page if not a grotesquely convoluted undocumented API with an unstable output format?
  - addingnumbers 5y ago
    Scraping refers specifically to extracting data from a format designed to be read by humans instead of machines. The gross inefficiency and low data-to-layout ratio are the key things being expressed through connotations of the word "scrape". To scrape is to extract a small amount of something from a much larger substrate. To call every query a scrape is to diminish the specificity and utility of the term.
    - ugjka 5y ago
      So you get a video feed in the end, that is viewed by human presumably?
      - addingnumbers 5y ago

kbenson 5y ago

That's not an accurate assessment of what's going on. The post linked states "no contact with the maintainers" not just inactivity, and if you follow the github issues linked, it's about people wondering what's going on and if anyone else can approve pull requests because there's lots of pull requests waiting. There's 843 pull requests at this time, and I just looked and over 50 are from just the last month. It's not that there's repo inactivity, it seems to be that this is an extremely active repo which saw everything grind to a halt when the admins went dark. That's quite a bit different then just "inactivity".

luovatek 5y ago
What if the maintainers just got a bad case of covid? I hear there's this awful pandemic, but I guess they might only be baseless rumors...
- kbenson 5y ago
  Then the repo might be dead because of that. I don't think people are (or should be) necessarily making assessments as to the reasoning of the admins without evidence, and there's a bunch of reasons why they might not be able to participate for a few months or even ever again, but if they are unavailable for a few months and nobody can even explain why, the end result is functionally the same if they're the only ones with the keys to the kingdom, which is that it's dead until something changes.
  - aabbcc1241 5y ago
    Is it a way to work on FOSS in decentralized or even distributed manner? Without relaying on the Github index as the critical point.
    - kbenson 5y ago
      Sure, fork it and have alternate implementations and releases. This will necessarily fragment the namespace for it though, and the only way around that is to have some shared responsibility for making releases, and at that point, you might as well have enough shared admin responsibilities on the one repo, and the problem is also solved. Another way to look at it is that this is not a problem of lack of solutions, it a problem that the person or people at the bottleneck (merging pull requests, testing and releasing) haven't shared that responsibility with enough additional people so this isn't a problem. That, or there's some legal case in the background and they are under advisement to not work on or respond to anything to do with this repository. That seems just as likely in my eyes. Doesn't change the situation though, this repo is effectively dead right now (but someone could fork and start applying an releasing if they wanted AFAICT).

umvi 5y ago

I guess it depends on if sub-2-month feature development is needed to keep up with YouTube's changes or not. I maintain a GCC code coverage tool on GitHub, and since GCC doesn't change very often and the feature set of the tool is fairly complete, I sometimes go 6+ months without commits. Usually I don't touch it unless someone opens an issue.

shp0ngle 5y ago

As other mentioned, it’s different with YouTube, as the site changes constantly

dspillett 5y ago
Also, in common with other massive online properties for instance Amazon, not consistently: changes sometimes roll out a bit at a time so users in different areas get different versions, either due to global roll-out being a staged process or because a UI experiment is being performed. It usually doesn't matter to a well written scraper as the core data is still accessible in the same way despite the UI sugar coat having changed, but sometimes there are significant enough changes under the hood too that the scraper needs to deal with while still supporting the older format(s).

tomaszs 5y ago

Exactly. It is ok to compete but announcing your competitor project is dead to promote your own, it is a little sketchy to say the least.

devrand 5y ago

Having contributed to youtube-dl in the past, long turnaround times from the maintainers was pretty normal. I've had (and still have some) open PRs that have been ready to merge for going on a year. The two months is really not that big of deal. That being said, the project probably could use some reorganization. It requires a lot of community contributions to keep all the extractors maintained so long turnaround times for reviews isn't ideal.

hatsunearu 5y ago

> Considering there's been weekly releases for years, two months is odd and worry some. https://old.reddit.com/r/DataHoarder/comments/p9riey/youtubedl_is_possibly_dead/h9zvalr/ https://old.reddit.com/r/DataHoarder/comments/p9riey/youtube... Sounds more like maybe the person is sick or something.

jms703 5y ago

It's August. Could the maintainers be on holiday?

momothereal 5y ago

Seriously. I've seen core/lead maintainers on projects I worked on go awol for months and years at a time, and come back guns blazing.

ChrisArchitect 5y ago

dark thought but I've had multiple instances in last year+ where someone hasn't posted in awhile, tweeted etc and I wonder for a second, maybe they died? The pandemic is real. And random ppl disappearing unexpectedly is part of it. I hope all is well.