6 ms·

It is interesting that YouTube isn't mentioned in this blogpost, despite there being good evidence that ffmpeg has been used there[1]. The fuzz testing they me

• chadillac 13y ago
  I'm not proud of this legacy code, but... it exists because it was a real world issue that would cripple a conversion server in production envs when fed certain files with timing/syncing errors as part of an automated upload and conversion process. When it would crash it would consume 100% of the cores and eat up enough RAM to force swap. This cron has been keeping ffmpeg in check for over 4 years (not 6, whoopsie) in a production environment at this point... it processes thousands of videos a day using a custom queuing and reviewing system. 1 <?php 2 /* 3 ** Cron responsible for detecting failed/hung ffmpeg 4 ** instances and killing them. 5 */ 6 require_once '/lib/class/dbmysqli.php'; 7 8 $output = shell_exec('ps -aeo pid,etime,args | grep ffmpeg | grep -v grep'); 9 10 11 12 13 preg_match_all("/^[ ]{0,}([0-9]*)[ ]{0,}(.*?) .*?([0-9]*?)\-[0-9]*?\.flv /m", $output, $preg_out, PREG_SET_ORDER); 14 15 if (!empty($preg_out)) { 16 17 $db = new DBmysqli('dbmaster'); 18 19 foreach ($preg_out as $process) { 20 $pid = intval($process[1]); 21 $etime = $process[2]; 22 $queue_id = intval($process[3]); 23 24 $etime = intval(str_replace(':','',$etime)); 25 26 // elapsed time >= 60:00 (1 hour) 27 if ($etime >= 6000) { 28 $fail_sql = "DELETE FROM media_upload_queue WHERE queue_id = $queue_id"; 29 shell_exec("kill -9 ".$pid); // kill hung ffmpeg process 30 $db->query($fail_sql); // remove file from DB 31 // debate if life is worth living... 32 } 33 } 34 } 35 ?> Yes I know there are better ways to do this now at the OS level, but it was a quick hack over a half decade ago and continues to work... ain't broke, don't fix it kinda deal.
  • rip747 13y ago
    "I'm not proud of this legacy code," "This cron has been keeping ffmpeg in check for over 6+ years in a production environment at this point... it processes thousands of videos a day using a custom queuing and reviewing system." why do we programmers always feel we need to apologize for something that we did quickly, but has been running without incident for a number of years. take a bow my friend. that was an awesome patch you came up with all those years ago.
    • MBCook 13y ago
      > why do we programmers always feel we need to apologize for something that we did quickly, but has been running without incident for a number of years. The script is only treating the symptom, not the problem. It would be better to detect the files before they waste an hour of time, so you could tell the user instead of having them silently disappear. Maybe there is something you could do to fix the files. The programmer part of me says there is a real fix that needs writing. It's obviously a great script if it's worked that long. Who knows how long it would take to track down and fix the bug(s) causing the issue. If they haven't needed to fix the bug in all these years just writing that script was obviously a good decision.
      • zaroth 13y ago
        > If they haven't needed to fix the bug in all these years just writing that script was obviously a good decision. Exactly this. There are an unbounded number of bugs which will cause this single symptom. I think detecting the symptom is exactly the right solution.
        • kbenson 13y ago
          In theory. In reality, after treating one or two causes, the occurrence rate may fall to once a month or year. I agree that mitigating the problem is a good first step though.
    • burntsushi 13y ago
• tttt1111 13y ago
  Google has never officially acknowledged that YouTube uses ffmpeg. I seem to recall it was because of legal reasons they are not allowed to. So you will never hear a Google employee mention anything about YouTube and ffmpeg ;)
  • dpe82 13y ago
    We do at least know that they use x264: http://x264licensing.com/adopters http://x264licensing.com/adopters