6 ms·

FFmpeg 8.1

438 points by gyan 6mo ago

gyan 6mo ago
Changelog: ffprobe -codec option EXIF Metadata Parsing gfxcapture: Windows.Graphics.Capture based window/monitor capture hxvs demuxer for HXVS/HXVT IP camera format MPEG-H 3D Audio decoding via mpeghdec D3D12 H.264 encoder drawvg filter via libcairo ffmpeg CLI tiled HEIF support D3D12 AV1 encoder ProRes Vulkan hwaccel DPX Vulkan hwaccel Rockchip H.264/HEVC hardware encoder Add vf_scale_d3d12 filter JPEG-XS parser JPEG-XS decoder and encoder through libsvtjpegxs JPEG-XS raw bitstream muxer and demuxer IAMF Projection mode Ambisonic Audio Elements muxing and demuxing Add vf_mestimate_d3d12 filter xHE-AAC Mps212 decoding support (experimental) Remove the old HLS protocol handler Vulkan compute codec optimizations swscale Vulkan support LCEVC metadata bitstream filter Add vf_deinterlace_d3d12 filter ffprobe: only show refs field in stream section when reading frames ProRes Vulkan encoder LCEVC parser LCEVC enhancement layer exporting in MPEG-TS
- yboris 6mo ago
  TIL: JPEG XS - an image and video codec that offers both visually and mathematically lossless quality for low latency implementations. Additionally, JPEG XS compressed content is indistinguishable from the original uncompressed content. https://en.wikipedia.org/wiki/JPEG_XS https://en.wikipedia.org/wiki/JPEG_XS
  - CharlesW 6mo ago
    > Additionally, JPEG XS compressed content is indistinguishable from the original uncompressed content. It can be indistinguishable, as long as you stick with lossless or very low compression ratios. It falls apart at typical JPEG XL compression ratios.
  - iso1631 6mo ago
    We use JXS when latency is critical. Most h24/265 decodes will have a 10 frame glass-glass delay, JXS drops that to 3 or 4, at a cost of bandwidth (our UHD jxs streams are 1.5gbit rather than 200mbit for hevc)
    - jamesfmilne 6mo ago
      Yeah, we've been deploying JPEG-XS for high bitrate streaming for a while. A lot of our customers are moving their grading systems into data centres and streaming the images over IP back to their grading suites. I've got it down to less than 1 frame for encode-transport-decode, but you've still got to copy the image to an SDI card and wait for that to clock out.
    - Dylan16807 6mo ago
      That's pretty depressing to read. x264 was handling the encoding side with sub-frame latency 15 years ago, and sub-frame decoding is significantly easier. "with –tune zerolatency, single-frame VBV, and intra refresh, x264 can achieve end-to-end latency (not including transport) of under 10 milliseconds for an 800×600 video stream" But for some reason you can't make use of that and have to burn bandwidth instead.
      - iso1631 6mo ago
        A small part of the end-to-end process https://www.obe.tv/how-to-lie-about-latency/ https://www.obe.tv/how-to-lie-about-latency/ Bandwidth is cheap -- basically free, especially at this bitrate.
        Dylan16807 6mo ago
        In theory it's a small part. But if you got that many frames of latency difference by changing codec, then it wasn't being a small part. It's not that you should have gotten a magical 10ms latency glass to glass, it's that you should have been able to get 4 frames latency on h.264. But something prevented that, so I'm sad about it. (And if you say the bandwidth was fine in your situation I won't argue, but using more than a gigabit extra is not usually thought of as free.)
  - voidUpdate 6mo ago
    Isn't the point of JPEG to have lossy compression for your photos that still looks fine? As opposed to something like PNG, which has lossless compression
    - pjc50 6mo ago
      Reading that it looks like the point of JPEG-XS is to have near-lossless compression for raw photo and video data while having extremely high throughput.
      - zamadatix 6mo ago
        JPEG XS supports either near lossless or truly lossless encoding depending on encoder configuration.
    - johntb86 6mo ago
      "JPEG" is short for Joint Photographic Experts Group, an ISO/ITU group that creates a lot of imaging standards. The JPEG image format you're thinking of is only one of the formats they've created.
    - bydo 6mo ago
      The Joint Photographic Experts Group manages many standards, generally each called "JPEG [something]". The one we most commonly call "JPEG" is just one of them.
    - deleted 6mo ago
      [deleted]
  - jamesfmilne 6mo ago
    I've had great results using JPEG-XS to transport video for colour grading in feature film & TV post production. At 3:1 or 4:1 compression ratio is effectively lossless. It is patent-encumbered though, you have to pay license fees to deploy it.
  - Almondsetat 6mo ago
    Not royalty free, unfortunately.
- bob1029 6mo ago
  > gfxcapture: Windows.Graphics.Capture based window/monitor capture > This source provides low overhead capture of application windows or entire monitors. The filter outputs hardware frames in d3d11 format; use hwdownload,format= if system memory frames are required. This would strongly alter my plans if I were to develop an OSS Discord alternative. Chromium originally looked like a better core to start with largely due to its mature screen capture API. WebRTC is the other big thing, but there are other ways to do that. Native desktop apps (i.e., not browser based) are beginning to look much more compelling to me now.
  - lostmsu 6mo ago
    You could always use Windows.Graphics.Capture directly.
  - pjc50 6mo ago
    If you were doing this, consider cribbing from https://github.com/obsproject/obs-studio/tree/master/plugins/win-capture https://github.com/obsproject/obs-studio/tree/master/plugins... which offers a variety of solutions including some rather exciting looking process injection (called "game" there). I wonder if "entire chat app functions as OBS plugin" would work? Would solve the AV streaming side of the functionality.
- HelloUsername 6mo ago
  Wait, are you https://www.gyan.dev/ffmpeg/builds/ https://www.gyan.dev/ffmpeg/builds/ ?
- TacticalCoder 6mo ago
  What I want to know is how much of these were written and/or debugged using AI tools and which ones? Using which workflow? For that's an actual project, with countless uses, on countless machines. Show me the AI. I want to see what AI has generated in those. (btw I pay religiously my Claude Code subscription plan)
megous 6mo ago
I wouldn't get too excited about rockchip hw encoding. It's rkmpp based, not an upstream solution. You'd need Rockchip kernel for this, I guess.
- stevemk14ebr 6mo ago
  It's still a big deal, you had to compile ffmpeg yourself before
deleted 6mo ago
[deleted]
pandaforce 6mo ago
Khronos published a post on the Vulkan compute codecs in FFmpeg: https://www.khronos.org/blog/video-encoding-and-decoding-with-vulkan-compute-shaders-in-ffmpeg https://www.khronos.org/blog/video-encoding-and-decoding-wit...
- altairprime 6mo ago
  https://news.ycombinator.com/item?id=47410271 https://news.ycombinator.com/item?id=47410271
- rixrax 6mo ago
  Is there a 'HW' guide to show the expected performance of Vulkan compute codecs anywhere?
ghgr 6mo ago
> Oh there's a new version of ffmpeg, I'll just quickly build it from source... no I can't wait I'll download the binary https://www.youtube.com/watch?v=9kaIXkImCAM https://www.youtube.com/watch?v=9kaIXkImCAM
- asveikau 6mo ago
  I tend to build ffmpeg from source because package managers don't usually include support for patented codecs. (Yes I know there are repos to get binaries for some, things like deb-multimedia.)
- shevy-java 6mo ago
  Building ffmpeg itself from source is actually quite easy. The hardest part IMO is getting the necessary codecs to work; this can take a little while. If you know what audio and video codecs you want and need, and if you get them installed properly, then compiling ffmpeg is really simple and straightforward. It works almost always for me, and I have compiled ffmpeg from source for like +10 or even +15 years. For reference purposes, my current configure options are: ./configure --prefix=/usr/ --enable-gnutls --enable-gpl --enable-libmp3lame --enable-libaom --enable-libopus --enable-libspeex --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-nonfree --enable-pthreads --enable-shared --enable-version3 --extra-libs=\"-ldl\" --disable-doc --disable-libopenjpeg --disable-libpulse --disable-static Probably more codecs could be added, and some options may not be necessary anymore (I changed this last ... years ago, too), but this works for the most part fairly well. One focus I have is mostly on a few .mp4 files, and for these I think you kind of want x264 x265 and so forth (I think one more codec from google too or so). But it is really quite trivial once you are past the codecs step. You can also start simple with just a few codecs, e. g. one good audio codec and one good video codec. One reason I like to have ffmpeg support many codecs is so I can use mpv, which in itself is really awesome; I like it more than vlc, which is also ok though.
  - dylan604 6mo ago
    quite easy? like learning to draw an owl? getting the stock ffmpeg to compile/build might be "easy", but once you start adding on additional codecs and other features that get you into dependency nightmares "easy" is not the word I would use. I have not been able to use the stock ffmpeg since forever. for example, i see no openssl enabled in your config. I see no freetype. I see you've disabled openjpeg. clearly, you and i use ffmpeg differently which just goes to show your "easy" is very misleading
brcmthrowaway 6mo ago
How much of this release was done by to corporate/big tech employees?
- overfeed 6mo ago
  I have no idea how much they contribute back, but pretty much every (big) tech company that does any media transcoding uses ffmpeg.
shevy-java 6mo ago
FFmpeg is really great. The only wish I'd have is for the usage to become simpler - both for regular stuff, but also for advanced filtering. If anyone remembers, avisynth was pretty cool back in the days. You could kind of script video/audio manipulations, a bit like a UNIX/Linux pipe, but kind of simpler, in my opinion. FFmpeg allows for many similar operations, but remembering anything here is ... hard. I'd love for the whole usage API to become much simpler, but it seems nobody among the ffmpeg dev team is considering this. :( I can't be the only one with that wish though ... It does not diminish ffmpeg being so great in general, but I think it could be better.
- daneel_w 6mo ago
  If you're just looking for an easier solution for encoding video ("regular stuff") then Handbrake is the go-to tool for that.
- qingcharles 6mo ago
  One of the best uses of LLMs is to help find the right command line options for tools like ffmpeg.
  - axus 6mo ago
    It won't always give you a perfect answer the first time, but it's much better than memorizing the manual or interpreting a forum discussion. Haven't used it for ffmpeg, but lots of other command lines.
    - sysworld 6mo ago
      I find it helps if you paste in the the ffmpeg manual and get the ai to use that as source. Helps it stick to real params.
    - andrew_lettuce 6mo ago
      Because ffmpeg is built on the Unix chained utility philosophy I find ai is also good at building scripts the use it as well
junelee77 6mo ago
nice job
edgarvaldes 6mo ago
One of the best open-source tools out there. I'm a frequent user of Plex, Jellyfin, Tunarr, local music files, etc. I use it weekly to extract subtitles, trim videos, convert music formats, and remove audio tracks. After writing the previous paragraph, I realized I've never donated to the project; it's time to change that.
- mihaelm 6mo ago
  It's a lower-level component in so much stuff we're not even aware of.
  - cybrox 6mo ago
    FFmpeg is a prime single-block-everything-is-built-on xkcd example.
    - comprev 6mo ago
      It's a sibling to curl in that way
      - Carioca 6mo ago
        Big difference for ffmpeg especially (but I imagine for curl too): it's not just one guy in Nebraska. Seems to have a very healthy community of devs involved in it.
        ldng 6mo ago
        Well, I don't how it is these days, but it hasn't always been describable as "Healthy".
      - 16000198 6mo ago
        I guess you guys have never heard of HarfBuzz, huh?
    - 8cvor6j844qw_d6 6mo ago
deleted 6mo ago
[deleted]
wei03288 6mo ago
[flagged]
jsrcout 6mo ago
Just realized they tag the releases with great names in math/computing. Very cool.
slartibardfast0 6mo ago
i'm in the process of adding bidirectional text to bitmap subtitle conversions with Claude Code: https://connollydavid.github.io/pgs-release/ https://connollydavid.github.io/pgs-release/ such a fun project!
valentinconan 6mo ago
Great, I'll update it. Thanks
skylake73 6mo ago
[dead]

438 points by gyan 6mo ago

gyan 6mo ago

Changelog: ffprobe -codec option EXIF Metadata Parsing gfxcapture: Windows.Graphics.Capture based window/monitor capture hxvs demuxer for HXVS/HXVT IP camera format MPEG-H 3D Audio decoding via mpeghdec D3D12 H.264 encoder drawvg filter via libcairo ffmpeg CLI tiled HEIF support D3D12 AV1 encoder ProRes Vulkan hwaccel DPX Vulkan hwaccel Rockchip H.264/HEVC hardware encoder Add vf_scale_d3d12 filter JPEG-XS parser JPEG-XS decoder and encoder through libsvtjpegxs JPEG-XS raw bitstream muxer and demuxer IAMF Projection mode Ambisonic Audio Elements muxing and demuxing Add vf_mestimate_d3d12 filter xHE-AAC Mps212 decoding support (experimental) Remove the old HLS protocol handler Vulkan compute codec optimizations swscale Vulkan support LCEVC metadata bitstream filter Add vf_deinterlace_d3d12 filter ffprobe: only show refs field in stream section when reading frames ProRes Vulkan encoder LCEVC parser LCEVC enhancement layer exporting in MPEG-TS

yboris 6mo ago
TIL: JPEG XS - an image and video codec that offers both visually and mathematically lossless quality for low latency implementations. Additionally, JPEG XS compressed content is indistinguishable from the original uncompressed content. https://en.wikipedia.org/wiki/JPEG_XS https://en.wikipedia.org/wiki/JPEG_XS
- CharlesW 6mo ago
  > Additionally, JPEG XS compressed content is indistinguishable from the original uncompressed content. It can be indistinguishable, as long as you stick with lossless or very low compression ratios. It falls apart at typical JPEG XL compression ratios.
- iso1631 6mo ago
  We use JXS when latency is critical. Most h24/265 decodes will have a 10 frame glass-glass delay, JXS drops that to 3 or 4, at a cost of bandwidth (our UHD jxs streams are 1.5gbit rather than 200mbit for hevc)
  - jamesfmilne 6mo ago
    Yeah, we've been deploying JPEG-XS for high bitrate streaming for a while. A lot of our customers are moving their grading systems into data centres and streaming the images over IP back to their grading suites. I've got it down to less than 1 frame for encode-transport-decode, but you've still got to copy the image to an SDI card and wait for that to clock out.
  - Dylan16807 6mo ago
    That's pretty depressing to read. x264 was handling the encoding side with sub-frame latency 15 years ago, and sub-frame decoding is significantly easier. "with –tune zerolatency, single-frame VBV, and intra refresh, x264 can achieve end-to-end latency (not including transport) of under 10 milliseconds for an 800×600 video stream" But for some reason you can't make use of that and have to burn bandwidth instead.
    - iso1631 6mo ago
      A small part of the end-to-end process https://www.obe.tv/how-to-lie-about-latency/ https://www.obe.tv/how-to-lie-about-latency/ Bandwidth is cheap -- basically free, especially at this bitrate.
      - Dylan16807 6mo ago
        In theory it's a small part. But if you got that many frames of latency difference by changing codec, then it wasn't being a small part. It's not that you should have gotten a magical 10ms latency glass to glass, it's that you should have been able to get 4 frames latency on h.264. But something prevented that, so I'm sad about it. (And if you say the bandwidth was fine in your situation I won't argue, but using more than a gigabit extra is not usually thought of as free.)
- voidUpdate 6mo ago
  Isn't the point of JPEG to have lossy compression for your photos that still looks fine? As opposed to something like PNG, which has lossless compression
  - pjc50 6mo ago
    Reading that it looks like the point of JPEG-XS is to have near-lossless compression for raw photo and video data while having extremely high throughput.
    - zamadatix 6mo ago
      JPEG XS supports either near lossless or truly lossless encoding depending on encoder configuration.
  - johntb86 6mo ago
    "JPEG" is short for Joint Photographic Experts Group, an ISO/ITU group that creates a lot of imaging standards. The JPEG image format you're thinking of is only one of the formats they've created.
  - bydo 6mo ago
    The Joint Photographic Experts Group manages many standards, generally each called "JPEG [something]". The one we most commonly call "JPEG" is just one of them.
  - deleted 6mo ago
    [deleted]
- jamesfmilne 6mo ago
  I've had great results using JPEG-XS to transport video for colour grading in feature film & TV post production. At 3:1 or 4:1 compression ratio is effectively lossless. It is patent-encumbered though, you have to pay license fees to deploy it.
- Almondsetat 6mo ago
  Not royalty free, unfortunately.
bob1029 6mo ago
> gfxcapture: Windows.Graphics.Capture based window/monitor capture > This source provides low overhead capture of application windows or entire monitors. The filter outputs hardware frames in d3d11 format; use hwdownload,format= if system memory frames are required. This would strongly alter my plans if I were to develop an OSS Discord alternative. Chromium originally looked like a better core to start with largely due to its mature screen capture API. WebRTC is the other big thing, but there are other ways to do that. Native desktop apps (i.e., not browser based) are beginning to look much more compelling to me now.
- lostmsu 6mo ago
  You could always use Windows.Graphics.Capture directly.
- pjc50 6mo ago
  If you were doing this, consider cribbing from https://github.com/obsproject/obs-studio/tree/master/plugins/win-capture https://github.com/obsproject/obs-studio/tree/master/plugins... which offers a variety of solutions including some rather exciting looking process injection (called "game" there). I wonder if "entire chat app functions as OBS plugin" would work? Would solve the AV streaming side of the functionality.
HelloUsername 6mo ago
Wait, are you https://www.gyan.dev/ffmpeg/builds/ https://www.gyan.dev/ffmpeg/builds/ ?
TacticalCoder 6mo ago
What I want to know is how much of these were written and/or debugged using AI tools and which ones? Using which workflow? For that's an actual project, with countless uses, on countless machines. Show me the AI. I want to see what AI has generated in those. (btw I pay religiously my Claude Code subscription plan)

megous 6mo ago

I wouldn't get too excited about rockchip hw encoding. It's rkmpp based, not an upstream solution. You'd need Rockchip kernel for this, I guess.

stevemk14ebr 6mo ago
It's still a big deal, you had to compile ffmpeg yourself before

deleted 6mo ago

[deleted]

pandaforce 6mo ago

Khronos published a post on the Vulkan compute codecs in FFmpeg: https://www.khronos.org/blog/video-encoding-and-decoding-with-vulkan-compute-shaders-in-ffmpeg https://www.khronos.org/blog/video-encoding-and-decoding-wit...

altairprime 6mo ago
https://news.ycombinator.com/item?id=47410271 https://news.ycombinator.com/item?id=47410271
rixrax 6mo ago
Is there a 'HW' guide to show the expected performance of Vulkan compute codecs anywhere?

ghgr 6mo ago

> Oh there's a new version of ffmpeg, I'll just quickly build it from source... no I can't wait I'll download the binary https://www.youtube.com/watch?v=9kaIXkImCAM https://www.youtube.com/watch?v=9kaIXkImCAM

asveikau 6mo ago
I tend to build ffmpeg from source because package managers don't usually include support for patented codecs. (Yes I know there are repos to get binaries for some, things like deb-multimedia.)
shevy-java 6mo ago
Building ffmpeg itself from source is actually quite easy. The hardest part IMO is getting the necessary codecs to work; this can take a little while. If you know what audio and video codecs you want and need, and if you get them installed properly, then compiling ffmpeg is really simple and straightforward. It works almost always for me, and I have compiled ffmpeg from source for like +10 or even +15 years. For reference purposes, my current configure options are: ./configure --prefix=/usr/ --enable-gnutls --enable-gpl --enable-libmp3lame --enable-libaom --enable-libopus --enable-libspeex --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-nonfree --enable-pthreads --enable-shared --enable-version3 --extra-libs=\"-ldl\" --disable-doc --disable-libopenjpeg --disable-libpulse --disable-static Probably more codecs could be added, and some options may not be necessary anymore (I changed this last ... years ago, too), but this works for the most part fairly well. One focus I have is mostly on a few .mp4 files, and for these I think you kind of want x264 x265 and so forth (I think one more codec from google too or so). But it is really quite trivial once you are past the codecs step. You can also start simple with just a few codecs, e. g. one good audio codec and one good video codec. One reason I like to have ffmpeg support many codecs is so I can use mpv, which in itself is really awesome; I like it more than vlc, which is also ok though.
- dylan604 6mo ago
  quite easy? like learning to draw an owl? getting the stock ffmpeg to compile/build might be "easy", but once you start adding on additional codecs and other features that get you into dependency nightmares "easy" is not the word I would use. I have not been able to use the stock ffmpeg since forever. for example, i see no openssl enabled in your config. I see no freetype. I see you've disabled openjpeg. clearly, you and i use ffmpeg differently which just goes to show your "easy" is very misleading

brcmthrowaway 6mo ago

How much of this release was done by to corporate/big tech employees?

overfeed 6mo ago
I have no idea how much they contribute back, but pretty much every (big) tech company that does any media transcoding uses ffmpeg.

shevy-java 6mo ago

FFmpeg is really great. The only wish I'd have is for the usage to become simpler - both for regular stuff, but also for advanced filtering. If anyone remembers, avisynth was pretty cool back in the days. You could kind of script video/audio manipulations, a bit like a UNIX/Linux pipe, but kind of simpler, in my opinion. FFmpeg allows for many similar operations, but remembering anything here is ... hard. I'd love for the whole usage API to become much simpler, but it seems nobody among the ffmpeg dev team is considering this. :( I can't be the only one with that wish though ... It does not diminish ffmpeg being so great in general, but I think it could be better.

daneel_w 6mo ago
If you're just looking for an easier solution for encoding video ("regular stuff") then Handbrake is the go-to tool for that.
qingcharles 6mo ago
One of the best uses of LLMs is to help find the right command line options for tools like ffmpeg.
- axus 6mo ago
  It won't always give you a perfect answer the first time, but it's much better than memorizing the manual or interpreting a forum discussion. Haven't used it for ffmpeg, but lots of other command lines.
  - sysworld 6mo ago
    I find it helps if you paste in the the ffmpeg manual and get the ai to use that as source. Helps it stick to real params.
  - andrew_lettuce 6mo ago
    Because ffmpeg is built on the Unix chained utility philosophy I find ai is also good at building scripts the use it as well

junelee77 6mo ago

nice job

edgarvaldes 6mo ago

One of the best open-source tools out there. I'm a frequent user of Plex, Jellyfin, Tunarr, local music files, etc. I use it weekly to extract subtitles, trim videos, convert music formats, and remove audio tracks. After writing the previous paragraph, I realized I've never donated to the project; it's time to change that.

mihaelm 6mo ago
It's a lower-level component in so much stuff we're not even aware of.
- cybrox 6mo ago
  FFmpeg is a prime single-block-everything-is-built-on xkcd example.
  - comprev 6mo ago
    It's a sibling to curl in that way
    - Carioca 6mo ago
      Big difference for ffmpeg especially (but I imagine for curl too): it's not just one guy in Nebraska. Seems to have a very healthy community of devs involved in it.
      - ldng 6mo ago
        Well, I don't how it is these days, but it hasn't always been describable as "Healthy".
    - 16000198 6mo ago
      I guess you guys have never heard of HarfBuzz, huh?
  - 8cvor6j844qw_d6 6mo ago

deleted 6mo ago

[deleted]

wei03288 6mo ago

[flagged]

jsrcout 6mo ago

Just realized they tag the releases with great names in math/computing. Very cool.

slartibardfast0 6mo ago

i'm in the process of adding bidirectional text to bitmap subtitle conversions with Claude Code: https://connollydavid.github.io/pgs-release/ https://connollydavid.github.io/pgs-release/ such a fun project!

valentinconan 6mo ago

Great, I'll update it. Thanks

skylake73 6mo ago

[dead]