6 ms·
Twitter's GIF hack
- deleted 12y ago[deleted]
- eph_unit 12y agoSo, what is an easy way to convert gifs to mp4? How did they do this. I run a small gif-sharing site and should probably do this.
- ushi 12y agoimagemagick + ffmpeg https://stackoverflow.com/questions/3212821/animated-gif-to-avi-on-linux/3212958#3212958 https://stackoverflow.com/questions/3212821/animated-gif-to-...
- kwntm 12y agoCheck out http://www.gfycat.com/about http://www.gfycat.com/about
- jdiez17 12y agoI made a website that does this, and a lot more. It's also open source, and has been around for a while. https://mediacru.sh https://mediacru.sh
- GHFigs 12y agoI had a convoluted method using gifsicle to extract the frames and then encode as a stream of frames, but then sometime in the past year or so ffmpeg improved native gif support. This is what I use (the options are both for compatibility): ffmpeg -i foo.gif -pix_fmt yuv420p -vf crop=floor (in_w/2)*2:floor(in_h/2)*2 foo.mp4 Works for WebM just by changing the extension, too.
- TazeTSchnitzel 12y agoMuch like what http://gfycat.com/ http://gfycat.com/ do, then? Interesting also that 4chan recently added looping, muted WebM videos for similar reasons.
- brickmort 12y agoI think now would be a good time to introduce HTML5 videos to the feed, or gfycats. At least on certain browsers.
- raverbashing 12y agoI had noticed that. Because of flashblock. Yeah, a 20+ years file format doesn't do compression between frames, who would have thought...
- ars 12y agoExcept that it does do that, the article is wrong. As I wrote in a different comment the actual reason is lossy jpeg compression, not intra frame compression.
- raverbashing 12y agoAah you're right. Yeah, for pictures jpeg is much better (and smaller) than gif. Not to mention in several gifs they really reduce the quality to have a smaller file size
- stonogo 12y agoThis is on a fully-patched RHEL6 workstation: http://i.imgur.com/NleqJTD.png http://i.imgur.com/NleqJTD.png Another example of "use bleeding-edge tech or go fuck yourself" from the modern web.
- fixermark 12y agoSupport for mp4 isn't bleeding-edge tech on other operating systems / distributions, unfortunately. It's a couple of years old in terms of being supported by Chrome; it's supported in Firefox as of last August. At this point, desktop configurations that can't play mp4s are at risk of being considered "broken."
- callahad 12y agoIs it really that difficult to supply a free format like WebM in addition to patent-encumbered H.264? If you only supply H.264, you're cutting out Firefox on OS X, Chromium, and Opera.
- garblegarble 12y agoIt does mean you need to transcode and store twice as many files, which can be a serious pain if you've got a number of different bitrate H.264 assets. It can be a big pain if your software assumes there'll only be a single media file. Not insurmountable but painful for small operations (and for large operations, where there may be a back-catalogue to worry about) I'm with you in spirit, but in practice I think we can all understand why people often just go for H.264
- kibibu 12y agoThis is a new feature launched by one of the biggest web brands there is, so presumably theres no back catalogue.
- xooyoozoo 12y agoInterestingly, the gif->mp4 in the source link uses Constrained Baseline H264, the only profile that's supported by Cisco's BSD-licensed OpenH264[1]. The MPEG-LA patent fees are already covered, so that small projects can freely use the decoder/encoder. [1] https://github.com/cisco/openh264 https://github.com/cisco/openh264
- bluedino 12y agoHaven't things like gfyCat been doing HTML5 video from GIF's for a while?
- discreditable 12y agoIn a way they're doing it better. Twitter only seems to support MP4 whereas gfycat et al will deliver WEBM or MP4 based on the user's browser.
- skrowl 12y agoAgreed. MP4 doesn't work on Firefox (out of the box).
- pyre 12y agoYou should probably qualify that somehow, because MP4 works for me on Firefox (on OSX 10.9).
- preek 12y agoIt does not work for me, too. FF30, OSX 10.9.3. https://www.evernote.com/shard/s19/sh/e81e070a-4e2e-4367-9eb1-83a7f6fe151f/42755f0246227bd75eb5e1099f857348/deep/0/About-This-Mac-and-Embedly-Blog,-What-Twitter-Isn't-Telling-You-About-GIFs---Vimperator.png https://www.evernote.com/shard/s19/sh/e81e070a-4e2e-4367-9eb...
- pyre 12y agoOSX 10.9.2 + FF30 here: http://i.imgur.com/uTlQkg7.jpg http://i.imgur.com/uTlQkg7.jpg
- DCKing 12y agoRight click the video. It's probably Flash. Plays using Flash in Firefox on my machine. OS X is the last platform where Firefox doesn't support H.264. They're working on it, but it doesn't seem to be landing soon.
- ars 12y agoThis is not true: "A GIF is literally a sequence of independent images squeezed into the same file. An mp4 video can take advantage of all kinds of fancy compression techniques like keyframes and forward-predictive frames." That's not why the mp4 is smaller. A GIF is not [just] independent images, you can have each frame reuse pixels from previous frames. The mp4 is smaller because lossy jpeg compression is used on each frame, while the gif stores each frames losslessly. This has implications from the predictive frames as well - the jpeg is lossy, so two almost similar parts of the images can be considered "the same", while the gif is not, so they must be identical in order to compress that way, and the realities of video noise makes that unlikely.
- underwater 12y agoGIF can have individual pixels that are preserved frame to frame, but an MP4 can do much more. A GIF is lossy because you lose color information.
- Kayou 12y agoSorry but H264 does not use JPEG for its keyframes, although it's not very different. There are a lot of differences between GIF and H264, like H264 specifies that you can use multiple reference frames, which is not the case with Animated GIF. Also H264 has motion compensation, which does not exist with GIF. GIF basically rewrites completely a part of the picture, while the remaining part is reused. And if you consider that a rewrite of a small part of the frame to reuse the remaining part from the previous frame is just displaying a new gif on top of a part of the old one, the you can consider that it is a sequence of independent images. Maybe they don't all have the same size and are not meant to be displayed at the same position, but they are independent in the sense that you don't need a previous or future frame to display one frame. And I wouldn't refer to GIF as lossless, except maybe if your original animation used only 256 simultaneous colors per frame. You lose information by down-sampling your colors.
- anon4 12y agoNot quite true yourself :). GIFs store deltas, true, but they are still raster images. H.264 on the other hand contains specific techniques for compressing sequential frames, including things like recording only pixel motion for some frames. You can see it in those "corrupt movies" gifs (which on twitter would be mp4s... talk about irony) where you take a clip and remove some keyframes from it - then transformations are applied on the wrong blocks and you get really weird things like someone's head splitting open and stuff flowing into each other. Additionally keyframes in movies are usually compressed with even more loss than standard jpegs. In most cases only the Y channel is recorded at full resolution and the two colour channels - U and V - are recorded at half. There is also motion-jpeg which really is literally a sequence of jpeg images.
- malkia 12y agoI would think it's to avoid any traces of steganography... naaah I'm just kidding....
- billmalarky 12y ago"Videos allow Twitter to leverage the browser. This means play, pause, and seek, but also cool things like slow-motion. Think your GIF is funny now? Wait til you see it in slow motion." Gifs have built in slow motion, it's called waiting for the 10mb file to download frame by frame upon initial play-through.
- lawl 12y agoSo. Who was first. 4chan or twitter? How long has twitter been doing this? Just out of curiousity. (Yes I know 4chan doesn't convert but there was some talk that 4chan might be a major player in pushing webm forward. Is it even webm or x264? On mobile. Can't check right now.)
- underwater 12y agogfycat.com and others have been doing this for a while.
- valarauca1 12y ago4chan pushed webm before twitter. gifcat (or gyfcat idk) was before 4chan. github beat gifcat. Basically twitter is doing what gifcat does, which has been doing it for a few years.
- protopete 12y agoUnfortunately the MP4 looks worse than the GIF, due to chroma compression in the YUV 420 colorspace. While each pixels luminance value is kept, the color information for a 4-pixel square is stored as a single CrCb pair, which is really obvious when you look at how the orange hat has artifacts against the blue background. Increasing the bitrate won't solve this either, since it's a limitation of the colorspace.
- vernie 12y agoIt's actually 4:2:0
- protopete 12y agoThanks, updated.
- ars 12y ago> Increasing the bitrate won't solve this either, since it's a limitation of the colorspace. You can encode at 4:4:4 if you want, although obviously you have no control over what twitter does.
- ferongr 12y agoOr alternatively browsers could use non-terrible chroma upsampling algorithms. http://screenshotcomparison.com/comparison/79805 http://screenshotcomparison.com/comparison/79805
- Kayou 12y agoThank you! I was gonna post the exact same thing. Nice website, though. I encoded the gif in 420 (subsampling), 422 and 444, having a nice quality with 444 with VLC, I embed the three different videos in a web page and when playing in my browser, they are all as bad as the twitter video! I thought that Firefox was using a standard library to play MP4?
- ferongr 12y ago
- RRRA 12y agoThe saddest part is that you can't add a zip to that gif and use twitter as a sharing platform. cat x.zip >> y.gif and upload would have been nice ;)
- shawnz 12y agoCombined with twitter's link shortening, you could use this instead: https://code.google.com/p/furl/ https://code.google.com/p/furl/
- anon4 12y agoEncode as image frames with enough redundancy to survive mp4 compression
- abemassry 12y agoNot exactly what you're looking for but it's similar https://wsend.net https://wsend.net
- Permit 12y agoI'm curious if anyone is noticed the flickering Loading icon when the .mp4 version loops? It only shows up for 50 ms, but seems kind of annoying. Or maybe it's just my machine? Here's a cap: http://imgur.com/2R84ImD http://imgur.com/2R84ImD
- Drakim 12y agoYeah, I see it too. Very annoying.
- mhurron 12y agoAlso annoying to have to allow javascript to see what is supposed to be an image.
- kremlin 12y agoI noticed it once, and then I kept watching and couldn't see it again so I thought I was just imagining it.
- TD-Linux 12y agoI saw it too. The odd thing is that 4chan's equivalent doesn't show any such loading indicator. Also, looping a video in HTML5 doesn't require Javascript.
- jgmmo 12y agoI saw it everytime the video looped. Left a distinct impression with me that the real GIF version is better.
- nixme 12y agoIf you right-click on that, is it a Flash player? Are you on Firefox?
- Permit 12y agoNope, it's an HTML video element. I'm on Chrome 35.
- 12y ago
- cbr 12y agoAnother option (for supporting browsers) is to use animated webp: https://developers.google.com/speed/webp/faq#why_should_i_use_animated_webp https://developers.google.com/speed/webp/faq#why_should_i_us...
- ahassan 12y agoThis is why 4chan supports WebM videos now in addition to the traditional gifs.
- johnnymonster 12y agoDoes anyone know what they are doing for the conversion? Any services out there do this?
- rudog 12y agoWe've been doing that at my company on our mobile applications for a few years now. There is no reason to be shoving 5-10mb GIFs down a users throat on a mobile device. Not only does are MP4s smaller (file size), frames per second will be constant on all browsers, unlike GIF (although most browsers are supporting up to 50fps these days). Although outdated, you can read more about that here. http://nullsleep.tumblr.com/post/16524517190/animated-gif-minimum-frame-delay-browser-compatibility http://nullsleep.tumblr.com/post/16524517190/animated-gif-mi... I wrote a simple js lib for detecting the duration of a GIF for each browser. It was a fun weekend project. https://github.com/rfrench/gify https://github.com/rfrench/gify If webp gains more adoption, it's another alternative now that it supports animation.
- gdulli 12y agoIs there a flash blocker solution that removes all traces of the page element rather than leave an ugly placeholder? It's going to be annoying to see this on Twitter from now on.
- kudu 12y agoThis article makes it sound like Twitter invented a whole new optimization, when they really just recycled a concept already in use by gfycat (http://www.gfycat.com/ http://www.gfycat.com/) and MediaCrush (https://mediacru.sh/ https://mediacru.sh/).
- Sir_Cmpwn 12y agoThe latter being open source (I made it): https://github.com/MediaCrush/MediaCrush https://github.com/MediaCrush/MediaCrush
- Sir_Cmpwn 12y agoI made a site that drove this innovation, MediaCrush. It's open source. https://github.com/MediaCrush/MediaCrush https://github.com/MediaCrush/MediaCrush if you want to try to do this yourself, our code is a good reference. https://mediacru.sh https://mediacru.sh
- sillysaurus3 12y agoWhy did Gfycat win? I've always wondered.
- Sir_Cmpwn 12y agoThey have a cooler name, I guess. It's not like MediaCrush isn't around, though.
- mkishi 12y agoI'd bet on UI and marketing. The player is very unobtrusive and minimalist, and simple things like pronounceable random names probably appeal to the user. Also, features that are well-known to heavy reddit users (such as easily resizeable videos) only helped with the word-of-mouth.
- notatoad 12y agoGfycat was promoted pretty agressively on reddit. I've never heard of MediaCrush before this thread.
- cookiecaper 12y agoI think it's probably too early to call Gfycat the "winner", and I don't necessarily see why there isn't room for multiple providers. There's no reason image sharing needs to be a "winner-take-all" game; in fact, it behooves everyone if there is a bit of diversity in the space. I understand that reddit (the organization, not only the community) likes to aggressively favor single image sharing vendors, and that sucks, but surely there are other places where such a site can find usage.
- neotoy 12y agoHopefully this will contribute to the extinction of gifs.
- Rudism 12y agoCorrect me if I'm wrong, but wasn't the whole reason gifs started proliferating on the net because people wanted to share short video loops, but didn't want to embed flash or video which may not load or play correctly in a user's browser? And now we are championing the conversion of those gifs back into video? Seems like a strange round-about way of doing things. I forsee a future where all video clips on the net have been converted back and forth between gif and video so often that they all slowly merge into a single amorphous blob of greyish-brown pixels. Actually, now that I think about it, that would almost certainly be an improvement over the current situation.
- Steer 12y agoI don't disagree with you, but isn't it reasonable to think that browsers now can play most videos without any issues?
- lotyrin 12y agoYes, in ancient history, video codecs sucked, browsers sucked, and the only way to play video was through plugins that sucked. Now, all the browsers are pretty good at dealing with video, the codecs are hugely improved (although, there's some headaches with IP) and the plugin is dead. Nobody ever actually wanted gifs. They want short, small videos that are guaranteed to work in their browser and that they can be pretty sure aren't going to have sound. We can now deliver that.
- roc 12y agoTo be fair: they don't want short, small videos any more than they specifically wanted gifs. They want moving images that load quickly. Which has meant short/small/gifs, because most US bandwidth still sucks. And thanks to the mobile explosion it now means short/small/h264. But I bet that as people become used to 5-10x space savings from h264, they'll quickly respond with videos having 5-10x more data. (larger/longer/higher-res clips)
- ObnoxiousJul 12y agowell, it loads quickly because contents are provided according to your user agent and black magic with js pinging your nearest best CDN possible near you and with optimal bandwidth, and then, according to a complex choice in a matrix of supported codec(your SW)/compression(CPU)/bandwidth(the net) a video is chosen through painful trial and errors, where every choices in the matrix (growing at the power of 3 and more) is almost made by humans. The "progress" we could benefits from these new techniques is so complicated and resource greedy and provoking a lock in of customers that I don't think it far outweigh the cost of the poor compression/control of the gif.
- goblin89 12y agoFor me the main lesson here is that I should be more attentive and investigate. I remember noticing some ungiffy vibe about animated clips on Twitter, but quickly dismissing the observation. Perhaps if my timeline contained a lot of GIFs I'd dig deeper, but it doesn't help feeling shallow.
- jstalin 12y agoIt's annoying to an oldtimer, like me, that animated gifs have now simply become gifs.
- tomphoolery 12y agoattn: reddit, please do this
- shmerl 12y agoGIFs indeed are bad for video (quality, size, no control, no sound, etc.) but in order for that to be really true, the site must allow embedding videos using the video tag (i.e. videos which people upload in proper video formats). Does Twitter allow it?
- pyrocat 12y agoThe twitter "GIF" is a black box. gg
- webXL 12y agoThis is an automated fix. Animated GIFs are the hack!
- cosmeen 12y agoThis is what Gfycat http://gfycat.com/ http://gfycat.com/ is been doing since last year but as a service.
- supercoder 12y agoCan I skip the GIF and have Twitter just play an MP4 I upload ?
- megablast 12y agoThis is what I see on the page: http://i.imgur.com/wFjEY8t.jpg http://i.imgur.com/wFjEY8t.jpg I can understand why twitter is saving money on hosting these new "gifs"
- zobzu 12y agoits like many other sites do it and its much better that way
- user24 12y agoGreat, and we've lost the ability to save them to your computer, and to share them outside of twitter's ecosystem.