5 ms·

Is it open source or are there plans to make it so? I'd be very interested how you do frame-accurate video annotations in the browser, as I found the currentTim

by mzur 4y ago

Is it open source or are there plans to make it so? I'd be very interested how you do frame-accurate video annotations in the browser, as I found the currentTime attribute of HTML video elements to be too unreliable for this. (Context: I maintain a video annotation tool that is mostly used for marine imaging.)

bjano 4y ago
In Chrome and Edge you can use WebCodecs to decode and display video frames one-by-one. (this is what I used for the video editor at https://vidmix.app https://vidmix.app ) In other browsers you could build FFmpeg with webassembly and use that for frame-by-frame decoding but it's not going to be nearly as performant.
- mzur 4y ago
  Decoding the video manually seems so much overkill when all that's needed would essentially be a currentFrame property. But it seems I have to explore this avenue as well. I only aim to support Chrome and Firefox but as much as I hate to admit it, Firefox is somewhat lacking on the video side anyway (as there is a bug that lets it display different video frames for the same currentTime than other browsers or FFmpeg [1]). Therefore I already discourage using Firefox for video annotation. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1735300 https://bugzilla.mozilla.org/show_bug.cgi?id=1735300
  - bjano 4y ago
    Yes, the shortcomings of the html video tag are quite unfortunate. Just some very small improvements would make it a lot more useful. One other issue I found at the time I was looking at it (not sure if it's still the case) was that stepping one frame forward wasn't any faster than seeking from a random location. Seemed like the video was decoded from the last keyframe every time, making it inefficient to iterate through the frames.
SpectralName 4y ago
> Context: I maintain a video annotation tool that is mostly used for marine imaging. Which one, out of curiosity? I'm just starting to look into this space and interested in knowing what's out there.
- mzur 4y ago
  BIIGLE (https://biigle.de https://biigle.de).
zackoverflow 4y ago
For Anno we are using HTML video elements / YouTube embeds which have the same frame inaccuracy from currentTime. For our editor we have a WebGL-based solution that seems to get better frame granularity but still has some floating point rounding errors converting frames <-> seconds There's also requestVideoFrameCallback() and seekToNextFrame() in the Web APIs but they are still experimental/not supported by all browser. The WebCodecs API is also experimental/not supported to all browsers but would allow you to decode and grab the individual frames from a video and draw them to a canvas
deleted 4y ago
[deleted]

by mzur 4y ago

bjano 4y ago

In Chrome and Edge you can use WebCodecs to decode and display video frames one-by-one. (this is what I used for the video editor at https://vidmix.app https://vidmix.app ) In other browsers you could build FFmpeg with webassembly and use that for frame-by-frame decoding but it's not going to be nearly as performant.

mzur 4y ago
Decoding the video manually seems so much overkill when all that's needed would essentially be a currentFrame property. But it seems I have to explore this avenue as well. I only aim to support Chrome and Firefox but as much as I hate to admit it, Firefox is somewhat lacking on the video side anyway (as there is a bug that lets it display different video frames for the same currentTime than other browsers or FFmpeg [1]). Therefore I already discourage using Firefox for video annotation. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1735300 https://bugzilla.mozilla.org/show_bug.cgi?id=1735300
- bjano 4y ago
  Yes, the shortcomings of the html video tag are quite unfortunate. Just some very small improvements would make it a lot more useful. One other issue I found at the time I was looking at it (not sure if it's still the case) was that stepping one frame forward wasn't any faster than seeking from a random location. Seemed like the video was decoded from the last keyframe every time, making it inefficient to iterate through the frames.

SpectralName 4y ago

> Context: I maintain a video annotation tool that is mostly used for marine imaging. Which one, out of curiosity? I'm just starting to look into this space and interested in knowing what's out there.

mzur 4y ago
BIIGLE (https://biigle.de https://biigle.de).

zackoverflow 4y ago

For Anno we are using HTML video elements / YouTube embeds which have the same frame inaccuracy from currentTime. For our editor we have a WebGL-based solution that seems to get better frame granularity but still has some floating point rounding errors converting frames <-> seconds There's also requestVideoFrameCallback() and seekToNextFrame() in the Web APIs but they are still experimental/not supported by all browser. The WebCodecs API is also experimental/not supported to all browsers but would allow you to decode and grab the individual frames from a video and draw them to a canvas

deleted 4y ago

[deleted]