Video time vs. real time & frame retrieval for multiple videos

ELAN
1

Hello,
I would like to understand better how ELAN controls playback / retrieval of image for a specific frame and how this works for multiple videos with different frame rates.
I read in the form that ELAN internally works with timestamps and going to next frame is based on the hypothesized frame rate. Furthermore, I read that the used media framework retrieves commands such as “Play” or “Pause”.

Are following assumptions correct:

  1. When pressing play, ELAN will initiate the playback of all opened video files starting at the timestamp selected in the timeline. During playback, there will be no re-synchronization but it is assumed that both videos play back without interruptions.
  2. The called media framework does not know that both videos are to be played synchronously (that is, it only retrieves n tasks for playback for the n video files at hand.
  3. When resuming playback or making a new selection in the timeline before playback, ELAN does the same as described in 1. (instead of simply sending a “resume” to the media framework).
  4. When jumping to the next frame via the control below the video, ELAN calculates the hypothesized timestamp of that frame based on the master video’s frame rate and uses that timestamp to retrieve images from the media framework. For instance, when jumping one frame forward from frame 999 with a master video frame rate of 30, ELAN calculates (999+1)/30=33.333s. For the other media, the image of the frame at that time will be retrieved, that is, they do not necessarily proceed by one frame if the frame rate is another one.
  5. We can assume that the underlying media framework will interpret the timestamp input as video time. That is, the time displayed in a video player which does not necessarily match real time for some (non drop frame) frame rates.

The reason why I ask these questions is that I would like to understand the implications of slightly different frame rates among videos. I have read that especially for the non-integer frame rates (29.97, 59.94 etc.) with non drop frame time code, video time deviates from real time (reason: for 29.97 FPS, the video time is still increased by 1 second after 30 frames, even though one real time second has already passed after 29.97 frames).

Let me define an example where following media files with recordings of the same scene at the same time are opened in an ELAN instance:

Video 1 (30.0 FPS) non drop frame
Video 2 (29.97 FPS) non drop frame

Used time formats: HH:mm:ss.SSS (HH:mm:ss:ff) where SSS are milliseconds, ff is a timecode frame number

If all 5 assumptions are true, I would expect for a recording:

  1. If the videos are played back in ELAN from start without any interaction, after 01:00:00.000 (1 hour) real time, the presented frames would be the ones displayed in a media player at:
Video 1: 01:00:00:00
Video 2: 00:59:56:12 (00:59:56.400)

However, the videos would stay synchronized in playback (given that the media framework is not interrupted for one of the videos).

  1. If I select the time 01:00:00.000 in the ELAN timeline and press “Play”, due to the video time of both videos being set to that time, the frames would be the ones displayed after 1 hour of video time, which equates to following real time:
Video 1: 01:00:00:00
Video 2: 01:00:03:18 (01:00:03.604)

That in turn would mean that the playback is no more synchronized.

Are these thoughts correct? If not, could you please clarify where ELAN behaves differently or where I draw incorrect conclusions?

Thank you for your help!
Best, ASE

2

Hello ASE,

Thank you for your detailed description.
First concerning the 5 assumptions:

  1. Yes
  2. Yes
  3. Yes, I guess. Resuming playback is also just a “Play” command to all players. When a player is paused, e.g. at the end of playing a selection, it can be that the “Pause” is followed by a command “Set Media Time”, depending on media framework and a preference setting. When making a new selection this also results in one or more “Set Media Time” commands. Therefore, the players should already be at the specific timestamp when resuming playback. (“resume” is just “play”, but maybe I don’t understand the “simply sending a ‘resume’ to the media framework” remark correctly).
  4. Yes, indeed, it can be that the second video jumps more (or less) than one frame, especially if the frame rates are very different.
  5. Yes, I think.

Whether your expectations following these assumptions are correct, I don’t know. I’m not sure if I have any non drop frame videos available, but certainly not one of an hour length. So I can’t really check unfortunately, but I could imagine that a supporting media framework could recalculate a request to jump to timestamp ‘1 hour’ to the same frame as would be reached by playback of one hour. But I’m not a specialist in these matters. If I understand correctly drop frame was invented to correct this kind of issues, so are there many non drop frame videos around?
Sorry for not having a clear answer at the moment.

Best,
Han