Printable Version of Topic Click here to view this topic in its original format

Unofficial VirtualDub Support Forums > Testing / Bug Reports > Audio/video Sync Problem

Posted by: jktla Mar 11 2003, 08:23 AM

I have captured MPEG2 video with MPEG2 audio, and I open it with VirtualDub, and when I cut off some part of video, and then convert it to DIVX audio does not sync with video.I convert audio to MP3 format. If I don't cut anything off, sync is OK. Anyone else has this same problem?

Posted by: ChristianHJW Mar 11 2003, 11:16 PM

Can you tell us how you can open MPEG2 in Virtualdub ? Or are you using VirtualdubMod or fcchandler's mod ?

Posted by: jktla Mar 12 2003, 05:47 AM

This forum is for VirtualdubMod so I use VirtualdubMod.

Posted by: fccHandler Mar 12 2003, 06:07 AM

Try http://home.attbi.com/~blade66 of VirtualDub.

Posted by: jktla Mar 17 2003, 08:42 AM

Your version had same problem. Video length I captured was 1 hour 30 minutes. At the beginning of video audio/video sync is Ok, but at end of recorded video audio comes about 0,2 seconds before video. What is the reason for this sync problem? Is there any way to fix it?

Posted by: fccHandler Mar 17 2003, 09:21 AM

Honestly, I'm not sure. But did you say that there were no sync problems when you encoded to DivX without cutting the MPEG? How about encoding the whole thing to DivX (with uncompressed audio) and then cutting up the DivX file? (Compress the audio after cutting.) Also, have you tried under Video->Frame rate, "change so video and audio durations match"?

Posted by: jktla Mar 17 2003, 09:55 AM

Reason why I don't encode whole video to Divx is that I can only cut from key frame to key frame in Divx video. Under Video->Frame rate, "change so video and audio durations match", I have read that VirtualDub adds extra frames to video and do nothing to audio.I would like to keep frame rate to 25 fps , so don't want to use that. It could be that your program does not cut audio from same place that video, and that causes the sync problem.

Posted by: fccHandler Mar 17 2003, 04:36 PM

QUOTE (jktla @ Mar 17 2003, 05:55 AM) Under Video->Frame rate, "change so video and audio durations match", I have read that VirtualDub adds extra frames to video and do nothing to audio.I would like to keep frame rate to 25 fps , so don't want to use that. VirtualDub doesn't add extra frames. Only the frame rate is changed. Anyway, if you're encoding to AVI to watch on a PC then the actual frame rate doesn't matter as long as it's in sync. QUOTE It could be that your program does not cut audio from same place that video, and that causes the sync problem. I've participated in other discussions about out-of-sync MPEG captures, and generally the capture program is to blame (especially ATI software).

Posted by: jktla Mar 18 2003, 12:23 PM

QUOTE VirtualDub doesn't add extra frames. Only the frame rate is changed. Anyway, if you're encoding to AVI to watch on a PC then the actual frame rate doesn't matter as long as it's in sync. I watch on TV. QUOTE I've participated in other discussions about out-of-sync MPEG captures, and generally the capture program is to blame (especially ATI software). When I watch original captured video sync is Ok. Could it be that conversion from MPEG2 audio to MP3 causes the sync problem?

Posted by: fccHandler Mar 18 2003, 04:10 PM

QUOTE (jktla @ Mar 18 2003, 08:23 AM) Could it be that conversion from MPEG2 audio to MP3 causes the sync problem? That's easy to test. Set audio to "direct stream copy" and save an AVI. I know the LAME MP3 codec can cause sync problems.

Posted by: jktla Mar 19 2003, 05:55 AM

I tested saving audio to wav and then I encoded it to MP3 and then I combined audio and video with virtualdub and same sync problem still exists. Am I the only one who has this problem?

Posted by: smball Mar 20 2003, 06:26 AM

This problem is usualy happen at capture mpeg2 file. because it was be save at varient FPS. (27.96 ~ 32.14) but its mpeg header is say 29.96 FPS. that why when you encode capture mpeg2 file will cause audio out of sync, but direct play this mpeg file(not encode) is fine. solution have many way 1. use vtdb change frame rate. its not reasonable but can slove some case. 2. correct the header info i search this forum for best solution. may fccHandler will summary and announce.

Posted by: fccHandler Mar 20 2003, 06:45 AM

QUOTE (jktla @ Mar 19 2003, 01:55 AM) Am I the only one who has this problem? I guess I'm lucky, because I've never had an out-of-sync MPEG from one of my own captures, but you are not alone. Similar problems have been reported in these forums and Doom9's forum: http://forum.doom9.org/showthread.php?s=&threadid=44897 http://forum.doom9.org/showthread.php?s=&threadid=48571 http://virtualdub.everwicked.com/index.php?act=ST&f=4&t=2452&hl=&s=b35180eefe180271dd1082204c4bb5f0 I still say the capture software (or hardware) is to blame. As Avery Lee said, "it's what we get for putting audio and video capture on separate clocks."

Posted by: Pamel Mar 22 2003, 06:51 PM

************************ Edit: Avery has alerted me(everyone) to the fact that this post is complete BS. In his words, "So please stop spreading this crap. MPEG does not support VFR, and AVI does not have the sync problem you describe." I am going to just leave this post here for the sake of completeness, but you are probably better off skipping the rest of it. ************************ The problem isn't MPEG or directly ATI's fault, it is the design of the AVI container. When you are capturing NTSC video using whatever software, you're usually going to be capturing at ~29.997fps, or ~30fps for our purposes. This is a constant framerate, as the video being recieved is a constant framerate. If something happens on the computer that causes the CPU usage to spike, then the encoding software cannot afford to spend time encoding the current frame and skips, or 'drops', it. Basically the encoding application doesn't give that frame to the codec, so it is never encoded and it is never stored. When this happens, you suddenly end up with a situation where the fps (frame per second) drops to 15fps for a frame. This is fine for storing in MPEG, because the container supports vfr (variable frame rate). It can go along at 30fps, switch to 15fps for a frame or so, and then switch back to 30fps. And because the container supports it, when playing it back, the video renderer will display a few frames when they should be seen. The problem is when you try and convert this vfr MPEG to AVI, or capture video with dropped frames directly to AVI. AVI ONLY SUPPORTS CFR (constant frame rate). So, if the video capture starts at 30fps, that is how AVI will store every frame, even if one was dropped and the framerate drops. What happens if you are displaying a 30fps video at 30fps, and the framerate briefly drops to 15fps, but you keep displaying it at 30fps? Well, if the audio is constant (and it basically always is) then the video gets ahead of the audio. Voila! Instant out of synch audio/video. The ATI software just isn't very efficient with the CPU usage, causing it to spike more often, and thus more dropped frames. The resulting files will work fine in MPEG, but when you try to convert to AVI, you get instant synch problems. This is one of the reasons that the Matroska container was invented. It can hold all types of video like AVI, but supports vfr. So, in theory, you should be able to convert a vfr MPEG video file to a Matroska file, and never lose synch.

Posted by: fccHandler Mar 23 2003, 12:19 AM

@Pamel: Great explanation, and I think you are probably correct with everything you said. Perhaps VirtualDub's MPEG parser could read the actual PTS stamps, and attach them to the frames for seeking purposes, rather than assuming (for instance) that 2997 frames equals 100 seconds into the clip. Seems like you would need to do this anyway to pass correct time stamps into the Matroska container. Using the PTS stamps means that converting a vfr MPEG to cfr AVI might cause some frames to be doubled, or others to be dropped, but at least it should stay in sync. Hmmmmm...

Posted by: Pamel Mar 23 2003, 04:54 AM

It think it would be a lot easier to change one of the AVISynth filters to detect a spot where a frame should probably be, and just output a duplicate decoded frame. But yes, that should work, and would be pretty handy.

Posted by: fccHandler Mar 23 2003, 06:41 AM

QUOTE (Pamel @ Mar 23 2003, 12:54 AM) It think it would be a lot easier to change one of the AVISynth filters ... Maybe so, but I don't know much about coding Avisynth filters. However, I know quite a lot about VirtualDub's MPEG parser, so for me, adding code to store the PTS stamps in VirtualDub would be less painful. Anyway, I don't use Avisynth for transcoding MPEG-2 (obviously). The only catch is that PTS stamps are not required on every PES packet. TMPGEnc and ATI seem to write them on almost every packet, but I see a lot of DVD VOBs only have PTS on packets containing an I-frame. Still, I suppose the missing PTS stamps could be interpolated from the existing ones...

Posted by: zib92130 Mar 23 2003, 08:58 AM

@Pamel I was parsing the forum since half a week to find an explanation/solution for my sync prob while converting MPEG (captured by ATI hard/software) to avi. All you have said answered my questions : - why captured MPEG is read correctly - why vdub doesn't show the same frame rate 'for duration match' for 2 different captured files - why the converting avi is 'suddenly' out of sync in the middle of the movie So now, due to your explanations, my quest is not 'WHY' anymore, but 'HOW' to overcome this... Let's go and search and make some tests...

Posted by: jktla Mar 27 2003, 05:24 AM

My sync problem solved when I started to use standard MP3 audio decoder codec when watching video. Now I can record at MPEG2 video and convert it Divx with MP3 sound without any sync proglems, and I don't have to change fps.

Posted by: phaeron Apr 1 2003, 09:14 AM

Nice try, but I call BS on Pamel's explanation. First, the argument that the AVI will desync because a section drops to 15 fps is complete bull. The whole purpose of zero-byte drop frames in an AVI video stream is to pad the video frames out to maintain sync and as indicators that has happened. So if the encoder is forced to drop to 15 fps, a drop frame is inserted between each encoded frame and sync is maintained. The decoder can trivially detect that this has been done and can even interpolate between the missing frames if desired. The code to calculate when drop frames are required for a stream that periodically drops below target rate is trivial. If you have a capture program that knows it is receiving a section of frames with twice the period that it is supposed to and writes them back-to-back in the video stream, ditch it and get one that actually works. Now, as for MPEG.... 2.5.1.4 of 11172-4 (MPEG-1 compliance requirements) requires that within any section of audio or video that has no discontinuities, the nominal frame rate must not vary from the average PTS delta over that set of samples by more than 100 parts per million. This means a section with no discontinuities must have precise PTS timestamps and can't have large amounts of jitter in the timing. So what are discontinuities? Well, according to 2.4.5.4 in 11172-1, a discontinuity exists whenever a frame's PTS is larger than the upper bound for that frame given required clock accuracy. That means: 1) PTSs cannot be anachronistic (go backward). 2) PTSs cannot encode frames at a higher rate than the specified picture rate, since that does not quality as a discontinuity and this would violate the precision requirement. So the local frame rate over any portion of the stream must have a framerate <= that specified. Well, guess what -- AVI drop frames allow you to compensate for such disturbances in the video stream with at most 1/2 frame error. At 30 fps, that's 17ms. Triple buffering in an FPS game will give you more delay than that. Never mind the fact that compliant decoders can't really be expected to continuously play through a discontinuity. Let's further read D.6.7 in ISO/IEC 11172-2 (MPEG-1 Video), where it says: QUOTE This part of ISO/IEC 11172 does not allow pictures to be dropped at the encoder.  This differs from the case of CCITT Recommendation H.261 where temporal sub-sampling may be done by omitting coded pictures from the sequence.  This part of ISO/IEC 11172 requires that all source pictures be encoded and that coded pictures must be inserted into the bitstream nominally at the rate defined by picture_rate field in the sequence header. It goes on to describe how an effective lower frame rate can be achieved by encoding P- and B-frames with all predictions and no DCT data. This is completely transparent to any decoder, including VirtualDub's, as it is indistinguishable from any other regular frame. So please stop spreading this crap. MPEG does not support VFR, and AVI does not have the sync problem you describe.

Posted by: fccHandler Apr 1 2003, 05:18 PM

Isn't it still possible that these captured MPEGs contain drifts in the PTS which are in violation of 11172-4? How would you explain the frequent reports that a captured MPEG is synchronized in WMP, but out of sync in VirtualDub? If we throw out Pamel's explanation, we're back to square one. I call upon anyone reading this thread to please provide us with a link to an MPEG which exhibits the behavior described in my previous paragraph. (So far I haven't been able to make one myself, using my own ATI hardware and software.)

Posted by: Pamel Apr 1 2003, 09:32 PM

Wow, that would be 2 strikes against me from Avery within the period of a week. Excuse me while I go out to purchase a new ego, the prevoius one having been burnt as an effigy of misinformation. I think it can be safely said to take anything I have said with a few grains of salt, C4, and a few blasting caps. I don't remember where I picked up my (mis)information, but I had always assumed it was correct. The only time that I have not had synch issues when making a long capture with dropped frames is when using ATI's native MPEG-2 vcr format. Of course, I had never actually tried to capture to MPEG-1, so I don't know if it is different. And the synch issues seemed related to dropped frames, as synch would be on in the beginning, but the further you got in the stream, the synch would just get a little further off every once in awhile. Changing the framerate would align it a little better, but it would still be off by varying amounts throughout the video. Of course, maybe I was an idiot there and selected the wrong settings in the different pieces of software I used? Anyway, comparing my fragmented memory with Avery's knowledge of video would be a fool's game. I will change my previous post to reflect my new found knowledge.

Posted by: Pamel Apr 1 2003, 10:58 PM

@fccHandler: Someone has offered an out of synch file for testing with http://virtualdub.everwicked.com/index.php?s=1df48c3bc18ed41f75a6f47378c96724&act=ST&f=6&t=389&st=0&#entry10788 I can offer nothing because since I moved, my pc is no longer hooked to any video source.

Posted by: phaeron Apr 2 2003, 03:39 AM

fccHandler: It could be an out-of-spec file, but here are other things that I can think of off the top of my head that could cause the desync: 1) Audio and video streams not starting at the same time. This can be solved by padding the late stream. 2) Error building up over long periods of time. I have heard stories of people recording MPEGs which, after, decompression, have >2^31 audio samples! Can be resolved by tweaking the video frame rate. 3) Actual discontinuity in the streams. Yuck. Cut off the streams at the discontinuity and then apply (1) to the next segment. 4) Audio or video parser screws up and misses frames. Hard to detect by reports, unfortunately. It's unfortunately hard to diagnose this problem when all people report is that their file is out of sync, without saying how long it is or if the sync changes over time. I too have not been able to repro the problem with my ATI card and software. ATI also has a funny sound resampling DirectShow filter in their capture software that adds an additional variable into the equation. The MPEG parser that ships with DirectShow copes with an awful lot of garbage in a file, including raw joins and incorrect GOP timestamps. I get the impression that it mainly syncs through the systems stream and then fine-tunes with the PTS. Works remarkably well for its age, really (dates back to Win95 OSR2!), and certainly better than the crappy DVD player software I got with my laptop, that's for sure. The main problems with including PTS support: 1) PTSes are a pain to decode because the PTS is tied to the pack that contains the first byte of the frame marker. Consider four packs, each of which hold one byte of the marker -- the PTS is three packs back by the time you have decoded the marker. This is even more annoying when you consider that markers can be preceeded by runs of 00 and you don't know which 00 snags a timestamp until you hit the third marker byte. 2) PTSes are incorrect when a raw join occurs. Enough said. Most MPEG files are nice and clean and have a video frame per pack, with a PTS+DTS on every I-frame. It's the once that are sliced up (VCD) that are scary.

Posted by: fccHandler Apr 2 2003, 04:11 AM

QUOTE (phaeron @ Apr 1 2003, 11:39 PM) Consider four packs, each of which hold one byte of the marker -- the PTS is three packs back by the time you have decoded the marker. Hehe. That's far-fetched, but I see your point. Anyway, if you don't think that PTS support will help then I'll strike it from my todo list for now. I'm still waiting to see if someone will post a link to that mythical out-of-sync out-of-spec MPEG they captured...

Posted by: LeQuack May 21 2003, 11:06 PM

Any solution to this yet ? I have a MPEG1 video/audio that I want to encode to xvid/mp3 but the problem is progressive asynchronisation (fine at start, up to 2sec delays at end). If i do "change so audio and video durations match" frame change in Vdub the video is synched in parts, but still not synched elsewhere. Someone said thet reencoding with TMPEGEnc fixes this, thou not for me. Even if I do just do a demux/mux with TMPEGEnc it loses sync.

Posted by: LeQuack May 22 2003, 03:34 PM

Okay so I now encoded with TMPEGEnc directly to AVI using xvid/mp3 and the audio/video are perfectly synched. Strange, I couldn't do this with Vdub or VdubMod, nor with nandub, flask...

Posted by: sternengaenger May 30 2003, 05:34 AM

The problem of the desync video/audio seems to grow. I heare many people talk about this problem using Vdub. It seems that playing with WMP or encoding with tmpg or flask the problem does not exist. Just a short review. problem on: mpeg1 29.97fps vdub suggests a conversion to (about) 30.02 fps to match audio/video. This suggests a shorter audio-track. So the suggested problem of dropped frames when capturing the video cannot be the problem, or else the video would be shorter than the audio (and this is not the case) @phaeron Do you regard the whole discussion as BS or are you seriously giving it some thoughts? Thanks for your work.

Posted by: phaeron May 30 2003, 06:17 AM

@&*#(&($# Your question is not as important as you think it is. If you send me an email, AND send me a private message, AND post in the forum, AND all three messages are exactly the same, you are going to PISS ME OFF. Ask your question ONCE. Now, to answer your question: yes

Posted by: fccHandler May 30 2003, 07:34 AM

I'm gonna repeat what I said 2 months ago: I call upon anyone reading this thread to please provide us with a link to an MPEG which exhibits the behavior I described before. Namely, that it's played in sync by WMP but out of sync in VirtualDub. (Until I see this firsthand, I refuse to believe it exists.)

Posted by: sternengaenger Jun 2 2003, 04:16 PM

OK, here is a link to a testclip http://130.75.21.166/daten/' target='_blank'>http://130.75.21.166/daten/[/URL] it is about 47MB in size. It would be nice if not just anyone is going to download the clip, sinc the traffic will slow everything down here... If you load this into Vdub you get an errormessge telling you that the stream is desynced. That did not happen with the original mpeg. So i guess it is just from cutting. But the desync ist the same in the real mpeg.

Posted by: sternengaenger Jun 2 2003, 04:18 PM

The above link doesn't seem to worke. Please copy and paste this one into a browser: http://130.75.21.166/daten/ and choose the testclip

Posted by: fccHandler Jun 2 2003, 06:28 PM

@LeQuack and sternengaenger: Thanks to both of you, you've made a believer out of me. I got both of your test clips and everything you said about them is true. I've been playing with them for a while and I think I'm finally ready to try and put this matter to rest. From studying the clips, and through many decodings/encodings, it's become apparent to me that their internal video rate doesn't match their audio rate, and it's as simple as that. Unfortunately, I have no idea how they got that way, but I'm guessing that Media Player drops or duplicates frames during playback to keep the timing synchronized. I tried reencoding the clips with TMPGEnc (as others had suggested) and indeed the output is synchronized. But stepping through the TMPGEnc-produced video and comparing it to the original, it's clear that no frames have been added or deleted, nor has the frame rate changed. However, the audio has been stretched! It was quite visible comparing the original audio and the TMPGEnc-produced audio in Cool Edit's multitrack mode. I also tried reencoding the clips with FlasK. Again the output was synchronized. But in this case the video frames don't match at all, and many frames have been duplicated compared to the original sequence. The point is, both programs are stretching one stream relative to the other in order to maintain sync. Conclusion: The plain and simple reason they appear out of sync in VirtualDub is because they really are out of sync internally, and what VirtualDub is showing you is the real deal. In order to correct it, it will be necessary to 1) stretch the audio, or 2) increase the frame rate. I think tweaking the frame rate to 30fps helps a lot with both clips.

Posted by: sternengaenger Jun 2 2003, 06:44 PM

Ok, that is what i thought so fare, too. I do have the feeling thought, that only stretching the video to the suggested framerate is at some points not enough. 1. with some Movies it seems that even the if the Framerate is matched with the suggested new video fps-rate the encoded file seems to be off just a fraction in some points wheras the overall synchonisation seems to matche. I unfortunaly cannot give you a testclip for that. It is just a feeling, but maybe someone can confirm that. 2. Even when choosing the suggested Framerate sometimes it is still out of sync. I think the much better way is the tmpeg way of stretching the audio. AND it would realy help to have an exact audiostretch possibility. VD 1.5.4 has the audiostretch option, but the precision is not enough, here a precision of 0.9e-6 would be helpfull to even correct a variance of 0.5sec desync on 90min movie. Again I think An automatic calculation and a activationbutten for audio matching to video not vice versa would do a much better job... And thank you for trying and the confermation. Besides that I think every streem should be sampled into avi right away. Why bother with the old mpeg.... jo

Posted by: fccHandler Jun 2 2003, 07:07 PM

QUOTE (sternengaenger @ Jun 2 2003, 02:44 PM) I think the much better way is the tmpeg way of stretching the audio. This should work: Turn down all of the video settings in TMPGEnc to the crappiest quality so it will do a really fast encode, then rip the audio out of the TMPGEnc-encoded file, and select that as the audio after you open the original MPEG in VirtualDub.

Posted by: Pamel Jun 13 2003, 02:20 PM

There is an interesting discussion going on at the Xiph mailling lists about how they are going to be storing video in Ogg. Anyway, they cover this exact same topic and give an interesting reason for it http://www.xiph.org/archives/theora-dev/200306/0053.html I would encourage you to check it out.

Powered by Invision Board (http://www.invisionboard.com)
© Invision Power Services (http://www.invisionpower.com)