|
|
| DrGuz |
| Posted: Jul 22 2013, 09:00 PM |
 |
|
Newbie
Group: Members
Posts: 9
Member No.: 25655
Joined: 8-June 09
|
Symptom:
AVI files created with "fast recompress" cannot be appended to source files; those created with "normal recompress" can.
Reproduce:
1) Create two AVI files, using the same codec. In my case, I use Xvid-1.2.2 (16:23:59, Jun 7 2009), unrestricted profile, single-pass encoding, target bitrate 10000kbps.
2) Verify that they can be concatenated: open file 1, then choose menu "File > Append AVI Segment" and open file 2. They should now be one long stream in the VirtualDub editor, with no errors.
3) Close that, and open file 2. Select some frames at the beginning (I chose 20). For example: Ctrl+G, "20", Enter key, End key.
4) Set menu "Video > Fast recompress", and save as (F7) "fast-recompress.avi".
5) Set menu "Video > Normal recompress", and save as (F7) "normal-recompress.avi".
6) Open file 1 again.
7) Choose menu "File > Append AVI Segment" and open "normal-recompress.avi". Observe no error.
8) Choose menu "File > Append AVI Segment" and open "fast-recompress.avi". Observe the following error dialog:
---------------------------
VirtualDub Error
---------------------------
Cannot append segment "fast-recompress.avi": The video streams have incompatible data formats.
(Mismatch detected in opaque codec data at byte 14 of the format data.)
---------------------------
OK
---------------------------
Versions:
First observed today in VirtualDub 1.9.10, reconfirmed in 1.9.11 and 1.10.3
Windows XP(SP3) 32-bit
|
|
| raffriff42 |
| Posted: Jul 25 2013, 08:09 PM |
|
|
Advanced Member
Group: Members
Posts: 384
Member No.: 35081
Joined: 25-June 12
|
Confirmed the same results. Don't know what it means, or if this is normal behavior.
MediaInfo shows no difference between fast- and normally-recompressed clips beyond a very minor difference in bitrate (Qf = 0.124 vs. 0.128). I can post the reports, but they fill several screens and show nothing. |
|
| dloneranger |
| Posted: Jul 25 2013, 08:30 PM |
|
|
Moderator
Group: Moderators
Posts: 2366
Member No.: 22158
Joined: 26-September 07
|
Fast mode hooks up the decoder codec to the encoder codec
Normal (and full) decompresses the video stream and sends the decompressed output to the encoder codec
| QUOTE | | (Mismatch detected in opaque codec data at byte 14 of the format data |
That's virtualdub comparing the two video streams encoded format data. The codecs own specific data is saved into the file by the encoder, Virtualdub doesn't know what this data is (or means), it's just reporting that the two are somehow 'different' (and it can only append when they are both the same)
XVid's recording 'something different' into it's format data - no idea what though
FWIW many other codecs don't seem to have the same problem
--------------------
MultiAdjust JoinWav WavNormalize FFMPeg Input Plugin v1827 UnSharpMask
Windows7/8 Codec Chooser
All FccHandlers Stuff inc. Installers for acm codecs AAC, AC3, LameMp3 |
|
| DrGuz |
| Posted: Jul 26 2013, 02:53 PM |
|
|
Newbie
Group: Members
Posts: 9
Member No.: 25655
Joined: 8-June 09
|
Ok, I can believe this is an issue with Xvid, not VirtualDub. Thanks for the follow-up, friends!
I'm sure there are many reasons it is unsafe for VirtualDub to permit concatenating two streams whose opaque data mismatch ... but would it be unwise to provide a preference option to override that safeguard? Or am I misguided in thinking that fast recompress might not be as lossy as normal recompress, and making a foolish request? |
|
| -vdub- |
| Posted: Jul 26 2013, 03:02 PM |
|
|
Advanced Member
Group: Members
Posts: 613
Member No.: 27087
Joined: 24-February 10
|
Using a YV12 colorspace http://forum.doom9.org/archive/index.php/t-123491.html |
|
| dloneranger |
| Posted: Jul 26 2013, 04:08 PM |
|
|
Moderator
Group: Moderators
Posts: 2366
Member No.: 22158
Joined: 26-September 07
|
| QUOTE | | Or am I misguided in thinking that fast recompress might not be as lossy as normal recompress, and making a foolish request? |
You might avoid some rgb<>yuv conversions, but you can set the colour depth options to a yuv format for normal/full processing modes anyway
TBH, I just use direct stream or full processing mode (once upon a time there was more of a difference in speed between the modes, but there's less of a difference now)
| QUOTE | | I'm sure there are many reasons it is unsafe for VirtualDub to permit concatenating two streams whose opaque data mismatch ... but would it be unwise to provide a preference option to override that safeguard? |
Depends on who you ask  from the dev's point of view, it cuts down on the 'I appended two videos and they didn't work' complaints
It's such a rare occurrence (somebody saving two parts in different modes and then trying to append them) that it's probably not worth the time coding an option for it
--------------------
MultiAdjust JoinWav WavNormalize FFMPeg Input Plugin v1827 UnSharpMask
Windows7/8 Codec Chooser
All FccHandlers Stuff inc. Installers for acm codecs AAC, AC3, LameMp3 |
|
| DrGuz |
| Posted: Jul 29 2013, 07:14 PM |
|
|
Newbie
Group: Members
Posts: 9
Member No.: 25655
Joined: 8-June 09
|
Research results! And issue resolved.
| QUOTE | | You might avoid some rgb<>yuv conversions, but you can set the colour depth options to a yuv format for normal/full processing modes anyway |
Excellent tip! This led to my discovery of complete data integrity for all modes.
I found that for Xvid, "Fast recompress" produces results that are byte-identical with "Normal recompress" configured for "Color Depth > Decompression format = YUY2". (This is slightly surprising, since that's a 4:2:2 format and MediaInfo v0.7.33 reports Xvid-encoded files as "YUV 4:2:0"). Further, "Full processing mode" will also produce identical results as long as no filters are used. So these modes may have a difference in speed (I didn't measure any), but not in data.
My original tests were not splice-compatible because I hadn't changed Color Depth settings from their default, 24-bit RGB. It didn't occur to me that could be the culprit.
YV12 turns out to generate the exact same datastream as YUY2. In fact, all of the YCbCr options that Xvid accepts (some are rejected at F7 time) give the same results. Same goes for the RGB formats, they all generate one datastream (which is different from YUV).
Actually, the entire AVI file, wrappers and all, is byte-identical except for one single byte, the one VirtualDub complains about when trying to append! The thread linked above says, "Byte 14 is not XVID related after all. Turns out it's an offset beginning at a BITMAPINFOHEADER. 14 is biBitCount." YUY2 sets this 14th byte to the value 16, YV12 sets it to 12, RGB sets it to 16, 24, or 32.
One solution is to use a 4:2:2 (16-bit) YUV format in the first place. Then any post-production splices will naturally append cleanly with "Fast recompress". However, if you're using filters, VirtualDub will convert the stream to RGB-32 for processing. You'll have to decide whether you'd rather have VirtualDub convert to YUV before handing to Xvid (Color Depth > Output format = YUY2) or let Xvid do the conversion internally (Color Depth > Output format = RGB-24). Results will vary, and testing is beyond the scope of this post.
Another solution is to use "Normal recompress" and choose a matching bit depth. Unfortunately, if the original was already output as RGB-24, then RGB-24 will have to be used again since there is no YUV format that will set biBitCount to 24. This method may degrade data through the double conversion of YUV->RGB->YUV.
My preferred solution, though, will be to trick VirtualDub by hex editing the spliced files. This is probably not for everyone, proceed with caution! I believe it is safe because VirtualDub happily appends an RGB15 to a YUY2, and Xvid doesn't seem to care. Simply find byte offset #14 after the "strf" chunk marker (and its 4-byte length field), two bytes before the "XVID" biCompression field, and change it to match what it's being spliced with (in my case, changing it from 0x10 to 0x18). Use your favorite hex editor, or VirtualDub's built-in one, or the command line batch scripting language of your choice.
| CODE | s t r f 28 00 00 00 28 00 00 00 C0 02 00 00 E0 01 00 00 01 00 [18] 00 X V I D
- - - - # # # # 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 - - - - |
|
|
| dloneranger |
| Posted: Jul 29 2013, 08:32 PM |
|
|
Moderator
Group: Moderators
Posts: 2366
Member No.: 22158
Joined: 26-September 07
|
Interesting detective work there 
Here's a bit more info on Virtualdubs format conversion and filters system
There's no inherent reason that Virtualdub will do a yuv<>rgb conversion when filtering
Virtualdub asks each filter what formats it supports, and tries a closest fit if a conversion is needed
You just have to avoid filters that are rgb only.....
In the filter list dialog, you can tick the 'show image formats' box and you'll get the format listed as well
--------------------
MultiAdjust JoinWav WavNormalize FFMPeg Input Plugin v1827 UnSharpMask
Windows7/8 Codec Chooser
All FccHandlers Stuff inc. Installers for acm codecs AAC, AC3, LameMp3 |
|
| DrGuz |
| Posted: Aug 3 2013, 07:21 PM |
|
|
Newbie
Group: Members
Posts: 9
Member No.: 25655
Joined: 8-June 09
|
Thanks, dloneranger.
I've discovered some unexpected behavior with 'show image formats', and started a separate thread since it's a separate issue. http://forums.virtualdub.org/index.php?act...ST&f=15&t=21751 |
|
| -vdub- |
| Posted: Oct 27 2013, 04:16 PM |
|
|
Advanced Member
Group: Members
Posts: 613
Member No.: 27087
Joined: 24-February 10
|
Normally I use intermediate files (lagarith) so to retain video quality as much as possible. Yesterday I was trying different things (without intermediate files)with mp4--avc1-aac and came across this. I used full compression on two clips with x264. Couldn't be overcome using x264 colour depth options or virtualdub colour depth options.
Though found with reading this thread again how easy it is to fix. The opaque error for byte 20 of the format data. Which is the fist 4 bytes after AVC1 byte 20 starts with 0C [18 00 A V C 1 0C 50 0D 00] in hex editor the previous above byte 14 [18 00 X V I D]
These were video edits saved to avi--avc1-aac (yuv420p) using virtualdub. Even though was able to append all files after quick edit so all were the same using virtualdub hex editor. They were wrong with green frames introduced after appending all file to one avi-avc1-ac3
Problem is will get a green frames after edit at start of the new video clip. Only noticed when append to another avi later and also then introduces lip sync problems (for all video clips appended after the first video clip). No video clips after edits have no lip sync issues only when appended. The same was noticed if first cuts also encoded to pcm audio
All edits were made on I frames (K0) start clip and on I frames at end of the clip. Also tried frame before I frame at end of the clip in case there was a difference (none)
The above opaque could affect just mp4 videos. Have never seen it before with other editors is it only virtualdub that has this problem. Neither the green frame issues or the lip sync issues are caused by hex editing to correct the opaque problem. Only when appending (joining) one avi to another with virtualdub
I have just encoded all the video edits to avi uncompressed lagarith with pcm audio direct copied. All clips are in sync when played with virtualdub, mpc-hc, wmp, vlc and others. After video appending each clip to one another to final intermediate video there are no green frames with uncompressed video. Although the lip sync returns all appended clips after the first clip is affected
Will have a look for other editors to overcome lip sync introduced when append all avi together with virtualdub. Avidemux (old frame edit version) while cannot decode lagarith. Did join all the uncompressed video edits with no green frames and no lip sync issues. Something must be wrong with virtualdub latest build I haven't tried this with any previous virtualdub builds. Would rather use virtualdub than another old or new editor to do the same
@Phaeron can anything be done to stop the green frames at clip start when appending (non uncompressed video with audio pcm or other) while also keeping the audio in sync for all video clips that are appended (added) |
|
