Printable Version of Topic
Click here to view this topic in its original format
Unofficial VirtualDub Support Forums > Testing / Bug Reports > Crash After Extend Luma Point In Capture


Posted by: element Mar 23 2011, 09:26 PM
I'm running VirtualDub 1.10.1 in Win7 x64. In Capture, selecting Video->Extend Luma Point (white or black) results in a crash. This is right after launching VirtualDub, once I've switched to Capture mode.

VirtualDub crash report -- build 34072 (release)
--------------------------------------

Disassembly:
0048fea0: 67c9 leave
0048fea2: 4d dec ebp
0048fea3: 0f7e4efc movd [esi-04h], mm1
0048fea7: 0f7e4ffc movd [edi-04h], mm1
0048feab: 758b jnz 0048fe38
0048fead: 03f0 add esi, eax
0048feaf: 03fb add edi, ebx
0048feb1: 49 dec ecx
0048feb2: 7582 jnz 0048fe36
0048feb4: 5b pop ebx
0048feb5: 5e pop esi
0048feb6: 5f pop edi
0048feb7: 5d pop ebp
0048feb8: 0f77 emms
0048feba: c3 ret
0048febb: cc int 3
0048febc: cc int 3
0048febd: cc int 3
0048febe: cc int 3
0048febf: cc int 3
0048fec0: 56 push esi
0048fec1: 57 push edi
0048fec2: 8b7c240c mov edi, [esp+0ch]
0048fec6: 8bc7 mov eax, edi
0048fec8: d1f8 sar eax, 1
0048feca: 6801000100 push 00010001
0048fecf: 83c001 add eax, 01h
0048fed2: 99 cdq
0048fed3: 6801000100 push 00010001
0048fed8: 52 push edx
0048fed9: 50 push eax
0048feda: 8bf1 mov esi, ecx
0048fedc: e82f981400 call 005d9710
0048fee1: 894618 mov [esi+18h], eax
0048fee4: 6801000100 push 00010001
0048fee9: 89561c mov [esi+1ch], edx
0048feec: 8bc7 mov eax, edi
0048feee: 99 cdq
0048feef: 6801000100 push 00010001
0048fef4: 52 push edx
0048fef5: 50 push eax
0048fef6: e815981400 call 005d9710
0048fefb: 85ff test edi, edi
0048fefd: 894620 mov [esi+20h], eax
0048ff00: 895624 mov [esi+24h], edx
0048ff03: 7508 jnz 0048ff0d
0048ff05: 8bca mov ecx, edx
0048ff07: 894618 mov [esi+18h], eax
0048ff0a: 894e1c mov [esi+1ch], ecx
0048ff0d: 5f pop edi
0048ff0e: 5e pop esi
0048ff0f: c20400 ret 0004
0048ff12: cc int 3
0048ff13: cc int 3
0048ff14: cc int 3
0048ff15: cc int 3
0048ff16: cc int 3
0048ff17: cc int 3
0048ff18: cc int 3
0048ff19: cc int 3
0048ff1a: cc int 3
0048ff1b: cc int 3
0048ff1c: cc int 3
0048ff1d: cc int 3
0048ff1e: cc int 3
0048ff1f: cc int 3
0048ff20: 55 push ebp
0048ff21: 57 push edi
0048ff22: 56 push esi
0048ff23: 53 push ebx
0048ff24: 8b442424 mov eax, [esp+24h]
0048ff28: 0f6f34c528ab61 movq mm6, [eax*8+0061ab28] <-- FAULT
00
0048ff30: 0f6f2cc5c8aa61 movq mm5, [eax*8+0061aac8]
00
0048ff38: 8b4c241c mov ecx, [esp+1ch]
0048ff3c: 8b742414 mov esi, [esp+14h]
0048ff40: 8b5c2420 mov ebx, [esp+20h]
0048ff44: 8b442418 mov eax, [esp+18h]
0048ff48: 8bd1 mov edx, ecx
0048ff4a: c1e202 shl edx, 02h
0048ff4d: 2bc2 sub eax, edx
0048ff4f: 8bd1 mov edx, ecx
0048ff51: f7c604000000 test esi, 00000004
0048ff57: 7424 jz 0048ff7d
0048ff59: 0f6e06 movd mm0, [esi]
0048ff5c: 0fefff pxor mm7, mm7
0048ff5f: 0f60c7 punpcklbw mm0, mm7
0048ff62: 83c604 add esi, 04h
0048ff65: 0f71f002 psllw mm0, 02h
0048ff69: 4a dec edx
0048ff6a: 0ffdc5 paddw mm0, mm5
0048ff6d: 0fe5c6 pmulhw mm0, mm6
0048ff70: 0f67c0 packuswb mm0, mm0
0048ff73: 0f7e46fc movd [esi-04h], mm0
0048ff77: 0f8484000000 jz 00490001
0048ff7d: 83ea03 sub edx, 03h
0048ff80: 767f jbe 00490001
0048ff82: 0f6f06 movq mm0, [esi]
0048ff85: 0fefff pxor mm7, mm7
0048ff88: 0f6f5608 movq mm2, [esi+08h]
0048ff8c: 0f7fc1 movq mm1, mm0
0048ff8f: 0f60c7 punpcklbw mm0, mm7
0048ff92: 0f7fd3 movq mm3, mm2
0048ff95: 0f71f002 psllw mm0, 02h
0048ff99: 83c610 add esi, 10h
0048ff9c: 0ffdc5 paddw mm0, mm5
0048ff9f: 0f db 0fh

Built on Althena on Sat Mar 19 16:51:45 2011 using compiler version 1400

Windows 6.1 (Windows 7 x64 build 7600) []
Memory status: virtual free 1901M/2048M, commit limit 8172M, physical total 4087M

EAX = 4b544d00
EBX = 023bc98c
ECX = 00000500
EDX = 05775400
EBP = 023bc920
ESI = 0000000b
EDI = 0608fcdc
ESP = 0608fc6c
EIP = 0048ff28
EFLAGS = 00010203
FPUCW = 027f
FPUTW = ffff

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'VirtualDub'...

...reading address 5B041328.

Pointer dumps:

EBX 023bc988: 0061abc8 023bc9cc 023bc964 4b544d00 00000000 0061abd4 53544e20 544d0043
EDX 05775400: 7a4fa24f 7850a454 7853a64e 784ca452 7855a451 784ea44f 7751a455 7753a44e
EDI 0608fcd8: 00000000 00498527 0608fd0c 023b89ec 023b8850 0608fdb8 bebab390 00000000
ESP 0608fc68: 05775400 023bc98c 0000000b 0608fcdc 023bc920 004900c7 05775400 00000500
0608fc88: 00000140 000001d6 4b544d00 0608fd34 00491096 0608fcb4 0608fd34 023b963c
0608fca8: 023b9600 bebab301 023bc868 05775400 023b978c 00000280 000001d6 00000500
0608fcc8: 0000000b 05774000 00000000 05774000 00000000 00498527 0608fd0c 023b89ec
EBP 023bc920: 023bc92c 023bc9cc 0061abbc 023bc964 023bc920 00001400 023b978c 00000280
023bc940: 000001d6 00000500 0000000b 00000000 00000000 00000000 00000000 6f675f64
023bc960: 0061ac44 023bc98c 023bc92c 03c20030 00024b80 70006f69 00060006 00060006
023bc980: 000a000a 000a000a 0061abc8 023bc9cc 023bc964 4b544d00 00000000 0061abd4

Thread call stack:
0048ff28: ?A0xc90e1093::lumasquish_MMX()
004900c7: VDCaptureFilterLumaSquish::Run()
00491096: VDCaptureFilterSystem::ProcessIn()
00498527: VDCaptureProject::CapProcessData2()
00498ea1: VDCaptureProject::CapProcessData()
005b73f1: VDCaptureDriverDS::CapProcessData()
76ce0849: KERNELBASE!WaitForSingleObjectEx [76cd0000+1077e+cb]
005b76de: VDCapDevDSVideoCallback::SampleCB()
6ca569bc: qedit!DllRegisterServer [6ca30000+10549+16473]
6ca910e8: qedit!DllRegisterServer [6ca30000+10549+50b9f]
747fe752: ksproxy!DllUnregisterServer [747f0000+42c0+a492]
747fe6cc: ksproxy!DllUnregisterServer [747f0000+42c0+a40c]
74818f54: ksproxy!DllGetClassObject [747f0000+23f02+5052]
76ce09cd: KERNELBASE!WaitForMultipleObjectsEx [76cd0000+10862+16b]
76d3162d: kernel32!WaitForMultipleObjectsEx [76d20000+1159f+8e]
747ff2f4: ksproxy!DllUnregisterServer [747f0000+42c0+b034]
7480aaa7: ksproxy!KsGetMediaTypeCount [747f0000+16d6f+3d38]
76d33677: kernel32!BaseThreadInitThunk [76d20000+13665+12]
77399f02: ntdll!RtlInitializeExceptionChain [77360000+39e9f+63]
77399ed5: ntdll!RtlInitializeExceptionChain [77360000+39e9f+36]

-- End of report

Posted by: element Mar 23 2011, 09:34 PM
Oops, I meant to post it in the Virtualdub 1.10.1 Test Thread, not as a separate thread. Maybe someone can move this post.

Posted by: phaeron Mar 27 2011, 12:48 AM
Nah, separate thread works fine, especially since it's a long-standing bug. I posted a fix in 1.10.1 test-6.

Powered by Invision Board (http://www.invisionboard.com)
© Invision Power Services (http://www.invisionpower.com)