Printable Version of Topic
Click here to view this topic in its original format
Unofficial VirtualDub Support Forums > Advanced Video Processing > Cool Effect


Posted by: evropej Jan 19 2011, 02:28 AM
Just found out some cool things.
Does anyone know this filter?
http://www.infognition.com/

Well, here is how you do in in vdub for free lol.
Oh, and much much better then those noobs.

Open your low res video in vdub.
Add msu deblockers, both
Now, re-size to 600%.
Add smoother 40
Add spacial smoother
Add warp sharp 40
Now re-size to 50 or 60% or close to 720

Enjoy

PS to remove excessive noise, add multiple smoother 40
to increase detail, increase warp sharp
could add denoising
could also add msu cartoon restore

Post a video i can gladly demonstrate

Must warn you, filters and vdub not stable for high resolution videos of 2000x2000 in pixels

Posted by: evropej Jan 19 2011, 06:48 AM
example

http://evropej.com/vdub.jpg

vdub settings
http://evropej.com/vdub_settings.jpg

almost unreal results

i will say this, filters crash on 1.1 or dont work if resolution is set to high.
i will try with 1.9 versions ...

Posted by: evropej Jan 20 2011, 04:06 PM
Can someone from the development teams test the resize filters at high resolution?
I did notice that at some point, some filters just stop working.
I am assuming this is a buffer issue in the filter design.

Also, anyone have contact with the people from the MSU filter team?

Posted by: dloneranger Jan 20 2011, 04:14 PM
The msu team have contact info on their website - but they seem to have abandoned most filter development

Is it about the msu filters crashing or producing incorrect output?
If so, that's been dealt with in the latest test versions, there's a 'force single framebuffer' option for these older filters (in the filter dialog)

What is the problem with high resolutions, your description is pretty vague?

Posted by: evropej Jan 21 2011, 12:11 AM
I figured as much for the MSU team since they have not replied to the email I have sent.

The issue is related to resizing video to extreme resolutions.
Once the pixel dimensions go beyond 2000x2000 pixels give or take a few,
the video output does not work at all. In some cases the input stop working as well.

The MSU filters are more sensitive to these parameter changes.
I will include a crash dump.

VirtualDub crash report -- build 32842 (release)
--------------------------------------

Disassembly:
769db6a0: 93 xchg eax, ebx
769db6a1: a99d76e878 test eax, 78e8769d
769db6a6: b5ff mov ch, 0ffh
769db6a8: ffc7 inc edi
769db6aa: 40 inc eax
769db6ab: 24bc and al, 0bch
769db6ad: a99d76c3cc test eax, ccc3769d
769db6b2: cc int 3
769db6b3: cc int 3
769db6b4: cc int 3
769db6b5: cc int 3
769db6b6: e866b5ffff call 769d6c21
769db6bb: 33c9 xor ecx, ecx
769db6bd: 8178206aa99d76 cmp dword ptr [eax+20h], 769da96a
769db6c4: 0f94c1 setz cl
769db6c7: 8bc1 mov eax, ecx
769db6c9: c3 ret
769db6ca: cc int 3
769db6cb: cc int 3
769db6cc: cc int 3
769db6cd: cc int 3
769db6ce: cc int 3
769db6cf: 8bff mov edi, edi
769db6d1: 55 push ebp
769db6d2: 8bec mov ebp, esp
769db6d4: 83ec50 sub esp, 50h
769db6d7: 8b4508 mov eax, [ebp+08h]
769db6da: 8945b0 mov [ebp-50h], eax
769db6dd: 8b450c mov eax, [ebp+0ch]
769db6e0: 83e001 and eax, 01h
769db6e3: 8945b4 mov [ebp-4ch], eax
769db6e6: 33c0 xor eax, eax
769db6e8: 8945b8 mov [ebp-48h], eax
769db6eb: c745bccfb69d76 mov dword ptr [ebp-44h], 769db6cf
769db6f2: 394514 cmp [ebp+14h], eax
769db6f5: 7423 jz 769db71a
769db6f7: 8b4510 mov eax, [ebp+10h]
769db6fa: 83f80f cmp eax, 0fh
769db6fd: 7603 jbe 769db702
769db6ff: 6a0f push 0fh
769db701: 58 pop eax
769db702: 8945c0 mov [ebp-40h], eax
769db705: c1e002 shl eax, 02h
769db708: 50 push eax
769db709: ff7514 push dword ptr [ebp+14h]
769db70c: 8d45c4 lea eax, [ebp-3ch]
769db70f: 50 push eax
769db710: e80da90200 call 76a06022
769db715: 83c40c add esp, 0ch
769db718: eb03 jmp 769db71d
769db71a: 8945c0 mov [ebp-40h], eax
769db71d: 8d45b0 lea eax, [ebp-50h]
769db720: 50 push eax
769db721: ff156c119d76 call dword ptr [769d116c]
769db727: c9 leave <-- FAULT
769db728: c21000 ret 0010
769db72b: cc int 3
769db72c: cc int 3
769db72d: cc int 3
769db72e: cc int 3
769db72f: cc int 3
769db730: 8bff mov edi, edi
769db732: 55 push ebp
769db733: 8bec mov ebp, esp
769db735: 56 push esi
769db736: 8b7508 mov esi, [ebp+08h]
769db739: 83fef4 cmp esi, 0f4h
769db73c: 7218 jc 769db756
769db73e: 83fef6 cmp esi, 0f6h
769db741: 7713 ja 769db756
769db743: 8d4508 lea eax, [ebp+08h]
769db746: 50 push eax
769db747: 6a00 push 00h
769db749: 56 push esi
769db74a: e8dd020000 call 769dba2c
769db74f: 85c0 test eax, eax
769db751: 7403 jz 769db756
769db753: 8b7508 mov esi, [ebp+08h]
769db756: 56 push esi
769db757: ff153c109d76 call dword ptr [769d103c]
769db75d: 5e pop esi
769db75e: 85c0 test eax, eax
769db760: 7c05 jl 769db767
769db762: 33c0 xor eax, eax
769db764: 40 inc eax
769db765: eb08 jmp 769db76f
769db767: 50 push eax
769db768: e8c5ae0200 call 76a06632
769db76d: 33c0 xor eax, eax
769db76f: 5d pop ebp
769db770: c20400 ret 0004
769db773: cc int 3
769db774: cc int 3
769db775: cc int 3
769db776: cc int 3
769db777: cc int 3
769db778: 8bff mov edi, edi
769db77a: 55 push ebp
769db77b: 8bec mov ebp, esp
769db77d: 8b450c mov eax, [ebp+0ch]
769db780: 83f8f4 cmp eax, 0f4h
769db783: 742c jz 769db7b1
769db785: 83f8f5 cmp eax, 0f5h
769db788: 7416 jz 769db7a0
769db78a: 83f8f6 cmp eax, 0f6h
769db78d: 7531 jnz 769db7c0
769db78f: 64a118000000 mov eax, fs:[00000018]
769db795: 8b4030 mov eax, [eax+30h]
769db798: 8b4010 mov eax, [eax+10h]
769db79b: 8b4018 mov eax, [eax+18h]
769db79e: eb20 jmp 769db7c0

Built on Aegis on Fri Dec 24 13:18:44 2010 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7600) []

EAX = 0018f8e8
EBX = 00000000
ECX = 00000003
EDX = 00000000
EBP = 0018f938
ESI = 049f233c
EDI = 01fbfe10
ESP = 0018f8e8
EIP = 769db727
EFLAGS = 00200216
FPUCW = 027f
FPUTW = ffff

Crash reason: Unhandled Microsoft C++ Exception

Crash context:
An exception occurred in module 'KERNELBASE'...

...while starting filter "Super temporary cleaner, version 1.1" (FilterInstance.cpp:1138).

Pointer dumps:

EAX 0018f8e8: e06d7363 00000001 00000000 769db727 00000003 19930520 0018f980 049efde0
ESI 049f2338: 0201e4b4 049ee294 049ee280 00000000 00000001 00000000 00000000 00000001
EDI 01fbfe10: 005de9b0 1e1e0000 00000000 00000020 00000ea0 000008a0 00003a80 00000000
ESP 0018f8e8: e06d7363 00000001 00000000 769db727 00000003 19930520 0018f980 049efde0
0018f908: 050809f8 050809f8 01fbfe10 00000000 049f233c 0018f928 049e77e2 049f25a8
0018f928: 0018f968 049e4d2f 00000008 049e37c9 0018f970 049e3979 e06d7363 00000001
0018f948: 00000003 0018f964 e06d7363 00000001 00000000 00000000 00000003 19930520
EBP 0018f938: 0018f970 049e3979 e06d7363 00000001 00000003 0018f964 e06d7363 00000001
0018f958: 00000000 00000000 00000003 19930520 0018f980 049efde0 0018f98c 049e1f08
0018f978: 0018f980 049efde0 049ee294 049ee280 00000000 0018f9a4 049e1835 01f89000
0018f998: 01fbfdc0 01fbfdb0 00000000 0018fa1c 00434ba1 01fbfdc0 0063f2c8 006431d0

Thread call stack:
769db727: KERNELBASE!RaiseException [769d0000+b6cf+58]
769db727: KERNELBASE!RaiseException [769d0000+b6cf+58]
049e77e2: super_tc!VirtualdubFilterModuleDeinit [049e0000+1036+67ac]
049e4d2f: super_tc!VirtualdubFilterModuleDeinit [049e0000+1036+3cf9]
049e37c9: super_tc!VirtualdubFilterModuleDeinit [049e0000+1036+2793]
049e3979: super_tc!VirtualdubFilterModuleDeinit [049e0000+1036+2943]
049e1f08: super_tc!VirtualdubFilterModuleDeinit [049e0000+1036+ed2]
049e1835: super_tc!VirtualdubFilterModuleDeinit [049e0000+1036+7ff]
00434ba1: FilterInstance::StartInner()
0043582a: VFBitmapInternal::BindToFrameBuffer()
00436e60: FilterInstance::Start()
005b2fb6: malloc()
0043e6c2: FilterSystem::ReadyFilters()
00461245: VDProject::StartFilters()
0043cdc6: FilterSystem::DeallocateBuffers()
004614b5: VDProject::UpdateDubParameters()
75483064: USER32!GetDCEx [75460000+23001+63]
00466bb0: VDProjectUI::SetVideoFiltersAsk()
754779cd: USER32!IsProcessDPIAware [75460000+179ab+22]
75478b70: USER32!LoadCursorW [75460000+189ee+182]
75483064: USER32!GetDCEx [75460000+23001+63]
75483064: USER32!GetDCEx [75460000+23001+63]
0046cc30: VDProjectUI::MenuHit()
75476ec0: USER32!GetClassNameW [75460000+16b04+3bc]
004794a0: VDUIFrame::DefProc()
0046da41: _catch$?MainWndProc@VDProjectUI@@IAEJIIJ@Z$0()
00465ac0: VDProjectUI::WndProc()
00479b65: VDUIFrame::StaticWndProc()
75476238: USER32!gapfnScSendMessage [75460000+15fc8+270]
754768ea: USER32!gapfnScSendMessage [75460000+15fc8+922]
75476899: USER32!gapfnScSendMessage [75460000+15fc8+8d1]
75477d31: USER32!LoadStringW [75460000+17c12+11f]
7549eec0: USER32!PeekMessageA [75460000+3ed58+168]
75478112: USER32!DispatchMessageA [75460000+18103+f]
00452596: WinMain@16()
005b78db: __tmainCRTStartup()
76c53677: kernel32!BaseThreadInitThunk [76c40000+13665+12]
77179d42: ntdll!RtlInitializeExceptionChain [77140000+39cdf+63]
77179d15: ntdll!RtlInitializeExceptionChain [77140000+39cdf+36]

-- End of report

Posted by: dloneranger Jan 21 2011, 12:35 AM
That's a horribly bugridden filter - produces awful artifacts even in 720*480 here
It's fairly pointless anyway, as the latest virtuadubs handle multithreading themselves
Better to use a non-broken filter instead

You wouldn't have to put up with the 2 frame delay and having to adjust the audio either

(I quite like http://neuron2.net/2dcleaner.html but there are lots of different ones)
-------------


Did you set the msu filters to single frame mode in the filter list?
All the msu filters I use work fine with that option set

Posted by: evropej Jan 21 2011, 01:26 AM
that fixed the msu cartoon filter lol. thanks

I did what you suggested and still issue.
what does that option really mean for filter groups?

Posted by: dloneranger Jan 21 2011, 01:50 AM
QUOTE
I did what you suggested and still issue.

I can't find a problem with any filters I use at high resolutions
Any more examples? Are these problems all with old thirdparty filters?

QUOTE
what does that option really mean for filter groups?

Do you mean, what does force single frame buffer do?
It's a fix for old filters that assume certain things
In the latest virtualdubs, those assumptions are wrong, and the filter breaks, produce rubbish output


Posted by: evropej Jan 21 2011, 03:19 AM
try doing a resize of 2000%, lanczos
then trying filters out


Posted by: dloneranger Jan 21 2011, 12:24 PM
You'd have to get step by step of what you do

I can resize to 12800*7040 and add a few filters - works fine

But with each frame being 343Mb in RGB32 virtualdub runs out of memory with a lot of filters added as a 32bit program only has 2Gb it can use and reports 'out of memory' in it's status bar

Most filters need 2 frames for each image, so at about 700MB per filter you're out of memory very fast

If you can open the video in YUV420 and only use YUV420 in all the filters, then each frame is only 108MB and you can use more before running out of memory

http://img195.imageshack.us/i/73984308.jpg/

Posted by: evropej Jan 21 2011, 10:37 PM
you have to load a lot of plugins
http://evropej.com/vdub_settings.jpg

add msu denoise and cartoon restore
i tried using both the latest and beta version and both crash
even with single buffer selected as suggested

there is a size that definately causes the crashes
its stable up to a point and then things start falling apart

Posted by: dloneranger Jan 21 2011, 11:13 PM
With the same filters and settings, I get no problems

Upping the resize to 6240x 3520 still works fine (though virtualdub's using 800Mb of ram by then)

You do know that the single frame buffer options is a separate option for each filter, and should be turned on for all filters that need it?


http://img696.imageshack.us/i/66913911.jpg/
I also moved all the msu filters in between the 2 resize filters, and it's still fine

QUOTE
there is a size that definately causes the crashes
its stable up to a point and then things start falling apart

Sorry, I'm just not seeing this at all
Maybe it's something else, like bad memory, overheating etc ?

[edit]
msu denoiser has a gpu option, you might want to try turning that off

Posted by: evropej Jan 22 2011, 02:26 AM
let me see if i can capture all the events and try to repeat them

i dont think its overheating, i have a h50 cooler and temps never go above 65C
i have an invidia gtx 275 which is running with the latest drivers
windows 7 ultimate 64 bit

i will try applying that setting to each filter and see what happens

and the results of applying the same filters as you have
http://evropej.com/halp/halp.html

Posted by: dloneranger Jan 22 2011, 03:07 AM
I don't see you actually setting the single framebuffer option on any of the filters

After you add eg the msu deblocking filter, select it, then press the options button->tick the single framebuffer option
Do that for all the msu's or any others that crash/show unexpected results

Here's a .vcf configuration file of it working on my pc
CODE
VirtualDub.audio.SetSource(1);
VirtualDub.audio.SetMode(0);
VirtualDub.audio.SetInterleave(1,500,1,0,0);
VirtualDub.audio.SetClipMode(1,1);
VirtualDub.audio.SetConversion(0,0,0,0,0);
VirtualDub.audio.SetVolume();
VirtualDub.audio.SetCompression();
VirtualDub.audio.EnableFilterGraph(0);
VirtualDub.video.SetInputFormat(15);
VirtualDub.video.SetOutputFormat(0);
VirtualDub.video.SetMode(3);
VirtualDub.video.SetSmartRendering(0);
VirtualDub.video.SetPreserveEmptyFrames(0);
VirtualDub.video.SetFrameRate2(0,0,1);
VirtualDub.video.SetIVTC(0, 0, 0, 0);
VirtualDub.video.SetCompression();
VirtualDub.video.filters.Clear();
VirtualDub.video.filters.Add("MSU Smart Deblocking 0.8");
VirtualDub.video.filters.instance[0].SetForceSingleFBEnabled(true);
VirtualDub.video.filters.instance[0].Config(1,1,1,1);
VirtualDub.video.filters.Add("resize");
VirtualDub.video.filters.instance[1].Config(600,600,1,4,3,1,320,240,4,3,0,7,8,0x000000);
VirtualDub.video.filters.Add("MSU Deblocking 2.2");
VirtualDub.video.filters.instance[2].SetForceSingleFBEnabled(true);
VirtualDub.video.filters.instance[2].Config(1, 1, 1, 1);
VirtualDub.video.filters.Add("smoother");
VirtualDub.video.filters.instance[3].Config(2000,0);
VirtualDub.video.filters.Add("resize");
VirtualDub.video.filters.instance[4].Config(360,203.077,0,4,3,1,320,240,4,3,0,4,8,0x000000);
VirtualDub.video.filters.Add("smart smoother HiQuality (2.11)");
VirtualDub.video.filters.instance[5].Config(5, 50, 0, 254, 1, 0, 1, 0);
VirtualDub.video.filters.Add("warp sharp");
VirtualDub.video.filters.instance[6].Config(52, 2);
VirtualDub.video.filters.Add("MSU Denoiser 2.5.1");
VirtualDub.video.filters.instance[7].SetForceSingleFBEnabled(true);
VirtualDub.video.filters.instance[7].ConfigAuto(0, 1, 50, 0);
VirtualDub.video.filters.Add("MSU Cartoon Restore 1.1");
VirtualDub.video.filters.instance[8].SetForceSingleFBEnabled(true);
VirtualDub.video.filters.instance[8].Config("fuzzy", 6, 6, 4, 4, 2, 2, 4, 6, 8, 10, 0);
VirtualDub.audio.filters.Clear();


Or, you can download this copy of my working folder
http://rapidshare.com/files/443867971/VirtualDubForEvropej.zip
Just run this virtualdub.exe, and it'll load the filter's automatically
See if that also crashes, it shouldn't as it's exactly the working version I use

Posted by: evropej Jan 22 2011, 04:26 AM
Tried the version you provided and no output, gray.
Tried setting to single buffer same sequence as before on every filter and crashed just like the video.
Is there a setting in vdub which is unstable?
I turned video gpu acc off for example.

Thanks for all the help so far.

Posted by: dloneranger Jan 22 2011, 04:33 AM
The only one I can think of that might possibly affect anything is in the option->preferences->3daccel->enable 3d video filter acceleration
Try turning that off

Possibly turning off directx in preferences->display ???

After that I'm stumped - totally out of ideas dry.gif

The one I uploaded didn't crash though?
So maybe you have a corrupted file somewhere?

ps
with the setting I used it does take quite a while to show the first frame - did you try waiting?
if the status bar says 'decoding frame' then it's still processing
if you don't have enough memory, the status bar will say 'out of memory'

Posted by: evropej Jan 22 2011, 06:01 AM
the only you uploaded did not crash, just gray screen.
directx is now off and still have a gray screen.
i have 12 gigs or memory and 768 of video memory.
i do see an error in the bottom which says it 'out of memory'.
how can that be?
vdub is showing only 53 megs in task manager.

Posted by: dloneranger Jan 22 2011, 01:44 PM
That's the problem then - it's trying to using more than 2Gb of ram and failing -
Lower the resize until it's ok

If virtualdubs multithreading is set to more than 1 thread, 1 frame - you'll increase it's memory use enormously

After it fails to get the memory in needs, the error message is shown and the rest of the memory it allocated is released

You'd need the 64bit version of virtualdub to access all your memory, but then you can't use the filters anyway.....

Posted by: ale5000 Jan 22 2011, 04:51 PM
If you have a 64-bit OS you can make available 4GB of RAM to the 32-bit VirtualDub.

You only need to set the largeaddressaware flag in the exe,
look here: http://www.gidhome.com/gid3gb/index.html (you only need to follow the instruction in the point 2 part 2 "setting this bit directly in the executable").

Posted by: dloneranger Jan 22 2011, 05:43 PM
The only problem with that can be if the app or any dlls etc (plugings and filters) don't like pointers above 2gb
Then unexpected crashes can result

Posted by: ale5000 Jan 22 2011, 05:50 PM
They should expect pointers above 2GB otherwise it is a bug that need to be fixed.

Posted by: dloneranger Jan 22 2011, 05:54 PM
Yup it's a programming error (usually of mixing unsigned and signed maths)
Fixing it's easy, but a lot of filters etc are abandonware and don't get updated

Most are ok and should be fine though

Posted by: ale5000 Jan 22 2011, 06:11 PM
Maybe can be added a compatibility option (Limit memory to 2GB) for filters like the "Force single framebuffer" option, so some filters can get full 4GB and others can be limited to 2GB.

Posted by: evropej Jan 22 2011, 07:51 PM
Ah, so now we are getting somewhere.
I figured out from the beginning that this was a memory issue.
I can resize up to a certain dimension and then things fall apart.

The gray screen is related to memory but what about the crash?

Posted by: dloneranger Jan 22 2011, 08:11 PM
Did my one also crash?

Posted by: evropej Jan 23 2011, 03:18 AM
yes it did, here is the crash info


VirtualDub crash report -- build 35670 (release)
--------------------------------------

Disassembly:
74d1b6a0: 93 xchg eax, ebx
74d1b6a1: a9d174e878 test eax, 78e874d1
74d1b6a6: b5ff mov ch, 0ffh
74d1b6a8: ffc7 inc edi
74d1b6aa: 40 inc eax
74d1b6ab: 24bc and al, 0bch
74d1b6ad: a9d174c3cc test eax, ccc374d1
74d1b6b2: cc int 3
74d1b6b3: cc int 3
74d1b6b4: cc int 3
74d1b6b5: cc int 3
74d1b6b6: e866b5ffff call 74d16c21
74d1b6bb: 33c9 xor ecx, ecx
74d1b6bd: 8178206aa9d174 cmp dword ptr [eax+20h], 74d1a96a
74d1b6c4: 0f94c1 setz cl
74d1b6c7: 8bc1 mov eax, ecx
74d1b6c9: c3 ret
74d1b6ca: cc int 3
74d1b6cb: cc int 3
74d1b6cc: cc int 3
74d1b6cd: cc int 3
74d1b6ce: cc int 3
74d1b6cf: 8bff mov edi, edi
74d1b6d1: 55 push ebp
74d1b6d2: 8bec mov ebp, esp
74d1b6d4: 83ec50 sub esp, 50h
74d1b6d7: 8b4508 mov eax, [ebp+08h]
74d1b6da: 8945b0 mov [ebp-50h], eax
74d1b6dd: 8b450c mov eax, [ebp+0ch]
74d1b6e0: 83e001 and eax, 01h
74d1b6e3: 8945b4 mov [ebp-4ch], eax
74d1b6e6: 33c0 xor eax, eax
74d1b6e8: 8945b8 mov [ebp-48h], eax
74d1b6eb: c745bccfb6d174 mov dword ptr [ebp-44h], 74d1b6cf
74d1b6f2: 394514 cmp [ebp+14h], eax
74d1b6f5: 7423 jz 74d1b71a
74d1b6f7: 8b4510 mov eax, [ebp+10h]
74d1b6fa: 83f80f cmp eax, 0fh
74d1b6fd: 7603 jbe 74d1b702
74d1b6ff: 6a0f push 0fh
74d1b701: 58 pop eax
74d1b702: 8945c0 mov [ebp-40h], eax
74d1b705: c1e002 shl eax, 02h
74d1b708: 50 push eax
74d1b709: ff7514 push dword ptr [ebp+14h]
74d1b70c: 8d45c4 lea eax, [ebp-3ch]
74d1b70f: 50 push eax
74d1b710: e80da90200 call 74d46022
74d1b715: 83c40c add esp, 0ch
74d1b718: eb03 jmp 74d1b71d
74d1b71a: 8945c0 mov [ebp-40h], eax
74d1b71d: 8d45b0 lea eax, [ebp-50h]
74d1b720: 50 push eax
74d1b721: ff156c11d174 call dword ptr [74d1116c]
74d1b727: c9 leave <-- FAULT
74d1b728: c21000 ret 0010
74d1b72b: cc int 3
74d1b72c: cc int 3
74d1b72d: cc int 3
74d1b72e: cc int 3
74d1b72f: cc int 3
74d1b730: 8bff mov edi, edi
74d1b732: 55 push ebp
74d1b733: 8bec mov ebp, esp
74d1b735: 56 push esi
74d1b736: 8b7508 mov esi, [ebp+08h]
74d1b739: 83fef4 cmp esi, 0f4h
74d1b73c: 7218 jc 74d1b756
74d1b73e: 83fef6 cmp esi, 0f6h
74d1b741: 7713 ja 74d1b756
74d1b743: 8d4508 lea eax, [ebp+08h]
74d1b746: 50 push eax
74d1b747: 6a00 push 00h
74d1b749: 56 push esi
74d1b74a: e8dd020000 call 74d1ba2c
74d1b74f: 85c0 test eax, eax
74d1b751: 7403 jz 74d1b756
74d1b753: 8b7508 mov esi, [ebp+08h]
74d1b756: 56 push esi
74d1b757: ff153c10d174 call dword ptr [74d1103c]
74d1b75d: 5e pop esi
74d1b75e: 85c0 test eax, eax
74d1b760: 7c05 jl 74d1b767
74d1b762: 33c0 xor eax, eax
74d1b764: 40 inc eax
74d1b765: eb08 jmp 74d1b76f
74d1b767: 50 push eax
74d1b768: e8c5ae0200 call 74d46632
74d1b76d: 33c0 xor eax, eax
74d1b76f: 5d pop ebp
74d1b770: c20400 ret 0004
74d1b773: cc int 3
74d1b774: cc int 3
74d1b775: cc int 3
74d1b776: cc int 3
74d1b777: cc int 3
74d1b778: 8bff mov edi, edi
74d1b77a: 55 push ebp
74d1b77b: 8bec mov ebp, esp
74d1b77d: 8b450c mov eax, [ebp+0ch]
74d1b780: 83f8f4 cmp eax, 0f4h
74d1b783: 742c jz 74d1b7b1
74d1b785: 83f8f5 cmp eax, 0f5h
74d1b788: 7416 jz 74d1b7a0
74d1b78a: 83f8f6 cmp eax, 0f6h
74d1b78d: 7531 jnz 74d1b7c0
74d1b78f: 64a118000000 mov eax, fs:[00000018]
74d1b795: 8b4030 mov eax, [eax+30h]
74d1b798: 8b4010 mov eax, [eax+10h]
74d1b79b: 8b4018 mov eax, [eax+18h]
74d1b79e: eb20 jmp 74d1b7c0

Built on Dlone on Sat Jan 22 02:52:51 2011 using compiler version 1500

Windows 6.1 (Windows Vista x64 build 7600) []

EAX = 0018f500
EBX = 02a30000
ECX = 00000003
EDX = 00000000
EBP = 0018f550
ESI = 0467e424
EDI = 02a30000
ESP = 0018f500
EIP = 74d1b727
EFLAGS = 00200216
FPUCW = 027f
FPUTW = ffff

Crash reason: Unhandled Microsoft C++ Exception

Crash context:
An exception occurred in module 'KERNELBASE'...

...while starting filter "MSU Denoiser 2.5.1" (FilterInstance.cpp:1218).

Pointer dumps:

EAX 0018f500: e06d7363 00000001 00000000 74d1b727 00000003 19930520 0018f598 04679d00
EBX 02a30000: 00ffffff 00ffffff 00ffffff 00ffffff 00ffffff 00ffffff 00ffffff 00ffffff
ESI 0467e420: 00000000 046732a4 04673320 00000000 00000001 400ba306 00000001 00000000
EDI 02a30000: 00ffffff 00ffffff 00ffffff 00ffffff 00ffffff 00ffffff 00ffffff 00ffffff
ESP 0018f500: e06d7363 00000001 00000000 74d1b727 00000003 19930520 0018f598 04679d00
0018f520: 00000020 04655162 04a90a10 02a30000 02a30000 0467e424 0018f544 04656071
0018f540: 0467e898 0018f584 04654607 00000008 0018f588 04650de0 e06d7363 00000001
0018f560: 00000003 0018f57c e06d7363 00000001 00000000 00000000 00000003 19930520
EBP 0018f550: 0018f588 04650de0 e06d7363 00000001 00000003 0018f57c e06d7363 00000001
0018f570: 00000000 00000000 00000003 19930520 0018f598 04679d00 0018f5a4 0465016d
0018f590: 0018f598 04679d00 046732a4 04673320 00000000 02168ac0 046462ea 0a8c0000
0018f5b0: 00000003 00000000 00384008 0464c39e 02a30000 00000000 02168ac0 00000000

Thread call stack:
74d1b727: KERNELBASE!RaiseException [74d10000+b6cf+58]
74d1b727: KERNELBASE!RaiseException [74d10000+b6cf+58]
04655162: MSUDenoiser251!00015162
04656071: MSUDenoiser251!00016071
04654607: MSUDenoiser251!00014607
04650de0: MSUDenoiser251!00010de0
0465016d: MSUDenoiser251!0001016d
046462ea: MSUDenoiser251!000062ea
0464c39e: MSUDenoiser251!0000c39e
0464c50c: MSUDenoiser251!0000c50c
0464c1d5: MSUDenoiser251!0000c1d5
04668398: MSUDenoiser251!VirtualdubFilterModuleInit2 [04640000+28000+398]
04667f2d: MSUDenoiser251!00027f2d
0046aca4: FilterInstance::StartInner()
0046a841: FilterInstance::Start()
006a81e6: malloc()
006a8452: (special)()
005f55c9: VDSignalBase::signal()
006a8714: (special)()
0047924c: FilterSystem::ReadyFilters()
7716e1cc: ntdll!RtlInitUnicodeString [77140000+2e068+164]
004bafc8: VDProject::StartFilters()
00479a22: FilterSystem::DeinitFilters()
0042fb23: InitVideoStreamValuesStatic()
004baab4: VDProject::StopFilters()
004bb802: VDProject::UpdateDubParameters()
004c266e: VDProjectUI::SetVideoFiltersAsk()
75717182: USER32!GetWindowLongW [75700000+16fed+195]
75717182: USER32!GetWindowLongW [75700000+16fed+195]
7572304f: USER32!GetDCEx [75700000+23001+4e]
7715f87a: ntdll!NtCallbackReturn [77140000+1f868+12]
75723064: USER32!GetDCEx [75700000+23001+63]
75717027: USER32!GetWindowLongW [75700000+16fed+3a]
004c392d: VDProjectUI::MenuHit()
71e60b96: uxtheme!00010b96
757161d3: USER32!gapfnScSendMessage [75700000+15fc8+20b]
75716ef4: USER32!GetClassNameW [75700000+16b04+3f0]
75716ec0: USER32!GetClassNameW [75700000+16b04+3bc]
7571c9da: USER32!GetWindowLongA [75700000+1c9ac+2e]
75716ec0: USER32!GetClassNameW [75700000+16b04+3bc]
004e2a30: VDUIFrame::DefProc()
004c5c9b: _catch$?MainWndProc@VDProjectUI@@IAEJIIJ@Z$0()
75716941: USER32!gapfnScSendMessage [75700000+15fc8+979]
7571c9da: USER32!GetWindowLongA [75700000+1c9ac+2e]
004c5a2b: VDProjectUI::WndProc()
004e30f7: VDUIFrame::StaticWndProc()
75716901: USER32!gapfnScSendMessage [75700000+15fc8+939]
75716899: USER32!gapfnScSendMessage [75700000+15fc8+8d1]
75716941: USER32!gapfnScSendMessage [75700000+15fc8+979]
75716238: USER32!gapfnScSendMessage [75700000+15fc8+270]
757168ea: USER32!gapfnScSendMessage [75700000+15fc8+922]
75716899: USER32!gapfnScSendMessage [75700000+15fc8+8d1]
75717d31: USER32!LoadStringW [75700000+17c12+11f]
7573eec0: USER32!PeekMessageA [75700000+3ed58+168]
75718112: USER32!DispatchMessageA [75700000+18103+f]
0049b42b: WinMain@16()
006b03cd: __set_flsgetvalue()
006a84b0: _LocaleUpdate::_LocaleUpdate()
006abc73: __tmainCRTStartup()
750a3677: kernel32!BaseThreadInitThunk [75090000+13665+12]
77179d42: ntdll!RtlInitializeExceptionChain [77140000+39cdf+63]
77179d15: ntdll!RtlInitializeExceptionChain [77140000+39cdf+36]

-- End of report

Posted by: dloneranger Jan 23 2011, 03:56 AM
I'm out of ideas then
It doesn't crash here

Posted by: evropej Jan 23 2011, 11:45 PM
The exact configuration does not crash. Add more filters to make it crash.

Posted by: dloneranger Jan 24 2011, 12:21 AM
Ok, added more filters, no crash, added more, no crash

I think I've reached the end of this discussion, as all your 'this crashes' examples don't crash here
And if I can't get them to crash, then I can't help pinpoint any error

Posted by: evropej Jan 24 2011, 01:29 AM
I understand. Thanks for the help so far. I will try to get a sequence going when I get home.thanks for the help.

Posted by: dloneranger Jan 24 2011, 01:51 AM
Finally got it to crash !!

I'm just looking through the source code now and seeing what should be submitted as possible problem

It seems to be triggered after an 'out of memory' error
Virtualdub can attempt to access a non-existent frame buffer

Unfortunately I'm only a beginner at c++, so while I can find the crash, I'll let Phaeron fix it properly


Posted by: evropej Jan 24 2011, 02:39 PM
I forgot to mention but how did you get vdub to open mov files automatically without entering the type and extended options?
Also, how do you set the filter set to appear automatically?

Out of this whole discussion, one great useful fact "Force single framebuffer" made some of my filter work finally and come crashes stopped.
Thanks for all the input thus far.

Posted by: dloneranger Jan 24 2011, 03:51 PM
QUOTE
I forgot to mention but how did you get vdub to open mov files automatically without entering the type and extended options?
Also, how do you set the filter set to appear automatically?


It's my modified version of virtualdub - it does some useful things and some bizarre ones
The only reason I wanted you to try that was to narrow down the point of failure that causes the crash - I wasn't sure if any changes I had made were causing it to crash less than the normal one


I posted a bug report on it, hopefully Phaeron can track it down from that smile.gif

Posted by: evropej Jan 25 2011, 05:10 AM
Thanks rolleyes.gif
I hope he fixes the bug because loading all those filters takes time lol

check this out lol
http://www.evropej.com/lol/lol.html

Posted by: phaeron Jan 30 2011, 01:45 AM
Basically, the way it works is that if you have a filter that needs the "force single framebuffer" option to work, then bad things can happen if it isn't -- specifically, the filter can corrupt memory when it runs. I need to put in an option to make this sticky by default for some filters.

Posted by: evropej Jan 31 2011, 03:22 AM
What if the option is already selected and there is a crash?

Side note, why do I get better performance with lower number of video buffers? I have 12 gigs of ram lol.

Posted by: ale5000 Jan 31 2011, 06:27 AM
If you are using the 32-bit version your 12GB are useless, the only way to use them is to use the 64-bit version.

Posted by: evropej Feb 1 2011, 05:45 PM
Yes thank you, I do realize that.
If you notice, I wrote that if windows xp supported more than 3 gigs of ram, I would not use a Windows 7 64 bit.
Point was, I have a lot of ram available.

Posted by: ale5000 Feb 1 2011, 08:38 PM
My point was mainly that if you use a 32-bit VirtualDub on a 64-bit Windows then your 12GB still can't be used biggrin.gif

However there is also Windows XP x64

Posted by: evropej Feb 2 2011, 02:13 AM
I have tried windows xp 64 and was not impressed with the support.
In either case, I hope the big chief decides to fix the issue.

Posted by: phaeron Feb 5 2011, 07:13 PM
QUOTE (evropej @ Jan 30 2011, 08:22 PM)
Side note, why do I get better performance with lower number of video buffers? I have 12 gigs of ram lol.

How many video buffers do you have? If you're running out of memory, this may be part of it.

Posted by: evropej Feb 6 2011, 12:15 AM
I cranked the video memory up and buffers all the way to right when I work on pixelation/resolution issues.
I usually have to crank these all the way to the left when I use deshaker.
Deshaker works much better on playback with minimal buffers.
The crashes still happen though no matter what buffer option I choose.

Posted by: phaeron Feb 13 2011, 11:35 PM
Found the problem. It appears to be a bug in the code that merges filter allocators, causing the smaller frames to use larger buffers than necessary. Unfortunately my dev branch is not in a good state to push a test release, but there are two fixes required.

FilterInstance.cpp:
CODE
void FilterInstance::RegisterAllocatorProxies(VDFilterFrameAllocatorManager *mgr, VDFilterFrameAllocatorProxy *prev) {
mSourceAllocator.Clear();

if (!(mFlags & FILTERPARAM_SWAP_BUFFERS))
 mResultAllocator.Link(prev);

mgr->AddAllocatorProxy(&mSourceAllocator);

if (!mbAccelerated)
 mSourceAllocator.AddSizeRequirement(mRealSrc.size + mRealSrc.offset);

prev->AddBorderRequirement(mRealSrc.mBorderWidth, mRealSrc.mBorderHeight);


FilterFrameAllocatorManager.cpp:
CODE

  } else {
   VDASSERT(ent1.mMaxSize <= ent2.mMinSize);

   float mergeRatio = 0.f;

   if (ent2.mMaxSize)
    mergeRatio = (float)ent1.mMinSize / (float)ent2.mMaxSize;

   if (mergeRatio > bestMergeRatio) {
    bestMerge = i;
    bestMergeRatio = mergeRatio;
   }
  }


In the filter chain I tested, this reduces memory usage by about half, so it won't solve all memory problems, but it should give more breathing room. A significant amount of memory is also being taken up by internal buffers in the filters themselves, which these fixes won't change. Beyond that I think the only quick fix might be to mark the executable /LARGEADDRESSAWARE, which on x64 systems will raise the 32-bit process commit limit from about 1.6GB to about 3.8GB. I haven't tried doing that yet, so I have no idea what might break (there can be compatibility problems with some DLLs, such as D3DX).

If you're still hitting the problem, please open Task Manager while the crash dialog is still up, enable the VM Size (Windows XP) or Process Commit Size (Windows Vista/7) column, and check how much memory VirtualDub is using.

Posted by: evropej Feb 20 2011, 12:20 AM
Thanks for the update, I will give it a try later.
Right now I have to wait for a day for all the videos to deshake from my ski trip.
I am putting this core i7 to work lol.

Posted by: evropej Feb 20 2011, 07:48 PM
I had to work hard to get it to crash but I did.
I am working on my machine with a monitor which supports a much smaller resoultion same machine though.

Anyway, the size used was 650 megs but the commit size was 1.35 gigs.
I am using 1.1 smile.gif

VirtualDub crash report -- build 33848 (release)
--------------------------------------

Disassembly:
091a2c80: b614 mov dh, 14h
091a2c82: 380f cmp [edi], cl
091a2c84: b60c mov dh, 0ch
091a2c86: 18895424300f sbb [ecx+f302454], cl
091a2c8c: b614 mov dh, 14h
091a2c8e: 28894c24348b sub [ecx-74cbdbb4], cl
091a2c94: 4c dec esp
091a2c95: 2454 and al, 54h
091a2c97: 2911 sub [ecx], edx
091a2c99: 83410cff add dword ptr [ecx+0ch], 0ffh
091a2c9d: 8b542434 mov edx, [esp+34h]
091a2ca1: 295104 sub [ecx+04h], edx
091a2ca4: 8b542430 mov edx, [esp+30h]
091a2ca8: 295108 sub [ecx+08h], edx
091a2cab: 0fb61418 movzx edx, byte ptr [eax+ebx]
091a2caf: 0fb60c38 movzx ecx, byte ptr [eax+edi]
091a2cb3: 89542454 mov [esp+54h], edx
091a2cb7: 0fb61428 movzx edx, byte ptr [eax+ebp]
091a2cbb: 0116 add [esi], edx
091a2cbd: 8b542454 mov edx, [esp+54h]
091a2cc1: 015604 add [esi+04h], edx
091a2cc4: 99 cdq
091a2cc5: f77c244c idiv eax, dword ptr [esp+4ch]
091a2cc9: 014e08 add [esi+08h], ecx
091a2ccc: 83460c01 add dword ptr [esi+0ch], 01h
091a2cd0: 85d2 test edx, edx
091a2cd2: 0f85bbf9ffff jnz 091a2693
091a2cd8: 5e pop esi
091a2cd9: 5f pop edi
091a2cda: 5d pop ebp
091a2cdb: 5b pop ebx
091a2cdc: 83c428 add esp, 28h
091a2cdf: c3 ret
091a2ce0: 81ec88000000 sub esp, 00000088
091a2ce6: 53 push ebx
091a2ce7: 55 push ebp
091a2ce8: 56 push esi
091a2ce9: 8bb424a8000000 mov esi, [esp+a8]
091a2cf0: 8bee mov ebp, esi
091a2cf2: 0fafac24ac0000 imul ebp, [esp+ac]
00
091a2cfa: 57 push edi
091a2cfb: 8bfd mov edi, ebp
091a2cfd: c1e704 shl edi, 04h
091a2d00: 57 push edi
091a2d01: e895810000 call 091aae9b
091a2d06: 8bd8 mov ebx, eax
091a2d08: 8bcf mov ecx, edi
091a2d0a: 8bd1 mov edx, ecx
091a2d0c: c1e902 shr ecx, 02h
091a2d0f: 33c0 xor eax, eax
091a2d11: 8bfb mov edi, ebx
091a2d13: f3ab rep stosd <-- FAULT
091a2d15: 8bca mov ecx, edx
091a2d17: 8b9424a0000000 mov edx, [esp+a0]
091a2d1e: 83e103 and ecx, 03h
091a2d21: f3aa rep stosb
091a2d23: 8bcd mov ecx, ebp
091a2d25: 33c0 xor eax, eax
091a2d27: 8bfa mov edi, edx
091a2d29: f3ab rep stosd
091a2d2b: 8b8424a4000000 mov eax, [esp+a4]
091a2d32: b901000000 mov ecx, 00000001
091a2d37: 890a mov [edx], ecx
091a2d39: 0fb638 movzx edi, byte ptr [eax]
091a2d3c: 017b10 add [ebx+10h], edi
091a2d3f: 8bbc24a8000000 mov edi, [esp+a8]
091a2d46: 0fb62f movzx ebp, byte ptr [edi]
091a2d49: 016b14 add [ebx+14h], ebp
091a2d4c: 8bac24ac000000 mov ebp, [esp+ac]
091a2d53: 0fb66d00 movzx ebp, byte ptr [ebp+00h]
091a2d57: 016b18 add [ebx+18h], ebp
091a2d5a: 014b1c add [ebx+1ch], ecx
091a2d5d: 8bac24bc000000 mov ebp, [esp+bc]
091a2d64: 83c404 add esp, 04h
091a2d67: 3bf1 cmp esi, ecx
091a2d69: 895c241c mov [esp+1ch], ebx
091a2d6d: 8b9c24bc000000 mov ebx, [esp+bc]
091a2d74: 894c2478 mov [esp+78h], ecx
091a2d78: 894c2428 mov [esp+28h], ecx
091a2d7c: 0f db 0fh
091a2d7d: 8e7601 mov ?6s, [esi+01h]

Built on Aegis on Fri Dec 24 13:11:24 2010 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7600) []

EAX = 00000000
EBX = 00000000
ECX = 0146d000
EDX = 051b4000
EBP = 0051b400
ESI = 00000a60
EDI = 00000000
ESP = 0018fbe0
EIP = 091a2d13
EFLAGS = 00210246
FPUCW = 027f
FPUTW = ffff

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'MSU_Cartoonizer'...

...writing address 00000000...

...while running filter "MSU Cartoonizer v 3.0" (FilterInstance.cpp:1954).

Pointer dumps:

EDX 051b4000: 00000000 00000000 00000000 6c720000 72206d68 706c6265 00006170 00001000
ESP 0018fbe0: 051b4000 071c3c84 071c3c58 00000000 071c3c78 48949520 000007dd 00000001
0018fc00: 4894a9e0 4b769f81 00000a5d 00000a5f 000014c0 00002980 000000ff 00000000
0018fc20: 00000a5f 4b769f80 000007de 4db45380 4b76a9e0 4b76a9e0 00000a50 000007de
0018fc40: 00000a5d 4d105380 00000a5f 00000a60 000007e0 00000a60 000007e0 000014c0
EBP 0051b400: 244c8d00 d0b6e814 4c8dfff2 44c70c24 6a241024 4c890062 8d561c24 c714244c
0051b420: 002c2444 e8000000 fff2da24 4c8dc085 950f1024 2444c7c3 ffffff28 d09ee8ff
0051b440: db84fff2 8b1c745b 89082454 b05f0857 4c8b5e01 89641424 0000000d 20c48300
0051b460: 8b0004c2 5f1c244c 645ec032 00000d89 c4830000 0004c220 cccccccc cccccccc

Thread call stack:
091a2d13: MSU_Cartoonizer!VirtualdubFilterModuleDeinit [09110000+3380+8f993]
0911471c: MSU_Cartoonizer!VirtualdubFilterModuleDeinit [09110000+3380+139c]
0911da94: MSU_Cartoonizer!VirtualdubFilterModuleDeinit [09110000+3380+a714]
091134df: MSU_Cartoonizer!VirtualdubFilterModuleDeinit [09110000+3380+15f]
0043d153: FilterInstance::RunFilterInner()
00438e10: VDFilterFrameManualSource::GetNextRequest()
0043eec6: _catch$?RunFilter@FilterInstance@@IAEXXZ$0()
0043f60a: FilterInstance::RunProcess()
00444431: FilterSystem::Run()
0046aa96: VDProject::UpdateFrame()
76666941: USER32!gapfnScSendMessage [76650000+15fc8+979]
76666901: USER32!gapfnScSendMessage [76650000+15fc8+939]
76667d31: USER32!LoadStringW [76650000+17c12+11f]
7667028d: USER32!PeekMessageW [76650000+20112+17b]
76670243: USER32!PeekMessageW [76650000+20112+131]
7668eec0: USER32!PeekMessageA [76650000+3ed58+168]
0046d613: VDProject::Tick()
0045a4fc: WinMain@16()
005d66cb: __tmainCRTStartup()
76763677: kernel32!BaseThreadInitThunk [76750000+13665+12]
77179f02: ntdll!RtlInitializeExceptionChain [77140000+39e9f+63]
77179ed5: ntdll!RtlInitializeExceptionChain [77140000+39e9f+36]

-- End of report

Posted by: phaeron Feb 20 2011, 09:17 PM
Yup, out of address space.

Try the 1.10.1 test-2 release in the testing forum... it contains the code fixes I outlined and should help a bit.

Posted by: evropej Feb 21 2011, 05:46 PM
Just ran the test build.
First, there is a significant performance increase at least in preview mode.
Second, I had to try really hard to get it to crash, loaded about 12 to 14 filters before it crashed.

Is there something with warp sharp that is causing this?
I say this because I have noticed that this filter in the past gave me issues.

I will try to run without warp sharp and see what happens.

VirtualDub crash report -- build 34005 (release)
--------------------------------------

Disassembly:
04852760: 5a pop edx
04852761: 080f or [edi], cl
04852763: f5 cmc
04852764: 6208 bound ecx,[eax]
04852766: 0ffecb paddd mm1, mm3
04852769: 0ffed4 paddd mm2, mm4
0485276c: 0ffe0d08418504 paddd mm1, [04854108]
04852773: 0ffe1508418504 paddd mm2, [04854108]
0485277a: 0f72e10e psrad mm1, 0eh
0485277e: 0f72e20e psrad mm2, 0eh
04852782: 0f6bca packssdw mm1, mm2
04852785: 03c1 add eax, ecx
04852787: 0f6e10 movd mm2, [eax]
0485278a: 0f6e7004 movd mm6, [eax+04h]
0485278e: 0f60d7 punpcklbw mm2, mm7
04852791: 0f7fd3 movq mm3, mm2
04852794: 0f60f7 punpcklbw mm6, mm7
04852797: 0f61d6 punpcklwd mm2, mm6
0485279a: 0f69de punpckhwd mm3, mm6
0485279d: 0ff512 pmaddwd mm2, [edx]
048527a0: 0ff51a pmaddwd mm3, [edx]
048527a3: 0f6e6008 movd mm4, [eax+08h]
048527a7: 0f6e700c movd mm6, [eax+0ch]
048527ab: 0f60e7 punpcklbw mm4, mm7
048527ae: 0f7fe5 movq mm5, mm4
048527b1: 0f60f7 punpcklbw mm6, mm7
048527b4: 0f61e6 punpcklwd mm4, mm6
048527b7: 0f69ee punpckhwd mm5, mm6
048527ba: 0ff56208 pmaddwd mm4, [edx+08h]
048527be: 0ff56a08 pmaddwd mm5, [edx+08h]
048527c2: 0ffed4 paddd mm2, mm4
048527c5: 0ffedd paddd mm3, mm5
048527c8: 0ffe1508418504 paddd mm2, [04854108]
048527cf: 0ffe1d08418504 paddd mm3, [04854108]
048527d6: 0f72e20e psrad mm2, 0eh
048527da: 0f72e30e psrad mm3, 0eh
048527de: 0f6bd3 packssdw mm2, mm3
048527e1: 03c1 add eax, ecx
048527e3: 0f6e18 movd mm3, [eax] <-- FAULT
048527e6: 0f6e7004 movd mm6, [eax+04h]
048527ea: 0f60df punpcklbw mm3, mm7
048527ed: 0f7fdc movq mm4, mm3
048527f0: 0f60f7 punpcklbw mm6, mm7
048527f3: 0f61de punpcklwd mm3, mm6
048527f6: 0f69e6 punpckhwd mm4, mm6
048527f9: 0ff51a pmaddwd mm3, [edx]
048527fc: 0ff522 pmaddwd mm4, [edx]
048527ff: 0f6e6808 movd mm5, [eax+08h]
04852803: 0f6e700c movd mm6, [eax+0ch]
04852807: 0f60ef punpcklbw mm5, mm7
0485280a: 0f60f7 punpcklbw mm6, mm7
0485280d: 0f7fef movq mm7, mm5
04852810: 0f61ee punpcklwd mm5, mm6
04852813: 0f69fe punpckhwd mm7, mm6
04852816: 0ff56a08 pmaddwd mm5, [edx+08h]
0485281a: 0ff57a08 pmaddwd mm7, [edx+08h]
0485281e: 0ffedd paddd mm3, mm5
04852821: 0ffee7 paddd mm4, mm7
04852824: 0ffe1d08418504 paddd mm3, [04854108]
0485282b: 0ffe2508418504 paddd mm4, [04854108]
04852832: 0f72e30e psrad mm3, 0eh
04852836: 0f72e40e psrad mm4, 0eh
0485283a: 0f6bdc packssdw mm3, mm4
0485283d: 8b542410 mov edx, [esp+10h]
04852841: 0f7fc4 movq mm4, mm0
04852844: 0f7fd5 movq mm5, mm2
04852847: 0f61c1 punpcklwd mm0, mm1
0485284a: 0f69e1 punpckhwd mm4, mm1
0485284d: 0f61d3 punpcklwd mm2, mm3
04852850: 0f69eb punpckhwd mm5, mm3
04852853: 0ff502 pmaddwd mm0, [edx]
04852856: 0ff522 pmaddwd mm4, [edx]
04852859: 0ff55208 pmaddwd mm2, [edx+08h]
0485285d: 0f db 0fh
0485285e: f5 cmc
0485285f: 6a db 6ah

Built on Althena on Sat Feb 19 22:12:40 2011 using compiler version 1400

Windows 6.1 (Windows 7 x64 build 7600) []

EAX = 5477ce80
EBX = 5476e790
ECX = 00002680
EDX = 04855e50
EBP = 517503a4
ESI = 00000000
EDI = 5aa4e790
ESP = 0018fc58
EIP = 048527e3
EFLAGS = 00210202
FPUCW = 027f
FPUTW = aaaa

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'warpsharp'...

...reading address 5477CE80...

...while running filter "warp sharp" (FilterInstance.cpp:1960).

Pointer dumps:

EBX 5476e790: 000f0b27 000f0b27 000f0b27 000f0b28 000f0b28 00100b28 00100c29 000f0c29
EDX 04855e50: 0af1fedd 0af1fedd fb323b00 fb323b00 0aa9fee7 0aa9fee7 fb3f3b31 fb3f3b31
EDI 5aa4e790: 00171322 00171322 00171322 00171323 00171322 00171322 00171322 00171323
ESP 0018fc58: 04852f0d 54775b00 00002680 04855e50 04855450 0c40bbc8 0c40bbb8 00000000
0018fc78: 00000000 007297a0 00002680 fffffd00 00099dff 5174fa03 003ef040 003ef9e0
0018fc98: fff97600 000006ff 00000330 00000998 000009a0 00000001 00099dff ffffecc0
0018fcb8: fffff661 000009a0 0043e3a3 0c40bbc8 006712c8 0072a61c 0c40bbb8 0018fdb0
EBP 517503a0: 524a423d 5f5e5b58 60606060 60606060 60606060 5f606060 6060605f 60606060
517503c0: 60606060 60606060 60606060 5f606060 5c5c5d5e 5c5c5c5c 5b5b5b5c 5b5b5b5b
517503e0: 5b5b5b5b 5b5b5b5b 5c5c5c5b 5c5c5c5c 5d5d5d5d 5e5e5e5d 5c5c5d5d 5b5b5b5b
51750400: 5b5b5b5b 5a5a5a5a 5b5b5b5a 5a5a5a5a 5a5a5a5a 5a5a5959 5959595a 5a5a5a5a

Thread call stack:
048527e3: warpsharp!VirtualdubFilterModuleDeinit [04850000+26a0+143]
04852f0d: warpsharp!VirtualdubFilterModuleDeinit [04850000+26a0+86d]
0043e3a3: FilterInstance::RunFilterInner()
0043a0e0: VDFilterFrameManualSource::GetNextRequest()
0043acff: VDFilterFrameRequest::AreSourcesReady()
004400f6: _catch$?RunFilter@FilterInstance@@IAEXXZ$0()
0044083a: FilterInstance::RunProcess()
00445661: FilterSystem::Run()
0046ca46: VDProject::UpdateFrame()
771500e6: ntdll!KiUserCallbackDispatcher [77140000+100b8+2e]
7667028d: USER32!PeekMessageW [76650000+20112+17b]
7668eed1: USER32!PeekMessageA [76650000+3ed58+179]
76667d4f: USER32!LoadStringW [76650000+17c12+13d]
7668eec0: USER32!PeekMessageA [76650000+3ed58+168]
0046f733: VDProject::Tick()
0045c28c: WinMain@16()
005dd31b: __tmainCRTStartup()
76763677: kernel32!BaseThreadInitThunk [76750000+13665+12]
77179f02: ntdll!RtlInitializeExceptionChain [77140000+39e9f+63]
77179ed5: ntdll!RtlInitializeExceptionChain [77140000+39e9f+36]

-- End of report


Posted by: phaeron Feb 28 2011, 12:03 AM
At some point you will always be able to get an out of memory condition, especially with these huge framebuffers... it's just a question of when it happens and how gracefully the error is handled.

In this case, I'm not sure the problem is out of memory... it looks like an out of bounds read in the warp sharp filter. Have you tried using the internal one? It's built-in in current versions, and the built-in one is newer than the external version you appear to be using.

Posted by: evropej Mar 7 2011, 05:21 PM
The reason I like going to such large sizes is the huge benefit of resizing with lanczos.
I have found a fine line where all the filters seem to work correctly.

I will take your advise and use the internal warp sharp.

Thanks for all the follow ups.
cool.gif

Posted by: Jam One Mar 7 2011, 05:34 PM
evropej -- do you think Lanczos-4 would have been of a bigger benefit to it?

Posted by: evropej Mar 7 2011, 10:32 PM
Jam One
I am not sure. This is new territory for me (filter resizing).

I am using resizing because I am starting to realize that each filter requires a minimum image size ( good for small dimensional movies).
This also applies to filters for photoshop as well.

This is the main reason why I use Lanczos resizing.
Plus the resizing seems to get rid of artifacts generated from pixelation.

Is Lanczos 4 available for vdub?

Posted by: Jam One Mar 8 2011, 09:59 PM
1)
QUOTE
Plus the resizing seems to get rid of artifacts generated from pixelation.

This seems to be interesting. "Supersampling"...

2)
Lanczos-4 is available only via Avisynth so far.

Posted by: arklight Apr 1 2011, 11:23 AM
cool thread!


More!


Heri Mkocha

Powered by Invision Board (http://www.invisionboard.com)
© Invision Power Services (http://www.invisionpower.com)