|
|
| jpsdr |
| Posted: Dec 21 2011, 09:20 AM |
 |
|
Advanced Member
Group: Members
Posts: 335
Member No.: 20490
Joined: 23-December 06
|
Note : After (i've lost count) number of edits, this post is a total mess... sorry 
Globaly, what i've big trouble with is the fact that in an filter chain, where you can put a specific order (A B C D E), and if you have put it in this order, it's for a reason, you can have a filter which affect previous. If C is a decimate filter, doing :
A B C D E or C A B D E or A B D E C is the same thing. This is where, still from my point of view : No, there is a problem. If i chose to put C after B, it's because i want the effect of C applied only after the ouput stream of B. So, if C is a decimate filter, decimation is made only after the ouput of B. Otherwise, there is no point of having a decimate filter (or possibility of decimate) in filter, considering the fact that a filter is something you put in a chain with a specific oder. In that case, decimate should be an option only aviable on the same level as "Frame rate..." (Video frame rate control), where you know, for sure, that it will be done before the chain filter. This is still, of course, my point of view.
As i use often VDub and found it very usefull, having this behavior very annoying for make me a little...  and  .
So, if i understand properly, unless you have a change of mind in behavior  ,what work now may not in the future, so, possibilities offer to me are :
- Using the new versions of my filters, but without decimating, and going back to have to split process in two parts, using intermediate file.
- Using new version and decimating, but stop updating VDub version when it stop working.
- Using my old version with LAG and decimating, but will not LAG be deprecated one day ?
- Having a very hard time to redesign filter, if it's even possible.
I think i've just had a revelation... Understanding prefetch.
You said my trick may not work in the future. Question is why ?
prefetching a frame n at stage C, mean : filter C needs frame n, so, filter B must produce frame n, so filter B will process frame n, and so on going up.
... I see... i understand now why decimate filter wich preftech only decimated frames will also affect upstream, and producing gap... It make sense... I'll think i begin to understand.... .....
Now, if i decimate k frames, prefetching a windows of +/-k will make me sure that all frames will be asked to the previous filter. More optimised, but always thinking during a job/process, not a seeking : If in my decimate, i keep a previous frame variable, compare it to the actual frame i have to prefetch, and prefetch only the frames in the gap, this will also make me sure all the frames will be asked to the previous filter. So, again, why it could not work in the future ?
So, after some understanding, i would said, from my point a view, problem is not from the way things are handled now, but how a decimating filter, is designed, and mine was bad. When creating a decimating filter, supposed to be a filter in a process chain, you must be sure that it will not affect upstream, and so it must prefetch the fames it'll drop. What make me more and more wondering : Why what i've done may not work in the future........... 
#######################
Questions :
What is the status of .mFrameNumber information before and after a pure decimate filter ?
I call a pure decimate filter wich prefetch only one frame, not doing the trick i've done. Is the following correct, for 1/5 frames decimated, with 200 frames input :
- Upstream filters will only see 160 frames, but with .mFrameNumber going from 0 to 199 with gaps in numbers.
- Downstream filters will see 160 frames, but with .mFrameNumber going from 0 to 159, without gaps.
A filter which increase number of frames, for exemple double, all the upstream filters will have their frames processed doubled ???? 
Now that i have a little understood prefetch, i would said no, but if i've understood decimating, i've not yet understood multiplying, but as i've no interest on it now, i leave it aside.
If i internaly in my filter, have a previous frame frame variable, wich store .mFrameNumber just before exiting Run(). Can i, at least, consider that in the next call of the Run() procedure, i'll have previous frame<.mFrameNumber ?
This way, i may try to detect gaps, and try to fill them (even if it'll absolutely not be easy).
Of course, i'm talking of when running a batch/job process, not a seeking situation !
I have, in my deinterlace filter, a special case i use in manual mode, the 0 mode.
If in my input file i have : 100 102 0
It will mean : Apply filter mode 0 from frame 100 to 102.
Filter mode 0 consist to duplicate previous frame of window filter in the window => duplicate frame 99 in frame 100 to 102 here. Actualy it works by checking if next frame will be mode 0. So, during process of frame 99, it will detect that next frame will have filter mode 0 (but current is not mode 0), and so buffer the ouput of the frame 99 (frame 99 may have to be deinterlaced). I see a possible big unsolvable problem... How to be sure to always being able to catch the output of frame number 99 ????
If a stupid decimating  filter below, of course, make me jump from frame 70 to 101, any idea of how to solve this issue ?
#################################
Finaly, if (or when) you have time, i'd realy like a feedback from you of what you think of the results of my IVTC filter. I've tested and compared with a little video i've used to test my filter when i've ported the new version, and mine works on places where yours failled.
If interested by the video i've used, and the specific process i've used, tell me, i'll post them and provide links. |
|
| phaeron |
| Posted: Dec 22 2011, 12:22 AM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
You can keep a previous frame number, but there is an important gotcha: you can't use it in Prefetch/Prefetch2. Why? Because the output of Prefetch/Prefetch2 has to be consistent and independent of Run(). Even in current versions, there is a delay between the time that upstream frame requests are prefetched and Run() gets called. Currently these requests get dispatched in order but this is not guaranteed. One of the features I would like to implement is 32/64-bit filter bridging, and in order to do that I would need to be able to speculatively call Prefetch[2] in order to reduce IPC overhead.
| QUOTE | I call a pure decimate filter wich prefetch only one frame, not doing the trick i've done. Is the following correct, for 1/5 frames decimated, with 200 frames input :
- Upstream filters will only see 160 frames, but with .mFrameNumber going from 0 to 199 with gaps in numbers.
- Downstream filters will see 160 frames, but with .mFrameNumber going from 0 to 159, without gaps. |
Yes, this is correct. A filter always produces the same sequence of frames -- it's just that the downstream filters may select only a subset of them.
| QUOTE | | A filter which increase number of frames, for exemple double, all the upstream filters will have their frames processed doubled ???? |
No, this shouldn't happen. VirtualDub has two mechanisms to combat this, one being a frame cache and the other being the merging of requests in the request queue. If the downstream filter requests frame 4 from the upstream filter twice, both of the requests are connected so that the frame is not released until both requests are discarded. Generally this means that when all filters run at the same frame step, each will only process frames at a 1:1 rate regardless of any prefetch overlaps.
| QUOTE | | If i internaly in my filter, have a previous frame frame variable, wich store .mFrameNumber just before exiting Run(). Can i, at least, consider that in the next call of the Run() procedure, i'll have previous frame<.mFrameNumber ? |
Usually, yes, although as you've guessed this is definitely not the case in preview windows.
| QUOTE | | Filter mode 0 consist to duplicate previous frame of window filter in the window => duplicate frame 99 in frame 100 to 102 here. Actualy it works by checking if next frame will be mode 0. So, during process of frame 99, it will detect that next frame will have filter mode 0 (but current is not mode 0), and so buffer the ouput of the frame 99 (frame 99 may have to be deinterlaced). I see a possible big unsolvable problem... How to be sure to always being able to catch the output of frame number 99 ???? |
You can't. There is no really good way around this. If you think about it, resolving this chain could effectively force all frames to be processed all the way from the start of the time line. This is one reason that VirtualDub doesn't allow you to prefetch your own filter's output (recursive filter).
There are a couple of ways around this. For many filters, the effect of previous output frames diminishes considerably over several frames, and so it's OK to preroll only from a certain distance. This is the logic used in the current runtime for laggy filters, and the way you would replicate this with prefetch is to prefetch N frames and use those frames if necessary to prime internal buffers, using a previous frame check as you have described. For this to work you need to prefetch those frames always since you don't know at that point if you will need them.
The other way, which is more directly applicable in your case of "duplicate previous frame," is to precompute or compute in prefetch the original output frame and use that instead. Since you know frames 100-102 are the same as frame 99, you can just prefetch for frame 99 instead. The cookies in the prefetches can be set so that Run() knows this is happening instead of having to re-deduce this.
|
|
| jpsdr |
| Posted: Dec 22 2011, 09:01 AM |
|
|
Advanced Member
Group: Members
Posts: 335
Member No.: 20490
Joined: 23-December 06
|
First, thanks for all your answers and time, i think i see some things clearer.
| QUOTE | | You can keep a previous frame number, but there is an important gotcha: you can't use it in Prefetch/Prefetch2. Why? Because the output of Prefetch/Prefetch2 has to be consistent and independent of Run(). |
For a decimate of k frame each n frame :
My idea was the following : previous frame initialised to -1 in start proc.
In prefetch2 : compute the frame i want to ouput. If previous frame+1 != frame => prefetch the k previous frames. I think i don't need to prefetch the next frames indeed. Prefetching from previous frame to actual frame may result in a disaster in case of seeking...
And after, still in prefetch2 function, update previous frame to frame. Independant to Run().
Can this, still only in a job process, not seeking, guarantee that all frame will be processed and in order by upstream filters ? If not, why ?
Edit : Bad idea... I'll stay with always prefetch +/- k windows.
| QUOTE | | he other way, which is more directly applicable in your case of "duplicate previous frame," is to precompute or compute in prefetch the original output frame and use that instead. Since you know frames 100-102 are the same as frame 99, you can just prefetch for frame 99 instead. The cookies in the prefetches can be set so that Run() knows this is happening instead of having to re-deduce this. |
Ok... found no real specific information on SDK v1.1 (Is there a new version btw ?).
So :
| CODE | | prefetcher->PrefetchFrame(0,frame,0); |
Where is the cookie ? How can it be retrieve in Run() proc ?
Do you think you'll have time to provide me feedback (what you think of) of tests results of my IVTC filter, or you will not have time or are you just not interested in ? Don't pushing, just want to know what to expect. |
|
| phaeron |
| Posted: Dec 22 2011, 10:06 PM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
| QUOTE | In prefetch2 : compute the frame i want to ouput. If previous frame+1 != frame => prefetch the k previous frames. I think i don't need to prefetch the next frames indeed. Prefetching from previous frame to actual frame may result in a disaster in case of seeking...
And after, still in prefetch2 function, update previous frame to frame. Independant to Run(). |
Sorry, but this is not allowed. It is not allowed because you are not guaranteed that prefetching happens in order or that there is a Run() call for every prefetch, and therefore your previous frame prediction in prefetch may be wrong. This will break if I implement speculative prefetch caching or multithreaded prefetching.
Also, I'd like to remind you that prefetch is required to be thread-safe, for similar reasons.
| QUOTE | | Can this, still only in a job process, not seeking, guarantee that all frame will be processed and in order by upstream filters ? If not, why ? |
Nope. This is not guaranteed for three reasons:
- There is no requirement that a filter's frame mapping is forward. You can implement a filter to play video backwards right now by prefetching (N-1-i); it's just marginally useful because you can't reverse the audio from a video filter.
- Sections can be repeated in the timeline, including single frames. You can create a script that reverses a video one frame at a time (again with similar caveats with regard to audio).
- There is no requirement that filters prefetch frames in ascending order. Currently VirtualDub processes prefetches and frame requests in order, and so if the upstream filter requests frames backwards, they will be processed FIFO. (This is not good for performance, so for this reason, you should prefetch in ascending order.)
| QUOTE | | Ok... found no real specific information on SDK v1.1 (Is there a new version btw ?). |
Working on it. I just did a big conversion to a new XML/.NET-based help builder, so I'm still working out the kinks.
| QUOTE | | Where is the cookie ? How can it be retrieve in Run() proc ? |
The cookie is the third argument to PrefetchFrame(). It's an arbitrary value you can supply with the prefetch request that is guaranteed to be returned with the VDXFBitmap::mCookie field that contains the corresponding frame. VirtualDub doesn't interpret this value in any way, so you can store any 64-bit value you want in it.
Be careful with storing pointers, though, since you have no idea if the prefetches will actually be used. You can't use the cookie to store pointers to allocated memory if you have no other way of freeing that memory.
| QUOTE |
Do you think you'll have time to provide me feedback (what you think of) of tests results of my IVTC filter, or you will not have time or are you just not interested in ? Don't pushing, just want to know what to expect.
|
Sorry, I'm afraid not, since I'm working on releases. I actually don't have telecined output available on my computer at the moment, anyway.
|
|
| ale5000 |
| Posted: Dec 22 2011, 11:32 PM |
|
|
Advanced Member
Group: Members
Posts: 1114
Member No.: 22180
Joined: 30-September 07
|
| QUOTE (phaeron @ Dec 22 2011, 11:06 PM) | | You can implement a filter to play video backwards right now by prefetching (N-1-i); it's just marginally useful because you can't reverse the audio from a video filter. |
Lol, I would like to see it
--------------------
New VirtualDub forum
VirtualDub AIO (All-in-One installer for VirtualDub and plugins)
Codec Toolbox RS (A tool to read/change merit of codecs and many other things)
Input plugins for VirtualDub / ACM codecs / VFW codecs |
|
| phaeron |
| Posted: Dec 23 2011, 08:21 AM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
Challenge Accepted.
Warning: This filter will perform pretty badly due to reverse decoding, which is another reason I don't encourage doing this with current versions.
|
|
| jpsdr |
| Posted: Dec 23 2011, 09:04 AM |
|
|
Advanced Member
Group: Members
Posts: 335
Member No.: 20490
Joined: 23-December 06
|
From what i've understood : If i use my filters the way they are designed, and not using any unknow filters which may change frame order in my chain, and the only decimating is mine, working as i've allready described, i should never encounter problems.
| QUOTE | There is no requirement that filters prefetch frames in ascending order. Currently VirtualDub processes prefetches and frame requests in order, and so if the upstream filter requests frames backwards, they will be processed FIFO. (This is not good for performance, so for this reason, you should prefetch in ascending order.)
|
To improve performance of my filters, globaly IVTC likes, i decided to not use SWAP_BUFFERS, because you have 3 cases :
- Nothing change.
- Only one field change.
- Both field change.
To minimize memory transfer, not using SWAP_BUFFER seems a good idea, beacause in first case you do nothing, and in second case, you move only one field.
But... effect i've discovered is in that case, that you can't prefecth in ascending order, if first frame is current-N... Because what you have in dst, is the 1rst prefecthed frame.
So prefetch is made in the following order : N, N-2,N-1,N+1,N+2,N+6,N+7 for IVTC.
Would it be better, for performance issue, to prefecth N-2, N-1, N, N+1, N+2, N+6, N+7 and always do memory transfer ? As this filter do a lot of process, and the way it's implemented now, changing the prefetch order and always forcing transfert will only take a few minutes, and time add by to transfer will probably have few effect on speed.
So, question : What is better ?
My decimate do the same thing : It 1rst prefecth N, and after N-k to N+k (except N, of course).
AS this filter does nothing, adding memory transfert will greatly increase it's processing time, and as Run() actualy is empty, i would have to add all mode transfert.
But again, what is better ?
But, doing this, despite prefetch order not in order, upstream process all frames in order.
| QUOTE | | Sorry, I'm afraid not, since I'm working on releases. I actually don't have telecined output available on my computer at the moment, anyway. |
Ok, if problem was only providing a little video source to test, i would gladly have provided a link (and of course, i was not asking to take a look at the code...). Well, you take time to answer my questions, that's already something. |
|
| jpsdr |
| Posted: Dec 23 2011, 09:10 AM |
|
|
Advanced Member
Group: Members
Posts: 335
Member No.: 20490
Joined: 23-December 06
|
I've just take a look at your backward.
Others questions :
Why the need of Prefetch, Prefetch2 is not enough ?
Is PrefetchFrameDirect more interesting to use instead of PrefetchFrame ? |
|
| phaeron |
| Posted: Dec 23 2011, 09:53 PM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
Hmmmm. This is an interesting case.
Yes, it would be slightly better if you could prefetch the in-place frame in the middle. As it turns out, it's a moot point. The problem with an in-place filter (!SWAP_BUFFERS) is that it forces frames to be removed from the cache because the frames are modified. What this means is that making your filter in-place does no good as it merely causes VirtualDub to do the copy instead. There is a predictor in the runtime which tracks whether frames are getting re-requested multiple times in quick succession, and if the predictor says yes, the frame is copied to a new buffer so the original can stay in the cache. This prevents the frame from being recomputed, which would be much more expensive than a memcpy().
I have a feeling that it won't be possible to come up with a good solution for this in a general prefetch model until late fetches are possible, i.e. the ability to re-request new frames from Run(). This would allow prefetching a minimal set of frames up front and then pushing the request back into the pending queue from Run() if it turns out more frames are needed. I'm not sure what effect this could have on pipelining, though. Hopefully if it only happens on seeks it would only occasionally stall the pipeline.
BTW, a memcpy should not be that slow. A modern CPU can copy large blocks of memory very quickly. One thing that may be slowing down a copy is if you are doing one row at a time. This is unavoidable for field copies, but for full frame copies, you can check whether the scan lines are contiguous and do a full plane copy if so, which is much faster. VirtualDub's VDMemcpyRect() does this.
| QUOTE |
Why the need of Prefetch, Prefetch2 is not enough ?
|
Prefetch2 is only available starting with the V14 API (1.9.1); the filter I posted only requires V12 and should work back to 1.7.4.
| QUOTE |
Is PrefetchFrameDirect more interesting to use instead of PrefetchFrame ?
|
Yes. If you know that your filter isn't going to modify the frame, then PrefetchFrameDirect() allows smart rendering to copy the compressed frame and avoid dropping out of direct stream copy mode. In this case it would likely only be advantageous for a key frame only format like Huffyuv or DV. The filter runtime could potentially also skip calling runProc() entirely although the current code doesn't do this.
|
|
| jpsdr |
| Posted: Dec 24 2011, 07:46 AM |
|
|
Advanced Member
Group: Members
Posts: 335
Member No.: 20490
Joined: 23-December 06
|
| QUOTE | | ...This is unavoidable for field copies... |
Like IVTC filters...
| QUOTE | | One thing that may be slowing down a copy is if you are doing one row at a time. |
Taken from sdk v1.2
| QUOTE |
Padding must not be overwritten. In particular, this is the wrong way to copy from one bitmap to the next:
// DO NOT DO THIS!
memcpy(dst.data, src.data, src.pitch * src.h);
You should instead copy the scanlines individually:
char *dstp = (char *)dst.data;
const char *srcp = (const char *)src.data;
for(int y=0; y< ++y) {
memcpy(dstp, srcp, w*4);
dstp += dst.pitch;
srcp += src.pitch;
}
|
| QUOTE | but for full frame copies, you can check whether the scan lines are contiguous
|
Does it mean : Testing if pitch=w*4 (or w or w*2) ?
I know pitch can be negative (this is why you forbid 1rst method), and i force it to negative and re-arange src pointer in my IVTC filters, to match the original old pattern of when i've develloped the filter. I've corrected it in my manual IVTC and another field manipulation filter i've made, to avoid recursed memory transfer (not very good for CPU memory caching), but for my IVTC filter, changing back has too much impact and is too much complicated.
So, considering the fact that actualy, for my IVTC filter :
- It's field transfert.
- Memory transfert are made backward
I think i'll stay with actual way of prefetch.
| QUOTE | | VirtualDub's VDMemcpyRect() does this. |
Is it in the src/include files provided in the sdk, or in some other file in the source code ?
|
|
| phaeron |
| Posted: Dec 24 2011, 08:54 PM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
VDMemcpyRect() is a VirtualDub internal function, not part of the SDK.
The warning in the SDK about not doing one big memcpy() in general is correct, but contiguous scan lines are a special case where that does work. If pitch == w*4, then there is no functional difference between copying one scan line at a time and copying the whole frame at once. This only works when there are no gaps, so that's why you need a pitch check before doing this.
|
|
| jpsdr |
| Posted: Dec 25 2011, 09:18 AM |
|
|
Advanced Member
Group: Members
Posts: 335
Member No.: 20490
Joined: 23-December 06
|
Thanks, and merry Xmas. |
|
| jpsdr |
| Posted: Mar 25 2012, 08:42 AM |
|
|
Advanced Member
Group: Members
Posts: 335
Member No.: 20490
Joined: 23-December 06
|
I've totaly forget one little thing... To finalise why my filter need a continuous input of frame, from begining, and can't work properly in preview : Because of one important thing in algorithm : History.
I've some kind of threshold to validate the detection of an IVTC pattern, and afterward, if this threshold is not validated, the detected pattern is kept. So, you can have a good pattern detected (strong interlaced information), followed by 100 frames where with IVTC almost undectable (and under threshold), in that case, "old" detected pattern is kept, until a scene change or another validated detected pattern. |
|
| phaeron |
| Posted: Mar 31 2012, 09:18 PM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
There's unfortunately no way to perfectly support seeking in this model without requiring all frames to be reprocessed potentially back to the beginning of the file.
You can approximate it by fetching a bunch of frames on a seek and omitting those frames when you have cached information, but the longer the look-back window the more memory this will require.
|
|
| jpsdr |
| Posted: Apr 1 2012, 08:24 AM |
|
|
Advanced Member
Group: Members
Posts: 335
Member No.: 20490
Joined: 23-December 06
|
I think there is something i may not have been clear enough : Seeking is absolutely not my concern. I've always considering seeking/preview purely be not compatible with it, this is why i've not implemented the preview button.
My only concern, now having changed my filter wich remove frames in adding to it to request by prefetch a windows of more frame (described in post 14) wich solved now the problem of not having all frames processed i had in the first place, is to be sure that all frames will be processed and in the correct order when i start a process, in a case where i perfectly know the filters i use (there is no filter wich will request frames backward ) => Only "direct" filter and only my remove filter wich can alter frames order. |
|
|
