|
|
| phaeron |
 Posted: Sep 16 2007, 08:06 AM |
 |
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
On a whim, I took VirtualDub's Direct3D effect display code and hacked it into a video filter. I'm curious if anyone has comments:
http://www.virtualdub.org/beta/gpufilter-0.5.zip
You need the April 2005 version of the D3DX library (d3dx9_25.dll) to use it; if you don't have it, you can run the DirectX End-User Runtime Web Installer:
https://www.microsoft.com/downloads/details...&displaylang=en
I've been surprised by the filter's performance, considering the horrible blocking readback mechanism that I'm using (I don't pipeline right now). For some operations, like bilateral filtering, it can be significantly faster than a CPU-based filter. It should run well on NVIDIA GeForce 6 or newer cards, and probably ATI X300+. On older cards, it might not run so well due to poor readback implementations in the driver -- when I tried it on a Radeon Mobility 8500 it sucked at around 7 fps.
Effect support is mostly similar to the 1.7.3 display effect support, except that the built-in "temp" and "temp2" textures aren't supported. I've supplied several samples, most of which work on any pixel shader card, although "warp resize" requires 2.0 and the bilateral noise reducers require 3.0.
|
|
| Gromozeka |
| Posted: Sep 16 2007, 03:03 PM |
|
|
Unregistered
|
Its not work!  Direct runtime installed |
|
| phaeron |
| Posted: Sep 16 2007, 07:34 PM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
Uh, what error message did you get?
|
|
| Gromozeka |
| Posted: Sep 17 2007, 10:34 AM |
|
|
Unregistered
|
"Unable to start filters
Cannot start filter 'GPU shader'. The technique in effect file
c:\Program Files\VirtualDub-mpeg2_1.6.19_eng\plugins\gpushaders\noise Reductins.fx cannot run on the installed video card"
My videocard is not supported? I installed new widows, may be not installed drivers or what?
P.S:But what is it algoritm Noise reduction 2? May be him supported on standart virtualDub plugins? |
|
| phaeron |
| Posted: Sep 18 2007, 03:23 AM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
The noise reducers are the most demanding in terms of video card requirements -- you need at least a GeForce 6x00 or ATI X1x00 to run them. Apparently your video card does not support shader model 3 and thus cannot run the necessary shaders. If you have a GeForce FX 5200+ or ATI X300+, you might be able to get NR1 to run by changing the PixelShader compile target in Noise Reduction.fx from ps_3_0 to ps_2_a or ps_2_b, respectively.
The algorithm used in the noise reducers is pretty simple -- it's the same as most of the regular spatial noise reduction filters you'll find for VirtualDub. The difference is that the GPU accelerated version can run faster.
|
|
| Gromozeka |
| Posted: Sep 18 2007, 02:17 PM |
|
|
Unregistered
|
Thank you very much |
|
| blink |
| Posted: Oct 5 2007, 08:29 AM |
|
|
Newbie
Group: Members
Posts: 2
Member No.: 22193
Joined: 3-October 07
|
Thank you, great job!
Any plans to release more video filters for GPU ? |
|
| phaeron |
| Posted: Oct 5 2007, 08:58 PM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
Not at the moment, but do you have any requests?
|
|
| blink |
| Posted: Oct 8 2007, 07:39 AM |
|
|
Newbie
Group: Members
Posts: 2
Member No.: 22193
Joined: 3-October 07
|
It would be great to have an opportunity to change some parameters for certain video filters. For example, strength of Noise Reduction 2. By default it is too high...
|
|
| phaeron |
| Posted: Oct 9 2007, 07:05 AM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
Yeah, I know... bit of a pain, though, because I have to parse the effect on another thread.
|
|
| Loadus |
| Posted: Oct 9 2007, 03:08 PM |
|
|
Advanced Member
Group: Members
Posts: 352
Member No.: 10881
Joined: 1-July 04
|
Wow. Works like a charm. Just tried to scale a normal DV clip to HD using the warp resizer and zap. Realtime baby. Sweeeeeet.
--------------------
deviantART behance |
|
| Loadus |
| Posted: Oct 11 2007, 08:32 AM |
|
|
Advanced Member
Group: Members
Posts: 352
Member No.: 10881
Joined: 1-July 04
|
| QUOTE (phaeron @ Oct 5 2007, 02:58 PM) | | Not at the moment, but do you have any requests? |
- 'Edge-Directed Resampler' as a GPU shader?
- A Gaussian Blur with a radius of 20? Or 50?
- A Color Linearization: Apply 2.2 gamma to color only.
Yummy. I'll have to dig up me ol' DirectX shader book I have laying
around. I might actually learn something.
EDIT:
@Phaeron:
Can you throw a quick .fx sample (if this is possible) how to
- Convert the texture from virtualdub to 16-bit/ch,
- linearize it (gamma 0.45),
- do blaa blaa, red = green etc. dummy
- Then apply gamma 2.2,
- convert it to 8-bit/ch [with dither?]
Just curious as to how much damage happens if color transformations
would be applied in linear space ...
--------------------
deviantART behance |
|
| Gromozeka |
| Posted: Oct 11 2007, 08:48 AM |
|
|
Unregistered
|
ooops/ Its not here  |
|
| Loadus |
| Posted: Oct 12 2007, 04:16 PM |
|
|
Advanced Member
Group: Members
Posts: 352
Member No.: 10881
Joined: 1-July 04
|
Here's more requests:
Effects that I use frequently - the filter chains now are slow and long,
one GPU Shader for each effect would be more than GodMode ...
I tried to edit the shaders myself, but soon found myself cyphering
Atlantis' mainframe computer written in ancient. I couldn't even find
a primer. Me <---- Not much of a coder.
Original image:
Effects:
Hardlight with grayscale blending:
This effect is useful, when creating a filmlike feel to footage. And it's simple.
Two layers, with the top one grayscaled (with equal weighting 33, 33, 33) and
blending with the original done using:
assuming value scale of 0.0 - 1.0
A = Layer
B = Original
C = The output
If A's tonal value is =< 0.5 then C = 2 * A * B
If A's tonal value is > 0.5 then C = 1 - 2 * ( 1 - A ) * ( 1 - B )
Hardlight with sepia toning:
This is the "300" effect, sometimes handy, maybe in flashback scenes or whatnot.
The same calculation as above, only applying a slight sepia toning on the top layer
before blending.
Hardlight with sepia toning and blur:
The same as above, only after sepia toning, a radius 30px Box Blur (don't know
if that can be done in pixel shader?) was applied.
Inversed Overlay:
This is bad for the example image, but with high contrast scenes, this
helps out to bring out the shadows. Two layers, top layer is grayscaled
(again, using equal weighting) then inversed to negative and then blended
with the original image using:
A = Layer
B = Original
C = The output
If B's tonal value is =< 0.5 then C = 2 * A * B
If B's tonal value is > 0.5 then C = 1 - 2 * ( 1 - A ) * ( 1 - B )
If I could get a "starting point" to how to do these calculations in a pixel shader,
I'd appreciate it. 
And cheers to Phaeron for creating the GPU shader filter.
--------------------
deviantART behance |
|
| phaeron |
| Posted: Oct 13 2007, 08:56 AM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
Alright, quick lesson in pixel shaders.
Almost everything you are looking for is in a function that looks something like this:
| CODE |
float4 PS(float2 uv : TEXCOORD0) : COLOR0 {
float4 pixel = tex2D(src, uv);
return pixel;
}
|
(The easiest shader to start with is the grayscale shader. You can just copy that, and replace its PS function with the one above. Also, for greater freedom, change ps_1_1 to ps_2_a, or you'll get lots of annoying messages about how pixel shader 1.1 sucks.)
The tex2D() call is what fetches the source pixel, and the return statement is what sets the output. Everything in between is where computations go. The incoming and outgoing pixels are both "float4" because they have four components -- alpha (A), red ®, green (G), and blue ( -- because that's how pixel shaders work, even though in this case alpha is ignored.
Now, while this is code, the good news is that the language involved here -- High Level Shader Language (HLSL) -- has rather natural syntax for working with colors. While you can always go mucking with red, green, and blue individually, it's easier to work with all of them at once, which is coincidentally what the hardware likes to do too. The values are also always normalized from 0-1, so you don't have to worry about the precision of the values. In fact, the precision is frequently a lot higher then even 16 bits/channel -- on modern GPUs it's actually 32-bit floating point.
Anyway, for the first example you requested:
| QUOTE |
- Convert the texture from virtualdub to 16-bit/ch,
- linearize it (gamma 0.45),
- do blaa blaa, red = green etc. dummy
- Then apply gamma 2.2,
- convert it to 8-bit/ch [with dither?]
|
Here's the corresponding effect, which sets red to original green, green to one-half, and adds one-half to blue. There's no dither, but that's because that's actually a bit involved to set up (you need a dither texture, since the built-in dither is only meant for precision lower than 8 bits/channel).
| CODE |
float4 PS(float2 uv : TEXCOORD0) : COLOR0 {
float4 pixel = tex2D(src, uv);
pixel = pow(pixel, 0.45);
pixel.r = pixel.g;
pixel.g = 0.5;
pixel.b += 0.5;
pixel = pow(pixel, 2.2);
return pixel;
}
|
As you can see, HLSL is set up such that you can easily work across all channels as well as with individual ones.
Now, for your inversed layer case, believe it or not, but your equations can almost be used verbatim. Here's what it looks like. dot() performs a dot product (inner product), which is red*red+green*green+blue*blue, and which I use to compute brightness below. One big change, though -- I had to change the tonal( < 0.5, because I'm not sure what you mean by tonal value.
Note also that there is only one input image here, which will be somewhat restricting -- but presumably if you are creating the top layer via calculations from the original, this won't be a problem in a grander shader.
| CODE |
float4 PS(float2 uv : TEXCOORD0) : COLOR0 {
float4 B = tex2D(src, uv);
float A = dot(float3(1.0/3.0, 1.0/3.0, 1.0/3.0), B.rgb);
float4 C;
if (A < 0.5)
C = 2 * A * B;
else
C = 1 - 2 * ( 1 - A ) * ( 1 - B );
return C;
}
|
It's also possible to wrap the above in a function for easier use, but that's for later. As I said, right now this just assumes that you want to blend a grayscale version of the original onto the original. There are other interesting ways to compute the blend layer. For instance, the expression saturate(B - 0.8) subtracts 0.8 from each component and then clamps the result to 0-1, so you can use it to determine highlights in the image.
Now, I do have a bit of bad news, and it's that high powered blurs are a pain to do on the GPU. The reason is that the pixel shader is rigged to be a straight function from textures to framebuffer, and that's not compatible with the popular ways of doing high-powered blurs. In particular, the box blur, which is a really fast way to get really big blur radii, relies on recursive computations, where parts of the image are dependent on other parts of the image in a big chain. On the GPU you can't do this, so you basically have to do a bunch of perturbed samples and average them. This is only practical up to about 30-40 samples (radius ~7), after which some serious slowness starts to set in. You can see this with the noise reduction filters. Multiple passes and separable processing can make radii in the range of 16-40 practical, but the passes and render targets are a PITA to set up. Worse yet, it's usually impossible to make this easily tunable, since the pass configurations are hardcoded in the effect. VirtualDub's display code and this filter actually have more support for things like this than most 3D engines -- some of the shaders that I shipped with gpufilter are damn near impossible to do in a tool like FXComposer, where the DXSAS annotation support is nearly useless for setting up 2D effects.
What would be really nice is a way to chain passes the same way that functions can be chained in a pixel shader -- but this requires a lot of work to implement, and there's only so much time available. In general, when you can structure your filter as an expression tree of color calculations, it's really easy to hack out, but when you start doing multiple passes and calculations across pixels, then it starts getting a bit hairy.
|
|
|
