Unofficial VirtualDub Support Forums [Powered by IB Forums]

Printable Version of Topic
Click here to view this topic in its original format

Unofficial VirtualDub Support Forums > VirtualDub Filters and Filter Development > How To Process In Yuv Colorspace

Posted by: levicki Feb 23 2012, 07:37 PM

I would like to create a filter that will do chroma upsampling based on a higher quality algorithm than the standard ones used.

Obviously, if video is encoded using RGB the filter can't do anything usefull.

What would be the best starting point for such a filter? How to check if source is encoded as YUV or YCbCr and only process those?

Any advice is appreciated.

Posted by: dloneranger Feb 23 2012, 08:34 PM

In the filters GetParams function, just return a valid response for formats you support, FILTERPARAM_NOT_SUPPORTED for the rest

eg, in virtualdub's brightness/contrast filter there is this

CODE

uint32 VDVFilterBrightCont::GetParams() {
const VDXPixmapLayout& srcLayout = *fa->src.mpPixmapLayout;
VDXPixmapLayout& dstLayout = *fa->dst.mpPixmapLayout;

switch(srcLayout.format) {
case nsVDXPixmap::kPixFormat_RGB888:
case nsVDXPixmap::kPixFormat_XRGB8888:
case nsVDXPixmap::kPixFormat_Y8:
case nsVDXPixmap::kPixFormat_YUV444_Planar:
case nsVDXPixmap::kPixFormat_YUV422_Planar:
case nsVDXPixmap::kPixFormat_YUV420_Planar:
case nsVDXPixmap::kPixFormat_YUV411_Planar:
case nsVDXPixmap::kPixFormat_YUV410_Planar:
break;

case nsVDXPixmap::kPixFormat_VDXA_RGB:
case nsVDXPixmap::kPixFormat_VDXA_YUV:
return FILTERPARAM_SUPPORTS_ALTFORMATS | FILTERPARAM_SWAP_BUFFERS | FILTERPARAM_PURE_TRANSFORM;

default:
return FILTERPARAM_NOT_SUPPORTED;
}

dstLayout.format = srcLayout.format;

return FILTERPARAM_SUPPORTS_ALTFORMATS | FILTERPARAM_PURE_TRANSFORM;
}

to just accept yuv formats you'd change it like this

CODE

Posted by: levicki Feb 24 2012, 05:56 PM

Thanks for the info.

Now I realize that I may have a problem -- I need to prevent decoder and/or VirtualDub from performing chroma upsampling.

Maybe Avery could offer some clues as to whether that would be possible?

Posted by: phaeron Feb 25 2012, 09:58 PM

No, this is not possible -- simply for the reasons that most of the time VirtualDub has no way of knowing what the native subsampling is, or that the codec has no way to pass it through. DV is a good example, as its native chroma layout doesn't have a corresponding FOURCC format.

If you want to prevent chroma conversions before your filter, you need to support the formats in question that would otherwise require such a conversion.

Posted by: levicki Feb 25 2012, 11:04 PM

Thanks everyone, I wrote the plugin and it seems to be working.

Is it possible to output RGB if the input is YUV?

Posted by: phaeron Mar 3 2012, 04:25 AM

Yes, your filter has complete control over its output format, independently of input.

Posted by: levicki May 8 2012, 06:01 PM

QUOTE (phaeron @ Mar 3 2012, 05:25 AM)

Yes, your filter has complete control over its output format, independently of input.

Ok, I made a filter, and I also make it multi-threaded and it seems to be working fine.

However, I need to handle interlaced video differently than progressive for obvious reasons (processing interlaced image would interpolate between fields which would be wrong).

So the question is -- how to programmatically detect that the source frame is interlaced?

I did a brief search in SDK but couldn't find anything.

I am aware that I can split the fields or perform deinterlace before my filter but IMO that is not a solution, just a workaround.

If there is currently no way to do that, what are the chances of expanding the SDK to pass this info to plugins?

Please advise.

Posted by: jpsdr May 9 2012, 08:38 AM

There is actualy nothing wich tag if video is interlaced (well, not realy totaly true for YV12, but not used by any filter actualy i think).
The best thing is to add in the UI an "Interlaced" option, wich if checked, tells your filter that video is interlaced.

I've a little question : If your code is not secret, is it possible to have it ?
I'm a little curious by it, and also by the multi-threading aspect, as i'm looking for some simple exemples of how to do it.

Posted by: levicki May 9 2012, 08:58 AM

That is what I was afraid of...

@Avery:

Can you please provide filter plugins with info whether the current frame is interlaced?
Other usefull info would be TFF/BFF flag and whether it is a key frame or not.

QUOTE (jpsdr @ May 9 2012, 09:38 AM)

I've a little question : If your code is not secret, is it possible to have it?

Sorry, but you cannot have the source code.

QUOTE (jpsdr @ May 9 2012, 09:38 AM)

I'm a little curious by it, and also by the multi-threading aspect, as i'm looking for some simple exemples of how to do it.

I can give you an example, sure:

CODE

// This means the for () loop will be executed in parallel
// For that to work all variables inside the loop tagged
// with pragma must be thread private
#pragma omp parallel for
for (int y = 0; y < ys; y++) {
for (int x = 0; x < xs; x++) {
// do your frame processing here
}
}

// this ensures that all threads have
// finished processing before continuing
// it is important or VirtualDub may already
// free the source or destination buffer while
// one of the threads is still working on it
#pragma omp barrier

This is the most basic example and it is based on OpenMP (it requires compiler that supports OpenMP and you need to distribute runtime libraries with the plugin).

Posted by: jpsdr May 10 2012, 08:24 AM

QUOTE (levicki @ May 9 2012, 08:58 AM)

@Avery:

Can you please provide filter plugins with info whether the current frame is interlaced?
Other usefull info would be TFF/BFF flag and whether it is a key frame or not.

As said, there is unfortunately no such thing. I don't know for sure for the key frame, but as far as i know, there is no such thing either.
The only thing you may have, but wich is probably actualy not use by any filter is the following :

CODE

case nsVDXPixmap::kPixFormat_YUV420i_Planar :
case nsVDXPixmap::kPixFormat_YUV420ib_Planar :
case nsVDXPixmap::kPixFormat_YUV420it_Planar :

in GetParams (take a look at vdplugin.h from the sdk for all the list).
And even if it was used, it would provide information only on the whole video, not frame by frame.

Posted by: levicki May 10 2012, 12:48 PM

QUOTE (jpsdr @ May 10 2012, 09:24 AM)

The only thing you may have, but wich is probably actualy not use by any filter is the following :

CODE

case nsVDXPixmap::kPixFormat_YUV420i_Planar :
case nsVDXPixmap::kPixFormat_YUV420ib_Planar :
case nsVDXPixmap::kPixFormat_YUV420it_Planar :

in GetParams (take a look at vdplugin.h from the sdk for all the list).
And even if it was used, it would provide information only on the whole video, not frame by frame.

If VirtualDub actually sends frames in kPixFormat_YUV420i_Planar to a plugin that would be ok with me.

I will look into it.

Posted by: jpsdr May 11 2012, 08:28 AM

QUOTE (levicki @ May 10 2012, 12:48 PM)

If VirtualDub actually sends frames in kPixFormat_YUV420i_Planar to a plugin that would be ok with me.

As i said, i'm affraid (not 100% sure but 99%) they simply exist, but are not used by VDub by default, and even less by third party pluggins. Actualy, realy, the only reliable way is to get the information from a setting on the UI, (or, in the color Depth input setting go to other and select what you want, but it's nearly the same).

Posted by: phaeron May 12 2012, 05:40 PM

Yes, unfortunately VirtualDub doesn't usually have this information. I'm undecided yet on whether it would be a good idea to provide a central checkbox for this or whether it should just be an option within the filters like I did with the resize filter. In a perfect world setting it upstream and having it magically propagate down the filters would just make everything work; in practice, it doesn't always work out that way, and at least with a checkbox on each filter the user has a good idea of which filters actually change behavior in this regard.

I suppose having it integrated has the benefit that you could tell when an edit has disrupted the field order... but even then, actually recovering from that kind of glitch isn't always the easiest....

Posted by: levicki May 12 2012, 06:49 PM

QUOTE (phaeron @ May 12 2012, 06:40 PM)

At the moment, I am not interested in field order. I am only interested whether I should process the image as a whole or as two halves. A flag in fa->src (fa->src.interlaced) or some equivalent would do for now.

Posted by: jpsdr May 13 2012, 07:47 AM

QUOTE (phaeron @ May 12 2012, 05:40 PM)

I'm undecided yet on whether it would be a good idea to provide a central checkbox for this or whether it should just be an option within the filters like I did with the resize filter.

I personnaly think an option in the filter is better, because if you have a deinterlacer in the chain filter, the status will change within the chain.
It's me who have asked you the deinterlace option in resize a long time ago...

(Even if i don't need it anymore now...

)

Posted by: levicki May 13 2012, 12:30 PM

QUOTE (jpsdr @ May 13 2012, 08:47 AM)

I personnaly think an option in the filter is better, because if you have a deinterlacer in the chain filter, the status will change within the chain.

And I think that deinterlacer should just do fa->dst.interlaced = FALSE and thus signal the change to the next plugin in the chain. Also, in filter dialog format should change from I to P after the plugin which does that.

What I also see as a problem in current filter design is that there is no way to differentiate between frames and fields -- if a plugin can split interlaced frame into fields and output that (at 2x framerate), there should be a way to signal whether a field is T or B and the clip parity (TFF/BFF) to the next plugin in the chain.