|
|
| alex_ts |
| Posted: Apr 27 2010, 12:42 AM |
 |
|
Newbie
Group: Members
Posts: 2
Member No.: 27495
Joined: 27-April 10
|
I encountered a problem when reading the class VDAVIReadIndex.  . That each stream(video stream,audio stream,or txt stream) has a member like:
| CODE |
class AVIStreamNode : public ListNode2<AVIStreamNode> {
public:
VDAVIReadIndex mIndex;
ˇˇ
}
|
In AVIReadHandler::_parseIndexBlock,VD adds a chunk to mIndex
| CODE |
pasn->mIndex.AddChunk(bytePos, sizeAndKey);
|
Now I consider that VDAVIReadIndex has indexs every data chunk in section 'movi',am i right?
but VDAVIReadIndex has members like:
| CODE |
struct SectorEntry {
sint64 mByteOffset;
sint64 mSampleOffset;
uint32 mChunkOffset;
bool mbOneSamplePerChunk;
};
struct IndexEntry {
uint32 mByteOffset;
uint32 mSampleOffset;
uint32 mSizeAndKeyFrameFlag;
uint16 mPrevKeyDistance;
uint16 mUnused0;
};
typedef vdfastvector<SectorEntry> Sectors;
Sectors mSectors;
typedef vdfastvector<IndexEntry *> Index;
Index mIndex;
sint64 GetSampleCount() const;
uint32 GetChunkCount() const;
uint32 FindSectorIndexByChunk(uint32 chunk) const;
uint32 FindSectorIndexBySample(sint64 sample) const;
IndexEntry *FindChunk(sint64 sample, uint32 sectorIndex, uint32& sampleOffsetOut, uint32& index) const;
|
Now I am confused  ,
what a 'chunk' means?
what a 'sample' means?
what a 'sector' means?
why VD has struct IndexEntry and SectorEntry?
Can several frame put into ONE chunk? |
|
| phaeron |
| Posted: Apr 27 2010, 03:50 AM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
Chunks and samples are AVI notions, whereas sectors are specific to VirtualDub's implementation.
An AVI stream is composed of a series of samples, where each sample takes a specific amount of time (dwScale/dwRate seconds, to be exact). For a video stream, one stream sample = one frame; for an audio stream, one stream sample = one audio sample. These samples are in turn stored in chunks, where each chunk consists of a FOURCC, a size, and raw data according to the RIFF standard. In a video stream, each chunk always represents one frame; in an audio stream, an N byte chunk may hold 0-N samples, depending on the nBlockAlign field of the audio format.
The most straightforward way to keep track of where all of the samples are is to store a flat index array of chunks and to keep track of how many samples are in each chunk. Problem is, this starts to become unwieldy when you have large AVI files. When I say large AVI files, I don't mean a few hundred megabytes -- I mean gigabytes. In fact, I've gotten bug reports about issues reading AVI files that are nearly a terabyte in size. At that point, a flat array index starts consuming a lot of memory, and traversing it is expensive as well -- unless you burn more memory on a starting sample number per entry, you have to walk the index from the beginning to figure out which chunk holds a given audio sample, due to the variable number of samples per chunk. The way VirtualDub attacks this problem is to split the index into sectors, where each sector keeps track of its starting sample number and all chunks in a sector fit within 4GB. This allows the file offset within each index entry to be reduced from 64-bit absolute to a 32-bit sector relative offset, and index search time is reduced because the correct sector can be found by binary search.
|
|
| alex_ts |
| Posted: Apr 27 2010, 12:03 PM |
|
|
Newbie
Group: Members
Posts: 2
Member No.: 27495
Joined: 27-April 10
|
Thanks for your response. It's a clear statement, and I think I get it now.
In the field of video programming, I am a novice, soˇˇperhaps my questions are simple and stupid  .
Sector keeps all chunks by mByteOffset and mChunkOffset,both of them are base value. Actually the chunk's information is stored in IndexEntry.Then index search time is reduced because the correct sector can be found by binary search.
| CODE |
typedef vdfastvector<IndexEntry *> Index;
Index mIndex;
|
When add a new chunk,do the following judges,achieve the goal that fit a sector within 4GB.
| CODE |
if ((uint64)(bytePos - sec->mByteOffset) >= (uint64)0x100000000 || mSampleCount - sec->mChunkOffset >= 0x10000) {ˇˇ}
|
I have three questions. 
first question:
If the AVI file is small,perhaps only a few hundred megabytes,then in VD, we should just get one sector for one stream,is that right?
But in VDAVIReadIndex::FindChunk, VD seens to consider that mSectors at least has two elements(not one),this made me very confused.
| CODE |
VDAVIReadIndex::IndexEntry *VDAVIReadIndex::FindChunk(sint64 sample, uint32 sectorIndex, uint32& sampleOffsetOut, uint32& index) const
{
const SectorEntry& sec1 = mSectors[sectorIndex];
const SectorEntry& sec2 = mSectors[sectorIndex + 1];
ˇˇ
}
|
second one:
| QUOTE (phaeron @ Apr 27 2010, 03:50 AM) |
For a video stream, one stream sample = one frame; for an audio stream, one stream sample = one audio sample.
ˇˇ
an N byte chunk may hold 0-N samples, depending on the nBlockAlign field of the audio format.
ˇˇ
|
So if chunk belongs to a video stream,there will be only one frame(one sample) inside it,if chunk belongs to a audio stream, perhaps there could be several samples inside the chunk. 
So you say"keep track of how many samples are in each chunk",just for chunks belongs to audio stream?
the last one:
Chunks in 'idx1' are in proper order?
If the first index chunk(named chunk_index_A) in 'idx1' refers to a data chunk(named chunk_data_A) that belongs to video stream. Does chunk_data_A represents the first frame in video stream?
Thank you. |
|
| phaeron |
| Posted: Apr 28 2010, 04:12 AM |
|
|
Virtualdub Developer
Group: Administrator
Posts: 7773
Member No.: 61
Joined: 30-July 02
|
| QUOTE | | But in VDAVIReadIndex::FindChunk, VD seens to consider that mSectors at least has two elements(not one),this made me very confused. |
This is a memory compaction trick. Each sector covers a specific contiguous range of bytes in the file, as well as a range of chunks in the RIFF structure and a range of samples in the stream. The trick is that in all three of these, the sectors are stored in-order and back to back. That means that instead of storing N sectors, each of which contain (start, count) pairs, it's possible to store N+1 sectors with just start points. The count in each sector is implicit from the difference of the start value with the next sector. If you look at the Finalize() method, you'll notice that it pushes an extra sector onto the end. That's the end sentinel used to bound the last sector.
| QUOTE | | So you say"keep track of how many samples are in each chunk",just for chunks belongs to audio stream? |
Yes, this is correct. In theory, it would be possible to store multiple video frames (samples) in a chunk, but this isn't possible due to special cases in Microsoft's parsers. However, the index code itself doesn't know about these; they're handled by the parsing code that initializes the index from the data in the file.
| QUOTE | Chunks in 'idx1' are in proper order?
If the first index chunk(named chunk_index_A) in 'idx1' refers to a data chunk(named chunk_data_A) that belongs to video stream. Does chunk_data_A represents the first frame in video stream? |
Half correct. For each stream, the chunks do appear in the idx1 chunk in the order that they play in the respective streams. However, between the streams, there is no playback order implied by their relative order; the chunks are instead indexed in the order they appear on disk. In other words, whether you see a chunk for stream A or for stream B first doesn't tell you anything about when those chunks should be played; that is determined solely by their position each stream and the timing parameters for that stream. You can reorder the file such that all of the chunks for stream A are before all of the chunks for stream B and it won't make any difference whatsoever to playback timing.
It is in theory possible to have an index that is ordered differently than the chunks in the 'movi' chunk and I believe the index is supposed to take precedence. However, there are a number of players which will break on such a file, due to parsing the movi chunk itself; I believe the OS/2 media player is one of them.
As for identifying which index entries belong to which streams, that is not determined by order. It is determined by the two character code (twocc) embedded within the chunk's four character code (fourcc). Specifically, the first two digits are the stream ID stored in hex, whereas the last two digits are a chunk type whose meaning is specific to the stream type. For instance, the video stream is conventionally stream 0 and has chunks tagged as either 00db or 00dc, and the audio stream is typically stream 1, which a fourcc of 01wb. However, it is perfectly possible to have an Indeo video stream encoded as stream 2, with chunks tagged as 02iv, or an audio stream 10 with 0awb. Such oddities do occasionally surface.
The index also occasionally stores entries that don't directly relate to chunks in a stream. I believe some older programs will write LIST/rec chunks to try to group chunks together for streaming, which will in turn be indexed. Modern parsers generally ignore these.
|
|
|
