|
|
| jcsston |
| Posted: Nov 23 2002, 04:31 PM |
 |
|
Matroska Dev
Group: Moderators
Posts: 553
Member No.: 652
Joined: 3-November 02
|
I tried it with DivX 5 1pass and the codec that NanDub uses. I tried with the video fast recompress and full processing and the audio set full processing and direct stream.
--------------------
Use the Matroska file format |
|
| jcsston |
| Posted: Nov 23 2002, 04:50 PM |
|
|
Matroska Dev
Group: Moderators
Posts: 553
Member No.: 652
Joined: 3-November 02
|
Here's the log files from the Master and Slave.
Job.log Files
--------------------
Use the Matroska file format |
|
| Thomas-Matern |
| Posted: Nov 23 2002, 05:13 PM |
|
|
Unregistered
|
Hi
Thx for the logfile, I found that line:
> VirtualDub.SaveCompatibleAVI("\JCSSTON\DATA\Videos\test.avi");
I think you use the option "save old format AVI", right?
Please try "Save as AVI" instead.
If you really need that option (why?), I could modify the code.
HTH, bye
Thomas
|
|
| jcsston |
| Posted: Nov 23 2002, 10:38 PM |
|
|
Matroska Dev
Group: Moderators
Posts: 553
Member No.: 652
Joined: 3-November 02
|
That fixed it.
I don't know why I keep choosing the old AVI for.
Thanks
BTW Do MPEG's work? If not I'll just make a avs script.
--------------------
Use the Matroska file format |
|
| Neo Neko |
| Posted: Nov 24 2002, 03:11 AM |
|
|
VirtualdubMod Alpha tester
Group: Vdubmod Alpha Testing Team
Posts: 474
Member No.: 24
Joined: 11-July 02
|
Interesting aplication. I tried it without much sucess. I could nearly get an encode done. But one of the clients would always crash or throw errors. This setup will never be viable for encoding perfectionists but it is handy for getting something done quickly. The nandub version and everything related to it is pretty much useless to me and most other people. Xvid can do as good as MSMPEG4 SBC with much less work. Though the built in stats do provide a problem as opposed to the outdated SBC setups. If you think you might considder continuing the project may I make a few suggestions.
1. Look at joining up with the Virtualdub-Mod guys. No sense in another isolated varriant no matter how good it is. Plus they could have valuable input. And vice versa.
2. The software should determine the speed of all systems involved. The data should then be split proportionally between all systems according to processor speed. As oposed to equal sized chunks which seem to be the case now. In this way all systems should finish with their chunks at approximatly the same time and should minimise the wait you might have to put up with from the slowest or slower systems involved.
3. All chunks should be as large as possible depending on system speed. The more chunks the more excess keyframes you will have. If you have 4 systems the video should be devided into 4 chunks sized by system power. It would be more efficient than jumping between and coordinating several smaller segment encodings and offer the greatest possible efficiency considdering the conditions.
4. If a slave should crash or throw an error durring encoding it should contact the master and try to restart the same or another segment. As it is now it just stops till the master restarts.
5. If a segment finishes sooner than expected the master should realocate part of the frame range of another slave to the one that just finished. This way if for some unknown reason a system suddenly got bogged down the rest should pick up the slack. And new systems could be added on the fly.
6. There should be some control over the number of segments. Going from the low end where the segments equals the number of systems at the start of the encode. This would be the slowest while still faster than normal processing as well as the most efficient in this setup. The other end should be unlimited where speed is more important than size. It will allocate chunks on the fly while still using the largest chunks proportionally possible.
7. Most people will have a general preffered master. Therefore pure slave clients would be usefull at some point in the future. They could be run as an actual system service and used at any time with no notice or startup. It would offer little or no configuration as the master would provide that.
8. Finally grouping or access control would be useful.
This could be a usefull adition to virtualdub. Keep it up.  |
|
| jcsston |
| Posted: Nov 24 2002, 04:13 AM |
|
|
Matroska Dev
Group: Moderators
Posts: 553
Member No.: 652
Joined: 3-November 02
|
Yes,
In reality I'll use this for jobs I need done as fast as possible. Mainly because I have to worry about the other computers being setup. A K6 300mhz, Pentium 166Mhz, and 486 120mhz 
I'll try run some benchmarks to see how much faster this is compared to my single computer.
--------------------
Use the Matroska file format |
|
| Neo Neko |
| Posted: Nov 24 2002, 08:52 AM |
|
|
VirtualdubMod Alpha tester
Group: Vdubmod Alpha Testing Team
Posts: 474
Member No.: 24
Joined: 11-July 02
|
It crashes every time I do an Xvid encode. I don't have Divx 5.02 pro on all systems to test that with it. I did a Cinepeak encode and it went off without a hitch. But if I use Xvid it crashes at or near the end. Before it coalates the segments. But it works with every other codec I have tried.  I had 2 systems getting ~20fps each plus a PIII 700 ~7fps. For single pass encoding it would cut the time needed by more than half. I suppose I could run it under Wine on my PII 400 Linux box, my AMD K6-3 550 laptop, and that PII 333 sitting downstairs. I might get another collective 10fps or so.  |
|
| Thomas-Matern |
| Posted: Nov 24 2002, 10:14 AM |
|
|
Unregistered
|
Hi
| QUOTE (Neo Neko @ Nov 23 2002, 09:11 PM) | | 1. Look at joining up with the Virtualdub-Mod guys. No sense in another isolated varriant no matter how good it is. Plus they could have valuable input. And vice versa. |
I started with patching Nandub because I never use VDub. After releasing the first version to a german newsgroup, there were some people asking me for a VDub-Version - so I applied the changes to VDub. And because I changed only 1 sourcefile, everything is done parallel to Nandub and VDub.
I never thought about a public release and so I didn't contact the VDub-developers. But now I have very little time to continue developing...
| QUOTE |
2. The software should determine the speed of all systems involved. The data should then be split proportionally between all systems according to processor speed.
|
Thats already done. But it needs some time to work properly.
| QUOTE | | 3. All chunks should be as large as possible depending on system speed. The more chunks the more excess keyframes you will have. If you have 4 systems the video should be devided into 4 chunks sized by system power. It would be more efficient than jumping between and coordinating several smaller segment encodings and offer the greatest possible efficiency considdering the conditions. |
You can select the chunk-size from the options-menu. But it's real size is channged slightly depending on system-speed.
It's a bad idea to divide the video into 4 chunks if you have 4 systems:
- what happens if one host is switched off or
- what happens if a 5th host is added
during encoding?
| QUOTE | 4. If a slave should crash or throw an error durring encoding it should contact the master and try to restart the same or another segment.
As it is now it just stops till the master restarts.
|
If a slave crashes, the master *should* note it and give the chunk to an other slave. But there are some errors the master can't note. There are 2 ways to detect an error:
- the slave sends an error-message
- the connection is closed.
If the slave "hangs", the master waits forever... :-(
| QUOTE |
5. If a segment finishes sooner than expected the master should realocate part of the frame range of another slave to the one that just finished. This way if for some unknown reason a system suddenly got bogged down the rest should pick up the slack. And new systems could be added on the fly.
|
Nice idea, but....
I only changed the job-processing - everything is done by executing sylia-scripts - but I can't change a running script.
| QUOTE |
6. There should be some control over the number of segments. Going from the low end where the segments equals the number of systems at the start of the encode. This would be the slowest while still faster than normal processing as well as the most efficient in this setup. The other end should be unlimited where speed is more important than size. It will allocate chunks on the fly while still using the largest chunks proportionally possible.
|
Have a look at the options-menu - you can select a chunksize from 1 to 15minutes.
| QUOTE |
7. Most people will have a general preffered master. Therefore pure slave clients would be usefull at some point in the future. They could be run as an actual system service and used at any time with no notice or startup. It would offer little or no configuration as the master would provide that.
|
I had a similar idea. Mark the "Send slave to tray" in the options-menu and put a link to Nandub/VDub into the autorun-folder with the option "/j".
| QUOTE |
8. Finally grouping or access control would be useful.
|
In which way???
Bye
Thomas
|
|
| Thomas-Matern |
| Posted: Nov 24 2002, 12:10 PM |
|
|
Unregistered
|
Hi
| QUOTE (Neo Neko @ Nov 24 2002, 02:52 AM) | | It crashes every time I do an Xvid encode. |
I fixed that. I never used XVid. There was a similar problem some month ago with DivX-4: the codecs configinfo was larger than the maximum line-length - I increased it to 1K but XVid needs ~1500 Byte... Please try the new Beta-2c - there where some more bugs in VirtualDub.
| QUOTE | | I suppose I could run it under Wine on my PII 400 Linux box, my AMD K6-3 550 laptop, and that PII 333 sitting downstairs. I might get another collective 10fps or so. |
Please tell me if you could run it under Wine - I have a Linux box too, with a Duron-950 but no X (not yet).
Bye
Thomas
|
|
| Neo Neko |
| Posted: Nov 24 2002, 07:30 PM |
|
|
VirtualdubMod Alpha tester
Group: Vdubmod Alpha Testing Team
Posts: 474
Member No.: 24
Joined: 11-July 02
|
[QUOTE=Thomas-Matern,Nov 24 2002, 04:14 AM]
[QUOTE]1. Look at joining up with the Virtualdub-Mod guys. No sense in another isolated varriant no matter how good it is. Plus they could have valuable input. And vice versa.[/QUOTE]
I started with patching Nandub because I never use VDub. After releasing the first version to a german newsgroup, there were some people asking me for a VDub-Version - so I applied the changes to VDub. And because I changed only 1 sourcefile, everything is done parallel to Nandub and VDub.
I never thought about a public release and so I didn't contact the VDub-developers. But now I have very little time to continue developing...
[/QUOTE]
It is a good idea. I simply thought no one would bother to do it because of the difficulties and problems involved. But you have managed to make a fairly open and workable solution. There are lots of people I am sure who would like to use it. And what you have is a good start. Not much less stable than Nandub itself which was never really finished. You should go public. And adding your work to a project like virtualdub mod will make your additions that much more usefull as well as possibly getting some help developing it further.
[QUOTE=Thomas-Matern,Nov 24 2002, 04:14 AM][QUOTE]
2. The software should determine the speed of all systems involved. The data should then be split proportionally between all systems according to processor speed.
[/QUOTE] Thats already done. But it needs some time to work properly.[/QUOTE]
I encoded a shorter clip with fewer segments and saw that the size did varry a "bit". But not much. The PIII 700's chunk was only slightly smaller than the chunks that the two Anthlon XP 1600+ systems were using.
[QUOTE=Thomas-Matern,Nov 24 2002, 04:14 AM]
[QUOTE]3. All chunks should be as large as possible depending on system speed. The more chunks the more excess keyframes you will have. If you have 4 systems the video should be devided into 4 chunks sized by system power. It would be more efficient than jumping between and coordinating several smaller segment encodings and offer the greatest possible efficiency considdering the conditions.[/QUOTE]
You can select the chunk-size from the options-menu. But it's real size is channged slightly depending on system-speed.
It's a bad idea to divide the video into 4 chunks if you have 4 systems:
- what happens if one host is switched off or
- what happens if a 5th host is added
during encoding?[/QUOTE]
The chances of one of the systems getting turned off etc should be pretty isolated. But using the maximum chunk sizes is still a good idea. It should at least be an option. For example on my LAN I can tell people not to fiddle with things and they don't. The fewer the segments the more efficient the encode.. As to adding in clients the pre rendered scripts at this point though genius become a liability. At some point actually coding functions/classes etc to handle the job allocation on the fly could fix that situation and make the process much more robust. But as I said these are all ideas for the future. No rush.
[QUOTE=Thomas-Matern,Nov 24 2002, 04:14 AM]
[QUOTE]4. If a slave should crash or throw an error durring encoding it should contact the master and try to restart the same or another segment.
As it is now it just stops till the master restarts.
[/QUOTE]If a slave crashes, the master *should* note it and give the chunk to an other slave. But there are some errors the master can't note. There are 2 ways to detect an error:
- the slave sends an error-message
- the connection is closed.
If the slave "hangs", the master waits forever... :-([/QUOTE]
Then would it be possible to have the Master poll the slaves at a regular interval or something of the sort?
[QUOTE=Thomas-Matern,Nov 24 2002, 04:14 AM][QUOTE]
5. If a segment finishes sooner than expected the master should realocate part of the frame range of another slave to the one that just finished. This way if for some unknown reason a system suddenly got bogged down the rest should pick up the slack. And new systems could be added on the fly.
[/QUOTE]
Nice idea, but....
I only changed the job-processing - everything is done by executing sylia-scripts - but I can't change a running script.[/QUOTE]
Hmmmm I see your problem. Well it is definatly something thatwould be nice if the moddification progresses.
[QUOTE=Thomas-Matern,Nov 24 2002, 04:14 AM][QUOTE]
6. There should be some control over the number of segments. Going from the low end where the segments equals the number of systems at the start of the encode. This would be the slowest while still faster than normal processing as well as the most efficient in this setup. The other end should be unlimited where speed is more important than size. It will allocate chunks on the fly while still using the largest chunks proportionally possible.
[/QUOTE]Have a look at the options-menu - you can select a chunksize from 1 to 15minutes.[/QUOTE]
I don't see it on the options menu.
[QUOTE]
7. Most people will have a general preffered master. Therefore pure slave clients would be usefull at some point in the future. They could be run as an actual system service and used at any time with no notice or startup. It would offer little or no configuration as the master would provide that.
[/QUOTE]I had a similar idea. Mark the "Send slave to tray" in the options-menu and put a link to Nandub/VDub into the autorun-folder with the option "/j".[/QUOTE]
I don't see it either. But if it is only on that new version you spoke of that is why. I will get it tonight and try it.
[QUOTE=Thomas-Matern,Nov 24 2002, 04:14 AM][QUOTE]
8. Finally grouping or access control would be useful.
[/QUOTE]In which way??? [/QUOTE]
In a large LAN environment you could really get some massive encoding speeds. Say like a college campus or corporate LAN. But what if you were not the only one with that idea. Who is to stop someone else from using your clients? And what if more than one person wants to use this same method for their own encodes on the same LAN segment. How can allocation of slaves be controlled. Since it appears you are using windows networking in this then I suppose you could break up groups of slaves by giving them different group/domain names. But that is not a very good or even regularly avalable solution. Also if you had control over IP address assignment and allocation then you could subnet a group slaves since NETBUI can not see across network segments by itself. But unless you run the network this is not an option.
|
|
| Neo Neko |
| Posted: Nov 24 2002, 08:11 PM |
|
|
VirtualdubMod Alpha tester
Group: Vdubmod Alpha Testing Team
Posts: 474
Member No.: 24
Joined: 11-July 02
|
Ok I see the options now! DOH! I was blind. Woould you honestly believe I forgot about the menu up there!? DOH!!!!! |
|
| Belgabor |
| Posted: Nov 25 2002, 08:18 PM |
|
|
Developer of VirtualdubMod
Group: VirtualdubMod Team
Posts: 100
Member No.: 998
Joined: 24-November 02
|
@Thomas: I think I speak for all our developers that you'd be a great addition to our project
If you don't have the time to work at it, please give us the changed source files so we can include it. (Maybe I'm dumb, but I coulnt find it on your page)
Cheers
Belgabor
--------------------
[VirtualDubMod Homepage]
Please submit any bugs/patches/feature requests also using the respective tracker on our sourceforge page |
|
| Suiryc |
| Posted: Nov 25 2002, 10:50 PM |
|
|
Developer of VirtualdubMod
Group: VirtualdubMod Team
Posts: 222
Member No.: 468
Joined: 10-October 02
|
I fully agree with Belgabor. Good ideas are allways welcomed 
--------------------
OGM tools, VirtualDubMod [SourceForge : Tracker/DL] (FAQ)
Don't forget the Needed DLLs for VirtualDubMod. Post bugs/requests in our Tracker.
We give 100% of your donations to the Open Source community |
|
| Thomas-Matern |
| Posted: Nov 25 2002, 10:53 PM |
|
|
Unregistered
|
@Belgabor: Sorry, but I just finished studying and now I'm looking for a (good) job - so I have not much sparetime.... but if that's done I will join your team.
I've uploaded the complete sourcecode - have a look at the download-section or click here ;-) http://www.matern-parkett.de/dub/nandub-vdub-src.exe
Nearly everything I added is in job.cpp and job.h but there are some changes in 2 or 3 other files....
Bye
Thomas
|
|
| Belgabor |
| Posted: Nov 25 2002, 11:18 PM |
|
|
Developer of VirtualdubMod
Group: VirtualdubMod Team
Posts: 100
Member No.: 998
Joined: 24-November 02
|
| QUOTE (Thomas-Matern @ Nov 25 2002, 04:53 PM) | @Belgabor: Sorry, but I just finished studying and now I'm looking for a (good) job - so I have not much sparetime.... but if that's done I will join your team.
I've uploaded the complete sourcecode - have a look at the download-section or click here ;-) http://www.matern-parkett.de/dub/nandub-vdub-src.exe
Nearly everything I added is in job.cpp and job.h but there are some changes in 2 or 3 other files....
Bye
Thomas |
Its ok, we'll wait for you
Just had a look at it, shouldnt be a real problem to include. (Thats what WinDiff's for ;p)
Cheers
Belgabor
--------------------
[VirtualDubMod Homepage]
Please submit any bugs/patches/feature requests also using the respective tracker on our sourceforge page |
|
