We're probably at the maximum performance now unless we want to scrounge for another second or two of CPU. Can't do much more with the IO on the application side. Use quality SSDs and stay off the windows partition. Don't do heavy work on other things while you are indexing. It would be interesting...

Sherman said he would be happy to implement an MPLS selector ala DGDemuxGUI. He wants to use it as a project to teach Curly how to code. I said go for it!

Here's a little challenge for coders. Following is the biggest CPU bottleneck function during indexing. If you think you can speed it up please submit your changes. You'll have to run it (say) 500000000 times in a loop to get good timings with timeGetTime(). This is the best I was able to do. It's f...

Here is rc3: * Further small performance improvement. * Removed the obsolete decode modes option. The line is retained in the INI file and ignored. * Fixed bugs in default save project path handling. * No longer clear the Info dialog after an indexing ends. http://rationalqm.us/misc/DGIndexNV_rc3.ra...

Good idea DJATOM. I'll do that.

Meanwhile, here is how to flush memory so you don't have to reboot:

https://chadaustin.me/2009/04/flushing-disk-cache/

Makes my life a lot easier! Now we need to find a way to flush the SSD hardware read cache.

That's great gonca , thank you. So it looks like there are significant gains for indexing but really massive ones for demuxing. But I may be able to improve the indexing further now that I know everything that is going on. I'll fix a few bugs while you are at work. For example, the default save path...

The rc build is updated so re-download. * The BUFFER_SIZE is reduced to optimize IO. * The priority is restored to the INI file for compatibility, so delete INI files before re-testing. rc and stable can now be in the same directory and share the same INI file. http://rationalqm.us/misc/DGIndexNV_rc...

OK, I get it all now. rc with demux after reboot: 2:37 stable with demux after reboot: 7:24 First we need the reduced buffer. I'll update the rc and post when that is done. Second we need to stay off the windows partition. I think that will make everything right but let's see. I'm surprised that the...

Nah, I won't be able to sleep. OK, I reduced the BUFFER_SIZE to 32K from 16MB. Now, after a reboot, I get: rc 1:22 at 410 MB/sec. That is the same as stable now. So the question arises, is it only an improvement in demuxing that we have, or is it no improvement at all? I'm so confused. Have to test ...

That's good but not time to relax (see below). It's great that we are exposing all these things. Here are my shocking results: After reboot: rc: 2:08 with read data rate 260 MB/sec stable 1:23 with read data rate 410 MB/sec That seems to duplicate your original report. It doesn't matter where the rc...

My process is going drive d: to drive d: (an SSD). I see 0% activity on D: during processing! But I have 64GB so the entire source file could be in windows memory from previous runs. I have to find a way to flush it out before running. Hmmm? Simple way is to reboot but that is draconian and a PITA. ...

I'll see what my rates are.

32GB is fine. You won't page.

I'd like you to get stuff off your windows partition.

I know your CPU kicks mine's ass so I'm baffled at this point.

Doubtful but worth a try. But go through everything I mentioned in my previous post, trying the things and answering my queries. It's the only hope because if I can't duplicate something, I can't fix it by debugging directly. renols had good results.

Is there is something weird with your system? I demuxed the .265 from CARS_2 and then did it both ways with no demuxing. I got: rc: 1:22 stable: 2:07 So why are you getting 6 and 4 minutes? That's terrible for both. Can you try with CARS_2 and give the timings? Also, please make sure you use the ver...

Your INI files show you loaded a .265 file. Please clarify everything you did.

EDIT: I see you said that. Let me test that.

What is the disk? Can you try on other disks? Try an AVC 1080p bluray.

I just can't believe this.

My results on bluray 1917 indexing only. Ran in separate directories with valid INI files for each.

old: 3:36
new: 1:02

If you are doing everything separately and you still see the issue please give your INI files for both.

Be careful that you don't run the fast one and the slow one from the same directory as the INI file will be wrong. Do everything in separate directories with fresh INI files.

I should leave that line in for backwards compatibility and just ignore it in the new version.

I am testing that now.

?????

Investigating...

Sorry, guys. Please re-download. Fix THD gaps processing was broken.

http://rationalqm.us/misc/DGIndexNV_rc.rar

Here is a release candidate for the high-speed DGIndexNV (includes 32 and 64 bit): http://rationalqm.us/misc/DGIndexNV_rc.rar Changes: * Built with VS 2019 and latest tools. WinXP is no longer supported. * Major performance improvement for indexing and demux. * Removed the process priority option. E...

Thank you. That's great. No need to compare to DGDemux.

Stand by for a release candidate with another big performace bump coming in minutes...

Thank you for your test results, renols.

renols wrote: ↑

Sun Nov 22, 2020 3:34 am

Impressive results I must say

You inspired the work, so right back at ya:

I fixed the bad extra byte on demuxing video. Off to bed now.

gonca wrote: ↑

Sat Nov 21, 2020 8:09 pm

So far the only thing is that there is a one byte difference in the hevc video when compared to DGDemux

Yeah, that is already on my list to fix. Maybe tomorrow. Thank you.