Support forum for DG Tools

I have been using an avisynth+ script that mixes a lot of conventional avisynth cpu based filters and GPU enabled filters as DGSource and DGBob. from DGdecNv.

My problem is the script is rather slow due to the conventional cpu avisynth filters if I run it in mono thread and I have not found a way to use them in MT mode with the GPU filters without crashing the script.

I have tried to Multi thread these cpu filters namely with SetFilterMTMode("DEFAULT_MT_MODE", 2)/ Prefetch(8) but my attempts seems to negatively affect the script included DGDecNv filters and the result is the script crash immediately or in a short delay . However it runs fine if I don’t try to use MT mode. Standard avisynth as a similar behavior as avisynth+.
I was seeking advice on how to MT the standard filters without causing the GPU filters to induce a crash.
Thanks in advance

Please post your script and tell me the version of Avisynth that you are using.

Avisynth version

Avisynth+ 3.7.0 (r3382, 3.7, x86_64)

Script:

That's a very complex script. Can you strip it down to a bare minimum that still crashes?

This script mixing CPU and GPU does not crash, so that's why I am asking for a minimal script that crashes.

loadplugin("d:\don\Programming\C++\dgdecnv\DGDecodeNV\x64\Release\dgdecodenv.dll")
loadplugin("D:\Don\Programming\C++\Avisynth filters\Deblock\Build\x64\Release\deblock.dll")
SetFilterMTMode("DEFAULT_MT_MODE", 2)
DGSource("alba.dgi").AssumeTFF()
DGBob(order=1, mode=0) #GPU
Deblock() #CPU
DGSharpen(strength=0.35 ) #GPU
Prefetch(8)

Initially I thought that declaring Avisynth+ MT modes SetFilterMTMode("DEFAULT_MT_MODE", 2) / Prefetch(8) could negatively impact GPU accelerated filters included in the script as I had no error notice at all and Vdub2 simply crashed. So my post was because I thought that declaring a default MT mode in avisynth+ script could eventually not be used if GPU accelerated filters namely the ones included in DGDecNv were included in the script.

However playing around with SetMemoryMax increasing it from default 1024k and reducing Prefetch to 4 or 2 threads I have been able to get the script to run in MT mode (2-3 times faster), however it´s unstable after some time could be some minutes or 1 hour or more gives out of memory errors or no error at all but freezes Vdub or FFmpeg.

So if declaring a default avisynth+ MT mode is not known to affect GPU accelerated filters namely the ones in DGDecNV , it seems I´m dealing with a standard problem of lack of memory resources in between avisynth and Vdub, to run a script that is somewhat complex and do include some old avisynth filters , running in 32 bit with limited memory resources and as nothing to do with the DGDecNv filters.

So if this situation is confirmed I have no choice if I do want speed as to forget the “unfilter” filter the one that only exists in 32 bit and forces me to run in 32 bit and look for a replacement and do a modified script to run in 64 bit where memory resources are much bigger and it will very unlikely that I will have the same problems or accept the 32 bit memory limitations and run the script slowly in 32 bit mon thread !...

MT'ing the GPU filters should not crash but it will require extra memory on the CPU and GPU that you may not have. I recommend that, instead of setting mode2 for everything using DEFAULT_MT_MODE, do it only for the CPU filters:

"AviSynth+ does it differently. In AviSynth+, you specify the MT-mode for only specific filters, and those filters will then automatically use their own mode, even if there were other MT-modes inbetween. This means you can specify all the MT modes at the beginning without polluting your script. You can even make a SetMTMode.avsi if you wish and let it autoload for all of your scripts, or import() it from their top. This is much cleaner, and it allows you to maintain all your MT-modes centrally at a single place. To make this distinction clear from AviSynth+, SetMTMode() is called SetFilterMTMode() in AviSynth+."

Thanks you so much for your suggestion. I´m going to test it.

However, I have a doubt that I would appreciate if you could clarify it.
Related to specifying all the MT modes at the script beginning for each individual filters it´s crystal clear, however is not clear for me if only one Prefetch at the end of the script is enough or if I must put one after each individual CPU filter.

Also both filters - dll and avsi MT modes should be declared or only dll.

One prefetch at the end. Test different values as too many can lead to thrashing. Find the sweet spot empirically.

To be honest, I never use avsi's. Guessing that the avsi itself should declare the modes for any of its invoked filters. Test it out and let us know what you find.

What do you think, Sherman?

I've stopped thinking. Curly says it is counter-productive. It's all about feeling. I feel like...

Ya see guys? This is why he'll always be a loser. Sherman and Britney...birds of a feather.

At leest I can spell!

The one on the right is Sherman. Submissive!

Rocky wrote: ↑

Thu Nov 18, 2021 9:43 pm

To be honest

Does that mean you are not normally honest?

There is something terribly wrong with you people. Get therapy!

Natasha is honestly confused.

new_guy, get a life...and a face.

Just in case anyone is still interested in the technical issue and the solution.

Declaring MT modes for each CPU filter as to avoid applying the default MT over the GPU filters seems to benefit a lot the stability. However may not be enough, with a high Prefetch thread number and a complex script these number may have to be adjusted as well ( I have initially used 8 the number of my logical cores). Randomly going to several frames in Vdub2 results sooner or later in memory errors as “Error reading source frame xxxx: avisynth read error : could not allocate video frame. out of memory, memory_max 1073741824, used =845512777 request=14775296”.
Reducing the threads prefect number to 6 improved the situation and 4 threads solved definitively the situation ,and despite everything it's still about 2 times faster than in mono thread.

I have declared MT modes only for filters (dll) even if sometimes only a function of the filter is used and with a different name: I have also not declared avsi based filters as they call sometimes a lot of filters . So far so good I run 2 jobs with a runtime of about 3-4 hours each and they ended successfully both of them.

So perhaps it will be good practice when running 32 bit scripts with limited memory resources avoid applying MT modes over the GPU filters, it seems to exacerbate the memory resources needed. Also if the script has some complexity prefect threads should be adjusted as well.

Just because we have sideline discussions doesn't mean we have lost interest! Thank you for the update and I'll mark this resolved.

Frankly, if you use 32-bit you deserve all the pain you get.

Thanks for the support

You are most welcome, VID240. Bullwinkle likes you.