MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Anything related to video and my tools that is not a support request.
Post Reply
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Hi. I'm considering now 2 workflow options for MPEG2 decoding - DGMPGDec vs DGdecNV.
Tests show that my scripts work faster with DGDecNV.
However, when comparing the output of both scripts I noticed there's a difference.

I compared "source" frames grabbed from VDub (.bmp) by Hash'er and they turned out to be different.

Then I compared them with ImageMagick and got this:

(Red highlights indicate the portions of the image that are different)

Image

Code: Select all

  Channel distortion: PSNR
    red: 59.8955
    green: 61.1464
    blue: 58.2305
    all: 59.5922
Image

Code: Select all

  Channel distortion: PSNR
    red: 57.8271
    green: 59.1479
    blue: 56.7779
    all: 57.8112
Here are the scripts:

Code: Select all

loadplugin("R:\DGDecode.dll")
mpeg2source("R:\101.d2v")

#loadplugin("R:\Decomb.dll")
AssumeTFF()
#Telecide(gthresh=5,guide=1,post=1)
#Decimate(mode=0,quality=3,cycle=5)

Code: Select all

loadplugin("R:\DGDecodeNV.dll")
DGsource("R:\101.dgi")

#loadplugin("R:\Decomb.dll")
AssumeTFF()
#Telecide(gthresh=5,guide=1,post=1)
#Decimate(mode=0,quality=3,cycle=5)
So am I doing something wrong here or this is just the way it is?
If they indeed output different data which one is "more correct"?
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

It may be due to different IDCT algorithms. Have you tried changing the IDCT in DGMPEGDec to see what effect it has? I'll have a look at it if you provide me your test sample and instructions to duplicate the issue.
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

So in theory it should be identical?

Ok I will try IDCT options. If I can't figure it out I will provide you the files to look at.
Thanks. Gone testing.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

If the same IDCT algorithms are used they should be identical.

But I don't fully trust your process, so that's why I'd like to get your sample and try to duplicate it with a minimal controlled process that I know and trust.
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Which IDCT algo is used by DGNV? I can't find the option in UI or in .dgi.

Update:
I couldn't figure it out on my own so I sent you my sample files, pm.
I tried all the IDCT options but none of them produced identical result to DGNV.
Maybe I should try different decode modes in DGNV? Would it make any difference?
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

I don't know which algorithm CUVID uses. Anyway, I set IEEE reference IDCT in DGIndex.

Then I made this script:

vid1=mpeg2source("test.d2v")
vid2=dgsource("test.dgi")
subtract(vid1,vid2)

Then I opened that in VirtualDub and added a Levels filter to really crank up the contrast. I was not able to detect any differences.

Please try my process and report your results. Also make sure you have honor pulldown enabled for both and no deinterlacing enabled.
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Ok, I tried it and I couldn't see any difference either. :agree:
This is good. One less thing to worry about.
Now I have one more unrelated question, if you don't mind.

What do you do with bad edit cuts combed frames in a pulled-down source? The most eye pleasing thing to do with them would be to remove them in addition to decimating in-cycle repeats. That's what I thought at first. But then I realized that removing even 24 frames would result in 1 second A/V desync right? So what else can u do about it? (Note: I only use post=1 in Telecide for this project.)
I'm thinking a deinterlace pass of already IVTC'ed and Decimated sequence. Supply specific list of frames to FieldDeinterlace() by ovr file.
What do you think about such a workflow?
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

If they can't be field matched the postprocessing will deinterlace them. If you disable deinterlacing generally then yes, you can do specific frames with an override file.
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Is there a way to force Decimate modes for specified frame ranges (maybe by ovr file?)? I have video portions of the sequence that I want to force mode 3 and a very low thresh while keep the rest of the project mode 0 because it would break it otherwise. Currently I'm doing it by appending separate avs scripts for each portion in a master script, but calculating the offsets and managing settings across them is a pain. Is there some faster, more automated way?

Also, I've found the debug output of dgdecodeNV very helpful for finding those video only portions - it shows the frame is using both its fields. So I was wondering if it's possible to capture this output (from DGNV or DGMPG) like Decomb's debug. This would be VERY helpful! For now I'm manually searching for any CGI portions and examine the fields information to see if it's video or not. This is ofc not at all convenient and IS time-consuming.
Anyway, thanks for all the help so far!
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Ok, DGDecode does output it to debug... Although it would be even better if there was a single row that would include in it both "Interlaced" AND frame number... so you could easily filter the debug output and create a frame list of that.
Well I guess I can try to modify the debug output to do just this... Can I?
Where would I need to go to add such a functionality? I bet it's DGDecodeSRC, but where exactly? I mean I'll find it myself but if you see this before that happens a hint is appreciated. ;)
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

I haven't looked at the DGMPGDec code for a long time so can't tell you offhand where the right code is. I'm sure you'll be able to find it. :D
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Error 54 error LNK1181: cannot open input file '.\Release\\.obj'

and in "File" column in the same row it says
E:\Coding\dgdecode158src\LINK
Omg how am I supposed to find out which file it can't find and correct it?

in the proj properties I have
C/C++ // Output files // Object file name .\Release\
is that correct?

OUTPUT END:
1> mc.cpp
1> mc3dnow.cpp
1> mcmmx.cpp
1> Generating Code...
1>LINK : fatal error LNK1181: cannot open input file '.\Release\\.obj'
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========

Is there some official guide to compiling DGMPGDec? Or at least some hints?
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

What toolset are you using? The DGDecode project uses VC6. If you want to use something else, you'll have to port it yourself.
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

I'm on VS2010.
Could you tell the specific version of IDE I need to compile it?
Version, SP... Any required Updates or patches?
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

All this because you can't parse the debug output?
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Yes! :D But it's very helpful, I could easily find video sections, you see. I can then process them appropriately.
Maybe you can recompile it? It's just a copy/paste of several symbols from one row to the other one right below it.
AVISynthAPI.cpp - Ln 445-446.
I just need Display Frame and Encoded Frame info in the same row.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

I don't understand why your parser cannot read the two lines to get the information it needs.
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Compiling resources...
Compiling...
Command line warning D4002 : ignoring unknown option '/O3'
Command line warning D4002 : ignoring unknown option '/Qip'
Command line warning D4002 : ignoring unknown option '/Qunroll'
Command line warning D4002 : ignoring unknown option '/Qparallel'
alloc.cpp
getbit.cpp
gethdr.cpp
getpic.cpp
global.cpp
fatal error C1600: unsupported data type
idctfpu.cpp
idctref.cpp
motion.cpp
MPEG2DEC.cpp
store.cpp
text-overlay.cpp
Utilities.cpp
vfapidec.cpp
AVISynthAPI.cpp
misc.cpp
C:\dgdecode158src\AVISynthAPI.cpp(999) : fatal error C1601: unsupported inline assembly opcode
PostProcess.cpp
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
C:\dgdecode158src\misc.cpp(114) : error C2520: conversion from unsigned __int64 to double not implemented, use signed __int64
mc.cpp
C:\dgdecode158src\PostProcess.cpp(2191) : fatal error C1601: unsupported inline assembly opcode
mc3dnow.cpp
mcmmx.cpp
C:\dgdecode158src\mc3dnow.cpp(95) : fatal error C1601: unsupported inline assembly opcode
Error executing cl.exe.

DGDecode.dll - 16 error(s), 272 warning(s)

:shock:
Do you know what is wrong?

What version of nasm I'm supposed to use? Probably not 2013 right? ))
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

If you keep ignoring me I will start ignoring you.

Again, I don't understand why your parser cannot read the two lines to get the information it needs.

If you answer my points, then I may be inclined to answer yours.
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Sorry, I'm not ignoring... Just messing with this old stuff in a virtual machine... My head kinda...
There is no parser, I'm not a programmer. My "parser" is DebugView, and 2 lines to capture for 1 frame would be... well, not good. If I can have it all in 1 line I'll get a nice log which I can save as text, then import for example in excel and do further processing - isolate consecutive interlaced frame sequences and throw out other pulled down portions.
Could you tell me what version of nasm I'm supposed to use and if I need a specific compiler?
Sorry again. I didn't mean to be rude.
User avatar
admin
Posts: 4551
Joined: Thu Sep 09, 2010 3:08 pm

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by admin »

It's Visual C++ 6.0 with service pack 5 and the 'processor pack'. I have uploaded the extra executables you need to here:

http://rationalqm.us/misc/stuff

I'm still curious, though, if you can extract a single line from the multiple lines that make up the debug output, why can't you extract two?
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

I can extract as many lines as I want but how am I supposed to process them afterwards?
If I have a list of a single line for a frame then it's a simple matter of removing the lines I don't need.
For example I can do
x=(Field.top - Field.bottom) (pseudocode)
and include Display frame number in the same line and then filter output to only capture those lines that have x=0 and then further process that list.
How to deal with a two lines for a frame situation I'm not sure... Well I could probably delete every other line break in the text file to make them into one. But still.
Thanks for the links. Much appreciated.
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Thank you very much, I'll try that tonight. Is it built from the official source 1.5.8?
DAE avatar
far.in.out
Posts: 14
Joined: Wed Oct 09, 2013 5:40 am

Re: MPEG2 decode: DGMPGDec vs DGDecNV - different output?

Post by far.in.out »

Ok then, I'm on i5 so I don't really care about 3dnow. :)

UPD:
Yes! It compiled! I guess SP5 and exe's did the trick. :P
I modified it to output lines like this
DGDecode: Encoded Frame: = 0 = ; 16 ;
for frames where both fields are the same frame number. Now I just filter output for "= 0 =" and in excel only import the last column between ; ;.
Thank you very much for your help and patience. It made a difference.
Post Reply