DGDemux development

DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Leave the abort button alone, my opinion
If someone is hitting it they should have a good reason

As to who to root for
Root for them both, you can't lose that way
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

gonca wrote:
Sun May 03, 2020 8:44 am
Root for them both, you can't lose that way
Why didn't I think of that? :facepalm:
DAE avatar
Guest 2
Posts: 903
Joined: Mon Sep 20, 2010 2:18 pm

Re: DGDemux development

Post by Guest 2 »

Rocky wrote:
Sun May 03, 2020 5:00 am
I consider that to be a feature. ;)
LOL! :mrgreen:

I want "woot woot" steamboat too!
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

You can replace finished.wav with a different one if you like. Is that what you meant?
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Re subtitles on John Wick 3 Parabellum
12a0.png
Please notice the timing of the last subtitle
12a1 and 12a2 and 12a3 are also fine on my copy of this movie
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

I don't know that app. Please tell me what I am supposed to notice.
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

The report on John Wick 3 was that the first subtitle stopped at 1:06 (half way thru movie)
The screenshot shows a sub at 2:01:55
It is in the white background area at the bottom
All four subtitles are fine (demuxed properly by DGDemux)
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Is that the European UHD version? Did you want to reply at the other forum?
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

It is the Canadian version (almost same as US version, if not identical)
Just pointing out that the issue might not be in DGDemux before you buy discs, unless you like the John Wick series
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Thank you, gonca. I have a good budget for disks and I do like the John Wick stuff. If he comes through with a link, I'll get it and see what's what. Hard to see a way DGDemux could make empty sup files.
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

@domy

Can you please post your current script with instructions how to use it? I could comment much more germanely with that in hand.

One thing I keep coming back to is the overlaps. For perfection we want to prefer to delete duplicated audio. For example, when there is duplicated audio is it between the [minor-first major] or between the [first major-second major]? Or even [second major-first minor]? Is it a whole frame's worth? What do you think?
DAE avatar
domy
Posts: 28
Joined: Fri Mar 20, 2020 10:50 am

Re: DGDemux development

Post by domy »

@Rocky

I wrote it up in my previous post a few pages back. I've kept it up to date, that's the latest version I got. Just paste it in a new .ps1 file and call it from the powershell console like in the given examples.

So far, the duplicated audio I've seen is always at the [minor-first major] boundary. It's always either 1 or 0 frames that overlap, never 2. This is true for all segments of Toy Story 4 and Monsters University.

I don't see a way to make 100% sure we only delete duplicated audio frames. In the general case, you can decode the two streams and match the last 40 samples of stream A against the first 40 samples of stream B (they might not match bit-perfectly though). That would work unless the gaps are completely silent, which happens more often than one would think. I'd guess around a quarter of the Monsters University gaps are silent or very close to it. How do you figure out if two THD frames contain duplicated audio if they're both silent?

Perhaps the ideal solution is to combine the Bresenham algorithm with duplicated audio detection. If a segment boundary isn't silent and contains duplicated audio, delete a frame. Otherwise, fall back to Bresenham and check the current accumulated overrun and determine from that whether to cut a frame or not.
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Thank you. I'll look back at that then.

Yup, I had asked earlier how duplications could be distinguished from intended repeating stuff. I guess we should just ignore the duplication/overlap issue entirely.

Whether we need Bresenham for THD is still open.
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

@gonca

Can you please re-test Wick using "no split thd"? Thank you.
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

will do
DAE avatar
Guest

Re: DGDemux development

Post by Guest »

Using GUI and full disk structure on local drive (folder)
Demuxing video, thd and subs
no split.png
File details
detail no split.png
Demuxing subs only
pgs.png
file details
detail pgs only.png
On the original demux the GUI showed 99% but DGDemux closed by itself (not visible in task manager)
The pgs files are also slightly smaller with the no split thd option checked
HOWEVER, there are no empty sub files
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Thank you, gonca. I duplicated the crash in DGDemux.exe with Hellboy and -nsthd. I know what the problem is and will release fix shortly.
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Fixed and replaced build 24. Didn't want to make a full release for this.
DAE avatar
domy
Posts: 28
Joined: Fri Mar 20, 2020 10:50 am

Re: DGDemux development

Post by domy »

@Rocky

What do you do, exactly, in order to delete the minor frame at the end of a THD stream? Do you do anything special when appending two THD streams together? I want to try and see if I can turn the theoretically perfect THD algorithm into practice.
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

I never delete the last minor of the stream because there is no following double major to trigger deletion. I just flush the unwritten frames from the frame write-behind buffer. I could suppress the final frame but see no reason for that.

By appending you mean when moving to the next M2TS? I don't do anything. Everything is triggered by the presence of double majors. It so happens that these double majors occur only at file joints.

For the non-THD audio, things are different. Bresenham is implemented when the next file is moved to. A frame(s) may or may not be removed depending on the accumulated desync.

Please keep us informed of your results. I'm open to improving things if it does not become too complex for little gain.
DAE avatar
domy
Posts: 28
Joined: Fri Mar 20, 2020 10:50 am

Re: DGDemux development

Post by domy »

So the end result is that after appending all THD frames of all segments together, you find all double-major frames and for each one, delete the preceding minor frame. And it just so happens that double-majors only occur at the start of a segment. Did I get that right?

Do you ever manipulate the contents of a THD frame? As in, change a flag, swap a bit, etc?
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Yes, that is exactly correct, with one caveat. I do not actually append anything. I simply keep a list of the input M2TS files and start reading from the next one when the previous one runs out. The output is written as one continuous file, of course, which has the effect of appending the THD data.

I do not currently modify the frame contents but will have to do that to modify dialog normalization. Checksums would be corrected, etc. Thanks to my packet-to-frame buffering (data is packetized in a transport stream),I have access to full frames at a time, so editing will not be difficult.

I generally follow a KISS philosophy. :P
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Working on wave wrapping for PCM now. Got the W64 header written just fine. But when I played the resulting .w64 file all I got was HISSSS. Compared my file to one produced by sox.exe and it was clear I have to do a byte re-ordering (MSB -> LSB) per sample. Thank you Microsoft. Not! Going back to DGIndex land, I see that it does the swaps on the fly as the transport packets are demuxed. But we can't do that when there are more than 2 channels because a packet no longer necessarily has an integral number of samples, i.e., it can end with a partial sample. That can't be guaranteed for DGIndex either but in practice it was OK for 2-channel stuff. So we have to play some buffering tricks to always have an integral number of samples to do the re-ordering. Sherman, are you busy? I'm going to go lie down and watch John Wick 3. At least the W64 header is written correctly.
User avatar
Sherman
Posts: 637
Joined: Mon Jan 06, 2020 10:19 pm

Re: DGDemux development

Post by Sherman »

You take a nap, Rocky. I'll take care of it. Good work on the header. Probably needed a lot of spec studying. I have some ideas based on what you did for THD buffering.
Sherman Peabody
Director of Linux Development
User avatar
Rocky
Posts: 3805
Joined: Fri Sep 06, 2019 12:57 pm

Re: DGDemux development

Post by Rocky »

Sherman wrote:
Fri May 08, 2020 1:57 pm
Probably needed a lot of spec studying.
A boatload!

Thanks for taking over, I'm outta here.
Post Reply