Problem with multi-edition MKV creation

[Curly]

im not busy today didja want me to do it

[hubblec4]

Yes, that's true. But as far as I understand, this only affects the first m2ts. So far, I haven't seen this behavior with seamless-branching material. Does the DG team perhaps have a Blu-ray where this is the case?

Haha, you gave me one. Please recall COLUMBO from this thread:

viewtopic.php?f=16&t=1308

The same M2TS was repeated with a different start time for each, in order to assemble a montage. Also, making assumptions about something allowed by spec sounds dangerous, and you should at least prepare/allow for it in your design, even if you do not immediately implement it. IMHO. Finally, we also have cases where it affects the last M2TS.

COLUMBO is not a multi-edition Blu-ray and does not have seamless branching because there is only one m2ts.
This Blu-ray is so special and falls under the 0.0001%.
However, the mpls file is used directly for MTX for muxing, so that you can now work specifically with the IN and OUT times and it probably shouldn't be a problem to create a time stamp file for the audio tracks.

As far as I can tell, there is excess data only in films that consist of one m2ts.
And yes, there can be data before the IN time and after the OUT time. But here too, the mpls file could be used directly for muxing.
But as I said, time stamp files are only needed for a multi-edition Blu-ray.

That sounds reasonable. If I did that is it something you would use, or would you be using your own stuff?

If you install it, I will definitely add it as an option in cE.
I assume that if you install it, it will be better/safer than my current workaround.
At the moment I am making it quite "easy" for myself, as I only check what codec is used for the audio track, and then the FPS is hard-coded. This tells me the playing time for the different audio frames.
Calculating the time stamps is then quite simple, you just keep adding the audio frame duration until the last video frame is reached (more precisely, the OUT time of the m2ts file, which I already determined when parsing).

I am also sure that I have not yet stored the FPS for all audio codecs in the source code.

I ask because you are the only person I know that is a potential user of this in a processing path or application.

There is me and the other user mentioned.
The problem is that this is completely new information and many multi-edition lovers don't even know it yet.

I can well imagine that if the process is built into well-known tools, word will get around.
cE already has many users who create multi-edition MKVs.

[Rocky]

Thank you for your reply.

COLUMBO is not a multi-edition Blu-ray and does not have seamless branching because there is only one m2ts.

I wasn't aware that you are interested only in multi-edition. Seems to me your new approach without demuxing could be equally valuable for single-edition. Formally, COLUMBO 00006.mpls is seamless branched, although if you are interested only in the main movie, or you are satisfied that the incidence of disks like this is vanishingly small, I can understand you disregarding it. If I was going to make a BD2MKV utility using your new approach, however, I would want it to be as general as possible.

As far as I can tell, there is excess data only in films that consist of one m2ts.

Sadly I discarded my notes by accident, but I specifically remember cases with multiple M2TSs where there was excess data at the start of the first M2TS and/or the end of the last M2TS (although I suppose such excess data in the last M2TS would be a minor irritant easily handled). I also remember that there were no cases where an intermediate M2TS had excess data. It was these cases and COLUMBO that made me decide to implement the window honoring for all M2TSs in DGDemux.

And yes, there can be data before the IN time and after the OUT time. But here too, the mpls file could be used directly for muxing. But as I said, time stamp files are only needed for a multi-edition Blu-ray.

Couldn't your new method be useful for single-edition seamless branched discs as well, in order to avoid the demuxing?

I am also sure that I have not yet stored the FPS for all audio codecs in the source code.

I can give you that information if you need it.

[hubblec4]

I wasn't aware that you are interested only in multi-edition. Seems to me your new approach without demuxing could be equally valuable for single-edition.

Since MTX only has problems when muxing seamless-branching discs, films that only consist of an m2ts are not really of interest. Because there are no overlapping audio tracks that need to be "prepared" with time-stamped files.

Formally, COLUMBO 00006.mpls is seamless branched, although if you are interested only in the main movie, or you are satisfied that the incidence of disks like this is vanishingly small, I can understand you disregarding it.

Yes, it is true that a single m2ts is used multiple times. Audio overlaps should also occur if muxed directly with MTX. So I think that the time stamp files could/will help here too.
Since my workaround sticks to the IN and OUT times, I assume that correct time stamp files are also generated.

If I was going to make a BD2MKV utility using your new approach, however, I would want it to be as general as possible.

Yes, that would make absolute sense.

Sadly I discarded my notes by accident, but I specifically remember cases with multiple M2TSs where there was excess data at the start of the first M2TS and/or the end of the last M2TS (although I suppose such excess data in the last M2TS would be a minor irritant easily handled). I also remember that there were no cases where an intermediate M2TS had excess data. It was these cases and COLUMBO that made me decide to implement the window honoring for all M2TSs in DGDemux.

It's a shame that the notes are gone, of course. Information like this is always worth its weight in gold. I would have liked to know which multi-edition films have this excess data. But I'll be paying more attention to this in the future and collecting data.

Yes, the COLUMBO disc is very special and has led to DGDemux now treating the m2ts files just like MTX if the IN and OUT times are known. That's absolutely fantastic.

Unfortunately, the topic is still quite new to me and so far I only know that there is such excess data, which MTX ignores if the mpls file is used for muxing.

But if multiple m2ts files are used, MTX is only fed with the m2ts files, and then the excess data could possibly be muxed.
Yes, that could be a problem. However, the ordered chapters can compensate for that without any problems.
To do that, I or cE would just need to know exactly what the time offset is until the actual film data begins. This offset time is simply added to all chapters.

Couldn't your new method be useful for single-edition seamless branched discs as well, in order to avoid the demuxing?

Yes, absolutely. As described above, the timestamp files could be used whenever seamless branching discs are involved, regardless of the format (single or multi m2ts files).

I am also sure that I have not yet stored the FPS for all audio codecs in the source code.

I can give you that information if you need it.

That would be great.

[Rocky]

Yes, absolutely. As described above, the timestamp files could be used whenever seamless branching discs are involved, regardless of the format (single or multi m2ts files).

??? Seamless branching implies > 1 M2TS file, so I don't understand this.

That would be great.

OK, gotta get it from our source code.

[Rocky]

OK, you're going to have to parse deeper in some cases. I'll give the access unit duration in seconds.

AC3: duration = 0.032

TrueHD embedded AC3: duration = 0.032

EAC3:

Code: Select all

// Assumes 48K sampling!
switch (numblkscod)
{
case 0:
	duration = 0.00533;
	break;
case 1:
	duration = 0.01067;
	break;
case 2:
	duration = 0.016;
	break;
case 3:
	duration = 0.032;
	break;
}

DTS: duration = 0.032 / 3

DTS Express:

Code: Select all

if (lbr_sample_rate < 16000)
	num_samples = 1024;
else if (lbr_sample_rate >= 16000 && lbr_sample_rate < 32000)
	num_samples = 2048;
else
	num_samples = 4096;
duration = (float)num_samples / lbr_sample_rate;

TrueHD: duration = 1/1200 for 48000 Hz/96000 Hz/192000 Hz; duration = 1/1102.5 for 44100 Hz/88200 Hz/176400 Hz

If you need help parsing deeper please post in Sherman's thread about M2TS parsing.

[hubblec4]

Yes, absolutely. As described above, the timestamp files could be used whenever seamless branching discs are involved, regardless of the format (single or multi m2ts files).

??? Seamless branching implies > 1 M2TS file, so I don't understand this.

I always assume that as soon as content/data is to be linked together, it is seamless branching.

There are the menu MPLS which append a single M2TS file together several times.

And then we have the COLUMBO Blu-ray, where only one M2TS file is used, and not even the entire file, but only fragments.

But they all have one thing in common: more than one Playitem is used in the MPLS.

So I assume that as soon as more than one Playitem is used, it is seamless branching.

EDIT:

Many thanks for the code, this will help me a lot, and I can compare if my existing hard coded values are fine.

[hubblec4]

About the Audio frame durations.

Do you have the value for PCM also?

I have one Blu-ray(single movie but it is splitted into 3 m2ts files) with a PCM track.

[Rocky]

Maybe it's a language issue but you said "whenever seamless branching discs are involved, regardless of the format (single or multi m2ts files)" which implies a seamless disk can have a single M2TS. But then you say "as soon as more than one Playitem is used". In my simple squirrel logic that is a contradiction. Anyway, let it go.

I'll try to get the values for LPCM. Which bluray is that?

[hubblec4]

Guys, it seems to me that Mr hubblec4's advance is all about correctly transcoding (or more precisely remuxing) from seamless branched bluray to MKV. The conventional approach is to demux with gaps correction and then remux to MKV. Mr hubblec4's new approach avoids all that. Of course, it is a wonderful thing, but as Rocky said DGDemux is a demuxer. Certainly we could create a whole new remuxing utility that implements the idea. But hasn't Mr hubblec4 already done that, and wouldn't we therefore be doing redundant development were we to make such a utility? We would never want to steal Mr hubblec4's thunder!

Up to now, everything is just theoretical and I for one am very happy that I can talk to a few people about this very specific topic.

I am also not sure whether the DG team should create a new tool. Because generating the time stamp files alone does not do much for a Blu-ray with seamless branching.
It only makes sense if ordered chapters are muxed into the MKV. And creating such a chapter structure is a completely different area.

In Madshi's eac3to tool, there was never an option to turn off gap correction. But this option is available in DGDemux. However, such a demuxed stream is no longer synchronized with the video. But if DGDemux were to create a time stamp file here, the audio in the MKV would be synchronized again.


The whole thing wasn't just my idea. The other user had a very big part in it.
It doesn't bother me at all if other programmers want to incorporate this into their tools. It's nice when we have lots of options.

[hubblec4]

Maybe it's a language issue but you said "whenever seamless branching discs are involved, regardless of the format (single or multi m2ts files)" which implies a seamless disk can have a single M2TS. But then you say "as soon as more than one Playitem is used". In my simple squirrel logic that is a contradiction. Anyway, let it go.

A PlayItem is part of the mpls. Each mpls can have one PlayItem or several. The m2ts that is used is stored in each PlayItem. And it doesn't matter whether there are different m2ts files, or just one m2ts that is repeated, or just fractions of an m2ts.

Therefore, the number of PlayItems in an mpls is crucial and not whether several m2ts files are used.
Of course, as soon as several m2ts files are used, it is always automatically seamless branching.
But it can also be seamless branching if only one m2ts file is used.

I hope Google translated it well this time.

[hubblec4]

I'll try to get the values for LPCM. Which bluray is that?

I have found it.
Public Enemy (Staatsfeind Nr.1).

[hubblec4]

I'll try to get the values for LPCM. Which bluray is that?

I have found a FPS value for PCM, it is 200. 5ms Frame duration.
Has LPCM another FPS value?

[Rocky]

Where did you find that? PCM and LPCM are the same thing.

[hubblec4]

Found directly isn't quite right.
I simply muxed an m2ts with PCM to Matroska and then looked in the MTX info tool to see what value it had.

[hubblec4]

Hi Rocky

I have a question about E-AC3.

OK, you're going to have to parse deeper in some cases. I'll give the access unit duration in seconds.

EAC3:

Code: Select all

// Assumes 48K sampling!
switch (numblkscod)
{
case 0:
	duration = 0.00533;
	break;
case 1:
	duration = 0.01067;
	break;
case 2:
	duration = 0.016;
	break;
case 3:
	duration = 0.032;
	break;
}

var "numblkscod": Is this value available in the mpls file or is this a parser task for the m2ts file?

[Rocky]

It's a field in the eac3 audio header:

https://www.atsc.org/wp-content/uploads ... 212-17.pdf

You would have to add deeper parsing (down to ES level) in the M2TS parser. Sherman can help with that if you need it. Ask him in his M2TS parser thread. School starts today but maybe he can crank it out for you fast.

[hubblec4]

Thank you Rocky.
I have now asked Sherman for deeper parsing.