Manually select M2TS to demux

User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Sun May 07, 2023 9:33 am
The problem arises when a stream is divided into many M2TS files. Because the granularity of video and audio access units is different, in a given M2TS the audio and video do not end at the same point. ...

++++++++++++++++++ (video)
---------------------------- (audio)
Yes, this was so far known to me and is the same issue with the Matroska container for merging splited mkv parts. 99,99% there are audio data after video data and that produces gaps.
Rocky wrote:
Sun May 07, 2023 9:33 am
....Normally, when the disk is played there are PTS timestamps so that audio sync can be maintained by the player. However, when demuxing, the PTS timestamps are lost.
That are new info, thanks. I had read already about PTS, but it is not fully clear to me. And I had never a use case for my end.
Rocky wrote:
Sun May 07, 2023 9:33 am
So in this case, during demuxing that excess audio must be removed. It's generally a very small amount, which would be inconsequential if there are not many M2TS files. However, it accumulates and for a large number of M2TS files, the desync becomes obvious.
Yes, and always noticeable for segmented-movies because the chapters are no longer in sync to video.
Such a file plays fine and while watching there is no disturbation, but the knowledge there is something wrong with audio overlapping disturbs.
It is a bit ironic that the issue comes from the audio but it affect the video stream.
My wild guess: a Matroska muxer like MKVToolNix try to keep all the data, and therefor some audio frames have no video frame for a certain timestamp.

Rocky wrote:
Sun May 07, 2023 9:33 am
My form of gaps correction works like this. As the stream plays we track the amount of video time and audio time accumulated. If the difference exceeds a threshold we decide that audio unit(s) must be dropped. They are dropped from the beginning of the following M2TS.
A noob question: Why are the data are dropped from the following M2TS and not from the current one?

Rocky wrote:
Sun May 07, 2023 9:33 am
So it should be clear that the goal is to maintain sync during playback and the tracking of the amount of accumulated audio and video requires the M2TS's to be demuxed in playing order. Trying to correct a gap between two M2TS files that are not in playing order is meaningless and useless.
Yes, sync of streams over all others!
And with removing the excess audio from a next M2TS which is not in play order makes no sense and is also a fault.
I would love to know how eac3to does it. I have a lot of multi Edition Blu-rays converted to mkv and there was never an issue after demuxing the audio. Very bad that this source code is not public.

Rocky wrote:
Sun May 07, 2023 9:33 am
... Let's first note that DGIndexNV's CLI can accept a list of input files and it does perform gaps correction. ...
OK, good to know.

Rocky wrote:
Sun May 07, 2023 9:33 am
Now, adding list acceptance to DGDemux would merely duplicate what DGIndexNV CLI can already do, except that (currently) DGIndexNV cannot disable gaps processing. However, there are two additional concerns for you: 1) DGIndexNV requires an nVidia card, and 2) there is no Linux version of DGIndexNV.
Missing of Linux support is the point why I don't want to use it.
Rocky wrote:
Sun May 07, 2023 9:33 am
So, a possible approach is to add the file list input option to DGDemux and warn users that the results may or may not be acceptable for disordered lists. That would be true with or without gaps processing, because with it, it's doing meaningless things, and without it, you may have the original desync problem possibly occurring. At least you would not need nVidia and you'd have the option to use or not use gaps processing. We could probably give you such a DGDemux test version and then if you find it is usable for your needs, it could be released with appropriate warnings.
Yeah, we need at the end some tests to see if this is working or not, I think it depends on your gap-removing algo.

Rocky wrote:
Sun May 07, 2023 9:33 am
Now, the issue of linux support. From my vantage point there has been zero interest in the linux version of DGDemux, and that's why I stopped updating it quite some time ago. Believe me, maintaining a linux system and porting updates from the windows versions is highly odious for me. It's very hard to justify it for one person's needs. So my question for you is, if things never get ported to linux, is it worth proceeding?
Mmh without Linux support it would be indeed bad. I was so happy to see there are Linux support, but I also realized the lower version number.
The other feature, "forced subs also" is not so important as like this.

I can understand it makes less sense to maintain a Linux version when nobody needs.
But now a friendly developer asks :-)
I'm a bit sure when I implement DGDemux in my chapterEditor you will have some downloads.
Multi-Edition mkv creation is more and more coming, I have many requests for that.
I don't bundle external programs so each user have to download these programs.

Rocky wrote:
Sun May 07, 2023 9:33 am
Final point. EAC3TO remains an option. The only thing you said that seems to rule it out is "it has issues with eac3". That seems ironic to me, so can you please share what these issues are?
eac3to is still a very good option, but I think DGDemux is a bit better.
One point is eac3to will NEVER be run on Linux native. DGDemux has a Linux support.
UHD topic: I had read about some issues but I have no UHD disc.

And yes it sounds like ironic that eac3to has issues with eac3 streams.
I can't remember me on the first disc, because it was not a multi edition disc and the eac3 stream could be demuxed with TsMuxeR and also read directly by MKVToolNix.
But currently I have here the "Ligthyear" movie which is a seamless branching disc.

Here the last lines of eac3to output

Code: Select all

a06 Extracting audio track number 6...                                         
a06 Removing AC3 dialog normalization...                                       
Applying (E-)AC3 delay failed.a06                                              
Aborted at file position 1048576.    
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

SomeHumanPerson wrote:
Sun May 07, 2023 12:07 pm

Back on the main subject, I'm failing to see how hubblec4's scenario ever works correctly unless MKVToolNix is somehow handling the sync problems by itself (I have no idea on the technical aspect of how that might be done), in which case your "possible approach" of adding the file input list to DGDemux and allowing the user to disable (or leave enabled) gaps processing should be workable, sidestepping the DGIndexNV issues (Nvidia + mandatory gaps processing).
MKVToolNix has only one bottleneck and that is the audio gap-processing.
Video frames and subtitle are ever fine, but audio uses a structure of 8 frames(not all audios).

I'm not an expert in technical details but I have a lot of experience with seamless branching discs.
With my chapterEditor is it easy to create an mkv from such discs and MKVToolNix is used to mux everything.
User avatar
Curly
Posts: 712
Joined: Sun Mar 15, 2020 11:05 am

Manually select M2TS to demux

Post by Curly »

I hear what yer saying but.
User avatar
Sherman
Posts: 576
Joined: Mon Jan 06, 2020 10:19 pm

Manually select M2TS to demux

Post by Sherman »

But what?
User avatar
Curly
Posts: 712
Joined: Sun Mar 15, 2020 11:05 am

Manually select M2TS to demux

Post by Curly »

Yer butt. Knerk.
User avatar
Britney
Posts: 145
Joined: Sun Aug 09, 2020 3:24 pm

Manually select M2TS to demux

Post by Britney »

Stop it you guys. This is important.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Thinking...
User avatar
Sherman
Posts: 576
Joined: Mon Jan 06, 2020 10:19 pm

Manually select M2TS to demux

Post by Sherman »

Pretty good, but don't get complacent.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Hi
I found this thread
https://www.rationalqm.us/board/viewtop ... 5dfa7b0b92
and first I thought it is for the gap-processing feature.
But after reading again carefully I realized it is an other feature for removing audio garbage.

First questions was directly: What is the difference between of this two features
"Chop excess audio" and "gap processing"?

If the "chop excess audio" feature removes data from the current m2ts, than I see no problems to use a random m2ts list. (OK I hope there is no issue)
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

gaps processing: Throughout the entire stream, strategically remove audio at M2TS junctions to maintain overall sync as the movie plays. Some gaps may not have any audio removed and some may; it depends on the accumulated desync due to M2TS junctions.

chop excess audio: Within each M2TS, remove audio with PTS beyond the out time of that M2TS.

I have questions for you that I'd like to understand before addressing your posts.

1. How do you know the in times and out times of the M2TS segments? The demuxer can give you only overall elementary streams. Are you parsing the disk structure and creating some sort of data structure?

2. How does MKV seek to a time when the segments are disordered?

3. Do you need the demuxer to demux chapters, or is that something you do on your own?

P.S. Ignore the children in the background. Never forget, though, they are our future. :?
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Mon May 08, 2023 5:47 pm
gaps processing: Throughout the entire stream, strategically remove audio at M2TS junctions to maintain overall sync as the movie plays. Some gaps may not have any audio removed and some may; it depends on the accumulated desync due to M2TS junctions.

chop excess audio: Within each M2TS, remove audio with PTS beyond the out time of that M2TS.
Ok, and is a combination of both methods possible? Could this be an advantage?

Rocky wrote:
Mon May 08, 2023 5:47 pm
1. How do you know the in times and out times of the M2TS segments? The demuxer can give you only overall elementary streams. Are you parsing the disk structure and creating some sort of data structure?
Yes, cE parses all the mpls files which are used for a multi-edition mkv creation and collect all the IN- and OUT- times. These times are used to create the magic chapter-structure.
Rocky wrote:
Mon May 08, 2023 5:47 pm
2. How does MKV seek to a time when the segments are disordered?
How exactly a player seeks in a Matroska file, I don't know.
But with the timestamps from the chapters and the Matroska Cues it is not heavy to get the correct stream position. If there are no Matroska Cues, seeking will take some "more" time.

Rocky wrote:
Mon May 08, 2023 5:47 pm
3. Do you need the demuxer to demux chapters, or is that something you do on your own?
No, I don't need it. I only trust on my chapter handling :-)
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Hi

After a little while I wanted to ask about the progress.

I already have another interested user who would like to edit UHD material and eac3to seems to fail again.

I took a closer look at the eac3to log.
So eac3to ALWAYS makes a second pass to remove the gaps. This seems to be necessary as apparently the "garbage" will be removed from the current m2ts and not the next (but that's just my guess).
eac3to collect all the gaps and perform removing it in the second pass.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Maybe good news.
The Doom9 user has successful created a mutli-Editon mkv by using DGIndexNV for demuxing the second Editions m2ts files.
https://forum.doom9.net/showthread.php? ... ost1988461

If you "simple?" implement this feature in DGDemux all is fine and you make me and many other mutli-Edition mkv fans very happy.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

I had asked the Doom9 user for another test where ALL m2ts for all Editions are loaded to DGIndexNV to get a single full Audio stream like with eac3to...and it works!!
https://forum.doom9.net/showthread.php? ... ost1988497

It seems your Audio-gap-remove-algo works perfect.

I hope it is not so much work to implement it in DGDemux.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Would behaviour like this be acceptable?

You still would open the main disk MPLS. However, you also provide a list of M2TS to add to the main disk list. Those would be the M2TS's needed for the extra editions.

The point is that everything becomes much harder if we don't do an MPLS open and go straight to an M2TS list. Simply adding some extra M2TS's is easy. You did say that the order does not matter.

If this is acceptable then I can go to implementation.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Wed Jun 21, 2023 6:39 am
Would behaviour like this be acceptable?

You still would open the main disk MPLS. However, you also provide a list of M2TS to add to the main disk list. Those would be the M2TS's needed for the extra editions.
I had also the same thoughts about this. I know a much info are located in the mpls file and it is very useful to have this info.

I have this in my mind:
DGDemux.exe -i 00000.mpls -m2ts(or short -m: a comma separated list of m2ts names without the .m2ts extension) 00100,00101,00199 -demux

How would you do that?

What is when I have to ignore the first or last m2ts from the main.mpls.
Are the switches -sf and -sl still recognized?

It is possible to set a main.mpls for the all the info you need but ignore the m2ts list from the mpls and use the entire m2ts list from the command line?

Of course it will be the best when I can set a fully personal m2ts list, but I guess all will work with a main.mpls plus extra m2ts files.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

hubblec4 wrote:
Wed Jun 21, 2023 7:16 am
I have this in my mind:
DGDemux.exe -i 00000.mpls -m2ts(or short -m: a comma separated list of m2ts names without the .m2ts extension) 00100,00101,00199 -demux
Let's go with just the list of extras for now and consider full freedom later.
How would you do that?

What is when I have to ignore the first or last m2ts from the main.mpls.
Are the switches -sf and -sl still recognized?
Sure, but last would mean the last one of the main M2TS list.
It is possible to set a main.mpls for the all the info you need but ignore the m2ts list from the mpls and use the entire m2ts list from the command line?
See above.
Of course it will be the best when I can set a fully personal m2ts list, but I guess all will work with a main.mpls plus extra m2ts files.
OK, good, your buddy RivDoVi seems to like the idea.

So, I ran a quick experiment appending an M2TS to the MPLS file list and everything worked fine. Almost trivial; the CLI parsing is the hardest thing and that is, well, trivial. The only question I have for you is how will you find the proper offsets for the pieces in the full combined ES files? Are you expecting me to output the file offsets or something like that? Or am I supposed to demux by M2TS as for episodes? You may have already told me. ;)
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Wed Jun 21, 2023 8:08 pm

So, I ran a quick experiment appending an M2TS to the MPLS file list and everything worked fine. Almost trivial;
Good to read.
Rocky wrote:
Wed Jun 21, 2023 8:08 pm
The only question I have for you is how will you find the proper offsets for the pieces in the full combined ES files? Are you expecting me to output the file offsets or something like that? Or am I supposed to demux by M2TS as for episodes? You may have already told me. ;)
I don't understand fully. What is "ES file"?
What do mean with "offsets"?

I had created a 2-edition mpls for Gladiator UHD
https://forum.doom9.net/showthread.php? ... ost1988695
All works fine and cE was able to create the chapters like ever.

Do you mean: if you append the m2ts files there are issues with time stamps?
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

hubblec4 wrote:
Thu Jun 22, 2023 6:16 am
I don't understand fully. What is "ES file"?
ES stands for elementary stream. It is the raw stream that is placed in a container, e.g., M2TS. When you demux a stream you remove it from the container. When removed all timestamps are lost, and the M2TS boundaries are lost. For example, all you'll have from DGDemux is a single large HEVC file.
What do mean with "offsets"?
It just means the position in the file, i.e., byte number.

Suppose we have a simple case with an MPLS with 5 M2TS files and one extra M2TS. The video ES output by DGDemux will be the concatenation of the video of all 6 of the M2TS files (with the extra one at the end). If you need to deliver video for the 2nd edition using the extra M2TS, you will need to be able to locate both the segment of video to be replaced and the location of the video to replace with. I do not understand how you are doing that. So I cannot determine whether what I am doing is what you need, and I won't try to code something I don't understand.

Let's put it this way. What exactly do you expect DGDemux to deliver to you for the case just described? If you say a single large HEVC, again, how will you find the parts that you need?
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Many thanks for this explanation.
Rocky wrote:
Thu Jun 22, 2023 7:50 am
... If you need to deliver video for the 2nd edition using the extra M2TS, you will need to be able to locate both the segment of video to be replaced and the location of the video to replace with. I do not understand how you are doing that. So I cannot determine whether what I am doing is what you need, and I won't try to code something I don't understand.
All what cE does is collect all the IN- and OUT- times of each m2ts file using the corresponding mpls files and generates a new Segment map with the correct order of the m2ts files. The same order is used for eac3to.
cE calculates the duration for each m2ts, and I trust that these durations are match with the time stamps of the ES.
Matroska uses intern the ordered chapters to get a straight forward virtual timeline for the second edition.

The algo to generate the ordered chapters is not so simple if the original chapters have to be preserved.

At the end you have be not worry about nothing. Appending the m2ts is total enough, the rest is cE's and Matroska's and a player's part.

Rocky wrote:
Thu Jun 22, 2023 7:50 am
What do mean with "offsets"?
It just means the position in the file, i.e., byte number.
OK, yes. And in this case you could output the offset for each appended m2ts for the demuxed ES?
For example
offset ES.hevc first appended m2ts -> 123456576789
that means at file position 123456789 in the ES.hevc starts data from the first appended m2ts
I'm correct with this?
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

You are not answering me directly for the questions I asked. Let me try again.

In the example I gave (5 M2TS in main edition, one extra M2TS to replace one of the main M2TS for a second edition), are you expecting to receive from DGDemux a single large HEVC file with the video contents all concatenated with the extra one at the end?
I trust that these durations are match with the time stamps of the ES.
There are no timestamps in the ES! That is my whole point.
Appending the m2ts is total enough
DGDemux does not append M2TS files. It concatenates ES video data demuxed from the M2TS files. This lack of precision in your answers is a big problem for me.

Your understanding of offset is correct, however, you have never asked for that, and everything you've said implies you don't need the offsets.

Please just tell me exactly what you are expecting DGDemux to deliver in this example case. If it is precise enough for me to code, I'll give you a test build and we can see if it works for you.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Thu Jun 22, 2023 9:52 am
In the example I gave (5 M2TS in main edition, one extra M2TS to replace one of the main M2TS for a second edition), are you expecting to receive from DGDemux a single large HEVC file with the video contents all concatenated with the extra one at the end?
Yes. "Simple" concatenated like all other m2ts content of the first edition to get a ES with all the content of the used m2ts files in a specific order( after the first edition's m2ts files).

Rocky wrote:
Thu Jun 22, 2023 9:52 am
I trust that these durations are match with the time stamps of the ES.
There are no timestamps in the ES! That is my whole point.
Ah, OK. All the timings are present in the mpls and I was always sure this must be enough.
Because mpls use the same thing with time stamps (in BD-Clock time).
Also Matroska uses time stamps and a player can read this time stamps and jump to the correct position.
Rocky wrote:
Thu Jun 22, 2023 9:52 am
Appending the m2ts is total enough
DGDemux does not append M2TS files. It concatenates ES video data demuxed from the M2TS files. This lack of precision in your answers is a big problem for me.
Please excuse me. I mean the content of each m2ts which are used to create the ES have to be appended.
Appending the m2ts files is a process in DGDemux before a demux start (I guess), and appending is in this case a virtual thing.
Rocky wrote:
Thu Jun 22, 2023 9:52 am
Your understanding of offset is correct, however, you have never asked for that, and everything you've said implies you don't need the offsets.
That's correct, I think I don't need this.
Rocky wrote:
Thu Jun 22, 2023 9:52 am
Please just tell me exactly what you are expecting DGDemux to deliver in this example case. If it is precise enough for me to code, I'll give you a test build and we can see if it works for you.
I hope I was precise this time.
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

OK. I still cannot visualize what you are doing with a combined ES after I present it to you, but I suppose that is not crucial for me to understand. I'll make a test build and see where things go from there.
User avatar
hubblec4
Posts: 219
Joined: Tue May 02, 2023 6:03 pm

Manually select M2TS to demux

Post by hubblec4 »

Rocky wrote:
Thu Jun 22, 2023 11:17 am
..
I'll make a test build and see where things go from there.
That sounds wonderful, thank you.

Yes it is not easy to understand what cE's heuristic is doing to create ordered chapters to get each edition while playing the file.
I had often rewritten the heuristic, but now any ordered input and user chapters can be processed.

I asked for some switches which are used(-sf and -sl) and forgot the most important thing: the Angles.
I guess also the Angle selection will be used to get the correct m2ts order for the first edition?
User avatar
Rocky
Posts: 3556
Joined: Fri Sep 06, 2019 12:57 pm

Manually select M2TS to demux

Post by Rocky »

Angles should be fine.

Please test this:

https://rationalqm.us/misc/DGDemux_hubblec4.rar

You will add an option like this:

DGDemux [existing stuff] -extras 00280,00281,00282

These extra M2TS's must be in the disk STREAM directory along with the main ones. There is no error handling yet, so be careful.
Post Reply