How to parse an M2TS file?

[Sherman]

Sure, Mr hubblec4, I am here for you.

We have in the spec for PES_packet():

Code: Select all

if (PTS_DTS_flags = = '10') {
'0010' 4 bits
PTS [32..30] 3 bits
marker_bit 1 bit
PTS [29..15] 15 bits
marker_bit 1 bit
PTS [14..0] 15 bits
marker_bit 1 bit
}

The 33 bits of the PTS value are spread out over 5 bytes. That is the reason for all the shifting, or'ing, and masking to parse out the 33 bits. Let's see how good the bit operators are in Lazarus.

[hubblec4]

Ah, OK. Now it is clear.

The code in FreePascal for PTS parsing:

Code: Select all

VIDEO_PES_PTS := (UInt64(b[0] and $e) shl 29) or
                 (UInt64(b[1]) shl 22) or
                 (UInt64(b[2] and $fe) shl 14) or
                 (UInt64(b[3]) shl 7) or
                 (UInt64(b[4]) shr 1);

I have tested the code for an m2ts from a Blu-ray (New Hope) which is a multi edition Blu-ray.
There are "missing PES packets" and you mentioned in the code I should skip/continue with the next PES packet.

Code: Select all

1011[0]: 1048560 [11650,666666667 ms]
missing PES packet header
missing PES packet header
missing PES packet header
missing PES packet header
missing PES packet header
missing PES packet header
missing PES packet header
missing PES packet header
missing PES packet header
missing PES packet header
1011[11]: 1086097 [12067,744444444 ms]

(The values for the c0, c1 and c2 are: 0, 1, 224)
This missing PES packets are unimportant? Or is there a difference to our test "wedding.tts" file?

And a third question about the first timestamp (first frame).
Why is this not a zero timestamp? Has this something to do with the IN- and OUT- times from the mpls?

[Sherman]

That should not be happening. I have duplicated it and will fix it.

Yes, the starting PTS need not be zero and it would typically be the IN_time, but not necessarily.

[Sherman]

OK, I updated main2.cpp and main3.cpp with the fix. There was a little bug in the case of afc = 3. For main3.cpp I also added some dots at the beginning to show progress. For video each dot is 1000 frames.

https://rationalqm.us/misc/main3.cpp

Here is the output for the bluray 1917 00013.m2ts:

...........................................................................................................................................................................
1011[0]: 54000000 [600000 ms]
1011[2]: 54003750 [600041 ms]
1011[1]: 54007500 [600083 ms]
1011[4]: 54011250 [600125 ms]
1011[3]: 54015000 [600166 ms]
1011[6]: 54018750 [600208 ms]
1011[7]: 54022500 [600250 ms]
1011[5]: 54026250 [600291 ms]
1011[9]: 54030000 [600333 ms]
1011[10]: 54033750 [600375 ms]
1011[8]: 54037500 [600416 ms]
1011[12]: 54041250 [600458 ms]
1011[11]: 54045000 [600500 ms]
1011[14]: 54048750 [600541 ms]
1011[13]: 54052500 [600583 ms]
1011[16]: 54056250 [600625 ms]
1011[15]: 54060000 [600666 ms]
1011[18]: 54063750 [600708 ms]
1011[17]: 54067500 [600750 ms]
1011[20]: 54071250 [600791 ms]
1011[19]: 54075000 [600833 ms]
1011[22]: 54078750 [600875 ms]
1011[21]: 54082500 [600916 ms]
1011[23]: 54086250 [600958 ms]
1011[25]: 54090000 [601000 ms]
1011[24]: 54093750 [601041 ms]
[etc.]

The frame rate is 24, so the PTS delta is 41.67 ms. Maybe I should have printed the ms as a decimal like you did.

[hubblec4]

That should not be happening. I have duplicated it and will fix it.

Great. Thank you.

Yes, the starting PTS need not be zero and it would typically be the IN_time, but not necessarily.

OK. For later processes it is necessary to start with a zero timestamp. But I guess it is easy to subtract the first PTS value from all PTS's.

[hubblec4]

OK, I updated main2.cpp and main3.cpp with the fix. There was a little bug in the case of afc = 3. For main3.cpp I also added some dots at the beginning to show progress. For video each dot is 1000 frames.

https://rationalqm.us/misc/main3.cpp

Very nice, thanks.

The frame rate is 24, so the PTS delta is 41.67 ms. Maybe I should have printed the ms as a decimal like you did.

Yes, I need later the PTS in ms with nano second accuracy.
(I see I could omit the last 3 digits, 6 digits are enough)

[Curly]

just a sec, can't find my lunch

[Sherman]

Good news, Mr hubblec4! I'll ask Rocky to mark this resolved then. Let me know if anything else arises that needs attention. Parsing doesn't have to be anything crazy difficult, as long as you follow the spec everything will be hunky-dory.

Curly is still looking.

[DG]

Great job, Sherman! A lot of people will benefit from your tutorial.

[Rocky]

Ditto!

If you parse out the payload and write it to a file, voila, you have a demuxer.

[Boris]

Special on surrender monkeys. Steep discount. Send PM!

[hubblec4]

Good news, Mr hubblec4! I'll ask Rocky to mark this resolved then. Let me know if anything else arises that needs attention.

A mega big thanks to you Sherman.

For the moment it is fine to parse only the PTS's.
But I will ask you if I have question and need more.

Parsing doesn't have to be anything crazy difficult, as long as you follow the spec everything will be hunky-dory.

OK. Now that I know where the extra 4 bytes come from (and a packet is therefore always 192 bytes long), everything should fit with the Spec.PDF and I can use it.

[hubblec4]

If you parse out the payload and write it to a file, voila, you have a demuxer.

Hi Rocky.

OK. Nice to know.
I had asked my self where are the data for the streams are stored.

Always we step by 192 Bytes forward and always is there on 5. position a Sync-Byte.
Means that the data for a frame is stored over multiple packets(in very small pieces)?

[Rocky]

Always we step by 192 Bytes forward and always is there on 5. position a Sync-Byte.
Means that the data for a frame a stored over multiple packets(in very small pieces)?

Yes, and that is why TS is a high-overhead format.

[hubblec4]

Yes, and that is why TS is a high-overhead format.

Oh yes, what a waste of Bytes.

How good that Matroska exists.

[Rocky]

TS has a high-overhead mainly for error resilience. Remember it was designed for broadcast applications.

https://www.obe.tv/why-does-mpeg-ts-still-exist/

[hubblec4]

Yes, that's true. Each format has it's own justification.

[hubblec4]

I have a question about duplicate timestamps for some Audio streams.

DTS-HD Master

Code: Select all

0.000000
0.000000
10.666667
10.666667
21.333333
21.333333
32.000000
32.000000
42.666667
42.666667
53.333333
53.333333
64.000000
64.000000

and E-AC3

Code: Select all

0.000000
0.000000
32.000000
32.000000
64.000000
64.000000
96.000000
96.000000

Has this something to do with the core/embedded streams of a track?
I guess I can simple remove all double lines?

[Sherman]

Yes, that is correct. If you needed to identify them as core versus embedded we'd have to parse more deeply.

[hubblec4]

For the moment it is fine. I delete/ignore the double lines.

[hubblec4]

Hi Sherman

I would like to ask for your help again.
It is better to distinguish whether the PTS belongs to the embedded part or not. Because with TruheHD it is not just duplicate time stamps.

And if it is not too much trouble, I would also like to deal with the parsing of the audio frame durations (especially for E-AC3).

[Sherman]

Sure Mr hubblec4. I'll get started on it right away.

[Sherman]

Please get main4.cpp from my misc directory as before. Here is the output when given a PID for an EAC3 track:

m2tsparse G:\Rips\A_NEW_HOPE\BDMV\STREAM\00902.m2ts 1101
EAC3 duration = 0.032000
..................................................................................................................................................................................
1101[1]: 1048560 [11650 ms]
1101[0]: 1048560 [11650 ms]
1101[3]: 1051440 [11682 ms]
1101[2]: 1051440 [11682 ms]
1101[4]: 1054320 [11714 ms]
1101[5]: 1054320 [11714 ms]
etc.

That's how you get duration for EAC3. If you have doubts about getting duration for other audio formats lemme know. When you have absorbed this I will move on to the identification/parsing of the embedded streams.

BONUS TIP: If you have already gotten the PTSs you can get the duration by taking the difference of two adjacent ones. Would need to take into account embedded streams of course. Might be a more general way that could avoid parsing.

[hubblec4]

Please get main4.cpp from my misc directory as before.
...

Many thanks. I'll take a closer look at it in the next few days, but my first look tells me it shouldn't be difficult.

If you have doubts about getting duration for other audio formats lemme know. When you have absorbed this I will move on to the identification/parsing of the embedded streams.

I suspect that I should parse other audio formats. But for example, with an AC-3 track, the duration is always fixed at 32ms. In that case, everything is known, but if I want to parse in m2ts, it would be enough if I recognized that it was AC-3 for the PID, right?

BONUS TIP: If you have already gotten the PTSs you can get the duration by taking the difference of two adjacent ones. Would need to take into account embedded streams of course. Might be a more general way that could avoid parsing.

It's funny that this was my first thought too, but I thought it wasn't clean and real professionals wouldn't program it like that. But it's good to know that it can be done like that.

So the question is whether I even need to parse the durations if the duration can also be determined using the PTSs.
Do you think that still makes sense?

[Sherman]

I suspect that I should parse other audio formats. But for example, with an AC-3 track, the duration is always fixed at 32ms. In that case, everything is known, but if I want to parse in m2ts, it would be enough if I recognized that it was AC-3 for the PID, right?

Right.

So the question is whether I even need to parse the durations if the duration can also be determined using the PTSs. Do you think that still makes sense?

Rocky (who used to work on set-top box backend HW/SW) told me there is a wrinkle involved. Since presentation times are evenly spaced, it is not essential to include a time stamp in every presentation unit. Instead, time stamps can be interpolated by the decoder, but they must not be more than 700 ms apart. Now, is timestamp interpolation something for you to worry about? He says he's never seen interpolation on bluray/UHD disks, nor on audio streams, although he has seen it on broadcast video streams (e.g., DVB, presumably for bandwidth saving). It's likely that in practice for bluray/UHD disks both ways are fine. But maybe you prefer to be safe and not sorry.

Another complication is that if the video stream uses field pictures (unlikely for bluray/UHD) then the PTS difference would be halved. That would also complicate frame counting, as a frame would be made up of two field pictures. In practice it's not serious because it doesn't apply to audio and for video you can get the duration from the video frame rate. However, if you are needing to count frames you need to take this into account and not just count the PUSIs.

Let's worry about these things if and when you run into them, which might be never.

BONUS TIP #2: If you want to avoid the sorting, which is only needed for video where pictures need to be re-ordered due to B pictures, you could parse the decode timestamp (DTS) instead of the PTS.