Support forum for DG Tools

Posted: **Sat Jul 31, 2021 7:01 am**

One more request: functionality to create a qpfile based on chapter timestamps during indexing. I've sometimes noticed that x264 and x265 miss some obvious scene changes and may not place a keyframe where the chapter begins (or if the chapter begins without a scene change, the issue is even more probable).

https://x265.readthedocs.io/en/master/c ... ion-qpfile for specification, most likely frametype K would do here.

Posted: **Sat Jul 31, 2021 7:25 am**

One more request: functionality to create a qpfile based on chapter timestamps during indexing

+1

Posted: **Sun Aug 01, 2021 8:20 am**

"Specify a text file which contains frametypes and QPs for some or all frames."

What is a QP and how is it obtained?

Posted: **Sun Aug 01, 2021 9:25 am**

Rocky wrote: ↑
Sun Aug 01, 2021 8:20 am
"Specify a text file which contains frametypes and QPs for some or all frames."

What is a QP and how is it obtained?

QP is not needed in this case, it can be left out. QP is basically the frame quantizer, it's used if you want to force a fixed quantizer on some frame instead of the one the encoder decides to use.

Posted: **Sun Aug 01, 2021 10:01 am**

Well then, what is it that you are asking for exactly? You want a file with the frame number and frame type for all frames that start chapters?

Posted: **Sun Aug 01, 2021 10:51 am**

QP file usually used to override encoder's frame type decisions. It's fine to declare IDR on chapter start, so player will navigate faster. But honestly I believe that's not that urgent and can be achieved by some python script:

Code: Select all

import re
with open(r'chapter_file.txt') as f:
    lines = f.read()
    bmDict = []
    tList = re.findall(r'(\d+)=(\d+):(\d+):(\d+\.\d+)', lines)
    fps = 24000 / 1001
    for index, hr, min, sec in tList:
        sec = int(hr)*3600 + int(min)*60 + float(sec)
        bmDict.append(str(round(sec*fps)))
    with open(f'qp_file.qp', 'w') as q:
        for n in bmDict:
            q.write(f'{n} I -1\n')

Posted: **Sun Aug 01, 2021 11:34 pm**

Rocky wrote: ↑
Sun Aug 01, 2021 10:01 am
Well then, what is it that you are asking for exactly? You want a file with the frame number and frame type for all frames that start chapters?

Yes, exactly that - to ensure the playback starts from the correct place when jumping to a chapter. DJATOM's Python script is definitely nice as well
:bravo:

Posted: **Tue Aug 03, 2021 8:27 am**

OK, thank you. It'll be a while as the MP4 upgrade is quite involved and we want to get that done first.

Posted: **Sun Aug 08, 2021 7:11 pm**

DJATOM wrote: ↑
Sun Aug 01, 2021 10:51 am
QP file usually used to override encoder's frame type decisions. It's fine to declare IDR on chapter start, so player will navigate faster. But honestly I believe that's not that urgent and can be achieved by some python script:

Very interesting, thank you DJ. I think we can just convert that to C++ and generate the qp file when the chapters file is generated. I'll try it out.

Posted: **Mon Aug 09, 2021 6:59 am**

I have the qpfile stuff working and will give slipstreams for DGIndexNV and DGDemux later today.

Posted: **Mon Aug 09, 2021 7:32 am**

Rocky wrote: ↑
Mon Aug 09, 2021 6:59 am
I have the qpfile stuff working but I have two questions for you:

1. I create *.chapters.txt and *.qp.txt. Should I delete the .txt and just create *.chapters and *.qp?

2. Should I do (for example)

456 I -1

or, because the x265 doc says the QP field is optional, just

456 I

The x265 docs suggest the latter. Does it matter? Examples I have seen online use the -1.

1. I'd keep chapters as .txt but for QP, .qp would be a good extension IMO.
2. The latter at least works properly so it's the safest bet

Posted: **Mon Aug 09, 2021 11:24 am**

Thank you. I had already concluded to use -1 before your post, hence the edit. Regarding the extensions, I'll use .txt for both because double click opens my text editor.

Posted: **Mon Aug 09, 2021 12:14 pm**

I released the QP stuff as DGDecNV slipstream 231. DGDemux to follow.

Posted: **Mon Aug 09, 2021 1:14 pm**

Rocky wrote: ↑
Mon Aug 09, 2021 11:24 am
Thank you. I had already concluded to use -1 before your post, hence the edit. Regarding the extensions, I'll use .txt for both because double click opens my text editor.

Hey, no worries regarding the extension and x265 will override that quantizer value anyway since it's out of the default range for minimum and maximum.

Thanks a lot! :bravo:

Posted: **Mon Aug 09, 2021 1:53 pm**

You are most welcome, and thank you for the suggestion.

Posted: **Tue Aug 10, 2021 7:02 am**

QP file generation added to DGDemux slipstream 52.

Recent DGDecNV (230 and 231) and DGDemux (52) changes will now be ported to the linux versions.

Posted: **Wed Aug 11, 2021 6:49 am**

Nice!

Posted: **Wed Aug 11, 2021 12:33 pm**

I tested this on Total Recall (should be this one: https://www.amazon.com/Total-Recall-4K- ... 08YNQ5CVD/) and there are a few chapter points where the frame number calculation is off by one, so maybe it's due to rounding?

Code: Select all

CHAPTER01=00:00:00.000
CHAPTER01NAME=
CHAPTER02=00:05:50.583
CHAPTER02NAME=
CHAPTER03=00:11:27.375
CHAPTER03NAME=
CHAPTER04=00:18:14.000
CHAPTER04NAME=
CHAPTER05=00:24:50.917
CHAPTER05NAME=
CHAPTER06=00:29:37.625
CHAPTER06NAME=
CHAPTER07=00:37:05.875
CHAPTER07NAME=
CHAPTER08=00:43:11.833
CHAPTER08NAME=
CHAPTER09=00:48:35.667
CHAPTER09NAME=
CHAPTER10=00:55:02.042
CHAPTER10NAME=
CHAPTER11=01:00:03.917
CHAPTER11NAME=
CHAPTER12=01:15:21.167
CHAPTER12NAME=
CHAPTER13=01:20:16.167
CHAPTER13NAME=
CHAPTER14=01:27:07.083
CHAPTER14NAME=
CHAPTER15=01:32:18.458
CHAPTER15NAME=
CHAPTER16=01:39:53.042
CHAPTER16NAME=
CHAPTER17=01:53:15.958
CHAPTER17NAME=

Code: Select all

0 I -1
8415 I -1  --- correct one is 8414
16497 I -1
26256 I -1
35782 I -1
42663 I -1
53421 I -1
62205 I -1  --- correct one is 62204, which is a B-frame in the original video  :o 
69976 I -1
79249 I -1
86494 I -1
108508 I -1
115588 I -1  --- it's the correct one but once again a B-frame in the original
125451 I -1  --- correct one is 125450
132924 I -1  --- correct one is 132923
143833 I -1
163104 I -1

Posted: **Wed Aug 11, 2021 1:27 pm**

What happens with DJATOM's script?

EDIT: OK, I think I know what is going on. But please do report results with DJATOM's script if possible.

EDIT2: I used your posted chapters file with DJATOM's program and fps 24.00. It matches the correct output you posted. So I will fix that and sneak in an update. Standby...

The fact that some are B frames is inconsequential.

Posted: **Wed Aug 11, 2021 2:20 pm**

Please re-download DGDecNV 231 and/or DGDemux 52 and test again. Should be working as expected. If not, please report.

Posted: **Wed Aug 11, 2021 11:58 pm**

Yes, seems to be correct now. I tested a 25fps source as well and looks good. The wondering about B-frames was just general, I thought all chapters must start at an I-frame but I guess I was very much wrong there.

I also tested a Matroska file but the QP file is not produced, got the chapter file though.

Posted: **Thu Aug 12, 2021 5:43 am**

Boulder wrote: ↑
Wed Aug 11, 2021 11:58 pm
Yes, seems to be correct now. I tested a 25fps source as well and looks good. The wondering about B-frames was just general, I thought all chapters must start at an I-frame but I guess I was very much wrong there.

You can have an open GOP with B frames that do not point backwards.

I also tested a Matroska file but the QP file is not produced, got the chapter file though.

I forgot about MKV. I'm using mkvextract now to get chapters from MKV, so I'll make a standalone chapters2qp executable and invoke it automatically.

Thank you for your testing.

Posted: **Fri Aug 13, 2021 3:41 am**

I added a binary chapters2qp.exe to the 231 distribution. You can use it to generate the QP file manually from a chapters file. Today I will revise DGIndexNV to invoke it automatically for MKV (use normal chapters file format, not XML; advise me if XML is important to you).

D:\Don\Programming\C++\chapters2qp\x64\Release>chapters2qp
chapters2qp 1.0 Copyright (c) 2021 Donald A. Graft, All rights reserved
Usage: chapters2qp frame_rate chapters_file
frame_rate: video frame rate (float)
chapters_file: chapters file to be used (character string holding path)

Posted: **Fri Aug 13, 2021 8:29 am**

Thanks, just tested it on a set of existing chapter files for my next encoding batch, works perfectly

I can use the regular format, no need for XML support.

Posted: **Fri Aug 13, 2021 10:09 am**

Great. Thank you for your testing. I have the automatic invocation working and will sneak it in to 231 later today.

Support forum for DG Tools

Generate QP file to force frame types

Generate QP file to force frame types

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Feature Requests

Generate QP file to force frame types

Generate QP file to force frame types

Generate QP file to force frame types

Generate QP file to force frame types

Generate QP file to force frame types

Generate QP file to force frame types

Generate QP file to force frame types

Generate QP file to force frame types

Generate QP file to force frame types