Page 6 of 12
Port Cube
Posted: Mon Aug 22, 2022 10:24 am
by Rocky
Good question.
Port Cube
Posted: Mon Aug 22, 2022 12:07 pm
by Guest 2
There is difference between Cube and DGCube when internal expansion / compression is used.
Code: Select all
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
LoadPlugin("D:\Eseguibili\Media\AVSCube\AVSCube.dll")
DGSource("F:\In\2_0446 Akira - TAiCHi\taichi.dgi",ct=48,cb=48,cl=0,cr=0)
propClearAll()
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:limited=>rgb:709:709:full", resample_filter_uv="spline64", dither_type="error_diffusion")
Cube("D:\Programmi\Media\AviSynth+\cube\1a_PQ1000_HLG_mode-nar_in-nar_out-nar_nocomp.cube", fullrange=true)
z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")
Code: Select all
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
LoadPlugin("D:\Eseguibili\Media\DGCube\DGCube.dll")
DGSource("F:\In\2_0446 Akira - TAiCHi\taichi.dgi",ct=48,cb=48,cl=0,cr=0)
propClearAll()
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:limited=>rgb:709:709:full", resample_filter_uv="spline64", dither_type="error_diffusion")
DGCube("D:\Programmi\Media\AviSynth+\cube\1a_PQ1000_HLG_mode-nar_in-nar_out-nar_nocomp.cube", expand_in=false, compress_out=false)
z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")
Code: Select all
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
LoadPlugin("D:\Eseguibili\Media\DGCube\DGCube.dll")
DGSource("F:\In\2_0446 Akira - TAiCHi\taichi.dgi",ct=48,cb=48,cl=0,cr=0)
propClearAll()
DGCube("D:\Programmi\Media\AviSynth+\cube\1a_PQ1000_HLG_mode-nar_in-nar_out-nar_nocomp.cube", expand_in=true, compress_out=true)
ConvertBits(10,dither=1)
You can see similar behaviour with colorbars:
Code: Select all
ColorBars(848, 480, pixel_type="YV24")
LoadPlugin("D:\Eseguibili\Media\AVSCube\AVSCube.dll")
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:limited=>rgb:709:709:full", resample_filter_uv="spline64", dither_type="error_diffusion")
Cube("D:\Programmi\Media\AviSynth+\cube\5a_BT709_HLG_UPCONVERT_DISPLAY_mode_nar_in_nar_out_nar_nocomp.cube", fullrange=true)
z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")
TurnRight.Histogram.TurnLeft
Code: Select all
ColorBars(848, 480, pixel_type="YV24")
LoadPlugin("D:\Eseguibili\Media\DGCube\DGCube.dll")
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:limited=>rgb:709:709:full", resample_filter_uv="spline64", dither_type="error_diffusion")
DGCube("D:\Programmi\Media\AviSynth+\cube\5a_BT709_HLG_UPCONVERT_DISPLAY_mode_nar_in_nar_out_nar_nocomp.cube", expand_in=false, compress_out=false)
z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")
TurnRight.Histogram.TurnLeft
Code: Select all
ColorBars(848, 480, pixel_type="YUV420P16")
LoadPlugin("D:\Eseguibili\Media\DGCube\DGCube.dll")
DGCube("D:\Programmi\Media\AviSynth+\cube\5a_BT709_HLG_UPCONVERT_DISPLAY_mode_nar_in_nar_out_nar_nocomp.cube", expand_in=true, compress_out=true)
TurnRight.Histogram.TurnLeft
Why oh why? Am I doing something wrong?
Port Cube
Posted: Mon Aug 22, 2022 12:54 pm
by Curly
Guest 2 made me upgrade my stable Windows 10 Enterprise workstation to the unstable crap called Windows 11 Enterprise which one day upgraded itself only to end up blue screening in loop, thus making me lose every program installed.
Great work
t!
Every program, Love it.
Could say this or could say that. Howz about brain-dead loser. Knurk nurk. Elbow in the side. Bop. Oww!
He's mad because
Boris unfriended him.
Port Cube
Posted: Mon Aug 22, 2022 12:59 pm
by Rocky
Thank you for the testing,
Guest 2. That's what we need to achieve world supremacy. Investigating...
Probably going to be down to different YUV<-->RGB matrices. Don't forget code comes from HDR world. Need to support different matrices, maybe with user-definable option. Let's see.
601 <--> RGB
709 <--> RGB
2020 <--> RGB
...
Separately configurable for input and output. Sure, ask about primaries and transfer. Maybe just port z to CUDA and run it all in CUDASynth mode?!
May I please have your source stream? Love those blues! Meanwhile, I can get started with ColorBars.
Port Cube
Posted: Mon Aug 22, 2022 1:05 pm
by Sherman
Guys, I'm re-ripping R&B to a dedicated SSD. Gonna get the restoration project going again. After chaoscoder showed what is possible let's get this job done! Definitely Rocky's solution requires dot crawl/rainbow cleanup, otherwise it's pretty good and fast. Gonna use Tcomb().
I think the denoising has been heavy handed. You wouldn't get anything all super clean like that back in the era. We need to back off on denoising strength. C'mon, we're watching 50s/60s over-the-air analog TV. Even Natasha agreed with me.
Port Cube
Posted: Mon Aug 22, 2022 1:16 pm
by DG
Hear, hear.
Port Cube
Posted: Mon Aug 22, 2022 1:35 pm
by Guest 2
Rocky wrote: ↑Mon Aug 22, 2022 12:59 pm
May I please have your source stream?
Sure:
https://krakenfiles.com/view/0VcYGMVjg5/file.html
Port Cube
Posted: Mon Aug 22, 2022 1:42 pm
by Guest 2
Sherman wrote: ↑Mon Aug 22, 2022 1:05 pm
Gonna get the restoration project going again.
Have a look at the brand new (porting) of
https://github.com/Asd-g/AviSynthPlus-RIFE.
Perhaps it's of some use to you.
Port Cube
Posted: Mon Aug 22, 2022 1:47 pm
by Sherman
Gee, that looks familiar.
On the restoration, I'm gonna stay close to the source so frame interpolation will be overkill. But thank you for the suggestion.
Port Cube
Posted: Mon Aug 22, 2022 1:56 pm
by Sherman
Have to wait until October for the 4090. I'm already buttering-up Albert for some dinero.
Port Cube
Posted: Tue Aug 23, 2022 4:23 am
by Guest 2
Rocky wrote: ↑Mon Aug 22, 2022 12:59 pm
Maybe just port z to CUDA and run it all in CUDASynth mode?
That would be nice but how much effort and how big the overhead? Perhaps it's time to use the CUDA extension of AVS+ (if working) to have a unified buffer to work with?
Port Cube
Posted: Tue Aug 23, 2022 7:29 am
by Rocky
It was tongue-in-cheek. The point was I didn't realize I was signing up to implement multiple color conversions. It's not exciting to me but I suppose now it has to be done.
Port Cube
Posted: Tue Aug 23, 2022 9:00 am
by Rocky
Your settings look kinda random. Let's get serious. Here is my test script for external conversion:
ColorBarsHD(848, 480, pixel_type="YV24")
ConvertToYUV420().ConvertBits(16)
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:limited=>rgb:709:709:full")
DGCube("IDENTITY.cube", expand_in=false, compress_out=false)
z_ConvertFormat(pixel_type="YUV420P10", colorspace_op="rgb:709:709:full=>709:709:709:limited")
Histogram("levels")
Here is the script for internal conversion:
ColorBarsHD(848, 480, pixel_type="YV24")
ConvertToYUV420().ConvertBits(16)
DGCube("IDENTITY.cube", expand_in=false, compress_out=false)
Histogram("levels")
These produce acceptably close results. It could be tweaked to be closer by adding 709->RGB matrix. I'll probably do that.
But wait! I have compress_in and compress_out set to false for the internal script. So here's where I went wrong. YUV is intrinsically limited. My conversions already convert limited to full on input, and full to limited on output. If you set compress_out=true you see from the histogram that it is wrong because it compresses twice.
So here is the plan.
1. Until I release a new version, realize that expand_in and compress_out apply only for RGBP16 input/output. For YUV420P16 they are inapplicable and should be left at false. Next version will enforce that.
2. I will add option for 709 <-> RGB versus 2020 <-> RGB matrixes in my conversions.
What do you think? Making any sense?
Port Cube
Posted: Tue Aug 23, 2022 10:53 am
by Guest 2
Rocky wrote: ↑Tue Aug 23, 2022 9:00 am
My conversions
already convert limited to full on input, and full to limited on output. If you set compress_out=true you see from the histogram that it is wrong because it compresses twice.
At this point, I've lost myself.
Your internal conversion should make the external source to comply to the cube needs.
If you generate a limited histogram but the cube needs full range, you need to set true to both operators.
If you generate a limited histogram and the cube needs limited range, you need to set false to both operators.
Am I wrong?
Rocky wrote: ↑Tue Aug 23, 2022 9:00 am
What do you think? Making any sense?
Sure it does, for the moment.
I mean: AVSCube was universal, in the meaning that it did just one thing and worked well but (very) slow.
If you start creating different conversions, you will get requests for any space out there. Much better, at this point, to be as generic as possible and port Z to CUDA...
Or to start thinking about a wrapper for OpenCV and its primitives.
Or (I don't know if working) to find (or create) LUTs to move from one space to RGB and back and apply DGCube 3 times in a row maximum.
P.S: Look at this
https://evs.com/documentation/xsquare/u ... %20LUT.htm
Port Cube
Posted: Tue Aug 23, 2022 4:10 pm
by Rocky
Guest 2 wrote: ↑Tue Aug 23, 2022 10:53 am
Your internal conversion should make the external source to comply to the cube needs.
Well, you're worried about cubes made in limited range. I have another idea for that. Let me see if it works.
Your link is helpful. Thank you for that.
Bear with me, I'm learning all this stuff
de novo.
Port Cube
Posted: Tue Aug 23, 2022 4:24 pm
by Guest 2
Rocky wrote: ↑Tue Aug 23, 2022 4:10 pm
Your link is helpful. Thank you for that.
You are always the most welcome.
Port Cube
Posted: Wed Aug 24, 2022 9:09 am
by mwittmann
I have a cube file in limited range to convert HDR PQ to SDR. If I convert the video to RGBP16 everything is fine.
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:limited=>rgb:709:709:limited")
DGCube("ACES_Gamma2.4_Legal.cube", expand_in=false, compress_out=false)
z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:709:709:limited=>709:709:709:limited", dither_type="ordered")
But using the cube file in YUV420P16 the picture is overbrighted, black is going to gray.
ConvertBits(16)
DGCube("ACES_Gamma2.4_Legal.cube", expand_in=false, compress_out=false)
ConvertBits (8, dither=0)
Am I doing something wrong or is the cube file intended for RGB only?
Port Cube
Posted: Wed Aug 24, 2022 9:13 am
by Rocky
The YUV support is still in development. This was never released but someone took it upon themselves to release it without consulting me. So please just follow this thread and when the dish is complete it will be released formally. If something is only linked from my website misc directory and does not have an announcement in the Binaries Update Notification thread, then it is not released.
Guest 2 has already pointed out the brightness issue for YUV and we are working on a solution. Be patient!
Finally, if you expect me to work on an issue, you need to provide the FULL script, a source stream fragment, and the cube file. I need those things to duplicate your issue and to test fixes. Please, don't make me beg for these things. Even squirrels working for free expect users to put in some effort too.
Port Cube
Posted: Wed Aug 24, 2022 9:38 am
by Guest 2
mwittmann wrote: ↑Wed Aug 24, 2022 9:09 am
I have a cube file in limited range to convert HDR PQ to SDR.
Can you share it?
Port Cube
Posted: Wed Aug 24, 2022 10:03 am
by mwittmann
I sent you a message, Guest 2. You can test the file now if you want.
Sorry, Rocky - I should have read the entire thread first. Then I would have seen the YUV-hint from Guest 2.
Thank you for your great work on DGCube.
Port Cube
Posted: Wed Aug 24, 2022 12:07 pm
by mwittmann
Message sent, I hope you have now everything you need. Please tell me if something is missing.
Port Cube
Posted: Wed Aug 24, 2022 12:09 pm
by Rocky
That's great, thank you. Can't tell if it's everything as it's predicting an hour to download your 112MB. No problem, I'm on my lunch hour.
Next time, I'll be happy to give you my FTP.
Your report and files are greatly appreciated.
EDIT: Hmm, might've got stuck.
EDIT2: Nope, just moving at a snail's pace.
Port Cube
Posted: Wed Aug 24, 2022 12:31 pm
by Rocky
Are you giving me encoded files? No! I need original source files. Please?
Port Cube
Posted: Wed Aug 24, 2022 12:34 pm
by Guest 2
mwittmann wrote: ↑Wed Aug 24, 2022 9:09 am
Am I doing something wrong or is the cube file intended for RGB only?
Cubes always work in RGB space (at least the ones I saw).
The only difference can be what kind of video they expect, i.e. full range or limited range.
Some software houses (such as xtremestuff) call them data (full range) and legal (limited range).
Port Cube
Posted: Wed Aug 24, 2022 12:37 pm
by mwittmann
File size of original hevc file is 57GB. Is there a way to split the hevc file to my chosen frames without reencoding?