Rocky and Bullwinkle restoration challenge

Anything related to video and my tools that is not a support request.
User avatar
Rocky
Moose Approved
Posts: 2207
Joined: Fri Sep 06, 2019 12:57 pm

Rocky and Bullwinkle restoration challenge

Post by Rocky »

The available streams for the culturally and historically important Rocky and Bullwinkle Show are of poor quality and they stand in need of major restoration. We announce therefore a restoration/encoding challenge for our forum members. Here is a link to a video stream of the very first R&B episode. I suppress the audio to try to keep this in the fair use domain.

http://rationalqm.us/misc/rb.m2v
(about 900MB)

There are significant problems such as noise, hard telecine, dot crawl, cross luma, cell defects, etc. So the challenge is to restore this video with the following constraints for the result:

* Encode in AVC video.
* Retain original resolution.
* Progressive video at 23.976 fps.
* Size of the encoded stream to be <= 300Mbytes
* Your script/process must be disclosed.
* The script must not require manual per-episode tweaking (e.g., no ApplyRange(), etc.)
as there are about 200 episodes to process.

The winning result will be determined by the votes on a poll to be created. The winner will receive $100 and the Curly-approved forum rank.

To submit an entry, post here expressing your interest (or send me a PM) and you will be sent a link to upload your submission. When all submissions are available, they will be posted blindly as part of the poll. If you have any questions, please post them here. Just so you know, I have an attempt of my own that I will also submit. My result is far from perfect and I am hoping y'all can do way better than me, a lowly squirrel.

Thank you and let's knock this out of the park!
User avatar
Sherman
Moose Approved
Posts: 333
Joined: Mon Jan 06, 2020 10:19 pm

Rocky and Bullwinkle restoration challenge

Post by Sherman »

Send me the link. You just wait, I'm going to crush this! :twisted:
User avatar
Curly
Moose Approved
Posts: 166
Joined: Sun Mar 15, 2020 11:05 am

Rocky and Bullwinkle restoration challenge

Post by Curly »

Make my day.
User avatar
Rocky
Moose Approved
Posts: 2207
Joined: Fri Sep 06, 2019 12:57 pm

Rocky and Bullwinkle restoration challenge

Post by Rocky »

Here is what I am using so far:

dgsource("D:\tmp\Rocky\rb.dgi")
dgtelecide(mode=0)
dgdecimate()
dgbob(mode=0) # residual interlacing cleanup
dgdenoise(blend=0.5,strength=0.2,cblend=0.5,cstrength=0.2)
warpsharp(depth=100)
tweak(sat=1.25)

It's a fairly light-handed approach. I don't want it looking plastic. The opening scene has awful chroma and noise compared to the rest of the episode, so I did not try to push things too hard just to fix that scene. I want a script that does not need to be manually edited per episode. All the DG filters make it super fast.

The dgbob() call acts on all frames but has surprisingly little effect on progressive frames, so it is effective for removing the residual interlacing of small mouth movements, etc. I was unsuccessful trying to apply it to only combed frames using IsCombed(). There is no good threshold to catch the mouth movements without it affecting most of the frames, so I just left it unconditional. I did not try IVTC using a fixed cycle, or using pattern guidance, although that might be worth trying. I found that the dot crawl remover do more harm than good so just let dgdenoise() mitigate it to some extent. I didn't do anything at all about rainbowing.

When you do your own script pay special attention to the interlacing of small mouth movements. Also, when you denoise don't kill detail on Bullwinkle's curtains at various scenes.
User avatar
Rocky
Moose Approved
Posts: 2207
Joined: Fri Sep 06, 2019 12:57 pm

Rocky and Bullwinkle restoration challenge

Post by Rocky »

Good old Decomb to the rescue! Got rid of the DGBob() call using Telecide() pattern guidance. Note that this is a version of Decomb compiled for 64-bit and for Avisynth+. If anybody wants it, just ask.

dgsource("D:\tmp\Rocky\VTS_01_1.dgi")
telecide(post=0,guide=1,show=false)
dgdecimate()
dgdenoise(blend=0.0,strength=0.15,cblend=0.0,cstrength=0.15)
warpsharp(depth=128)
tweak(sat=1.25)

Still don't know what to do about dot crawl and rainbowing, if anything.

Clarification on the challenge: DG tools staff (other than Rocky) are ineligible to enter or vote. The $100 will go to the best result that is voted better than my attempt linked above.
User avatar
thechaoscoder
Moose Approved
Posts: 33
Joined: Tue Jul 14, 2020 8:34 am

Rocky and Bullwinkle restoration challenge

Post by thechaoscoder »

I get an 404 Error for http://rationalqm.us/misc/rb.m2v
User avatar
Rocky
Moose Approved
Posts: 2207
Joined: Fri Sep 06, 2019 12:57 pm

Rocky and Bullwinkle restoration challenge

Post by Rocky »

Ha ha. The Archive subforum was not supposed to be accessible except to moderators. I had moved this thread over there and deleted the stream. The reason was that after 7 full days there was not even a nibble, nor even a comment. So I just assumed there was no interest (which was a little surprising and embarrassing to tell the truth), and I wanted to get the episode processing under way. After completing a quarter of the episodes I saw your post above. That's great of course, and thank you, so I have moved the thread back to the General subforum, put the source stream back online, and put my processing on hold.
DAE avatar
Guest

Rocky and Bullwinkle restoration challenge

Post by Guest »

Tried to post a sample but keep getting HTTP error
File is about 320MB
Also, it appears that avc and 264 are invalid extensions
DAE avatar
Guest

Rocky and Bullwinkle restoration challenge

Post by Guest »

script

Code: Select all

import vapoursynth as vs
from vapoursynth import core

#####FRAME SERVER#####
core.std.LoadPlugin("C:/Program Files (Portable)/dgdecodenv/DGDecodeNV.dll")
clip = core.dgdecodenv.DGSource(r'F:\DVD\rb.dgi',  fieldop=0)

#####AVSCOMPAT#####
core.avs.LoadPlugin("C:/Program Files (Portable)/dgdecodenv/DGDecodeNV.dll")

#####IVTC#####
clip = core.avs.DGTelecide(clip, mode=1, dthresh=3.5)
clip = core.avs.DGDecimate(clip, cycle=5, keep=4)

#####DENOISE#####
clip = core.avs.DGDenoise(clip, strength=0.075)

#####RESIZE#####
clip = core.std.SetFieldBased(clip, 0)
clip = core.resize.Spline36(clip, width=1440, height=960)

#####SHARPEN#####
clip = core.avs.DGSharpen(clip, strength=0.200)

#####RESIZE#####
clip = core.std.SetFieldBased(clip, 0)
clip = core.resize.Spline36(clip, width=720, height=480)

clip.set_output()

apparently I cannot upload any files, mkv and vpy are also not allowed
Maybe something with pulling it out of the archive?
User avatar
thechaoscoder
Moose Approved
Posts: 33
Joined: Tue Jul 14, 2020 8:34 am

Rocky and Bullwinkle restoration challenge

Post by thechaoscoder »

Size of the encoded stream to be <= 300Kbytes
You mean Megabytes? So under 300MB? 300KB would be very challenging :scratch:
User avatar
Rocky
Moose Approved
Posts: 2207
Joined: Fri Sep 06, 2019 12:57 pm

Rocky and Bullwinkle restoration challenge

Post by Rocky »

thechaoscoder wrote:
Wed Jan 12, 2022 6:12 am
You mean Megabytes? So under 300MB? 300KB would be very challenging :scratch:
Yes, 300MB. I have corrected the post. Thank you for pointing it out.
User avatar
Rocky
Moose Approved
Posts: 2207
Joined: Fri Sep 06, 2019 12:57 pm

Rocky and Bullwinkle restoration challenge

Post by Rocky »

apparently I cannot upload any files, mkv and vpy are also not allowed
Maybe something with pulling it out of the archive?
Regarding avs and vpy, I have added them as allowed extensions. Regarding streams like mkv and avc/264, I want you to upload them to our ftp. If you need the access details please send me a PM.

You are free to post your scripts here as we go but I require only that you disclose them after the result is determined. So you can keep things secret if you want to stop people from using your ideas. Your body, your choice. ;)
User avatar
thechaoscoder
Moose Approved
Posts: 33
Joined: Tue Jul 14, 2020 8:34 am

Rocky and Bullwinkle restoration challenge

Post by thechaoscoder »

Not really tweaked or encoded yet. But preview & "playback" are looking good so far. I'm currently at work :belly-laugh:
https://i.imgur.com/2YyMpL1.png
Image
User avatar
Rocky
Moose Approved
Posts: 2207
Joined: Fri Sep 06, 2019 12:57 pm

Rocky and Bullwinkle restoration challenge

Post by Rocky »

Thank you, looking forward to your completed encode.

Work hard, like a squirrel. :lol:
User avatar
thechaoscoder
Moose Approved
Posts: 33
Joined: Tue Jul 14, 2020 8:34 am

Rocky and Bullwinkle restoration challenge

Post by thechaoscoder »

My first attempt
See attachment link.txt for test encode.

EDIT
Oh, I just noticed there will be a blind test. So I removed my script + file for now. :belly-laugh:

So far I can say that dot crawl and rainbows are eliminated. Only some background instability is left. Not sure if it can be removed completly. :scratch:
DAE avatar
Guest

Rocky and Bullwinkle restoration challenge

Post by Guest »

rb.264 and rb.vpy uploaded.
Simple little script. Not cleaning too much so Detail isn't lost
DAE avatar
Guest

Rocky and Bullwinkle restoration challenge

Post by Guest »

sample frame
rb.bmp
DAE avatar
Guest

Rocky and Bullwinkle restoration challenge

Post by Guest »

I am not in the contest, just trying options, so you can make mine public if you wish.
Any opinions, criticism, or advice might improve my end result.
User avatar
Rocky
Moose Approved
Posts: 2207
Joined: Fri Sep 06, 2019 12:57 pm

Rocky and Bullwinkle restoration challenge

Post by Rocky »

thechaoscoder wrote:
Wed Jan 12, 2022 1:08 pm
So far I can say that dot crawl and rainbows are eliminated. Only some background instability is left. Not sure if it can be removed completley. :scratch:
Ooh, that sounds good. I have the IVTC under control but the dot crawl and rainbows were flummoxing me.

FTP details sent by PM.
User avatar
Rocky
Moose Approved
Posts: 2207
Joined: Fri Sep 06, 2019 12:57 pm

Rocky and Bullwinkle restoration challenge

Post by Rocky »

Guest wrote:
Wed Jan 12, 2022 4:39 pm
I am not in the contest, just trying options, so you can make mine public if you wish.
Any opinions, criticism, or advice might improve my end result.
Thank you, Guest. If I put you in the contest you can always refuse the $100 if you win.
DAE avatar
Guest

Rocky and Bullwinkle restoration challenge

Post by Guest »

New and improved ( I hope) rb_1 on the way
User avatar
Sherman
Moose Approved
Posts: 333
Joined: Mon Jan 06, 2020 10:19 pm

Rocky and Bullwinkle restoration challenge

Post by Sherman »

Don't give up, Guest, you'll get it one day. I did my submission, so don't get your hopes up. Rocky, all you guys, it's over.

Rocky, when do submissions end?
DAE avatar
Guest

Rocky and Bullwinkle restoration challenge

Post by Guest »

Sherman wrote:
Wed Jan 12, 2022 8:00 pm
Don't give up, Guest, you'll get it one day. I did my submission, so don't get your hopes up. Rocky, all you guys, it's over.

Rocky, when do submissions end?
Clarification on the challenge: DG tools staff (other than Rocky) are ineligible to enter or vote.
User avatar
Sherman
Moose Approved
Posts: 333
Joined: Mon Jan 06, 2020 10:19 pm

Rocky and Bullwinkle restoration challenge

Post by Sherman »

Whaa? Looks like I'm going to have to throw a tantrum.
DAE avatar
Guest

Rocky and Bullwinkle restoration challenge

Post by Guest »

Sherman wrote:
Thu Jan 13, 2022 11:54 am
Whaa? Looks like I'm going to have to throw a tantrum.
You are getting quite good at that, Shermy
Post Reply