Script for deflickering and ramping ACR (.xmp) settings in Bridge

Started by dmilligan, October 17, 2013, 12:32:09 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

itsskin


dmilligan

Quote from: itsskin on November 17, 2014, 01:44:50 AM
1. do 1 pass of deflicker
2. wait for bridge to finish rendering new previews
3. start 2nd pass
4. go to step 2
For the most part, that is already what it does. When you ask for a preview, Bridge is supposed to keep you waiting until it is ready.

Quote from: itsskin on November 17, 2014, 01:44:50 AM
Or I guess adding dialog with "delay between iterations" will also work awesome :)

I have a feeling its going to work faster and more stable
I see no reason why that would be the case. Adding delays do not make computer programs more reliable (in general, they actually tend to make programs less reliable, because they now depend on arbitrary delays)

itsskin

I totally agree with theory, but script never fails with one pass. Probably Bridge problem? And why there is no window with log output when debug version is used?

dmilligan


itsskin

Ok, got debug output. Extendscript Toolkit is required for this. Was not installed on my machine. So here is the screenshot - can't stop the process without killing it.

https://dl.dropboxusercontent.com/u/17452116/Screen%20Shot%202014-11-30%20at%202.19.30%20PM.jpg

After that I killed Extendscript Toolkit, process went on for several frames and halted again.

https://dl.dropboxusercontent.com/u/17452116/Screen%20Shot%202014-11-30%20at%202.20.42%20PM.jpg

NickZee

Hello @dmilligan,

Hope you are having a great new year.  Sorry to always be coming with problems.  I updated to Bridge CC, LR 5.7, and PS CC and now each time I run Ramp Multiple I get this error:


Adobe Bridge

Error: XMP Exception: Indexing applied to non-array.


I don't not get this error when running a single ramp or running a deflicker.

In Ramp Multiple, all options were selected except the Incrementals that you added for tifs.

In LR I edited 3 images and 1 stared them (beginning, middle and end) with gradients, circle gradients, the basic sliders (E. C. H. S. W. B. C. V. S.) temp and tint, HSL (only S & L).  No lens correction, Post Processing or Effect.

Here is a screen shot of the error.
https://www.dropbox.com/s/7s7aqt5kznybu38/xmp%20error.JPG?dl=0

I've searched this error and haven't found any reference to it.  The closest I could find was an issues with LR 5.7 xmp and it saying that every image needed to be Updated From File.  Although, this didn't happen to me.  https://forums.adobe.com/thread/1657300

I downloaded a fresh copy of the script from your website and github and tried both. 

Windows 8.1
CR2 from t3i

If you need me to upload the images and xmps just let me know.

Thank you David

EDIT/UPDATE:

I deleted all the XMP's created by LR and made the edits in ACR within Bridge and the Ramp Multiple worked. Something is wrong with LR 5.7's xmp writing.
Architectural Photography
NickZimmerman.com
5D MK3 & 600D | 24-70mm USMII L 2.8 | Nikon 14-24mm 2.8 | Nikon Nikkor 24mm 2.8 AIS & 50mm 1.4 AIS ** Windows 8.1 Pro | 32GB Ram | i7-4770 @ 3.4GHz

PtboPete

Please excuse my ignorance. I am new to ml and have mastered  ;) AETTR and Intervalometer and now, also turned on the Flicker module. From what I gather, the Flicker module makes a (ramped) flicker free setting in each xmp file. Although I don't see any changes when loaded in LR after copying all the files, CR2 and xmp then importing. Now I read this and it seems I need to run all the photos through this script to achieve the flicker free result. If the latter is true then what does the ml flicker module do? Do I need both?

I do know that if the script is required I have to do that in Bridge with ACR as LR does not handle scripts and that is sad but fine.

Secondly, it has been stated that no changes to exposure settings be done or any setting in the basic develop module. Only changes to levels are acceptable. Can I not make static changes to all photos in the area of clarity, vibrance, saturation, highlights, shadows, whites and blacks? I do already give each photo a static wb setting usually around the 6500 mark.

Any help is greatly appreciated, thanks. :)

dmilligan

You can use the in camera module to deflicker OR you can use this script to deflicker, you don't have to use both. This script is a little more flexible. Also you have to make sure to give the in camera deflicker module time to run in between each frame (~5 seconds). Using this script you don't have to worry about that and time between frames can be shorter.

You can also use this script to 'modify' an in camera deflicker, by applying some offset or ramp to existing "Exposure" values. In camera you can't do ramps and such like you can with this script.

Quote from: PtboPete on January 27, 2015, 08:24:37 PM
Secondly, it has been stated that no changes to exposure settings be done
No, I think you misunderstand. That's no changes to THE "Exposure" setting after running the script. That's the setting that the script sets to achieve deflickering. You can change any of the other settings you want (you can also change exposure *before* running the script if you like). Some of these other settings can cause additional flicker, so you need to be careful with them (they are 'image-adpative' meaning they will have different effects depending on the content of the frame, which causes the additional flicker). This script can help somewhat removing the additional flicker from those settings, but if you crank something like clarity to 100%, then you might find the flicker is too strong even for this script to remove.

You can also use this script to 'ramp' values of various settings. This is useful for WB because it changes drastically during a sunrise/sunset.

PtboPete

Thanks for the clarification. For now I'll go with in camera and progress from there. I'm not heavy handed with pp and for time lapse will be sure to make the changes identical for each. Now if the temperature outside would rise a little and the cloud cover dissipate then I could do some serious testing. Thanks again. 8)

PtboPete

Me again... last nights test I did with AETTR, DUAL ISO and Flicker. I ubnderstand that the dual ISO will only come into play when dr is quite large. The first few dual ISO shots by themselves I could plainly see the banding, however in last nights test I could see none of that so I figured dual ISO was not used. Once I completed my timelapse, the entire run shows blatant dual ISO banding.  >:(
1. How can I tell if dual ISO was used?
2. If dual ISO is only used on some shots do I need to process only those ones in cr2dng?
3. If I just grab the lot and run them all through cr2dng will that cause problems?

Thanks in advance. :)

dmilligan


caballe

Hello Dmilligan,

Sorry my ignorance, but I'm trying to install this script and when I click on the link you provided, it opens a browser window with a text file (I assume it is the script code). I tried to save it thru text editor with a jsx extension but it doesnt't work. I probably doing something wrong, but I understand this text file should be compiled in some way to be used. Obviously I'm doing something wrong, but I don't have a clue... can you help me?

Thank you,
Albert.

Walter Schulz

Open context menu on that link (Windows: Right mouse button, OS X: Control-Click) and use "save as .." option

classicbs

Hi There, loving the script.  I've just been doing loads of testing with ETTR and now I'm testing the processing.  I have a few questions that will help me understand this better and also help other people understand it.  I will go and test it myself but I think it wold be valuable if anyone was prepared to walk me through how they would approach the post processing of a time-lapse I shot.  It was shot overnight on the Great Wall of china in early Nov as I froze my ass of off camera.  It looks amazing but due to the cloud cover and the full moon it looks like day time.  Also due to my settings in ETTR (percent of highlights to ignore) once the moon comes into frame the exposure changes dramatically.  Have a look and if some one could give me a rough outline of how they would approach this using the script and roughly where they would place the keyframes and for what values (WB, Exposure etc) that would be amazing.  I'll do my own tests and post the results but it'd be great to get your opinions.   

A few specific questions.  If i want to reduce the exposure of everything can I do this at anytime or only at the beginning?
Why would I want to keyframe the de-flickering process I'm not clear on why I'd want to do this or in what situation?


dmilligan

When you make a keyframe its like telling the script "don't touch this frame", it becomes the target for what frames around it to look like. By default the first frame and last frame are keyframes.

So, pick a couple a frames that represent important sections of your time lapse. Then simply adjust them (along with first and last frames) to look however you like. If you modified settings besides exposure, then first do a "ramp multiple" this will make all settings smoothly ramp from one keyframe to the next (if you change WB, make sure all frames are set to Custom WB). Then run the deflicker. It will adjust exposure, making all the frames smoothly transition from one keyframe to the next.

If you want to adjust exposure after the deflicker, you can use the "ramp" in additive mode. It should be easy to avoid that though. Simply make sure your keyframes look like what you want for you final output before running the deflicker.

classicbs

Thanks David, that's great so I really get it now one can just edit a few photos and everything will RAMP.  My problem now is that the keyframes don't seem to work.  I'll try again but say I set a keyframe in the middle and one at each end it only rams to the middle keyframe and not all the way.  It's fine as I can do it in two batches but I wonder what I'm doing wrong to have it stop in the middle at the one I gave one start rating to, this has happened twice now.  Also separate issue I can't seem to flip between bridge (and camera raw) and light room.  They photos seem to have there own settings and the edits don't seem to be carried over even though (as far as I can tell) they are referencing the same cr.'s and XMP's?  Any Ideas?

surami

@classicbs: Maybe this workflow will help for you. I could get the best results till now:
1. Delete your old XMP-s (or save them somewhere as backup).
2a. In LR read in the metadatas for your sequence. Select all files on the library tab (ctrl + a) -> right click metadata -> read metadata.
2b. In LR set custom whitebalance for your files + crop for 16:9. Select all files on develope tab -> switch on autosync -> basic settings: wb custom + crop overlay: aspect 16:9. -> save metadatas (ctrl + s)
2c. In LR set 1 star for your keyframes + adjust the settings for them. Rightclick on the selected images -> set rating: 1 star (switch off autosync!) + filter for 1 star -> adjust the settings for the keyframe as you wish (don't touch the shadows, highlights, blacks, whites) -> save metadatas again.
3a. In BR select all files and right click -> purge cache for selections. Now you will see the settings, what you did in LR.
3b. In BR select all files and run David's awesome ramp multiple script (check all), wait till the process ends.
4a. Before you deflicker the sequence I suggest you to download the visionlog raw camera profile (read the installation guide) and apply it. This will help the deflicker script to work more properly.
5. In LR setup the visionlog raw profile for you sequence. Read in the metadatas for your full sequence (filters off) on the develope tab -> switch on autosync -> camera calibration, profile: visionlog -> save metadatas.
6a. In BR select all files and right click -> purge cache for selections. Now you will see very flat images.
6b. Run deflicker.
7. In LR read metadatas for your sequence and swicth back your camera profile to adobe standard -> save metadatas.
8. In BR select all files and right click -> purge cache for selections. Now you will see everywhere the final multi ramped deflickered sequence.

Maybe it's too long to read and every settings could be done in BR + ACR, but I like the LR's layout.
550D + nightly ML

classicbs

Hi Surami,

That's great I think I've nailed it now everything seems to work as expected.  I have two questions.

2c, why do you say (don't touch the shadows, highlights, blacks, whites)?  At what point can I adjust these in your workflow and why.

Do you know about changing Lightroom's catalog settings in the preferences: Lightroom>Prefrences>Go To Catalog Settings>Metadata>Automatically write changes to XMP

Should I use this or not and if not why not.

Also more generally how do I get notified in this forum if someone replies to my post?

Mant thanks and I'll post the time-lapse and some others once they are done,

James


classicbs

Also just my tuppence worth, I do really appreciate the effort that has gone into this but a couple of comments.  I keep selecting the undo log thing when I deflicker and it hangs my computer and I have to force quit bridge.  It'd be better without that being there somehow (although it is useful).  Also is it possible to make it run until it doesn't need to anymore.  At the moment I'm just selecting it to run 10 times so it stops itself eventually .  Is there a reason why one might want to deflicker less than what the script thinks?

Clip imminent of several day to night lapses from China shot for National Geographic.  Once I've nailed this I'll donate something on the PayPal,  much appreciated!

DeafEyeJedi

Thanks for sharing your workflow @surami -- will definitely test it out!

Solid questions @classicbs -- I wouldn't touch the S,H,B,W because it would then cause more flicker, I suppose?

Also you can get notifications through Tapatalk via app on your phones, or tablet. (I use them on iPhone, iPad)

works pretty well for me.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

classicbs

Okay great many thanks @deafeyejedi, @Surami, @dmilligan.  I'm really getting into this now.  I'm still a bit unsure as to when the best time to do batch post processing is?  For the nighttime star time lapses it really helps to adjust some settings on all the photos to make the stars really pop in the sky.  If one knows they will change all the photos  (i.e. no need to ramp) then are these edits best left to after the de-flicklering?  Can I ramp after de-flickering?  Do I need to run de-flickering again if I keep ramping.

Here's my first three efforts using this amazing script in 4k should you have a great monitor.  You can see the great wall of china time-lapse has progresses.  Would anyone care to criticise my exposure ramping and WB tramping, how does it look I'd be interested to know, the original un-ramped version is above.

Cheers

p.s. please don't share this YouTube link anywhere else as I'm not supposed to be making these images public yet


dmilligan

Quote from: classicbs on February 10, 2015, 10:07:46 AM
If one knows they will change all the photos  (i.e. no need to ramp) then are these edits best left to after the de-flicklering?  Can I ramp after de-flickering?
It's usually best to do these edits before deflickering in case they introduce any extra flicker themselves, the deflicker will help take care of it somewhat. (ACR has a lot of settings that are 'image adaptive' and they can have a different effect on different source material, causing flicker; the worst offenders are: clarity, highlights, shadows). Of course there are things that are not going to cause extra flicker (like WB, noise reduction, lens corrections), so it doesn't really matter when you do them.

Because of the image-adaptive nature of some of these settings, even the deflicker script won't be able to fully fix the flicker introduced because for example highlights, midtones, and shadows may be affected differently, and the deflicker can only match overall exposure. So you have to be careful when using these settings, there will be limits to the values you can "safely" use (without introducing too much additional flicker).

classicbs

Okay great many thanks @dmilligan.

Do you have an opinion on the following statement?

Quote4a. Before you deflicker the sequence I suggest you to download the visionlog raw camera profile (read the installation guide) and apply it. This will help the deflicker script to work more properly.

surami

@classicbs: The idea of using the visionlog raw camera profile came from the post #260, they spoke about flat images... ;)
550D + nightly ML

classicbs

Okay so thanks all, 3 days getting my head around this bit the results are great here the full run down of my de-flickered time-lapses from my last shoot.


I thought I'd compile some of the advice that helped me get my head around this:

@classicbs: Maybe this workflow will help for you. I could get the best results till now:
1. Delete your old XMP-s (or save them somewhere as backup).
2a. In LR read in the metadatas for your sequence. Select all files on the library tab (ctrl + a) -> right click metadata -> read metadata.
2b. In LR set custom whitebalance for your files + crop for 16:9. Select all files on develope tab -> switch on autosync -> basic settings: wb custom + crop overlay: aspect 16:9. -> save metadatas (ctrl + s)
2c. In LR set 1 star for your keyframes + adjust the settings for them. Rightclick on the selected images -> set rating: 1 star (switch off autosync!) + filter for 1 star -> adjust the settings for the keyframe as you wish (don't touch the shadows, highlights, blacks, whites) -> save metadatas again.
3a. In BR select all files and right click -> purge cache for selections. Now you will see the settings, what you did in LR.
3b. In BR select all files and run David's awesome ramp multiple script (check all), wait till the process ends.
4a. Before you deflicker the sequence I suggest you to download the visionlog raw camera profile (read the installation guide) and apply it. This will help the deflicker script to work more properly.
5. In LR setup the visionlog raw profile for you sequence. Read in the metadatas for your full sequence (filters off) on the develope tab -> switch on autosync -> camera calibration, profile: visionlog -> save metadatas.
6a. In BR select all files and right click -> purge cache for selections. Now you will see very flat images.
6b. Run deflicker.
7. In LR read metadatas for your sequence and swicth back your camera profile to adobe standard -> save metadatas.
8. In BR select all files and right click -> purge cache for selections. Now you will see everywhere the final multi ramped deflickered sequence.

Maybe it's too long to read and every settings could be done in BR + ACR, but I like the LR's layout.

D-milligan's words of wisdom

------------------------------------

When you make a keyframe its like telling the script "don't touch this frame", it becomes the target for what frames around it to look like. By default the first frame and last frame are keyframes.

So, pick a couple a frames that represent important sections of your time lapse. Then simply adjust them (along with first and last frames) to look however you like. If you modified settings besides exposure, then first do a "ramp multiple" this will make all settings smoothly ramp from one keyframe to the next (if you change WB, make sure all frames are set to Custom WB). Then run the deflicker. It will adjust exposure, making all the frames smoothly transition from one keyframe to the next.

If you want to adjust exposure after the deflicker, you can use the "ramp" in additive mode. It should be easy to avoid that though. Simply make sure your keyframes look like what you want for you final output before running the deflicker.

---------------------------------

It's usually best to do these edits before deflickering in case they introduce any extra flicker themselves, the deflicker will help take care of it somewhat. (ACR has a lot of settings that are 'image adaptive' and they can have a different effect on different source material, causing flicker; the worst offenders are: clarity, highlights, shadows). Of course there are things that are not going to cause extra flicker (like WB, noise reduction, lens corrections), so it doesn't really matter when you do them.

Because of the image-adaptive nature of some of these settings, even the deflicker script won't be able to fully fix the flicker introduced because for example highlights, midtones, and shadows may be affected differently, and the deflicker can only match overall exposure. So you have to be careful when using these settings, there will be limits to the values you can "safely" use (without introducing too much additional flicker).

-----------------------------------

The preview should show you the analysis area and it will highlight the percentile you selected.

Analysis size refers to how big the image is scaled down to before computing the histogram. The smaller you make it the faster the script but the histogram will be less accurate. (the reason for this setting is that the rescale operation is native and very fast, however computing the histogram is done in JavaScript which is very very slow, so rescaling the image down makes the histogram calculation much faster)



My Hint's and tips.


If you're not happy with the de-flicker of a particular section then an easy workflow is to just re-keyframe that section and only select the photos in the section you want to re-de-flicker so you don't have to wait for the whole time-lapse to run again.  Just remember to uncheck any previous keyframes you may have set so as to not affect the behaviour of the new keyframes in the section you are modifying.  Otherwise you might experience some unwanted behaviour

If you want to ramp some other settings after the de-flicker then be sure to uncheck exposure in the Ramp Multiple check box otherwise it will mess up your de-flickering

When de-flickering, if you "select all" in Bridge be sure to uncheck the undoDatoa.jdon file otherwise it will hang the de-flickering and you'll have to force quit bridge (at least I had to) and run it again

I found that often more than 3 passes were necessary before the script was happy so I just set it to 10 passes every time then let it run so I wouldn't have to do it twice every time

Use the percentile measure to fine tune areas that aren't de-flickering well by changing where the de-flickering occurs (but be careful as of you change one area another may get worse best to stick with the sky