Dynamic Range, Mid Tones, Metering and ETTR

Posted on September 28, 2018 by Andrew Astbury

Dynamic Range, Mid Tones, Metering and ETTR

I recently uploaded a video to my YouTube channel showing you an easy way to find the ‘usable dynamic range’ of you dSLR:

The other day I was out with Paul Atkins for a landscape session in the awesome Dinorwic Quarry in Llanberis, Snowdonia. Highly dynamic clouds and moody light made the place look more like Mordor!

Looking towards the top of the Llanberis Pass from the middle level of Dinorwic Quarry and Electric Mountain.

Here are the 6 unedited shots that make this finished panoramic view:

As you can see, the images are are shot in a vertical aspect ratio. Shooting at 200mm on the D800E this yields an assembled pano that is 16,000 x 7000 pixels; the advantages for both digital sales and print should be obvious to you!

As you can see, the bright parts of the sky are a lot brighter in the captures than they are in the finished image, but they are not ‘blown’. Also the shadows in the foreground are not choked or blocked.

In other words the captures are shot ETTR.

Meter – in camera or external.

Any light meter basically looks at a scene (or part thereof) and AVERAGES the tones that it sees. This average value value is then classed by the meter is MID GREY and the exposure is calculated in terms of the 3 variables you set – Time, Intensity and Applied Gain, or shutter, aperture and ISO.

But this leads to all sorts of problems.

All meters are calibrated to an ANSI Standard of 12% grey (though this gets a bit ambiguous between manufactures and testers). But you can get a good idea of what ‘light meter mid grey/mid tone” looks like by mentally assigning an RGB value of 118,118,118 to it.

However, we – humans – find 18% grey a more acceptable ‘mid tone grey’ both in print and on our modern monitors.

NOTE: 18% grey refers to the level of REFLECTANCE – it reflects 18% of the light falling on it. It can also be reproduced in Photoshop using a grey with 128,128.128 RGB values.

So problem number 1 is that of mid tone perception and the difference between what you ‘see’ and what the camera sees and then does in terms of exposure (if you let the camera make a decision for you).

128RGB grey versus 118RGB meter mid grey

Click on the pano image from Dinorwic to view it bigger, then try to FIND a mid grey that you could point your camera meter at – you can’t.

Remember, the grey you try to measure MUST be exactly mid-grey – try it, it’ll drive you nuts trying to find it!

This leads us to problem number 2.

Take your camera outside, find a white wall. Fill your frame with it and take a shot using ZERO exposure compensation – the wall will look GREY in the resulting shot not WHITE.

Next, find something matte black or near to it. Fill your frame with it and take another shot – the black will look grey in the shot not black(ish).

Problem number 3 is this – and it’s a bit of a two-headed serpent. An exposure meter of any kind is COLOUR BLIND but YOU can SEE colours but are tonally blinded to them to some degree or other:

Simple primary red, green and blue translate to vastly different grey tones which comes as a big surprise to a lot of folk, especially how tonally light green is.

Scene or Subject Brightness Range

Any scene in front of you and your camera has a range of tones from brightest to darkest, and this tonal range is the subject brightness range or SBR for short. Some folk even refer to it as the scene dynamic range.

If you put your camera meter into spot mode you can meter around your chosen scene and make note of the different exposure values for the brightest and darkest areas of your potential shot.

You camera spot meter isn’t the most accurate of spot meters because its ‘spot’ is just too big, typically between 4mm and 5mm, but it will serve to give you a pretty good idea of your potential SBR.

A 1 degree spot meter will, with correct usage, yield a somewhat more accurate picture (pun intended) of the precise SBR of the scene in front of you.

Right about now some of you will be thinking I’m hair-splitting and talking about unnecessary things in todays modern world of post-processing shadow and highlight recovery.

Photography today is full of folk who are prepared to forego the CRAFT of the expert photographer in favour of getting it half-right in camera and then using the crutch of software recovery to correct their mistakes.

Here’s the news – recovery of popped highlights is IMPOSSIBLE and recovery of shadows to anymore than a small degree results in pixel artifacting. Get this, two WRONGS do NOT make a RIGHT!

If the Mercedes F1 team went racing with the same attitude as the majority of camera users take pictures with, then F1 would be banned because drivers would die at an alarming rate and no car would ever make the finish line!

So, one way or another we can quantify our potential scene SBR.

“But Andy I don’t need to do that because my camera meter does that for me…….”

Oh no it does NOT, it just averages it to what IT THINKS is a correct mid tone grey – which it invariably isn’t!

This whole mid tone/mid grey ‘thing’ is a complete waste of time because:

It’s near impossible to find a true mid tone in your scene to take a reading off.
What you want as a mid tone will be at odds with your camera meter by at least 1/2stop.
If you are shooting wildlife or landscapes you can’t introduce a ‘grey card’.
Because of the above, your shot WILL BE UNDER EXPOSED.

“Yeah, but I can always bracket my shots and do an exposure blend Andy so you’re still talking crap….”

Two answers to that one:

You can’t bracket shots and blend if your MAIN subject is moving – de-ghosting is only effective on small parts of a scene with minimal movement between frames.
The popular “shoot and bracket two each end” makes you look like total dickhead and illustrates that you know less than zero about exposure. Try doing that on a paying job in front of the client and see how long you last in a commercial environment.

By far the BEST way of calculating exposure is the ETTR method.

ETTR, Expose to the Right.

If you meter for a highlight, your camera will treat that as a mid tone because your camera ASSUMES it’s a mid tone.

Your camera meter is a robot programmed to react to anything it sees in EXACTLY the same way. It doesn’t matter if your subject is a black cat in the coal house or a snow man in a snow storm, the result will be the same 118,118,118 grey sludge.

Mid tones are as we’ve already ascertained, difficult to pin down and full of ambiguity but highlights are not. So let’s meter the brightest area of the image and expose it hard over to the right of the histogram.

The simplest way to achieve this is to use your live view histogram with the camera in full manual mode.

Unlike the post-shot review histogram, the live-view histogram is not subject to jpeg compression, and can be thought of as something of a direct readout of scene tonality/brightness.

Using your exposure controls (usually shutter speed for landscape photography) you can increase your exposure to push the highlight peak of the histogram to the right as far as you can go before ‘hitting the wall’ on the right hand side of the histogram axis – in other words the camera sensor highlight clipping point.

Of course, this has the added benefit of shifting ALL the other tones ( mids and shadows) to the right as well,resulting in far less clipping potential in your shadow areas.

So back to Dinorwic again and here’s a shot that has been exposed ETTR on the live view histogram using spot metering over what I deemed to be the brightest area of the sky:

The red square indicates the approximate size of the spot meter area.

I was a naughty boy not recording this on video for you but I forgot to pack the HDMI lead for the video recorder – I’ll do one shortly!

The problem with using the Live View Histogram is that it can be a bit of a struggle to see it. your live view screen itself can be hard to see in certain light conditions outside, and the live view histogram itself is usually a bit on the small side – no where near as big as the image review histogram you can see here.

But looking at the review histogram above you can see that there’s a ‘little bit more juice’ to be had in terms of exposure of the highlights because of that tiny gap between the right end of the histogram and the ‘wall’ at the end of the axis.

Going back to the video the maximum ETTR ‘tipping point’ was centered around these three shots:

Clipped

Not Clipped (the one we allocated the star rating to). Exposure is -1/3rd stop below clipped.

Safe, but -2/3rd stop below Clipped.

The review histogram puts the Dinorwic shot highlights firmly in the same exposure bracket as ‘Safe, but -2/3rd stop below Clipped, and tells us there is another 1/3rd stop ‘more juice’ to be had!

So lengthening the exposure by 1/3rd stop and changing from 160th sec to 1/50th sec gives us this:

The red square indicates the approximate size of the spot meter area.

Live View Histogram ETTR

Live View Histogram plus 1/3 stop more juice! Highlights STILL below Clipping Point and shadows get 1/3rd stop more exposure.

That’s what it’s all about baby – MORE JUICE!

And you will not be in a position to confidently acquire more juice unless you find the USABLE DYNAMIC RANGE of your camera sensor.

The whole purpose of finding that usable DR is to discover where your highlight and shadow clipping points are – and they are very different between camera models.

For instance, the highlight clipping point value of the Nikon D850 is different from that of the Nikon D800E, but the shadow clipping point is pretty similar.

There is an awful lot more use to discovering your cameras usable dynamic range than a lot of folk imagine.

And if you do it the precise way then you can acquire a separate meter that will accept camera profiling:

You can create a dynamic range profile for your camera (and lens combo*) and then load it into the meter:

and then have your cameras usable dynamic range as part of the metering scale – so then you have NO EXCUSE for producing a less than optimum exposure.

(*)Note: yes, the lens does have an effect on dynamic range due to micro-contrast and light transmission variables – if you want to be super-picky!

AND THEY SAY HANDHELD METERS ARE DEAD, OLD TECH and of NO USE!!!

Anyone who says or even thinks that is a total KNOB.

Your camera dynamic range, the truthful one – FIND IT, KNOW IT, USE IT.

And don’t listen to the idiots and know-nothings, just listen and heed the advice of those of us who actually know what we’re doing.

NOTE: The value of grey (gray) cards and how to use them for accurate measurement is a subject in its own right and provides the curious with some really interesting reading. Believe me it’s far more expansive than the info I’ve given here. But adopting an ETTR approach when exposing to sensor that you KNOW the physical behavior of (dynamic response to light/dynamic range) can alleviate you of all critical mid-tone concerns.

This article has taken me over 8 hours to produce in total, and is yours to view for FREE. If you feel I deserve some support for doing this then please consider joining my membership site over on Patreon by using the link below.

Alternatively you could donate via PayPal to tuition@wildlifeinpixels.net

More ISO Settings Misinformation

Posted on January 6, 2017 by Andrew Astbury

More ISO Settings Misinformation

This WAS going to be a post about exposure…….!

But, this morning I was on the Facebook page of a friend where I came across a link he’d shared to this page which makes a feature of this diagram:

Please Note: I’m “hot linking” this image so’s not to be accused of theft!

This style of schematic for the Exposure Triangle is years old and so is nothing new.

When using FILM the ISO value IS a measure of sensitivity to light – that of the film, in other words its SPEED. Higher ISO film is more sensitive to light than lower ISO film, and the increased sensitivity brings about larger ‘grain’ in the image.

When we talk ‘digital photography’ however the ISO value HAS NOTHING TO WITH SENSITIVITY TO LIGHT – of anything inside your camera, including the damn sensor.

ISO in digital cameras is APPLIED GAIN. Applied ‘after the exposure has been made’..after the fact…after Elvis has left the freaking building!

Your sensors sensitivity to light is FIXED and dictated by the size of the photosites that make up the sensor – that is, the sensor pixel pitch.

People who persist in leading you guys into thinking that ISO controls sensor sensitivity should be shot, or better still strapped over the muzzle of an artillery piece……..

The article then goes on to advise the following pile of horse crap:

Recommended ISO settings:

ISO 100 or 200 for sunny and bright daylight
ISO 400 ISO for cloudy days, or indoors
ISO 800 for indoors (without a flash)
ISO 1600+ for very low light situations

WTF??? What year are we in – 2007??

And this pile of new 2017 junk is on a website dedicated to a certain camera manufacturer who’s cameras have produced superb images at ISO settings way higher than the parameters stated above for ages.

Take this shot from a Canon 1DX Mk1 – old tech/off-sensor ADCs etc:

Canon 1DX Mark 1 ISO 10,000 1/8000th @ f7.1 – click for the full size image.

ISO settings are at the bottom of the pile when it comes to good action photography – the overriding importance at all times is SHUTTER SPEED and AF performance.

I don’t care about ‘ISO noise’ anywhere near as much as I care about focus and freezing the action, and neither should you guys.

What have the above and below shots got in common – apart from the wildlife category?

1/8000th shutter speed and an aperture of 7.1 – aperture for DoF and shutter speed to freeze the action – stuff the ‘noise’.

And speaking of ‘noise’ – there isn’t anywhere near enough to screw the shot up for stock sale even at full size, and I’ll tell you again, noise hardly prints at all!

Here’s another ‘old tech’ Canon 1DX Mk1 shot:

I don’t really want to wheel the same shots out over and over but don’t forget the Canon 5D Mk4 Great Tit at 10,000ISO or 1DX Mk2 Musk Ox at 16,000ISO either!

Don’t get me wrong, when I want maximum Dynamic Range I shoot at base ISO, but generally you’ll never find me shooting at any fixed ISO other than base; other than when shooting astro landscapes. Everything else is Auto ISO.

So a fan website, in 2017, is basically telling you not to use the ISO speeds that I use all the damn time – and they are justifying that with bad information.

Please people, 90% plus of what you see on the web is total garbage, please don’t take it as gospel truth until you check with someone who actually knows what they are talking about.

Do I know what I’m talking about, well, only you can judge that one. But everything I do tell you can be justified with full resolution images – not meaningless little jpegs on a web site.

Become a patron from as little as $1 per month, and help me produce more free content.

Patrons gain access to a variety of FREE rewards, discounts and bonuses.

Anyway, that’s it – rant over!

As ever, if you like the info in this post hit the subscribe button. Hop over to my YouTube channel and subscribe there too and if you are feeling generous then a couple of bucks donation via PayPal to tuition@wildlifeinpixels.net would be gratefully appreciated!

Thanks Folks!

Twilight & Astro Landscape Photography

Posted on August 19, 2016 by Andrew Astbury

Twilight & Astro Landscape Photography

Everyone likes a nice moody sunset, but great images await those camera operators that start shooting after most folks have started packing their gear away and heading home.

For me, twilight is where the fun starts.

The rock stack lying off the boulder-strewn beach of Porth Saint, Rhoscolyn Head, Anglesey.

The low light levels on a scene once ‘civil daylight’ has ended mean you get awesome light with lower contrast shadows, subtle skies, and nice long shutter speeds for dreamy water effects without needing expensive 10 stop ND filters.

However, that awesome light vanishes very quickly, so you have to be ready! I waited nearly 90 minutes for the shot above.

But that time was spent doing ‘dry runs’ and rehearsals – once the composition was set how I wanted it, the foreground was outside of DoF, so I knew I needed to shoot a focus stack as well as an exposure blend…mmmm….yummy!

Once we have made the long transition from civil daylight end to astronomical daylight end the fun really begins though.

Astro Landscape Photography

The Milky Way over the derelict buildings of Magpie Mine in Derbyshire.

Astro landscape photography, or wide field astro as it’s sometimes known, is not as difficult as a lot of photographers imagine.

But astro landscape photography IS very demanding of your familiarity with your gear, and will require some expenditure on additional bits of kit if disappointment is to be avoided.

Here’s the kit I usually venture out at night with:

Dew Heater Band (A).

An essential bit of kit for astro landscape photography – it’s amazing how rapidly a lot of lenses, especially super-wides like the Nikon 14-24 f2.8 encounter a problem with dew at night. This will in effect fog the front element, starting at its centre and if left unchecked it can spread across the entire face of the lens.

Heating the lens front sufficiently to keep its temperature above the dew point for your current location and time will prevent a ruined session – don’t leave home without one!

This dew heater is powered by a battery (C) via a dew heater controller (D) with is basically a simple rotary rheostat which controls the level of current driving the heater band.

I use mine at about 75% of ‘full chat’ and it seems to work just fine.

A final note on dew heater bands – these are designed for use by those strange folk who spend hours behind telescopes. They tape the bands in place and leave them there. As photographers we need to add or remove them as needed. The bands can prove fragile, need I say more?

Yes, it pays to carry a spare, and it pays to treat them with care and not just throw them in the camera bag – I’m on band number 3 with number 4 in reserve!

Intervalometer (B).

You will need to shoot a long exposure of you scene foreground, slightly re-focuused closer to you, at a much lower ISO, and perhaps at a slightly narrower aperture; this shot might well be 20 minutes long or more and with long exposure NR engaged to produce a black subtraction.

Yes, a lockable cable release and the timer on your watch will do the job, hence (F) and (G) in case (B) stops working!

But an intervalometer will make this easier – as long as you’ve read the instructions..doh!

If you want to shoot star trails the external intervalometer is vastly superior to your cameras built in one. That’s because the in-camera intervalometer on nearly all cameras except the Nikon D810A is limited to a 30 second shutter speed.

An hours worth of star rotation is barely enough:

Twilight & Astro Landscape Photography

But at 30 seconds shutter speed you will end up with 120 frames at fairly high ISO.

Far better to shoot at ‘bulb’ with a 5 minute exposure and lower ISO – then you’ll only have 12 frames – your computer with thank you for the lower number when it comes to stacking the shots in Photoshop.

There is also another problem, for certain marks of Nikon cameras. The D800E that I use has a stupid cap on continuous shooting. The much touted method of setting the shutter to 30 seconds and putting the camera in continuous low speed shooting mode and locking the cable release button down does NOT work – it only allows you to take 100 frames then the camera just STOPS taking pictures.

But if you use an external intervalometer set to a 30 second exposure, continuous and just drop the camera in BULB and Single Shot then the D800E and its like will sit there and fill your cards up with frames.

Other Essentials.

Micro fibre cloths, bin liners and gaffer tape (B,I and J).

After a couple of hours of full darkness your gear (and I mean all of it) will most likely be wet with dew, especially here in the UK. Micro fibre cloths are great for getting the majority of this dampness off your camera gear when you put it away for the trip home.

Bin liners are great for keeping any passing rain shower off your camera gear when its set up – just drop one (opened of course) over your camera and tape it to the tripod legs with a bit of gaffer tape. Leave the dew heater ON.

Also, stick the battery supply in one – rain water and 13 volts DC at 4000MAh don’t mix well.

Photopills on your iPhone (G) is incredibly useful for showing you where the Milky Way is during that extended period between civil and astronomical daylight end. Being able to see it in relationship to your scene with the Night Augmented Reality feature cetainly makes shot composition somewhat easier.

Head Lamp (H) – preferably one which has a red light mode. Red light does not kill off your carefully tuned night vision when you need to see some camera setting control lever or button.

Accurate GPS positioner (K). Not entirely an ‘essential’ but it’s mighty useful for all sorts of reasons, especially when forward planning a shot, or getting to a set position in the dark.

The Milky Way towering over the National Coastwatch Institution (NCI) station at Rhoscolyn on Anglesey.

I love taking someone who’s never seen the Milky Way out at night to capture it with their own equipment – the constant stream of ‘WOWS’ makes me all warm ‘n fuzzy! This year has seen me take more folk out than ever; and even though we are going to loose the galactic centre in the next few weeks the opportunities for night photography get better as the nights grow longer.

The Milky Way over the derelict buildings of Magpie Mine in Derbyshire.

So if you want to get out there with me then just give me shout at tution@wildlifeinpixels.net

The Milky Way will still be a prominent feature in the sky until October, and will be in a more westerly position, so lots of great bays on the North Wales & Anglesey coast will come into their own as locations.

The Milky Way over the Afon Glaslyn Valley looking towards Beddgelert and Porthmadog. The patchy green colour of the sky is cause by a large amount of airglow, another natural phenomenon that very few people actually see.

And just look at that star detail:

Over the next few weeks I’m going to be putting together a training video title on processing astro landscape photography images, and if the next new moon phase at the end of this month comes with favorable weather I’m going to try and supplement these with a couple of practical shooting videos – so fingers crossed.

Become a patron from as little as $1 per month, and help me produce more free content.

Patrons gain access to a variety of FREE rewards, discounts and bonuses.

More Thoughts on the Nikon D5

Posted on March 24, 2016 by Andrew Astbury

More Thoughts on the Nikon D5

Okay, so the Nikon D5 has started to slowly trickle into the hands of people now (though sadly not those belonging to yours truly) and yesterday I was sent a link to some downloadable D5 RAW files.

That link is HERE for those of you that might want a look for yourself.

If you have received this post via email PLEASE view it on the blog itself.

Also, as a matter of interest, Nikon have made the D5 User Manual available HERE.

As I’ve said in earlier posts, I’m quite excited at the thought of the new AF system giving the Nikon shooter access to more Canon-esque controls, but image quality in terms of sensor output and the recorded .NEF are always paramount in my mind.

So I jumped all over the above-linked RAW files, but I have to say that looking at them in Lightroom (neutralised of course as per my previous post HERE) I’m not as overly enamoured as I thought I was going to be.

I’ve seen this camera called ‘The New Lord of Darkness’ with much play being made of its high ISO capability, so let’s have a look at that shall we. ISO range is 100 to 102,000 expandable to 50 and 3,276,800 – ISO stupid and then some!

Before we go any further, I suspect that the downloadable files are Lossless Compressed!

Want to see what 3,276,800ISO looks like?

All shots are by a user named Andy (not me) posted on NikonGear.net – thanks go to him for sharing.

ISO 3,286,800 – Image is NOT full resolution as it’s too big for WordPress!

ISO 3,286,800 or H5 – full resolution crop – CLICK to view at full size.

This image is, honestly, unusable SO WHY charge you the buyer for the ability to produce it??

Let’s have a look at the high native ISO 102,400:

Nikon D5 highest native 102,400 ISO – click for full rez view.

Okay, so in certain circumstances this image would be useful for press reproduction, and I can see the appeal for photojournalists – this level of performance will earn them money, and lots of it.

But I suspect that 75%+ of all global D5 purchasers in its first 12 months will NOT benefit from this performance because they are not in that market place. If you produce weddings shots that look like this then you’re going to get sued up the Ying Yang for sure.

What is interesting is a link on Nikon Rumours which was kindly sent to me yesterday by Paul Atkins:

Photographic Dynamic Range comparison of Nikon D4 and Nikon D5.

This is a ‘live graph’ which you can access directly via this link HERE

This is a comparison of PDR, not EDR, and you will not find the D5 listed at DXO Mark at this moment in time. If you want to get your head around the difference between PDR and EDR then click HERE or HERE. But be warned, MATHS ALERT!

Below 1600 ISO the D5 has a significantly lower PDR than the D4, putting it very much in line with the Canon 1DX at <1600ISO – see HERE.

To my mind the D5 is an all-action camera with good low light capabilities; as is/was the D3 in its time, D4 and D4s and Canons 1DX.

As such, lower ISO performance is not really important – it’s a question of ‘horses for courses’ and the right tool for the job. But the fact that the PDR is lower came as a surprise.

Time was, not so long ago, that I was ‘capped’ at sub 800 ISO for wildlife/action photography – the D3 put paid to that and 1200 to 1600 ISO became my working values when needed.

The D4 and Canon 1DX shifted the goal posts again – 3200 ISO became a standard AND both cameras had AutoISO that worked perfectly.

Nobody with a working brain chooses to work at high ISOs unless they are driven to do so by a need for high shutter speeds in low light – no matter how well a camera sensor functions, image quality will always increase with decreasing ISO.

So examination of the above PDR curves clearly indicate that the true advantage of the D5 over the D4 is on average around 1.3 stops above 1600 ISO – which is a good thing, but it’s not exactly what I’d call revolutionary. We experience pretty much the same increase with every Nikon D FX release.

If PDR increases then the Signal to Noise ratio – S/N – pretty much appears to increase by the same value, so a visual comparison of D4 and D5 images shot at higher than 1600 ISO will show around 1.3Ev to 1.5Ev of reduced ISO noise.

What I do like is the IQ improvements at 8000 ISO and above. 8000 ISO on a D4 is bad, and its top native 12800 ISO is awful. Based on the downloaded raw files, anyone could process a D5 12800 ISO image at full resolution to pass QC at ANY stock agency – just go and download those RAWS on the link at the top of the post and see for yourself.

25,600 ISO – well I might be tempted to down-res those by perhaps 1000 to 1500 pixels on the long edge to help with noise reduction a bit, and chucked onto A3 or A3+ print you would never really notice the noise.

Do I like what I see – yes I do!

Is the D5 the new ‘Lord of Darkness’ – no it bloomin’ well isn’t! Lord of Low Light – quite possibly. The ISO H1 to H5 images go from questionable to crap in my opinion.

Like the Canon 1DX, I’m not impressed at lower ISO values than 1600 – I can get the same or better performance with a D4 or 4S – admittedly though with a lower pixel count.

So overall Andy, does the D5 impress? Well, still being in a hands-off situation I’m not going to commit to a full answer there. When all is said and done, the AF performance will be the key issue for me – a high DR/low noise image of an out of focus subject in no use to me – or anyone else for that matter!

The Way I See Things As They Stand At This Very Moment.

The KING of low ISO with high resolution DSLRs is the Nikon D800E – but it’s not without its limitations. And before you start screaming 5DS at me – it’s a nail, go away..

The best all-round VFM DSLR is the Nikon D810 – a proper jack of all trades who’s only weakness is the occasionally questionable Nikon AF.

The best DSLR autofocus for action is without doubt the Canon 1DX – fabulous AF, crap ergonomics, crap sensor.

The best DSLR sensor for action is the Nikon D4 or 4S – great ergonomics, great sensor, sometimes dubious AF.

But, going on the raw files I’ve downloaded, I strongly suspect that the D5 is going to have the best action sensor title stitched up and dethrone the D4/4S.

Will it dethrone the Canon 1DX in the action AF department – no idea is my truthful answer. I suppose anything is possible, but if it did, would the soon-to-be-released 1DXMk2 take the throne back – quite possibly.

Become a patron from as little as $1 per month, and help me produce more free content.

Patrons gain access to a variety of FREE rewards, discounts and bonuses.

Lightroom – Neutralise Hidden Exposure Compensation

Posted on March 22, 2016 by Andrew Astbury

Lightroom – Neutralise Hidden Exposure Compensation.

We all know how good Lightroom is – but it’s also a total pain in the arse!

Ages ago, I did a post about Lightroom 5 and accurate colour HERE and, according to this blogs page-view stats, that post still gets a large global viewing figure every month – so it’s something of an on-going problem for a lot of users.

But things have moved on a bit since then, and we are now working with the v5 release of Lightroom 6/CC 2015 – and things haven’t got any better, sadly, from the perspective of actually “seeing what you captured”.

The problem lies in the fact that Lightroom, for a long time, ceased to be a “neutral” RAW handler. It uses a variety of ‘behind the scenes’ algorithms to add what it thinks are good adjustments in terms of exposure brightness and contrast. In other words Lightroom adds hidden adjustments which we cannot see because they are not registered on the adjustment sliders under process version 2012.

Why does it do this – God only knows!

But when I take into account the support Lightroom currently offers for mobile phone cameras, cloud synch etc, I can’t help thinking that Adobe are trying to give Lightroom some sort of mass-market appeal by adding what the designers and coders think is some sort of WOW-factor to image previews – though I might be wrong!

But whatever Adobes reasoning, the fact remains that SOME OF US want to see our raw files for what they are – straight gamma 2.2 encoded versions of what the sensor recorded. Only by learning how to Neutralise Hidden Exposure Compensation can we actually arrive at a suitable starting point for the development process.

The Case To Answer

Firstly, a lot of you might be wondering WTF I’m ranting on about – your RAW image previews look great before you start doing anything to them – mmmmm….

If that’s the case then NEWS FLASH – RAW files should look as flat as dish-water pre-process, and you have do some work to make them look good. So believe me, if your raws look “nice ‘n punchy” from the get-go then something is wrong somewhere!

Out there in photography land there are two RAW file handlers that are notorious for being “neutral” in their initial raw render – Raw Digger, and Iridient Developer.

Let me demonstrate the “case to answer” by using the same image I used the other day when giving Canon an indirect slagging off over lossless compression:

Neutralise Hidden Exposure Compensation in Lightroom

Raw file opened in Lightroom with no user adjustments BUT WITH Lightroom ‘hidden exposure compensation’.

Now let’s open the same file in Raw Digger:

Raw file opened in Raw Digger with no user adjustments.

And now in Iridient Developer:

Raw file opened in Iridient Developer with no user adjustments.

And now, just for good measure, my Lightroom-processed version of the image:

Raw file processed in Lightroom WITH user adjustments.

Both RAW Digger and Iridient Developer give the user a much better processing start point simply because they are neutral and don’t go about making contrast-loaded ‘background adjustments’. And I’m sure you can see that the final Lightroom processed version of the image bares more resemblance to the RAW Digger and Iridient screen grabs than the Lightroom ‘as is’ preview.

Now if you are a total maniac then you can go and download either of the two aforementioned raw developers and get yourself super-confused or you can learn how to ‘neutralise’ the Lightroom background adjustment ‘crap’ – which is far easier!

How to Neutralise Hidden Exposure Compensation in Lightroom.

Step 1. Scroll down to the Camera Calibration Panel in the Develop module and switch the Process Version from PV2012 to PV 2010:

Step 1 in Neutralising Lightroom Hidden Exposure Compensation.

Step 2. Scroll up to the Basics panel (a very different looking one if you never used Lightroom 3!) and make the following changes:

Blacks from 5 to 0
Brightness from +50 to 0
Contrast from +25 to 0

Step 2 in Neutralising Lightroom Hidden Exposure Compensation.

Step 3. Move to the Tone Curve and change the Medium Contrast tone curve to Linear:

Step 3 in Neutralising Lightroom Hidden Exposure Compensation.

DO NOT concern yourself with the fact that your image has gone dark and flat, it’s to be expected!

Step 4. Scroll back down to Camera Calibration and switch the process version BACK to PV2012, then scroll back up to the Basics Panel:

Step 4 in Neutralising Lightroom Hidden Exposure Compensation.

Step 5. Yes I know it still looks awful, but if you now change that -1EV to 0 on the exposure slider you’ll get a great process start image:

Step 5 in Neutralising Lightroom Hidden Exposure Compensation.

Looking at the before and after images you can see that we have got contrast under control – in other words we have removed the excess contrast added to the image with the Lightroom hidden background shenanigans.

Indeed, we can see exactly how much contrast has been removed with this ‘by the numbers’ process by looking at the -33 Contrast value – DO NOT RESET THIS BACK TO 0!!!!

The process has decreased contrast still further by lifting the Blacks value to +25. You need to check the shadow areas on the image in this respect. If they are looking a bit noisy (Hello Canon!) you might want to drop the blacks value to maybe +5 to +10 and open the shadows a bit more with a small positive adjustment to the Shadows slider in the basics panel.

And so processing is just a matter of a few subjective tweaks until I’m happy with the image:

Click to view larger image.

In the Tone Curve panel you can see the multi-point Custom Curve the process has added. If you click the up/down arrows to the right of the word Custom you will see a menu giving you the option to save the curve:

Saving the custom curve.

I save the curve with the name 2010to2012 – by default it saves as an .xmp file, and to the user/Library/Application Support/Adobe/CameraRaw/Curves file path (Mac).

Saving the curve is useful as it makes for a very quick adjustment of further images.

However, there is a caveat (isn’t there always!) and it’s this:

The majority of adjustments in Lightroom are specific to camera sensor and ISO. In simple terms the same numeric value of adjustment to any control slider can have differing effects depending on the sensor it was made by and the ISO at which it was shot. It’s very important that you wrap your head around this fact.

The curve I’ve produced here is correct for a Canon 1DX at the shot ISO which was 1000 or 800 if my memory serves correctly. I could apply this curve to a 100 ISO image shot with a Nikon D800E, and it would do a good job, but I might get a slightly better result if I go through the whole process again to produce a custom curve for the D800E using a 100 ISO shot to begin with. But even if that new curve visually gives a different result it will still have the same numeric values in the basics panel!

If I save the curve and then apply it to another image via the Tone Curve panel the contrast and blacks Basic Panel values do NOT change – but you will get a better distribution of contrast.

You may want to generate and save at least a low and high ISO variant of the curve for each of your camera bodies; or you could be a smart-arse like me by just using one curve and eye-balling the finer tweaks.

You can also create the curve and then save the settings as a User Develop Preset and then apply it to future imports via the import module.

So there you have it, how to Neutralise Hidden Exposure Compensation in Lightroom and see you images properly – have fun folks!

Become a patron from as little as $1 per month, and help me produce more free content.

Patrons gain access to a variety of FREE rewards, discounts and bonuses.

Raw File Compression

Posted on March 14, 2016 by Andrew Astbury

Raw File Compression.

Today I’m going to give you my point of view over that most vexatious question – is LOSSLESS raw file compression TRULY lossless?

I’m going to upset one heck of a lot of people here, and my chances of Canon letting me have any new kit to test are going to disappear over the horizon at a great rate of knots, but I feel compelled to post!

What prompts me to commit this act of potential suicide?

It’s this shot from my recent trip to Norway:

Direct from Camera

Processed in Lightroom

I had originally intended to shoot Nikon on this trip using a hire 400mm f2.8, but right at the last minute there was a problem with the lens that couldn’t be sorted out in time, so Calumet supplied me with a 1DX and a 200-400 f4 to basically get me out of a sticky situation.

As you should all know by now, the only problems I have with Canon cameras are their short Dynamic Range, and Canons steadfast refusal to allow for uncompressed raw recording.

The less experienced shooter/processor might look at the shot “ex camera” and be disappointed – it looks like crap, with far too much contrast, overly dark shadows and near-blown highlights.

Shot on Nikon the same image would look more in keeping with the processed version IF SHOT using the uncompressed raw option, which is something I always do without fail; and the extra 3/4 stop dynamic range of the D4 would make a world of difference too.

Would the AF have done as good a job – who knows!

The lighting in the shot is epic from a visual PoV, but bad from a camera exposure one. A wider dynamic range and zero raw compression on my Nikon D4 would allow me to have a little more ‘cavalier attitude’ to lighting scenarios like this – usually I’d shoot with +2/3Ev permanently dialled into the camera. Overall the extra dynamic range would give me less contrast, and I’d have more highlight detail and less need to bump up the shadow areas in post.

In other words processing would be easier, faster and a lot less convoluted.

But I can’t stress enough just how much detrimental difference LOSSLESS raw file compression CAN SOMETIMES make to a shot.

Now there is a lot – and I mean A LOT – of opinionated garbage written all over the internet on various forums etc about lossless raw file compression, and it drives me nuts. Some say it’s bad, most say it makes no difference – and both camps are WRONG!

Sometimes there is NO visual difference between UNCOMPRESSED and LOSSLESS, and sometimes there IS. It all depends on the lighting and the nature of the scene/subject colours and how they interact with said lighting.

The main problem with the ‘it makes no difference’ camp is that they never substantiate their claims; and if they are Canon shooters they can’t – because they can’t produce an image with zero raw file compression to compare their standard lossless CR2 files to!

So I’ve come up with a way of illustrating visually the differences between various levels of raw file compression on Nikon using the D800E and Photoshop.

But before we ‘get to it’ let’s firstly refresh your understanding. A camera raw file is basically a gamma 1.0, or LINEAR gamma file:

Linear (top) vs Encoded Gamma

The right hand 50% of the linear gamma gradient represents the brightest whole stop of exposure – that’s one heck of a lot of potential for recording subtle highlight detail in a raw file.

It also represents the area of tonal range that is frequently most effected by any form of raw file compression.

Neither Nikon or Canon will reveal to the world the algorithm-based methods they use for lossless or lossy raw file compression, but it usually works by a process of ‘Bayer Binning’.

Bayer_Pattern

If we take a 2×2 block, it contains 2 green, 1 red and 1 blue photosite photon value – if we average the green value and then interpolate new values for red and blue output we will successfully compress the raw file. But the data will be ‘faux’ data, not real data.

The other method we could use is to compress the tonal values in that brightest stop of recorded highlight tone – which is massive don’t forget – but this will result in a ’rounding up or down’ of certain bright tonal values thus potentially reducing some of the more subtle highlight details.

We could also use some variant of the same type of algorithm to ‘rationalise’ shadow detail as well – with pretty much the same result.

In the face of Nikon and Canons refusal to divulge their methodologies behind raw file compression, especially lossless, we can only guess what is actually happening.

I read somewhere that with lossless raw file compression the compression algorithms leave a trace instruction about what they have done and where they’ve done it in order that a raw handler programme such as Lightroom can actually ‘undo’ the compression effects – that sounds like a recipe for disaster if you ask me!

Personally I neither know nor do I care – I know that lossless raw file compression CAN be detrimental to images shot under certain conditions, and here’s the proof – of a fashion:

Let’s look at the following files:

Image 1: 14 bit UNCOMPRESSED

Image 2: 14 bit UNCOMPRESSED

Image 3: 14 bit LOSSLESS compression

Image 4: 14 bit LOSSY compression

Image 5: 12 bit UNCOMPRESSED

Yes, there are 2 files which are identical, that is 14 bit uncompressed – and there’s a reason for that which will become apparent in a minute.

First, some basic Photoshop ‘stuff’. If I open TWO images in Photoshop as separate layers in the same document, and change the blend mode of the top layer to DIFFERENCE I can then see the differences between the two ‘images’. It’s not a perfect way of proving my point because of the phenomenon of photon flux.

Photon Flux Andy??? WTF is that?

Well, here’s where shooting two identical 14 bit uncompressed files comes in – they themselves are NOT identical!:

The result of overlaying the two identical uncompressed raw files (above left) – it looks almost black all over indicating that the two shots are indeed pretty much the same in every pixel. But if I amplify the image with a levels layer (above right) you can see the differences more clearly.

So there you have it – Photon Flux! The difference between two 14 bit UNCOMPRESSED raw files shot at the same time, same ISO, shutter speed AND with a FULLY MANUAL APERTURE. The only difference between the two shots is the ratio and number of photons striking the subject and being reflected into the lens.

Firstly 14 Bit UNCOMPRESSED compared to 14 bit LOSSLESS (the important one!):

14 bit UNCOMPRESSED vs 14 bit LOSSLESS

Please remember, the above ‘difference’ image contains photon flux variations too, but if you look carefully you will see greater differences than in the ‘flux only’ image above.

The two images above illustrate the differences between 14 bit uncompressed and 14 bit LOSSY compression (left) and 14 bit UNCOMPRESSED and 12 bit UNCOMPRESSED (right) just for good measure!

In Conclusion

As I indicated earlier in the post, this is not a definitive testing method, sequential shots will always contain a photon flux variation that ‘pollutes’ the ‘difference’ image.

I purposefully chose this white subject with textured aluminium fittings and a blackish LED screen because the majority of sensor response will lie in that brightest gamma 1.0 stop.

The exposure was a constant +1EV, 1/30th @ f 18 and 100 ISO – nearly maximum dynamic range for the D800E, and f18 was set manually to avoid any aperture flicker caused by auto stop down.

You can see from all the ‘difference’ images that the part of the subject that seems to suffer the most is the aluminium part, not the white areas. The aluminium has a stippled texture causing a myriad of small specular highlights – brighter than the white parts of the subject.

What would 14 bit uncompressed minus 14 bit lossless minus photon flux look like? In a perfect world I’d be able to show you accurately, but we don’t live in one of those so I can’t!

We can try it using the flux shot from earlier:

But this is wildly inaccurate as the flux component is not pertinent to the photons at the actual time the lossless compression shot was taken. But the fact that you CAN see an image does HINT that there is a real difference between UNCOMPRESSED and LOSSLESS compression – in certain circumstances at least.

If you have never used a camera that offers the zero raw file compression option then basically what you’ve never had you never miss. But as a Nikon shooter I shoot uncompressed all the time – 90% of the time I don’t need to, but it just saves me having to remember something when I do need the option.

raw file compression

Would this 1DX shot be served any better through UNCOMPRESSED raw recording? Most likely NO – why? Low Dynamic Range caused in the main by flat low contrast lighting means no deep dark shadows and nothing approaching a highlight.

I don’t see it as a costly option in terms of buffer capacity or on-board storage, and when it comes to processing I would much rather have a surfeit of sensor data rather than a lack of it – no matter how small that deficit might be.

Lossless raw file compression has NO positive effect on your images, and it’s sole purpose in life is to allow you to fit more shots on the storage media – that’s it pure and simple. If you have the option to shoot uncompressed then do so, and buy a bigger card!

What pisses my off about Canon is that it would only take, I’m sure, a firmware upgrade to give the 1DX et al the ability to record with zero raw file compression – and, whether needed or not, it would stop miserable grumpy gits like me banging on about it!

Become a patron from as little as $1 per month, and help me produce more free content.

Patrons gain access to a variety of FREE rewards, discounts and bonuses.