More Thoughts on the Nikon D5

More Thoughts on the Nikon D5

Nikon D5

Okay, so the Nikon D5 has started to slowly trickle into the hands of people now (though sadly not those belonging to yours truly) and yesterday I was sent a link to some downloadable D5 RAW files.

That link is HERE for those of you that might want a look for yourself.

If you have received this post via email PLEASE view it on the blog itself.

Also, as a matter of interest, Nikon have made the D5 User Manual available HERE.

As I’ve said in earlier posts, I’m quite excited at the thought of the new AF system giving the Nikon shooter access to more Canon-esque controls, but image quality in terms of sensor output and the recorded .NEF are always paramount in my mind.

So I jumped all over the above-linked RAW files, but I have to say that looking at them in Lightroom (neutralised of course as per my previous post HERE) I’m not as overly enamoured as I thought I was going to be.

I’ve seen this camera called ‘The New Lord of Darkness’ with much play being made of its high ISO capability, so let’s have a look at that shall we.  ISO range is 100 to 102,000 expandable to 50 and 3,276,800 – ISO stupid and then some!

Before we go any further, I suspect that the downloadable files are Lossless Compressed!

Want to see what 3,276,800ISO looks like?

All shots are by a user named Andy (not me) posted on NikonGear.net – thanks go to him for sharing.

Nikon D5

ISO 3,286,800 – Image is NOT full resolution as it’s too big for WordPress!

Nikon D5

ISO 3,286,800 or H5 – full resolution crop – CLICK to view at full size.

This image is, honestly, unusable SO WHY charge you the buyer for the ability to produce it??

Let’s have a look at the high native ISO 102,400:

Nikon D5

Nikon D5 highest native 102,400 ISO – click for full rez view.

Okay, so in certain circumstances this image would be useful for press reproduction, and I can see the appeal for photojournalists – this level of performance will earn them money, and lots of it.

But I suspect that 75%+ of all global D5 purchasers in its first 12 months will NOT benefit from this performance because they are not in that market place. If you produce weddings shots that look like this then you’re going to get sued up the Ying Yang for sure.

What is interesting is a link on Nikon Rumours which was kindly sent to me yesterday by Paul Atkins:

Nikon D5

Photographic Dynamic Range comparison of Nikon D4 and Nikon D5.

This is a ‘live graph’ which you can access directly via this link HERE

This is a comparison of PDR, not EDR, and you will not find the D5 listed at DXO Mark at this moment in time. If you want to get your head around the difference between PDR and EDR then click HERE or HERE. But be warned, MATHS ALERT!

Below 1600 ISO the D5 has a significantly lower PDR than the D4, putting it very much in line with the Canon 1DX at <1600ISO – see HERE.

To my mind the D5 is an all-action camera with good low light capabilities; as is/was the D3 in its time, D4 and D4s and Canons 1DX.

As such, lower ISO performance is not really important – it’s a question of ‘horses for courses’ and the right tool for the job.  But the fact that the PDR is lower came as a surprise.

Time was, not so long ago, that I was ‘capped’ at sub 800 ISO for wildlife/action photography – the D3 put paid to that and 1200 to 1600 ISO became my working values when needed.

The D4 and Canon 1DX shifted the goal posts again – 3200 ISO became a standard AND both cameras had AutoISO that worked perfectly.

Nobody with a working brain chooses to work at high ISOs unless they are driven to do so by a need for high shutter speeds in low light – no matter how well a camera sensor functions, image quality will always increase with decreasing ISO.

So examination of the above PDR curves clearly indicate that the true advantage of the D5 over the D4 is on average around 1.3 stops above 1600 ISO – which is a good thing, but it’s not exactly what I’d call revolutionary.  We experience pretty much the same increase with every Nikon D FX release.

If PDR increases then the Signal to Noise ratio – S/N – pretty much appears to increase by the same value, so a visual comparison of D4 and D5 images shot at higher than 1600 ISO will show around 1.3Ev to 1.5Ev of reduced ISO noise.

What I do like is the IQ improvements at 8000 ISO and above.  8000 ISO on a D4 is bad, and its top native 12800 ISO is awful.  Based on the downloaded raw files, anyone could process a D5 12800 ISO image at full resolution to pass QC at ANY stock agency – just go and download those RAWS on the link at the top of the post and see for yourself.

25,600 ISO – well I might be tempted to down-res those by perhaps 1000 to 1500 pixels on the long edge to help with noise reduction a bit, and chucked onto A3 or A3+ print you would never really notice the noise.

Do I like what I see – yes I do!

Is the D5 the new ‘Lord of Darkness’ – no it bloomin’ well isn’t!  Lord of Low Light – quite possibly.  The ISO H1 to H5 images go from questionable to crap in my opinion.

Like the Canon 1DX, I’m not impressed at lower ISO values than 1600 – I can get the same or better performance with a D4 or 4S – admittedly though with a lower pixel count.

So overall Andy, does the D5 impress?  Well, still being in a hands-off situation I’m not going to commit to a full answer there.  When all is said and done, the AF performance will be the key issue for me – a high DR/low noise image of an out of focus subject in no use to me – or anyone else for that matter!

The Way I See Things As They Stand At This Very Moment.

The KING of low ISO with high resolution DSLRs is the Nikon D800E – but it’s not without its limitations. And before you start screaming 5DS at me – it’s a nail, go away..

The best all-round VFM DSLR is the Nikon D810 – a proper jack of all trades who’s only weakness is the occasionally questionable Nikon AF.

The best DSLR autofocus for action is without doubt the Canon 1DX – fabulous AF, crap ergonomics, crap sensor.

The best DSLR sensor for action is the Nikon D4 or 4S – great ergonomics, great sensor, sometimes dubious AF.

But, going on the raw files I’ve downloaded, I strongly suspect that the D5 is going to have the best action sensor title stitched up and dethrone the D4/4S.

Will it dethrone the Canon 1DX in the action AF department – no idea is my truthful answer.  I suppose anything is possible, but if it did, would the soon-to-be-released 1DXMk2 take the throne back – quite possibly.

Become a patron from as little as $1 per month, and help me produce more free content.

Patrons gain access to a variety of FREE rewards, discounts and bonuses.

Lightroom – Neutralise Hidden Exposure Compensation

Lightroom – Neutralise Hidden Exposure Compensation.

We all know how good Lightroom is – but it’s also a total pain in the arse!

Ages ago, I did a post about Lightroom 5 and accurate colour HERE and, according to this blogs page-view stats, that post still gets a large global viewing figure every month – so it’s something of an on-going problem for a lot of users.

But things have moved on a bit since then, and we are now working with the v5 release of Lightroom 6/CC 2015 – and things haven’t got any better, sadly, from the perspective of actually “seeing what you captured”.

The problem lies in the fact that Lightroom, for a long time, ceased to be a “neutral” RAW handler.  It uses a variety of ‘behind the scenes’ algorithms to add what it thinks are good adjustments in terms of exposure brightness and contrast.  In other words Lightroom adds hidden adjustments which we cannot see because they are not registered on the adjustment sliders under process version 2012.

Why does it do this – God only knows!

But when I take into account the support Lightroom currently offers for mobile phone cameras, cloud synch etc, I can’t help thinking that Adobe are trying to give Lightroom some sort of mass-market appeal by adding what the designers and coders think is some sort of WOW-factor to image previews – though I might be wrong!

But whatever Adobes reasoning, the fact remains that SOME OF US want to see our raw files for what they are – straight gamma 2.2 encoded versions of what the sensor recorded.  Only by learning how to Neutralise Hidden Exposure Compensation  can we actually arrive at a suitable starting point for the development process.

The Case To Answer

Firstly, a lot of you might be wondering WTF I’m ranting on about – your RAW image previews look great before you start doing anything to them – mmmmm….

If that’s the case then NEWS FLASH – RAW files should look as flat as dish-water pre-process, and you have do some work to make them look good.  So believe me, if your raws look “nice ‘n punchy” from the get-go then something is wrong somewhere!

Out there in photography land there are two RAW file handlers that are notorious for being “neutral” in their initial raw render – Raw Digger, and Iridient Developer.

Let me demonstrate the “case to answer” by using the same image I used the other day when giving Canon an indirect slagging off over lossless compression:

Neutralise Hidden Exposure Compensation in Lightroom

Raw file opened in Lightroom with no user adjustments BUT WITH Lightroom ‘hidden exposure compensation’.

Now let’s open the same file in Raw Digger:

Neutralise Hidden Exposure Compensation in Lightroom

Raw file opened in Raw Digger with no user adjustments.

And now in Iridient Developer:

Neutralise Hidden Exposure Compensation in Lightroom

Raw file opened in Iridient Developer with no user adjustments.

And now, just for good measure, my Lightroom-processed version of the image:

Neutralise Hidden Exposure Compensation in Lightroom

Raw file processed in Lightroom WITH user adjustments.

Both RAW Digger and Iridient Developer give the user a much better processing start point simply because they are neutral and don’t go about making contrast-loaded ‘background adjustments’.  And I’m sure you can see that the final Lightroom processed version of the image bares more resemblance to the RAW Digger and Iridient screen grabs than the Lightroom ‘as is’ preview.

Now if you are a total maniac then you can go and download either of the two aforementioned raw developers and get yourself super-confused or you can learn how to ‘neutralise’ the Lightroom background adjustment ‘crap’ – which is far easier!

How to Neutralise Hidden Exposure Compensation in Lightroom.

Step 1.  Scroll down to the Camera Calibration Panel in the Develop module and switch the Process Version from PV2012 to PV 2010:

Neutralise Hidden Exposure Compensation in Lightroom

Step 1 in Neutralising Lightroom Hidden Exposure Compensation.

Step 2.  Scroll up to the Basics panel (a very different looking one if you never used Lightroom 3!) and make the following changes:

  1. Blacks from 5 to 0
  2. Brightness from +50 to 0
  3. Contrast from +25 to 0
Neutralise Hidden Exposure Compensation in Lightroom

Step 2 in Neutralising Lightroom Hidden Exposure Compensation.

Step 3.  Move to the Tone Curve and change the Medium Contrast tone curve to Linear:

Neutralise Hidden Exposure Compensation in Lightroom

Step 3 in Neutralising Lightroom Hidden Exposure Compensation.

DO NOT concern yourself with the fact that your image has gone dark and flat, it’s to be expected!

Step 4.  Scroll back down to Camera Calibration and switch the process version BACK to PV2012, then scroll back up to the Basics Panel:

Neutralise Hidden Exposure Compensation in Lightroom

Step 4 in Neutralising Lightroom Hidden Exposure Compensation.

Step 5.  Yes I know it still looks awful, but if you now change that -1EV to 0 on the exposure slider you’ll get a great process start image:

Neutralise Hidden Exposure Compensation in Lightroom

Step 5 in Neutralising Lightroom Hidden Exposure Compensation.

Looking at the before and after images you can see that we have got contrast under control – in other words we have removed the excess contrast added to the image with the  Lightroom hidden background shenanigans.

Indeed, we can see exactly how much contrast has been removed with this ‘by the numbers’ process by looking at the -33 Contrast value – DO NOT RESET THIS BACK TO 0!!!!

The process has decreased contrast still further by lifting the Blacks value to +25.  You need to check the shadow areas on the image in this respect.  If they are looking a bit noisy (Hello Canon!) you might want to drop the blacks value to maybe +5 to +10 and open the shadows a bit more with a small positive adjustment to the Shadows slider in the basics panel.

And so processing is just a matter of a few subjective tweaks until I’m happy with the image:

Neutralise Hidden Exposure Compensation in Lightroom

Click to view larger image.

In the Tone Curve panel you can see the multi-point Custom Curve the process has added.  If you click the up/down arrows to the right of the word Custom you will see a menu giving you the option to save the curve:

Neutralise Hidden Exposure Compensation in Lightroom

Saving the custom curve.

I save the curve with the name 2010to2012 – by default it saves as an .xmp file, and to the user/Library/Application Support/Adobe/CameraRaw/Curves file path (Mac).

Saving the curve is useful as it makes for a very quick adjustment of further images.

However, there is a caveat (isn’t there always!) and it’s this:

The majority of  adjustments in Lightroom are specific to camera sensor and ISO.  In simple terms the same numeric value of adjustment to any control slider can have differing effects depending on the sensor it was made by and the ISO at which it was shot.  It’s very important that you wrap your head around this fact.

The curve I’ve produced here is correct for a Canon 1DX at the shot ISO which was 1000 or 800 if my memory serves correctly.  I could apply this curve to a 100 ISO image shot with a Nikon D800E, and it would do a good job, but I might get a slightly better result if I go through the whole process again to produce a custom curve for the D800E using a 100 ISO shot to begin with.  But even if that new curve visually gives a different result it will still have the same numeric values in the basics panel!

If I save the curve and then apply it to another image via the Tone Curve panel the contrast and blacks Basic Panel values do NOT change – but you will get a better distribution of contrast.

You may want to generate and save at least a low and high ISO variant of the curve for each of your camera bodies; or you could be a smart-arse like me by just using one curve and eye-balling the finer tweaks.

You can also create the curve and then save the settings as a User Develop Preset and then apply it to future imports via the import module.

So there you have it, how to Neutralise Hidden Exposure Compensation in Lightroom and see you images properly – have fun folks!

Become a patron from as little as $1 per month, and help me produce more free content.

Patrons gain access to a variety of FREE rewards, discounts and bonuses.

Raw File Compression

Raw File Compression.

Today I’m going to give you my point of view over that most vexatious question – is LOSSLESS raw file compression TRULY lossless?

I’m going to upset one heck of a lot of people here, and my chances of Canon letting me have any new kit to test are going to disappear over the horizon at a great rate of knots, but I feel compelled to post!

What prompts me to commit this act of potential suicide?

It’s this shot from my recent trip to Norway:

FW1Q1351-2

Direct from Camera

FW1Q1351

Processed in Lightroom

I had originally intended to shoot Nikon on this trip using a hire 400mm f2.8, but right at the last minute there was a problem with the lens that couldn’t be sorted out in time, so Calumet supplied me with a 1DX and a 200-400 f4 to basically get me out of a sticky situation.

As you should all know by now, the only problems I have with Canon cameras are their  short Dynamic Range, and Canons steadfast refusal to allow for uncompressed raw recording.

The less experienced shooter/processor might look at the shot “ex camera” and be disappointed – it looks like crap, with far too much contrast, overly dark shadows and near-blown highlights.

Shot on Nikon the same image would look more in keeping with the processed version IF SHOT using the uncompressed raw option, which is something I always do without fail; and the extra 3/4 stop dynamic range of the D4 would make a world of difference too.

Would the AF have done as good a job – who knows!

The lighting in the shot is epic from a visual PoV, but bad from a camera exposure one. A wider dynamic range and zero raw compression on my Nikon D4 would allow me to have a little more ‘cavalier attitude’ to lighting scenarios like this – usually I’d shoot with +2/3Ev permanently dialled into the camera.  Overall the extra dynamic range would give me less contrast, and I’d have more highlight detail and less need to bump up the shadow areas in post.

In other words processing would be easier, faster and a lot less convoluted.

But I can’t stress enough just how much detrimental difference LOSSLESS raw file compression CAN SOMETIMES make to a shot.

Now there is a lot – and I mean A LOT – of opinionated garbage written all over the internet on various forums etc about lossless raw file compression, and it drives me nuts.  Some say it’s bad, most say it makes no difference – and both camps are WRONG!

Sometimes there is NO visual difference between UNCOMPRESSED and LOSSLESS, and sometimes there IS.  It all depends on the lighting and the nature of the scene/subject colours and how they interact with said lighting.

The main problem with the ‘it makes no difference’ camp is that they never substantiate their claims; and if they are Canon shooters they can’t – because they can’t produce an image with zero raw file compression to compare their standard lossless CR2 files to!

So I’ve come up with a way of illustrating visually the differences between various levels of raw file compression on Nikon using the D800E and Photoshop.

But before we ‘get to it’ let’s firstly refresh your understanding. A camera raw file is basically a gamma 1.0, or LINEAR gamma file:

gamma,gamma encoding,Andy Astbury

Linear (top) vs Encoded Gamma

The right hand 50% of the linear gamma gradient represents the brightest whole stop of exposure – that’s one heck of a lot of potential for recording subtle highlight detail in a raw file.

It also represents the area of tonal range that is frequently most effected by any form of raw file compression.

Neither Nikon or Canon will reveal to the world the algorithm-based methods they use for lossless or lossy raw file compression, but it usually works by a process of ‘Bayer Binning’.

Bayer_Pattern

If we take a 2×2 block, it contains 2 green, 1 red and 1 blue photosite photon value – if we average the green value and then interpolate new values for red and blue output we will successfully compress the raw file.  But the data will be ‘faux’ data, not real data.

The other method we could use is to compress the tonal values in that brightest stop of recorded highlight tone – which is massive don’t forget – but this will result in a ’rounding up or down’ of certain bright tonal values thus potentially reducing some of the more subtle highlight details.

We could also use some variant of the same type of algorithm to ‘rationalise’ shadow detail as well – with pretty much the same result.

In the face of Nikon and Canons refusal to divulge their methodologies behind raw file compression, especially lossless, we can only guess what is actually happening.

I read somewhere that with lossless raw file compression the compression algorithms leave a trace instruction about what they have done and where they’ve done it in order that a raw handler programme such as Lightroom can actually ‘undo’ the compression effects – that sounds like a recipe for disaster if you ask me!

Personally I neither know nor do I care – I know that lossless raw file compression CAN be detrimental to images shot under certain conditions, and here’s the proof – of a fashion:

Let’s look at the following files:

raw file compression

Image 1: 14 bit UNCOMPRESSED

raw file compression

Image 2: 14 bit UNCOMPRESSED

raw file compression

Image 3: 14 bit LOSSLESS compression

raw file compression

Image 4: 14 bit LOSSY compression

raw file compression

Image 5: 12 bit UNCOMPRESSED

Yes, there are 2 files which are identical, that is 14 bit uncompressed – and there’s a reason for that which will become apparent in a minute.

First, some basic Photoshop ‘stuff’.  If I open TWO images in Photoshop as separate layers in the same document, and change the blend mode of the top layer to DIFFERENCE I can then see the differences between the two ‘images’.  It’s not a perfect way of proving my point because of the phenomenon of photon flux.

Photon Flux Andy??? WTF is that?

Well, here’s where shooting two identical 14 bit uncompressed files comes in – they themselves are NOT identical!:

controlunamplified control

The result of overlaying the two identical uncompressed raw files (above left) – it looks almost black all over indicating that the two shots are indeed pretty much the same in every pixel.  But if I amplify the image with a levels layer (above right) you can see the differences more clearly.

So there you have it – Photon Flux! The difference between two 14 bit UNCOMPRESSED raw files shot at the same time, same ISO, shutter speed AND with a FULLY MANUAL APERTURE.  The only difference between the two shots is the ratio and number of photons striking the subject and being reflected into the lens.

Firstly 14 Bit UNCOMPRESSED compared to 14 bit LOSSLESS (the important one!):

raw file compression

14 bit UNCOMPRESSED vs 14 bit LOSSLESS

Please remember, the above ‘difference’ image contains photon flux variations too, but if you look carefully you will see greater differences than in the ‘flux only’ image above.

raw file compression raw file compression

The two images above illustrate the differences between 14 bit uncompressed and 14 bit LOSSY compression (left) and 14 bit UNCOMPRESSED and 12 bit UNCOMPRESSED (right) just for good measure!

In Conclusion

As I indicated earlier in the post, this is not a definitive testing method, sequential shots will always contain a photon flux variation that ‘pollutes’ the ‘difference’ image.

I purposefully chose this white subject with textured aluminium fittings and a blackish LED screen because the majority of sensor response will lie in that brightest gamma 1.0 stop.

The exposure was a constant +1EV, 1/30th @ f 18 and 100 ISO – nearly maximum dynamic range for the D800E, and f18 was set manually to avoid any aperture flicker caused by auto stop down.

You can see from all the ‘difference’ images that the part of the subject that seems to suffer the most is the aluminium part, not the white areas.  The aluminium has a stippled texture causing a myriad of small specular highlights – brighter than the white parts of the subject.

What would 14 bit uncompressed minus 14 bit lossless minus photon flux look like?  In a perfect world I’d be able to show you accurately, but we don’t live in one of those so I can’t!

We can try it using the flux shot from earlier:

raw file compression

But this is wildly inaccurate as the flux component is not pertinent to the photons at the actual time the lossless compression shot was taken.  But the fact that you CAN see an image does HINT that there is a real difference between UNCOMPRESSED and LOSSLESS compression – in certain circumstances at least.

If you have never used a camera that offers the zero raw file compression option then basically what you’ve never had you never miss.  But as a Nikon shooter I shoot uncompressed all the time – 90% of the time I don’t need to, but it just saves me having to remember something when I do need the option.

raw file compression

Would this 1DX shot be served any better through UNCOMPRESSED raw recording?  Most likely NO – why?  Low Dynamic Range caused in the main by flat low contrast lighting means no deep dark shadows and nothing approaching a highlight.

I don’t see it as a costly option in terms of buffer capacity or on-board storage, and when it comes to processing I would much rather have a surfeit of sensor data rather than a lack of it – no matter how small that deficit might be.

Lossless raw file compression has NO positive effect on your images, and it’s sole purpose in life is to allow you to fit more shots on the storage media – that’s it pure and simple.  If you have the option to shoot uncompressed then do so, and buy a bigger card!

What pisses my off about Canon is that it would only take, I’m sure, a firmware upgrade to give the 1DX et al the ability to record with zero raw file compression – and, whether needed or not, it would stop miserable grumpy gits like me banging on about it!

Become a patron from as little as $1 per month, and help me produce more free content.

Patrons gain access to a variety of FREE rewards, discounts and bonuses.