Raw File Compression

Raw File Compression.

Today I’m going to give you my point of view over that most vexatious question – is LOSSLESS raw file compression TRULY lossless?

I’m going to upset one heck of a lot of people here, and my chances of Canon letting me have any new kit to test are going to disappear over the horizon at a great rate of knots, but I feel compelled to post!

What prompts me to commit this act of potential suicide?

It’s this shot from my recent trip to Norway:

FW1Q1351-2

Direct from Camera

FW1Q1351

Processed in Lightroom

I had originally intended to shoot Nikon on this trip using a hire 400mm f2.8, but right at the last minute there was a problem with the lens that couldn’t be sorted out in time, so Calumet supplied me with a 1DX and a 200-400 f4 to basically get me out of a sticky situation.

As you should all know by now, the only problems I have with Canon cameras are their  short Dynamic Range, and Canons steadfast refusal to allow for uncompressed raw recording.

The less experienced shooter/processor might look at the shot “ex camera” and be disappointed – it looks like crap, with far too much contrast, overly dark shadows and near-blown highlights.

Shot on Nikon the same image would look more in keeping with the processed version IF SHOT using the uncompressed raw option, which is something I always do without fail; and the extra 3/4 stop dynamic range of the D4 would make a world of difference too.

Would the AF have done as good a job – who knows!

The lighting in the shot is epic from a visual PoV, but bad from a camera exposure one. A wider dynamic range and zero raw compression on my Nikon D4 would allow me to have a little more ‘cavalier attitude’ to lighting scenarios like this – usually I’d shoot with +2/3Ev permanently dialled into the camera.  Overall the extra dynamic range would give me less contrast, and I’d have more highlight detail and less need to bump up the shadow areas in post.

In other words processing would be easier, faster and a lot less convoluted.

But I can’t stress enough just how much detrimental difference LOSSLESS raw file compression CAN SOMETIMES make to a shot.

Now there is a lot – and I mean A LOT – of opinionated garbage written all over the internet on various forums etc about lossless raw file compression, and it drives me nuts.  Some say it’s bad, most say it makes no difference – and both camps are WRONG!

Sometimes there is NO visual difference between UNCOMPRESSED and LOSSLESS, and sometimes there IS.  It all depends on the lighting and the nature of the scene/subject colours and how they interact with said lighting.

The main problem with the ‘it makes no difference’ camp is that they never substantiate their claims; and if they are Canon shooters they can’t – because they can’t produce an image with zero raw file compression to compare their standard lossless CR2 files to!

So I’ve come up with a way of illustrating visually the differences between various levels of raw file compression on Nikon using the D800E and Photoshop.

But before we ‘get to it’ let’s firstly refresh your understanding. A camera raw file is basically a gamma 1.0, or LINEAR gamma file:

gamma,gamma encoding,Andy Astbury

Linear (top) vs Encoded Gamma

The right hand 50% of the linear gamma gradient represents the brightest whole stop of exposure – that’s one heck of a lot of potential for recording subtle highlight detail in a raw file.

It also represents the area of tonal range that is frequently most effected by any form of raw file compression.

Neither Nikon or Canon will reveal to the world the algorithm-based methods they use for lossless or lossy raw file compression, but it usually works by a process of ‘Bayer Binning’.

Bayer_Pattern

If we take a 2×2 block, it contains 2 green, 1 red and 1 blue photosite photon value – if we average the green value and then interpolate new values for red and blue output we will successfully compress the raw file.  But the data will be ‘faux’ data, not real data.

The other method we could use is to compress the tonal values in that brightest stop of recorded highlight tone – which is massive don’t forget – but this will result in a ’rounding up or down’ of certain bright tonal values thus potentially reducing some of the more subtle highlight details.

We could also use some variant of the same type of algorithm to ‘rationalise’ shadow detail as well – with pretty much the same result.

In the face of Nikon and Canons refusal to divulge their methodologies behind raw file compression, especially lossless, we can only guess what is actually happening.

I read somewhere that with lossless raw file compression the compression algorithms leave a trace instruction about what they have done and where they’ve done it in order that a raw handler programme such as Lightroom can actually ‘undo’ the compression effects – that sounds like a recipe for disaster if you ask me!

Personally I neither know nor do I care – I know that lossless raw file compression CAN be detrimental to images shot under certain conditions, and here’s the proof – of a fashion:

Let’s look at the following files:

raw file compression

Image 1: 14 bit UNCOMPRESSED

raw file compression

Image 2: 14 bit UNCOMPRESSED

raw file compression

Image 3: 14 bit LOSSLESS compression

raw file compression

Image 4: 14 bit LOSSY compression

raw file compression

Image 5: 12 bit UNCOMPRESSED

Yes, there are 2 files which are identical, that is 14 bit uncompressed – and there’s a reason for that which will become apparent in a minute.

First, some basic Photoshop ‘stuff’.  If I open TWO images in Photoshop as separate layers in the same document, and change the blend mode of the top layer to DIFFERENCE I can then see the differences between the two ‘images’.  It’s not a perfect way of proving my point because of the phenomenon of photon flux.

Photon Flux Andy??? WTF is that?

Well, here’s where shooting two identical 14 bit uncompressed files comes in – they themselves are NOT identical!:

controlunamplified control

The result of overlaying the two identical uncompressed raw files (above left) – it looks almost black all over indicating that the two shots are indeed pretty much the same in every pixel.  But if I amplify the image with a levels layer (above right) you can see the differences more clearly.

So there you have it – Photon Flux! The difference between two 14 bit UNCOMPRESSED raw files shot at the same time, same ISO, shutter speed AND with a FULLY MANUAL APERTURE.  The only difference between the two shots is the ratio and number of photons striking the subject and being reflected into the lens.

Firstly 14 Bit UNCOMPRESSED compared to 14 bit LOSSLESS (the important one!):

raw file compression

14 bit UNCOMPRESSED vs 14 bit LOSSLESS

Please remember, the above ‘difference’ image contains photon flux variations too, but if you look carefully you will see greater differences than in the ‘flux only’ image above.

raw file compression raw file compression

The two images above illustrate the differences between 14 bit uncompressed and 14 bit LOSSY compression (left) and 14 bit UNCOMPRESSED and 12 bit UNCOMPRESSED (right) just for good measure!

In Conclusion

As I indicated earlier in the post, this is not a definitive testing method, sequential shots will always contain a photon flux variation that ‘pollutes’ the ‘difference’ image.

I purposefully chose this white subject with textured aluminium fittings and a blackish LED screen because the majority of sensor response will lie in that brightest gamma 1.0 stop.

The exposure was a constant +1EV, 1/30th @ f 18 and 100 ISO – nearly maximum dynamic range for the D800E, and f18 was set manually to avoid any aperture flicker caused by auto stop down.

You can see from all the ‘difference’ images that the part of the subject that seems to suffer the most is the aluminium part, not the white areas.  The aluminium has a stippled texture causing a myriad of small specular highlights – brighter than the white parts of the subject.

What would 14 bit uncompressed minus 14 bit lossless minus photon flux look like?  In a perfect world I’d be able to show you accurately, but we don’t live in one of those so I can’t!

We can try it using the flux shot from earlier:

raw file compression

But this is wildly inaccurate as the flux component is not pertinent to the photons at the actual time the lossless compression shot was taken.  But the fact that you CAN see an image does HINT that there is a real difference between UNCOMPRESSED and LOSSLESS compression – in certain circumstances at least.

If you have never used a camera that offers the zero raw file compression option then basically what you’ve never had you never miss.  But as a Nikon shooter I shoot uncompressed all the time – 90% of the time I don’t need to, but it just saves me having to remember something when I do need the option.

raw file compression

Would this 1DX shot be served any better through UNCOMPRESSED raw recording?  Most likely NO – why?  Low Dynamic Range caused in the main by flat low contrast lighting means no deep dark shadows and nothing approaching a highlight.

I don’t see it as a costly option in terms of buffer capacity or on-board storage, and when it comes to processing I would much rather have a surfeit of sensor data rather than a lack of it – no matter how small that deficit might be.

Lossless raw file compression has NO positive effect on your images, and it’s sole purpose in life is to allow you to fit more shots on the storage media – that’s it pure and simple.  If you have the option to shoot uncompressed then do so, and buy a bigger card!

What pisses my off about Canon is that it would only take, I’m sure, a firmware upgrade to give the 1DX et al the ability to record with zero raw file compression – and, whether needed or not, it would stop miserable grumpy gits like me banging on about it!

Become a patron from as little as $1 per month, and help me produce more free content.

Patrons gain access to a variety of FREE rewards, discounts and bonuses.

6 thoughts on “Raw File Compression

  1. Thanks for this very useful information. Please could you re affirm for me when it would be appropriate to use a compressed raw file rather than the uncompressed one.Sorry to bother you this.
    Many thanks Cora

    • Hi Cora – no bother! As I said, personally I always shoot uncompressed on Nikon, and on Canon I have no option but to shoot lossless compressed because there is no uncompressed option.

  2. I have tried to understand everything you wrote Andy, but it’s probably a part of photography that ‘s going to fare for me sometimes ,it’s a bit like trying to understand how a computer ore camera works and I know we have to understand our camera’s almost inside and out ,….but having said that we both know that photographing birds at speed and ever changing angles and from different directions in a boat is not at all easy . What I am trying to say is that shadows and over exposed area’s is for most of us something we need to rectify in post processing and I agree the less we need to rectify the better but almost inevitable ,…..don’t know if I am losing the plot now and I do try to improve on the more technical side of photography and take better pictures,but God it sometimes get so bloody boring to keep up with these camera’s……….but I shall read your article again and see if it make’s sense to me second time around ,….thanks Andy and nice to share mutual friends and pictures,….regards Rudy Muts

    • Hi Rudy – I tell everyone I take to Oles that it is the hardest photography they will ever do – because it is.

      You’re a Nikon shooter like me – just shoot 14bit uncompressed and expose to the right a bit and you’ll get great exposures.

      Raw file compression serves no purpose except to increase card capacity – there is NO other benefit. If you want more frames per card then buy a bigger card.

      A Nikon D4 will shoot nearly 80 uncompressed frames before it hits the buffer and I never use that many in a burst because high frame rates cripple auto focus.

      • Thanks Andy ,I always shoot 14 bit uncompressed and I am also not a high burst frame shooter because I also found out that it kills the AF, so I am glad I learned something over these years ! Q, would you tackle Ole’s eagles with only a D 800 and D 810 ? and 80-400 vr II + 400 f2.8 vr ore risking loosing many shots with only these cameras, I have a D5 on order but can’t see me affording it before May,…they have the D4s on special here in Norway but it doesn’t seem right to buy that with D500 at half that price, if you have the time to answer I would appreciate your thoughts,regards Rudy

        • Hi Rudy

          D800 – crap camera as the low-pass is too thick, gimme the D800E anyday! D810 – epic camera!

          I had intended to shoot 400 f2.8 with the D800E and D4, but the lens arrived from the hire place in a f**ked up state two days before departure, hence I was stuck with gear I didn’t want to use – but overall it’s good gear to be lumbered with!!!!!!!!!!!!!!!!

          80-400 – mmmmmmm, I’ve used 4 or 5 examples of this lens and one impressed the heck out of me, the others I found a little crappy in various ways – QC isn’t brilliant on these lenses.

          I never bothered to upgrade my D4 for a 4S – no point.

          D5 – we’ll reserve judgement on that one until I’ve given a test one a good hammering – hopefully this September trip to Dovre plus a few outings with the eagles will answer all my questions.

          D500 – crop sensor plus too many pixels equals one thing Rudy – you’re heading into sensor diffraction territory, and the natives are hostile!

          All the high-speed bodies I’ve ever shot Oles eagles with – D3/3S/4 and 1DX – have always been slowed down to 5 or 6 fps to give the AF a fighting chance of keeping up with Brutus, Miguel and Luigi when they come hurtling at the camera head-on at 50mph and sub 20 metres distance. Basic physics will never change, so no matter what ‘improvements’ Nikon or Canon make to their predictive AF performance those improvements will never be as accurate as real-time data – and that real-time data ONLY comes from a slower frame rate, end of story….

          I fancy a D5 like crazy, but only because it offers the POTENTIAL of Nikon AF having caught up with Canon at long f**king last. BUT a. that’s only a POTENTIAL, and b. Canon MIGHT have moved the goal post with the 1DXMk2 (yes, the 1DX AF is not infallible, far from it.).

          So the whole “latest model” question is full of questions with no real answers – not until both the D5 and 1DX2 have been put head to head. Who’s the best person to come up with the definitive test for these two bodies – well I’m bound to say ME aren’t I, but I’ve made too many enemies with both Nikon and Canon for that to happen, unless a retailer decides to help out! Yes both manufacturers avoid me like poison! I can only put this down to the fact that I call a spade a spade and if something on a camera is crap then I say so!

          Personally Rudy, if I was you I would not spring for a D5 at the first opportunity – nor a 1DX2. ALL cameras are a bit questionable when they hit the market – they need a firmware upgrade or two to get them working right.

          If you run a D4 like me then I’d wait. If you are on a D3 or 3S then the D4S upgrade could be worth a go. But then again a D810 could definitely be the way to go now, and this could not be out-dated if you sprang for a D5 down the line once it had been out in ‘the jungle’ for 10-12 months.

          Just my thoughts Rudy!

Add your comments and feedback

This site uses Akismet to reduce spam. Learn how your comment data is processed.