RAW Files Explained In-depth

RAW files

Simply put, a RAW file represents what the camera sensor captured during the exposure of the image (like an undeveloped negative in the days of film).  Having the RAW file gives the user maximum freedom to decide how the image should look later when using a RAW converter to convert the RAW file to a JPG or TIF file.

For example, an under exposed image with wrong White Balance can be corrected perfectly. The only limitation of RAW is if the image is way to much overexposed. In that case the RAW file will not contain any information in the highlights about the object photographed.

What is a RAW file?

In order to understand in more depth what a RAW file is we need to look at how a digital image is created in the camera.

During the exposure of an image, light hits the sensor which consist of a large array of light sensitive pixels arranged in a grid.  There are mainly 2 different sensor technologies used in digital capture – CCD sensors and CMOS sensors.  Both technologies can produce top quality images. It is not the technology that determines the quality but how it has been designed and manufactured.  Common for both types of sensor is that in order for the sensor to record color, the pixels are divided in green, red and blue pixels. This means that some pixels in the sensor only see Green light (G pixels), some only see Red light (R pixels) and some only see Blue light (B pixels). By far the most common pattern for organizing the R, G and B pixels is the Bayer pattern.  For each 2×2 pixels in the image sensor there will be 2 G pixels in diagonal and 1 B and 1 R pixel.

This arrangement is known as the Bayer pattern.

A/D conversion

After exposure each pixel holds an analogue electrical value representing the amount of light reaching that pixel position.  These electrical values need to be converted to digital values readable by the internal camera computer.  This process is called Analogue to Digitial conversion (or A/D conversion).  The precision of the conversion is limited by the number of bits the A/D conversion provides. In the very beginning of consumer digital photography the cameras only contained 8 converters.  With 8 bit you could get 2 to the exponent of 8 = 2x2x2x2x2x2x2x2 = 256 different levels.  Today the best cameras have 16 bit A/D conversion giving 2 to the exponent of 16 = 65536 different levels.  Taking into account that the human eye only can distinguish between approximately 300 different levels of gray it may sound like way to many levels. However, when the image is processed and deep shadow details need to be visible then it becomes extremely important to have all the levels possible available.

Correcting Sensor Defects

We now have the digital RAW signal from the sensor but sensors consisting of millions of pixels are never perfect.  There will be lots of so-called “dead pixels” – pixels that do not react on light and other manufacturing defects. In order to reconstruct an image from the RAW data, all the dead pixels and other imperfections need to be fixed.  When the camera is manufactured all the sensor imperfections are mapped out in a calibration file. The camera can now correct the RAW data or add a description of what need to be corrected to the RAW data.

Different RAW file formats

At this point we have all the data ready for making a RAW file.  In principle the camera could now save the RAW data to a CF or SD memory card in a unified RAW file format. During the development of digital capture the different camera manufactures started making cameras with unique RAW files, the option of a universal format didn’t exist.

As a consequence of this each camera manufacturer uses their own file formats today.  For example Canon‘s RAW files are named .cr2. Nikons RAW files are named .nef and Phase One’s RAW files are named .iiq or .tif. – even a standard tif file can contain RAW file data!  For the Phase One RAW file all the RAW file information is kept in a special private tag that cannot be read by a normal TIF reader. Only the thumbnail is in the normal part of the .tif file.  Both Phase One and Canon started making RAW files with the .tif extension, the benefit of which meant that the user could see the contents of the RAW file in the OS file system. For many other RAW file formats the contents of the RAW file can only be displayed if the user installs specific SW components and once you moved the file to another computer you had to make sure that the same SW component was installed.

RAW file compression

As the RAW file represents all the data with a giving bit depth that comes from the image sensor it may be a large amount of data.  As an example a Canon 5D MKII RAW file comes from a sensor of 5616 x 3744 pixels with a bit depth of 14 bit.  This gives RAW data of 5616 x 3744 x 14 / 8 bytes = 36MB.  When looking at a real RAW file from the Canon 5D MKII you will see that it is only about 29MB.  The trick here is to compress the data.  It is possible to compress totally lossless without loosing any information. As with the Music industry’s MP3 format, it is also possible to do a similar lossless compression with picture data. The signal may be changed but changed in a way that makes it impossible to distinguish it from the original.  The different camera manufacturers use different methods of compression of their RAW files but having a fast compression with a good compression ratio whilst preserving all the data is essential for being able to shoot fast for longer periods. RAW file formats from Phase One, Leaf, Canon and Nikon all provide this.