Wednesday 2 July 2014

Pixels and the art of backup



My first digital camera had a massive 1.1 mega pixels(MP) sensor. Today with even low end mobile phones packing 5 MP this would now be considered pretty low resolution, But in the day I got some great photos out of it and it helped show me the power of digital cameras (up to that point I had been doubtful whether digital would ever overshadow film).

My present camera packs a 16.1 MP sensor and this is considered the low end for a DSLR. For example, my camera's successor has 20.1 MP sensor and even this for a camera is thought of as mid-range. For example at the top end the latest Nikon, the D810, packs a massive 36.3 Mega Pixels.

I bring this up because it seems every time a manufacturer produces a new camera, they feel the need to increase the pixel count. But the rarely asked question is why?  Take for example my cameras successor, the alpha 58. At the same time as they increased the pixel count, they reduced the frame rate. Is this a reasonable compromise?  Personally I am not convinced.  In the same way that most of today's PC's have a processor which is fast enough for 95% of tasks, I feel we are reaching the  point of max pixels. Basically we are at a point where any increase in pixel count will make little difference to your photos.

To put this in context, it has been calculated that at it's best old chemical film had a digital equivalent resolution of 20 MP. This means that even a mid-range camera will exceed the best film resolution. Remember this is film at it's best. Therefore increasing pixel count to a figure beyond has very little effect on what people will see on a photo in terms of detail. Remember also that the majority of photos will actually be reduced in size before display, so any gain in resolution will most likely be lost in post processing.

Of course there are some reasons why you may feel the need for more pixels. If you are in the habit of blowing up your photos to the size of a house,you may feel the more pixels the better. But remember such large images will likely only be examined from a distance, so any gain in resolution is likely to be moot.

The other reason often mentioned is to allow the cropping of a photo without losing resolution. However if you do such extensive cropping you are most likely to find all sort of artefacts as you zoom in. In reality a better lens is more likely to improve your resolution that a huge number of pixels. Don't believe me? Well read this guy take on it.

Then there are the downsides of having all those pixels. Depending on your sensor size, more pixels means packing more electronics onto the same size of silicon. This in turn means each pixel becomes more sensitive to noise. The only way around this is to make the sensors bigger, which in turn is more expensive. More pixels means that your frame rate is likely to suffer, which for pure landscape photographers may not be an issue, but for general .usage could be an important deficiency.

There is in fact a good argument that what actually is required is a larger sensor, containing fewer, not more, pixels. This is the path Sony have gone with the A7S which packs 12MP into a full frame sensor. 12MP does not seem a lot, but it does allow the camera to achieve almost unheard of low light capability with perfectly adequate resolution (See this review if you don't believe me).

So if we already reached the optimum level of pixels for camera, why do manufacturers continually pump out cameras with  greater and greater pixel counts? The answer is that pixel count provides an easy selling point when potential customers compare camera models. It is easier to compare 16 MP against 20MP models than things like low light performance or ease of use.

Another problem with a high pixel count is that is your photos are going to take up a lot of space when you download them to your computer. This in turn has a knock on effect when you go to back up your photos.

That is of course if you do back up your photos. Many don't and live continually with the risk of being only one hard disk failure away from losing all their irreplaceable memories, So unless you actually enjoy living on the edge, it is highly recommended that you have some sort of backup plan in place.

The problem is how best to do it.

Currently each of my cameras 16MP Raw files takes up approximately 16 MBytes of space on my disk. On a good day I can take easily 200 photos and when you upload them you suddenly see your disk space being eaten by up to 3 GBytes each time. Even on a 1 TB hard disk, you are going to quickly run out of space. OK, you can whittle your pictures down, but even so you will find that you are using a large amount of disk space just by storing your photos. This is only likely to get worse if the pixel count on your camera increases.

For a long time, I would religiously back up my photos onto cheap DVD's twice a year(hoping I had no failures between times). A DVD can take about 4.8 GB of files, but that is only about 300 photos worth, or one days photography. You can see from that, backing up 6 months work took a long time.

This would still of been acceptable if it wasn't for the fact that when I went back to recover some files from a backup DVD, I often found that after 4 years they were basically unreadable. DVD's, especially cheap writeable ones, are notorious for degrading. You can buy specialist archive DVD's, which are supposed to last 100 years. However these cost about £5 a pop and you will need a lot for a backup. Also the long term prognosis for these are only a estimate and it  may be difficult to locate the original manufacturer to get your money back if you find in 25 years time you can no longer access your photos(if DVD drives still even exist by then).

So what is required is a long term backup solution  that is convenient, has high capacity, low cost, is fast, and can secure your data for generations to come. DVD's obviously are out (Blue Ray for the same reasons), so what are the alternatives.

For a while I used to flash dongles to keep my data. These can be got in 32, 64 GB or greater sizes, are fast to copy to, easy to store and are not that expensive. However there is no real evidence that they will retain their data for a long period and it is likely that after a number of years of sitting on your shelf the data will be inaccessible.

Big corporations use tape as a their long term back up solution.  However tape  is slow to store and even slower to retrieve. Also there are few consumer solutions on the market.

One option is to invest in a RAID NAS server. This is a box containing 2 or more hard disks that sits on your network allowing you copy files to it. A properly configured RAID solution ensures that if one disk fails, the data is safe, copied or mirrored to the other disk. However there is a motto in computing that says "Raid is not backup". While generally reliable, RAID can fail ugly and should not be relied on for long term storage of your data.

Then there is always the cloud. There are loads of places online all happy to store your photos. These start  from the free, to more expensive solutions depending on facilities and quantity.  However two problems. Unless you have very fast upload speeds(and most don't) it will take ages to upload all those files. The second issue is, do you trust your life work to another company? What if it the company goes bust, or you cannot pay for the storage? Can you get the photos back or transfer them somewhere else? By all means use the cloud to store your files, but don't assume they will be safe forever,

So if you have managed to got to this far, you are probably guessing I have a solution for you.

Well here I have to disappoint you.

I do have a solution that is fast, local and uses technology known to retain information for considerable periods, but there is a gotcha.

What I do is buy cheap generic SATA hard drives and put them in these cardboard containers.  I backup my pictures to them using an USB adapter and place them in the cardboard box, and place them on a nice dark shelf somewhere. The price of a generic 500GB hard disk is quite low, especially in terms of GB per buck.

It sounds good, so whats the problem? Well no one really knows how long disks will last. They are pretty mature technology and therefore we have a good track record. Because we are only using them occasionally there will  not be a lot of wear and tear. However disks are designed  to be running continuously and there is some doubts whether a disk left on a shelf for a long time will start up again. One additional solution is to make duplicate copies of your photos from disk to disk (which is basically what Google does) but this can get complicated.

The truth is you do not know the true effectiveness of your backup solution until you try and recover your data. By then it maybe to late.

In reality, at present there is no perfect solution for backup that will guarantee your photos are secure for you and your descendants. Even if there was, there is no guarantee that such technology will be usable in the future. If you do not believe me try and read data of a 5 inch floppy disk.

There is only one long term solution that has stood the test of time, but it is expensive.

Get your photo printed. It maybe the only way that your descendants will actually get to see your  photos.

It will mean however that you will have to throw away a lot of pixels in the process

Addendum

I started a  thread on pixel count on  dpReview  which is actually quite an interesting read

Links found useful in my research

http://gizmodo.com/5155942/giz-explains-why-more-megapixels-isnt-always-more-better
http://news.cnet.com/More-megapixels,-better-photos-Fact-or-fiction/2100-1041_3-6156398.html
http://www.luminous-landscape.com/essays/full_frame_myth.shtml
http://www.mdavid.com.au/photography/megapixels.shtml




1 comment:

  1. There's a wonderful phrase that I've heard they will call this period... "the digital dark ages"

    It's a big problem that many seem to turn to online services to solve, but will they be available in 10 years time, let alone 50?

    I have something like 23 thousand family photos that I can't possibly print off, and in fact only a small percentage are worth the bother, but as taking more costs nothing, we keep more than we used to. As a sporadic keen photographer I overshoot, and ditch the rubbish, but it still results in many many photos.

    My strategy (also being a software developer) involved creating a web based solution so that our virtual albums can no only be organised, searchable and supplemented with titles, dates & descriptions, but it's just as easily accessed by our family members.

    I don't know what I will do with this long term but I like to think I have a while yet to think about it.

    ReplyDelete