Here they periodically write posts about how they store and back up their photos - well, just files. In the last such post, I wrote a rather long comment, thought a little and decided to expand it into a post. Moreover, I have somewhat changed the backup method to the cloud, it may be useful to someone.
The home server where much of the following happens:
What should be preserved?
The most important and voluminous for me are photographs. Occasionally video, but very occasionally - it takes up too much space and takes too much time, because I donβt like it too much, I shoot only short videos that lie in the same pile as photos. At the moment, my photo archive occupies about 1,6 terabytes and is growing at about 200 gigabytes per year. Other important things are much less voluminous and with them there are fewer questions in terms of storage and backup, a dozen or two gigabytes can be crammed into a bunch of free or very cheap places, ranging from DVD to flash drives and clouds.
How is it stored and backed up?
My entire photo archive is about 1,6 terabytes at the moment. The master copy is stored on a XNUMX terabyte SSD in the home computer. On memory cards, I try not to keep photos longer than necessary, I merge my desktop or laptop as soon as possible (when I'm on the road). Although I do not delete it from the flash drive, if there is still space. An extra copy never hurts. From the laptop, upon arrival home, everything is also thrown off to the desktop.
A copy of the folder with photos is made daily to the home server (with a type of mirror based on Drivepool, where duplication of important folders is configured). By the way, I still recommend Drivepool - for all the years of use, not a single glitch. It just works. You don't just need to look at its Russian interface, I sent a more decent translation to the developers, but I don't know when it will be implemented. In the meantime, in Russian, this is a program for managing the pool (manage pool).
You can, of course, make copies more often, if a lot of work has been done in a day, then I can force the task to run. Although now Iβm still thinking about starting copying by changing files, I want to stop keeping the desktop turned on around the clock, let the server work more. The program is GoodSync.
Until recently, from the same desktop, the same GoodSync files were uploaded to the Onedrive cloud. Most of my files are not personal, so I uploaded them as is, without encryption. What was personal was uploaded as a separate task, with encryption.
Onedrive was chosen because the 365 Office 2000 Home Premium subscription provided five (now six) terabytes of cloud space per year. Let and pieces on a terabyte. Now, however, the freebie has risen in price a little, but a few weeks ago there was another option for 2600-2700 per year (you have to look for retailers). I foresaw this when last year MS raised prices, and even stopped selling subscriptions on the site, so I activated the subscription for five years in advance while there were still 1800-2000 boxes on sale (of course, there were also several boxes in reserve take, but so I did not dare to guess).
The download speed is the maximum for my tariff, 4-5 megabytes / sec., at night up to 10. At one time I looked at crashplan - itβs good there if the megabytes per second were loaded.
Lifetime 5TB for $2-3 from ebay is a very random thing. For the life span can be very short, so far three months is a record. It's not a matter of backing up to a place that can be covered at any moment. Even for pennies.
But now, due to the fact that I decided to drag some of the tasks from the desktop to the server, I transferred copying to Onedrive to Duplicati. Although it is a beta, I have been using it for several months and so far it works quite stably. Since Duplicati still stores its backups in archives, and not in bulk, I decided to encrypt everything downloaded with built-in tools. All the same, if anything, you will have to restore it through Duplicati. So let's encrypt everything.
Considering that I have terabytes in pieces, a backup to the cloud consists of several tasks. This is where the backup is being uploaded to the cloud. 2019 filled up quickly - there are fifty photos in a couple of days, I havenβt traveled much yet, and 2018 is slowly pouring. The current download speed is not the maximum - a day, the channels are loaded and all that.
In the cloud, the backup folder looks like this - a lot of zip archives, the size of the archive is configured when creating the task:
About once a month I make a copy on an external drive, which is stored in a closet. I connect and manually start the task with the same GoodSync. Although, of course, you can also set up a launch upon connecting a disk, but I donβt always need to make a copy when I connect a disk.
For good, one more remote storage location would be needed - its own and not very cloudy. On my server, which is on the provider's site, I have long prepared a disk for this business, but all my hands do not reach. But since I started dragging everything under duplicati, I think I will do it now, after I re-upload everything to onedrive.
How is it cataloged?
Here the question is divided into two - the file system level, where cataloging takes place at the folder level and logical cataloging by more parameters, because the folder tree is still limited in its capabilities.
Yes, I take pictures. Because you can make jpg from raw at any time, but not vice versa. Once I shot in raw + jpg, so that I could quickly transfer the photo to the phone and send it to the Internet (it was difficult to transfer raw to the phone). jpg then erased when copying to the desktop. But now the phone began to suit me in terms of photo quality (for uploading to the Internet), so I completely refused jpg on cameras. Remained either from those times when I did not have a mirrorless camera, or they come from the phone.
At the file system level, it looks something like this: at the top level of folders - the source. The names of the photographers are usually.
One level down are topics. Everyone is more or less the same, there may be personal topics (for example, βDogsβ, there may not be any topics.
Next is the year. Inside the year folder by day. There may be separate photo shoots in the folder if the photos for the day are divided into topics.
As a result, the path to the file may look something like this: MyTravel20182018-04-11 BerlinFrench StationP4110029.ORF
I take pictures with two cameras, usually in turn, but occasionally I take both with me - then I dump the photos from them in one folder. The main thing is that the time is synchronized, otherwise you then have to calculate the difference and adjust the shooting date for all files (in Lightroom itβs easy, but itβs tedious to calculate the time difference).
There is a separate folder on the second level for photos from the phone, but if necessary, the photo can be sent to the thematic folder.
Logical cataloging on top of folders β Adobe Lightroom. Of course, there are a lot of programs for cataloging and processing, but lightroom suits me, it costs quite well (and they even give Photoshop in the kit), and over the past couple of years it has also become less slow. Although, of course, there is also a complete transition to the SSD helped.
All photos live in one directory. The folder structure from the previous paragraph is used as a base, on top of it - EXIF ββinformation, geotags, tags and color marks. You can also enable face recognition, but I do not use it.
Based on all of the above, you can create "smart collections" - dynamic selections for certain file properties - from shooting parameters to text in comments.
All tags are stored in files, the editing history is in XMP files next to the raves. The Lightroom catalog is backed up by means of the Lightroom itself once a week to a certain folder, from where it is then uploaded to OneDrive. Well, plus, through the veeam agent, the desktop system disk is uploaded to the server every day - and the catalog is stored on the system disk.
And what about the photo? What, there are no other types of files?
There is why not. Backup methods do not differ (if you need to backup at all), and cataloging methods depend on the type of content.
Basically sorting at the folder level is enough, tags are not needed. Only for movies and TV series a separate cataloger is used. β Plex Media Server. It is also a media server, as the name suggests. But there the horse was not lying around, normally sorted well if a quarter of the film library, and the rest is lying around in the β! To sortβ folder.
Source: habr.com