🥇How I Organized Photo Storage

Hello Habr! Each of us stores some kind of information; some use tricks and hacks for this. Personally, I love to press the shutter button, and today I would like to share my experience in information storage that I have worked on and refined.

Let me warn you right away: there is no 'silver bullet' under the cut that will eliminate the chaos in your files. There are no lines about neural networks, recognition of anything by anyone, or other nanotechnologies. What’s under the cut is a bit of text and a wooden table that will need to be filled out manually =) But it works.

Intro

Before understanding the problem I wanted to solve, let me briefly explain it =) I don't consider myself a full-fledged photographer, but still:

I have a camera and shoot in RAW (each photo weighs about 20-25 MB)
I faced the issue of storing and structuring photos (or rather their originals)

Now, let’s delve a bit deeper.

I use 1-2 memory cards of 64 GB each (not the ones in the photo below, although I know they have already caught your eye)) — buying larger cards (128-256) makes me feel uneasy. It's not so much the cost, as treating the card as a kind of consumable that could fail at any moment: I've lost cards, bent them, and once, someone simply stole one right out of the camera. Plus, putting 'all eggs in one basket' isn't the wisest approach.

Here's what happens when you forget to take the card out of your laptop, place it on the passenger seat, and suddenly brake. And I've tripped over this twice.

64 GB holds about 2000-2500 RAW photos. In my case, this translates to 4-6 event photo sets or about 10 'gadget' sets. Look at my previous publications to see why I have so many. Some might ask, 'why go through the trouble of pressing the shutter so much?' and they'd be right, but as I've said, I'm a bit of a newbie. Moreover, I have the bad habit of taking two shots — if the first one turns out blurry, perhaps the second will save the day. This instinct is hardwired into me, and I haven’t figured out how to change it. This explains why I shoot in RAW — simply to fix my own mistakes later, such as overexposure, underexposure, and other geometric misalignments.

The Problem

For a long time, I couldn't find a program that would fully meet my data storage needs. There are catalogers, convenient work with meta-tags, face recognition, and mapping photos — a whole bunch of cool features, but... scattered across different applications. Let me list a few pitfalls that almost all applications stumble upon.

Problem number 1: here lies a memory card — what’s on it? You never know. Of course, you could scroll through 2000 photos on your camera, plug it into your laptop, or take notes on your smartphone, but that won’t give you the 'big picture'. And it won't answer the question, 'have I already backed up this data, or can I delete it permanently?' if, for example, I urgently need to free up space? After all, a spare 64 GB might not be available.

Problem number 2: you never know the condition of the photos. Are they sorted? Edited? Can I delete them, or should I first transfer them to the computer? You're familiar with those endless folders like 'From SD', 'SD64 LAST', '!UNSORTED', '2018 ALL', 'iPhone_before_update', and so on, right? =) On the laptop, on the memory card, on the external drive, with tons of duplicates? And that oppressive feeling of, 'I should really get this all organized — I'll do it when I have free weekends...' But those free weekends never seem to come.

Problem number 3: how can I quickly find the right photos? For instance, I recently needed to make a collage of all 'first days of September' from several years. Store them on the laptop? It won't fit. Sifting through different drives? Well, that's one option. But that's so inconvenient...

I would greatly appreciate it if you could suggest a more functional and flexible option than what I’ve come up with through trial and error (below). To clarify, I’m not talking about a photo viewer/sorter, but rather about convenience/clarity/informativeness.

Solution

I've decided to use a cool tool like Google Docs spreadsheets =) It's free, cross-platform, and etc. — I think it requires no introduction. Before creating the structure of the spreadsheet, I tried to understand what fields I needed. You can come up with hundreds, but it should be convenient to use and not tedious to fill out every time. Plus, I aimed to consider future scaling: so that it's easy to use the spreadsheet in a year, two, or three.

I focused my thoughts on the following set of fields:

CategoryI analyzed what I photograph and divided it into categories. It turned out like this:
Cars — cars
Events — events
Gadgets — gadgets
Girls — you know what I mean
Home — something domestic, family-related
Life — any activity that doesn't fit into the categories above
— =)
Travel — travels

All photo sets will be sorted into these sections. If you take a lot of photos, it’s easier to keep each section on a separate sheet (at the bottom of the table).

Important: try to avoid creating a category called 'Other,' as this is where chaos will arise and collapse the universe. At most, use '!Temp' to stash files for further sorting into other categories.
Title. Within each category, every photo set has a title — it’s important to give names that are easy to remember or find. There are two convenient options: alphabetical or chronological order. I alternate both options: in gadgets, it's easier to use device names, while in events — a title like '2018-03-08 — International Women's Day.' If needed, there's always CMD+F.
Where now. In this column, I indicate where the photos are stored — on the camera's memory card, on the laptop, on an external drive, or in the cloud. If the data's location changes, the table is updated. It's important to provide information about the photo set right away; otherwise, it will be forgotten later.
Number before sorting. It's not always possible (actually, it's never possible) to sort gigabytes of RAW files immediately; usually, you just dump them from the memory card. Here, it's crucial to understand how many photos are in the photo set — to roughly estimate how much time it will take to sort and edit.
Life hack: it will be helpful to know the average speed of sorting and editing photos, if you are into that at all. Just set a timer for 5-10 minutes and then see how many you managed to improve. I typically take about 2-5 minutes per photo (assuming I know the Photoshop hotkeys well). See point 8.
Sorting and processing. Just two columns, where the cells are colored either green (= 'Done') or red (= 'Not Done'). You can also add, for example, blue — if processing is not required. Such a color legend will clearly show what is in which state. Optionally, you can display numbers — the speed of work multiplied by the number of photos after sorting (see point 11).
By sorting, I mean selecting the best frames (removing duplicates and defects) for further processing, and by processing itself — their path from RAW to JPEG (the one that you wouldn't be ashamed to show others). In the future, each folder will contain the processed JPEGs, while the subfolder 'Originals' will hold the RAW files and *.xmp files.
Copy in the cloud. Usually, there's no point in uploading unsorted piles of photos to the cloud; it's a waste of time and space. It's reasonable to back up already sorted photos there. Better yet — those that have already been processed. If I upload files to the cloud, I create a clickable link to the folder — so I can jump to the right place from the table in one click, rather than rummaging through an online file manager (which usually lags).
Copy on disk. Clouds are considered reliable, but something inside tells me that it's better to have a backup locally as well (at least for particularly important data). Or if it concerns any 'sensitive' data that I wouldn't want to upload to the internet.
Quantity, size. The number of photos after sorting, as well as the size of the space they occupy. This column is optional, but I'll try to explain why I created it.
If I see a photoshoot with a green cell labeled 'Sorting' and a red one for 'Processing'—it means I just need some free time for quite dull and repetitive mechanical work. Knowing the number and size of the photos, I can schedule this task. For example, next weekend, I need to travel from Moscow to St. Petersburg and back on the 'Sapsan', which means I know I will have my laptop and 8 hours without stable internet (= excellent conditions for processing photos). I roughly estimate how many photos we can process in that time and upload the necessary photo sets to the laptop. It helps to know at least the approximate processing speed for one photo. It takes me from 2 to 5 minutes per photo, and with 8 hours being 480 minutes, there's little point in copying more than 300 photos onto the laptop (which amounts to about 6 to 9 GB). My MacBook has a 256 GB disk, so sometimes I have to 'play 15-puzzle', but with the table, the total size of the photo sets is never a surprise for me.

And then I simply need to arrive early at the train station to grab a table in the restaurant car =)
Shoot Date. It's an important parameter closely related to the next column.
On the phone. It often happens that alongside the main camera, you also need to capture something on the phone. For instance, if you're photographing a dynamic scene (like a race) and ask a friend to shoot a video. Or if you're doing renovations and have dirty hands—it's not appealing to pull out the camera, but capturing it on your phone is just right. As a result—right now I have 25,000 photos on my 128-gig iPhone. Yes, there's a lot of nonsense, but also a sufficient amount of needed ones.
To ensure important phone photos don't lead a separate life, it’s better to add them to the thematic photo set folder. Searching by date makes it fastest to find what you need (although geotags help significantly here too). If there’s a 'Yes' mark on the phone—it means I need to separately upload the photos from the phone. If 'No'—it means either they didn’t exist or have already been uploaded.
FlawThis column may not be necessary for you, but I've decided to keep it for myself. It shows the percentage of rejects from the photo set that I delete — on average, it's 50%. As I mentioned, my problem is that I make duplicate shots. Overall, I don't see anything wrong with it; I'm not worried about the shutter count =) but it still irritates me a bit, which I notice every time I open the spreadsheet, and I always think, "I need to learn to take better photos, improve my knowledge and skills." One day I'll get fed up and deal with it!
Draft and postIn case I need to write something about the subject being photographed (for example, a device review, of which I've had quite a few in my profile), I first create a draft in Google Docs, linking it to the word "Here." Green means the draft is finished, yellow means in progress, and red means I haven't started it yet. The same applies to posts — adding a link to a post allows me to jump to the right one with a single click, without any Google searches.
It's immediately clear what the state of all publications is and what the size of the "technical debt" is.

Clickable:

So, this is the table I came up with =) It's quite extensive, but I made it for myself. If you like my way of thinking, feel free to adapt it to your own needs — add or remove as necessary.

Optionally, you can sum up the total weight of all photo sets and calculate the percentage of space occupied on a storage device of known capacity (a sort of progress bar).

Speaking of storage devices.

Initially, I only stored files on my laptop, but I quickly ran out of space. I bought a 2.5" external drive — it soon died due to my negligence, as I constantly carried it in my backpack and didn't protect it one day.

I decided to try Yandex.Disk and purchased 1TB — overall it's quite convenient, but there are many inconveniences as well: upload and download speeds, cost, privacy (what if some beta version of a new algorithm deems my photos unacceptable and deletes the entire account?) and much more.

Therefore, I ultimately settled on a hybrid version: I took two stationary disks and kept my subscription to Yandex.Disk active as a transit point and spare wheel. The 'non-sensitive' data that may potentially be needed in the foreseeable future goes to the cloud — for example, photographs of the device I'm going to write about, or photos from children's events that I need to share with other parents (having a DSLR automatically condemns you to this function at kindergarten and school). Everything that doesn't belong in the cloud is on the disks.

At the beginning of the year, I chose two 3.5″ Seagate Ironwolf drives as stationary storage — a series of disks specifically for NAS systems. This line has models ranging from 1 to 14 TB — 1 and 2 TB are not serious, while 6 and more are quite expensive. I settled on the 4 TB model — at first, I thought I'd create a JBOD of 8 TB, but then I calculated and realized I haven't photographed that much yet =) So, in the end, I configured them in RAID 1 — just to be sure I won't regret it. The disks run at 5900 RPM, so they're quiet, don’t heat up much, and the speed is more than acceptable (although I didn’t make an exact measurement).

1 TB on Yandex.Disk costs 2000 ₽ per year, so 4 TB will be an annual 8K (life hack: if you have a Yandex.Plus subscription for 1500 a year, you get a 30% discount on Yandex.Disk), one of the advantages is that you can add space in a couple of clicks. The Seagate Ironwolf 4 TB drives are priced at 7K each (I managed to snag them for 6), but once you buy them, you just set them up and forget about them — they can autonomously hum away somewhere in a closet and not ask for money every year.

Out of curiosity, I checked the rates at Oblako@mail.ru — 1 TB costs from 699 ₽ per month! ) That makes it 8400 a year. 4 TB — from 2690 ₽ a month (32K a year).

I currently have more than enough space with 4 TB for photos, but if you're into video editing, it might not be enough. In general, calculate according to your own needs =)

An important point to consider in the calculations. Recently, I spoke with two wedding photographers — they mentioned that they try to send the photos to the client within a month (that includes retouching). Then they keep the photos for another couple of months, after which they ruthlessly delete them, leaving only a few photos from each shoot for their portfolio (and originals in case they need to prove something, both have had that happen). At first, I considered such an approach: "Hmm, maybe forget about it?! After all, why keep all those photos of other people's weddings and gadgets if you're never going to look at them?». Waiting for the magical “what if they come in handy”? If you haven't had such an adventure in the past year, trust me, it won’t come in handy. But then I thought, there is indeed a difference between personal and someone else's — yes, family photos and videos you won't watch now, but it will be very nice to look back in 5-10-15 years. And that's when you realize that it's better to stock up on free space.

Browser life hack

I use Chrome and it has a convenient bookmarks panel (CMD+Shift+B). Let's create a bookmark folder for files and rename it, giving it the name:

(phew, Habr doesn't support emoji, so I had to insert it as an image). If you have a lot of bookmarks, you can use a separator; I like this one — “⬝”. It looks like this:

The end

I've been using this table for about six months and I generally like everything about it; I've gotten used to filling it out while files are copying. So I suggest not wasting time trying to convince me otherwise =) But at the same time, I understand that it's from the stone age and there are certainly (not maybe, but definitely!) many things that can be improved or automated (for which more knowledge and time are needed). Collective intelligence, let’s think together about how we can improve/redesign/optimize all this to achieve better results with minimal effort? Any suggestions are welcome.

Or maybe you have your own file storage hacks — share them.

I hope this was helpful =) Good luck!

Only registered users can participate in the survey. Please log in, please.