{"id":34151,"date":"2019-10-31T21:56:41","date_gmt":"2019-10-31T18:56:41","guid":{"rendered":"https:\/\/prohoster.info\/blog\/hranenie-i-avtomaticheskaya-sortirovka-foto-i-drugih-fajlov-rabota-s-fajlohranilishhem-na-baze-nas-synology\/"},"modified":"2019-10-31T21:56:41","modified_gmt":"2019-10-31T18:56:41","slug":"hranenie-i-avtomaticheskaya-sortirovka-foto-i-drugih-fajlov-rabota-s-fajlohranilishhem-na-baze-nas-synology","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/hranenie-i-avtomaticheskaya-sortirovka-foto-i-drugih-fajlov-rabota-s-fajlohranilishhem-na-baze-nas-synology","title":{"rendered":"Storage and automatic sorting of photos and other files. Working with NAS Synology-based file storage","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

I have long wanted to write about how I store my files and make backups, but I just haven't gotten around to it. Recently, an article appeared that is somewhat similar to mine but with a different approach.
\nThe article itself.<\/a><\/noindex><\/p>\n

For many years, I have been trying to find the perfect method for file storage. I think I've found it, but there's always room for improvement. If you have any thoughts on how to do better, I'd love to hear them.<\/p>\n

I'll start by sharing a few words about myself: I am a web developer and in my spare time, I enjoy photography. This means that I need to store both work and personal projects, photos, videos, and other files.<\/p>\n

I have about 680 GB of files, of which approximately 90% are photos and videos.<\/p>\n

The cycle of files in my storage:<\/h2>\n

\n\"Storage
\n
\nHere's a rough diagram of how and where all my files are stored.
\n<\/a><\/noindex>
\nNow in more detail.<\/p>\n

As you can see, the heart of everything is my NAS, specifically the Synology DS214, one of the simplest NAS devices from Synology. Nevertheless, it handles everything I need.<\/p>\n

Dropbox<\/h3>\n

\nMy work machine is a MacBook Pro 13 from 2015. It has 512GB, but naturally, not all files fit there; I only store what I need at the moment. I sync all my personal files and folders with Dropbox. I know it's not very reliable, but it serves only the purpose of synchronization. And it does this best, at least compared to everything I've tried. I have tested all the well-known and not-so-well-known cloud services.<\/p>\n

Synology also has its own cloud service, which can be set up on your NAS. I have tried several times to switch from Dropbox to Synology Cloud Station, but there were always synchronization issues\u2014always some errors, or not all files synchronized.<\/p>\n

All important files are stored in the Dropbox folder. Sometimes, I save something to the desktop, and to avoid losing it, I created a symlink to the Dropbox folder using the MacDropAny program.
\nMy Download folder is not synced, but there's nothing important there, just temporary files. If I download something important, I copy it to the appropriate folder in Dropbox. <\/p>\n

My adventures with Dropbox<\/b>Once upon a time, back in 2013-2014, I stored all my files only in Dropbox, without any backups. At that time, I didn't have 1TB, meaning I didn't pay for it; I had about 25GB, which I earned by inviting friends or completing other tasks. <\/p>\n

One fine morning, I turned on my computer and all my files had disappeared. I also received an email from Dropbox apologizing for the loss of my files due to their error. They provided me with a link to restore my files, but of course, nothing was recovered. Instead, they gifted me 1TB for a year, and somehow, I became their client. As strange as it may sound, I never trusted them.<\/p>\n

As I mentioned earlier, I couldn't find a better cloud service for myself. Firstly, I had no synchronization issues, and secondly, many services only work with Dropbox.<\/p>\n

Git<\/h3>\n

\nWork files are stored on the work server; personal projects are on GitLab, it's all quite simple.<\/p>\n

Time Machine<\/h3>\n

\nI also back up the entire system, excluding the Dropbox folder and Downloads, of course, to save space. I back up using Time Machine, which is an excellent tool that has saved me more than once. I do this on the same NAS, as it has that function. You can also back up to an external HDD, but it\u2019s not as convenient. You have to connect the external disk each time and start Time Machine manually. Due to laziness, I often only did such backups once every few weeks. On the server, it automatically backs up, and I don't even notice when it does. I work from home, so I always have a fresh backup of my entire system. Backups are made several times a day; I haven't counted how often.<\/p>\n

NAS<\/h3>\n

\nThis is where the magic happens.<\/p>\n

Synology has an excellent tool called Cloud Sync, which, as the name suggests, does what it says.<\/p>\n

It can synchronize many cloud systems with each other, specifically, synchronizing files from the NAS server with other clouds. There's probably a review of this program online; I won\u2019t go into details. I'll better describe how I use it.<\/p>\n

\"Storage
\n
\nOn the server, I have a disk folder named Dropbox, which is a copy of my Dropbox account, and Cloud Sync takes care of the synchronization. If something happens to the files in Dropbox, it happens on the server as well, regardless of whether it's editing, deletion, or creation. In general, it\u2019s classic synchronization.<\/p>\n

Yandex Disk<\/h3>\n

\nNext, I upload all these files to my Yandex Disk, which I use as a homemade backup disk, meaning I upload files there but don\u2019t delete anything from it. It turns into a bit of a file dump, but it has saved me a couple of times.<\/p>\n

Google Drive<\/h3>\n

\nI only send the \"Photos\" folder there, also in sync mode. I do this solely for convenient viewing of photos in Google Photos with the option to delete them from there, and they get deleted everywhere (except Yandex Disk, of course). I'll write about the photos below; I could even write a separate article on that.<\/p>\n

HyperBackup<\/h3>\n

\nBut all of this isn't very reliable; if you accidentally delete a file, it gets deleted everywhere and can be considered lost. Of course, you can restore it from Yandex Disk, but firstly, a backup in one place isn't very reliable in itself, and secondly, Yandex Disk isn't a service you can trust 100%, even though I've never had problems with it.<\/p>\n

That's why I've always tried to keep files somewhere else, with a proper backup system in place.<\/p>\n

\"Storage
\n
\nSynology also has a tool for this called HyperBackup, which allows you to back up files either to other Synology servers or to some cloud solutions from third-party vendors.
\nIt can also back up to external drives connected to NAS, which I used to do until recently. But that's also not reliable; for example, if there's a fire, it's the end for both the server and the HDD.<\/p>\n

Synology C2<\/h3>\n

\nHere we smoothly transition to another service, from Synology itself. It has its own cloud for backup storage. It's specifically designed for HyperBackup, backing up data there every day, but it's a well-thought-out backup solution with file versions, a timeline, and even clients for Windows and macOS.<\/p>\n

\"Storage
\n
\nThat's all regarding file storage; I hope my files are safe.<\/p>\n

Now let's move on to file sorting. <\/p>\n

Regular files, books, scanned documents, and other minor files, I sort by hand into folders like everyone else. They typically aren't many, and I open them rarely.<\/p>\n

The most challenging part is sorting photos and videos, as I have a lot of them. <\/p>\n

I take anywhere from several dozen to several hundred photos a month. I shoot with a DSLR, a drone, and sometimes on my phone. The photos can be personal or for stock. I also occasionally shoot home videos (not what you might think\u2014just family videos, often with my daughter). They also need to be stored and sorted in a way that prevents chaos.<\/p>\n

I have a folder in the same Dropbox called Sort Images, which contains subfolders where all photos and videos go; from there, they are taken and sorted where needed.<\/p>\n

\"Storage
\n
\nSorting occurs on the NAS server, where bash scripts run automatically once a day to perform their tasks. The NAS is also responsible for executing these scripts; there is a task scheduler that manages the execution of all scripts and other tasks. You can configure how often and when tasks will run, using a cron interface for simplicity.<\/p>\n

\"Storage
\n
\nEach folder has its own script. Now let's talk in detail about the folders:<\/p>\n

Drone<\/b> \u2014 this folder contains photos from the drone that I took for personal purposes. First, I process all the photos in Lightroom, then I export JPGs to this folder. From there, they are moved to another folder in Dropbox, named \u2018Photo\u2019.<\/p>\n

There is a folder \u2018Drone\u2019 there, and they are sorted by year and month. The scripts automatically create the necessary folders and rename the photos according to my template, usually the date and time the photo was taken; I also add a random number at the end to avoid files with the same name. I don't remember why setting seconds in the file name didn't work for these purposes.<\/p>\n

\u0414\u0435\u0440\u0435\u0432\u043e \u0438\u043c\u0435\u0435\u0442 \u0442\u0430\u043a\u043e\u0439 \u0432\u0438\u0434: Photo\/Drone\/2019\/05 \u2014 May\/01 \u2014 May \u2014 2019_19.25.53_37.jpg<\/p>\n

\"Storage
\n
\nDrone Video<\/b> \u2014 I am currently not filming videos with the drone; there is a lot to learn, and I don\u2019t have time for that right now, but I have already created the folder.<\/p>\n

Images Operations<\/b> \u2014 this folder contains two internal folders; when files end up in there, they are either compressed to a maximum side of 2000px for publishing on the internet or images are flipped. I don\u2019t need that right now, but I haven\u2019t deleted the folder yet.<\/p>\n

Panoramas<\/b> \u2014 as the name suggests, this folder contains panoramas, which I keep separately since it's a specific type of photo, usually taken by the drone. I create standard panoramas as well as 360-degree panoramas and sometimes spheres, which look like little planets, also captured with the drone. All photos from this folder go to Photo\/Panoramas\/2019\/01 \u2014 May \u2014 2019_19.25.53_37.jpg. I don\u2019t sort them by month since there aren\u2019t that many panoramas.<\/p>\n

Personal Photo<\/b> \u2014 this is where photos taken with my DSLR go, usually family pictures or travel photos, in general, shots taken for memories and personal use. I also process the raw photos in Lightroom and then export them here.<\/p>\n

\u041e\u0442\u0441\u044e\u0434\u0430 \u043e\u043d\u0438 \u043f\u043e\u043f\u0430\u0434\u0430\u044e\u0442 \u0441\u044e\u0434\u0430: Photo\/2019\/05 \u2014 May\/01 \u2014 May \u2014 2019_19.25.53_37.jpg<\/p>\n

If I have filmed some event or something else that should be stored separately, I create a folder named after the event in the 2019 folder and copy the photos there manually.<\/p>\n

RAW<\/b> \u2014 this is where the source photos go. I always shoot in RAW, I store all my photos in JPG, but sometimes I want to keep the RAW files as well to process a shot differently. Usually, these are nature shots, and only the best ones are kept, not just any random ones.<\/p>\n

Stock Photo<\/b> \u2014 here I drop photos for stock that I take either with a DSLR or a drone. The sorting is the same as for other photos, just in a separate folder.<\/p>\n

In the root directory of Dropbox, there is a Camera Uploads folder, which is the default folder where the Dropbox mobile app uploads all photos and videos. All my wife's photos from her phone are uploaded this way. I also upload all my photos and videos from my phone here and from there I sort them into a separate folder. However, I do this in a different, more convenient way. There is a program for Android called FolderSync; it allows you to take all photos from your mobile, upload them to Dropbox, and then delete them from your phone. It has many settings, I recommend it. This folder also receives videos from the phone, which are sorted just like all the photos, by years and months.<\/p>\n

I gathered all the scripts myself from various online instructions; I didn't find any ready-made solutions. I really don't know anything about bash scripts; there might be some errors or improvements that could be made, but the most important thing \u2014 for me, they perform their job and do what I need.<\/p>\n

I uploaded the scripts to GitHub: https:\/\/github.com\/pelinoleg\/bash-scripts<\/a><\/noindex><\/p>\n

Previously, for sorting photos and videos, I used Hazel on macOS; everything is easier there, as all tasks are created visually without needing to write code. However, there are two downsides. First, I need to keep all folders on the computer for it to work well, and second, if I ever switch to Windows or Linux, there aren\u2019t any such programs. I tried looking for alternatives, but none of them were suitable. A server-based script solution is a more universal approach.<\/p>\n

All scripts are configured to run once a day, usually at night. But if you don't have time to wait and need to execute a necessary script now, there are two solutions: connect to the server via SSH and run the desired script, or log into the admin panel and manually start the script. I found this inconvenient, so I discovered a third solution. There is a program for Android that can send SSH commands. I created several commands, each with its own button, so now if I need to sort, for example, photos taken with a drone, I just press one button, and the script runs. The program is called SSHing; there are other similar ones, but for me, this one is the most convenient.<\/p>\n

\"Storage
\n
\nI also have several of my own websites; they are more for show, as hardly anyone visits them, but it's still good to make backups. I host the sites on DigitalOcean, where I installed the aaPanel. It has the option to back up all files and databases, but on the same disk.<\/p>\n

Storing the backup on the same disk isn\u2019t a good idea, so I also use a bash script to connect there and copy everything to my server, archiving everything into a single archive with the date in the name.<\/p>\n

I hope the methods I use and have shared help at least someone.<\/p>\n

As can be seen from the article, I love automation and try to automate everything I can; I didn't describe many things from the perspective of automation since those are already different topics and other articles.
\n
Source: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u0414\u0430\u0432\u043d\u043e \u0445\u043e\u0442\u0435\u043b \u043d\u0430\u043f\u0438\u0441\u0430\u0442\u044c \u043f\u0440\u043e \u0442\u043e \u043a\u0430\u043a \u044f \u0445\u0440\u0430\u043d\u044e \u0441\u0432\u043e\u0438 \u0444\u0430\u0439\u043b\u044b, \u043a\u0430\u043a \u0434\u0435\u043b\u0430\u044e \u0431\u044d\u043a\u0430\u043f\u044b, \u043d\u043e \u0432\u0441\u0435 \u043d\u0438\u043a\u0430\u043a \u043d\u0435 \u0434\u043e\u0445\u043e\u0434\u0438\u043b\u0438 \u0440\u0443\u043a\u0438. \u041d\u0435\u0434\u0430\u0432\u043d\u043e \u0442\u0443\u0442 \u043f\u043e\u044f\u0432\u0438\u043b\u0430\u0441\u044c \u0441\u0442\u0430\u0442\u044c\u044f, \u0432 \u0447\u0435\u043c-\u0442\u043e \u043f\u043e\u0445\u043e\u0436\u0443\u044e \u043d\u0430 \u043c\u043e\u044e \u043d\u043e \u0441 \u0434\u0440\u0443\u0433\u0438\u043c \u043f\u043e\u0434\u0445\u043e\u0434\u043e\u043c. \u0421\u0430\u043c\u0430 \u0441\u0442\u0430\u0442\u044c\u044f. \u042f \u0443\u0436\u0435 \u043c\u043d\u043e\u0433\u043e \u043b\u0435\u0442 \u043f\u044b\u0442\u0430\u044e\u0441\u044c \u043d\u0430\u0439\u0442\u0438 \u0438\u0434\u0435\u0430\u043b\u044c\u043d\u044b\u0439 \u0434\u043b\u044f \u0441\u0435\u0431\u044f \u043c\u0435\u0442\u043e\u0434 \u0434\u043b\u044f \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0444\u0430\u0439\u043b\u043e\u0432. \u0414\u0443\u043c\u0430\u044e \u044f \u0435\u0433\u043e \u043d\u0430\u0448\u0435\u043b, \u043d\u043e \u0432\u0441\u0435\u0433\u0434\u0430 \u0435\u0441\u0442\u044c \u0447\u0442\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":25771,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-34151","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Storage and automatic sorting of photos and other files. Working with a NAS Synology file storage | ProHoster","description":"I've wanted to write about how I store my files and make backups for a long time, but I just never got around to it.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/hranenie-i-avtomaticheskaya-sortirovka-foto-i-drugih-fajlov-rabota-s-fajlohranilishhem-na-baze-nas-synology","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0425\u0440\u0430\u043d\u0435\u043d\u0438\u0435 \u0438 \u0430\u0432\u0442\u043e\u043c\u0430\u0442\u0438\u0447\u0435\u0441\u043a\u0430\u044f \u0441\u043e\u0440\u0442\u0438\u0440\u043e\u0432\u043a\u0430 \u0444\u043e\u0442\u043e \u0438 \u0434\u0440\u0443\u0433\u0438\u0445 \u0444\u0430\u0439\u043b\u043e\u0432. \u0420\u0430\u0431\u043e\u0442\u0430 \u0441 \u0444\u0430\u0439\u043b\u043e\u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0435\u043c \u043d\u0430 \u0431\u0430\u0437\u0435 NAS Synology | ProHoster","og:description":"\u0414\u0430\u0432\u043d\u043e \u0445\u043e\u0442\u0435\u043b \u043d\u0430\u043f\u0438\u0441\u0430\u0442\u044c \u043f\u0440\u043e \u0442\u043e \u043a\u0430\u043a \u044f \u0445\u0440\u0430\u043d\u044e \u0441\u0432\u043e\u0438 \u0444\u0430\u0439\u043b\u044b, \u043a\u0430\u043a \u0434\u0435\u043b\u0430\u044e \u0431\u044d\u043a\u0430\u043f\u044b, \u043d\u043e \u0432\u0441\u0435 \u043d\u0438\u043a\u0430\u043a \u043d\u0435 \u0434\u043e\u0445\u043e\u0434\u0438\u043b\u0438 \u0440\u0443\u043a\u0438.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/hranenie-i-avtomaticheskaya-sortirovka-foto-i-drugih-fajlov-rabota-s-fajlohranilishhem-na-baze-nas-synology","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T18:56:41+00:00","article:modified_time":"2019-10-31T18:56:41+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"34151","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-21 18:07:19","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 02:27:30","updated":"2026-01-21 18:07:19","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/34151","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=34151"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/34151\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/25771"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=34151"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=34151"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=34151"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}