{"id":38928,"date":"2019-10-31T22:26:48","date_gmt":"2019-10-31T19:26:48","guid":{"rendered":"https:\/\/prohoster.info\/blog\/komanda-cp-pravilnoe-kopirovanie-papok-s-fajlami-v-nix\/"},"modified":"2019-10-31T22:26:48","modified_gmt":"2019-10-31T19:26:48","slug":"komanda-cp-pravilnoe-kopirovanie-papok-s-fajlami-v-nix","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/komanda-cp-pravilnoe-kopirovanie-papok-s-fajlami-v-nix","title":{"rendered":"cp team: correct copying of folders with files in *nix","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"cp
\n
\nThis article will reveal some less obvious aspects related to the use of wildcards<\/a><\/noindex> when copying, the ambiguous behavior of the command cp<\/code> when copying, as well as methods to correctly copy a large number of files without omissions or crashes.<\/p>\n

Let's say we need to copy everything from the folder \/source to the folder \/target.
\n<\/a><\/noindex>
\nThe first thing that comes to mind is:<\/p>\n

cp \/source\/* \/target<\/code><\/pre>\n
\nLet's immediately correct this command to:<\/p>\n
cp -a \/source\/* \/target<\/code><\/pre>\n
\n\u2014 some characteristic of the node (for example, a number). The key is needed to identify the element of the tree corresponding to this key. Example of a binary search tree: -a<\/code> will include copying all attributes, permissions, and will add recursion. When exact reproduction of permissions is not required, it is sufficient to use the key -r<\/code>.<\/p>\n
After copying, we will find that not all files were copied \u2014 files starting with a dot, like:<\/p>\n
.profile
\n.local
\n.mc<\/code>
\nand similar ones were ignored.<\/p>\n
Why did this happen?<\/p>\n
Because wildcards are processed by the shell (bash<\/code> in the typical case). By default, bash ignores all files starting with dots, as it treats them as hidden. To avoid this behavior, we need to change the behavior bash<\/code> using the command:<\/p>\n
shopt -s dotglob<\/code><\/pre>\n
\nTo preserve this behavior change after a reboot, you can create a file wildcard.sh with this command in the folder \/etc\/profile.d<\/code> (it may be a different folder in your distribution).<\/p>\n
And if there are no files in the source directory, then the shell will not be able to substitute anything for the asterisk, and the copying will also terminate with an error. To handle such a situation, there are the options failglob<\/code> and nullglob<\/code>. We will need to set failglob<\/code>, which will prevent the command from executing. nullglob<\/code> won't work, as it converts the string with wildcards that found no matches into an empty string (zero length), which for cp<\/code> will cause an error.<\/p>\n
However, if there are thousands of files or more in the folder, it's best to completely abandon the approach using wildcards. The thing is that bash<\/code> expands wildcards into a very long command line like:<\/p>\n
cp -a \/souce\/a \/source\/b \/source\/c \u2026\u2026 \/target<\/code><\/pre>\n
\nThere is a limit to the length of the command line, which we can find out using the command:<\/p>\n
getconf ARG_MAX<\/code><\/pre>\n
\nWe will get the maximum command line length in bytes:<\/p>\n
2097152<\/code><\/pre>\n
\nOr:<\/p>\n
xargs --show-limits<\/code><\/pre>\n
\nWe will get something like:<\/p>\n
\u2026.\nMaximum length of command we could actually use: 2089314\n\u2026.<\/code><\/pre>\n
\nSo, let's manage without wildcards at all.<\/p>\n
Let's just write<\/p>\n
cp -a \/source \/target<\/code><\/pre>\n
\nHere we encounter ambiguous behavior. cp<\/code>If the folder \/target does not exist, we will get what we need.<\/p>\n
However, if the target folder exists, the files will be copied into the \/target\/source folder.<\/p>\n
We can't always delete the \/target folder in advance, as it may contain necessary files, and our goal is, for example, to supplement the files in \/target with files from \/source.<\/p>\n
If the source and target folders were named the same, for example, if we copied from \/source to \/home\/source, we could use the command:<\/p>\n
cp -a \/source \/home<\/code><\/pre>\n
\nAnd after copying, the files in \/home\/source would be supplemented with files from \/source.<\/p>\n
Here\u2019s a logical puzzle: we can supplement files in the target directory if the folders are named the same, but if they are different, the source folder will be placed inside the target. How do we copy files from \/source to \/target using cp without wildcards?<\/p><\/blockquote>\n
\nTo bypass this harmful limitation, we use a non-obvious solution:<\/p>\n
cp -a \/source\/. \/target<\/code><\/pre>\n
\nThose who are well acquainted with DOS and Linux already understand: within each folder, there are 2 invisible folders «.» and «..», which are pseudo-folders linking to the current and parent directories.<\/p>\n