A classic said that happy hours are not observed. In those wild times, there were neither programmers nor Unix, but nowadays programmers firmly know: instead of them, cron will track the time.
Command line utilities are both my weakness and routine. Sed, awk, wc, cut, and other old programs are run by scripts on our servers daily. Many of them are configured as cron jobs, a scheduler from the 70s.
I used cron superficially for a long time without delving into the details, but one day, when faced with an error while running a script, I decided to investigate thoroughly. Thus, this article was born, during the writing of which I familiarized myself with POSIX crontab, the main variants of cron in popular Linux distributions, and the structure of some of them.
Do you use Linux and run tasks in cron? Are you interested in the architecture of system applications in Unix? Then we are on the same path!
Content
The Origin of Species
Periodic execution of user or system programs is an obvious necessity in all operating systems. Therefore, the need for services that allow centralized scheduling and execution of tasks has long been recognized by programmers.
Unix-like operating systems trace their lineage back to Version 7 Unix, developed in the 1970s at Bell Labs by the famous Ken Thompson among others. Along with Version 7 Unix, cron was supplied, a service for the regular execution of tasks by the superuser.
A typical modern cron is a simple program, but the algorithm of the original version was even simpler: the service would wake up once a minute, read a table of tasks from a single file (/etc/lib/crontab), and execute for the superuser those tasks that needed to be performed in the current minute.
Subsequently, improved versions of the simple and useful service were shipped with all Unix-like operating systems.
General descriptions of the crontab format and basic principles of the utility's operation were included in the main standard for Unix-like operating systems — POSIX — in 1992, thus making cron from a de facto standard into a de jure standard.
In 1987, Paul Vixie surveyed Unix users for their wishes regarding cron and released another version of the daemon that fixed some issues with traditional cron and expanded the syntax of table files.
By the third version, Vixie cron met POSIX requirements, and the program had a liberal license—or rather, there was no license at all, except for the wishes stated in the README: the author provides no guarantees, the author's name cannot be removed, and the program can only be sold together with the source code. These requirements turned out to be compatible with the principles of free software, which was gaining popularity at the time, so some key Linux distributions that emerged in the early 90s adopted Vixie cron as their system cron and continue to develop it to this day.
In particular, Red Hat and SUSE have developed a fork of Vixie cron called cronie, while Debian and Ubuntu use the original edition of Vixie cron with many patches applied.
Let's start by getting acquainted with the user utility crontab described in POSIX, after which we will discuss the syntax extensions presented in Vixie cron and the use of variations of Vixie cron in popular Linux distributions. And finally, the cherry on top — an analysis of the cron daemon's structure.
POSIX crontab
Whereas the original cron only worked for the superuser, modern schedulers more often deal with tasks for regular users, which is safer and more convenient.
Cron comes in a set of two programs: a constantly running cron daemon and a user-accessible utility called crontab. The latter allows users to edit task tables specific to each user in the system, while the daemon runs tasks based on both user and system tables.
In does not describe the behavior of the daemon, and only the user program is formalized. . The existence of mechanisms to launch user tasks is implied, but not described in detail.
With the crontab utility, you can do four things: edit the user task table in an editor, load a table from a file, display the current task table, and clear the task table. Examples of how to use the crontab utility:
crontab -e # edit the task table
crontab -l # display the task table
crontab -r # remove the task table
crontab path/to/file.crontab # load the task table from a fileWhen invoked crontab -e The editor specified in the standard environment variable will be used. EDITOR.
The tasks themselves are described in the following format:
# строки-комментарии игнорируются
#
# задача, выполняемая ежеминутно
* * * * * /path/to/exec -a -b -c
# задача, выполняемая на 10-й минуте каждого часа
10 * * * * /path/to/exec -a -b -c
# задача, выполняемая на 10-й минуте второго часа каждого дня и использующая перенаправление стандартного потока вывода
10 2 * * * /path/to/exec -a -b -c > /tmp/cron-job-output.logThe first five fields of the entries: minutes [1..60], hours [0..23], days of the month [1..31], months [1..12], days of the week [0..6], where 0 is Sunday. The last, sixth field, is a string that will be executed by the standard command interpreter.
In the first five fields, values can be listed separated by commas:
# задача, выполняемая в первую и десятую минуты каждого часа
1,10 * * * * /path/to/exec -a -b -cOr separated by hyphens:
# задача, выполняемая в каждую из первых десяти минут каждого часа
0-9 * * * * /path/to/exec -a -b -cUser access to task scheduling is regulated in POSIX files cron.allow and cron.deny which list, respectively, users with access to crontab and users without access to the program. The location of these files is not standardized by any regulation.
According to the standard, at least four environment variables must be passed to the executable programs:
- HOME — the user's home directory.
- LOGNAME — the user's login name.
- PATH — the path where standard system utilities can be found.
- SHELL — the path to the command interpreter used.
Notably, POSIX does not specify where the values for these variables come from.
Best Seller — Vixie cron 3.0pl1
The common ancestor of popular cron variants is Vixie cron 3.0pl1, introduced in the comp.sources.unix mailing list in 1992. We will look in more detail at the main features of this version.
Vixie cron is provided in two programs (cron and crontab). As usual, the daemon is responsible for reading and executing tasks from the system task table and the individual user task tables, while the crontab utility is responsible for editing user tables.
Task tables and configuration files
The superuser's task table is located at /etc/crontab. The syntax of the system table corresponds to the syntax of Vixie cron with the addition that the sixth column specifies the username under which the task is run:
# Запускается ежеминутно от пользователя vlad
* * * * * vlad /path/to/execRegular user task tables are located in /var/cron/tabs/username and use the common syntax. When the crontab utility is run on behalf of the user, these files are specifically edited.
Management of user lists with access to crontab occurs in the files /var/cron/allow and /var/cron/deny, where it is sufficient to add the username on a separate line.
Extended Syntax
Compared to the POSIX crontab, Paul Vixie's solution contains several very useful modifications in the syntax of the task tables of the utility.
A new syntax for tables is now available: for example, you can specify days of the week or months by name (Mon, Tue, etc.):
# Запускается ежеминутно по понедельникам и вторникам в январе
* * * Jan Mon,Tue /path/to/execYou can specify a step interval for running tasks:
# Запускается с шагом в две минуты
*/2 * * * Mon,Tue /path/to/execSteps and intervals can be mixed:
# Запускается с шагом в две минуты в первых десять минут каждого часа
0-10/2 * * * * /path/to/execIntuitive alternatives to the regular syntax are supported (reboot, yearly, annually, monthly, weekly, daily, midnight, hourly):
# Запускается после перезагрузки системы
@reboot /exec/on/reboot
# Запускается раз в день
@daily /exec/daily
# Запускается раз в час
@hourly /exec/dailyTask execution environment
Vixie cron allows modification of the environment for the started applications.
The environment variables USER, LOGNAME, and HOME are not just provided by the daemon but are taken from the file . The PATH variable gets the value '/usr/bin:/bin', while SHELL is set to '/bin/sh'. The values of all variables, except LOGNAME, can be changed in user tables.
Some environment variables (primarily SHELL and HOME) are used by cron itself to run the task. Here’s how using bash instead of the standard sh for user tasks can look:
SHELL=/bin/bash
HOME=/tmp/
# exec will be launched by bash in /tmp/
* * * * * /path/to/execUltimately, all environment variables defined in the table (used by cron or required by the process) will be passed to the launched task.
To edit files with the crontab utility, the editor specified in the VISUAL or EDITOR environment variable is used. If these variables are not set in the environment where crontab was launched, '/usr/ucb/vi' is used (ucb likely stands for University of California, Berkeley).
cron in Debian and Ubuntu
Debian developers and derivative distributions have released of Vixie cron 3.0pl1. There are no differences in the syntax of the table files; for users, it’s the same Vixie cron. The major new features include support for , and .
Among the less noticeable but tangible changes are the location of configuration files and task tables.
User tables in Debian are located in the directory /var/spool/cron/crontabs, with the system table in the same old place — /etc/crontab. Debian package-specific task tables are placed in /etc/cron.d, from where the cron daemon reads them automatically. User access control is regulated by the /etc/cron.allow and /etc/cron.deny files.
The default command shell is still /bin/sh, which in Debian is represented by a small POSIX-compliant shell , launched without reading any configuration (in non-interactive mode).
Cron in the latest versions of Debian is run through systemd, and the startup configuration can be viewed in /lib/systemd/system/cron.service. There is nothing special in the service configuration, any more granular task management can be achieved through environment variables declared directly in each user's crontab.
Cronie in RedHat, Fedora, and CentOS
— is a fork of Vixie cron version 4.1. Like in Debian, the syntax has not changed, but support for PAM and SELinux, cluster operations, file monitoring using inotify, and other capabilities have been added.
The default configuration is located in standard places: the system table is in /etc/crontab, packages place their tables in /etc/cron.d, and user tables are found in /var/spool/cron/crontabs.
The daemon is managed by systemd, and the service configuration is located at /lib/systemd/system/crond.service.
In Red Hat-based distributions, /bin/sh is used by default upon startup, with the standard bash as its shell. It should be noted that when running cron tasks through /bin/sh, the bash shell is started in POSIX-compliant mode and does not read any additional configuration, operating in non-interactive mode.
cronie in SLES and openSUSE
The German distribution SLES and its derivative openSUSE also use cronie. The daemon is similarly started under systemd, with the service configuration located at /usr/lib/systemd/system/cron.service. The configuration files are at /etc/crontab, /etc/cron.d, and /var/spool/cron/tabs. The same bash, launched in POSIX-compliant non-interactive mode, serves as /bin/sh.
Structure of Vixie cron
Modern versions of cron have not changed radically compared to Vixie cron, but they have acquired new features that are not necessary for understanding the program's operational principles. Many of these extensions are poorly implemented and confuse the code. However, the original source code of cron as implemented by Paul Vixie is a pleasure to read.
Therefore, I decided to analyze the internal workings of cron using a common version from both branches of cron's development — Vixie cron 3.0pl1. I will simplify the examples by removing complicating ifdefs and omitting secondary details.
The operation of the daemon can be divided into several stages:
- Initialization of the program.
- Aggregating and updating the list of tasks to run.
- The main loop of cron.
- Launching a task.
Let’s examine them in order.
Initialization
When starting up, after checking the arguments, the cron process sets up the signal handlers SIGCHLD and SIGHUP. The first logs the termination of a child process, while the second closes the log file descriptor:
signal(SIGCHLD, sigchld_handler);
signal(SIGHUP, sighup_handler);The cron daemon in the system always runs as a single instance, only as the superuser and from the main cron directory. The following calls create a lock file with the daemon process's PID, ensure the user is correct, and change the current directory to the main one:
acquire_daemonlock(0);
set_cron_uid();
set_cron_cwd();A default path is set that will be used when starting processes:
setenv("PATH", _PATH_DEFPATH, 1);Next, the process is 'daemonized': a child copy of the process is created by calling fork, and a new session is established in the child process (calling setsid). There is no longer any need for the parent process — so it terminates:
switch (fork()) {
case -1:
/* critical error and termination */
exit(0);
break;
case 0:
/* child process */
(void) setsid();
break;
default:
/* parent process terminates */
_exit(0);
}
The termination of the parent process releases the lock on the lock file. Additionally, the PID in the file needs to be updated to the child's. After this, the task database is filled:
/* повторный захват лока */
acquire_daemonlock(0);
/* Заполнение БД */
database.head = NULL;
database.tail = NULL;
database.mtime = (time_t) 0;
load_database(&database);Next, cron moves to the main working loop. But before that, let's take a look at loading the task list.
Collecting and updating the task list
The function load_database is responsible for loading the task list. It checks the main system crontab and the user files directory. If the files and directory have not changed, the task list is not re-read. Otherwise, a new task list starts to be formed.
Loading the system file with special filenames and tables:
/* если файл системной таблицы изменился, перечитываем */
if (syscron_stat.st_mtime) {
process_crontab("root", "*system*",
SYSCRONTAB, &syscron_stat,
&new_db, old_db);
}Loading the user tables in a loop:
while (NULL != (dp = readdir(dir))) {
char fname[MAXNAMLEN+1],
tabname[MAXNAMLEN+1];
/* do not read files starting with a dot */
if (dp->d_name[0] == '.')
continue;
(void) strcpy(fname, dp->d_name);
sprintf(tabname, CRON_TAB(fname));
process_crontab(fname, fname, tabname,
&statbuf, &new_db, old_db);
}
After which the old database is replaced with the new one.
In the examples above, the call to the process_crontab function verifies the existence of the user corresponding to the table filename (unless it is the superuser), after which it calls load_user. The latter reads the file line by line:
while ((status = load_env(envstr, file)) >= OK) {
switch (status) {
case ERR:
free_user(u);
u = NULL;
goto done;
case FALSE:
e = load_entry(file, NULL, pw, envp);
if (e) {
e->next = u->crontab;
u->crontab = e;
}
break;
case TRUE:
envp = env_set(envp, envstr);
break;
}
}Here, either an environment variable (strings like VAR=value) is set by the functions load_env / env_set, or a task description (***** /path/to/exec) is read by the load_entry function.
The entity entry returned by load_entry is our task, placed in the shared task list. The function carries out a verbose parsing of the time format; we are more interested in the formation of environment variables and task launch parameters:
/* пользователь и группа для запуска задачи берутся из passwd*/
e->uid = pw->pw_uid;
e->gid = pw->pw_gid;
/* шелл по умолчанию (/bin/sh), если пользователь не указал другое */
e->envp = env_copy(envp);
if (!env_get("SHELL", e->envp)) {
sprintf(envstr, "SHELL=%s", _PATH_BSHELL);
e->envp = env_set(e->envp, envstr);
}
/* домашняя директория */
if (!env_get("HOME", e->envp)) {
sprintf(envstr, "HOME=%s", pw->pw_dir);
e->envp = env_set(e->envp, envstr);
}
/* путь для поиска программ */
if (!env_get("PATH", e->envp)) {
sprintf(envstr, "PATH=%s", _PATH_DEFPATH);
e->envp = env_set(e->envp, envstr);
}
/* имя пользовтеля всегда из passwd */
sprintf(envstr, "%s=%s", "LOGNAME", pw->pw_name);
e->envp = env_set(e->envp, envstr);The main loop operates with the current task list.
Main Loop
The original cron from Version 7 Unix worked very simply: it read the configuration in a loop, ran tasks scheduled for the current minute as the superuser, and slept until the start of the next minute. This simple approach required too many resources on older machines.
In SysV, an alternative version was proposed, where the daemon would sleep either until the nearest minute defined for a task or for 30 minutes. This consumed fewer resources for reading the configuration and checking tasks, but quickly updating the task list became inconvenient.
Vixie cron reverted to checking task lists every minute; fortunately, by the end of the 1980s, resources on standard Unix machines had significantly increased:
/* первичная загрузка задач */
load_database(&database);
/* запустить задачи, поставленные к выполнению после перезагрузки системы */
run_reboot_jobs(&database);
/* сделать TargetTime началом ближайшей минуты */
cron_sync();
while (TRUE) {
/* выполнить задачи, после чего спать до TargetTime с поправкой на время, потраченное на задачи */
cron_sleep();
/* перечитать конфигурацию */
load_database(&database);
/* собрать задачи для данной минуты */
cron_tick(&database);
/* перевести TargetTime на начало следующей минуты */
TargetTime += 60;
}
The execution of tasks is handled by the cron_sleep function, which calls job_runqueue (iterating over and launching tasks) and do_command (launching each individual task). The latter function deserves a closer examination.
Task Execution
The do_command function is implemented in good Unix style, meaning that for asynchronous task execution, it performs a fork. The parent process continues to launch tasks, while the child process prepares the task process:
switch (fork()) {
case -1:
/* failed to perform fork */
break;
case 0:
/* child process: just in case, try to acquire the main lock again */
acquire_daemonlock(1);
/* proceed to form the task process */
child_process(e, u);
/* upon completion, the child process exits */
_exit(OK_EXIT);
break;
default:
/* parent process continues working */
break;
}The child_process has quite a bit of logic: it takes the standard output and error streams on itself and then forwards them by email (if the MAILTO environment variable is specified in the task table), and finally, it waits for the completion of the main task process.
The task process is formed by another fork:
switch (vfork()) {
case -1:
/* If an error occurs, the process exits immediately */
exit(ERROR_EXIT);
case 0:
/* The child process creates a new session, terminal, etc. */
(void) setsid();
/*
* Further verbose settings for process output will be omitted for brevity
* /
/* Change directory, user, and group,
* meaning the process is no longer a superuser */
setgid(e->gid);
setuid(e->uid);
chdir(env_get("HOME", e->envp));
/* Start the actual command */
{
/* The SHELL environment variable indicates the interpreter to execute */
char *shell = env_get("SHELL", e->envp);
/* The process is started without passing the parent process's environment,
* just as described in the user's task table */
execle(shell, shell, "-c", e->cmd, (char *)0, e->envp);
/* Error — if the process did not start? Exit */
perror("execl");
_exit(ERROR_EXIT);
}
break;
default:
/* The main process continues its work: waits for completion and output */
break;
}That's basically the whole cron. I skipped some interesting details, like accounting for remote users, but the main points have been covered.
Afterword
Cron is surprisingly simple and useful software, done in the best traditions of the Unix world. It doesn't do anything extra, yet it performs its job excellently for several decades. Familiarizing myself with the code of the version shipped with Ubuntu took no more than an hour, and I gained a lot of pleasure from it! I hope I was able to share that with you.
I don't know about you, but it makes me a bit sad to realize that modern programming, with its tendency towards overcomplication and overabstraction, has long since moved away from such simplicity.
There are many modern alternatives to cron: systemd-timers allow for the organization of complex systems with dependencies, while fcron offers more flexibility in regulating task resource consumption. But personally, I've always found the simplest crontab sufficient.
In short, love Unix, use simple programs, and don't forget to read the manuals for your platform!
Source: habr.com
