Cron is a standard scheduler for Unix servers. There are several cron implementations. All
follow the principle of fire & forget. They have no ability to monitor jobs success/failures
and to provide dependency-based scheduling. The only thing they can do is to mail the output to the
user under which job was invoked (typically root). Along with calendar based invocation, cron has basic
batch facilities (see batch command ), but
grid engine which is also open source component of Unix/Linux
provides much more sophisticated and powerful batch environment. Batch command is actually an example
of dependency based scheduling as command is executed when load of the server falls below specified
for atd daemon threshold (1.5 by default). There are several queues that you can populate and
each queue can be populated with multiple jobs which will be executed sequentially, one after another.
See batch command for more details.
Absence of dependency-based scheduling is less of a limitation then one might think. A simple status-based
dependency mechanism is easy to implement in shell. In this case the start of each job can depend of
existence of a particular "status-file" that was created (or not, in case of failure) during some previous
step. Those status-files can survive reboot or crash of the server.
With at command the same idea can be implemented by cancelling the
execution of next scheduled step from a central location using ssh . In this case ssh
can serve as central scheduling daemon and at daemon as local agent. With the universal adoption
of ssh daemon remote scheduling is as easy as local. For example, if a backup job fails it
did not create the success file, then each job dependent on it should checks the existence of the file
and exit if the file is not found. In more general way one can implement script "envelope" -- a special
script that creates status file and send messages at the beginning and at the end of each step to the
monitoring system of your choice. Using at commands allow you to cancel or move to a different
time all at commands dependent on successful completion of the backup.
Another weakness of cron is that its calendaring function is not sophisticated enough to
understand national holidays, furlough, closure, maintenance periods, plants shut downs, etc. Avoiding
running workload on holidays or specific days (e.g. an inventory day) is relatively easy to implement
via standard envelope which performs such checks. Additionally you can store all the commands with parameters
in /root/cronrun.d or similar directory and run checks within them via functions or external
scripts. You can also specify in cron command /usr/local/bin/run with appropriate
parameters (the first parameter is the name of script command to run, everything parameters passed to
the script). For example:
@dayly /usr/local/bin/run tar cvzf /var/bak/etc`date%y%m%d`.tgz
@daily /root/cronrun/backup_image
You can also prefix command with the "checking script" (let's name in canrun) script,
and exit, if it returns "false" value. For example
@daily/usr/local/bin/run && tar cvzf /var/bak/etc`date%y%m%d`.tgz
Another, more flexible, but slightly more complex way way is to use the concept of "scheduling day"
with particular start time and length (in some cases it is beneficial to limit length of scheduling
day to just one shift or extend it to 48 hours period). In this case the "start of scheduling day"
special script filters a sequence of at commands that is scheduled that day and then propagates
this script to all servers that need such commands via ssh or some
parallel
command execution command. In a simplest case you can generate such a sequence
using cron itself, when all running command are prefixed by echo command that generated
at command with the appropriate parameter and redirects output to "the schedule of
the day" file. For example
This allows to centralize all such the checks on a single server -- mothership.
After this one time "feed" servers become autonomous and execute the generated sequence of jobs,
providing built-in redundancy mechanism based on independence of local cron/at daemons from
the scheduler on the "mothership" node. Failure of the central server (aka mothership) does not affect
execution of jobs of satellite servers until the current scheduling day ends.
Due to day light saving time changes it is prudent not to schedule anything important between midnight
and 3 AM.
Due to day light saving time changes it is prudent not to schedule anything important between
midnight and 3 AM.
Linux and OpenSolaris use Vixie cron which is somewhat richer in facilities than traditional
SysV cron daemon. Two important extensions are slash notation for specified period ("*/5"
means each five minutes) and "@" keywords such as @reboot which allow to specify scripts
that will be executed after reboot. This is a simpler way then populating /etc/init.d/local
script which serves for the same purpose.
Without those two extensions Vixie cron is a compatible with SysV cron.
To determine if Vixie cron is installed, use the rpm -q cron command on Suse
10 and rpm -qi vixie-cron on Red Hat.
To determine if the service is running, your can use the command /sbin/service crond status.
Cron is one of the few daemons that does not require restart when the configuration changed via
crontab command.
Vixie cron extensions include:
Ability to specify step after asterisk which simplify simple sequences. For example to run job
each five minutes you can put "*/5" in the minute field. Increments are supported in all
fields. For example */2 in day field will mean "each second day".
Vixie cron supports several simple macro definitions, including a special macro
@reboot for execution of scripts on reboot. For example
@reboot echo `hostname` was rebooted at `date` | mail -s "Reboot notification" [email protected]
Other macros include:
Entry
Description
Equivalent To
@reboot
Run once, at startup.
None
@monthly
@weekly
Run once a week
0 0 * * 0
@daily
Run once a day
0 0 * * *
@midnight
(same as @daily)
0 0 * * *
@hourly
Run once an hour
0 * * * *
Other features of Vixie cron are pretty standard.
Setting of environment variables in crontabs (PATH, SHELL, HOME, etc.)
Each user can have his own crontab;
Control of access by cron.allow and cron.deny files
A emails messages with output send to the user if command fails (this setting is controllable
and can be changed from the default user under which the cron is running to somebody else)
Linux distributions make cron somewhat more complex and flexible by splitting crontab into
several include file with predefined names cron.daily, cron.hourly, cron.weekly
and cron.monthly that are executed from the "central" crontab.
/etc/crontab (Red Hat & Suse10) -- the master crontab file which like /etc/profile
is always executed first and contains several important settings including shell to be used as well
as invocation of the script which process predefined five directories:
/etc/cron.d/
/etc/cron.daily
/etc/cron.hourly
/etc/cron.weekly
/etc/cron.monthly
Implementations of this feature are different in Red Hat
and Suse 10. See below for details of each implementation.
etc/cron.allow - list of users for which
cron is allowed. The files "allow" and "deny" can be used to control access to the "crontab"
command (which serves for listing and editing of crontabs; direct access to them is discouraged).
Cron does not need to be restarted of send HUP signal to reread those files.
/etc/cron.deny - list of users for which
cron is denied. Note: if both cron.allow and cron.deny files exist the
cron.deny is ignored.
crontab files: there are multiple (one per user) crontab files which list tasks and their
invocation conditions. All crontab files are stored in a read-protected directory, typically
/var/cron). Those files is not edited directly. There is a special command crontab
which serves for listing and editing them.
In both Suse and Red Hat there is a master crontab file /etc/crontab which like
/etc/profile is always executed first (or to be more correct is a hidden prefix of "root"
crontab file).
By default it contains several settings and invocation of script which runs each 15 min) and processes
four predefined directories (which represent "Linux extension" of standard cron functionality).
/etc/cron.daily
/etc/cron.hourly
/etc/cron.weekly
/etc/cron.monthly
Here is example from Suse:
SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * * root test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons >/dev/null 2>&1
Notes:
You can set any environment variables you wish in this "hidden" root crontab
You can specify different recipient for mailing the output of root cronjobs by changing the line
MAILTO.
Similarly you can change the shell that will run the jobs to bash or ksh93.
As we already mentioned, details of implementation are different for Red Hat and Suse.
In Red Hat the master crontab file /etc/crontab uses /usr/bin/run-parts script
to execute content of predefined directories. It contains the following lines
Bash script run-parts contains for loop which executes all components of corresponding
directory one by one. This is a pretty simple script. You can modify if you wish:
#!/bin/bash
# run-parts - concept taken from Debian
# keep going when something fails
set +e
if [ $# -lt 1 ]; then
echo "Usage: run-parts <dir>"
exit 1
fi
if [ ! -d $1 ]; then
echo "Not a directory: $1"
exit 1
fi
# Ignore *~ and *, scripts
for i in $1/*[^~,] ; do
[ -d $i ] && continue
# Don't run *.{rpmsave,rpmorig,rpmnew,swp} scripts
[ "${i%.rpmsave}" != "${i}" ] && continue
[ "${i%.rpmorig}" != "${i}" ] && continue
[ "${i%.rpmnew}" != "${i}" ] && continue
[ "${i%.swp}" != "${i}" ] && continue
[ "${i%,v}" != "${i}" ] && continue
if [ -x $i ]; then
$i 2>&1 | awk -v "progname=$i" \
'progname {
print progname ":\n"
progname="";
}
{ print; }'
fi
done
exit 0
This extension allows adding cronjobs by simply writing a file containing an invocation line into
appropriate directory. For example, by default /etc/cron.daily directory contains:
# ll
total 48
-rwxr-xr-x 1 root root 379 Dec 18 2006 0anacron
lrwxrwxrwx 1 root root 39 Jul 24 2012 0logwatch -> /usr/share/logwatch/scripts/logwatch.pl
-rwxr-xr-x 1 root root 118 Jan 18 2012 cups
-rwxr-xr-x 1 root root 180 Mar 30 2011 logrotate
-rwxr-xr-x 1 root root 418 Mar 17 2011 makewhatis.cron
-rwxr-xr-x 1 root root 137 Mar 17 2009 mlocate.cron
-rwxr-xr-x 1 root root 2181 Jun 21 2006 prelink
-rwxr-xr-x 1 root root 296 Feb 29 2012 rpm
-rwxr-xr-x 1 root root 354 Aug 7 2010 tmpwatch
You can study those scripts to add your own fragments or rewrite them completely.
Invocation of tmpwatch from /etc/cron.daily
Another "standard" component of /etc/cron.daily that deserves attention
is a cleaning script for standard/tmplocations calledtmpwatch:
cat tmpwatch
flags=-umc
/usr/sbin/tmpwatch "$flags" -x /tmp/.X11-unix -x /tmp/.XIM-unix \
-x /tmp/.font-unix -x /tmp/.ICE-unix -x /tmp/.Test-unix \
-X '/tmp/hsperfdata_*' 240 /tmp
/usr/sbin/tmpwatch "$flags" 720 /var/tmp
for d in /var/{cache/man,catman}/{cat?,X11R6/cat?,local/cat?}; do
if [ -d "$d" ]; then
/usr/sbin/tmpwatch "$flags" -f 720 "$d"
fi
done
The utility can is also invoked from several other scripts. For example cups script provides
for removal of temp files from /var/spool/cups/tmp tree using /usr/sbin/tmpwatch utility
#!/bin/sh
for d in /var/spool/cups/tmp
do
if [ -d "$d" ]; then
/usr/sbin/tmpwatch -f 720 "$d"
fi
done
exit 0
If you do not use caps it is redundant, and you can rename it to ~cups to exclude
from execution. You can also use it as a prototype for creating your own script(s) to clean temp directories
of applications that you do have on the server.
Suse 10 and 11 have identical implementations of this facility.
There is also "master" crontab at /etc/crontab that like /etc/profile is always
executed. But details and the script used (/usr/lib/cron/run-crons) are different from Red
Hat implementation:
SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * * root test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons >/dev/nul
l 2>&1
Suse also uses the same five directories as Red Hat. Each of those can contain scripts which will
be executed by shell script /usr/lib/cron/run-crons. The latter is invoked each 15 minutes
like in Red Hat. But the script itself is quite different, and as is typical for Suse
much more complex then one used by Red Hat. To make things more confusing for sysadmins, at the beginning
of its execution it sources the additional file /etc/sysconfig/cron which contains additional
settings which control the script (and by extension cron) behavior:
/home/bezroun # cat /etc/sysconfig/cron
## Path: System/Cron/Man
## Description: cron configuration for man utility
## Type: yesno
## Default: yes
#
# Should mandb and whatis be recreated by cron.daily ("yes" or "no")
#
REINIT_MANDB="yes"
## Type: yesno
## Default: yes
#
# Should old preformatted man pages (in /var/cache/man) be deleted? (yes/no)
#
DELETE_OLD_CATMAN="yes"
## Type: integer
## Default: 7
#
# How long should old preformatted man pages be kept before deletion? (days)
#
CATMAN_ATIME="7"
## Path: System/Cron
## Description: days to keep old files in tmp-dirs, 0 to disable
## Type: integer
## Default: 0
## Config:
#
# cron.daily can check for old files in tmp-dirs. It will delete all files
# not accessed for more than MAX_DAYS_IN_TMP. If MAX_DAYS_IN_TMP is not set
# or set to 0, this feature will be disabled.
#
MAX_DAYS_IN_TMP="0"
## Type: integer
## Default: 0
#
# see MAX_DAYS_IN_TMP. This allows to specify another frequency for
# a second set of directories.
#
MAX_DAYS_IN_LONG_TMP="0"
## Type: string
## Default: "/tmp"
#
# This variable contains a list of directories, in which old files are to
# be searched and deleted. The frequency is determined by MAX_DAYS_IN_TMP
#
TMP_DIRS_TO_CLEAR="/tmp"
## Type: string
## Default: ""
#
# This variable contains a list of directories, in which old files are to
# be searched and deleted. The frequency is determined by MAX_DAYS_IN_LONG_TMP
# If cleaning of /var/tmp is wanted add it here.
#
LONG_TMP_DIRS_TO_CLEAR=""
## Type: string
## Default: root
#
# In OWNER_TO_KEEP_IN_TMP, you can specify, whose files shall not be deleted.
#
OWNER_TO_KEEP_IN_TMP="root"
## Type: string
## Default: no
#
# "Set this to "yes" to entirely remove (rm -rf) all files and subdirectories
# from the temporary directories defined in TMP_DIRS_TO_CLEAR on bootup.
# Please note, that this feature ignores OWNER_TO_KEEP_IN_TMP - all files will
# be removed without exception."
#
# If this is set to a list of directories (i.e. starts with a "/"), these
# directories will be cleared instead of those listed in TMP_DIRS_TO_CLEAR.
# This can be used to clear directories at boot as well as clearing unused
# files out of other directories.
#
CLEAR_TMP_DIRS_AT_BOOTUP="no"
## Type: string
## Default: ""
#
# At which time cron.daily should start. Default is 15 minutes after booting
# the system. Example setting would be "14:00".
# Due to the fact that cron script runs only every 15 minutes,
# it will only run on xx:00, xx:15, xx:30, xx:45, not at the accurate time
# you set.
DAILY_TIME=""
## Type: integer
## Default: 5
#
# Maximum days not running when using a fixed time set in DAILY_TIME.
# 0 to skip this. This is for users who will power off their system.
#
# There is a fixed max. of 14 days set, if you want to override this
# change MAX_NOT_RUN_FORCE in /usr/lib/cron/run-crons
MAX_NOT_RUN="5"
## Type: yesno
## Default: no
#
# send status email even if all scripts in
# cron.{hourly,daily,weekly,monthly}
# returned without error? (yes/no)
#
SEND_MAIL_ON_NO_ERROR="no"
## Type: yesno
## Default: no
#
# generate syslog message for all scripts in
# cron.{hourly,daily,weekly,monthly}
# even if they haven't returned an error? (yes/no)
#
SYSLOG_ON_NO_ERROR="yes"
## Type: yesno
## Default: no
#
# send email containing output from all successful jobs in
# cron.{hourly,daily,weekly,monthly}. Output from failed
# jobs is always sent. If SEND_MAIL_ON_NO_ERROR is yes, this
# setting is ignored. (yes/no)
#
SEND_OUTPUT_ON_NO_ERROR="no"
From the content of the script you can see, for example, that time of execution of scripts in
/etc/cron.daily is controlled by env. variable $DAILY_TIME which can be set in /etc/sysconfig/cron
system file. This way you can run daily jobs at time slot different from hourly, weekly and monthly
jobs.
This is an example of unnecessary complexity dictated by desire to create a more flexible environment,
but which just confuses most sysadmin, which neither appreciate, no use provided facilities.
As you can see from the content of the file listed above, considerable attention in/etc/sysconfig/cron is devoted to the problem of deletion of temp files. Which is an
interesting problem in a sense that you can not just delete temporary files on running system. Still
you can use an init task that runs tmpwatch utility before any
application daemons are started.
NOTE: Creating a backup file before any non-trivial crontab modification is a must.
To list and edit cron file one should use crontab command which copies the specified file
or standard input if no file is specified, into a directory that holds all users' crontabs. It can also
invoke the editor with option -e to edit exiting crontab.
crontab Command Switches
-uuser— Lists the name of
a different user whose crontab is to be modified
-l— Lists the current crontab
The option -l lists the crontab file for the invoking user. By redirecting output of this command to a file you can create a backup of your crontab. For example
crontab -l > `whoami"date +"%y%m%d_%H%M"`.crontab
crontab -uusername-l List crontab file for particular user.
Convenient for root as it allow to inspect other users crontabs without switch to particular user
via su command.
crontab-u usernamefile Installs cron file for a
given user from the file, specified as an argument. This file can be created by writing output of
crontab -l
crontab [-u username] -e . This mode of invocation is for editing
of the cron files and it presupposes that editor is defined, for example via export
EDITOR=vi . After you exit crontab modified cron file is installed automatically and
is active immediately
crontab [-u username] -r The option -r removes a
user's crontab from the crontab directory. This is a dangerous operation as if you do not
have a backup content of cron file for a particular user will be lost. Option -i
should be used to prompt the user for a 'y/Y' response before actually removing the crontab.
crontab [-u username] -s SELinux only: It will append the current
SELinux security context string
If option -u is not specified crontab operates with cron files for the current user.
There are two man pages for crontab. You can view them via WWW browser from die.net:
Users are permitted to use crontab, if their names appear in the file /etc/cron.allow.
If that file does not exist, the file /etc/cron.deny is checked to determine if the user should
be denied access to crontab. If neither file exists, only a process with appropriate privileges
is allowed to submit a job. If only /etc/cron.deny exists and is empty, global usage is permitted.
The cron.allow and cron.deny files consist of one user name per line.
A crontab file can contain two types of instructions to the cron daemon:
Set an environment variable
Run this command at this time on this date.
Each user has their own crontab, and commands in any given crontab will be executed as the user who
owns the crontab.
Blank lines and leading spaces and tabs are ignored. Lines whose first non-space character is a pound-sign
(#) are comments, and are ignored. Note that comments are not allowed on the same line as cron
commands, since they will be taken to be part of the command. Similarly, comments are not allowed on
the same line as environment variable settings.
An active line in a crontab can be either an environment setting line or a cron command
line.
An environment setting should be in the form, name = value . The spaces
around the equal-sign (=) are optional, and any subsequent non-leading spaces in value will
be interpreted as the value assigned to name.
The value string may be placed in quotes (single or double, but matching) to preserve leading
or trailing blanks. The name string may also be placed in quote (single or double, but matching).
NOTE: Several environment variables should be set via /etc/crontab whichwe discussed above (SHELL, PATH, MAILTO). Actually in view of
/etc/crontab existence, setting environment variables via cron looks redundant and
should be avoided.
Each cron command is a single line that consists of six fields. One line of
cron table specifies one cron job. A cron job is a specific task that
runs a certain number of times per minute, day, week, or month. For example, you can use a cron
job to automate a daily MySQL database backup. The main problem with cron jobs is that if
they aren't properly configured and all start at the same time they can cause high server loads.
So it is prudent to have different time for start of hour and daily jobs. In case of important jobs
which run once a day it makes sense to configure your cron job so that the results of running
the scheduled script are emailed to you, not to root under which the job was run.
There are two main ways by which you create a cron job. On is using your Web administration
panel ( most *nix webhosting providers offer Web-base interface to cron) or using shell access to
your server.
A crontab expression is a string comprising 6 or 7 (with year) fields separated by white space.
Its structure has the structure shown in the table:
Field
Meaning
Allowed range
Allowed special characters
Example
1
Minutes that have to pass after the selected hour in order to execute the task
0-59
* / , -
30, which means 30 minutes after the selected hour
2
Hours at which the task has to be executed
0-23
* / , -
04, which means at 4 O'clock in the morning
3
Days of the month on which this task has to be executed
1-31
* / , -
*, which means that every day of the selected month
4
Months during which the task has to be executed
1-12
* / , -
3-5, which means run the task in the months of March, April & May First
3 letters of the Month name. Case doesn't matter. E.g. Jan
5
Days of the week on which this task has to be run
0-7
* / , -
* means all days of the selected weeks Numeric value or first 3 letters of
the Day name. Case doesn't matter (Sun or sun)
(0 or 7 is Sun, 1 is Mon...)
6
Name of the program (task) to be executed
Any program
%
absolute path to executable required
Special characters
Support for each special character depends on specific distributions and versions of cron
Asterisk ( * )
The asterisk indicates that the cron expression will match for all values of the field;
e.g., using an asterisk in the 4th field (month) would indicate every month.
Slash ( / )
Slashes are used to describe increments of ranges. For example 3-59/15 in the 1st field (minutes)
would indicate the 3rd minute of the hour and every 15 minutes thereafter. The form "*/..." is equivalent
to the form "first-last/...", that is, an increment over the largest possible range of the field.
Percent ( % )
Percent-signs (%) in the command, unless
escaped with backslash (\), will be changed into newline characters, and all data after the first
% will be sent to the command as standard input.
Comma ( , )
Commas are used to separate items of a list. For example, using "MON,WED,FRI" in the 5th field
(day of week) would mean Mondays, Wednesdays and Fridays.
Hyphen ( - )
Hyphens are used to define ranges. For example, 2000-2010 would indicate every year between 2000
and 2010 CE inclusive.
A field may be an asterisk (*), which always stands for "first-last''.
Ranges of numbers are allowed. Ranges are two numbers separated with a hyphen. The specified
range is inclusive. For example, 8-11 for an "hours'' entry specifies execution at hours 8, 9, 10
and 11.
Lists are allowed. A list is a set of numbers (or ranges) separated by commas.
Examples: "1,2,5,9'', "0-4,8-12''.
Step values can be used in conjunction with ranges. Following a range with "/''
specifies skips of the number's value through the range. For example, "0-23/2'' can be used in the
hours field to specify command execution every other hour. Steps are also permitted after an asterisk,
so if you want to say "every ten minutes'', just use "*/10''. Ranges can include "steps",
so "1-9/2" is the same as "1,3,5,7,9".
Names can also be used for the "month'' and "day of week'' fields. Use the first three
letters of the particular day or month (case doesn't matter). Ranges or lists of names are not allowed.
The last field (the rest of the line) specifies the command to be run. The entire command
portion of the line, up to a newline or % character, will be executed by /bin/sh
or by the shell specified in the SHELL variable of the /etc/crontab
Percent-signs (%) in the command, unless escaped with backslash (\), will be changed into newline
characters, and all data after the first % will be sent to the command as standard input.
Note: The day of a command's execution can be specified by two fields --- day of month, and
day of week. If both fields are restricted (i.e, aren't *), the command will be run when either
field matches the current time. For example, "30 4 1,15 * 5'' would cause a command to be run at 4:30
am on the 1st and 15th of each month, plus every Friday.
To avoid mistakes it is recommended to include the following header in the crontab
# minute (0-59),
# | hour (0-23),
# | | day of the month (1-31),
# | | | month of the year (1-12),
# | | | | day of the week (0-6 with 0=Sunday, 1-Monday).
# | | | | | commands
Instead of the first five fields, in vixie-cron you can use one of eight predefined macros:
string meaning
------ -------
@reboot Run once, at startup.
@yearly Run once a year, "0 0 1 1 *".
@annually (same as @yearly)
@monthly Run once a month, "0 0 1 * *".
@weekly Run once a week, "0 0 * * 0".
@daily Run once a day, "0 0 * * *".
@midnight (same as @daily)
@hourly Run once an hour, "0 * * * *".
The most useful is @reboot which does provide new functionality. Usefulness of
others macros is questionable.
If you are in one of the countries that observe Daylight Savings Time, jobs scheduled during the
rollback or advance will be affected. In general, it is not a good idea to schedule jobs from
1pm to 3pm.
For US timezones (except parts of IN, AZ, and HI) the time shift occurs at 2AM local time. For others,
the output of the
zdump(8) program's verbose (-v) option can be used to determine the moment of time shift.
Cron has a built in feature of allowing you to specify who may, and who may not use it. It does this
by the use of /etc/cron.allow and /etc/cron.deny files:
/etc/cron.allow - list of users for
which cron is allowed. The files /etc/cron.allow and /etc/cron.deny can be used to control
access to the "crontab" command (which serves for listing and editing of crontabs; direct
access to them is discouraged). Cron does not need to be restarted of send HUP signal to reread those
files.
/etc/cron.deny - list of users for which
cron is denied.
Note: if both cron.allow and cron.deny files exist the cron.deny
is ignored.
If you wanted that only selected users can use cron, you could add the line ALL
to the cron.deny file and put the list of those users into cron.allow file:
echo ALL >>/etc/cron.deny
If you want user apache to be able to use cron you need to add the appropriate line
to /etc/cron.allow file. For example:
echo apache >>/etc/cron.allow
If there is neither a cron.allow nor a cron.deny file, then the use of cron
is unrestricted (i.e. every user can use it). If you put a name (or several names) into cron.allow
file, without creating a cron.deny file, it would have the same effect as creating a cron.deny
file with ALL in it. This means that any subsequent users that require cron access should be
put in to the cron.allow file.
By default the output from cron gets mailed to the person specified in the MAILTO variable.
If this variable is not defined it is mailed to the owner of the process. If you want to mail
the output to someone else, you can just pipe the output to the command mailx command. For
example:
If you have a command that is run often, and you don't want to be emailed the output every time,
you can redirect the output to a log file (or /dev/null, if you really don't want the output).
For example
cmd >> log.file
Now you can create a separate cron job that analyses the log file and mail you only if this
is an important message. Or just once a day, if the script is not crucial. This way you also can organize
log rotation. See logrotate for details
Hi All,
I created a crontab entry in a cron.txt file accidentally entered
crontab cron.txt.
Now my previous crontab -l entries are not showing up, that means i removed the scheduling
of the previous jobs by running this command "crontab cron.txt"
How do I revert back to previously schedule jobs.
Please help. this is urgent.,
Thanks.
In this case, if you do not have a backup, you only remedy is to try to extract cron commands
from /var/log/messages.
For classic cron backup and restore are simple. You just pipe the crontab in backup file or pipe
the backup from the crontab backup file.
To backup the current crontab settings for current user:
If we have a set of files named for example root.crontab131010, root.crontab131011,
root.crontab131012, you can restore content of the most recent backup of the crontab using
the command:
The “-“ specifies that crontab will use the standard input.
With Suse and RHEL structure /etc/cron.d directory should be backed up too. Remember that
crontab data can exist for each user of the system, not just root. For example the user
oracle often has crontab entries.
...I had to come up with a solution for a customer using AWS/EC2 to make their crontab resilient
to server re-builds in case it goes down by autoscaler or other situations. That’s why I created
a script that is called daily that backups the crontab of a specific user to a file in an EBS mounted
volume maintaining a 7 days library just in case:
Use
the Bash shell in Linux to manage foreground and background processes. You can use Bash's job control functions and
signals to give you more flexibility in how you run commands. We show you how.
How to Speed Up a Slow PC
https://imasdk.googleapis.com/js/core/bridge3.401.2_en.html#goog_863166184
All About Processes
Whenever a program is executed in a Linux or Unix-like operating
system, a process is started. "Process" is the name for the internal representation of the executing program in the
computer's memory. There is a process for every active program. In fact, there is a process for nearly everything that
is running on your computer. That includes the components of your
graphical
desktop environment
(GDE) such as
GNOME
or
KDE
,
and system
daemons
that
are launched at start-up.
Why
nearly
everything
that is running? Well, Bash built-ins such as
cd
,
pwd
,
and
alias
do
not need to have a process launched (or "spawned") when they are run. Bash executes these commands within the instance
of the Bash shell that is running in your terminal window. These commands are fast precisely because they don't need to
have a process launched for them to execute. (You can type
help
in
a terminal window to see the list of Bash built-ins.)
Processes can be running in the foreground, in which case they take
over your terminal until they have completed, or they can be run in the background. Processes that run in the background
don't dominate the terminal window and you can continue to work in it. Or at least, they don't dominate the terminal
window if they don't generate screen output.
A Messy Example
We'll start a simple
ping
trace
running
. We're going to
ping
the
How-To Geek domain. This will execute as a foreground process.
ping www.howtogeek.com
We get the expected results, scrolling down the terminal window. We
can't do anything else in the terminal window while
ping
is
running. To terminate the command hit
Ctrl+C
.
Ctrl+C
The visible effect of the
Ctrl+C
is
highlighted in the screenshot.
ping
gives
a short summary and then stops.
Let's repeat that. But this time we'll hit
Ctrl+Z
instead
of
Ctrl+C
.
The task won't be terminated. It will become a background task. We get control of the terminal window returned to us.
ping www.howtogeek.com
Ctrl+Z
The visible effect of hitting
Ctrl+Z
is
highlighted in the screenshot.
This time we are told the process is stopped. Stopped doesn't mean
terminated. It's like a car at a stop sign. We haven't scrapped it and thrown it away. It's still on the road,
stationary, waiting to go. The process is now a background
job
.
The
jobs
command
will
list the jobs
that have been started in the current terminal session. And because jobs are (inevitably) processes,
we can also use the
ps
command
to see them. Let's use both commands and compare their outputs. We'll use the
T
option
(terminal) option to only list the processes that are running in this terminal window. Note that there is no need to use
a hyphen
-
with
the
T
option.
jobs
ps T
The
jobs
command
tells us:
[1]
:
The number in square brackets is the job number. We can use this to refer to the job when we need to control it with
job control commands.
+
:
The plus sign
+
shows
that this is the job that will be acted upon if we use a job control command without a specific job number. It is
called the default job. The default job is always the one most recently added to the list of jobs.
Stopped
:
The process is not running.
ping
www.howtogeek.com
: The command line that launched the process.
The
ps
command
tells us:
PID
:
The process ID of the process. Each process has a unique ID.
TTY
:
The pseudo-teletype (terminal window) that the process was executed from.
STAT
:
The status of the process.
TIME
:
The amount of CPU time consumed by the process.
COMMAND
:
The command that launched the process.
These are common values for the STAT column:
D
:
Uninterruptible sleep. The process is in a waiting state, usually waiting for input or output, and cannot be
interrupted.
I
:
Idle.
R
:
Running.
S
:
Interruptible sleep.
T
:
Stopped by a job control signal.
Z
:
A zombie process. The process has been terminated but hasn't been "cleaned down" by its parent process.
The value in the STAT column can be followed by one of these extra
indicators:
<
:
High-priority task (not nice to other processes).
N
:
Low-priority (nice to other processes).
L
:
process has pages locked into memory (typically used by real-time processes).
s
:
A session leader. A session leader is a process that has launched process groups. A shell is a session leader.
l
:
Multi-thread process.
+
:
A foreground process.
We can see that Bash has a state of
Ss
.
The uppercase "S" tell us the Bash shell is sleeping, and it is interruptible. As soon as we need it, it will respond.
The lowercase "s" tells us that the shell is a session leader.
The ping command has a state of
T
.
This tells us that
ping
has
been stopped by a job control signal. In this example, that was the
Ctrl+Z
we
used to put it into the background.
The
ps
T
command has a state of
R
,
which stands for running. The
+
indicates
that this process is a member of the foreground group. So the
ps
T
command is running in the foreground.
The bg Command
The
bg
command
is used to resume a background process. It can be used with or without a job number. If you use it without a job number
the default job is brought to the foreground. The process still runs in the background. You cannot send any input to it.
If we issue the
bg
command,
we will resume our
ping
command:
bg
The
ping
command
resumes and we see the scrolling output in the terminal window once more. The name of the command that has been
restarted is displayed for you. This is highlighted in the screenshot.
But we have a problem. The task is running in the background and
won't accept input. So how do we stop it?
Ctrl+C
doesn't
do anything. We can see it when we type it but the background task doesn't receive those keystrokes so it keeps pinging
merrily away.
In fact, we're now in a strange blended mode. We can type in the
terminal window but what we type is quickly swept away by the scrolling output from the
ping
command.
Anything we type takes effect in the foregound.
To stop our background task we need to bring it to the foreground
and then stop it.
The fg Command
The
fg
command
will bring a background task into the foreground. Just like the
bg
command,
it can be used with or without a job number. Using it with a job number means it will operate on a specific job. If it
is used without a job number the last command that was sent to the background is used.
If we type
fg
our
ping
command
will be brought to the foreground. The characters we type are mixed up with the output from the
ping
command,
but they are operated on by the shell as if they had been entered on the command line as usual. And in fact, from the
Bash shell's point of view, that is exactly what has happened.
fg
And now that we have the
ping
command
running in the foreground once more, we can use
Ctrl+C
to
kill it.
Ctrl+C
We Need to Send the Right Signals
That wasn't exactly pretty. Evidently running a process in the
background works best when the process doesn't produce output and doesn't require input.
But, messy or not, our example did accomplish:
Putting a process into the background.
Restoring the process to a running state in the background.
Returning the process to the foreground.
Terminating the process.
When you use
Ctrl+C
and
Ctrl+Z
,
you are sending signals to the process. These are
shorthand
ways
of using the
kill
command.
There are
64
different signals
that
kill
can
send. Use
kill
-l
at the command line to list them.
kill
isn't
the only source of these signals. Some of them are raised automatically by other processes within the system
Here are some of the commonly used ones.
SIGHUP
:
Signal 1. Automatically sent to a process when the terminal it is running in is closed.
SIGINT
:
Signal 2. Sent to a process you hit
Ctrl+C
.
The process is interrupted and told to terminate.
SIGQUIT
:
Signal 3. Sent to a process if the user sends a quit signal
Ctrl+D
.
SIGKILL
:
Signal 9. The process is immediately killed and will not attempt to close down cleanly. The process does not go down
gracefully.
SIGTERM
: Signal
15. This is the default signal sent by
kill
.
It is the standard program termination signal.
SIGTSTP
: Signal
20. Sent to a process when you use
Ctrl+Z
.
It stops the process and puts it in the background.
We must use the
kill
command
to issue signals that do not have key combinations assigned to them.
Further Job Control
A process moved into the background by using
Ctrl+Z
is
placed in the stopped state. We have to use the
bg
command
to start it running again. To launch a program as a running background process is simple. Append an ampersand
&
to
the end of the command line.
Although it is best that background processes do not write to the
terminal window, we're going to use examples that do. We need to have something in the screenshots that we can refer to.
This command will start an endless loop as a background process:
while true; do echo "How-To Geek Loop
Process"; sleep 3; done &
We are told the job number and process ID id of the process. Our
job number is 1, and the process id is 1979. We can use these identifiers to control the process.
The output from our endless loop starts to appear in the terminal
window. As before, we can use the command line but any commands we issue are interspersed with the output from the loop
process.
ls
To stop our process we can use
jobs
to
remind ourselves what the job number is, and then use
kill
.
jobs
reports that our process is job number 1. To use that number with
kill
we
must precede it with a percent sign
%
.
jobs
kill %1
kill
sends the
SIGTERM
signal,
signal number 15, to the process and it is terminated. When the Enter key is next pressed, a status of the job is shown.
It lists the process as "terminated." If the process does not respond to the
kill
command
you can take it up a notch. Use
kill
with
SIGKILL
,
signal number 9. Just put the number 9 between the
kill
command
the job number.
kill 9 %1
Things We've Covered
Ctrl+C
:
Sends
SIGINT
,
signal 2, to the process -- if it is accepting input -- and tells it to terminate.
Ctrl+D
: Sends
SISQUIT
,
signal 3, to the process -- if it is accepting input -- and tells it to quit.
Ctrl+Z
: Sends
SIGSTP
,
signal 20, to the process and tells it to stop (suspend) and become a background process.
jobs
:
Lists the background jobs and shows their job number.
bg
job_number
:
Restarts a background process. If you don't provide a job number the last process that was turned into a background
task is used.
fg
job_number
:
brings a background process into the foreground and restarts it. If you don't provide a job number the last process
that was turned into a background task is used.
commandline
&
:
Adding an ampersand
&
to
the end of a command line executes that command as a background task, that is running.
kill %
job_number
:
Sends
SIGTERM
,
signal 15, to the process to terminate it.
kill 9
%
job_number
:
Sends
SIGKILL
,
signal 9, to the process and terminates it abruptly.
In this quick tutorial, I want to look at
the
jobs
command
and a few of the ways that we can manipulate the jobs running on our systems. In short, controlling jobs lets you
suspend and resume processes started in your Linux shell.
Jobs
The
jobs
command
will list all jobs on the system; active, stopped, or otherwise. Before I explore the command and output, I'll create
a job on my system.
I will use the
sleep
job
as it won't change my system in any meaningful way.
First, I issued the
sleep
command,
and then I received the
Job number
[1].
I
then immediately stopped the job by using
Ctl+Z
.
Next, I run the
jobs
command
to view the newly created job:
[tcarrigan@rhel ~]$ jobs
[1]+ Stopped sleep 500
You can see that I have a single stopped job
identified by the job number
[1]
.
Other options to know for this command
include:
-l - list PIDs in addition to default info
-n - list only processes that have changed since the last notification
-p - list PIDs only
-r - show only running jobs
-s - show only stopped jobs
Background
Next, I'll resume the
sleep
job
in the background. To do this, I use the
bg
command.
Now, the
bg
command
has a pretty simple syntax, as seen here:
bg [JOB_SPEC]
Where JOB_SPEC can be any of the following:
%n - where
n
is the job number
%abc - refers to a job started by a command beginning with
abc
%?abc - refers to a job started by a command containing
abc
%- - specifies the previous job
NOTE
:
bg
and
fg
operate
on the current job if no JOB_SPEC is provided.
I can move this job to the background by
using the job number
[1]
.
[tcarrigan@rhel ~]$ bg %1
[1]+ sleep 500 &
You can see now that I have a single running
job in the background.
[tcarrigan@rhel ~]$ jobs
[1]+ Running sleep 500 &
Foreground
Now, let's look at how to move a background
job into the foreground. To do this, I use the
fg
command.
The command syntax is the same for the foreground command as with the background command.
fg [JOB_SPEC]
Refer to the above bullets for details on
JOB_SPEC.
I have started a new
sleep
in
the background:
[tcarrigan@rhel ~]$ sleep 500 &
[2] 5599
Now, I'll move it to the foreground by using
the following command:
[tcarrigan@rhel ~]$ fg %2
sleep 500
The
fg
command
has now brought my system back into a sleep state.
The end
While I realize that the jobs presented here
were trivial, these concepts can be applied to more than just the
sleep
command.
If you run into a situation that requires it, you now have the knowledge to move running or stopped jobs from the
foreground to background and back again.
No time for
commands? Scheduling tasks with cron means programs can run but you don't have to stay up
late.9 comments Image by :
Internet Archive Book Images. Modified by Opensource.com. CC BY-SA 4.0 x Subscribe now
Instead, I use two service utilities that allow me to run commands, programs, and tasks at
predetermined times. The cron
and at services enable sysadmins to schedule tasks to run at a specific time in the future. The
at service specifies a one-time task that runs at a certain time. The cron service can schedule
tasks on a repetitive basis, such as daily, weekly, or monthly.
In this article, I'll introduce the cron service and how to use it.
Common (and
uncommon) cron uses
I use the cron service to schedule obvious things, such as regular backups that occur daily
at 2 a.m. I also use it for less obvious things.
The system times (i.e., the operating system time) on my many computers are set using the
Network Time Protocol (NTP). While NTP sets the system time, it does not set the hardware
time, which can drift. I use cron to set the hardware time based on the system time.
I also have a Bash program I run early every morning that creates a new "message of the
day" (MOTD) on each computer. It contains information, such as disk usage, that should be
current in order to be useful.
Many system processes and services, like Logwatch , logrotate , and Rootkit Hunter , use the cron service to schedule
tasks and run programs every day.
The crond daemon is the background service that enables cron functionality.
The cron service checks for files in the /var/spool/cron and /etc/cron.d directories and the
/etc/anacrontab file. The contents of these files define cron jobs that are to be run at
various intervals. The individual user cron files are located in /var/spool/cron , and system
services and applications generally add cron job files in the /etc/cron.d directory. The
/etc/anacrontab is a special case that will be covered later in this article.
Using
crontab
The cron utility runs based on commands specified in a cron table ( crontab ). Each user,
including root, can have a cron file. These files don't exist by default, but can be created in
the /var/spool/cron directory using the crontab -e command that's also used to edit a cron file
(see the script below). I strongly recommend that you not use a standard editor (such as
Vi, Vim, Emacs, Nano, or any of the many other editors that are available). Using the crontab
command not only allows you to edit the command, it also restarts the crond daemon when you
save and exit the editor. The crontab command uses Vi as its underlying editor, because Vi is
always present (on even the most basic of installations).
New cron files are empty, so commands must be added from scratch. I added the job definition
example below to my own cron files, just as a quick reference, so I know what the various parts
of a command mean. Feel free to copy it for your own use.
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
# backup using the rsbu program to the internal 4TB HDD and then 4TB external
01 01 * * * / usr / local / bin / rsbu -vbd1 ; / usr / local / bin / rsbu -vbd2
# Set the hardware clock to keep it in sync with the more accurate system clock
03 05 * * * / sbin / hwclock --systohc
# Perform monthly updates on the first of the month
# 25 04 1 * * /usr/bin/dnf -y update
The crontab command is used to view or edit the cron files.
The first three lines in the code above set up a default environment. The environment must
be set to whatever is necessary for a given user because cron does not provide an environment
of any kind. The SHELL variable specifies the shell to use when commands are executed. This
example specifies the Bash shell. The MAILTO variable sets the email address where cron job
results will be sent. These emails can provide the status of the cron job (backups, updates,
etc.) and consist of the output you would see if you ran the program manually from the command
line. The third line sets up the PATH for the environment. Even though the path is set here, I
always prepend the fully qualified path to each executable.
There are several comment lines in the example above that detail the syntax required to
define a cron job. I'll break those commands down, then add a few more to show you some more
advanced capabilities of crontab files.
This line in my /etc/crontab runs a script that performs backups for my
systems.
This line runs my self-written Bash shell script, rsbu , that backs up all my systems. This
job kicks off at 1:01 a.m. (01 01) every day. The asterisks (*) in positions three, four, and
five of the time specification are like file globs, or wildcards, for other time divisions;
they specify "every day of the month," "every month," and "every day of the week." This line
runs my backups twice; one backs up to an internal dedicated backup hard drive, and the other
backs up to an external USB drive that I can take to the safe deposit box.
The following line sets the hardware clock on the computer using the system clock as the
source of an accurate time. This line is set to run at 5:03 a.m. (03 05) every day.
03 05 * * * /sbin/hwclock --systohc
This line sets the hardware clock using the system time as the source.
I was using the third and final cron job (commented out) to perform a dnf or yum update at
04:25 a.m. on the first day of each month, but I commented it out so it no longer runs.
# 25 04 1 * * /usr/bin/dnf -y update
This line used to perform a monthly update, but I've commented it
out.
Other scheduling tricks
Now let's do some things that are a little more interesting than these basics. Suppose you
want to run a particular job every Thursday at 3 p.m.:
00 15 * * Thu /usr/local/bin/mycronjob.sh
This line runs mycronjob.sh every Thursday at 3 p.m.
Or, maybe you need to run quarterly reports after the end of each quarter. The cron service
has no option for "The last day of the month," so instead you can use the first day of the
following month, as shown below. (This assumes that the data needed for the reports will be
ready when the job is set to run.)
02 03 1 1,4,7,10 * /usr/local/bin/reports.sh
This cron job runs quarterly reports on the first day of the month after a quarter
ends.
The following shows a job that runs one minute past every hour between 9:01 a.m. and 5:01
p.m.
01 09-17 * * * /usr/local/bin/hourlyreminder.sh
Sometimes you want to run jobs at regular times during normal business hours.
I have encountered situations where I need to run a job every two, three, or four hours.
That can be accomplished by dividing the hours by the desired interval, such as */3 for every
three hours, or 6-18/3 to run every three hours between 6 a.m. and 6 p.m. Other intervals can
be divided similarly; for example, the expression */15 in the minutes position means "run the
job every 15 minutes."
*/5 08-18/2 * * * /usr/local/bin/mycronjob.sh
This cron job runs every five minutes during every hour between 8 a.m. and 5:58
p.m.
One thing to note: The division expressions must result in a remainder of zero for the job
to run. That's why, in this example, the job is set to run every five minutes (08:05, 08:10,
08:15, etc.) during even-numbered hours from 8 a.m. to 6 p.m., but not during any odd-numbered
hours. For example, the job will not run at all from 9 p.m. to 9:59 a.m.
I am sure you can come up with many other possibilities based on these
examples.
Regular users with cron access could make mistakes that, for example, might cause system
resources (such as memory and CPU time) to be swamped. To prevent possible misuse, the sysadmin
can limit user access by creating a /etc/cron.allow file that contains a list of all users with
permission to create cron jobs. The root user cannot be prevented from using cron.
By preventing non-root users from creating their own cron jobs, it may be necessary for root
to add their cron jobs to the root crontab. "But wait!" you say. "Doesn't that run those jobs
as root?" Not necessarily. In the first example in this article, the username field shown in
the comments can be used to specify the user ID a job is to have when it runs. This prevents
the specified non-root user's jobs from running as root. The following example shows a job
definition that runs a job as the user "student":
04 07 * * * student /usr/local/bin/mycronjob.sh
If no user is specified, the job is run as the user that owns the crontab file, root in this
case.
cron.d
The directory /etc/cron.d is where some applications, such as SpamAssassin and sysstat , install cron files. Because there is no
spamassassin or sysstat user, these programs need a place to locate cron files, so they are
placed in /etc/cron.d .
The /etc/cron.d/sysstat file below contains cron jobs that relate to system activity
reporting (SAR). These cron files have the same format as a user cron file.
# Run system
activity accounting tool every 10 minutes
*/ 10 * * * * root / usr / lib64 / sa / sa1 1 1
# Generate a daily summary of process accounting at 23:53
53 23 * * * root / usr / lib64 / sa / sa2 -A
The sysstat package installs the /etc/cron.d/sysstat cron file to run programs for
SAR.
The sysstat cron file has two lines that perform tasks. The first line runs the sa1 program
every 10 minutes to collect data stored in special binary files in the /var/log/sa directory.
Then, every night at 23:53, the sa2 program runs to create a daily summary.
Scheduling
tips
Some of the times I set in the crontab files seem rather random -- and to some extent they
are. Trying to schedule cron jobs can be challenging, especially as the number of jobs
increases. I usually have only a few tasks to schedule on each of my computers, which is
simpler than in some of the production and lab environments where I have worked.
One system I administered had around a dozen cron jobs that ran every night and an
additional three or four that ran on weekends or the first of the month. That was a challenge,
because if too many jobs ran at the same time -- especially the backups and compiles -- the
system would run out of RAM and nearly fill the swap file, which resulted in system thrashing
while performance tanked, so nothing got done. We added more memory and improved how we
scheduled tasks. We also removed a task that was very poorly written and used large amounts of
memory.
The crond service assumes that the host computer runs all the time. That means that if the
computer is turned off during a period when cron jobs were scheduled to run, they will not run
until the next time they are scheduled. This might cause problems if they are critical cron
jobs. Fortunately, there is another option for running jobs at regular intervals: anacron
.
anacron
The anacron program
performs the same function as crond, but it adds the ability to run jobs that were skipped,
such as if the computer was off or otherwise unable to run the job for one or more cycles. This
is very useful for laptops and other computers that are turned off or put into sleep mode.
As soon as the computer is turned on and booted, anacron checks to see whether configured
jobs missed their last scheduled run. If they have, those jobs run immediately, but only once
(no matter how many cycles have been missed). For example, if a weekly job was not run for
three weeks because the system was shut down while you were on vacation, it would be run soon
after you turn the computer on, but only once, not three times.
The anacron program provides some easy options for running regularly scheduled tasks. Just
install your scripts in the /etc/cron.[hourly|daily|weekly|monthly] directories, depending how
frequently they need to be run.
How does this work? The sequence is simpler than it first appears.
The crond service runs the cron job specified in /etc/cron.d/0hourly .
# Run the hourly jobs
SHELL = / bin / bash
PATH = / sbin: / bin: / usr / sbin: / usr / bin
MAILTO =root
01 * * * * root run-parts / etc / cron.hourly
The contents of /etc/cron.d/0hourly cause the shell scripts located in /etc/cron.hourly
to run.
The cron job specified in /etc/cron.d/0hourly runs the run-parts program once per
hour.
The run-parts program runs all the scripts located in the /etc/cron.hourly
directory.
The /etc/cron.hourly directory contains the 0anacron script, which runs the anacron
program using the /etdc/anacrontab configuration file shown here.
# /etc/anacrontab: configuration file for anacron
# See anacron(8) and anacrontab(5) for details.
SHELL = / bin / sh
PATH = / sbin: / bin: / usr / sbin: / usr / bin
MAILTO =root
# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY = 45
# the jobs will be started during the following hours only
START_HOURS_RANGE = 3 - 22
The contents of /etc/anacrontab file runs the executable files in the
cron.[daily|weekly|monthly] directories at the appropriate times.
The anacron program runs the programs located in /etc/cron.daily once per day; it runs
the jobs located in /etc/cron.weekly once per week, and the jobs in cron.monthly once per
month. Note the specified delay times in each line that help prevent these jobs from
overlapping themselves and other cron jobs.
Instead of placing complete Bash programs in the cron.X directories, I install them in the
/usr/local/bin directory, which allows me to run them easily from the command line. Then I add
a symlink in the appropriate cron directory, such as /etc/cron.daily .
The anacron program is not designed to run programs at specific times. Rather, it is
intended to run programs at intervals that begin at the specified times, such as 3 a.m. (see
the START_HOURS_RANGE line in the script just above) of each day, on Sunday (to begin the
week), and on the first day of the month. If any one or more cycles are missed, anacron will
run the missed jobs once, as soon as possible.
More on setting limits
I use most of these methods for scheduling tasks to run on my computers. All those tasks are
ones that need to run with root privileges. It's rare in my experience that regular users
really need a cron job. One case was a developer user who needed a cron job to kick off a daily
compile in a development lab.
It is important to restrict access to cron functions by non-root users. However, there are
circumstances when a user needs to set a task to run at pre-specified times, and cron can allow
them to do that. Many users do not understand how to properly configure these tasks using cron
and they make mistakes. Those mistakes may be harmless, but, more often than not, they can
cause problems. By setting functional policies that cause users to interact with the sysadmin,
individual cron jobs are much less likely to interfere with other users and other system
functions.
It is possible to set limits on the total resources that can be allocated to individual
users or groups, but that is an article for another time.
For more information, the man pages for cron , crontab , anacron , anacrontab , and run-parts
all have excellent information and descriptions of how the cron system works.
Cron is definitely a good tool. But if you need to do more advanced scheduling then Apache
Airflow is great for this.
Airflow has a number of advantages over Cron. The most important are: Dependencies (let
tasks run after other tasks), nice web based overview, automatic failure recovery and a
centralized scheduler. The disadvantages are that you will need to setup the scheduler and
some other centralized components on one server and a worker on each machine you want to run
stuff on.
You definitely want to use Cron for some stuff. But if you find that Cron is too limited
for your use case I would recommend looking into Airflow.
Hi David,
you have a well done article. Much appreciated. I make use of the @reboot crontab entry. With
crontab and root. I run the following.
@reboot /bin/dofstrim.sh
I wanted to run fstrim for my SSD drive once and only once per week.
dofstrim.sh is a script that runs the "fstrim" program once per week, irrespective of the
number of times the system is rebooted. I happen to have several Linux systems sharing one
computer, and each system has a root crontab with that entry. Since I may hop from Linux to
Linux in the day or several times per week, my dofstrim.sh only runs fstrim once per week,
irrespective which Linux system I boot. I make use of a common partition to all Linux
systems, a partition mounted as "/scratch" and the wonderful Linux command line "date"
program.
The dofstrim.sh listing follows below.
#!/bin/bash
# run fstrim either once/week or once/day not once for every reboot
#
# Use the date function to extract today's day number or week number
# the day number range is 1..366, weekno is 1 to 53
#WEEKLY=0 #once per day
WEEKLY=1 #once per week
lockdir='/scratch/lock/'
if [[ WEEKLY -eq 1 ]]; then
dayno="$lockdir/dofstrim.weekno"
today=$(date +%V)
else
dayno=$lockdir/dofstrim.dayno
today=$(date +%j)
fi
prevval="000"
if [ -f "$dayno" ]
then
prevval=$(cat ${dayno} )
if [ x$prevval = x ];then
prevval="000"
fi
else
mkdir -p $lockdir
fi
if [ ${prevval} -ne ${today} ]
then
/sbin/fstrim -a
echo $today > $dayno
fi
I had thought to use anacron, but then fstrim would be run frequently as each linux's
anacron would have a similar entry.
The "date" program produces a day number or a week number, depending upon the +%V or +%j
Running a report on the last day of the month is easy if you use the date program. Use the
date function from Linux as shown
*/9 15 28-31 * * [ `date -d +'1 day' +\%d` -eq 1 ] && echo "Tomorrow is the first
of month Today(now) is `date`" >> /root/message
Once per day from the 28th to the 31st, the date function is executed.
If the result of date +1day is the first of the month, today must be the last day of the
month.
The
Task Spooler
project allows you
to queue up tasks from the shell for batch execution. Task Spooler is simple to use and requires no
configuration. You can view and edit queued commands, and you can view the output of queued commands at any
time.
Task Spooler has some similarities with other delayed and batch execution projects, such as "
at
."
While both Task Spooler and at handle multiple queues and allow the execution of commands at a later point, the
at project handles output from commands by emailing the results to the user who queued the command, while Task
Spooler allows you to get at the results from the command line instead. Another major difference is that Task
Spooler is not aimed at executing commands at a specific time, but rather at simply adding to and executing
commands from queues.
The main repositories for Fedora, openSUSE, and Ubuntu do not contain packages for Task Spooler. There are
packages for some versions of Debian, Ubuntu, and openSUSE 10.x available along with the source code on the
project's homepage. In this article I'll use a 64-bit Fedora 9 machine and install version 0.6 of Task Spooler
from source. Task Spooler does not use autotools to build, so to install it, simply run
make; sudo make
install
. This will install the main Task Spooler command
ts
and its manual page into /usr/local.
A simple interaction with Task Spooler is shown below. First I add a new job to the queue and check the
status. As the command is a very simple one, it is likely to have been executed immediately. Executing ts by
itself with no arguments shows the executing queue, including tasks that have completed. I then use
ts -c
to get at the stdout of the executed command. The
-c
option uses
cat
to display the
output file for a task. Using
ts -i
shows you information about the job. To clear finished jobs
from the queue, use the
ts -C
command, not shown in the example.
$ ts echo "hello world"
6
$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/1]
6 finished /tmp/ts-out.QoKfo9 0 0.00/0.00/0.00 echo hello world
The
-t
option operates like
tail -f
, showing you the last few lines of output and
continuing to show you any new output from the task. If you would like to be notified when a task has
completed, you can use the
-m
option to have the results mailed to you, or you can queue another
command to be executed that just performs the notification. For example, I might add a tar command and want to
know when it has completed. The below commands will create a tarball and use
libnotify
commands to create an
inobtrusive popup window on my desktop when the tarball creation is complete. The popup will be dismissed
automatically after a timeout.
$ ts tar czvf /tmp/mytarball.tar.gz liberror-2.1.80011
11
$ ts notify-send "tarball creation" "the long running tar creation process is complete."
12
$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/1]
11 finished /tmp/ts-out.O6epsS 0 4.64/4.31/0.29 tar czvf /tmp/mytarball.tar.gz liberror-2.1.80011
12 finished /tmp/ts-out.4KbPSE 0 0.05/0.00/0.02 notify-send tarball creation the long... is complete.
Notice in the output above, toward the far right of the header information, the
run=0/1
line.
This tells you that Task Spooler is executing nothing, and can possibly execute one task. Task spooler allows
you to execute multiple tasks at once from your task queue to take advantage of multicore CPUs. The
-S
option allows you to set how many tasks can be executed in parallel from the queue, as shown below.
$ ts -S 2
$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/2]
6 finished /tmp/ts-out.QoKfo9 0 0.00/0.00/0.00 echo hello world
If you have two tasks that you want to execute with Task Spooler but one depends on the other having already
been executed (and perhaps that the previous job has succeeded too) you can handle this by having one task wait
for the other to complete before executing. This becomes more important on a quad core machine when you might
have told Task Spooler that it can execute three tasks in parallel. The commands shown below create an explicit
dependency, making sure that the second command is executed only if the first has completed successfully, even
when the queue allows multiple tasks to be executed. The first command is queued normally using
ts
.
I use a subshell to execute the commands by having
ts
explicitly start a new bash shell. The
second command uses the
-d
option, which tells
ts
to execute the command only after
the successful completion of the last command that was appended to the queue. When I first inspect the queue I
can see that the first command (28) is executing. The second command is queued but has not been added to the
list of executing tasks because Task Spooler is aware that it cannot execute until task 28 is complete. The
second time I view the queue, both tasks have completed.
$ ts bash -c "sleep 10; echo hi"
28
$ ts -d echo there
29
$ ts
ID State Output E-Level Times(r/u/s) Command [run=1/2]
28 running /tmp/ts-out.hKqDva bash -c sleep 10; echo hi
29 queued (file) && echo there
$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/2]
28 finished /tmp/ts-out.hKqDva 0 10.01/0.00/0.01 bash -c sleep 10; echo hi
29 finished /tmp/ts-out.VDtVp7 0 0.00/0.00/0.00 && echo there
$ cat /tmp/ts-out.hKqDva
hi
$ cat /tmp/ts-out.VDtVp7
there
You can also explicitly set dependencies on other tasks as shown below. Because the
ts
command
prints the ID of a new task to the console, the first command puts that ID into a shell variable for use in the
second command. The second command passes the task ID of the first task to ts, telling it to wait for the task
with that ID to complete before returning. Because this is joined with the command we wish to execute with the
&&
operation, the second command will execute only if the first one has finished
and
succeeded.
The first time we view the queue you can see that both tasks are running. The first task will be in the
sleep
command that we used explicitly to slow down its execution. The second command will be
executing
ts
, which will be waiting for the first task to complete. One downside of tracking
dependencies this way is that the second command is added to the running queue even though it cannot do
anything until the first task is complete.
$ FIRST_TASKID=`ts bash -c "sleep 10; echo hi"`
$ ts sh -c "ts -w $FIRST_TASKID && echo there"
25
$ ts
ID State Output E-Level Times(r/u/s) Command [run=2/2]
24 running /tmp/ts-out.La9Gmz bash -c sleep 10; echo hi
25 running /tmp/ts-out.Zr2n5u sh -c ts -w 24 && echo there
$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/2]
24 finished /tmp/ts-out.La9Gmz 0 10.01/0.00/0.00 bash -c sleep 10; echo hi
25 finished /tmp/ts-out.Zr2n5u 0 9.47/0.00/0.01 sh -c ts -w 24 && echo there
$ ts -c 24
hi
$ ts -c 25
there
Wrap-up
Task Spooler allows you to convert a shell command to a queued command by simply prepending
ts
to the command line. One major advantage of using ts over something like the
at
command is that
you can effectively run
tail -f
on the output of a running task and also get at the output of
completed tasks from the command line. The utility's ability to execute multiple tasks in parallel is very
handy if you are running on a multicore CPU. Because you can explicitly wait for a task, you can set up very
complex interactions where you might have several tasks running at once and have jobs that depend on multiple
other tasks to complete successfully before they can execute.
Because you can make explicitly dependant tasks take up slots in the actively running task queue, you can
effectively delay the execution of the queue until a time of your choosing. For example, if you queue up a task
that waits for a specific time before returning successfully and have a small group of other tasks that are
dependent on this first task to complete, then no tasks in the queue will execute until the first task
completes.
Run the commands listed in the ' my-at-jobs.txt ' file at 1:35 AM. All output from the job will be mailed to the
user running the task. When this command has been successfully entered you should receive a prompt similar to the example below:
commands will be executed using /bin/sh
job 1 at Wed Dec 24 00:22:00 2014
at -l
This command will list each of the scheduled jobs in a format like the following:
1 Wed Dec 24 00:22:00 2003
...this is the same as running the command atq .
at -r 1
Deletes job 1 . This command is the same as running the command atrm 1 .
atrm 23
Deletes job 23. This command is the same as running the command at -r 23 .
But processing each line until the command is finished then moving to the next one is very
time consuming, I want to process for instance 20 lines at once then when they're finished
another 20 lines are processed.
I thought of wget LINK1 >/dev/null 2>&1 & to send the command
to the background and carry on, but there are 4000 lines here this means I will have
performance issues, not to mention being limited in how many processes I should start at the
same time so this is not a good idea.
One solution that I'm thinking of right now is checking whether one of the commands is
still running or not, for instance after 20 lines I can add this loop:
Of course in this case I will need to append & to the end of the line! But I'm feeling
this is not the right way to do it.
So how do I actually group each 20 lines together and wait for them to finish before going
to the next 20 lines, this script is dynamically generated so I can do whatever math I want
on it while it's being generated, but it DOES NOT have to use wget, it was just an example so
any solution that is wget specific is not gonna do me any good.
wait is the right answer here, but your while [ $(ps would be much
better written while pkill -0 $KEYWORD – using proctools that is, for legitimate reasons to
check if a process with a specific name is still running. – kojiro
Oct 23 '13 at 13:46
I think this question should be re-opened. The "possible duplicate" QA is all about running a
finite number of programs in parallel. Like 2-3 commands. This question, however, is
focused on running commands in e.g. a loop. (see "but there are 4000 lines"). –
VasyaNovikov
Jan 11 at 19:01
@VasyaNovikov Have you readall the answers to both this question and the
duplicate? Every single answer to this question here, can also be found in the answers to the
duplicate question. That is precisely the definition of a duplicate question. It makes
absolutely no difference whether or not you are running the commands in a loop. –
robinCTS
Jan 11 at 23:08
@robinCTS there are intersections, but questions themselves are different. Also, 6 of the
most popular answers on the linked QA deal with 2 processes only. – VasyaNovikov
Jan 12 at 4:09
I recommend reopening this question because its answer is clearer, cleaner, better, and much
more highly upvoted than the answer at the linked question, though it is three years more
recent. – Dan Nissenbaum
Apr 20 at 15:35
For the above example, 4 processes process1 .. process4 would be
started in the background, and the shell would wait until those are completed before starting
the next set ..
Wait until the child process specified by each process ID pid or job specification
jobspec exits and return the exit status of the last command waited for. If a job spec is
given, all processes in the job are waited for. If no arguments are given, all currently
active child processes are waited for, and the return status is zero. If neither jobspec
nor pid specifies an active child process of the shell, the return status is 127.
So basically i=0; waitevery=4; for link in "${links[@]}"; do wget "$link" & ((
i++%waitevery==0 )) && wait; done >/dev/null 2>&1 – kojiro
Oct 23 '13 at 13:48
Unless you're sure that each process will finish at the exact same time, this is a bad idea.
You need to start up new jobs to keep the current total jobs at a certain cap .... parallel is the answer.
– rsaw
Jul 18 '14 at 17:26
I've tried this but it seems that variable assignments done in one block are not available in
the next block. Is this because they are separate processes? Is there a way to communicate
the variables back to the main process? – Bobby
Apr 27 '17 at 7:55
This is better than using wait , since it takes care of starting new jobs as old
ones complete, instead of waiting for an entire batch to finish before starting the next.
– chepner
Oct 23 '13 at 14:35
For example, if you have the list of links in a file, you can do cat list_of_links.txt
| parallel -j 4 wget {} which will keep four wget s running at a time.
– Mr.
Llama
Aug 13 '15 at 19:30
I am using xargs to call a python script to process about 30 million small
files. I hope to use xargs to parallelize the process. The command I am using
is:
Basically, Convert.py will read in a small json file (4kb), do some
processing and write to another 4kb file. I am running on a server with 40 CPU cores. And no
other CPU-intense process is running on this server.
By monitoring htop (btw, is there any other good way to monitor the CPU performance?), I
find that -P 40 is not as fast as expected. Sometimes all cores will freeze and
decrease almost to zero for 3-4 seconds, then will recover to 60-70%. Then I try to decrease
the number of parallel processes to -P 20-30 , but it's still not very fast. The
ideal behavior should be linear speed-up. Any suggestions for the parallel usage of xargs
?
You are most likely hit by I/O: The system cannot read the files fast enough. Try starting
more than 40: This way it will be fine if some of the processes have to wait for I/O. –
Ole Tange
Apr 19 '15 at 8:45
I second @OleTange. That is the expected behavior if you run as many processes as you have
cores and your tasks are IO bound. First the cores will wait on IO for their task (sleep),
then they will process, and then repeat. If you add more processes, then the additional
processes that currently aren't running on a physical core will have kicked off parallel IO
operations, which will, when finished, eliminate or at least reduce the sleep periods on your
cores. – PSkocik
Apr 19 '15 at 11:41
1- Do you have hyperthreading enabled? 2- in what you have up there, log.txt is actually
overwritten with each call to convert.py ... not sure if this is the intended behavior or
not. – Bichoy
Apr 20 '15 at 3:32
I'd be willing to bet that your problem is python . You didn't say what kind of processing is
being done on each file, but assuming you are just doing in-memory processing of the data,
the running time will be dominated by starting up 30 million python virtual machines
(interpreters).
If you can restructure your python program to take a list of files, instead of just one,
you will get a huge improvement in performance. You can then still use xargs to further
improve performance. For example, 40 processes, each processing 1000 files:
This isn't to say that python is a bad/slow language; it's just not optimized for startup
time. You'll see this with any virtual machine-based or interpreted language. Java, for
example, would be even worse. If your program was written in C, there would still be a cost
of starting a separate operating system process to handle each file, but it would be much
less.
From there you can fiddle with -P to see if you can squeeze out a bit more
speed, perhaps by increasing the number of processes to take advantage of idle processors
while data is being read/written.
What is the constraint on each job? If it's I/O you can probably get away with
multiple jobs per CPU core up till you hit the limit of I/O, but if it's CPU intensive, its
going to be worse than pointless running more jobs concurrently than you have CPU cores.
My understanding of these things is that GNU Parallel would give you better control over
the queue of jobs etc.
As others said, check whether you're I/O-bound. Also, xargs' man page suggests using
-n with -P , you don't mention the number of
Convert.py processes you see running in parallel.
As a suggestion, if you're I/O-bound, you might try using an SSD block device, or try
doing the processing in a tmpfs (of course, in this case you should check for enough memory,
avoiding swap due to tmpfs pressure (I think), and the overhead of copying the data to it in
the first place).
I want the ability to schedule commands to be run in a FIFO queue. I DON'T want them to be
run at a specified time in the future as would be the case with the "at" command. I want them
to start running now, but not simultaneously. The next scheduled command in the queue should
be run only after the first command finishes executing. Alternatively, it would be nice if I
could specify a maximum number of commands from the queue that could be run simultaneously;
for example if the maximum number of simultaneous commands is 2, then only at most 2 commands
scheduled in the queue would be taken from the queue in a FIFO manner to be executed, the
next command in the remaining queue being started only when one of the currently 2 running
commands finishes.
I've heard task-spooler could do something like this, but this package doesn't appear to
be well supported/tested and is not in the Ubuntu standard repositories (Ubuntu being what
I'm using). If that's the best alternative then let me know and I'll use task-spooler,
otherwise, I'm interested to find out what's the best, easiest, most tested, bug-free,
canonical way to do such a thing with bash.
UPDATE:
Simple solutions like ; or && from bash do not work. I need to schedule these
commands from an external program, when an event occurs. I just don't want to have hundreds
of instances of my command running simultaneously, hence the need for a queue. There's an
external program that will trigger events where I can run my own commands. I want to handle
ALL triggered events, I don't want to miss any event, but I also don't want my system to
crash, so that's why I want a queue to handle my commands triggered from the external
program.
That will list the directory. Only after ls has run it will run touch
test which will create a file named test. And only after that has finished it will run
the next command. (In this case another ls which will show the old contents and
the newly created file).
Similar commands are || and && .
; will always run the next command.
&& will only run the next command it the first returned success.
Example: rm -rf *.mp3 && echo "Success! All MP3s deleted!"
|| will only run the next command if the first command returned a failure
(non-zero) return value. Example: rm -rf *.mp3 || echo "Error! Some files could not be
deleted! Check permissions!"
If you want to run a command in the background, append an ampersand ( &
).
Example: make bzimage & mp3blaster sound.mp3 make mytestsoftware ; ls ; firefox ; make clean
Will run two commands int he background (in this case a kernel build which will take some
time and a program to play some music). And in the foregrounds it runs another compile job
and, once that is finished ls, firefox and a make clean (all sequentially)
For more details, see man bash
[Edit after comment]
in pseudo code, something like this?
Program run_queue:
While(true)
{
Wait_for_a_signal();
While( queue not empty )
{
run next command from the queue.
remove this command from the queue.
// If commands where added to the queue during execution then
// the queue is not empty, keep processing them all.
}
// Queue is now empty, returning to wait_for_a_signal
}
//
// Wait forever on commands and add them to a queue
// Signal run_quueu when something gets added.
//
program add_to_queue()
{
While(true)
{
Wait_for_event();
Append command to queue
signal run_queue
}
}
The easiest way would be to simply run the commands sequentially:
cmd1; cmd2; cmd3; cmdN
If you want the next command to run only if the previous command exited
successfully, use && :
cmd1 && cmd2 && cmd3 && cmdN
That is the only bash native way I know of doing what you want. If you need job control
(setting a number of parallel jobs etc), you could try installing a queue manager such as
TORQUE but that
seems like overkill if all you want to do is launch jobs sequentially.
You are looking for at 's twin brother: batch . It uses the same
daemon but instead of scheduling a specific time, the jobs are queued and will be run
whenever the system load average is low.
Apart from dedicated queuing systems (like the Sun Grid Engine ) which you can also
use locally on one machine and which offer dozens of possibilities, you can use something
like
command1 && command2 && command3
which is the other extreme -- a very simple approach. The latter neither does provide
multiple simultaneous processes nor gradually filling of the "queue".
task spooler is a Unix batch system where the tasks spooled run one after the other. The
amount of jobs to run at once can be set at any time. Each user in each system has his own
job queue. The tasks are run in the correct context (that of enqueue) from any shell/process,
and its output/results can be easily watched. It is very useful when you know that your
commands depend on a lot of RAM, a lot of disk use, give a lot of output, or for whatever
reason it's better not to run them all at the same time, while you want to keep your
resources busy for maximum benfit. Its interface allows using it easily in scripts.
For your first contact, you can read an article at linux.com , which I like
as overview, guide and
examples(original url) .
On more advanced usage, don't neglect the TRICKS file in the package.
Features
I wrote Task Spooler because I didn't have any comfortable way of running batch
jobs in my linux computer. I wanted to:
Queue jobs from different terminals.
Use it locally in my machine (not as in network queues).
Have a good way of seeing the output of the processes (tail, errorlevels, ...).
Easy use: almost no configuration.
Easy to use in scripts.
At the end, after some time using and developing ts , it can do something
more:
It works in most systems I use and some others, like GNU/Linux, Darwin, Cygwin, and
FreeBSD.
No configuration at all for a simple queue.
Good integration with renice, kill, etc. (through `ts -p` and process
groups).
Have any amount of queues identified by name, writting a simple wrapper script for each
(I use ts2, tsio, tsprint, etc).
Control how many jobs may run at once in any queue (taking profit of multicores).
It never removes the result files, so they can be reached even after we've lost the
ts task list.
I created a GoogleGroup for the program. You look for the archive and the join methods in
the taskspooler google
group page .
Alessandro Öhler once maintained a mailing list for discussing newer functionalities
and interchanging use experiences. I think this doesn't work anymore , but you can
look at the old archive
or even try to subscribe .
How
it works
The queue is maintained by a server process. This server process is started if it isn't
there already. The communication goes through a unix socket usually in /tmp/ .
When the user requests a job (using a ts client), the client waits for the server message to
know when it can start. When the server allows starting , this client usually forks, and runs
the command with the proper environment, because the client runs run the job and not
the server, like in 'at' or 'cron'. So, the ulimits, environment, pwd,. apply.
When the job finishes, the client notifies the server. At this time, the server may notify
any waiting client, and stores the output and the errorlevel of the finished job.
Moreover the client can take advantage of many information from the server: when a job
finishes, where does the job output go to, etc.
Download
Download the latest version (GPLv2+ licensed): ts-1.0.tar.gz - v1.0
(2016-10-19) - Changelog
Look at the version repository if you are
interested in its development.
Андрей
Пантюхин (Andrew Pantyukhin) maintains the
BSD port .
Eric Keller wrote a nodejs web server showing the status of the task spooler queue (
github project
).
Manual
Look at its manpage (v0.6.1). Here you also
have a copy of the help for the same version:
usage: ./ts [action] [-ngfmd] [-L <lab>] [cmd...]
Env vars:
TS_SOCKET the path to the unix socket used by the ts command.
TS_MAILTO where to mail the result (on -m). Local user by default.
TS_MAXFINISHED maximum finished jobs in the queue.
TS_ONFINISH binary called on job end (passes jobid, error, outfile, command).
TS_ENV command called on enqueue. Its output determines the job information.
TS_SAVELIST filename which will store the list, if the server dies.
TS_SLOTS amount of jobs which can run at once, read on server start.
Actions:
-K kill the task spooler server
-C clear the list of finished jobs
-l show the job list (default action)
-S [num] set the number of max simultanious jobs of the server.
-t [id] tail -f the output of the job. Last run if not specified.
-c [id] cat the output of the job. Last run if not specified.
-p [id] show the pid of the job. Last run if not specified.
-o [id] show the output file. Of last job run, if not specified.
-i [id] show job information. Of last job run, if not specified.
-s [id] show the job state. Of the last added, if not specified.
-r [id] remove a job. The last added, if not specified.
-w [id] wait for a job. The last added, if not specified.
-u [id] put that job first. The last added, if not specified.
-U <id-id> swap two jobs in the queue.
-h show this help
-V show the program version
Options adding jobs:
-n don't store the output of the command.
-g gzip the stored output (if not -n).
-f don't fork into background.
-m send the output by e-mail (uses sendmail).
-d the job will be run only if the job before ends well
-L <lab> name this task with a label, to be distinguished on listing.
Thanks
To Raúl Salinas, for his inspiring ideas
To Alessandro Öhler, the first non-acquaintance user, who proposed and created the
mailing list.
Андрею
Пантюхину, who created the BSD
port .
To the useful, although sometimes uncomfortable, UNIX interface.
To Alexander V. Inyukhin, for the debian packages.
To Pascal Bleser, for the SuSE packages.
To Sergio Ballestrero, who sent code and motivated the development of a multislot version
of ts.
To GNU, an ugly but working and helpful ol' UNIX implementation.
The time-based job scheduler cron(8)
has been around since Version 7 Unix, and its
crontab(5) syntax is
familiar even for people who don't do much Unix system administration. It's
standardised
, reasonably flexible, simple to configure, and works reliably, and so it's trusted by both system
packages and users to manage many important tasks.
However, like many older Unix tools, cron(8) 's simplicity has a drawback: it relies
upon the user to know some detail of how it works, and to correctly implement any other safety checking
behaviour around it. Specifically, all it does is try and run the job at an appropriate time, and
email the output. For simple and unimportant per-user jobs, that may be just fine, but for more crucial
system tasks it's worthwhile to wrap a little extra infrastructure around it and the tasks it calls.
There are a few ways to make the way you use cron(8) more robust if you're in a situation
where keeping track of the running job is desirable.
Apply the principle of least privilege
The sixth column of a system crontab(5) file is the username of the user as which
the task should run:
0 * * * * root cron-task
To the extent that is practical, you should run the task as a user with only the privileges it
needs to run, and nothing else. This can sometimes make it worthwhile to create a dedicated system
user purely for running scheduled tasks relevant to your application.
0 * * * * myappcron cron-task
This is not just for security reasons, although those are good ones; it helps protect you against
nasties like scripting errors attempting to
remove entire
system directories .
Similarly, for tasks with database systems such as MySQL, don't use the administrative root
user if you can avoid it; instead, use or even create a dedicated user with a unique random password
stored in a locked-down ~/.my.cnf file, with only the needed permissions. For a MySQL
backup task, for example, only a few permissions should be required, including SELECT
, SHOW VIEW , and LOCK TABLES .
In some cases, of course, you really will need to be root . In particularly sensitive
contexts you might even consider using sudo(8) with appropriate NOPASSWD
options, to allow the dedicated user to run only the appropriate tasks as root , and
nothing else.
Test the tasks
Before placing a task in a crontab(5) file, you should test it on the command line,
as the user configured to run the task and with the appropriate environment set. If you're going
to run the task as root , use something like su or sudo -i
to get a root shell with the user's expected environment first:
$ sudo -i -u cronuser
$ cron-task
Once the task works on the command line, place it in the crontab(5) file with the
timing settings modified to run the task a few minutes later, and then watch /var/log/syslog
with tail -f to check that the task actually runs without errors, and that the task
itself completes properly:
May 7 13:30:01 yourhost CRON[20249]: (you) CMD (cron-task)
This may seem pedantic at first, but it becomes routine very quickly, and it saves a lot of hassles
down the line as it's very easy to make an assumption about something in your environment that doesn't
actually hold in the one that cron(8) will use. It's also a necessary acid test to make
sure that your crontab(5) file is well-formed, as some implementations of cron(8)
will refuse to load the entire file if one of the lines is malformed.
If necessary, you can set arbitrary environment variables for the tasks at the top of the file:
MYVAR=myvalue
0 * * * * you cron-task
Don't throw away errors or useful output
You've probably seen tutorials on the web where in order to keep the crontab(5) job
from sending standard output and/or standard error emails every five minutes, shell redirection operators
are included at the end of the job specification to discard both the standard output and standard
error. This kluge is particularly common for running web development tasks by automating a request
to a URL with curl(1)
or wget(1) :
Ignoring the output completely is generally not a good idea, because unless you have other tasks
or monitoring ensuring the job does its work, you won't notice problems (or know what they are),
when the job emits output or errors that you actually care about.
In the case of curl(1) , there are just way too many things that could go wrong,
that you might notice far too late:
The script could get broken and return 500 errors.
The URL of the cron.php task could change, and someone could forget to add a
HTTP 301 redirect.
Even if a HTTP 301 redirect is added, if you don't use -L or --location
for curl(1) , it won't follow it.
The client could get blacklisted, firewalled, or otherwise impeded by automatic or manual
processes that falsely flag the request as spam.
If using HTTPS, connectivity could break due to cipher or protocol mismatch.
The author has seen all of the above happen, in some cases very frequently.
As a general policy, it's worth taking the time to read the manual page of the task you're calling,
and to look for ways to correctly control its output so that it emits only the output you actually
want. In the case of curl(1) , for example, I've found the following formula works well:
curl -fLsS -o /dev/null http://example.com/
-f : If the HTTP response code is an error, emit an error message rather than
the 404 page.
-L : If there's an HTTP 301 redirect given, try to follow it.
-sS : Don't show progress meter ( -S stops -s from
also blocking error messages).
-o /dev/null : Send the standard output (the actual page returned) to /dev/null
.
This way, the curl(1) request should stay silent if everything is well, per the old
Unix philosophy Rule of Silence
.
You may not agree with some of the choices above; you might think it important to e.g. log the
complete output of the returned page, or to fail rather than silently accept a 301 redirect, or you
might prefer to use wget(1) . The point is that you take the time to understand in more
depth what the called program will actually emit under what circumstances, and make it match your
requirements as closely as possible, rather than blindly discarding all the output and (worse) the
errors. Work with Murphy's law
; assume that anything that can go wrong eventually will.
Send the output somewhere useful
Another common mistake is failing to set a useful MAILTO at the top of the
crontab(5) file, as the specified destination for any output and errors from the tasks.
cron(8) uses the system mail implementation to send its messages, and typically, default
configurations for mail agents will simply send the message to an mbox file in
/var/mail/$USER , that they may not ever read. This defeats much of the point of mailing output
and errors.
This is easily dealt with, though; ensure that you can send a message to an address you actually
do check from the server, perhaps using mail(1) :
Once you've verified that your mail agent is correctly configured and that the mail arrives in
your inbox, set the address in a MAILTO variable at the top of your file:
[email protected]
0 * * * * you cron-task-1
*/5 * * * * you cron-task-2
If you don't want to use email for routine output, another method that works is sending the output
to syslog with a tool like
logger(1) :
0 * * * * you cron-task | logger -it cron-task
Alternatively, you can configure aliases on your system to forward system mail destined for you
on to an address you check. For Postfix, you'd use an
aliases(5) file.
I sometimes use this setup in cases where the task is expected to emit a few lines of output which
might be useful for later review, but send stderr output via MAILTO as
normal. If you'd rather not use syslog , perhaps because the output is high in volume
and/or frequency, you can always set up a log file /var/log/cron-task.log but don't
forget to add a logrotate(8)
rule for it!
Put the tasks in their own shell script file
Ideally, the commands in your crontab(5) definitions should only be a few words,
in one or two commands. If the command is running off the screen, it's likely too long to be in the
crontab(5) file, and you should instead put it into its own script. This is a particularly
good idea if you want to reliably use features of bash or some other shell besides POSIX/Bourne
/bin/sh for your commands, or even a scripting language like Awk or Perl; by default,
cron(8) uses the system's /bin/sh implementation for parsing the commands.
Because crontab(5) files don't allow multi-line commands, and have other gotchas
like the need to escape percent signs % with backslashes, keeping as much configuration
out of the actual crontab(5) file as you can is generally a good idea.
If you're running cron(8) tasks as a non-system user, and can't add scripts into
a system bindir like /usr/local/bin , a tidy method is to start your own, and include
a reference to it as part of your PATH . I favour ~/.local/bin , and have
seen references to ~/bin as well. Save the script in ~/.local/bin/cron-task
, make it executable with chmod +x , and include the directory in the PATH
environment definition at the top of the file:
PATH=/home/you/.local/bin:/usr/local/bin:/usr/bin:/bin
[email protected]
0 * * * * you cron-task
Having your own directory with custom scripts for your own purposes has a host of other benefits,
but that's another article
Avoid /etc/crontab
If your implementation of cron(8) supports it, rather than having an /etc/crontab
file a mile long, you can put tasks into separate files in /etc/cron.d :
$ ls /etc/cron.d
system-a
system-b
raid-maint
This approach allows you to group the configuration files meaningfully, so that you and other
administrators can find the appropriate tasks more easily; it also allows you to make some files
editable by some users and not others, and reduces the chance of edit conflicts. Using sudoedit(8)
helps here too. Another advantage is that it works better with version control; if I start collecting
more than a few of these task files or to update them more often than every few months, I start a
Git repository to track them:
If you're editing a crontab(5) file for tasks related only to the individual user,
use the crontab(1) tool; you can edit your own crontab(5) by typing
crontab -e , which will open your $EDITOR to edit a temporary file that
will be installed on exit. This will save the files into a dedicated directory, which on my system
is /var/spool/cron/crontabs .
On the systems maintained by the author, it's quite normal for /etc/crontab never
to change from its packaged template.
Include a timeout
cron(8) will normally allow a task to run indefinitely, so if this is not desirable,
you should consider either using options of the program you're calling to implement a timeout, or
including one in the script. If there's no option for the command itself, the
timeout(1) command
wrapper in coreutils is one possible way of implementing this:
cron(8) will start a new process regardless of whether its previous runs have completed,
so if you wish to avoid locking for long-running task, on GNU/Linux you could use the
flock(1) wrapper for
the flock(2) system call
to set an exclusive lockfile, in order to prevent the task from running more than one instance in
parallel.
0 * * * * you flock -nx /var/lock/cron-task cron-task
Greg's wiki has some more in-depth discussion of the
file locking problem for scripts
in a general sense, including important information about the caveats of "rolling your own" when
flock(1) is not available.
If it's important that your tasks run in a certain order, consider whether it's necessary to have
them in separate tasks at all; it may be easier to guarantee they're run sequentially by collecting
them in a single shell script.
Do something useful with exit statuses
If your cron(8) task or commands within its script exit non-zero, it can be useful
to run commands that handle the failure appropriately, including cleanup of appropriate resources,
and sending information to monitoring tools about the current status of the job. If you're using
Nagios Core or one of its derivatives, you could consider using send_nsca to send passive
checks reporting the status of jobs to your monitoring server. I've written
a simple script called
nscaw to do this for me:
0 * * * * you nscaw CRON_TASK -- cron-task
Consider alternatives to cron(8)
If your machine isn't always on and your task doesn't need to run at a specific time, but rather
needs to run once daily or weekly, you can install
anacron and drop scripts
into the cron.hourly , cron.daily , cron.monthly , and
cron.weekly directories in /etc , as appropriate. Note that on Debian and
Ubuntu GNU/Linux systems, the default /etc/crontab contains hooks that run these, but
they run only if anacron(8)
is not installed.
If you're using cron(8) to poll a directory for changes and run a script if there
are such changes, on GNU/Linux you could consider using a daemon based on inotifywait(1)
instead.
Finally, if you require more advanced control over when and how your task runs than cron(8)
can provide, you could perhaps consider writing a daemon to run on the server consistently and fork
processes for its task. This would allow running a task more often than once a minute, as an example.
Don't get too bogged down into thinking that cron(8) is your only option for any kind
of asynchronous task management!
Oftentimes you may
wish to start a process on the Bash shell without having to wait for it to actually complete,
but still be notified when it does. Similarly, it may be helpful to temporarily stop a task
while it's running without actually quitting it, so that you can do other things with the
terminal. For these kinds of tasks, Bash's built-in job control is very useful.
Backgrounding processes
If you have a process that you expect to take a long time, such as a long cp or
scp operation, you can start it in the background of your current shell by adding
an ampersand to it as a suffix:
$ cp -r /mnt/bigdir /home &
[1] 2305
This will start the copy operation as a child process of your bash instance,
but will return you to the prompt to enter any other commands you might want to run while
that's going.
The output from this command shown above gives both the job number of 1, and the process ID
of the new task, 2305. You can view the list of jobs for the current shell with the builtin
jobs :
$ jobs
[1]+ Running cp -r /mnt/bigdir /home &
If the job finishes or otherwise terminates while it's backgrounded, you should see a
message in the terminal the next time you update it with a newline:
[1]+ Done cp -r /mnt/bigdir /home &
Foregrounding processes
If you want to return a job in the background to the foreground, you can type
fg :
$ fg
cp -r /mnt/bigdir /home &
If you have more than one job backgrounded, you should specify the particular job to bring
to the foreground with a parameter to fg :
$ fg %1
In this case, for shorthand, you can optionally omit fg and it will work just
the same:
$ %1
Suspending processes
To temporarily suspend a process, you can press Ctrl+Z:
You can then continue it in the foreground or background with fg %1 or bg
%1 respectively, as above.
This is particularly useful while in a text editor; instead of quitting the editor to get
back to a shell, or dropping into a subshell from it, you can suspend it temporarily and return
to it with fg once you're ready.
Dealing with output
While a job is running in the background, it may still print its standard output and
standard error streams to your terminal. You can head this off by redirecting both streams to
/dev/null for verbose commands:
$ cp -rv /mnt/bigdir /home &>/dev/null
However, if the output of the task is actually of interest to you, this may be a case where
you should fire up another terminal emulator, perhaps in GNU Screen or tmux , rather than using simple job control.
Suspending SSH
sessions
As a special case, you can suspend an SSH session using an SSH escape sequence . Type a
newline followed by a ~ character, and finally press Ctrl+Z to background your SSH session and
return to the terminal from which you invoked it.
...The crontab (chronological table)
command maintains a list of jobs for cron to execute. Each user has his or her own crontab
table. The -l (list) switch lists currently scheduled tasks.
Linux reports an error if you don't have permission to use cron. Because jobs are added
or removed from the crontab table as a group, always start with the -l switch,
saving the current table to a file.
$ crontab -l > cron.txt
After the current table is saved, the file can be edited. There are five
columns for specifying the times when a program is to run: The minute, hour, day, month, and the
day of the week. Unused columns are marked with an asterisk, indicating any appropriate time.
Times are represented in a variety of formats: Individually (1), comma-separated
lists (1,15), ranges (0-6, 9-17), and ranges with step values (1-31/2). Names can be used for months
or days of the week.
The final column contains the name of the command to execute. The following line
runs a script called cleanup.sh at 1:00 AM every morning.
* 1 * * * /home/kburtch/cleanup.sh
Environment variables can also be initialized in the crontab. When a
shell script is started by cron, it is not started from a login session and none of the
profile files are executed. Only a handful of variables are defined: PWD, HOSTNAME,
MACHTYPE, LOGNAME, SHLVL, SHELL, HOSTTYPE, OSTYPE,
HOME, TERM, and PATH. You have to explicitly set any other values in the
script or in the crontab list.
PATH is defined as only /usr/bin:/bin. Other paths are normally
added by profile files and so are unavailable.
Because a script running under cron is not in a login session, there is no screen
to write standard output to. Anything that is normally written to standard output is instead captured
by cron and mailed to the account owning the cron script. The mail has the unhelpful
subject line of cron. Even printing a blank line results in a seemingly empty email being
sent. For this reason, scripts designed to run under cron should either write their output
to a log file, or should create and forward their own email with a meaningful subject line. It is
common practice to write a wrapper script to capture the output from the script doing the actual
work.
#!/bin/bash
#
# show_users.sh: show all users in the database table "users"
shopt -s -o nounset
declare -rx SCRIPT=${0##*/}
declare -r SQL_CMDS="sort_inventory.sql"
declare -rx ON_ERROR_STOP
if [ ! -r "$SQL_CMDS" ] ; then
printf "$SCRIPT: the SQL script $SQL_CMDS doesn't exist or is not \
readable" >&2
exit 192
fi
RESULTS=`psql -user gordon -dbname custinfo –quiet -no-align -tuples-only \
-field-separator "," -file "$SQL_CMDS"`
if [ $? -ne 0 ] ; then
printf "$SCRIPT: SQL statements failed." >&2
exit 192
fi
#!/bin/bash
# show_users_wrapper.sh - show_users.sh wrapper script
shopt -s -o nounset
declare -rx SCRIPT=${0##*/}
declare -rx USER="kburtch"
declare -rx mail="/bin/mail"
declare -rx OUTPUT='mktemp /tmp/script_out.XXXXXX'
declare -rx SCRIPT2RUN="./show_users.sh"
# sanity checks
if test ! -x "$mail" ; then
printf "$SCRIPT:$LINENO: the command $mail is not available - aborting" >&2
exit 1
fi
if test ! -x "$SCRIPT2RUN" ; then
printf "$SCRIPT: $LINENO: the command $SCRIPT2RUN is not available\
- aborting" >&2
exit 1
fi
# record the date for any errors, and create the OUTPUT file
date > $OUTPUT
# run the script
$SCRIPT2RUN 2>&1 > "$OUTPUT"
# mail errors to USER
if [ $? -ne 0 ] ; then
$mail -s "$SCRIPT2RUN failed" "$USER" < "$OUTPUT"
fi
# cleanup
rm "$OUTPUT"
exit 0
It is possible to run gui applications via cronjobs. This can be done by telling cron which display
to use.
00 06 * * * env DISPLAY=:0 gui_appname
The env DISPLAY=:0 portion will tell cron to use the current display (desktop) for the
program "gui_appname".
And if you have multiple monitors, don't forget to specify on which one the program is to be run.
For example, to run it on the first screen (default screen) use :
00 06 * * * env DISPLAY=:0.0 gui_appname
The env DISPLAY=:0.0 portion will tell cron to use the first screen of the current display
for the program "gui_appname".
Note: GUI users may prefer to use gnome-schedule (aka "Scheduled tasks") to configure
GUI cron jobs. In gnome-schedule, when editing a GUI task, you have to select "X application" in
a dropdown next to the command field.
Note: In Karmic(9.10), you have to enable X ACL for localhost to connect to for
GUI applications to work.
~$ xhost +local:
non-network local connections being added to access control list
~$ xhost
access control enabled, only authorized clients can connect
LOCAL:
...
Tips
crontab -e uses the EDITOR environment variable. to change the editor to your
own choice just set that. You may want to set EDITOR in you .bashrc because many commands use this
variable. Let's set the EDITOR to nano a very easy editor to use:
export EDITOR=nano
There are also files you can edit for system-wide cron jobs. The most common file is located at
/etc/crontab, and this file follows a slightly different syntax than a normal crontab
file. Since it is the base crontab that applies system-wide, you need to specify what user to run
the job as; thus, the syntax is now:
minute(s) hour(s) day(s)_of_month month(s) day(s)_of_week user command
It is recommended, however, that you try to avoid using /etc/crontab unless you
need the flexibility offered by it, or if you'd like to create your own simplified anacron-like
system using run-parts for example. For all cron jobs that you want to have run
under your own user account, you should stick with using crontab -e to edit your
local cron jobs rather than editting the system-wide /etc/crontab.
Crontab Example
Below is an example of how to setup a crontab to run updatedb, which updates the slocate database:
Open a term, type "crontab -e" (without the double quotes) and press enter. Type the following line,
substituting the full path of the application you wish to run for the one shown below, into the editor:
45 04 * * * /usr/bin/updatedb
Save your changes and exit the editor.
Crontab will let you know if you made any mistakes. The crontab will be installed and begin running
if there are no errors. That's it. You now have a cronjob setup to run updatedb, which updates the
slocate database, every morning at 4:45.
Note: The double-ampersand (&&) can also be used in the "command" section to run multiple commands
consecutively, but only if the previous command exits successfully. A string of commands joined by
the double-ampersand will only get to the last command if all the previous commands are run successfully.
If exit error-checking is not of a concern, string commands together, separated with a semi-colon
(;)
The above example will run chkrootkit followed by updatedb at 4:45am daily - providing you have
all listed apps installed. If chkrootkit fails, updatedb will NOT be run.
Hi All,
I created a crontab entry in a cron.txt file accidentally entered
crontab cron.txt.
Now my previous crontab -l entries are not showing up, that means i removed the scheduling
of the previous jobs by running this command "crontab cron.txt"
How do I revert back to previously schedule jobs.
Please help. this is urgent.,
Thanks.
lestat_ecuador:
Try to go to /tmp and do
Code:
ls -ltr cron*
Then will appear kind a history of the cron, see the creation dates an look into the one you want.
For example I have:
Code:
This is a quick advice of those who have ran crontab -r accidentally and would like to restore
it. Well letter r and letter e is close together therefore this is a very easy mistake to do.
If you don't have a backup of /var/spool/cron/*user* the only way you could find to
recover you crontab commands would be taking a look at the logs of your system.
For example in RedHat / CentOS Linux distributions you can issue: # cat /var/log/cron And you
will see a list of commands executed and the time, from there you will be able to rebuild your crontab.
Unfortunately incron doesn't have recursive auto subscriptions, i.e. if I want to watch an entire
tree and automatically subscribe to new directories being created. look at lsyncd, although it is primarily
meant as a syncing tool, you can configure any action on an event.
incron is very similar in concept and usage to using cron, as the interface is a clone
of it.
Each user who is allowed to use incron may use the incrontab command to
view, or edit, their rule list. These rules are processed via the daemon, and when a match occurs
the relevant command is executed.
To list the current rules you've got defined run "incrontab -l", and to edit them use
"incrontab -e". If you do that just now you'll receive the following error message:
rt:~# incrontab -l
user 'root' is not allowed to use incron
This error may be fixed in one of two ways:
Allow the root user to make use of incron: By editing /etc/incron.allow,
adding 'root' to it.
Allowing all local users the ability to use incron: By removing the file /etc/incron.allow.
The user table rows have the following syntax (use one or more spaces between elements):
[Path] [mask] [command]
Where:
Path is a filesystem path (each whitespace must be prepended by a backslash)
mask is a symbolic (use commas for separating symbols) or numeric mask for events
command is an application or script to run on the events
The full list of supported flags for mask include:
This small script backup all files in the etc and myProject directory.
vi /root/inotify.sh
--
#!/bin/sh
# Create a inotify backup dir (if not exists)
#
mkdir /var/backups/inotify
# Make a copy off the full path and file
#
cp -p --parents $1 /var/backups/inotify
# move the file to a file with datetime-stamp
#
mv /var/backups/inotify$1 /var/backups/inotify$1_`date +'%Y-%m-%d_%H:%M'
This module provides a simple but complete cron like scheduler. I.e this modules can be used for
periodically executing Perl subroutines. The dates and parameters for the subroutines to be called
are specified with a format known as crontab entry (METHODS, add_entry()
and crontab(5))
The philosophy behind Schedule::Cron is to call subroutines periodically from within
one single Perl program instead of letting cron trigger several (possibly different)
perl scripts. Everything under one roof. Furthermore Schedule::Cron provides mechanism
to create crontab entries dynamically, which isn't that easy with cron.
Schedule::Cron knows about all extensions (well, at least all extensions I'm aware
of, i.e those of the so called "Vixie'' cron) for crontab entries like ranges including 'steps',
specification of month and days of the week by name or coexistence of lists and ranges in the same
field. And even a bit more (like lists and ranges with symbolic names).
There is an easy way to start a program during system boot. Just put this in your crontab: @reboot /path/to/my/program
The command will be executed on every (re)boot. Crontab can be modified by running
#crontab -e
Other available Options
string meaning
------ -----------
@reboot Run once, at startup.
@yearly Run once a year, "0 0 1 1 *".
@annually (same as @yearly)
@monthly Run once a month, "0 0 1 * *".
@weekly Run once a week, "0 0 * * 0".
@daily Run once a day, "0 0 * * *".
@midnight (same as @daily)
@hourly Run once an hour, "0 * * * *"
More information about crontab options is available in the man page check here
How to Use percentage sign (%) in a crontab entry
Usually, a % is used to denote a new line in a crontab entry. The first % is special in that it denotes
the start of STDIN for the crontab entry's command. A trivial example is:
* * * * * cat - % another minute has passed
This would output the text
another minute has passed
After the first %, all other %s in a crontab entry indicate a new line. So a slightly different trivial
example is:
* * * * * cat - % another % minute % has % passed
This would output the text
another
minute
has
passed
Note how the % has been used to indicate a new line.
The problem is how to use a % in a crontab line to as a % and not as a new line. Many manuals
will say escape it with a \. This certainly stops its interpretation as a new line but the shell
running the cron job can leave the \ in. For example:
* * * * * echo '\% another \% minute \% has \% passed'
would output the text
\% another \% minute \% has \% passed
Clearly, not what was intended.
A solution is to pass the text through sed. The crontab example now becomes:
* * * * * echo '\% another \% minute \% has \% passed'| sed -e 's|\\||g'
This would output the text
% another % minute % has % passed
which is what was intended.
This technique is very useful when using a MySQL command within a crontab. MySQL command can often
have a % in them. Some example are:
SET @monyy=DATE_FORMAT(NOW(),"%M %Y")
SELECT * FROM table WHERE name LIKE 'fred%'
So, to have a crontab entry to run the MySQL command
mysql -vv -e "SELECT * FROM table WHERE name LIKE Fred%'" member_list
would have to appear in the crontab as
echo "SELECT * FROM table WHERE name LIKE 'Fred\%'" | sed -e 's|\\||g' | mysql -vv member_list
Pulling the crontab entry apart there is:
the echo command sends the MySQL command to STDOUT where it is piped into
sed which removes any back slashes before sending the output to STDOUT where it is piped into
the mysql command processor which reads its commands from STDIN
Windows-compatible implementation of cron in Python
This article will discuss using a Cron type system, as used on Unix and Linux systems, to bring
the flexibility, scalability and a need for more out of a task automation tool, to the Win32 environment.
Internals: Replacing Task Scheduler with Cron
As our dependency upon machines for various tasks grow, time becomes an integral factor that may
work against us, when pressed upon deadlines. Automation of such tasks, without the need for human
intervention, becomes vital to whether one is able to square away enough time to complete more and
more tasks with little help from human hands.
Task Scheduler, which comes bundled with Windows attempts to make automation of tasks effortless.
Unfortunately, it is not very configurable and basic in what it is capable of.
On Unix and Linux systems, Cron is what is used for task scheduling. This scheduler is very configurable,
and is capable of well more then its Windows counterpart.
This article will discuss using a Cron type system, as used on Unix and Linux systems, to bring
the flexibility, scalability and a need for more out of a task automation tool, to the Win32 environment.
Install the product and make sure to install it as a service (default option on the last dialog
box of the installer) if your Win32 operating system supports this.
Then click Start->Run->services.msc (hit enter)
Scroll down to and highlight Task Scheduler->right click->Properties->Toggle to Manual->hit Stop->then
Apply->OK
Then scroll up to Python Cron Service->highlight->right click->Properties->Toggle to Automatic->Apply->OK
We will be working with a file called crontab.txt for all of the scheduled entries. This file
must be created in the Pycron Program directory which is located in the pycron folder under the Program
Files folder.
Create a new file called crontab.txt in the Pycron Program Directory and put this in to it
* * * * * replace replace
Save your file.
Now launch the crontab editor (Start->Programs->Pycron->Pycron CronTab Editor)
By default it will load up the contents of the crontab.txt file in the Pycron Program Directory.
The parameters of a crontab entry from left to right are as follows.
0-59 - Minute
0-23 - Hour
1-31 - Day
1-12 - Month
0-6 - Day of the week (0 For Mon, and 6 for Sunday)
Command/Application to execute
Parameter to the application/command.
entry:
Minute Hour Day Month Day_of_the_week Command Parameter
Note:
* is a wildcard and matches all values
/ is every (ex: every 10 minutes = */10) (new in this Win32 version)
, is execute each value (ex: every 10 minutes = 0,10,20,30,40,50)
- is to execute each value in the range (ex: from 1st (:01) to 10th (:10) minute = 1-10)
Double click the 1st entry of " * * * * replace replace" to edit the entry.
For an example, we will run defrag on every Friday at 11:00 (23:00) PM against the C:\ volume.
On the Command line hit Browse, and navigate to your System32 Folder inside your Windows folder
and double click on defrag.exe
On the Parameter line enter in c:
Always run a Test Execution to make sure your command is valid. If all was successful, you will
see your command/application run and a kick back message of Successful will be observed.
For Minute, erase the * and enter in 0
For Hour, erase the * and enter in 23
For Week Day enter in 4.
Then hit OK, File->Save.
Note: You can use the wizard to enter in values for each entry as well.
Now open up a command prompt (start->run->cmd), and type:
net start pycron
You can leave it running now, and every time you append and or change your crontab.txt, the schedule
will be updated.
To add another entry using the crontab GUI, add in a * * * * * replace replace to crontab.txt
on the next free line, save it, then open up crontab.txt with the GUI editor and make the desired
changes on the entry by double clicking to edit.
It is recommended that every time the GUI is used to edit entries, that you observe the entry
in crontab. After you become comfortable with the syntax of cron entries, there will be no need for
the GUI editor.
The entry for our defrag command becomes:
0 23 * * 4 "C:\WINDOWS\System32\defrag.exe" c:
This same task can be performed with the Task Scheduler with one entry.
Let us go through another example of more complexity, which would not as easily be accomplished
with the Task Scheduler.
I want to back up my work files every 3 hours, from Monday through Friday, between the hours of
9AM to 6PM, for all the months of the year. The work folder is C:\WORK and the backup folder is C:\BACKUP.
Open up crontab.txt and on the next free line, enter in * * * * * replace replace, then save it.
Open up the crontab editor and import crontab.txt. Double click the "* * * * replace replace"
entry.
For Command, browse to xcopy located in System32 within your Windows folder.
For Parameter: C:\WORK\* C:\BACKUP /Y
For Minute: 0
For Hour: 9-18/3
For Day: *
For Month: *
For Week Day: 0-4
Click OK->File->Save.
The entry for this task as reflected in our crontab.txt becomes
0 9-18/3 * * 0-4 "C:\WINDOWS\System32\xcopy.exe" C:\WORK\* C:\BACKUP /Y
If we were to schedule the above example with the Task Scheduler that comes with windows, then
a separate entry for every 3rd hour mark in terms of time (AM/PM) between the aforementioned times
would have to be entered, for the task.
Note: Cron can work with your own written batch/script files as well.
Note: You can view other examples in crontab.sample, located in the pycron program directory.
As you can see, Cron has a lot more to offer then the Task Scheduler. Not to say that the Windows
application is not useable, but for those scenarios where you need the ability to be flexible, and
configurable without all the extra headaches, then this is the ideal replacement for you. It proves
to be much more efficient as well in practice.
Scripts in /etc/cron.daily run each day from 4:02 to 0:02. I want do this so that updatedb (started
by slocate.cron) finishes before people start their workday. My quesiton isn't how, but whether this
will create any problems.
Does anyone see any problems with running cron.daily at 2 minutes after midnight
instead of 2 minutes after 4?
The tasks that run are mostly cleaning up log files and deleting old maid Hat EL. The daily tasks
are all defaults. (ls /etc/cron.daily returns: 00-logwatch 00webalizer certwatch logrotate makewhatis.cron
prelink rpm slocate.cron tetex.cron tmpwatch).
Thanks in advance!
===
Originally Posted by Tim65
Does anyone see any problems with running cron.daily at 2 minutes after midnight instead of 2
minutes after 4?
This should be no problem.
However: if you need to apply any patches / security fixes to cron in the future, you will want
to confirm that your changes weren't overwritten.
===
n.yaghoobi.s : Thanks. I know how to change it -
I was just wondering if anyone thought it was a bad idea.
anomie : Thanks. Good point about confirming the change doesn't get undone
by patches. I'm going to make the change. If I ever do discover a problem caused by this change,
I'll be sure to look up this thread and post the info here.
There are several special entries, some which are just shortcuts, that you can use instead of
specifying the full cron entry. The most useful of these is probably @reboot which allows you to
run a command each time the computer gets reboot. You can alert yourself when server is back online
after a reboot. Also becomes useful if you want to run certain services or commands at start up.
The complete list of special entries are:
Entry
Description
Equivalent To
@reboot
Run once, at startup.
None
@monthly
Run once a month
0 0 1 * *
@weekly
Run once a week
0 0 * * 0
@daily
Run once a day
0 0 * * *
@midnight
(same as @daily)
0 0 * * *
@hourly
Run once an hour
0 * * * *
The most useful again is @reboot. Use it to notify you when your server gets rebooted!
Users are permitted to use crontab if their names appear in the file /usr/lib/cron/cron.allow.
If that file does not exist, the file /usr/lib/cron/cron.deny is checked to determine if the
user should be denied access to crontab. If neither file exists, only a process with appropriate
privileges is allowed to submit a job. If only cron.deny exists and is empty, global usage
is permitted. The cron.allow and cron.deny files consist of one user name per line.
if both cron.allow and cron.deny files exist the cron.deny is ignored.
This can be accomplished by either listing users permitted to use the command in the file /var/spool/cron/cron.allow
and the /var/spool/cron/at.allow or in the list of user not permitted to access the command
in the file /var/spool/cron/cron.deny
[ian@attic4 ~]$ cat ./runclock3.sh #!/bin/bash runtime=${1:-10m} mypid=$$ # Run xclock in background
xclock& clockpid=$! echo "My PID=$mypid. Clock's PID=$clockpid" ps -f $clockpid #Sleep for the specified
time. sleep $runtime kill -s SIGTERM $clockpid echo "All done"
Listing 5 shows what happens when you execute runclock3.sh. The final kill
command confirms that the xclock process (PID 9285) was, indeed, terminated.
[ian@attic4 ~]$ ./runclock3.sh 5s
My PID=9284. Clock's PID=9285
UID PID PPID C STIME TTY STAT TIME CMD
ian 9285 9284 0 22:14 pts/1 S+ 0:00 xclock
All done
[ian@attic4 ~]$ kill -0 9285
bash: kill: (9285) - No such process
If you omit the signal specification, then SIGTERM is the default signal. The
SIG part of a signal name is optional. Instead of using -s and a signal name,
you can just prefix the signal number with a -, so the four forms shown in Listing 6
are equivalent ways of killing process 9285. Note that the special value -0, as used
in Listing 4 above, tests whether a signal could be sent to a process.
If you need just a one-shot timer to drive an application, such as you have just seen here, you
might consider the timeout command, which is part of the AppleTalk networking package
(Netatalk). You may need to install this package (see Resources below for
details), since most installations do not include it automatically.
You now have the basic tools to run a process for a fixed amount of time. Before going deeper
into signal handling, let's consider how to handle other termination requirements, such as repetitively
capturing information for a finite time, terminating when a file becomes a certain size, or terminating
when a file contains a particular string. This kind of work is best done using a loop, such as
for, while, or until, with the loop executed repeatedly with
some built-in delay provided by the sleep command. If you need finer granularity than
seconds, you can also use the usleep command.
You can add a second hand to the clock, and you can customize colors. Use the showrgb
command to explore available color names. Suppose you use the command xclock -bg Thistle -update
1& to start a clock with a second hand, and a Thistle-colored background.
Now you can use a loop with what you have learned already to capture images of the clock face
every second and then combine the images to make an animated GIF image. Listing 7 shows how to use
the xwininfo command to find the window id for the xclock command. Then
use ImageMagick command-line tools to capture 60 clock face images at one-second intervals (see
Resources for details on ImageMagick). And finally combine these into an
infinitely looping animated GIF file that is 50% of the dimensions of the original clock.
Timing of this type is always subject to some variation, so the import command to
grab the clock image is run in the background, leaving the main shell free to keep time. Nevertheless,
some drift is likely to occur because it does take a finite amount of time to launch each subshell
for the background processing. This example also builds in a 5-second delay at the start to allow
the shell script to be started and then give you time to click on the clock to bring it to the foreground.
Even with these caveats, some of my runs resulted in one missed tick and an extra copy of the starting
tick because the script took slightly over 60 seconds to run. One way around this problem would be
to use the usleep command with a number of microseconds that is enough less than one
second to account for the overhead, as shown by the commented line in the script. If all goes as
planned, your output image should be something like that in Figure 2.
This example shows you how to take a fixed number of snapshots of some system condition at regular
intervals. Using the techniques here, you can take snapshots of other conditions. You might want
to check the size of an output file to ensure it does not pass some limit, or check whether a file
contains a certain message, or check system status using a command such as vmstat. Your
needs and your imagination are the only limits.
If you run the getclock.sh script of Listing 7 yourself, and you close the clock
window while the script is running, the script will continue to run but will print error messages
each time it attempts to take a snapshot of the clock window. Similarly, if you run the runclock3.sh
script of Listing 4, and press Ctrl-c in the terminal window where the script is running,
the script will immediately terminate without shutting down the clock. To solve these problems, your
script needs to be able to catch or trap some of the signals discussed in
Terminating a child process.
If you execute runclock3.sh in the background and run the ps -f command
while it is running, you will see output similar to Listing 8.
[ian@attic4 ~]$ ./runclock3.sh 20s&
[1] 10101
[ian@attic4 ~]$ My PID=10101. Clock's PID=10102
UID PID PPID C STIME TTY STAT TIME CMD
ian 10102 10101 0 06:37 pts/1 S 0:00 xclock
ps -f
UID PID PPID C STIME TTY TIME CMD
ian 4598 12455 0 Jul29 pts/1 00:00:00 bash
ian 10101 4598 0 06:37 pts/1 00:00:00 /bin/bash ./runclock3.sh 20s
ian 10102 10101 0 06:37 pts/1 00:00:00 xclock
ian 10104 10101 0 06:37 pts/1 00:00:00 sleep 20s
ian 10105 4598 0 06:37 pts/1 00:00:00 ps -f
[ian@attic4 ~]$ All done
[1]+ Done ./runclock3.sh 20s
Note that the ps -f output has three entries related to the runclock3.sh process
(PID 10101). In particular, the sleep command is running as a separate process. One
way to handle premature death of the xclock process or the use of Ctrl-c to terminate the
running script is to catch these signals and then use the kill command to kill the
sleep command.
There are many ways to accomplish the task of determining the process for the sleep command. Listing
9 shows the latest version of our script, runclock4.sh. Note the following points:
The sleep command is explicitly run in the background.
The wait command is used to wait for termination of the sleep command.
The first trap command causes the stopsleep function to be run whenever
a SIGCHLD, SIGINT, or SIGTERM signal is received. The PID of the sleeper process is passed as
a parameter.
The stopsleep function is run as the result of a signal. It prints a status message
and sends the sleep command a SIGINT signal.
When the sleep command terminates, for whatever reason, the wait
command is satisfied. Traps are then cleared, and the xclock command is terminated.
[ian@attic4 ~]$ cat runclock4.sh
#!/bin/bash
stopsleep() {
sleeppid=$1
echo "$(date +'%T') Awaken $sleeppid!"
kill -s SIGINT $sleeppid >/dev/null 2>&1
}
runtime=${1:-10m}
mypid=$$
# Enable immediate notification of SIGCHLD
set -bm
# Run xclock in background
xclock&
clockpid=$!
#Sleep for the specified time.
sleep $runtime&
sleeppid=$!
echo "$(date +'%T') My PID=$mypid. Clock's PID=$clockpid sleep PID=$sleeppid"
# Set a trap
trap 'stopsleep $sleeppid' CHLD INT TERM
# Wait for sleeper to awaken
wait $sleeppid
# Disable traps
trap SIGCHLD
trap SIGINT
trap SIGTERM
# Clean up child (if still running)
echo "$(date +'%T') terminating"
kill -s SIGTERM $clockpid >/dev/null 2>&1 && echo "$(date +'%T') Stopping $clockpid"
echo "$(date +'%T') All done"
Listing 10 shows the output from running runclock4.sh three times. The first time,
everything runs to its natural completion. The second time, the xclock is prematurely
closed. And the third time, the shell script is interrupted with Ctrl-c.
Note how many times the stopsleep function is called as evidenced by the "Awaken"
messages. If you are not sure why, you might try making a separate copy of this function for each
interrupt type that you catch and see what causes the extra calls.
You will also note that some job control messages tell you about termination of the xclock
command and interrupting the sleep command. When you run a job in the background with
default bash terminal settings, bash normally catches SIGCHLD signals and prints a message after
the next terminal output line is printed. The set -bm command in the script tells bash
to report SIGCHLD signals immediately and to enable job control monitoring, The alarm clock example
in the next section shows you how to suppress these messages.
Our final exercise returns to the original problem that motivated this article: how to record
a radio program. We will actually build an alarm clock. If your laws allow recording of such material
for your proposed use, you can build a recorder instead by adding a program such as vsound.
For this exercise, we will use the GNOME rhythmbox application to illustrate some additional points.
Even if you use another media player, this discussion should still be useful.
An alarm clock could make any kind of noise you want, including playing your own CDs, or MP3 files.
In central North Carolina, we have a
radio station, WCPE, that broadcasts
classical music 24 hours a day. In addition to broadcasting, WCPE also streams over the Internet
in several formats, including Ogg Vorbis. Pick your own streaming source if you prefer something
else.
To start rhythmbox from an X Windows terminal session playing the WCPE Ogg Vorbis stream, you
use the command shown in Listing 11.
The first interesting point about rhythmbox is that the running program can respond
to commands, including a command to terminate. So you don't need to use the kill command
to terminate it, although you still could if you wanted to.
The second point is that most media players, like the clock that we have used in the earlier examples,
need a graphical display. Normally, you run commands with the cron and at
facilities at some point when you may not be around, so the usual assumption is that these scheduled
jobs do not have access to a display. The rhythmbox command allows you to specify
a display to use. You probably need to be logged on, even if your screen is locked, but you can explore
those variations for yourself. Listing 12 shows the alarmclock.sh script that you can
use for the basis of your alarm clock. It takes a single parameter, which specifies the amount of
time to run for, with a default of one hour.
Note the use of set +bm in the stopsleep function to reset the job control
settings and suppress the messages that you saw earlier with runclock4.sh
Listing 13 shows an example crontab that will run the alarm from 6 a.m. to 7 a.m.
each weekday (Monday to Friday) and from 7 a.m. for two hours each Saturday and from 8:30 a.m. for
an hour and a half each Sunday.
In more complex tasks, you may have several child processes. The cleanup routine shows how to
use the ps command to find the children of your script process. You can extend the idea
to loop through an arbitrary set of children and terminate each one.
Saw your "find" tip in issue #9, and thought you might like a quicker method. I don't know about
other distributions, but Slackware and Redhat come with the GNU versions of locate(1) and updatedb(1)
which use an index to find the files you want. The updatedb(1) program should be run once a night
from the crontab facility. To ignore certain sub-directories (like your /cdrom) use the following
syntax for the crontab file:
This would run every morning at 5:41am, and update the database with filenames from everywhere
but the subdirectories (and those below) the ones listed.
Q: I edited the crontab file but the commands still don't get executed.
A: Be sure user is not editing the crontab file directly with a simple text editor such as vi.
Use crontab -e which will invoke
the vi editor and then signal cron that changes have been made. Cron will only read the crontab file
when the daemon
is started, so if crontab has been edited directly, cron will need to be killed, /etc/cron.d/FIFO
removed, and the cron daemon
restarted in order to recover the situation.
'''Q: I deleted all my crontab entries using crontab -e but crontab -l shows that
they are still there.'''
A: Use crontab -r to remove an entire crontab file. Crontab -e does not know what to do
with empty files,
so it does not update any changes.
Q: Can I use my **** editor ?
A: Yes, by setting the environment variable EDITOR to ****.
Q: Why do I receive email when my cron job dies?
A: Because there is no standard output for it to write to. To avoid this, redirect the output of
the command to a
device (/dev/console, /dev/null) or a file.
Q: If I have a job that is running and my system goes down, will that job complete once the
system is brought back up?
A: No, the job will not run again or pick up where it left off.
Q: If a job is scheduled to run at a time when the system is down, will that job run once the
system is brought back up?
A: No, the job will not be executed once the system comes back up.
Q: How can I check if my cron is running correctly ?
A: Add the entry * * * * * date > /dev/console to your crontab file. It should print the
date in the console every minute.
Q: How can I regulate who can use the cron.
A: The file /var/spool/cron/cron.allow can be used to regulate who can submit cron jobs.
If /var/spool/cron/cron.allow does not exist, then crontab checks /var/spool/cron/cron.deny
to see who should not be
allowed to submit jobs.
If both files are missing only root can run cron jobs.
TROUBLESHOOTING CRON
If a user is experiencing a problem with cron, ask the user the following few questions to help
debug the problem.
1. Is the cron daemon running?
#ps -ef |grep cron
2. Is there any cron.allow/deny file?
#ls -lt /etc/cron*
3. Is it the root crontab or a non-su crontab?
#crontab -e "USER NAME"
4. If you are calling a script through crontab, does the script run from the command line?
Run the script at the command line and look for errors
5. Check that the first 5 fields of an entry are VALID or NOT commented out.
(minute, hours, day of the month, month and weekday)
6. Check for crontab related patches.
(check with sunsolve and the solaris version installed on the system
for exact patch match)
7. Check for recommended and security related patches?
(recommend to the customer to install all recommended and security patches
relevant to the OS installed)
Many times admins forget the field order of the crontab file
and alway reference the man pages over-and-over.
Make your life easy. Just put the field definitions in your crontab file
and comment (#) the lines out so the crontab file ignores it.
# minute (0-59),
# | hour (0-23),
# | | day of the month (1-31),
# | | | month of the year (1-12),
# | | | | day of the week (0-6 with 0=Sunday).
# | | | | | commands
3 2 * * 0,6 /some/command/to/run
3 2 * * 1-5 /another/command/to/run
if both cron.allow and cron.deny files exist the cron.deny is ignored.
This can be accomplished by either listing users permitted to use the command in the file /var/spool/cron/cron.allow
and the /var/spool/cron/at.allow or in the list of user not permitted to access the command
in the file /var/spool/cron/cron.deny.
Have you ever wandered near your Linux box in the middle of the night, only to discover the hard
disk working furiously? If you have, or just want a way for some task to occur at regular intervals,
cron is the answer.
Debian GNU-Linux
-- anacrona cron-like program that does not assume that the system is running continuously.
Anacron (like `anac(h)ronistic') is a periodic command scheduler. It executes commands at intervals
specified in days. Unlike cron, it does not assume that the system is running continuously.
It can therefore be used to control the execution of daily, weekly and monthly jobs (or anything
with a period of n days), on systems that don't run 24 hours a day. When installed and configured
properly, Anacron will make sure that the commands are run at the specified intervals as closely
as machine-uptime permits.
This package is pre-configured to execute the daily jobs of the Debian system. You should install
this program if your system isn't powered on 24 hours a day to make sure the maintenance jobs of
other Debian packages are executed each day.
crontab command creates, lists or edits the file containing control statements to be interpreted
by the cron command (cron table). Each statement consists of a time pattern and a command. The
cron program reads your crontab file and executes the commands at the time specified in the time
patterns. The commands are usually executed by a Bourne shell (sh).
The crontab command reads a file or the standard input to a directory that contains all users'
crontab files. You can use crontab to remove your crontab file or display it. You cannot access
other users' crontab files in the crontab directory.
COMMAND FORMAT
Following is the general format of the crontab command.
The following options may be used to control how crontab functions.
-e
Edit your crontab file using the editor defined by the EDITOR variable.
-r
Removes your current crontab file. If username is specified then remove that user's crontab
file. Only root can remove other users' crontab files.
-l
List the contents of your current crontab file.
Crontab File Format
The crontab file contains lines that consist of six fields separated by blanks (tabs or spaces).
The first five fields are integers that specify the time the command is to be executed by cron. The
following table defines the ranges and meanings of the first five fields.
Field
Range
Meaning
1
0-59
Minutes
2
0-23
Hours (Midnight is 0, 11 P.M. is 23)
3
1-31
Day of Month
4
1-12
Month of the Year
5
0-6
Day of the Week (Sunday is 0, Saturday is 6)
Each field can contain an integer, a range, a list, or an asterisk (*). The integers specify exact
times. The ranges specify a range of times. A list consists of integers and ranges. The asterisk (*)
indicates all legal values (all possible times).
The following examples illustrate the format of typical crontab time patterns.
Time Pattern
Description
0 0 * * 5
Run the command only on Thursday at midnight.
0 6 1,15 * 1
Run the command at 6 a.m. on the first and fifteenth of each month and every Monday.
00,30 7-20 * * *
Run the command every 30 minutes from 7 a.m. to 8 p.m. every day.
NOTE:
The day of the week and day of the month fields are interpreted separately if both are defined. To
specify days to run by only one field, the other field must be set to an asterisk (*). In this case
the asterisk means that no times are specified.
The sixth field contains the command that is executed by cron at the specified times. The
command string is terminated by a new-line or a percent sign (%). Any text following the percent sign
is sent to the command as standard input. The percent sign can be escaped by preceding it with a backslash
(\%).
A line beginning with a # sign is a comment.
Each command in a crontab file is executed via the shell. The shell is invoked from your HOME directory
(defined by $HOME variable). If you wish to have your (dot) .profile executed, you must specify so in
the crontab file. For example,
0 0 * * 1 . ./.profile ; databaseclnup
would cause the shell started by cron to execute your .profile, then execute the program
databaseclnup. If you do not have your own .profile executed to set up your environment, cron
supplies a default environment. Your HOME, LOGNAME, SHELL, and PATH variables are set. The HOME and
LOGNAME are set appropriately for your login. SHELL is set to /bin/sh and PATH is set to :/bin:/usr/bin:/usr/lbin.
NOTE:
Remember not to have any read commands in your .profile which prompt for input. This causes
problems when the cron job executes.
Command Output
If you do not redirect the standard output and standard error of a command executed from your crontab
file, the output is mailed to you.
Access
To use the crontab command you must have access permission. Your system administrator can
make the crontab command available to all users, specific users, or no users. Two files are used
to control who can and cannot access the command. The cron.allow file contains a list of all users who
are allowed to use crontab. The cron.deny file contains a list of all users who are denied access
to crontab. If the cron.allow file exists but is empty, then all users can use the crontab
command. If neither file exists, then no users other than the super-user can use crontab.
Displaying your crontab
If you have a crontab file in the system crontab area, you can list it by typing crontab -l.
If you do not have a crontab file, crontab returns the following message:
crontab: can't open your crontab file.
DIAGNOSTICS AND BUGS
The crontab command will complain about various syntax errors and time patterns not being
in the valid range.
CAUTION:
If you type crontab and press Return without a filename, the standard input is read
as the new crontab entries. Therefore, if you inadvertently enter crontab this way and you want to
exit without destroying the contents of your current crontab file, press the Del key. Do not press
the Ctrl-D key; if you do, your crontab file will only contain what you have currently typed.
Related Commands
/usr/sbin/cron.d
The main directory for the cron process.
/usr/sbin/cron.d/log
Accounting information for cron processing.
/usr/sbin/cron.d/crontab.allow
A file containing a list of users allowed to use crontab.
/usr/sbin/cron.d/crontab.deny
A file containing a list of users not allowed to use crontab.
If cron.allow exists then you MUST be listed in it to use crontab (so make sure all the system accounts
like root are listed), this is very effective for limiting cron to a small number of users. If cron
allow does not exists, then cron.deny is checked and if it exists then you will not be allowed to use
crontab unless you are listed ("locked out")
In both cases users are listed one per line, so you can use something like:
I assume you are on a Linux system. Then, you have a small syntax error in viewing other users
crontabs, try "crontab -l -u username" instead.
Here is how it works: Two config files, /etc/cron.deny and /etc/cron.allow (on SuSE systems these
files are /var/spool/cron.deny and .../allow), specify who can use crontab.
If the allow file exists, then it contains a list of all users that may submit crontabs, one per
line. No unlisted user can invoke the crontab command. If the allow file does not exist, then
the deny file is checked.
If neither the allow file nor the deny file exists, only root can submit crontabs.
This seems to be your case, so you should create one of these files ... on my system I have a
deny file just containing user "guest", so all others are allowed.
One caveat: this access control is implemented by crontab, not by cron. If a user manages to put
a crontab file into the appropriate directory by other means, cron will blindly execute ...
[from the book "Linux Administration Handbook" by Nemeth/Snyder/Hein and validated locally here]
You can control access to the crontab command by using two files in the /etc/cron.d
directory: cron.deny and cron.allow. These files permit only specified users
to perform the crontab command tasks such as creating, editing, displaying, or removing
their own crontab files.
The cron.deny and cron.allow files consist of a list of user names, one
per line. These access control files work together as follows:
If cron.allow exists, only the users listed in this file can create, edit, display,
or remove crontab files.
If cron.allow does not exist, all users can submit crontab files, except
for users listed in cron.deny.
If neither cron.allow nor cron.deny exists, superuser privileges are
required to run the crontab command.
Superuser privileges are required to edit or create the cron.deny and cron.allow
files.
The cron.deny file, created during SunOS software installation, contains the following
user names:
daemon
bin
smtp
nuucp
listen
nobody
noaccess
None of the user names in the default cron.deny file can access the crontab
command. You can edit this file to add other user names that will be denied access to the crontab
command.
No default cron.allow file is supplied. So, after Solaris software installation, all
users (except the ones listed in the default cron.deny file) can access the crontab
command. If you create a cron.allow file, only these users can access the crontab
command.
To verify if a specific user can access crontab, use the crontab -l command
while you are logged into the user account. $ crontab -l
If the user can access crontab, and already has created a crontab file,
the file is displayed. Otherwise, if the user can access crontab but no crontab
file exists, a message such as the following is displayed: crontab: can't open your crontab file
This user either is listed in cron.allow (if the file exists), or the user is not listed
in cron.deny.
If the user cannot access the crontab command, the following message is displayed whether
or not a previous crontab file exists: crontab: you are not authorized to use cron. Sorry.
This message means that either the user is not listed in cron.allow (if the file exists),
or the user is listed in cron.deny.
Determining if you have crontab access is relatively easy. A Unix system administrator has two
possible files to help manage the use of crontab. The administrator can explicitly give permission
to specific users by entering their user identification in the file:
/etc/cron.d/cron.allow
Alternatively, the administrator can let anyone use crontab and exclude specific user with the
file:
/etc/cron.d/cron.deny
To determine how your system is configured, first enter the following at the command line:
more /etc/cron.d/cron.allow
If you get the message, "/etc/cron.d/cron.allow: No such file or directory" you're probably in
fat city. One last step, make sure you are not specifically excluded. Go back to the command line
and enter:
more /etc/cron.d/cron.deny
If the file exists and you're not included therein, skip to setup instruction.
If there are entries in the cron.allow file, and you're not among the chosen few, or if you are listed
in the cron.deny file, you will have to contact the administrator and tell him/her you are an upstanding
citizen and would like to be able to schedule crontab jobs.
In summary, users are permitted to use crontab if their names appear in the file /etc/cron.d/cron.allow.
If that file does not exist, the file /etc/cron.d/cron.deny is checked to determine if the user should
be denied access to crontab. If neither file exists, only the system administrator -- or someone
with root access -- is allowed to submit a job. If cron.allow does not exist and cron.deny exists
but is empty, global usage is permitted. The allow/deny files consist of one user name per line.
The Last but not LeastTechnology is dominated by
two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt.
Ph.D
FAIR USE NOTICEThis site contains
copyrighted material the use of which has not always been specifically
authorized by the copyright owner. We are making such material available
to advance understanding of computer science, IT technology, economic, scientific, and social
issues. We believe this constitutes a 'fair use' of any such
copyrighted material as provided by section 107 of the US Copyright Law according to which
such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free)
site written by people for whom English is not a native language. Grammar and spelling errors should
be expected. The site contain some broken links as it develops like a living tree...
You can use PayPal to to buy a cup of coffee for authors
of this site
Disclaimer:
The statements, views and opinions presented on this web page are those of the author (or
referenced source) and are
not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society.We do not warrant the correctness
of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be
tracked by Google please disable Javascript for this site. This site is perfectly usable without
Javascript.
DAVE MCKAY @thegurkha
