SFX Maintenance Tasks

Recommended SFX Maintenance Tasks

There are several maintenance procedures that are recommended to ensure the system's ongoing performance. It is highly recommended to perform all recommended maintenance procedures once a month.

Database Maintenance

We recommend running the CHECK and REPAIR functions using the Server Admin Utility once a month. This process is described in Database Maintenance .

File System Maintenance

We recommend running a cleanup of the /exlibris directory every month. The cleanup clears enough disk space for the revision update to run and allows SFX to function smoothly.

Do the following in order to clean up the /exlibris directory:

  • Archive statistics – Archiving statistics is recommended whenever your offline table reaches over 200,000 records. For more information regarding statistics maintenance, refer to the SFX Statistics Module section of the SFX Advanced User's Guide.
  • Clean the session and temporary files – These files are created under each instance in the <instance> /dbs/scratch/ directory. These also include files created using export tools in SFX Admin.

The number of files created varies depending on use of the different SFX tools. Do not to remove any files when someone is working in SFX Admin.

To clean this directory, log on to the SFX server as an instance user and select option 5 Clean /exlibris/sfx_ver/sfx4_1/sfxlcl41/dbs/scratch/ from the Filesystem Maintenance menu of the Server Admin Utility. (For more information, see Clean /exlibris/sfx_ver/sfx4_1/sfxlcl41/dbs/scratch/.)

Do not manually delete the entire <instance> /dbs/scratch/sessions directory. If you do, SFX Admin access is disabled.
  • Clean the files created during the revision update – The files such as logs, downloaded packages, and backup files are created during the revision update in the /exlibris/sfx_ver/sfx4_1/tmp directory. Clean this directory every few months before running a revision update.

To clean this directory, log on to the SFX server as the sfxglb41 user and select option 1 Clean /exlibris/sfx_ver/sfx4_1/tmp/ from the Filesystem Maintenance menu of the Server Admin Utility. (For more information, see Clean /exlibris/sfx_ver/sfx4_1/tmp/.)

  • Clean the reverse proxy Apache logs – These log files are located in the /exlibris/sfx_ver/sfx4_1/proxy/logs directory.

To clean this directory, log on to the SFX server as any user and select option 2 Clean Apache log from the Filesystem Maintenance menu of the Server Admin Utility. (For more information, see Clean Apache and MySQL Logs.)

  • Clean the instance Apache logs – Each Apache instance also creates log files in the <instance> /logs/apache/ directory. To clean these files, log on to the SFX server as an instance user. There are two options:
    • Automatically – Use the server_admin_util by selecting option 4 /exlibris/sfx_ver/sfx4_1/<instance> /logs/ from the Filesystem Maintenance menu of the Server Admin Utility. (For more information, see Clean /exlibris/sfx_ver/sfx4_1/sfxlcl41/logs/ .)
    • Manually – Navigate to the <instance> /logs/apache/ directory and clean the access_log and error_log files, as follows:
  1. Type the following:

lg

cd apache

rm access_log

rm error_log

  1. Restart Apache by typing:

apachectl restart

When cleaning files, do not remove the httpd.pid file or you will be unable to restart Apache.
  • Clean the MySQL logs – If you enabled MySQL binary logging (it is not enabled by default in SFX), binary log files are created in the /exlibris/sfx_ver/sfx4_1/data directory.

To clean this directory, log on to the SFX server as any user and select option 3 Clean MySQL log from the Filesystem Maintenance menu of the Server Admin Utility. (For more information, see Clean Apache and MySQL Logs .)

Cleaning the MySQL logs requires the MySQL root password and the UNIX password of the sfxglb41 user.

  • Clean the MARCit! export files – When exporting files to MARCit!, several files are created in different locations on the SFX server, such as the <instance> /export and <instance> /home directories. Clean these files once they are no longer used.

Note that the recommended frequency of cleaning files can vary depending on the amount of usage different SFX components receive and the amount of free disk space available on the SFX server.

  • Clean the SFX_REQUEST table – The SFX_REQUEST table is used to store data required to generate each SFX menu screen. This data is of requests before they get processed by the statistics module. It is not directly related to statistics tables, so cleaning it does not result in a loss of statistical data.

When you clean the SFX_REQUEST table using the Server Admin Utility, the last 24 hours of requests in the table are saved. This information is necessary for SFX_REQUEST to function so that the end users can continue to use the system.

For information on cleaning the SFX_REQUEST table immediately, see Clear SFX_REQUEST Table.

To schedule the cleaning of the SFX_REQUEST table for a later time, see Schedule Clearing of SFX_REQUEST Table.

Centralized Management of Maintenance Tasks

Centralized Management of Maintenance Tasks is designed for SFX installations with multiple SFX instances. It allows the SFX administrator to schedule recommended maintenance tasks centrally, by configuring the tasks to be run as the global user. The maintenance tasks are run on each instance for which they are configured one after another, with no time gap between them. This saves time over the method of scheduling individual maintenance tasks manually.

When using centralized management, the tasks covered by centralized management are disabled in the Server Admin Utility of the local instances. Although you can run these tasks manually on the local instance, you cannot locally schedule these tasks to be run. If you attempt to, the following is displayed:

Maintenance tasks can be scheduled only by the system administrator of this SFX installation. This option is not available for local SFX instances.

If you activate centralized management, you must remove existing cron jobs from the local instances manually - they are not migrated automatically.

You can request email notification in case of errors and statistic information on the time it took for the maintenance tasks to be run for each instance via the Server Admin Utility. For more information, see Report Centralized Management Maintenance Tasks.

The following table lists the tasks performed by the Centralized Management of Maintenance Tasks:

Centralized Management of Maintenance Tasks
Task Type Default Frequency Description
statistics_Online2offline Fundamental Every day
cleanup_sfx_request_table Fundamental Every day
Cleanup_logs Fundamental Every day
Cleanup_tmp Fundamental Once a week The only instance that can be chosen is #global#
cleanup_ip_from_tables Fundamental Once a week This option cleans IP information from statistics offline statistics data. It should be run before the Offline2file task.
statistics_Offline2file Not set up by default. Moves to an archive file all SFX usage statistics older than 1 year
indices_RSI_complete Not set up by default
indices_RSI_incremental Not set up by default
indices_RSI_incremental_unless_complete_required Fundamental Every day
indices_AZ Subscription Each Sunday
indices_ebookSearch_complete Not set up by default
indices_ebookSearch_incremental unless_complete_required Subscription Every day
indices_ebookSearch_incremental Not set up by default
Elsevier_SD_Autoloader Subscription Once a month Not set up by default
CDI_export Only when using CDI Every day Set up when configured to use CDI

Configuring Centralized Management of Maintenance Tasks

The Centralized Management of Maintenance tasks is configured via a configuration file in the sfxglb4x instance.

The configuration file contains the following for each maintenance task:

  • A switch to turn centralized management on/off
  • The frequency and day of the week or month the task is run. Values can be
    • every_day
    • one or more weekdays (comma delimited)
    • day_of_month_>
    • specific dates by setting the frequency parameter in the following format: specific_dates_. It is possible to specify more than one date (comma delimited) for example:

Section "cleanup_sfx_request_table" active "1" frequency "specific_dates_JAN-01,APR-015,JUL-01,OCT-15" instances "#all#"

  • A list of instances for which the job should be run. Values can be #all#, #all_except_tst#, #file#, #global# or a comma delimited list of one or more instances.
    • If you entered #all# the config/instances.config list of local instances is used.
    • If #all_except_tst# is entered, the config/instances.config list of local instances is used except for any instances that have tst or test in the instance name.
    • If #file# is entered, the list of instances is taken from sfxglb41/config/rsi_ebook_setup.config_ in the index_build section.
In the cleanup_tmp job, the only instance that can be entered is #global#.

At the end of the configuration file, a list of commands is included in the all_jobs section (one command for each maintenance task). The order of the commands in this section determines the order in which the tasks are run. This is important in case more than one task is run in the same session.

You can run the same task for a different set of instances. Note that in this case, the instances need to be specified - it is not possible to use all, all_except_tst or file.

To run the same task for a different set of instances:
  1. Add a new section with name of the tasks in the config/central_management.config_ file:

Section "indices_AZ_Journals_1"

active "1"

frequency "sunday,wednesday"

instances "instance1,instance2"

EndSection

Section "indices_AZ_Journals_2"

active "1"

frequency "monday,thursday"

instances "instance3,instance4,instance5"

EndSection

  1. Add the following lines in the all_jobs section:

indices_AZ_Journals_1 "admin/aztools/az_index_builder.pl"

indices_AZ_Journals_2 "admin/aztools/az_index_builder.pl"

To run the CDI_export consecutively for multiple institutes in the same SFX instance:

In the CDI_export job, it is possible to define for a specific instance the institutes for which a separate CDI export should be created. This can be done by adding the parameter --institute=<institute name>at the end of the configuration file, in the all_jobs section > CDI_export command.

For example:

Section "CDI_export_1"

active "1"

frequency "every_day"

instances "A"

EndSection

Section "AllJobs"

CDI_export_1 "admin/kbtools/cdi_export.cron -service=getFullTxt --service=getSelectedFullTxt --original_title_format=both --object_type=JOURNAL --object_type=BOOK --mode=manual --format=CDI --institute=InstA --institute=InstB --institute=InstC"

Create a separate section per instance where you want to run multiple CDI exports, one for each institute.

Additionally, if there are instances on the SFX server for which you want to run the CDI export without creating institute specific export files, list them all in the following manner:

# Multiple instances - no institutes

Section "CDI_export_2"

active "0"

frequency "every_day"

instances "B,C,D"

EndSection

Section "AllJobs"

# Multiple instances - no institutes

CDI_export_2 "admin/kbtools/cdi_export.cron --service=getFullTxt --service=getSelectedFullTxt --original_title_format=both --object_type=JOURNAL --object_type=BOOK --mode=manual --format=CDI"

To configure the Centralized Management of Maintenance Tasks:
  1. As the root user, create a cronjob that starts the centralized maintenance script every night (for example at midnight) to check if any tasks should be run that night:
    1. Type crontab -e.
    2. Enter the following at the top of the maintenance window of the server.:

      00 20 * * * /exlibris/sfx_ver/sfx4_1/sfxglb41/admin/CentralJobManagement/central_crons.cron

      For more information on cron jobs, including the structure of the scheduling time/date definition, see http://en.wikipedia.org/wiki/Cron.
  2. Activate Central Job Management:
    1. In the centralized_management section of the /exlibris/sfx_ver/sfx4_1/sfxglb41/config/central_management.config_ file, set the active parameter to 1.
    2. Set the email parameter to a comma separated list of email addresses (no spaces) to which you to send a report of errors in jobs.
  3. For each maintenance task, check its frequency and the list of instances for which the task should be run.
  4. For the eBook Search index, the configuration file is set up to use the list of instances specified in the sfxglb41/config/rsi_ebook_setup.config_ file in the index_build section. If this configuration file does not contain the list of instances for which you want to build the index, use one of the following options to configure the relevant instances:
    • List of specific instances
    • #all# - not recommended
    • #all_except_test#
  5. Validate the configuration file. As the root user, run /exlibris/sfx_ver/sfx4_1/sfxglb41/admin/CentralJobManagement/central_crons.cron --validate. The output should be Configuration is valid.
  6. Remove or comment out the existing crontabs entries that have been replaced by the centralized management of maintenance tasks script.

    1. Log on to the SFX server as each instance user and type:

      crontab -e

    2. Remove the following crontab jobs:

      • admin/stats/online2offline.pl --f

      • admin/filesystem/clean_dir.pl --directory=logs --type=scheduled

      • admin/filesystem/clean_dir.pl --directory=scratch --type=scheduled

      • admin/filesystem/clean_dir.pl --directory=export --type=scheduled

      • admin/filesystem/clean_dir.pl --directory=tmp --type=scheduled

      • admin/database/clear_sfx_requests.pl --force

      • admin/rsi_tools/rsi_index_builder.pl

      • admin/rsi_tools/rsi_index_builder.pl --object_type=m --build_type=c

      • admin/rsi_tools/rsi_index_builder.pl --object_type=m --build_type=i --no_complete_build

      • admin/rsi_tools/rsi_index_builder.pl --object_type=m --build_type=i

      • admin/aztools/az_index_builder.pl

      • admin/aztools/solr_index_builder.pl --build_type=c

      • admin/aztools/solr_index_builder.pl --build_type=i --no_complete_build

      • admin/aztools/solr_index_builder.pl --build_type=i

We recommend that you add a small number of servers to use the Centralized Management of Maintenance Tasks option when you first configure it to make trouble shooting easier.