Statistics

To access this option, select 5 Statistics from the main menu. The Statistics menu is displayed:

Statistics

1 Move data from online to offline

2 Archive statistics

3 Load statistics from a file

4 Show stats tables monitor information

5 Scheduling for statistics maintenance tasks

6 Clear SFX_REQUEST table

7 bX Registration & Configuration

8 Publish Historical Stats for bX

9 Clean OAI_REPOSITORY table

10 Remove SFX requests as part of IE8 fix from online and offline tables

11 Fix statistics file to make it OS independent

12 Clean CENTRAL_CRON_STATS table (only by sfxglb41)

13 Clean IP information from statistics data

Please select [1-13,q,m]:

Move Data from Online to Offline

In order for statistics to be queried, they must be in the offline tables. Select 1 Move data from online to offline from the Statistics menu to move statistics to the offline tables.

You are asked whether you want to specify a date range to move all data in the online tables. Press enter if you do not want to specify a date range. See the following example. (Bold text is typed by the user):

Please select: [1-13,q,m]: 1

Type the begin date (YYYYMMDD) of the date range of data to move or to ignore: [ignore]

Type the end date (YYYYMMDD) of the date range of data to move or to ignore: [ignore]

About to move from online to offline... Do you want to continue? [Y/n] y

Transferring records, please wait...

Alternatively, you can perform this task directly from the server. Type the following command from the admin/stats directory of your instance:

./online2offline.pl --f

To see more information on the different options, type:

./online2offline.pl --help

You can schedule this task to be done automatically every day or week. More information about scheduling statistics tasks can be found in Scheduling for Statistics Maintenance Tasks.

Archiving Statistics

When the offline table contains a large amount of statistics data, it is best to archive the statistics records and store them on a different server. (Guidelines for archiving can be found in the Guidelines for Statistics Maintenance section of the SFX Advanced User’s Guide.) Make sure to run all desired statistics queries on the statistics records before they are archived.

Select option 2 Archive Statistics from the Statistics menu. This option removes the statistics data from the SFX database and stores the data instead in an archive file.

You are asked if you want to archive all statistics data except for the current calendar year statistics data or specify a date range to move all data in the online tables.

The archive file is placed in the following location:

/exlibris/sfx_ver/sfx4_1/<sfx_instance> /archive/

Type y to archive all statistics data except for the current calendar year statistics data. See the following example (Bold text is typed by the user):

Please select [1-11,q,m]: 2

Do you want to exclude the current year statistics data from the archiving task? [y/n]: y

About to move stats from offline to file.

Do you want to continue? [Y/n] y

Done. Moved 1200/1200 records

Cleaning up...

Compressing the file '/exlibris/sfx_ver/sfx4_1/sst1_3/archive/stats.20160524-12:07:40_from_2016-04-05_11:28:24_to_2016-05-24_12:03:43.saf'

Done. File compressed successfully into '/exlibris/sfx_ver/sfx4_1/sst1_3/archive/stats.20160524-12:07:40_from_2016-04-05_11:28:24_to_2016-05-24_12:03:43_sfx_compress.saf.gz'

The file name and location of the archive file is:

/exlibris/sfx_ver/sfx4_1/<sfx_instance> /archive/stats.

The file name includes the date and time of the last (from_) and first (to_) statistics record included in the file.

Type n to specify a date range to archive. See the following example (Bold text is typed by the user):

Please select [1-11,q,m]: 2

Do you want to exclude the current year statistics data from the archiving task? [y/n]: n

Type the begin date (YYYYMMDD) of the date range to archive or to ignore: [ignore] 20130101

Type the end date (YYYYMMDD) of the date range to archive or to ignore: [ignore] 20131231

About to move stats from offline to file.

Do you want to continue? [Y/n] y

Done. Moved 1200/1200 records

Cleaning up...

Compressing the file '/exlibris/sfx_ver/sfx4_1/sst1_3/archive/stats.20160524-12:07:40_from_2016-04-05_11:28:24_to_2016-05-24_12:03:43.saf'

Done. File compressed successfully into '/exlibris/sfx_ver/sfx4_1/sst1_3/archive/stats.20160524-12:07:40_from_2016-04-05_11:28:24_to_2016-05-24_12:03:43_sfx_compress.saf.gz'

The file name and location of the archive file is:

exlibris/sfx_ver/sfx4_1/<sfx_instance> /archive/stats.

The file name includes the date and time of the last (from_) and first (to_) statistics record included in the file.

Alternatively, you can perform this task directly from the server. Type the following command from the admin/stats directory of your instance:

./offline2file.pl --f --file=

For more information on the different options, type:

./offline2file.pl --help

To exclude the current year statistics data from the archiving task, type the following command:

./offline2file.pl --f --leaveLastYear

To exclude a specific number of days from the archiving task, type the following command:

./offline2file.pl --f --leaveDays=<nr_of_days_to_exclude_from_archiving>

You can schedule this task to be done automatically every day or week. More information about scheduling statistics tasks can be found in Scheduling for Statistics Maintenance Tasks.

Load Statistics from a File

This option allows you to import an archived flat file back into the offline tables so that the statistical information can be queried. To access this option, select 3 Load statistics from a file from the Statistics menu.

You are asked whether you want to specify a date range (optional) or copy all data into the offline tables. Press enter if you do not want to specify a date range.

The program asks you to specify the location of the archived file. See the following example:

Please select [1-6,q,m]: 3

Enter complete path and filename to extract the records from: /exlibris/sfx_ver/sfx4_1/sst1_3/archive/stats.20160524-12:07:40_from_2016-04-05_11:28:24_to_2016-05-24_12:03:43_sfx_compress.saf.gz

About to load offline stats from file:

/exlibris/sfx_ver/sfx4_1/sst1_3/archive/stats.20160524-12:07:40_from_2016-04-05_11:28:24_to_2016-05-24_12:03:43_sfx_compress.saf.gz

Do you want to continue? [Y/n]

Extracting the file '/exlibris/sfx_ver/sfx4_1/sst1_3/archive/stats.20160524-12:07:40_from_2016-04-05_11:28:24_to_2016-05-24_12:03:43_sfx_compress.saf.gz'

Done. File '/exlibris/sfx_ver/sfx4_1/sst1_3/archive/stats.20160524-12:07:40_from_2016-04-05_11:28:24_to_2016-05-24_12:03:43_sfx_compress.saf.gz' was extracted successfully

Done. Loaded 54/54 records

Removing uncompressed file '/exlibris/sfx_ver/sfx4_1/sst1_3/archive/stats.20160524-12:07:40_from_2016-04-05_11:28:24_to_2016-05-24_12:03:43.saf'

Alternatively, you can perform this task by typing the following command from the admin/stats directory of your instance:

./file2offline.pl --f --file=<file name> <options>

For more information on the different options, type:

./file2offline.pl --help

Show Stats Tables Monitor Information

This option allows you to check the size of the statistics tables and whether statistics maintenance tasks (such as archiving) need to be performed. Select option 4 Show Stats Tables Monitor Information from the Statistics menu.

The following is an example of the information displayed on the screen:

+----------------+------------+-----------------------+------------+------------+

+----------------+------------+-----------------------+------------+------------+

| SFX Requests | 747 | 18.87 | 0.03 | |

| Online Stats | 0 | 0.25 | 0.32 | |

| Offline Stats | 580 | 1.04 | 2.40 | 2004-09-27 | 2004-10-25

+----------------+------------+-----------------------+------------+------------

Stats Tables Monitor Information

In this example, the online statistics tables are empty. This means that all information was recently moved from the online to the offline statistics tables.

In this example, the offline statistics tables contain information about the usage of SFX in this instance between 2004-09-27 and 2004-10-25.

Scheduling for Statistics Maintenance Tasks

This option allows you to:

Schedule maintenance tasks
List currently scheduled maintenance tasks
Remove currently scheduled maintenance tasks

To access this option, select 5 Scheduling for statistics maintenance tasks from the Statistics menu. The following menu is displayed:

Scheduling for statistics maintenance tasks

1 List scheduled tasks for online to offline process

2 Schedule online to offline process

3 List scheduled archiving tasks

4 Schedule archiving task

5 Schedule clearing of SFX_REQUEST table

6 List scheduled tasks of clearing CENTRAL_CRON_STATS table (only by sfxglb41)

7 Schedule clearing of CENTRAL_CRON_STATS table (only by sfxglb41)

8 Show scheduled IP clean-up tasks

9 Schedule IP information clean-up task

Please select [1-8,q,m]:

List Scheduled Tasks for Online to Offline Process

Select 1 List Scheduled Tasks for Online to Offline Process from the Scheduling for statistics maintenance tasks menu to list and delete scheduled tasks. A screen such as the following is displayed:

Online to offline schedule is as follows:

[1] 22:00 of every day of every month

[2] 00:00 of every Monday of every month

Would you like to remove any of these? [y/N] Y

Please enter the number of the task to remove or to cancel: 2

Successfully deleted the crontab line

Type the number of a task to delete it.

Schedule Online to Offline Process

This task cannot be scheduled when Centralized Management of Maintenance Tasks is activated. If you attempt to do so, the following is displayed: Maintenance tasks can be scheduled only by the system administrator of this SFX installation. This option is not available for local SFX instances.

Select 2 Schedule online to offline process from the Scheduling for statistics maintenance tasks menu to schedule online to offline processes. The following screen is displayed:

Please enter scheduling parameters

Time must be in the form HH:MM where

HH can be: 00-23 for midnight-11PM or * for every hour

MM can be: 00-59 or * for every minute

Enter the time to run [00:00]:

Enter a time to run. The following is displayed:

Would you like to schedule for a certain weekday? [Y/n]

If you would like to schedule for a specific day of the week, type Y. The following is displayed:

Please select a weekday (1-7 Sunday-Saturday respectively) [1]: 2 Successfully added the refresh task to your crontab.

If you do not want to schedule for a specific day of the week, type n. The following is displayed:

Please enter month (1-12 for January-December respectively or '*' for every month) [*]

Either type the number of the month for which you would like to schedule or type an asterisk (*) to schedule for every month. The following is displayed:

Please enter day of the month (1-31 or '*' for every day) [1]:

Type the day of the month or type an asterisk (*) to schedule for every day. The following is displayed:

Successfully added the refresh task to your crontab.

List Scheduled Archiving Tasks

Select 3 List Scheduled Archiving Tasks from the Scheduling for statistics maintenance tasks menu to list and delete scheduled archiving tasks. A screen such as the following is displayed:

Archiving schedule is as follows:

[1] 00:00 of day 01 of January

[2] 12:12 of every Sunday of every month

Would you like to remove any of these? [y/N]y

Please enter the number of the task to remove or to cancel: 1

Successfully deleted the crontab line

Type the number of an archiving task to delete it.

Schedule Archiving Task

Select 4 Schedule archiving task from the Scheduling for statistics maintenance tasks menu to schedule archiving tasks. The following screen is displayed:

Do you want to exclude the current year statistics data from the archiving task? [Y/n] y

Please enter scheduling parameters

Time must be in the form HH:MM where

HH can be: 00-23 for midnight-11PM or * for every hour

MM can be: 00-59 or * for every minute

Enter the time to run [00:00]:

Enter a time at which to run the task. The following is displayed:

Would you like to schedule for a certain weekday? [Y/n]

If you would like to schedule for a specific day of the week, type Y. The following is displayed. (Bold text is typed by the user):

Please select a weekday (1-7 Sunday-Saturday respectively) [1]: 2 Successfully added the refresh task to your crontab.

If you do not want to schedule for a specific day of the week, type n. The following is displayed:

Please enter month (1-12 for January-December respectively or '*' for every month) [*]

Either type the number of the month for which you would like to schedule or type an asterisk (*) to schedule for every month. The following is displayed:

Please enter day of the month (1-31 or '*' for every day) [1]:

Type the day of the month or type an asterisk (*) to schedule for every day. The following is displayed:

Successfully added the archiving task to your crontab.

Schedule Clearing of SFX_REQUEST Table

Select 5 Schedule clearing of SFX_REQUEST table from the Scheduling for statistics maintenance tasks menu to schedule the clearing of the sfx_request table. The following screen is displayed:

Please enter scheduling parameters

Time must be in the form HH:MM where

HH can be: 00-23 for midnight-11PM or * for every hour

MM can be: 00-59 or * for every minute

Enter the time to run [00:00]:

Enter a time at which to run the task. The following is displayed:

Would you like to schedule for a certain weekday? [Y/n]

If you would like to schedule for a specific day of the week, type Y. The following is displayed:

Please select a weekday (1-7 Sunday-Saturday respectively) [1]: 2 Successfully added the refresh task to your crontab.

If you do not want to schedule for a specific day of the week, type n. The following is displayed:

Please enter month (1-12 for January-December respectively or '*' for every month) [*]

Either type the number of the month for which you would like to schedule or type an asterisk (*) to schedule for every month. The following is displayed:

Please enter day of the month (1-31 or '*' for every day) [1]:

Type the day of the month or type an asterisk (*) to schedule for every day. The following is displayed:

Successfully added the refresh task to your crontab.

Show Scheduled IP Clean-Up Tasks

Select 8 Show scheduled IP clean-up tasks. One of the following is displayed, depending on if you have scheduled IP clean -up tasks:

There are no scheduled IP clean-up tasks

or:

IP clean-up task schedule is as follows:

[1] 00:00 of every Sunday of every month

[2] 00:00 of day 01 of every month

Would you like to remove any of these? [y/N]

Schedule IP Information Clean-Up Task

Select 9Schedule IP information clean-up task. One of the following is displayed:

If you want to schedule for a specific weekday:

Please enter scheduling parameters Time must be in the form HH:MM where HH can be: 00-23 for midnight-11PM or * for every hour MM can be: 00-59 or * for every minute Enter the time to run [00:00]: Would you like to schedule for a certain weekday? [Y/n] y Please select a weekday (1-7 Sunday-Saturday respectively) [1]: 1 Successfully added the refresh task to your crontab.

If you want to schedule for a specific date:

Please enter scheduling parameters

Time must be in the form HH:MM where

HH can be: 00-23 for midnight-11PM or * for every hour

MM can be: 00-59 or * for every minute

Enter the time to run [00:00]:

Would you like to schedule for a certain weekday? [Y/n] n

Please enter month (1-12 for January-December respectively or '*' for every month) [*]: *

Please enter day of the month (1-31 or '*' for every day) [1]: 1

Successfully added the refresh task to your crontab.

Clear SFX_REQUEST Table

The SFX_REQUEST table is used to store data required to generate each SFX menu screen. This table increases in size and needs to be cleared periodically.

To clear the SFX_REQUEST table, select 6 Clear SFX_REQUEST Table from the Statistics menu. The following screen is displayed:

To look at the contents of the SFX_REQUEST table, you may want to export the table before taking this step.

Are you sure you want to remove all sfx requests? [y/n]

Type y to delete all SFX requests.

Menu Options for bX

The following options are only available to customers who have purchased bX as well as SFX:

bX Registration & Configuration
Publish Historical Stats for bX
Clean OAI_REPOSITORY table

For more information about these options, refer to the bX-SFX Configuration Guide.

For information about statistics administrative tasks, refer to the SFX Statistics Module section of the SFX Advanced User's Guide.

Remove SFX Requests as Part of IE8 Fix from Online and Offline Tables

A previously released IE8 Windows issue fix did not work for some customers and was found to affect SFX usage statistics. As part of the SFX 4.0.3 release, this fix was changed so that it no longer affects SFX statistics. Additionally, a script was included in the SFX 4.0.3 release to fix SFX usage statistics. OpenURL requests in statistics that consist only of empty base URLs are changed in the STAT_OBJECT and STAT_OBJECT_OFFLINE tables so that these requests no longer appear in statistics queries by subtracting 50 years from the date. As a result, the empty baseURL requests are no longer counted in the usage statistics.

This script is executed as part of the SFX 4.0.3 release, but can also be run from the Server Admin Utility using this option.

Select 10 Remove SFX requests as part of IE8 fix from online and offline tables to remove the OpenURL requests that consist only of empty base URLs. The following is displayed:

Cleaning STAT_OBJECT from problematic statistics

200 rows cleaned

Cleaning STAT_OBJECT_OFFLINE from problematic statistics

1000 rows cleaned

Done for /exlibris/sfx_ver/sfx_version_3/<instance> /admin/stats/ie8_stat_fix.pl

Fix Statistics File to Make It OS Independent

Archived SFX Statistics had been operating system dependent. This meant that archive files created in one operating system could not be loaded back into a different operating system. This caused problems for customers who changed operating systems and could no longer access their statistic tables.

When fixing existing archive files using this option, it is very important to fix the files in the same SFX installation from which the archive file had originally been created. This is because the operating system where the archive file was originally created has to be identical to the one where it is fixed.

Select option 11 Fix Statistics File to Make It OS Independent to convert statistic files to an operating system independent format. The following is displayed:

Enter complete path and filename to extract the records from: /exlibris/sfx_ver/sfx4_1/sfxlcl41/archive/stats.20110727-16:31:01.saf

About to fix file:

/exlibris/sfx_ver/sfx4_1/sfxlcl41/archive/stats.20110727-16:31:01.saf

Do you want to continue? [Y/n]

Done. File is fixed

A new archive file is created in the directory of the original file with the same name as the original file, but with _fixed added before the extension. For example:

stats.20110727-16:31:01.saf (This is the original file.)

stats.20110727-16:31:01_fixed.saf (This is the OS independent file.)

Clean IP Information from Statistics Data

This option removes all IP information from statistics data.

Select 13 from the Statistics menu. The following is displayed:

Clean IP information from statistics data

1 Clean all IP information from the offline Statistics table records

2 Clean all IP information from Statistics archive files

3 Show scheduled IP clean-up tasks

4 Schedule IP information clean-up task

If you select 1, the following is displayed:

This query will erase all the IP information in the statistics Offline tables. Are you sure that you want to continue? [y/n]

If you select 2, the following is displayed:

This query will erase all the IP information from statistics archive files specified during this option. You can either specify a specific archive file (with complete path) or a directory (in which case all SFX statistics archive files in the directory will be cleaned). Type (f) for file, or (d) for all files in a specific directory: [f] d - Enter complete directory path to clean the records from (all SFX statistics archive files in the directory will be cleaned): f - Enter complete path and filename to clean the records from: About to clean IP information from file(s): Do you want to continue? [Y/n]

If you select 3, one of the following is displayed, depending on if you have scheduled IP clean -up tasks:

There are no scheduled IP clean-up tasks

or:

IP clean-up task schedule is as follows:

[1] 00:00 of every Sunday of every month

[2] 00:00 of day 01 of every month

Would you like to remove any of these? [y/N]

If you select 4, one of the following is displayed:

If you want to schedule for a specific weekday:

If you want to schedule for a specific date:

Please enter scheduling parameters

Time must be in the form HH:MM where

HH can be: 00-23 for midnight-11PM or * for every hour

MM can be: 00-59 or * for every minute

Enter the time to run [00:00]:

Would you like to schedule for a certain weekday? [Y/n] n

Please enter month (1-12 for January-December respectively or '*' for every month) [*]: *

Please enter day of the month (1-31 or '*' for every day) [1]: 1

Successfully added the refresh task to your crontab.