KBTools Command Lines
The command line versions of the KBTools can be useful for SFX administrators who prefer to work in the UNIX environment, and for crontab scheduling purposes.
Command-Line DataLoader
The following contains information about the command-line version of the DataLoader:
|
Usage: ./dataloader.pl --mode= Values: portfolios, update_objects, add_remove_objects, target_cdi_discovery_activation --file= Data file. Full path, or found in '/exlibris/sfx_ver/sfx4_1/sst1_4/dbs/scratch'. --target_service= Name of target-service(s). Required. --sub_target_service= Name of sub-target-service. Should be used in case --target_service is specified which has multiple sub-target-services. The sub-target-service specified should be connected to the Target Service specified in the --target_service. --all_sub_targets (portfolios only) Should be used in case --target_service is specified which has multiple sub-target-services. Will activate all subtargets. --institute= Optional name of institute/group. --column1= Type of Primary Key column. Required for portfolios, update_objects, add_remove_objects. Do not use for target_cdi_discovery_activation Possible values for portfolios, update_objects: ISSN, ISBN, LCCN, OBJECT_ID Possible values for add_remove_objects: OBJECT_TYPE (for load_type=add) OBJECT_ID (for load_type=remove) --column#= Types of other columns. To skip a column in the data file, don't specify a --column# option for that column. Possible values for portfolios: THRESHOLD, TARGET_PARSER_PROGRAM, PARSE_PARAM, ACTIVATION_STATUS, AUTHENTICATION_NOTE, GENERAL_NOTE, INTERNAL_DESCRIPTION Possible values for update_objects: OBJECT_TYPE, LOCAL, TITLE_LANGUAGE, TITLE_TYPE, TITLE_VALUE, TITLE_NON_FILING_CHAR, LANGUAGE, PUBLICATION_DATE, PUBLICATION_PLACE, PUBLISHER Possible values for add_remove_objects: LOCAL, TITLE_LANGUAGE, TITLE_TYPE, TITLE_VALUE, TITLE_NON_FILING_CHAR, LANGUAGE, ISSN, EISSN, ISBN_ELECTRONIC, ISBN_HARDBACK, ISBN_PAPERBACK, LCCN, CODEN, DOI, OCLC_NR, PUBLICATION_DATE, PUBLICATION_PLACE, PUBLISHER --kbart_format (portfolios only) This option indicates that --column definition is not used - but kbart format is used instead --kbart_activation_status= Possible value for This option is required in case kbart_format is used --kbart_ignore_coverage_depth Optional - when included - coverage_depth will be ignored during load --kbart_parse_param= Possible value for Optional - when included - PARSE_PARAM field will be loaded with either jkey=title_url or jkey=title_id if exists --report_mode Report what would be done with the data file, but do not actually update the Database. --add (portfolios only) Add portfolios to the target-service. --update (portfolios only) Update portfolios for the target-service. --remove (portfolios only) Remove portfolios from the target-service. --load_type= Possible values for update_objects: keep, overwrite, remove Possible values for add_remove_objects: add, remove Possible values for target_cdi_discovery_activation: activate_targets_cdi_discovery, deactivate_targets_cdi_discovery --activate (portfolios only) Activate the chosen target and target_services --quiet Don't print summary on success OR ./dataloader.pl --help OR ./dataloader.pl --list --change_activation_status_multi_ops (update portfolios only) Allow updating multiple OPs (under same object and same target-target_service) by Parse Params and Threshold matching Note: For threshold matching, update can only be for availability_status (active/inactive) |
Command-Line Information
The following files contain command-line information:
- Cleaned input file – /exlibris/sfx_ver/sfx4_1/
/dbs/scratch/ .ok - Report of loading – /exlibris/sfx_ver/sfx4_1/
/dbs/scratch/ .ok.err - Log file of dataloader – /exlibris/sfx_ver/sfx4_1/
/dbs/scratch/dataloader.pl.log
Command-Line Examples
To run the DataLoader program on a file called testAmerican_Medical_Association that exists in the scratch directory, use the following command:
|
./dataloader.pl --mode=portfolios -- file=testAmerican_Medical_Association -- target_service=AMERICAN_MEDICAL_ASSOCIATION-getFullTxt --column1=ISSN -column2=AVAILABILITY --column3=THRESHOLD --update --activate |
In this example, the DataLoader is used to update the OBJECT_PORTFOLIOs for the American Medical Association target (activate and add a local threshold). At the same time, the target and target_service will be activated.
The tab-delimited TXT input file looks like the following:
|
0098-7484 ACTIVE $obj->parsedDate(">=",2002,131,1) |
The following is an example for commandline dataloader to add objects:
|
./dataloader.pl --mode=add_remove_objects --load_type=add --file=test10.txt --column1=OBJECT_TYPE --column2=ISBN_HARDBACK -column3=TITLE_VALUE --column4=PUBLICATION_PLACE --column5=PUBLISHER --column6=PUBLICATION_DATE |
An example for commandline dataloader in kbart format is the following:
|
./dataloader.pl --mode=portfolios --file=/exlibris/sfx_ver/sfx4_1/sst1_4/dbs/scratch/dataloader.txt --target_service=ADISONLINE-getFullTxt --all_sub_targets --kbart_format --kbart_activation_status=ACTIVE --kbart_ignore_coverage_depth --kbart_parse_param=title_url --add |
The tab-delimited TXT input file looks like the following (only the first four columns are displayed here):
|
publication_title print_identifier date_first_issue_online num_first_vol_online Philosophical Transactions 0370-2316 1665 1 |
For a complete list of the columns of the KBART format, see the KBART section of the SFX General User’s Guide.
Command-Line Lookup Tool
The following contains information about the lookup tool:
|
Help information: ./lookup.pl --input-file= --lookup-content= input_file location defaults to /exlibris/sfx_ver/sfx4_1/dana_3/dbs/scratch if no path is given. Possible field_name values are: ISSN, eISSN, ISBN, LCCN, TITLE, OBJECT |
Command-Line Lookup Tool Examples
The following are lookup tool examples:
|
./lookup.pl --input-file=test1.txt --input-content=TITLE --lookup-content=ISSN ./lookup.pl --input-file=test1.txt --input-content=TITLE --lookup-content=LCCN |
Command-Line Export Tool
The command-line version of the Export tool allows you to run all of the advanced export options that can be performed from the Export Tool option of the SFX Admin Center. Additionally, it is possible to run the basic export query export_urls. No other basic export options can be performed from the command-line version of the Export tool. All other basic export options can be run only via the Export Tool option of the SFX Admin Center. For more information, see Appendix C: External Export.
The command-line export tool is located in: /exlibris/sfx_ver/sfx4_1/
This tool allows you to do the following actions in a command-line mode:
- Create different types of advanced export files.
- Export URLs from active targets, target services, and active object portfolios. (This can also be performed from the basic export section of the SFX Admin Center).
- From the UNIX command prompt, type da.
- Type cd kbtools/.
- Type ./export.pl [Options].
The following is a list of all the options that can be used in the command-line export tool:
|
Usage: ./export.pl Basic Queries: --basic Values: urls --institute Optional name of institute/group to filter results. Multiple option. --service Optional name of service(s) to export. Multiple option. If not given, export ALL services. --output_dir (profile or manual) Optional alternate location of output files. Output dir defaults to '/exlibris/sfx_ver/sfx4_1/sfxlcl41/dbs/scratch'. Advanced Queries: --mode Values: profile, manual --output_dir (profile or manual) Optional alternate location of output files. Output dir defaults to '/exlibris/sfx_ver/sfx4_1/sfxlcl41/dbs/scratch'. --filename (manual only) Specify output file name. If filename value is contains a relative or absolute path, this is the same as specifing --output_dir. i.e., --filename=/some/path/to/file.xml is the same as specifying --output_dir=/some/path/to --filename=file.xml This option is not available for the Google export.
--profile (profile only) Name of profile. Required. --institute (manual only) Optional name of institute/group to filter results. Multiple option. --format (manual only) |
|
Required. Values: TXT, HTML, XML, ALEPH_SEQ, PUBMED, KBART Output file format. --external_input (profile or manual) Optional name of external TXT format export file to include. Location defaults to '/exlibris/sfx_ver/sfx4_1/sfxlcl41/dbs/scratch' if no path is given --target (manual only) Optional name of target to export. Multiple option. If not given, export ALL active targets/target_services. --service (manual only) Optional name of service to export. Multiple option. If not given, export ALL services from the following: Values: getFullTxt, getHolding, getTOC, getAbstract --object_type Specify types of objects to export (e.g., JOURNAL, BOOK). Multiple option. If not given defaults to JOURNAL --compare (profile or manual) For XML export only. Optional name of previous TXT format export file to compare to. Must be located in '/exlibris/sfx_ver/sfx4_1/sfxlcl41/dbs/scratch/e_collection_update' For profile, only used and is required if profile |
|
is 'compare'. --prefix (manual only) For XML export only. Optional file prefix. --link_text (manual only) For XML export only. '856 $y' tag --inst_info (manual only) For XML export only. '852 $a' tag --target_note (manual only) For XML export only. '245 $h' tag --base_url (manual only) For XML export: '856 $u' tag For PUBMED export: required Base URL. --gif_url (manual only) For PUBMED export only. Required gif URL. --provider_id (manual only) For PUBMED export only. Required ProviderId. --no_issn_info (manual only) For GOOGLE export, do not include issn --no_portfolio_info (manual only) For GOOGLE export, do not include 866 a --[no]include_categories (profile or manual) For XML export, will include (or exclude) categories in the export file. Not included by default --[no]include_local_attributes (profile or manual) For XML export, will include (or exclude) local_attributes in the export file. Not included by default |
|
--[no]include_servicetype (profile or manual) Include or exclude "sfx.servicetype=yes" from 856 url. Included by default. --[no]institute_info (profile or manual) For XML export, will include (or exclude) institute_info in the export file. Not included by default --[both]original_title_format Which title will be included in export in case of Chinese object Values: both, china, taiwan --include_alternative_titles (profile or manual) For TXT, XML exports, will include (or exclude) alternative title in the export file. Not included by Default Display list of target names: --show_targets --compress (google only) ZIP the output file --exclude_not_changed_objects - to exclude from the export file objects that were not changed, both when manually exporting or using profiles |
Command-Line Export Tool Examples
The following are export tool examples:
- ./export.pl --mode=manual --format=TXT --target=AMERICAN_MATHEMATICAL_SOCIETY --service=getFullTxt
- ./export.pl --mode=manual --format=ALEPH --service=getFullTxt
--output_dir=/exlibris/sfx_ver/sfx4_1/<instance>
- ./export.pl --mode=manual --format=XML --service=getFullTxt
---filename=/exlibris/sfx_ver/sfx4_1/<instance>/lieve.xml --target=NATURE
--target=ERUDIT -->
- ./export.pl --mode=manual --format=XML --service=getHolding --institute=InstA
--prefix=OPAC-XML --base_url=http://sfxserver:3210/<instance>
--link_text="SFX linking" --inst_info=InstitutionA --target_note="[electronic content]"
- ./export.pl --mode=manual --format=XML --service=getFullTxt
--base_url http://sfxserver:3210/<instance> --noinclude_servicetype --include_categories
--external_input=additional.txt --compare=TEST-e-collection.20060613120211
- ./export.pl --mode=profile --profile=metalib
- ./export.pl --mode=profile --profile=primo --compare=e-collection.20120321165005
- ./export.pl --mode=manual --format=KBART --service=getFullTxt
The following is an example to create Google Export file with holdings information for three institutes:
- ./export.pl --mode=manual --format=GOOGLE --object_type=JOURNAL --object_type=BOOK --institute=<institute_name_1> --institute=<institute_name_2> --institute=<institute_name_3> --service=getFullTxt
- ./export.pl --format=XML --mode=manual --compare=e-collection.20110612125533 -exclude_not_changed_objects
- If no --institute=<institute> is included, anything active for either default or any of the institutes/groups is exported.
- If one or more --institute=<institute> is specified, anything active for the institute(s) or active for DEFAULT or a group the institute is affiliated with and not explicitly inactive for the institute is exported.
- If one or more --institute=<group> is specified, anything active for the group(s) is exported.
- If --institute=DEFAULT is specified, anything active for DEFAULT is exported.
Command-Line TargetMatcher
|
./target_matcher.pl
--file Name of input file, file should exist in scratch directory.
--target_service Name of target service format: TARGET_INTERNAL_NAME-SERVICE e.g CAMBRIDGE_UNIVERSITY_PRESS-getFullTxt multiple argument
--key primary key, valid values: ISSN,LCCN, ISBN
--mode type of action valid values: report,activation
--target_type match against group of targets valid values: ACTIVE,INACTIVE,SPECIFIC,ALL, ACTIVE_FULLTXT, INACTIVE_FULLTXT,ALL_FULLTXT |
Running the TargetMatcher Program Examples
|
./target_matcher.pl --file=test.txt --key=ISSN --mode=report --target_type=ACTIVE |
|
./target_matcher.pl --file=test.txt --key=ISBN --mode=activation --target_type=ACTIVE_FULLTXT |
- AMERICAN_INSTITUTE_OF_PHYSICS_OJPS-getFullTxt
- AMERICAN_CHEMICAL_SOCIETY-getFullTxt
|
./target_matcher.pl --file=test.txt --key=ISSN --mode=activation --target_type=SPECIFIC --target_service=AMERICAN_INSTITUTE_OF_PHYSICS_ OJPS-getFullTxt --target_service=AMERICAN_CHEMICAL_SOCIETY-getFullTxt |
Log and Report Files
- In report mode:
- target_matcher.pl.result – contains a list of created files
- Export files per target service for which hits were found
- <input_file>.ok.err – contains an analysis of input data (only contains information if input data is incorrect)
- target_matcher.pl.log – a general log file
- In active mode:
- target_matcher.pl.result – contains a list of all targets, target services, and portfolios activated
- <input_file>.ok.err – contains an analysis of input data (only contains information if input data is incorrect)
- <input_file>.ok.portfolios.out – a complete file with all portfolios
- target_matcher.pl.log – a general log file
Command-Line Category Tool
- Type da.
- Type cd category.
- Type ./category_export <options> to start the export or type
./category_load <options> to start loading.
Category Export
|
category_export [--help] <command> [--input=<file name>] --help Print this screen --output=<file name | 'standardoutput'> File where the data will be stored or 'standardoutput' to print to screen --export_global Export global and translated categories --export_non_translated Export global categories for which no translated counterpart exists --export_local_without_assigned Export local categories without assignments --export_local_with_assigned Export local categories with assignments --export_local_with_count Export local categories with assignment record count --export_all Export all category information for list of objects (using input file) --input=<file name> File containing data for --export_all command --key= (issn|lccn|object_id) |
- /category_export --output=standardoutput --export_global
- ./category_export --output=../dbs/scratch/test1.txt --export_global
- ./category_export --output=standardoutput --export_non_translated
- ./category_export --output=../dbs/scratch/test2.txt --export_non_translated
- ./category_export --output=standardoutput --export_local_without_assigned
- ./category_export --output=standardoutput --export_local_with_count
- ./category_export --output=standardoutput --export_all --input=export9.txt --key=object_id
Category Load
The following is the help screen for the Category Load Tool:
|
category_load [--help] <command> --input=<file name> [--replace] --help Print this screen --input=<file name> File containing the records to import --encoding=<utf8 | latin1> Encoding of the input file
command: --load_translated Import translated categories\subcategories from file --load_local Import local categories\subcategories from file --load_global_assign Import local category assignments to global --load_local_assign Import local category assignments to local
options: --key=<ISSN_ID|ISBN_ID|LCCN_ID|OBJECT_ID> Key included in input file for adding local assignments --option_local=<add|addAndRemove|update> Options for adding local categories\subcategories --option_assign=<add|replace> Options for adding local assignments |
The following are examples of Category Load output:
- ./category_load --load_translated --input=../dbs/scratch/test1.txt --encoding=utf8
- ./category_load --load_local --input=../dbs/scratch/test2.txt --encoding=utf8 --option_local=add
- ./category_load --load_local --input=../dbs/scratch/local_cat1.txt --encoding=utf8 --option_local=add
- ./category_load --load_local --input=../dbs/scratch/test2.txt --encoding=utf8 --option_local=addAndRemove
- ./category_load --load_local_assign --input=../dbs/scratch/local_cat_assign4.txt --encoding=utf8 --key=LCCN_ID --option_assign=add
- ./category_load --load_local_assign --input=../dbs/scratch/local_cat_assign2.txt --encoding=utf8 --key=ISSN_ID --option_assign=add
- ./category_load --load_global_assign --input=../dbs/scratch/test3.txt --encoding=utf8 --key=OBJECT_ID --option_assign=add
- ./category_load --load_global_assign --input=../dbs/scratch/global_cat_assign1.txt --encoding=utf8 --key=ISSN_ID --option_local=add
- ./category_load --load_local_assign --input=../dbs/scratch/local_cat_assign3.txt --encoding=utf8 --key=OBJECT_ID --option_assign=add
- ./category_load --load_local_assign --input=../dbs/scratch/local_cat_assign3.txt --encoding=utf8 --key=OBJECT_ID --option_assign=replace
Command-Line Tool: change_availability.pl
- Enter da.
- Enter cd kbtools.
- Enter ./change_availability.pl [option].
Examples
- ./change_availability.pl --ACTION=ACTIVE --ALL_TARGETS
This makes all targets, target services, and portfolios active.
- ./change_availability.pl --ACTION=INACTIVE --ALL_TARGETS
This makes all targets, target services, and portfolios inactive.
- ./change_availability.pl --ACTION=ACTIVE --ALL_FREE_TARGETS
--EXCLUDE_T=AMAZON
This activates all targets, target services, and portfolios where target service has a status of free, except for the AMAZON target.
- ./change_availability.pl
--TARGET_SERVICE=JSTOR_COMPLETE-getFullTxt --TARGET_SERVICE=PROJECT_MUSE_STANDARD_COLLECTION-getFullTxt --AUTO_ACTIVE=YES --ACTION=ACTIVE
This activates the targets in the command, their full-text target services, and full-text portfolios. It also sets the AutoActive option to yes so that portfolios added in future to these target services are active.
- ./change_availability.pl
--TARGET="JSTOR_COMPLETE" --TARGET="PROJECT_MUSE_STANDARD_COLLECTION" --ACTION=SEARCH_ACTIVE
This activates the two targets listed in the command for CDI Discovery.
|
./change_availability.pl [command] commands: |
|
|
--ACTION |
ACTIVE/INACTIVE/SEARCH_ACTIVE/SEARCH_INACTIVE - mandatory option.
Note: SEARCH_ACTIVE and SEARCH_INACTIVE can only be used for targets that can be activated for CDI search/discovery. It cannot be used in combination with other options related to target services or portfolios. |
|
--AUTO_ACTIVE |
Value can be YES/NO. Sets AUTOACTIVE for target services which were Activated/Deactivated. |
|
--ALL_FREE_TARGETS |
Activate/Deactivate ALL targets, target services and portfolios where target service has status 'free'.
Should not be combined with options: --ALL_TARGETS, --ALL_TARGET_SERVICES, --TARGET, --TARGET_SERVICE, --PORTFOLIOS |
|
--ALL_TARGETS |
Activate/Deactivate ALL targets, target services and portfolios.
Should not be combined with options: --ALL_FREE_TARGETS, --ALL_TARGET_SERVICES, --TARGET, --TARGET_SERVICE, --PORTFOLIOS |
|
--ALL_TARGET_SERVICES |
Activate/Deactivate ALL target_services and portfolios.
Should not be combined with options: --ALL_FREE_TARGETS, --ALL_TARGET_SERVICES, --TARGET, --TARGET_SERVICE --PORTFOLIOS |
|
--ALL_SERVICE=<service> |
Activate/Deactivate ALL targets, target services and portfolios from a specific service type (repeatable option).
Should not be combined with options: --TARGET_SERVICE, --PORTFOLIOS |
|
--TARGET=<target> |
Activate/Deactivate specific target and all target services and portfolios connected that to target (repeatable option). Should not be combined with options: --ALL_FREE_TARGETS, --ALL_TARGET_SERVICES, --TARGET_SERVICE --PORTFOLIOS |
|
--TARGET_SERVICE=<target service> |
Activate/Deactivate specific target_service and all targets and portfolios connected to it (repeatable option). Should not be combined with options: --ALL_FREE_TARGETS, --ALL_TARGET_SERVICES, --TARGET --PORTFOLIOS |
|
--PORTFOLIOS=<target service> |
Activate/Deactivate only the portfolios of a specific target_service (repeatable option). Should not be combined with options: --ALL_FREE_TARGETS, --ALL_TARGET_SERVICES, --TARGET, --TARGET_SERVICE |
|
The following options can be used in combination with the options above - to further limit the items that will be activated/deactivated: |
|
|
--OBJECT_TYPE=<object_type> |
Activate/Deactivate only the portfolios linked to object of selected type (repeatable option). |
|
--EXCLUDE_T=<target> |
Excludes the target (with all target services and portfolios connected) listed from being activated or deactivated. Repeatable option - cannot be used together with --TARGET and --ALL_TARGET_SERVICES options. |
|
--EXCLUDE_TS=<target_service> |
Excludes the target service (and its portfolios) listed from being activated or deactivated. Repeatable option - cannot be used together with --TARGET_SERVICE option. |
|
--EXCLUDE_OP =<target_service> |
Excludes the portfolios connected to the target service listed from being activated or deactivated. Repeatable option - cannot be used together with --PORTFOLIOS option. |
|
Note: It is possible to use '%' to indicate truncation in the target internal name. |
|
|
Example: ./change_availability.pl --TARGET="EBSCO%" --ACTION=INACTIVE will deactivate all targets, target services and portfolios for targets with internal name starting with EBSCO. |
|
|
./change_availability.pl --TARGET="EBSCO%" --ACTION=INACTIVE |
ONIX Import Tool: onix_import.pl
Overview (from http://www.editeur.org/onixserials.html)
- Place the SOH XML file obtained from the provider in the SFX file system for the instance. A recommended location is /exlibris/sfx_ver/ sfx4_1/<sfx_instance>/dbs/scratch.
The command-line ONIX import tool is located in /exlibris/sfx_ver/ sfx4_1/<sfx_instance>/admin/kbtools.
-
Access the tool by entering the following commands at the UNIX prompt:
- da
- cd kbtools/
|
usage:./onix_import.pl [--options] FILE.xml --vendor=<vendor> which vendor mapping table to use (will prompt if not supplied) --report_mode don't change anything, just report what would happen --deactivation, --nodeactivation deactivate all relevant targets, target services and portfolios (enabled by default) --loadlocal, --noloadlocal load local thresholds if onix value differs from global (enabled by default) --list display a list of available vendors --log_file=<logfile> --verbose, --noverbose default is off --help --debug |
|
./onix_import.pl /exlibris/sfx_ver/ sfx4_1/sfxlcl4/dbs/scratch/ ebsco_onix.20050731.xml --vendor=ebsco
./onix_import.pl /exlibris/sfx_ver/ sfx4_1/sfxlcl41/dbs/scratch/ ebsco_onix.20050731.xml --vendor=ebsco --noloadlocal
./onix_import.pl /exlibris/sfx_ver/ sfx4_1/sfxlcl41/dbs/scratch/ ebsco_onix.20050731.xml --vendor=ebsco --verbose |
Reports
|
/exlibris/sfx_ver/ sfx4_1/sfxlcl41/dbs/scratch/ebsco_onix.20050731.xml.ok.err /exlibris/sfx_ver/ sfx4_1/sfxlcl41/dbs/scratch/onix_import.log |