Entity Extraction Workflow

This guide outlines the step-by-step process for using Entity Extraction.

Step 1: Upload Resources to Collections

To extract entities from digital resources, start by adding digital representations to a collection.

Note: Supported formats include JPG, PNG, TIFF, and textual PDF.

Once the titles are uploaded, you can run the Entity Extraction process.

  1. Go to Specto > Data Management > Manage Collections.
  2. In the list of collections, locate the relevant collection that contains digital content.
  3. Select the More Actions (ellipses) button and select the Digital Metadata Assistant option. This is the Specto homepage.

Step 2: Run Entity Extraction

Once on the Specto homepage for your collection, follow the steps to run entity extraction:

For the best view of the titles and resources in your collection, switch to Gallery view in Specto by selecting the view options in the right-hand corner.

For each title, the Specto process will run against one Representation, following this order, as available:

  1. First Derivative Representation

  2. First Primary Representation

  3. First Modified Primary Representation

  4. If none of these exist, Specto will use the first available Representation, not including Auxiliary content.

Images

  1. Select the Entity Extraction button. This process reviews all relevant titles in the collection to identify and differentiate between images (pictures) and documents (containing text).

  2. The AI extracts faces and objects from the images. The process groups entities by similarity (e.g., all pictures of the same person are in one group).
  3. At each title level, face entities are visible in the image via a bounding box that highlights the extracted entity in the new Viewer.

Entities are listed on the Viewer's left pane under Entities, allowing users to select and highlight the bounding boxes.

To be detected, a face must be no smaller than 40x40 pixels in an image with 1920X1080 pixels. Images with dimensions higher than 1920X1080 pixels will need a larger minimum face size proportionally.

Documents

The purpose of the ALTO file is to enable hit highlighting for document search, where the search term is highlighted directly in the document. This happens in the Specto environment in Alma.

  1. If none are available, the AI generates a full-text document and a corresponding ALTO for each file.
  2. The AI extracts names of people, locations and organizations from images.

  3. The process groups these entities by similarity (e.g., all mentions of the same person are in one group, all mentions of the same city are in another group).
  4. At the title level, each entity appears in the document under the Viewer Entities pane.
  5. Selecting an entity highlights the entity's mentions.
  6. With the generated full-text, search functionality is also available in the Viewer's Search pane and highlights keyword search results in the document.

To learn more, see Tesseract OCR extraction and ALTO generation tool.

Step 3: Review Extracted Entities

  1. Once the extraction and grouping process is complete, the entities appear in the left-side panel, under the collection name.
  2. The Digital Titles section leads to the collection homepage and displays all the titles in the collection.
  3. Select the type of entity you would like to review:
    1. Faces – to review the groups of faces detected in the images.
    2. Text Mentions – to review the groups of people, organizations and locations mentioned in the documents.
    3. Objects – to review the objects detected in the images.
A Search feature enables users to easily locate specific tag suggestions for faces and text mentions.
  1. Within each section, you can select specific groups and review the groupings' accuracy to ensure that all group members are appropriately assigned before tagging. You can use the Sort by filter to change how you preview the results.
    1. For grouped faces and text mentions, if one or more entities in a group are inaccurately associated, select the relevant items and use the Remove from Group bulk functionality to ensure the group is accurate for the next tagging step.

    2. Object grouping is broader and is further elaborated in tagging with or without linking to authority.
  2. If you do not want to tag a detected entity or entities, you can dismiss the group in one of the following ways:
    1. On the entity section level, select the More Actions button (ellipses) under the image and select the option to Dismiss Entity.
    2. Within the entity itself, select the Dismiss Entity function from the left-side panel. Dismissed entities are not removed from the collection, but they are not available for tagging in this collection or marked on individual titles. The grouped entities are marked as Dismissed and can be restored later if necessary.
      You can multi-select different entity groups by selecting the checkboxes , and dismiss selected entity groups in bulk.
    3. For objects, you can dismiss a group by selecting the ellipses at the end of the row in the object table and selecting the Dismiss Entity Group option.

    Once you dismiss a group of objects, you can restore it by going to the Entity Group Tagging > Dismissed Group > Restore Entity Group.

    Resore Dismissed Entity button shown restoring an entity group

Each Entity Group, whether faces, text mentions, or objects, will have a status depending on where it is in the extraction process. The status appears as an icon in the lower left corner of the Entity group image. The status types are marked with the following indicators:

  • Untagged: Blue Question mark icon

  • Tagged: Green checkmark icon

  • Dismissed: Gray X icon

The actions you can perform on an Entity Group will also depend on the current status:

  • Untagged: Edit Entity Group, Dismiss Entity

  • Tagged: Edit Entity Group (triggers re-tagging ), Dismiss Entity (clears tagging and converts to recommendation)

  • Dismissed: View Recommendation, Restore Entity (restores recommendation and sets status to Untagged)

  • In Progress: All actions disabled until tagging completes

Step 4: Tag With or Without Linking to an Authority

Entity tagging can be performed manually or derived from metadata. For manual tagging, follow the process detailed in the steps below.

Metadata-derived Entity Tagging

Entity tagging has been enhanced to derive tag names from the linked metadata field rather than the group name. As a result, tags displayed in the Viewer reflect the value and language used in each record's metadata.

Changes to linked metadata fields automatically update the associated entity tags (like the manual update action), while removing a linked field removes the tag (like the manual untag action). In addition, authority updates processed through the Preferred Term Correction job automatically update related entity and group tags, ensuring that the displayed values remain synchronized with the latest authority data.

Use authority linking whenever possible to ensure consistency and interoperability across cataloging systems, and to support linked data in Primo NDE.

When tagging without an authority, you can create entity tags by entering either a simple name (for example, “Einstein, Albert”) or a complete metadata field with subfields (for example, $$a Einstein, Albert $$x ...), when cataloging in MARC21 or UNIMARC. When only a name is entered, the system automatically treats it as subfield $$a for MARC records.

Faces

Once you are ready to tag the entities, select Edit Entity Group or select the item to begin. You can tag:

  • With Authority Linking
  • Without Authority Linking

Tag suggestions based on AI or previous user tagging will appear along with banners denoting the source of the tag. An orange banner (lightbulb icon) will appear for user tagging suggestions, while a purple one (stars icon) will appear for AI tagging recommendations .


User may review the record by selecting the View Authority Record to determine if this is the Authority to which you want to link the grouped entity.
notification showing entty suggestion is based on a tagged member in a groupa notification showing the authority recommendations were generated by AI

Tag with Authority Link
  1. Select the Search an Authority option in the Entity Tagging box.

  2. Search the authority database and select the authority you wish to link to. Use filters and facets to narrow your search, and select individual records to view their metadata to ensure accuracy.
  3. The authority search results are pre-filtered based on the institution configuration setup for allowed Authority Vocabularies. If there are relevant suggested authorities prepopulated in the search, it is suggested to select them. This setting is available under the Customer Parameter authority_name.
  4. Select the relevant record.

  5. Select the Tag Entity button on the left panel. This tags ALL titles associated with the extracted entities in the group.
Tag without Authority Link
  1. Select the Without an Authority option in the Entity Tagging box.
  2. When a name is available for the entity, an authority search is auto-populated. This happens due to previous tagging or AI suggestions. Otherwise, enter the name in the Name field, meeting your institutional cataloging requirements.
  3. Select the Tag Entity button. This tags ALL titles associated with the extracted entities in the group.

Use authority linking whenever possible to ensure consistency and interoperability across cataloging systems, and to support linked data in Primo.

Text Mentions

The Text Mentions table includes the following columns:

  • Entity: identified name of the grouped entity.
  • Tag Suggestion:
    • Stars: AI-generated recommendation for an Authority record dentified as appropriate for this entity.
    • Lightbulb: user-generated suggestion based on a previously tagged member of this group by a cataloger.
  • Entity Type: distinguishes between people, locations and organizations.
  • Status: untagged, tagged, dismissed.
  • Number of Titles: number of titles in which this entity was mentioned. Note that there may be multiple mentions of the entity within each title.

Once you are ready to tag the entities, select Edit or select the item to begin. You can tag:

  • With Authority Linking
  • Without Authority Linking
Tag With Authority Link

If an AI recommendation or user suggestion is already available, the Name field is pre-populated with the proposed Authority. You may review the record by selecting the View Authority Record to determine if this is the Authority to which you want to link the grouped entity.

  1. If no recommendations or suggestions are available, select the Search an Authority option in the Entity Tagging box.
  2. Search the authority database and select the authority you wish to link to. Use filters and facets to narrow your search, and select individual records to view their metadata to ensure accuracy.
  3. Select the relevant record.
  4. Select the Tag Entity button on the left panel. This tags ALL titles associated with the extracted entities in the group.
Tag Without Authority Link
  1. Select the Without an Authority option in the Entity Tagging box.
  2. Enter the name manually in the Name field below, meeting your institutional cataloging requirements.
  3. Select the Tag Entity button below. This tags ALL titles associated with the extracted entities in the group.

Objects

  1. To tag extracted objects, select the object's name in the Entity Group column in the object table or select the ellipses at the end of the table row and select the Edit Entity Group option. A new page opens. On the left side when in split view, you see all the appearances of the selected object type in the collection, and on the right side, you have the viewer.

  2. Choose the object or objects you wish to tag by selecting the checkbox under the title and selecting the Tag Entities button in the upper right side of the screen. Once tagged, the objects status is changed to Tagged they appear in the viewer within the Specto Title window.
Tag with Authority Link
  1. Select the Search an Authority option in the Entity Group Tagging pop-up.
  2. Search the authority database and select the authority you wish to link to. Use filters and facets to narrow your search, and select individual records to view their metadata to ensure accuracy.
  3. Select the relevant record.

  4. Select the Tag button in the bottom right corner of the tagging popu-up. This tags all SELECTED entities in the group.
Tag without Authority Link
  1. Select the Without an Authority option in the Entity Group Tagging box.
    entity tagging without authority
  2. Enter the metadata field number (a three-digit number) in the Field box and enter the 1st and 2nd indicators according to your institutional cataloging requirements.
  3. Enter the subfield and name in the Field Value box below, meeting your institutional cataloging requirements. Recommendation is to use proper subfield syntax, though if you enter only a name, the system assumes this falls under $$a .
  4. Select the Tag button below. This tags the selected titles in the group. The tag appears in the metadata under the field you entered with the indicators you chose.