Merging file metadata with records

Introduction

The Merge from file wizard enables Super Administrators to import metadata from a file and use the values to replace, combine and remove metadata of existing records. This wizard is similar to the Update via file wizard except has much greater control over the manner in which metadata is integrated with existing records.

The Merge from file wizard will import the following file formats:

• Comma Separated Values (CSV) or Tab Separated Values (TAB or TSV) spreadsheets, with either ‘.csv’, ‘.tsv’ or ‘.tab’ file extensions.
  The file must contain a column that identifies the record in Keeping Culture. This column can be a record ID, UUID, Archive Number, Archive View Number, or a unique record name/title.
  See Creating CSV metadata files for importing and updating records.
• KC Structured Data format, which is a special format that only Keeping Culture archives can process and is created when exporting records from a Keeping Culture archive. These files must have a ‘.csj’ file extension.
  If metadata includes audio or video annotations, you must also provide the corresponding media files. Be sure to include annotation media in the export and supply both the ‘media’ folder and metadata file when using the Merge from file wizard.
  See Using ‘KC Structured Data’ metadata import and export format.

Your prepared media and import file need to be uploaded to the Archive Files directory before commencing the merge process.

The merge operation does not replace media files for Archive Item classes. If you wish to replace media files in the archive, refer to Replacing media and/or thumbnails.

A record is only updated if it is correctly identified. If the record is not found, it will be skipped.

You will be required to login using a Super Administrator account to access the Merge from file wizard.

Merge operations and other actions

Below is a list of all merge operations, though some may not be applicable to every attribute:

• Replace with file value: the record’s attribute value is overwritten by the import file value.
• Replace with file value when existing value is empty: if the record’s attribute has no value, the import file value is applied to the record.
• Merge file values into existing values: (repeatable attributes only) combines the record’s attribute values with the import file’s values without duplicate entries.
• Append file value when different from existing value: (textbox attributes only) If the record’s attribute value is different to the import file’s value, combine the texts by adding the file’s value to the end of the record’s value.
• Prepend file value when different from existing value: (textbox attributes only) If the record’s attribute value is different to the import file’s value, combine the texts by adding the file’s value to the beginning of the record’s value.
• Remove the file values from the existing value: If the import file’s value matches the record’s attribute value, then remove the value from the record value. For single value attributes, the record value will be empty after removal has occurred. For repeatable attributes, any entries not found in the import file will be retained by the record.

When the Skip merge operation if the file value is empty setting is enabled, the attribute value of the existing record will remain unchanged if the corresponding file value is absent.

 The On duplicate entries, assign the value from either the Import File or Existing Record action applies to the follow attributes:

• Annotations: If a duplicate annotation is detected, you can either update its date, contributors, text, recorded media, and source with import file values, or keep the current record’s values.
• Features: If a duplicate feature is detected, you have the option to either update its shape, location, and reference association based on the import file, or keep the values already present in the existing records.
• Compound attributes: when a duplicate primary (or parent) value is detected, you have the options to apply the child attribute values from the import file, or keep the existing child values.

A careful approach is required

When conducting a Merge from file operation, please be aware of the following:

• Record edit locks are forcibly removed on records being edited by another user, which could result in the loss of another user’s changes.
• If you make a mistake with your merge, the magnitude of the error could lead to thousands of records being changed, and the only course of action to correct this mistake is to restore the database from the overnight backup.

For these reasons, it’s recommended that you only perform a merge operation when you’re certain no one else is using the system and as early as possible in the morning. Why the morning? The less time that elapses between the backup and the mistake, the less likely it is that other changes will be lost when rolling the database back to the last backup.

How to merge records

Steps for CSV & TAB formatted merge file

1. To access the Merge from file wizard, do one of the following:
   • For Archive Item and Knowledge classes: go to the Main Menu → Administration tab, then in the ‘cog’ button menu next to the name of the record class you’re merging, select Merge from file.
   • For List classes: go to the Main Menu → Administration tab → Lists hyperlink. Locate the class of the records you’re merging and click the Merge via file hyperlink in the row actions.
2. In the wizard, click Choose file.
3. In the Choose file window, navigate to, and select the CSV or TAB file that you’ve previously uploaded to the Archive Files directory. Click Select.
4. Select ‘CSV & TAB File Importer’ from the Format list. Click Continue.
5. Click Continue again.
6. In the ‘Identification’ wizard step, you will need to do the following:
   
   In Column, select the column in the CSV file that contains the value to use to identify the record in the archive. This could be the record’s ID, UUID, Archive Number, Archive View Number, or a unique record name/title depending on the record class.
   
   In Format, select the type of the value in the column you’ve selected above. This tells the wizard how to interpret the value you’ve selected in Column.
   
   When processing Archive Items, the wizard will display options for identifying both the Archive Item and Archive View records. You must identify at least one of these record classes.
   
7. Click Continue. The wizard will now check that the merge file contains at least one record that can be identified. If records are not identifiable, then you’ll be redirected back to the previous step.
8. In the Assignment wizard step, systematically work through the list of class attributes (fields) assigning a column that contains the value for the attributes. It is not necessary to assign a column to every attribute.
   
   Set the preferred merge operate and actions for each attribute as described in the Merge operations and other actions section above.
   
   For attributes that accept multiple values, the column’s value can be separated into multiple values using a delimiter, usually a comma. For example, a column value of ‘Matt Smith, Jill Green, Ken Bell’ can be broken into three separate people using a comma ‘,’ delimiter.
   
   For attributes that reference their values from multiple classes, you must specify the class that the column’s value will be imported into.
   
   Once you have finished making your column assignments, click Continue.
9. Review your proposed merge changes. If you’re happy to proceed, click Continue. Otherwise click Back to refine your selection.
10. The merge process will begin, once complete you will receive statistic and possibly error information on the outcome of the merge. Click Done.

Steps for KC Structured Data formatted merge file

1. To access the Merge from file wizard, do one of the following:
   • For Archive Item and Knowledge classes: go to the Main Menu → Administration tab, then in the ‘cog’ button menu next to the name of the record class you’re merging, select Merge from file.
   • For List classes: go to the Main Menu → Administration tab → Lists hyperlink. Locate the class of the records you’re merging and click the Merge from file hyperlink in the row actions.
2. In the wizard, click Choose file.
3. In the Choose file window, navigate to, and select the KC Structured Data (.csj) file that you’ve previously uploaded to the Archive Files directory. Click Select.
4. Select ‘KC Structured Data Importer’ from the Format list. Click Continue.
5. In step two of the wizard, under the File Reader configuration heading, pair any comparable content restrictions from this archive with those from the source archive. Any unassigned restrictions will be ignored during the merge.
   
   Review the content restrictions applied to the classes and attributes of the source archive. You will need to carefully consider whether similar restrictions should be applied to the classes and attributes of this archive.
   
   Click Continue.
6. In the Identification wizard step, you will need to do the following:
   
   In Column, select the file property that contains the UUID value to use, to identify the record in the archive. This could be the ‘Item Record UUID’, ‘View Record UUID’ or ‘Record UUID ‘depending on the record class.
   
   In Format, select ‘Universally Unique ID (UUID)’ option if it’s not already selected.
   
   When processing Archive Items, the wizard will display options for identifying both the Archive Item and Archive View records. You must identify at least one of these record classes.
   
7. Click Continue. The wizard will now check that the import file contains at least one record that can be identified. If records are not identifiable, then you’ll be redirected back to the previous step.
8. In the Assignment wizard step, systematically work through the list of attributes (fields) assigning a property from the import file to each attribute in the archive.
   
   Set the preferred merge operate and actions for each attribute as described in the Merge operations and other actions section above.
   
   Only compatible properties will be display for each attribute. The system will attempt to recommend a ‘Good Match’ based on the properties of the source archive attributes.
   
   Any attributes that are unassigned a property for the import file will be skipped.
   
   For an explanation of why some attributes have ‘Unknown Reference!’, please refer to Using ‘KC Structured Data’ metadata import and export format for more information.
   
   Once you have finished making your attribute assignments, click Continue.
9. Review your proposed merge changes. If you’re happy to proceed, click Continue. Otherwise click Back to refine your selection.
10. The merge process will begin, once complete you will receive statistic and possibly error information on the outcome of the merge. Click Done.