Duplicate Check

Sometimes you might want Link to check whether an incoming document already exists and in that case make it fail to avoid duplicates. This is done using duplicate check.

Here are some statements that can help you understanding the duplicate check functionality: 

• Duplicate check values are stored in relation to the distribution where you set up duplicate checks. This also means that if you delete the distribution and create it again, the documents that were processed by the first distribution will not be checked against the documents that were processed by the second distribution.

• If you Edit a distribution and disable a duplicate check type in a period of time and then enable it again. The documents that are processed in that period of time will NOT be subject to duplicate check for the duplicate check type you disabled.

• If you Edit a distribution and change something that is not related to the duplicate check functionality like the "Format/Variant/Version" on either the "From Format" or "To Format". This will not affect the duplicate check functionality.

• When a document has been deleted by the Maintenance job, that documents duplicate check values are deleted with it. This means that an identical document will not be seen as a duplicate. Note: There is a "hidden" feature where its possible to delete duplicate check values earlier (not later, unfortunately) than when the document is deleted (the hidden configuration key description is "Days to keep duplicate check records"). It could make sense to delete documents but keep duplicate check values but this will require a feature request.

There are several different types of duplicate checks available, and we will describe them in detail on this page. Depending on your permissions, some types may not be available.

Note that four of the duplicate check types are simply comparing document field values (interchange number, transmission id, document number and filename) - these are described in the Document detail page article.

It is possible to enable multiple duplicate check types at the same time.

MD5HashValue

Hash value check uses the hash value of the first 30.000 bytes of a file when checking for duplicates. Be aware that the duplicate check is performed on the Message (not the interchange) in the file. This means that newlines, tabs and other non-visible characters could have been removed before the duplicate check.

Interchange number

Checking this option will determine whether a document with the same Interchange Number already exists in the database. 

Document number

Checking this option will determine whether a document with the same Document Number already exists in the database. 

Transmission id

Checking this option will determine whether a document with the same Transmission id already exists in the database. 

Filename

Checking this option will determine whether a document with the same filename already exists in the database.