Keep extracted mentions separate from catalogue matches
CalOn offers compact, schema-guided extraction with direct links to source text. Use that capability to build a mention layer, then resolve catalogue identities separately so a correct extraction does not become an unsupported business record.
Keep development and test groups separate before comparing performance. No measured results are shown.
A service note can name a piece of equipment without identifying which asset the organisation owns. That distinction becomes more important when an extraction model can turn documents into structured records at scale. The useful opportunity is to make the source easier to inspect while keeping uncertain catalogue matches visible.
CalOn, described in a preprint dated 12 October, is a compact encoder for extracting entities, relations and events under user-supplied type definitions. It predicts spans within the input rather than generating replacement wording. Its design separates whether a candidate should be extracted from which requested type it receives. (Method, sections 3.1–3.4)
For a document-processing owner, that supports a candidate design with a useful intermediate product: a typed mention linked to the text. Build that layer before deciding which catalogue entry, supplier or asset the mention represents. A correct label and a correct business identity require different evidence.
Make the source mention a usable record
The released interface returns mention text, character positions, a type and a probability. Relations retain their endpoint mentions, while events have triggers and arguments. Type descriptions can accompany names. This makes the release a practical candidate for a bounded extraction comparison, although the examples in its documentation are not independent validation.
Consider a hypothetical maintenance note: “Falcon was inspected at the west depot.” The team wants to find references to equipment in approved service notes. Its schema, the set of requested types and their definitions, describes equipment as a named physical item being discussed in the maintenance activity. A candidate extraction identifies “Falcon”. That could help a reviewer find the relevant passage without reading every note.
The proposed mention record should retain the source identifier and version, exact extraction input, character positions, type definition and model configuration. Verify that those positions reproduce the returned text. If a parser normalises whitespace or removes a page header, positions in the transformed text may no longer point into the original document. Preserve a mapping where reviewers need the original page, rather than silently treating the two texts as interchangeable.
This is an interface check, not a test of semantic accuracy. A position can correctly point to “Falcon” while the assigned type is wrong. Review the words in context as well as checking that the reference resolves.
Resolve identity as a separate decision
Suppose the fictional asset catalogue contains two pumps named Falcon. The service note contains no serial number. Extracting the name does not supply the missing discriminator, even if the extractor assigns it a high score. Attaching the inspection to the first catalogue result would introduce a new claim that the note did not establish.
Keep the matching result alongside the mention. It should identify the catalogue version, candidate records, matching evidence and unresolved ambiguity. A reviewer can then obtain a serial number or another approved source before choosing the asset. If no distinguishing evidence is available, retain an unresolved mention without updating either asset’s maintenance history.
Follow a mention into a proposed catalogue match
Prepare a reviewable asset reference from a fictional service note.
Swipe or scroll for the full diagram →
Review obtains evidence
An authorised reviewer obtains the serial number from the source owner and records the chosen match separately from the original extraction.
- Source preserved
- The approved note reads: Falcon was inspected at the west depot. Retain the note version and the exact text supplied to extraction.
- Mention proposed
- The hypothetical extractor returns Falcon as equipment, with its character positions and the schema used. No asset identifier has been established.
- Catalogue searched
- A separate matching step finds two pump records named Falcon. Preserve both candidates and the catalogue version.
- Identity left unresolved
- The note has no serial number. Do not convert either candidate into a confirmed asset link or attach the inspection to its maintenance history.
- Review obtains evidence
- An authorised reviewer obtains the serial number from the source owner and records the chosen match separately from the original extraction.
Constructed operational records, not CalOn output, hidden reasoning or a measured service result. Replay follows the boundary between a source mention and a business identity.
A fictional service note names Falcon as a pump. Extraction retains the exact source mention. A separate catalogue lookup finds two possible pump records, so the asset identity remains unresolved until a reviewer obtains distinguishing evidence.
Reviewed 2026-10-18
This proposed separation also makes correction more precise. If the extractor missed the name, repair extraction. If the name was found but the wrong asset was selected, repair matching. If the note itself is inaccurate, the source owner must address that record. Re-running the model should not silently overwrite a reviewed catalogue decision, and correcting a match should not erase what the source originally said.
The trace is a constructed workflow, not a CalOn demonstration. Its value is a testable boundary: can the system retain a useful source reference when it cannot yet establish the business identity?
Define the types before comparing extractors
Ask the data owner to provide short definitions and difficult examples. Does “equipment” include a product family, a proposed replacement and a decommissioned asset? Should a negated inspection appear as an event, and with what status? These choices belong in the extraction brief and review labels. Leaving them implicit makes disagreement between reviewers hard to distinguish from model error.
CalOn’s authors report that descriptions help disambiguate relation types. They also state that every evaluated entity type appeared by name in training. Their evidence therefore does not establish unrestricted transfer to entirely new type vocabularies. (Schema analysis, section 5.2)
Treat a changed definition as a changed extraction task. Compare the affected records under the old and proposed definitions, then decide whether earlier records need reprocessing. Keep both outputs distinguishable during review. This differs from the archive’s corpus-table adaptation article: that changes information supplied to a generator. Here the owner is defining what a document-processing service is being asked to record.
Test the complete document route
Begin with approved English service notes and independently reviewed mentions, including ambiguous names and documents with no relevant equipment. Compare the specialist extractor with the current method on the same text and definitions. Measure missed mentions, incorrect spans and types, unresolved matches and wrongly confirmed identities separately. Include reviewer effort and elapsed time through parsing, extraction, matching and correction.
The implementation’s stated limits include inputs of at most 1,024 tokens, spans of at most 32 tokens and relations within one input. Splitting a long note can therefore separate a name from the information needed to interpret it. Test the actual splitting policy and preserve document-level references. Do not infer support for cross-document resolution from within-input relation extraction.
The paper reports English-only evaluation and a single training seed. Calibration uses validation data from training sources and may not hold after domain change. (Limitations) No model experiment was run for this article. The earlier confidence-threshold analysis explains why a published score scale cannot set a local review policy by itself.
The data owner should approve the mention definitions and identity evidence required for the intended record. The service owner should fund a bounded trial only where source-linked extraction reduces useful review work without concealing unresolved matches. Where the source lacks the discriminator, preserve the gap and obtain better evidence before updating the business register.