Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Vocabulary

Namespace doc: is https://ns.flur.ee/doc#. Structure elements additionally use DoCO (doco:, http://purl.org/spar/doco/), the pattern ontology it extends (po:, http://www.essepuntato.it/2008/12/pattern#), NIF (nif:, http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core#), Dublin Core (dcterms:, http://purl.org/dc/terms/) and FOAF (foaf:, http://xmlns.com/foaf/0.1/) for the document root’s class. The structure terms are those of the parsing engine and are documented in full in its vocabulary reference; the terms below are what the ingest adds.

Classes

ClassMeaning
doc:ChunkA retrieval unit cut from a document’s structure.
doc:SourceDocumentThe file a graph was extracted from; the doc:sourceDocument target.
doc:MentionA span of a chunk naming an entity. Also typed nif:RFC5147String.
doc:RelationOne relation the language model reported, reified with its evidence and verdict.
doc:EntityAn entity extraction minted because no --entities source knew it. Known entities keep their own IRI and get no node here.

Chunk properties

PropertyRangeMeaning
doc:chunkIndexintegerPosition in the document.
doc:textstringThe chunk’s text, whitespace collapsed.
doc:headerPathstringSection titles above it, outermost first, joined with /. Absent when there are none.
doc:sourceElementIRI, repeatedThe elements it was built from, in order.
doc:sourceDocumentIRIThe document. Also on every structure node but a table cell, which is reached through its table.
doc:embedding@vectorIts embedding, when produced.

Document properties

PropertyRangeMeaning
doc:fileNamestringThe file’s name.
doc:relativePathstringPath relative to the ingested directory, /-separated.
doc:sha256stringHex digest of the bytes.
doc:mediaTypestringThe media type inferred from the extension.
doc:byteSizeintegerSize in bytes.
doc:pageCountintegerPages, for sources that have them; 0 otherwise.
doc:escalatedCropsintegerCrops a vision model read for this extraction.
doc:parserRevisionstringThe fluree-doc-parse revision that produced the structure graph.
doc:chunkCountintegerChunks written.
doc:chunkingstringmin/max characters the chunks were cut with.
doc:embeddingModelstringThe embedding model, when one ran.
doc:embeddingDimensionsintegerIts vector width.
doc:ingestedAtxsd:dateTimeWhen this extraction was written.
doc:extractionModelstringThe language model that extracted, when one ran.
doc:extractionFingerprintstringHash of the ontology, entity sources, model, guidance and relation mode.
doc:mentionCountintegerMentions written.
doc:entityCountintegerDistinct entities mentioned, known and new.
doc:relationCountintegerRelations written, every verdict included.

Mention properties

PropertyRangeMeaning
nif:beginIndex, nif:endIndexintegerCharacter offsets into the chunk’s doc:text.
nif:anchorOfstringThe text as written.
nif:referenceContextIRIThe chunk.
nif:entityIRIThe entity, under the IRI its source gave it.
doc:sourceElementIRIThe structure element the span sits in.
doc:extractedBystringgazetteer or llm.
doc:sourceDocumentIRIThe document.

Relation properties

PropertyRangeMeaning
rdf:subject, rdf:predicateIRIThe statement.
rdf:objectIRI or literalThe statement’s object: an entity, or the literal the model gave when the object was a value or named no entity in the chunk.
rdfs:labelstring`subject
doc:excerptstringThe text supporting it, as the model quoted it.
doc:verdictstringvalid, repaired or rejected.
doc:assertionModestringasserted, hedged, attributed or negated: how the source states it. Present only when a custom --system-prompt asked for it. Not a confidence score and not ordered — see Assertion mode.
doc:assertedbooleanWhether a direct edge was written for it.
doc:originalPredicate, doc:repairNotestringOn a repaired relation: what the model wrote, and how it was resolved.
doc:rejectionReasonstringOn a rejected relation.
doc:sourceChunk, doc:sourceDocumentIRIWhere it was found.

Minted entity properties

PropertyRangeMeaning
schema:namestringThe canonical name.
skos:altLabelstring, repeatedOther surface forms seen.
doc:nerLabelstringCoarse label: PERSON, ORG, GPE, LOC, FAC, EVENT, PRODUCT, WORK_OF_ART, LANGUAGE, MISC, CONCEPT.
doc:offModelbooleanPresent and true when the model’s class for the entity is not in the ontology; the node carries no class.
ontology datatype propertiesliteralAttributes the model stated and the ontology admits.

Structure terms used by the chunker

The chunker starts at the foaf:Document root and follows po:contains, visiting a node’s po:containsAsHeader child first. It takes text from doco:Title and doco:Paragraph (nif:isString, else rdfs:label), turns each doco:Section’s doco:SectionTitle and doc:sectionLevel into the header path, and reads each doc:TableCell as doc:cellValue with its doc:rowHeader and doc:columnHeader. Every other node is a container it walks through. See What gets built for the rest of the structure graph.