fluree.doc

Read documents with an extraction model reached over HTTP.

read() parses a file (PDF, Word, PowerPoint, HTML, Markdown, text, spreadsheets, transcripts, email) the way fluree doc parse does, sends its text and structure with your ontology and catalog to a reader, and returns its reading: a Fluree DB transaction holding the run (the document read, the release that read it), the facts in your ontology’s terms, the claims behind them and the mentions, each anchored to character offsets in the parser’s text (what fluree doc parse --format text prints):

import fluree

reading = fluree.doc.read("invoice.pdf", ontology="invoices.ttl",
                          catalog="suppliers.ttl", model="reader-small")
for m in reading.mentions():
    print(m["text"], m["entity"])

with fluree.connect("./data") as conn:
    conn.create("invoices").insert(reading.to_jsonld())

include= says what comes with the facts, which always come: "claims", "mentions", both (the default) or "none".

The read is the fluree CLI’s own, so a document is cut into calls, sent, resent when the reader has no capacity yet or rate-limits it, and kept the same way. Waits (a model still starting can take minutes) are logged as warnings on the fluree.engine logger. Python does no vision escalation: scanned pages and images are read with the parser’s deterministic tier only, and Reading.notes says what it couldn’t read.

Signing in.

  1. api_key with auth_header: that key, sent as is in that header, to url.

  2. api_key alone: a Fluree.ai API key (flk_…) is traded for an access token (reused for the hour it lasts), or, when the service won’t trade it (a developer key, which calls the reader itself), sent as it is; any other key is sent as Authorization: Bearer, as the CLI’s [doc.reader] api_key is.

  3. No api_key: this machine’s fluree login, else the FLUREE_AI_API_KEY environment variable (as in 2). Each is sent only to its own service; a url elsewhere is called with no credential.

With no url, a Fluree.ai sign-in reads through the account.

Rows and terms. The reading is kept as the reader wrote it, in the extraction vocabulary (xt:, https://ns.flur.ee/extract#): Reading.transactions holds one transaction per call (a document longer than one call’s limit takes several, each its own run, its offsets moved into the whole document’s text). Reading.mentions() and Reading.claims() are rows read out of it.

Module Contents

class fluree.doc.Usage

What a read was billed, as the reader reported it.

input_tokens: int
windows: int
cache_read_tokens: int
cache_write_tokens: int
fuel: float | None
note: str

The same, as one line: 1,032 tokens, cache 900 read / 0 written.

class fluree.doc.Reading

One document’s reading, as the reader wrote it.

transactions: list[dict[str, Any]]

One Fluree DB transaction (JSON-LD) per call: the run in its txn-meta, the facts and the claims annotating them in its @graph, the mentions in a named graph of their own.

fact_count: int

Statements in the facts, the claims aside.

new_entities: list[str]

Entities the runs minted for things the catalog doesn’t hold.

linked: list[str]

Catalog records the mentions link to.

usage: Usage
calls: int

Calls the document took; cached of them were answered from the cache.

cached: int
release: str | None
runs: list[dict[str, Any]] = []

One per call: run (the run’s IRI), release, request_id (the endpoint’s ID for the call) and cached.

ontology_notes: list[str] = []

What the reader set aside or adapted in the ontology.

notes: list[str] = []

What the parse left out: pages it couldn’t read (no vision escalation from Python), an email’s attachments, and the like.

property mention_count: int
property claim_count: int
to_jsonld() → dict[str, Any]

The reading as a transaction, ready for Ledger.insert: the run becomes the commit’s metadata. A document read in several calls has a transaction per call; insert each of transactions.

to_trig() → str

The reading as TriG: the run in <#txn-meta>, each claim behind a fact reifying it.

to_turtle() → str

The reading’s statements as Turtle, without the run.

mentions() → list[dict[str, Any]]

One row per mention (xt:Occurrence), in text order: id, text (nif:anchorOf), begin and end (offsets in the parser’s text), class (xt:occurrenceClass), entity (the catalog record it links to, xt:entity, or the entity the run minted, xt:localEntity, and then new_entity), confidence (xt:linkConfidence for a link, else xt:score) and element (the structure element it sits in).

claims() → list[dict[str, Any]]

One row per claim (xt:Claim): id, subject, predicate, object (an IRI, or a literal’s text), status (xt:assertionStatus), accepted, score, and asserted (the claim annotates its fact, which is in the facts).

fluree.doc.read(path: str | os.PathLike[str] | None = None, *, text: str | None = None, ontology: str | os.PathLike[str], catalog: str | os.PathLike[str] | Sequence[str | os.PathLike[str] | tuple[str | os.PathLike[str], str]] | None = None, model: str, cache: str | None = None, include: str | Sequence[str] = _INCLUDE, url: str | None = None, api_key: str | None = None, auth_header: str | None = None, doc_iri: str | None = None, base_iri: str = 'urn:fluree:doc:', cache_dir: str | os.PathLike[str] | None = None) → Reading

Read one document with model and return its Reading.

Give a file as path, or plain text as text. A text is read under doc_iri, by default <base_iri>text/<first 16 hex of its SHA-256>, so two different texts never share a document’s nodes. ontology is a .ttl, .nt, .jsonld or .json file. catalog is one such file, or a list of them, each optionally a (file, class IRI) pair to take only that class’s records; a bare 2-tuple is read as such a pair unless its second item is itself an RDF file. cache ("ontology", "catalog" or "off") lets the reader keep that much for the next request sharing it. include says what comes with the facts: "claims", "mentions", both (the default, as ("claims", "mentions") or "claims,mentions"), or "none". With cache_dir, parses and answers are kept there, so reading the same request again costs nothing.

Raises fluree.errors.InvalidRequestError for arguments that can’t make a read (a bad cache, auth_header without api_key, no way to sign in), fluree.errors.PermissionDeniedError when a credential is refused, and fluree.errors.FlureeError for a reader’s other refusals, with its message.