fluree.doc¶
Read documents with an extraction model reached over HTTP.
read() parses a file (PDF, Word, PowerPoint, HTML, Markdown, text,
spreadsheets, transcripts, email) the way fluree doc parse does, sends
its text and structure with your ontology and catalog to a reader, and
returns its reading: a Fluree DB transaction holding the run (the document
read, the release that read it), the facts in your ontology’s terms, the
claims behind them and the mentions, each anchored to character offsets in
the parser’s text (what fluree doc parse --format text prints):
import fluree
reading = fluree.doc.read("invoice.pdf", ontology="invoices.ttl",
catalog="suppliers.ttl", model="reader-small")
for m in reading.mentions():
print(m["text"], m["entity"])
with fluree.connect("./data") as conn:
conn.create("invoices").insert(reading.to_jsonld())
include= says what comes with the facts, which always come: "claims",
"mentions", both (the default) or "none".
The read is the fluree CLI’s own, so a document is cut into calls, sent,
resent when the reader has no capacity yet or rate-limits it, and kept the
same way. Waits (a model still starting can take minutes) are logged as warnings
on the fluree.engine logger. Python does no vision escalation: scanned
pages and images are read with the parser’s deterministic tier only, and
Reading.notes says what it couldn’t read.
Signing in.
api_keywithauth_header: that key, sent as is in that header, tourl.api_keyalone: a Fluree.ai API key (flk_…) is traded for an access token (reused for the hour it lasts), or, when the service won’t trade it (a developer key, which calls the reader itself), sent as it is; any other key is sent asAuthorization: Bearer, as the CLI’s[doc.reader] api_keyis.No
api_key: this machine’sfluree login, else theFLUREE_AI_API_KEYenvironment variable (as in 2). Each is sent only to its own service; aurlelsewhere is called with no credential.
With no url, a Fluree.ai sign-in reads through the account.
Rows and terms. The reading is kept as the reader wrote it, in the
extraction vocabulary (xt:, https://ns.flur.ee/extract#):
Reading.transactions holds one transaction per call (a document
longer than one call’s limit takes several, each its own run, its offsets
moved into the whole document’s text). Reading.mentions() and
Reading.claims() are rows read out of it.
Module Contents¶
- class fluree.doc.Usage¶
What a read was billed, as the reader reported it.
- class fluree.doc.Reading¶
One document’s reading, as the reader wrote it.
- transactions: list[dict[str, Any]]¶
One Fluree DB transaction (JSON-LD) per call: the run in its
txn-meta, the facts and the claims annotating them in its@graph, the mentions in a named graph of their own.
- runs: list[dict[str, Any]] = []¶
One per call:
run(the run’s IRI),release,request_id(the endpoint’s ID for the call) andcached.
- notes: list[str] = []¶
What the parse left out: pages it couldn’t read (no vision escalation from Python), an email’s attachments, and the like.
- to_jsonld() dict[str, Any]¶
The reading as a transaction, ready for
Ledger.insert: the run becomes the commit’s metadata. A document read in several calls has a transaction per call; insert each oftransactions.
- mentions() list[dict[str, Any]]¶
One row per mention (
xt:Occurrence), in text order:id,text(nif:anchorOf),beginandend(offsets in the parser’s text),class(xt:occurrenceClass),entity(the catalog record it links to,xt:entity, or the entity the run minted,xt:localEntity, and thennew_entity),confidence(xt:linkConfidencefor a link, elsext:score) andelement(the structure element it sits in).
- fluree.doc.read(path: str | os.PathLike[str] | None = None, *, text: str | None = None, ontology: str | os.PathLike[str], catalog: str | os.PathLike[str] | Sequence[str | os.PathLike[str] | tuple[str | os.PathLike[str], str]] | None = None, model: str, cache: str | None = None, include: str | Sequence[str] = _INCLUDE, url: str | None = None, api_key: str | None = None, auth_header: str | None = None, doc_iri: str | None = None, base_iri: str = 'urn:fluree:doc:', cache_dir: str | os.PathLike[str] | None = None) Reading¶
Read one document with
modeland return itsReading.Give a file as
path, or plain text astext. A text is read underdoc_iri, by default<base_iri>text/<first 16 hex of its SHA-256>, so two different texts never share a document’s nodes.ontologyis a.ttl,.nt,.jsonldor.jsonfile.catalogis one such file, or a list of them, each optionally a(file, class IRI)pair to take only that class’s records; a bare 2-tuple is read as such a pair unless its second item is itself an RDF file.cache("ontology","catalog"or"off") lets the reader keep that much for the next request sharing it.includesays what comes with the facts:"claims","mentions", both (the default, as("claims", "mentions")or"claims,mentions"), or"none". Withcache_dir, parses and answers are kept there, so reading the same request again costs nothing.Raises
fluree.errors.InvalidRequestErrorfor arguments that can’t make a read (a badcache,auth_headerwithoutapi_key, no way to sign in),fluree.errors.PermissionDeniedErrorwhen a credential is refused, andfluree.errors.FlureeErrorfor a reader’s other refusals, with its message.