Core concepts & pipeline · docuflow-ai Docs

docuflow-ai is built around a small set of objects and one automated pipeline. Knowing them makes every other page of this documentation easier to follow.

The building blocks#

Object	Description	Where to manage it
Organization	Your tenant. It owns users, documents, settings, credits, an API key/secret pair, a capture email address and one search index per document type. Data is never shared between organizations.	Admin → Organization Management
Document	A stored file with name, extension, page count, version number, tags, status, type, extracted text and extracted JSON.	Documents
Content type	An accepted file extension and the OCR engine that reads it. Only active content types can be uploaded.	Admin → System administration → Content Types
Document type	A business category (invoice, bill of lading, patient intake form …) linked to industries. It carries the JSON schema used for extraction and its own OpenSearch index.	Admin → Organization Management → Document settings
Tag	A free label you attach to documents to group and filter them.	Admin → Organization Management → Tag Editor
AI job	A queued request to the AI service with a function (split, map JSON, search, compare …), a priority, a status and the response.	AI Jobs
System task	A scheduled background job, such as Convert documents to text, that moves documents forward.	Admin → Platform management

The processing pipeline#

Uploading only stores the file. Everything else happens asynchronously in background tasks, each of which picks up a batch of documents in a given status, does its work and moves them to the next status.

1. SplitBundled PDFs are cut into business documents; the type is detected.Waiting for pre processing

2. VersioningSame name + source key → next version number.Waiting for versioning

3. OCRText is extracted with the content type's engine.Waiting for OCR start

4. IndexText goes into the organization's full-text/semantic index.Indexed

5. Map JSONAI fits the content to the type's schema and indexes the fields.Searchable

1. Split and type detection#

PDF files are sent to the AI service, which recognizes where one business document ends and the next begins and returns the pieces with their extracted content. Each piece becomes its own document whose Parent is the original file. When AutoDetectDocType is on, the AI also chooses the most suitable of your active document types for each piece. The KeepOriginalAfterSplit setting decides whether the bundled original is kept. Other formats skip this step and start at versioning (or straight at OCR when versioning is off).

2. Versioning#

With UseVersioning on, a new file with the same name and extension (and the same source key, when it comes through the API) as an existing document becomes a new version of it instead of a separate document. With UseAutoCompare on, AI then compares the new version with the previous one and emails a summary of the changes.

3. Text extraction (OCR)#

Documents are sent in batches to the OCR service. Scanned images and PDFs are read with an OCR engine; Office, e-mail and markup formats are parsed directly. See Supported file types.

4. Indexing#

The extracted text is stored in OpenSearch with both a keyword (BM25) mapping and a vector embedding, so it can be found by exact words and by meaning.

5. JSON mapping#

For documents whose type has a JSON schema and a created index, AI maps the extracted content onto the schema. The result is written to the document type's own index, where every field can be queried, and the document becomes Searchable. See Document types & JSON schemas.

Document statuses#

Id	Status	Meaning
3	Waiting for pre processing	Uploaded; a PDF waiting for (or being processed by) the AI split step.
4	Waiting for original delete	A bundled original that has been split and will be removed because `KeepOriginalAfterSplit` is off.
5	Waiting for versioning	Waiting for version detection.
9	Permanently failed	OCR or split failed the maximum number of retries (3 by default) and will not be retried automatically.
15	Waiting for OCR start	Queued for text extraction.
25	OCR completed	Text is available; waiting to be indexed.
30	Indexed	Full-text and semantic search work. Field-level search needs the JSON mapping step.
40	Searchable	Fully processed: extracted fields are in the document type's index.

The Dashboard shows live counters for the statuses that matter most, together with totals by type, extension and user, your credit balance and the disk space you use.

Retries and failures#

Every AI step (split, JSON mapping, auto compare) and OCR is retried automatically. A document that fails AIJobMaxRetryCount times is marked Permanently failed. Failed AI requests are listed under Admin → Platform logs → AI job failures, and the individual OCR and AI service calls under OCR service logs and AI service logs.

Security model#

Isolation: every query is scoped to your organization, and each organization has its own search indexes and storage folder.
Authentication: users sign in with a Microsoft account or email and password; systems use an API key plus secret.
Authorization: each page is a menu item granted to access roles, so users can only open what their role allows.
Retention: StoreDocuments = N deletes files once processing finishes. Deleted organizations are purged, including files and indexes, after the configured number of days (30 by default).