Documentation / Extraction

Document types & JSON schemas

Define what to extract from each kind of document.

A document type tells docuflow-ai what a document is and which fields to pull out of it. Its JSON template (schema) defines those fields, and its search index makes them queryable. Getting these right is the most important step for good extraction and search.

Document types#

Manage types under Admin → Organization Management → Document settings → Document Types. Each type has a name, industries it belongs to and an active flag. docuflow-ai ships with ready-made types for finance and health care, including:

IndustryExamples
FinanceGeneral Invoice, General Purchase Order, General Shipping Order, Bill of Landing, Work Order, Acord 25 / Acord 130 statements, IRS statement, bank statements (General, BoA, Chase, Capital One, HSBC, Wells Fargo), AMEX statement
Health carePatient Intake Form, Out Patient Form, Consent Form, Discharge Summary, Lab Results

Platform administrators maintain the master catalogue (Reference management → Master Document Types, Master Industry - Document Types) that new organizations start from.

How a document gets its type#

  • Automatically: with AutoDetectDocType = Y (default), the AI split step picks the best match among your active types for every PDF it processes.
  • Manually: right-click documents on the Documents page and choose Document type. Use this for non-PDF files or to correct a detection.
Detection works best with distinct types
Keep only the types you actually receive active, and give them clear, specific names. Ten well-separated types are detected more reliably than fifty overlapping ones.

Building a JSON template#

Click the JSON templates button on a document type row to open the template editor.

  1. Pick sample documents. The left panel lists documents of this type. Tick a few representative ones (the eye icon shows each one's extracted JSON). Their structures are merged into a draft template on the right. Your selection is remembered.
  2. Simplify with AI. Click Simplify. AI turns the merged draft into a clean schema: consistent field names, sensible types and no one-off fields. The result appears when the background job finishes.
  3. Review and edit. Adjust field names and types in the editor. Expand or collapse sections to navigate large schemas.
  4. Save. Click Save. If the type has no index yet, you are offered to create it; if it has one, you are shown what changed and offered to apply it.

Schema design tips#

  • Name fields for what they mean, e.g. invoiceNumber, issuerName, totalAmount, not field1.
  • Use the right type. Numbers and dates become numeric and date fields in the index, which enables range queries such as amount over 1,000.
  • Model repeating data as arrays (for example lineItems with description, quantity, unitPrice). Arrays are indexed as nested objects so conditions match within the same line.
  • Keep it focused. Every field costs extraction effort; leave out what nobody will search or report on.
  • Iterate. After changing a template, use Recreate JSON on a few documents to check the result before re-indexing everything.

Search indexes#

Each document type with a template gets its own OpenSearch index. Documents only become Searchable once the index exists. Open Index operations in the template editor:

OperationUse it whenUses credits
Create indexThe type has a saved template but no index yet.No
Analyze indexYou want to see health, size, shards, how many documents are searchable, waiting or still in the pipeline, and how the template differs from the index.No
Apply template changesYou saved a changed template. You are shown the previous and new template and asked to apply the change: new fields are added to the existing index, and existing documents get values for them after a re-index. (The Modify index menu entry itself is not available yet.)No
Re-create indexYou changed a field's type or want removed fields gone. The index is deleted, rebuilt from the current template and every document of the type is processed again.Yes
Re-index documentsYou want existing documents re-extracted into the same index, e.g. after adding fields.Yes
Schema changes and existing indexes
OpenSearch cannot change the type of an existing field or remove a field from an index. Added fields are applied in place; removed fields stay (new documents simply don't fill them); type changes need Re-create index. Re-create and re-index show how many documents will be processed before they start, because each one consumes credits.

How extraction works#

When a PDF is split, AI returns each piece's content as structured JSON. The Map JSON task then asks AI to fit that content onto the type's template, writes the result to the type's index and marks the document Searchable. The mapped JSON is what you see on the Json tab of the viewer and what field-level search queries.

An unhandled error has occurred. Reload 🗙