The first question
Why models are needed for unbounded layouts
Parsers excel when the input shape is known. Loan files are different: payroll providers, banks, phone photos, screenshots, changed layouts, and inconsistent labels create an effectively unbounded input space. Template coverage therefore requires continuing code changes; models can recognize structure in layouts not anticipated in advance.
Coverage behavior
Template coverage rises for anticipated layouts then requires a code change for each new layout; model-based recognition can generalize to unfamiliar layouts but still requires validation.
OCR reads characters. Semantic extraction assigns meaning.
OCR may return 4,231.18 among dozens of numbers. It does not determine whether that value is current gross pay, year-to-date earnings, a deduction, or an unrelated identifier. Document layout and domain context supply that meaning.
OCR versus semantic extraction
4,231.18 · 1,102.44
38,079.62 · 401
12.50 · 2,847.31gross_pay_current → 4,231.18
gross_pay_ytd → 38,079.62
pay_period_end → source regionOCR produces an unstructured list of characters and numbers; semantic extraction connects selected source regions to named fields, each with provenance and a confidence signal that still requires calibration.
Division of labor
Unbounded documents enter a model perception band; only verified structured fields cross into deterministic calculations; results are presented to licensed people who approve or decide.
Where models are deliberately not used
When an authoritative structured source is available, the reference design prefers it. Bureau files, account aggregation data, transcript data, verification-service responses, and native system records do not need visual interpretation. Models are reserved for artifacts without a reliable machine-readable source.
Source preference ladder
- Structured API · no model
- Machine-readable file · no model
- PDF text layer · validate structure
- Scanned image · model recognition
- Photograph · model recognition + stronger review
The system prefers structured APIs, then machine-readable files, text-layer PDFs, scans, and photographs; model involvement increases only as structure decreases.
Model deployment
Model choice is a validated implementation decision
Open-weight, hosted, and private-service options can be assessed against privacy, document accuracy, provenance, procurement, latency, ecosystem, and operational capacity. No single family is assumed. Pinning and hashing artifacts, capturing runtime and library versions, controlling inference settings, and regression-testing a representative set improve traceability and repeatability; pinned weights alone do not promise bit-for-bit reproducibility.
Controlled model supply chain
Candidate model artifacts pass review, hashing, provenance capture, regression tests, and promotion into a private registry before a controlled runtime serves them; production need not pull weights live.
Infrastructure options
Hardware follows the client’s threat model and workload
Encryption at rest and in transit are standard controls. Confidential-computing designs can also reduce exposure while data is in use, but hardware protection remains one layer among access control, logging, credential hygiene, patching, and egress restriction.
Three states of data
Data is encrypted at rest and in transit; confidential-computing options can protect selected data while in use, subject to platform capability and validation.
H100, H200, MI300X, confidential-capable CPU platforms, and related infrastructure are options—not claims about equipment Zinits owns or operates and not mandates. Selection depends on privacy, capacity, budget, throughput, software ecosystem, and procurement requirements.
Attested boot sequence
A confidential-capable environment boots, requests an attestation report, verifies it against the expected trust chain and policy, and either loads approved artifacts or halts before intake opens.
Protected processing boundary
A protected boundary may include confidential-capable CPU memory, transfer path, and accelerator memory; administrative layers remain outside and still require access controls, logging, and egress restrictions.
Runtime
Software layers keep responsibilities narrow
Durable workflow state matters because a loan file can wait on people or third parties for days. Retries must be idempotent, and a restart must not silently duplicate a consequential action.
Controls
Every extracted value earns its way into the file
Thresholds are calibrated and validated by field class against representative client documents. Raw model confidence is not inherently calibrated. Two calls to the same model are correlated; meaningful verification may combine independent prompts, preprocessing, models, deterministic checks, or human review according to field risk.
Gate cascade
Extracted values pass schema, calibrated confidence, independent verification, cross-document consistency, plausibility, configurable business rules, required human verification, and obligation gates. A failure enters review, then must be corrected and revalidated, rejected, or escalated; it never flows directly into verified data.
DTI thresholds, document age, conditions, notices, and similar lending logic are illustrative and configurable by product, investor, lender policy, and jurisdiction. No example threshold is universal.
Field-level traceability
A figure should lead back to its exact source
A field record can retain document ID, page, source coordinates, model and artifact hash, runtime/version capture, controlled inference settings, gate results, corrections, and reviewer attribution.
Click-through to source
A file summary highlights gross monthly income; selecting it opens the source pay statement at the exact outlined region with extraction history, calibrated confidence context, and reviewer status.
Trust boundaries
Untrusted documents do not receive capabilities
The component that interprets borrower uploads should not send email, update a system of record, reach the public internet, or satisfy a condition. Its permitted output is structured data that still faces validation. Action-capable services belong in a separate zone and operate on gate-passed data.
Zone architecture
An untrusted intake zone validates artifacts before a no-egress processing zone performs extraction and gates; only gate-passed structured data crosses into the trusted record zone for deterministic calculations, audit records, and controlled connectors.
The complete path
One document, start to finish
End-to-end master schematic
Borrower uploads move through intake validation, isolated model extraction, the gate cascade, correction and revalidation when needed, deterministic calculations and obligation checks, team approval of communications, final licensed review, and an attributed decision record.
Readable sequence: borrower upload → artifact validation → isolated extraction → validation gates → correction and revalidation, rejection, or escalation → gate-passed fields → versioned calculations and obligation checks → human-approved requests → licensed review and attributed decision.
Client implementation
Audit, implement, validate, then operate
Implementation phases
- DiscoverDocument inventory, threat model, workflow and baseline
- PilotRepresentative real files; calibrate by field class
- Implement + validateControls, integrations, failure cases, regression set
- OperateApproved go-live, monitoring, reviewed upgrades
A discovery and baseline phase leads to a real-file pilot without required integration, then controlled implementation and validation, followed by an explicitly approved go-live and monitored operation.
What this does not do
Honest limits are part of the architecture
- It does not make credit decisions. Approval authority stays with licensed staff.
- It does not achieve perfect extraction. Uncertainty must stop and receive proportionate handling.
- It does not make a brokerage compliant. Controls support obligations; responsibility remains with the brokerage.
- Hardware protection is not a guarantee. It complements access control, egress restriction, credential hygiene, logging, and operations.
- It requires clean handoffs. Integration choices must avoid creating a competing system of record.
Discuss the choices that fit your workflow
Bring a document sample, workflow map, or vendor questionnaire. We can use this reference architecture to identify the decisions that need client-specific validation.
Discuss implementation choices