Document AI: when general-purpose model APIs are not enough

Today, creating a prototype to extract data from PDFs, scans or images using artificial intelligence is relatively simple.

With APIs from general-purpose models such as OpenAI, Anthropic, Google or other providers, it is possible, for example, to send an invoice, specify which information to retrieve and quickly obtain structured output:

• invoice number;
• supplier company name;
• date;
• taxable amount;
• VAT;
• total;
• IBAN;
• currency.

With a limited number of documents, and especially when the files are fairly homogeneous, the results can be very convincing.

The demo works.
The JSON output arrives correctly.
The data seems accurate.

At that point, it is natural to think that the document extraction automation project is almost complete.

In reality, that is precisely when the most complex part begins.

Because there is a major difference between having an AI model read a document and building a reliable, controllable Document AI process that can be used every day in production.

From a data extraction prototype to production

During a Proof of Concept, the workflow can be extremely simple.

The team selects a few documents, prepares a prompt, defines the output schema and manually checks the results.

This approach makes it possible to experiment quickly and represents one of the major advantages of general-purpose AI models.

The problem emerges when the system has to start processing real documents every day.

No longer ten manually selected invoices, but thousands or hundreds of thousands of files coming from customers, suppliers, intermediaries or different systems.

At that point, much harder-to-control variables come into play:

• native PDFs and scanned documents;
• photos taken with smartphones;
• tilted or blurred images;
• different layouts;
• multipage documents;
• tables continuing across multiple pages;
• missing fields;
• poorly legible values;
• contradictory information;
• new versions of the same document;
• new document types never seen before.

An invoice from one supplier may be completely different from that of another.

A payslip may change structure depending on the software used.

An electricity or gas bill may show POD, PDR, consumption and contract data in different positions.

An identity document photographed with a smartphone may be partially covered, tilted or poorly lit.

The system must continue to work even in these cases.

And this is where extraction through a simple API call is no longer sufficient.

A plausible answer is not yet reliable data

General-purpose AI models are extremely effective at understanding the content of a document.

But in a business process, obtaining a plausible answer is not enough.

You need to obtain usable data.

Suppose, for example, that the following is extracted from an invoice:

Total: €12,450

For a person who is simply reading the document, this information may be sufficient.

An accounting system, however, needs to answer many other questions.

Does that value really represent the final total?

Is the currency actually EUR?

Was the decimal separator interpreted correctly?

Are taxable amount, VAT and total consistent?

Is the document really an invoice, or is it a credit note?

Are there multiple totals on the page?

Are there similar values that the model may have confused?

If the data does not pass these checks, the system needs to know how to behave.

In other words: extracting a value from a document does not yet mean that reliable data has been produced for a business process.

Before being sent to an ERP, CRM, management system or decision engine, the result must be structured, normalized, validated and verified.

The AI model is only one part of the solution

In an enterprise Document AI system, the model used to interpret the document is only one of the components in the pipeline.

A complete process may include several stages.

1. Document acquisition

Files may arrive from:

• web portals;
• mobile apps;
• email;
• APIs;
• management systems;
• CRM;
• internal workflows;
• onboarding platforms.

The system must acquire them, check their format and prepare them for processing.

2. Automatic classification

Before extracting the data, it is often necessary to understand which document has been received.

For example:

• identity card;
• passport;
• payslip;
• CU;
• invoice;
• energy bill;
• contract;
• insurance policy.

Classification determines which pipeline to use and which information to look for.

3. Data extraction

At this point, OCR, multimodal models, VLM and other Document AI technologies come into play.

The system identifies and structures the required information.

4. Normalization

The extracted data must be transformed into a consistent format.

A date such as “12 August 2026” can become “2026-08-12”.

In the same way, amounts, addresses, tax codes, IBANs, document numbers and other values can be normalized.

5. Validation

This is one of the main differences between simple AI extraction and a dedicated platform.

The data can be checked using:

• business rules;
• syntactic checks;
• mathematical checks;
• cross-checks;
• consistency checks between multiple fields;
• checks against data already stored in company systems.

A tax code, for example, can be formally validated.

An invoice total can be compared with the taxable amount and VAT.

An IBAN can undergo structural checks.

6. Exception management

Not all documents can be processed automatically.

When a value is ambiguous or a validation fails, the system must be able to route the document for manual review.

This is the principle of human-in-the-loop.

The operator intervenes only in cases that genuinely require attention, while most documents continue to be processed automatically.

7. Integration

The final data must be transferred to company systems.

For example:

• ERP;
• CRM;
• insurance software;
• credit platforms;
• onboarding systems;
• administrative software;
• data warehouses.

The system must also handle timeouts, errors, duplicates and retry attempts.

8. Monitoring

Finally, it is necessary to know what is happening in production.

For example:

• how many documents are being processed;
• which types generate the most errors;
• which fields require the most reviews;
• how long processing takes;
• how many documents end up in manual review.

Without monitoring, a deterioration in quality can go unnoticed for weeks.

The problem of errors at high volumes

In prototypes, occasional errors may seem relatively insignificant.

At scale, however, they become an operational problem.

Imagine a system processing 100,000 documents per month.

Even an apparently low percentage of problematic cases can result in hundreds or thousands of documents requiring review.

And not all errors carry the same weight.

Misinterpreting a descriptive field may have limited consequences.

Incorrectly extracting:

• an amount;
• an IBAN;
• a tax code;
• a POD;
• a PDR;
• a due date;
• a document number;

can instead directly affect an operational process.

For this reason, a production solution must be designed around a fundamental assumption: the model can make mistakes.

The goal is therefore not to eliminate every error completely, but to build a system capable of recognizing, intercepting and managing it.

Accuracy must be measured at individual field level

Simply asking “Can the model read this PDF?” is not enough.

The quality of a document data extraction system should be evaluated field by field.

For example, for an energy bill it may be useful to separately measure the accuracy of:

• POD;
• PDR;
• tax code;
• supply address;
• account holder;
• contracted power;
• consumption;
• customer number.

A system may achieve high average accuracy while performing worse on one of the fields that is most important to the process.

Evaluation must therefore be carried out on the documents actually received by the company, not only on files selected for a demo.

Test datasets and regression testing

There is also another element that is often underestimated. AI systems change.

What may change includes:

• the model;
• the model version;
• the prompt;
• the OCR pipeline;
• a configuration;
• a validation rule;
• image preprocessing.

A change may improve some documents while making others worse.

For this reason, a Document AI platform must be able to rely on representative test datasets and regression testing procedures.

Before introducing a new configuration into production, it is necessary to verify that the change does not reduce quality on cases that were already being handled correctly.

Versioning therefore becomes an integral part of the process.

The cost is not only the cost of the APIs

When comparing an internally built solution with a Document AI platform, one of the most common mistakes is to consider only the cost of model calls.

Inference cost is certainly an important variable.

But Total Cost of Ownership includes much more.

It is necessary to develop and maintain:

• document acquisition;
• preprocessing;
• classification;
• extraction;
• validations;
• error management;
• review interfaces;
• logging;
• monitoring;
• integrations;
• retry systems;
• test datasets;
• regression testing;
• update procedures.

On top of this comes the time required from technical teams to identify anomalies and keep the system reliable.

The right question, therefore, is not simply “How much does it cost to process a page with the API?”, but “How much does it cost to keep the entire document process reliable?”.

Governance and data control

The issue becomes even more important when processing documents containing personal data or sensitive business information.

Consider:

• identity documents;
• passports;
• payslips;
• contracts;
• financial documentation;
• insurance files;
• healthcare documentation.

In these scenarios, extraction quality represents only part of the problem.

The company must be able to control aspects such as:

• access to documents;
• user permissions;
• audit trail;
• environment segregation;
• data retention;
• data location;
• operation logging.

The specific characteristics naturally depend on the provider, the service used, the chosen configuration and the contractual agreements.

But in an enterprise project, these aspects must be analyzed alongside model quality.

General-purpose APIs or a Document AI platform?

The answer does not necessarily involve choosing one technology and excluding the other.

General-purpose model APIs are extremely flexible.

They are particularly useful when you need to:

• understand a document;
• summarize a contract;
• identify a clause;
• compare documents;
• answer questions about content;
• interpret unstructured text;
• quickly experiment with new use cases.

When, on the other hand, the goal is to transform documents into structured data automatically used by a business process, the requirements change.

This is where Intelligent Document Processing (IDP) comes into play.

A vertical Document AI platform is designed not only to understand a document, but to govern the entire processing cycle.

The role of myBiros

At myBiros, the value does not simply lie in the model used to extract a specific piece of information.

The platform combines different components within a process designed for document processing:

• OCR and AI technologies for complex documents;
• handling of different layouts;
• extraction from tables and multipage documents;
• document classification;
• data structuring and normalization;
• validation workflows;
• business rules;
• exception management;
• human-in-the-loop review;
• integration via API with company systems;
• data control and governance tools.

The goal is not simply to read a document.

It is to transform that document into reliable data that can be automatically used by the business process.

General-purpose models and Document AI can coexist

The two technologies do not necessarily need to be considered competitors.

In many projects, they can be used together.

A specialized platform can handle repetitive and structured activities:

document → classification → extraction → validation → structured data → company system

A general-purpose model can instead be used for activities that require greater flexibility.

For example, it can:

• generate a summary of the document;
• explain an anomaly to an operator;
• compare two contracts;
• interpret non-standardized content;
• support the review of complex cases.

The model therefore becomes one of the components of the solution.

The platform is responsible for governing it within a controlled process.

From “document understanding” to the document process

The most important evolution happens precisely here.

A prototype demonstrates that AI is capable of understanding the document.

A Document AI system must demonstrate something different: that this capability can be used reliably every day within a business process.

The difference is not only technological. It is above all architectural and operational.

It means moving from “Can the model find this data?” to “Can we automatically use this data within our process?”.

This is the gap between a convincing demo and a system that is genuinely ready for production.

Conclusions

The APIs from OpenAI, Anthropic, Google and other general-purpose models have made it much easier to experiment with documents.

They are extremely powerful tools and can represent an important component of many Document AI systems.

But when a company needs to process documents continuously, integrate results into its systems and keep quality, exceptions and governance under control, the model alone does not represent the entire solution.

Pipelines, validations, monitoring, integrations and procedures are needed to manage cases where AI does not produce the expected result.

It is at this level that a vertical Intelligent Document Processing platform creates value.

Not simply by transforming a PDF into an answer.

But by transforming real documents into structured, verified data ready to power real business processes.

Want to understand how to automate data extraction from your company's documents?

Find out how myBiros can integrate with your document acquisition, control and processing workflows.

Contact us or book a demo to discover the enterprise solutions developed by the experts at myBiros.

FAQ

Can OpenAI APIs extract data from PDFs and documents?

Yes. Multimodal models can be used to interpret documents and return structured information. In a large-scale business process, however, extraction must be supported by checks, validations, exception management, monitoring and integrations.

What is the difference between an LLM and a Document AI platform?

An LLM is a model capable of interpreting and generating information. A Document AI platform, on the other hand, integrates models, OCR, classification, extraction, validations, workflows and control systems to transform documents into data that can be used in business processes.

What is Intelligent Document Processing?

Intelligent Document Processing, or IDP, is the combined use of OCR, artificial intelligence, machine learning and software workflows to classify documents, extract information, validate it and automatically transfer it to other systems.

When is it worth using a dedicated Document AI solution?

A dedicated platform becomes particularly useful when large volumes of documents are processed, many different layouts exist, data needs to be validated or must automatically feed ERP, CRM and other business processes.

Can general-purpose models and IDP platforms be used together?

Yes. An IDP platform can manage structured processing, validations and integrations, while a general-purpose model can be used for summaries, interpretation of complex cases, document comparison or operator support.