Intelligent document processing (IDP) is the automated capture and structuring of data from documents. In a world where one person generates 1.7MB of data per second, sophisticated data management tools – such as IDP – have evolved from a luxury afforded by the largest enterprises to a necessity for most firms in financial services.
Finding and implementing an IDP solution may seem daunting, not the least due to the influx of IDP vendors. As of 2023, Gartner estimates that over 60 IDP companies are delivering solutions. Even more, as the SaaS sales process can span weeks, we’d recommend asking the right questions at the initial demo.
In this article, we’ll break down the right questions to ask, enabling your company to unlock the multiple benefits of IDP.
First, it’s best to determine the limits of an IDP tool’s functionality. Consider asking a vendor the following questions:
Confidence scores validate data quality. For each data point, a score indicates the likelihood that the AI has captured and attributed the data correctly.
Some IDP tools release unstructured data or data in fixed formats (such as CSV), which may not be immediately actionable and require manual conversion.
A fully end-to-end data extraction service will include creating and updating the existing AI extraction models to fit your company’s use case, adding a human-in-the-loop (HITL) system, implementing the model (including any post processing) and providing ongoing support.
Even if the tool you’re considering offers impressive functionality, it’s important to check that its quality extends to its outputs. Consider verifying a few fundamental facts before proceeding.
If the system struggles to learn from incorrectly captured data, then it’s likely some kind of rule-bound OCR, not true AI. An IDP vendor should be able to prove the AI model’s learning capabilities during a demo.
IDP technology may be packaged as a managed service, where human analysts or annotators validate the captured data. If so, the vendor should be able to guarantee complete accuracy.
AI has advanced beyond the stage where 99% accuracy is acceptable. Therefore, it’s important to assess how IDP vendors can vouch for the product's data quality.
So far, during the demo, the IDP company representative has supplied basic information. Now it’s time to ask them questions specific to your business’s use case.
Ask them to talk you through a case study matching your company’s use case. In particular, you might ask:
For example, you might ask:
In addition to assuring you that their annotators possess demonstrable experience with the document type(s) your business requires, it’s also important the IDP vendor answers your questions confidently. A well-informed and connected team is the only way a data extraction vendor can provide the level of service your business deserves.
At this point, you’ll have acquired an overview of the IDP service. The next step is to explore the product’s technical details. Enquiring about these details will be useful on two levels. For one, it will allow you to check the robustness of their tool, and it will provide the vendor with a better understanding of your business’s use case.
Let’s explore some of the technical details you’ll want to discuss with the vendor.
Part of the API’s accessibility can be demonstrated through seamless integration with your company’s architecture. What do we mean by ‘seamless integration’? Firstly, no IDP tool should require a lengthy configuration period. The tool should also be minimally disruptive to your company’s IT architecture.
A poorly implemented AI technology will inevitably cause issues downstream due to its slow performance or network and storage complications. Therefore, it’s important that a vendor’s API demonstrates robustness, which it can do by:
This capability distinguishes sophisticated intelligent document processing platforms from LLMs like GPT-4, which are extremely unwieldy to train.
When it comes to this requirement, there are no compromises.
Some of the top red flags to watch out for with a potential vendor include whether they:
Understandably, you may have reservations about letting a third party access your company’s data. Checking that the IDP vendor commits to data security is essential, so it’s good practice to check that the following are in place:
Once you’ve validated the technical features of the IDP tool, it’s time to negotiate a price. There are several pertinent questions that will determine whether the IDP platform’s pricing matches your project’s requirements, such as whether the vendor offers a volume discount. Let’s explore the others.
Transparent pricing – whether per page or by license – is essential to make an informed purchasing decision.
Some IDP vendors will charge a hefty upfront fee, which you’ll need to factor into the project’s costs.
Self-service refers to the IDP tool itself, whereas a managed service will deploy a HITL system to validate the data before it is returned. Consider which is best suited to your company culture, your budget and your current IT architecture.
When you’re ready to book a discovery call with an IDP vendor, here are a few general tips:
In summary, though shopping for an IDP solution may feel like a tedious trawl through 60+ technologies, with an organised approach, you can find the right tool for your company’s requirements.
If you have any questions or comments, please feel free to reach out at hello@evolution.ai.
You can also download a PDF of our IDP Buyer’s Guide here.
