A useful first AI project has a bounded job, examples of that job, and someone who can judge the result. “Add AI to operations” is too broad to evaluate. “Extract these six fields from incoming requests and put exceptions in a review queue” can become a testable scope.
The following method is a planning framework. The quantities in the example are illustrative, not a client result or a forecast.
Describe the workflow before the technology
Follow one item from arrival to completion. Record what comes in, where it is stored, which decisions people make, which systems change, and what happens when information is missing.
Include the work around the task. Reading an invoice may be quick; finding the related order, resolving a mismatch, and contacting the supplier may consume most of the time. An extraction model helps only if its output improves that broader process.
Ask the person doing the work to identify a normal example and a difficult example. Differences between the two often determine the real scope.
Find a narrow, observable outcome
Good candidates have repeatable inputs and results that can be checked. Examples include document classification, field extraction, draft preparation, and retrieving a relevant policy.
A first pilot should define the supported input types and explicitly handle everything outside them. “Only these three request categories, with all others routed to review” is an actionable boundary.
Decide which parts need a model. Matching a known account identifier, checking required fields, and recording an approval can usually be deterministic. Use AI for the parts where language or document variation makes conventional rules unwieldy.
Estimate the whole cost of doing the work
Suppose a team processes 200 requests a week at eight minutes each: 1,600 minutes. An illustrative pilot reduces routine work to three minutes per request, but 20% require five additional minutes of correction.
That becomes 600 minutes of routine handling plus 200 minutes of correction, or 800 minutes before maintenance and support. The potential time saving is 800 minutes, not the saving suggested by measuring only the fast path.
Put your own observed quantities into the calculation:
Net time saved =
previous handling time
- new handling time
- correction and exception time
- ongoing operating timeTime saved is not automatically cash saved. Establish whether the freed capacity improves turnaround, reduces backlog, avoids additional staffing, or serves another concrete business objective.
Build a representative acceptance set
Collect examples across common and difficult cases. Include missing fields, inconsistent terminology, duplicate requests, and inputs the system should decline to process. Keep a held-out set that is not used to tune the prompts or rules.
Agree what counts as correct. For extraction, compare individual fields and consequential mistakes. For drafting, define the required facts and prohibited unsupported claims. For search, identify the expected source and whether the answer is supported.
Do not evaluate only with attractive demonstrations. Measure what happens on the cases that currently consume staff attention.
Specify the review and failure path
A reviewer needs the original input, the proposed result, and an understandable way to correct it. Corrections should be recorded so the team can distinguish recurring defects from new exceptions.
For system updates, use stable identifiers and explicit retry behavior. If a request times out after an update succeeds, a blind retry should not create another record. The process owner should know how to pause the automation and continue the work manually.
Decide whether to proceed
At the end of the pilot, compare the measured workflow against the original baseline. Include review effort, failure handling, user feedback, and expected operating cost. Proceed, revise the scope, or stop with a useful finding.
The deliverable is a decision supported by examples and measurements. The engagement process shows how that evaluation fits between assessment and implementation.