Straight Through Processing: Measure and Raise It

Ask a team how well its data entry automation works and the answer is usually an impression: "most documents go through fine". Ask how many went through without anyone touching them last week, and how that compares with the month before, and there is often no answer. Straight through processing is the share of documents or transactions that complete from start to finish with no manual intervention. It is the most useful single measure of whether automation is doing its job, and tracking it, along with the reasons items fall out, tells you where to improve and when something has broken. This guide explains how to define it, measure it, find the causes behind it and respond automatically when it drops.

What straight through processing means

A document is processed straight through when it is captured, classified, extracted, validated and exported to its destination without a person reviewing, correcting or re-submitting it. The straight through processing rate is the number of such documents divided by the total processed in a period.

The definition needs care in three places:

• Scope: decide whether you measure per document, per transaction or per field, and keep it consistent
• Exclusions: decide how to count documents rejected as out of scope, such as spam attachments
• Time: count a document once it has finished, so items still in progress do not distort the figure

Write the definition down. A rate that changes because someone changed what counts is worse than no rate at all.

Why the rate matters more than speed

Automation projects often report processing time: documents extracted in seconds instead of minutes. But if half of them then wait in a review queue, the business has not gained much. The straight through rate captures what the business cares about: how much work no longer needs people. It also connects directly to cost, since every document that falls out of the straight path needs someone's time.

The rate should never be pushed up by lowering standards. Removing validation checks would raise it immediately and send errors into business systems. Pair it with an accuracy measure, such as corrections made later in the destination system, so improvements are real.

Instrument every stage

To understand the rate, you need to know where items leave the straight path. Record an event at each stage for every document:

• Captured, with source and document type
• Classified, with the type and confidence
• Extracted, with confidence per key field
• Validated, with each failed check named
• Reviewed, if it was, with who changed what
• Exported, or failed at export with the destination's error

With these events, every document that was not processed straight through has a recorded reason. Group those reasons and the improvement plan writes itself: if most fall-outs are supplier-not-found failures, fix supplier matching; if most are low confidence on dates from one source, look at that source's documents.

Our guide to human in the loop review covers how to design the review step that catches these items.

Watch the rate as a health signal

A sudden drop in the straight through rate is often the first sign that something has broken. Typical causes:

• A large supplier changed its invoice layout
• An extraction service changed behaviour or started throttling
• A reference list used in validation, such as product codes, was not updated
• The destination system started rejecting records after a configuration change

Track the rate daily, broken down by source and document type, and alert when it falls well below its recent level. A drop for one supplier points to a layout change; a drop across everything points to a service or system problem.

Automate the routine responses

Some problems have known, safe responses that can run without waiting for a person. This is where automated remediation helps:

• Temporary failures, such as throttling or timeouts, are retried with backoff
• Export failures caused by an unavailable destination are held and re-sent when it recovers
• Documents classified as unknown are routed to a review queue rather than failing
• When a reference list is refreshed, items that failed lookup against it are re-validated automatically

Each automated response should be logged, so you can see how often it runs. An automated retry that runs constantly is hiding a problem that needs a real fix. Keep anything that changes business data or makes judgements, such as correcting values, with people.

Worked example: a monitoring view for a document pipeline

The table below shows a hypothetical set of measures and alerts for an invoice processing pipeline. Thresholds are placeholders; set yours from your own normal levels.

MeasureBreakdownAlert whenLikely first actionStraight through rateBy source and document type, dailyFalls well below the recent averageCheck fall-out reasons for the affected groupTop fall-out reasonsBy failed checkA new reason appears in the top fiveInspect sample documentsReview queue ageOldest itemExceeds the agreed targetReassign or add reviewersExport failuresBy destinationAny sustained failuresCheck destination status and recent changesAutomated retriesBy causeRetries rising day on dayInvestigate the underlying causeCorrections after exportBy fieldRising corrections in one fieldTighten validation or thresholds

The first row tells you something changed; the others tell you where.

Improve the rate deliberately

Use the fall-out reasons to choose improvements, one at a time, and measure each:

• Add templates or rules for sources that fail often
• Improve reference data, such as supplier and product lists, used in validation
• Adjust confidence thresholds where reviewers rarely make corrections
• Work with senders to improve document quality or switch to digital formats

Our guide to data entry automation tools covers choosing tools that report confidence and validation results in a form you can measure.

Common mistakes with straight through processing

• No written definition, so the rate means different things over time
• Raising the rate by removing validation
• Measuring only the overall rate, hiding problems with one source
• No recorded reason when items fall out
• Automated retries that run constantly without anyone noticing
• Treating speed as the main measure of success

Frequently Asked Questions

What is straight through processing?

It is the completion of a document or transaction from start to finish without manual intervention. The straight through processing rate is the share of items that do so.

How do you calculate the straight through processing rate?

Divide the number of items completed with no manual review, correction or resubmission by the total number completed in the same period, using a consistent written definition.

What is a good straight through processing rate?

It depends on document types and quality. Track your own rate over time and by source; a stable or rising rate with stable accuracy is the goal.

Why would the rate suddenly drop?

Common causes include a sender changing document layout, an extraction service problem, outdated reference data or a destination system rejecting records.

Know how well your automation really works

We instrument document pipelines so you can see straight through rates, fall-out reasons and problems as they happen. See our AI automation solutions, or talk to us about your pipeline.

About the Author

Manav Sharma is the founder of Insyrge and a CRM solutions architect. He works across Zoho, Salesforce, HubSpot, Odoo and more, designing the CRM setups, integrations and automations that keep sales and operations teams working from the same data.