How often do borrower document uploads fail a first-pass review?

A borrower uploads a pay stub, and the processor rejects it because the page is cut off. The borrower uploads it again, and the second version has the right page but the wrong pay period. Each round trip costs a day and adds a message to a queue that is already busy. This protocol measures how often documents fail a first-pass review, why they fail, and whether the failure rate differs by document type or channel.

Question and scope

The question is what share of borrower document submissions fail the first review after intake, under a defined set of review rules. The scope names the loan types, the document categories, the channels, and the reviewers. A team might study purchase files in the processing stage across two portals and one email inbox. The result describes that setting and should not be generalized to every mortgage product or every borrower population.

Define first-pass review before collecting data. A submission passes if the reviewer accepts it for the condition without asking for a replacement. It fails if the reviewer requests a new version, a missing page, a clearer image, or a different document entirely. A submission that is accepted but later replaced for an unrelated reason is not a first-pass failure.

Separate the failure from the condition outcome. A document can pass the first review and still fail to satisfy the condition because the condition itself was written incorrectly. The protocol should measure the document review, not the condition design. That distinction prevents the study from blaming borrowers for a request the team should have phrased better.

Unit of analysis

The unit is one submission of one document for one condition. A borrower who uploads three statements for one condition contributes three units, but the report should also show the borrower-level result. That prevents a borrower with many small documents from dominating the rate. Record the loan key, the condition key, the document type, the channel, the submission time, and the review result.

Record the reviewer and the review time as well. Reviewer differences are a known source of variation, and the protocol should be able to show them. If one reviewer rejects images that another accepts, the team has a calibration problem rather than a borrower problem. The reviewer field makes that visible without turning the study into a performance review.

Keep borrower identity out of the analysis file. Use a loan key that can be linked back to the file by an authorized person, but do not copy names, addresses, or account numbers into the working dataset. The loan processing support lead should approve the data fields before collection begins. A study that creates a new privacy risk is not worth the insight.

Failure categories

Define a short list of failure reasons before scoring. Common categories include a missing page, an unreadable image, an expired document, a name mismatch, an account mismatch, a wrong document type, and a document that does not cover the requested period. Add a category for a request that was unclear, because that failure belongs to the team rather than the borrower. A condition with a vague request should be scored under the unclear category rather than forced into a document category.

Each category needs one example and one test. A name mismatch means the name on the document differs from the name on the application in a way the reviewer cannot explain. An expired document means the statement or letter falls outside the accepted age for that condition. The test should be specific enough that two reviewers reach the same result when they apply it.

Do not use a catch-all category until the specific categories are exhausted. A large other bucket hides the pattern the study is meant to find. If the other bucket grows beyond a small share, review the examples and add a category. The category list can evolve between observation periods, but it should be frozen within one period.

Sampling and observation

Use consecutive submissions from a defined window, or a random sample with a recorded start. Include every channel the team uses and every reviewer who handles the document type. If the sample excludes email submissions because they are harder to export, the result will understate the failure rate for the channel that may need the most attention. A sample that covers only one channel will describe that channel, not the intake process.

Record the number of submissions excluded and the reason. A submission with no review record cannot be scored, and a submission that was withdrawn before review is not a failure. Report those exclusions beside the rate. A high exclusion count is itself a finding about the intake process, because it means the team cannot reconstruct what happened.

Give the reviewers the same instructions and the same category list. Do not let them see each other's scores before recording their own. If the study uses the normal workflow, capture the classification at the moment of review rather than reconstructing it later. A classification made after a discussion is not an independent first-pass decision.

Data quality checks

Check that every submission has a condition, a document type, a channel, and a review result. A submission without a condition cannot be scored for first-pass failure, though it can still be counted as an intake record. Mark missing fields separately rather than treating them as passes. A high missing-field rate weakens the study more than a small sample does.

Compare a subsample with a second reviewer and report agreement by category. Reviewers may agree on the overall pass rate and disagree sharply on name mismatches. The category-level result tells the team where the rules are ambiguous. Keep one example of each disagreement so the discussion stays on the rule rather than the person.

Check the timestamps against the channel's own record. An email received at five in the evening may be logged the next morning, which changes the first-pass interval but not the pass result. Be clear about which measures use the original timestamp and which use the working receipt. The mortgage operations support routine should define the working receipt rule before the study starts.

Analysis and reporting

Report the number of submissions, the first-pass failure rate, and the failure rate by category and channel. Add a file-level view showing how many files had at least one failed submission. A file with one failure is different from a file with six, and the team needs both views to judge the workload. Include the reviewer count so a reader can see whether the sample represents the team.

Compare the failure rate with the rework rate. A failed submission usually creates a new request, a new message, and a new review, so the failure rate is an early indicator of rework. If the two measures diverge, the team may be correcting some failures without a new submission. That is useful information, not a contradiction.

Report the time between the first submission and the accepted version. A high failure rate with a fast correction may be less costly than a low failure rate with a slow one. The interval shows the borrower experience and the processor workload together. Use the median and the spread rather than an average alone, because a few very slow files can distort the mean.

Limits and sources of error

The study measures the team's review decisions, not the objective quality of every document. A reviewer can reject a usable document or accept an unusable one. Include a small audit against the condition standard to estimate that error. Without the audit, a low failure rate may only show that the team is lenient.

The sample may miss documents that were never submitted. A borrower who gave up after a confusing request does not appear as a failed submission, even though the request failed. Add the number of open conditions with no submission and the age of those conditions. That count gives the reader a sense of the work that the failure rate does not capture.

Channel and document mix can change during the observation period. A new portal or a holiday week can shift the rate for reasons unrelated to the process. Record the dates of any channel or policy change and avoid comparing periods across them without a note. The protocol describes one window, and the next window should be designed to test a specific change.

Review cadence

Repeat the study after a change to the request templates, the portal, or the review checklist. Use the same categories and sampling rule so the comparison is meaningful. If the failure rate falls but the time to correction rises, the team may have made the request easier to understand without making the review faster. Report both movements.

Share the category results with the people who write the condition requests. A request that produces repeated missing-page failures may need a clearer example or a different document option. The borrower communication support team can test the revised wording on a small group before a full rollout. The change should be visible in the next sample.

End the report with one open question and a named owner. The question should name the category the team still cannot explain and the evidence it needs. The owner should have a date to bring an answer. A study that ends with a number but no next action leaves the same failures in the queue.