How consistent is fee verification across mortgage loan files?

Two reviewers can look at the same loan estimate and reach different conclusions about a fee. One treats a changed title charge as a tolerance issue, while the other treats it as a revised quote to document. The difference may not affect the borrower, but it affects whether the file is handled the same way each time. This protocol measures fee verification consistency without pretending that every fee question has a single mechanical answer.

Question and scope

The question is how consistently reviewers classify and document fees across a defined set of mortgage files. The scope should name the loan type, the disclosure stage, the fee categories, and the reviewers. A team might study purchase files at the closing disclosure stage across two processing pods. The result describes that setting and should not be generalized to every mortgage product.

Separate the review of fee accuracy from the review of fee tolerance. Accuracy asks whether the amount on the disclosure matches the supporting source. Tolerance asks whether a change exceeds the allowed limit. A file can be accurate and still have a tolerance problem, or the reverse. The protocol should state which question it is answering.

Unit of analysis

The unit is one fee line on one file at one disclosure stage. A file with twenty fee lines contributes twenty units, but the report should also show the file-level result. That prevents one file with many small fees from dominating the analysis. Record the file key, the fee category, the disclosure version, the source document, and the reviewer classification.

Keep the borrower identity out of the analysis file. Use the fee line as the smallest unit because reviewers often agree on the total while disagreeing on a single line. A file-level average can hide that disagreement. The line-level view shows where the training or the checklist needs attention. Add the disclosure stage to every line so a closing disclosure fee is not compared with an early loan estimate fee.

Defining fee categories

Group fees by the rule that governs them, not by the label on the disclosure. Some fees have a zero tolerance, some have a ten percent cumulative tolerance, and some can change with a valid reason. The categories should match the team's approved policy and the relevant disclosure rules. A category that mixes two rules will produce disagreement that looks like reviewer error.

Write a short definition and an example for each category. Include the source the reviewer should check, such as the service provider quote or the title company invoice. The disclosure review lead should approve the category list before the sample is scored. If the list is too long, combine categories that share the same rule.

A fee category should also record whether the reviewer needs to compare the amount, the payee, or both. A title fee can change amount without changing the provider, while a recording fee can change both. The reviewer's task is different in each case. Adding that field prevents a match on the payee from being treated as a match on the charge.

Sampling and observation

Use consecutive files from the chosen stage, or a random sample with a recorded start. Include files from each reviewer and each loan officer if the volume allows. Record the number of files excluded and the reason, such as a missing disclosure or an incomplete source document. A sample that excludes the files with missing sources will overstate consistency.

Give each reviewer the same materials and the same instructions. Do not let one reviewer see another's classification before recording their own. If the review is done in the normal workflow, capture the classification before any discussion. A consensus reached after a meeting is not the same as an independent review. Add the time spent on each file so the study can show whether consistency comes at an unreasonable cost.

Data quality checks

Check that every fee line has a source and a category. A line with no source cannot be scored for accuracy, though it may still be scored for documentation. Mark missing sources separately rather than treating them as correct. A high missing-source rate is a finding about the file, not about the reviewer. Report the missing-source count beside the agreement rate so the two are not confused.

Compare a subsample of classifications with a second reviewer. Report agreement by category, not only overall. Reviewers may agree on most fees and disagree sharply on title or recording charges. The category-level result tells the team where to focus. Keep a short example of each disagreement so the team can discuss the rule rather than the person. The CFPB's Loan Estimate and Closing Disclosure guides are useful references for the borrower-facing forms.

If the reviewers disagree, ask them to explain the rule they applied before showing the other answer. The explanation often reveals that one person used an older policy. Record the rule version with each classification. A disagreement caused by a stale reference is a training issue, while one caused by an ambiguous rule is a policy issue.

Build a small decision tree for the categories with the most disagreement. The tree should ask two or three yes or no questions and end in a category. Test it on the same sample and compare the new agreement rate. A simple tool that reviewers actually use is better than a long guide they ignore.

Analysis and reporting

Report the number of fee lines, the agreement rate by category, and the number of lines with missing documentation. Add a file-level view showing how many files had at least one disputed line. A category with low agreement and high volume deserves attention before a category with low agreement and one occurrence. Include the reviewer count so a reader can judge whether the sample represents the team.

Separate a difference in classification from a difference in the underlying amount. Two reviewers may agree that a fee changed and disagree about whether the change is allowed. That is a rule interpretation issue. Two reviewers may disagree about the amount itself, which is a data or source issue. The report should not merge the two.

Limits and sources of error

The sample measures agreement, not correctness. Reviewers can agree on the wrong answer. Include a small audit against the approved policy or a subject matter expert to estimate the correct classification. Without that check, a high agreement rate may only show that the team shares the same misunderstanding. Report the correct-classification check separately from the agreement rate.

The fee categories may change during the study if the policy is updated. Freeze the category list for the observation period and record any change. A file reviewed under an old rule should be scored against the rule in effect at the time. Mixing versions creates false disagreement and undermines the comparison. Keep the rule version next to each line so the analysis can be filtered later.

Review cadence

Repeat the protocol after a training session, a checklist change, or a new disclosure rule. Use the same categories and sample design so the comparison is meaningful. If agreement rises but missing documentation also rises, the team may have standardized the easy part while leaving the hard part untouched. Report both movements. Add the date of the training or rule change so the comparison has a clear boundary.

Add one open question to the end of the report. The question should name the category the team still cannot classify consistently and the evidence it needs. A short open question gives the next review a purpose. It also prevents the report from ending as a scorecard with no next action. Assign the question to a named owner before the report is distributed.

Keep the category definitions, sample list, and disagreement examples in one place. A later reviewer should be able to see why a line was classified a certain way. The mortgage operations support team can use the result to decide which fee category needs a decision tree or a second review. That change is more useful than a general instruction to be more careful.