Quality Metrics: Are Your Numbers Driving Better Decisions?

What should this number help you decide?

Investigations are closing on time. Training completion is high. The overdue action list is shrinking. Your quality dashboard is green, yet the same problems keep returning.

A useful quality metric helps you understand what is happening, where attention is needed, and whether an action made a meaningful difference. An accurate number can still leave you with an incomplete picture.

Start with a decision rather than an available chart. If a team wants to reduce batch record errors, training completion shows whether an activity happened. To understand performance during the work, ask which entries need correction, how often, and under what conditions. Those questions can direct attention toward the form, instructions, workflow, or access to information.

ICH Q10, sections 1.7(g) and 2.3(e), connects performance indicators with quality-system effectiveness and progress against quality objectives. The practical challenge is choosing measures that make those discussions useful.

This guide accompanies the Quality Metrics video. The batch record example below is hypothetical and illustrates internal quality review, not a regulatory reporting definition.

Agree on the count, population, and period

For our example, measure the percentage of batch records requiring at least one correction during their first review. Use a consistent definition:

• Numerator: batch records with at least one qualifying correction identified during that first review.
• Denominator: all batch records completing their first review in the same reporting period.
• Calculation: numerator divided by denominator, multiplied by 100.

A record with five corrections counts once in this measure. Track correction counts and types separately if those details support another decision. Agree on what qualifies as a correction, the included record types, the data source, the period-assignment rule, and how exceptions are handled.

Two teams can use the same metric name while measuring different things. Compare definitions before comparing performance. If you change the definition, explain when it changed and whether earlier results can still be compared fairly.

More corrections, but a lower rate?

Consider these two hypothetical months:

• Month one: 8 of 100 reviewed records require correction, or 8%.
• Month two: 12 of 200 reviewed records require correction, or 6%.

The number of affected records increased from 8 to 12. The proportion decreased from 8% to 6%. Both statements are true.

The count helps describe the workload associated with affected records. The rate helps compare results when review volume changes. Neither establishes that the process improved. You still need to know whether the records, review practices, and operating conditions were comparable, and whether the pattern continues.

Keep the denominator visible beside the percentage. A result of 1 in 10 and a result of 10 in 100 both equal 10%, but they provide different amounts of information about performance.

Look beneath the average

Most corrections might involve a missing reference. Another could concern information needed to assess whether a critical process step was performed correctly. A single percentage cannot explain the difference in potential impact.

Classify and assess issues through your established procedures. Keep consequential categories visible rather than letting a large volume of lower-risk corrections dominate the overall picture. Respond to a serious individual issue when it occurs; do not wait for the monthly dashboard to show a trend.

Separate results by process, record type, or another relevant factor when that helps interpretation. A change in the mix of work can move the overall percentage even if performance within each group stays the same. Ask what changed in the work as well as what changed in the number.

Two points do not establish improvement

Plot comparable results over time and keep the volume behind each point available. Note relevant changes in products, staffing, procedures, review practices, and data collection. With small numbers, one additional event can move a percentage substantially.

Choose an analytical method that fits the question and the data. Statistical process control can support interpretation when its assumptions and sampling approach are appropriate. The NIST explanation of proportions control charts shows how the sample size affects the variability of a proportion and the chart's limits. It also describes assumptions behind the binomial model, including independent observations.

Our two-month example is not enough to establish statistical control or justify a control chart. Do not add control limits merely to make a dashboard look more rigorous.

Keep three concepts distinct: a business target states a desired result; statistical control limits help assess process behavior; a product specification defines an applicable acceptance requirement. Meeting one does not establish that the others are satisfied.

Pair speed with quality

Investigation closure time can reveal delays. Pair it with suitable evidence about investigation quality, recurrence, or CAPA effectiveness to understand what the completed work achieved. A shorter closure time is difficult to celebrate if the same issue keeps returning.

Also consider earlier signals of potential trouble. The batch record correction rate describes an outcome that has already occurred. Checking whether required instructions and materials are available before work begins could be an earlier signal if that readiness issue is a known contributor to errors.

The relationship needs evidence. Calling a measure a leading indicator does not establish that it predicts an outcome. Choose complementary measures that answer useful questions, and revisit them when the process or risks change.

What behavior does the target encourage?

A target for fewer reported deviations can create pressure to avoid reporting. A lower count may reflect improvement, weaker detection, or reluctance to raise concerns. The number alone cannot tell you which explanation fits.

Likewise, a strong focus on rapid closure can encourage teams to finish an investigation before the evidence supports its conclusion. Examine the work behind the result and how people experience the target.

Make it safe to raise concerns. Discuss what the metric could hide, and look for evidence that distinguishes a genuine improvement from a change in reporting behavior. Keep required records and reporting obligations intact when revising an internal dashboard.

A chart cannot repair an unreliable denominator

Assign an owner for each metric and make the calculation traceable to its source records. Check for missing or duplicated records, inconsistent classifications, and records assigned to the wrong period. Make exclusions and limitations visible.

In our example, incomplete review logs could make the correction rate look better because difficult records were left out. A zero should mean that a defined search of the relevant population found no qualifying events. Missing data is a different condition and should be shown as such.

Keep an explanation of changes to definitions and historical results. If a correction to the source data changes a prior report, preserve enough context for someone to understand what changed and why.

Use the signal to ask a better question

Suppose the record data shows repeated corrections around one confusing instruction. Review the affected records, speak with the people doing the work, and test whether the evidence supports that explanation. The metric has identified a question; it has not established the cause.

If the evidence supports a change, use the applicable change-control process. Assign an owner and decide how the result will be assessed. The review may also identify resource needs, unresolved risks, or reasons to continue monitoring. Record the reasoning behind the decision.

Return to the result under comparable operating conditions. Did the targeted correction type become less common? Were new difficulties introduced? Keep the learning available to the next reviewer through knowledge management, including the evidence and the limits of the conclusion.

Six questions for your next review

1. What decision does this measure support?
2. Are its definition, population, calculation, and source clear?
3. What do the denominator, risk, and mix of work add to the picture?
4. What could make the result misleading, including incentives or missing data?
5. Who responds, and what evidence would justify action?
6. How will you check whether the decision helped?

Start with one measure that people regularly discuss but rarely act on. Improve its definition or context, then see whether the conversation produces a clearer next step.

Quality metrics become useful when the number, its context, the risk, and a decision stay connected. A focused dashboard should help people understand the process and its uncertainties well enough to choose what happens next.

Guidance, statistical background, and practical judgment

• ICH Q10: Pharmaceutical Quality System, sections 1.7(g), 2.3, 3.2.1, 3.2.4, and 4.1–4.3, addresses performance indicators, monitoring, management review, and improvement. The guideline is used alongside applicable regional GMP requirements. It does not prescribe the batch record metric or numerical targets in this article.
• NIST/SEMATECH e-Handbook: Proportions Control Charts explains the statistical model and assumptions for p-charts. It is statistical reference material, not a GMP requirement or a recommendation to apply that chart to every dashboard.
• FDA: Quality Metrics for Drug Manufacturing explains how metrics can support pharmaceutical quality and provides background on FDA's reporting-program proposals. An internal dashboard is not the same as a regulatory submission; this guide does not establish a reporting obligation.

The example, dashboard questions, and suggested pairings are Quality Unpacked educational applications. They are not a mandatory template, a validated metric set, or regulatory endorsement. Apply your approved procedures and the requirements relevant to your product, operation, and markets.