Statistical Data Evaluation

Statistical Data Evaluation models and functions are used to evaluate analytical method validations, estimate measurement uncertainties, maintain control charts and calibration models, and evaluate interlaboratory comparisons.

Scope and use:

  • Execution, assessment and reporting of method validation studies
  • Estimation of uncertainties
  • Development and use of control charts
  • Development of use of calibration models
  • Evaluation of inter-laboratory comparisons

Method validation

Repeatability

Repeatability is a measure of the method’s precision obtained under the ideal measurement conditions (single instrument, single analyst, a short period of time) and can be evaluated in two ways in EffiChem:
– From multiple measurements on a level
– From parallel measurements

Intermediate precision

Intermediate precision is a precision measure between repeatability and reproducibility. It can be evaluated from multiple measurements of parallel measurements per level.

Reproducibility

Reproducibility is a precision measure obtained under the worst measurement conditions (multiple instruments, multiple analysts, a long period of time) and can be evaluated from multiple measurements or from parallel measurements per level.

Accuracy

Method accuracy describes the degree of agreement of a measurement with a reference value/method. The accuracy can be evaluated in several ways, depending on the concentration range and the data availability:
– Limited concentration range – sample reconstitution possible
– Limited concentration range – reference material available
– Large concentration range – blank available: regression
– Large concentration range – blank not available: t-test
– Large concentration range – blank not available : regression
– Comparison of two methods/laboratories: t-test by levels
– Comparison of two methods/labs: t-test for difference in results

Linearity

Linearity describes the degree of linear dependence between the concentration and the measured signal. It is evaluated in different ways depending on the availability, type, and design of the validation data:
– Correlation and QC coefficient
– ANOVA for Lack of fit
– Significance of the quadratic term
– Sign test

Limit of detection and Limit of quantification

The detection and quantification limits are concentration levels at which the signal can be distinguished from the noise or the concentration of the analyte being measured can be reliably quantified. The limits can be evaluated in several ways:
– 3s – IUPAC
– 3s – blank correction
– 3s – continuously measured blank
– From the calibration line

Robustness

Robustness is an ability of the method to cope with small changes in the method setup or in the adherence to the prescribed conditions, such as pH of the mobile phase in HPLC or detector temperature in GC, without having impact on result. Robustness can be evaluated in EffiChem in two ways:
– Dong’s algorithm
– AOAC with evaluation of

Selectivity

Selectivity is the ability of a method to selectively measure the concentration of the analyte of interest in the presence of potent interferents. Selectivity can be evaluated in several ways:
– By comparing calibration lines
– By comparing the results with a standard
– By comparing the results with measurements without interferents

Sensitivity

Sensitivity is the measure of change in the measurand (signal) per unit change in the concentration. It is calculated as the slope of the calibration line between the concentration and the measurand.

Blank

A blank experiment measures the magnitude of the measurand in a sample with zero concentration. It is the basis for calculating the limit of detection and the limit of quantification.

System Suitability Check

System suitability check is the confirmation that the system meets the prescribed parameters, namely PhEur requirements or USP/FDA requirements:
– Repeatability of injection according USP
– Capacity factor according to USP
– Resolution according to USP
– Tailing according USP
– Theoretical plate number according USP
– Theoretical plate number according PhEur
– Resolution according PhEur
– Symmetry factor according PhEur

Uncertainties, control charts, and other

Control charts

Control charts are developed to monitor and evaluate the degree of variability in the method or results over time, to evaluate trends and to set warning and action limits. In EffiChem, there are several types of control charts to select from, depending on data availability, design, and range:
– Individual values – control characteristics determined
– Individual values – control characteristics not determined
– Repeated measurements – control characteristics determined
– Repeated measurements – control characteristics not determined
– Multivariate-variate control chart
– X-diagram (Control chart for Average)
– R-diagram (Control chart for Range)
– Western-Electric rules for evaluation of control charts

Uncertainty

Quantifying uncertainty in analytical measurement is one of the key requirements of ISO17025. Several approaches or methods can be used for this purpose, depending on the situation and data availability:
– From Precision data – multiple measurements
– From Precision data – parallel measurements
– From Control charts – individual measurements
– From Control chart – multiple measurements

Calibration

A linear or quadratic calibration model allows to estimate the concentration or the property of interest of the unknown sample from the value of the measurand. In addition, the calibration model is used for other purposes, e.g. to determine linearity, limit of detection, limit of quantification, and to determine the method sensitivity.

Inter-laboratory comparison

Inter-laboratory comparisons is used to assess whether a laboratory participating in an interlaboratory comparison is reporting results that are significantly different from those of other laboratories, or not. Furthermore, the inter-laboratory comparison can be used to evaluate the method’s repeatability and reproducibility. The following algorithms are included:
– Mandel statistics H
– Mandel statistics K
– Collaborative studies

Audit trail and History

The audit trail and history are kept for all Statistical Data Evaluations conducted. All records are 100% traceable and cannot be falsified or deleted. Undesired records can be moved to a bin with a rationale provided.

Connection with LIMS modules

Statistical Data Evaluation can be either linked to the Methods LIMS module, or configured separately, within one single validated system, meeting the data integrity requirements defined.

 

Examples EffiChem 5.0 Statistical Data Evaluation

https://www.effichem.com/wp-content/uploads/2022/04/2__Procedures-hotovo.jpg

Methods module

The Methods module enables to create, review, approve, publish and use test methods and to enter and evaluate the corresponding method validation data, control charts, uncertainties and calibration models. All data and statistical evaluations are recorded in the audit trail and history.

https://www.effichem.com/wp-content/uploads/2022/04/3__Procedures_-_select_validation-hotovo.jpg

Methods: selection of the validation parameter to be evaluated

The drop down menu first lets you choose between full validation, exploratory validation, etc., then from the list of parameters to be validated, (Repeatability, Accuracy, Linearity, etc.). In the last step, you can choose from the list of algorithms that can be applied (Level by level from multiple measurements, or from parallel measurements).

https://www.effichem.com/wp-content/uploads/2022/04/4.-Linearity-graph.png

Linearity data

An example of data for evaluation of Linearity. The evaluation is done numerically and graphically

https://www.effichem.com/wp-content/uploads/2022/04/4.-Linearity-graph-1.png

Linearity evaluation

An example of graphical evaluation of Linearity in the method validation module, showing the relationship between the concentration and the measured values. For a more detailed evaluation, a plot of residuals or supporting statistics can be used.

https://www.effichem.com/wp-content/uploads/2022/04/5.-Repeatability-data.png

Repeatability data

An example of data for evaluation of Repeatability. Rows in the table correspond to the individual concentration levels, the columns to the repeated measurements at the given level. The data evaluation is done numerically and graphically.

https://www.effichem.com/wp-content/uploads/2022/04/5.-Repeatability-graph.png

Graphical evaluation of repeatability

An example of the graphical evaluation of repeatability in the method validation module, showing the concentration levels (x-axis) plotted against the difference between the measurement and the average measured values at the given concentration level (y-axis). This graph shows the method variability/repeatability.

https://www.effichem.com/wp-content/uploads/2022/04/5.-Repeatability-results.png

Numerical evaluation of repeatability

An example of the numerical evaluation of repeatability: rows show repeatability and the relative repeatability at the individual concentration levels. The conclusion line reports the average repeatability and the average relative repeatability.