» SPECTROSCOPY
» Calibration Test Set
The EMA indicates “A calibration test set may be used (instead of cross validation within the calibration set) to provide the first ‘test’ or check of the validity of the model”. The EMA document also indicates that the calibration test set does not represent independent validation of the NIRS procedure, since the samples are taken from different batches within the same historical population of batches. The EMA document further indicates that the calibration set often consists of two thirds of the available sample population and the calibration test set is the remaining third. The calibration test set is similar to the use of cross- validation and this is acknowledged by the EMA and FDA documents. The EMA states in its glossary section that internal validation is not a substitute for the external validation of the model.
The term “calibration test set” is used to describe the first test of the validity of the model and acknowledges that multivariate calibration models, such as partial least squares regression, are rich in diagnostic tools that are very useful to optimize a calibration model.13
The
calibration test set samples may be used to optimize the calibration model and progress towards the prediction of an “independent validation set for external validation of the model”.
Validation Sets
The EMA guideline indicates: “Chemometric data analysis works by correlating the variance in the NIRS signal to a number of latent variables or factors, constrained by a set of calibration reference data. There is always a risk that the correlations identified by the software are due to chance only and not to changes in the analyte; therefore chemometric models should always be validated with an independent set of samples”. Thus, the mission of the “independent data set” is clearly established as assuring that predictions are due to changes in the analyte and not a result of chance. Previous studies have shown that this is a reasonable concern.14,15
However, the FDA and EMA documents do not clearly describe the expected “independent data set”
The two documents appear to indicate that a first requirement for external validation (“independence”) is demonstration that production- scale batches are adequately predicted. The EMA document states “the external validation set should cover the calibration range of the NIRS model, including all variation seen in the commercial process and should include pilot and production-scale batches, where possible”. The FDA draft guidance indicates that “samples used for external validation should span a suitable range of operating conditions (i.e. ranges expected during commercial production) and should be obtained independently from the calibration and internal validation samples used during the development of the NIR models.” The documents appear to acknowledge that many calibration models are developed in laboratories where it is easier and more cost efficient to vary the concentration of samples than in a pilot or production facility. However, the end goal is to use this calibration model for the prediction of samples in a manufacturing facility. This prediction in the manufacturing facility is thus considered independent of the laboratory efforts.
50 | | May/June 2016
The EMA guidelines also indicate an expectation that spectroscopic results be compared to a reference analytical procedure based on destructive testing. The EMA guidelines state: “Interpretation of the complex spectra of unprepared samples generated by NIRS measurement usually requires the use of chemometric calibration models. These models are developed using carefully selected and representative samples, which normally require qualification by independent, reference analytical procedures (normally requiring destructive sample preparation to extract or isolate the analyte of interest and calibration and validation using analytical reference standards”. The words “normally requiring destructive sample preparation to extract or isolate the analyte of interest”, are indicative of the use of High Performance Liquid Chromatography (HPLC). HPLC is able to quantify both an active pharmaceutical ingredient (API), process impurities and degradation products. The use of HPLC usually requires analysis in a separate laboratory, and this can be considered an “independent” result. However, both NIR and reference methods depend on the absolute calibration and use of analytical balances.
The importance of the analytical balance is frequently forgotten. This author has attended conferences where a speaker has mentioned the need to use HPLC, that gravimetric calibration samples are not sufficient. However, HPLC methods also depend on analytical balances and the weight standards used. HPLC methods often calculate drug concentration based on the weighing of 10 milligrams or less of a costly reference standard. In turn, the gravimetric preparations for the calibration blends in many NIR methods may involve the weighing 1 – 100 grams of the excipients and API. Thus, the gravimetric mea- surements in the preparation of NIR calibrations models could be more reliable than those in HPLC methods. HPLC methods also have a number of errors that are not found in NIR spectroscopy such as incomplete extractions and dilution errors.
The approach recommended by this author is develop the calibration model with samples prepared gravimetrically in laboratory facilities. The calibration model could then be challenged with a one or more test sets prepared gravimetrically within the developer’s laboratory. The test sets could involve:
• samples prepared with batches of excipients and API different from those used in the calibration set, or prepared using different sieve cuts. These determinations would also be part of the robustness challenges to the calibration model.
• samples prepared with separate weighing steps (should not be prepared from a single stock blend).
• samples with different proportions of excipients and API. These samples will challenge the method's ability to predict adequately in spite of subtle process variations.
• spectra that are not obtained on the same day as the calibration samples, to determine whether the calibration model is able to handle subtle variations in instrument noise. Noise spectra may also be obtained by using the reference material as sample. This noise spectrum may be multiplied by a factor of 2 or 3 and added to the NIR spectra
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52 |
Page 53 |
Page 54 |
Page 55 |
Page 56 |
Page 57 |
Page 58 |
Page 59 |
Page 60 |
Page 61 |
Page 62 |
Page 63 |
Page 64 |
Page 65 |
Page 66 |
Page 67 |
Page 68 |
Page 69 |
Page 70 |
Page 71 |
Page 72 |
Page 73 |
Page 74 |
Page 75 |
Page 76