On the evaluation of reliability,repeatability, and reproducibility of instrumental evaluation methods and measurement systems

ABSTRACT

Validation of instrumental evaluation methods or measurement systems plays an important role in both pharmaceutical and cosmetic research and development. In practice, it is suggested that validation should be performed according to performance characteristics as described in the United States Pharmacopedia and National Formulary (USP/NF, 2000) for analytical methods validation. A validated method or measurement system is expected to achieve a certain degree of accuracy and reliability. However, it is a concern whether the test results obtained are repeatable (with similar test samples) and/or reproducible (under similar but slightly different experimental conditions). In this article, reliability and repeatability/reproducibility of a measurement system estimated within a mixed-effects nested design are monitored by relevant variability acceptance limits. A method based on the concept of empirical power (reproducibility) is used to determine these acceptance limits and thus ensure that there is a high probability of repeatability/reproducibility of the tests results. Formulas or procedures for sample size requirements for comparing the variabilities between products are derived. An example is presented to illustrate the use of the proposed method.     

Qualifying ELISA data: combining information

With immunoassay or bioassay data, the assay standards often exhibit considerable inter-assay variability. However, the assay controls, which are used to monitor the assay performance and set acceptance criteria, should have no or less interassay variability. In this paper, we develop a mixed-effect calibration model for the assay controls to set new acceptance criteria and qualify the enzyme-linked immunosorbent assay (ELISA) data, which incorporates the interassay variation of assay standards and the nature of the assay controls, and overcomes the problems caused by traditional fixed-effect calibration model.     

Setting acceptance criteria for validation of analytical methods of drug eluting stents: Minimum requirements for analytical variability

Accuracy and reliability of the analytical results are crucial for ensuring quality, safety and efficacy of drug eluting stents (DESs). Method validation is the process used to confirm that the analytical procedure employed for a specific test is suitable for its intended use. Results from method validation can be used to judge the quality, reliability and consistency of analytical results. Validation of analytical methods includes the identification of the performance parameters relevant for the given procedure, the definition of appropriate acceptance criteria and the appropriate design of the validation studies. Achieving an appropriate consideration of the analytical variability in assay procedures and setting acceptance criteria for analytical validations is however much more difficult than usually described. Criteria which are too wide may lead to unnecessary and incorrect out-of-specification (OOS) cases, resulting in bad reject decision for products. This study concentrates on analysis, through simulation, of the relation of method variability with specification limits for the total loaded dose of the active substance on the DES. The findings of this study point what levels of precision and accuracy are needed, in other words what is the magnitude of the allowable total error from all possible effects (both systematic and random) in an assay method in order to achieve the level of performance required for the methods applied routinely for the evaluation of the total loaded dose of DES as part of lot release/stability testing.     

A Quantitative Assay of Telomerase Activity

Purpose. Telomerase is a ribonucleoprotein that extends telomeres at the ends of chromosome. Increased telomerase activity is associated with cellular immortality. The currently available assay for telomerase, i.e., telomeric repeat amplification protocol (TRAP), consists of 2 steps: (a) telomerase-mediated extension of an oligonucleotide primer by the enzyme-containing extracts of cells and tissues, and (b) amplification of the telomerase-extended primer products by polymerase chain reaction (PCR) and detection of the PCR products. It is generally accepted that the current TRAP assay lacks quantitative precision. The present study was to develop a quantitative telomerase assay with greater precision and sensitivity. Methods. This new method used the primer extension method as in TRAP, plus the following modifications: (a) used a lysis buffer that yielded complete lysis of nuclei; (b) removal of PCR inhibitors by phenol/chloroform extraction after primer extension; and (c) used primers for the internal standard that were designed to reduce their competition with the telomerase products for PCR. Results. The modified method showed a good correlation (r² = 0.99, P < 0.001) between telomerase amount (expressed as total protein in cell lysate) and its activity (expressed as telomerase products). Compared to the conventional TRAP, the new method (a) was more sensitive (average of 5.5-fold in cultured cancer cells and >5.9-fold in patient tumors), (b) had a lower inter- and intra-day variability (>3-fold), and (c) showed a 2 to 4-fold broader range of linearity in the standard curve. The higher assay sensitivity further enabled the use of a nonradioactive method, i.e., ethidium bromide staining of DNA, to detect the TRAP products, as opposed to the use of radioactive nucleotide and the more labor-intensive autoradiography mandated by the conventional TRAP. Conclusion. We report here a quantitative assay for telomerase activity in cultured human cancer cells and patient tumors.     

Validation of immunoassay for protein biomarkers: bioanalytical study plan implementation to support pre-clinical and clinical studies

Biomarkers have emerged as an important tool to optimize the benefit/risk ratio of therapeutics. The scientific impact of biomarker studies is directly related to the quality of the underlying data. It is therefore important that guidance be established for validation of assays used to support drug development. This paper specifically focuses on validation of immunoassay for protein biomarker to support pre-clinical and clinical studies. Therapeutics (small- and macro-molecules) and their respective target/ligand are out of scope. This paper describes the implementation of a bioanalytical study plan for the validation of immunoassays to support decision-making biomarkers and biomarker selection during preclinical and clinical studies. It establishes the complete operating procedure as well as the parameters and their respective acceptance criteria and defines milestones and decision points to be followed during the assay validation that should result in high quality bioanalytical data in a limited timeframe and with reduced costs. The bioanalytical study plan can be applied to the validation of a wild range of immunoassay technology such as monoplex ELISA, automated analyzer, multiplex assays or cutting edge technology. Before any validation, a feasibility study is performed to assess the performance of the immunoassay using biological samples which should mimic the clinical population. The feasibility study addresses the likelihood that an assay will be able to achieve its intended purpose with parallelism being the most critical element (milestone 1). At the end of the feasibility study, a decision is taken to either continue with the validation or change the assay (milestone 2). The milestone 3 consists of the establishment of the nominal value of quality control to be used during the validation. The quality controls used to validate an assay should preferentially be prepared using neat (non-spiked) biological matrix (ideally derived from the specific trial population). The last milestone (milestone 4), the formal validation, includes demonstration of the assay performance meeting accuracy and precision acceptance criteria within (intra-run) and between (inter-run) validation runs for each QC sample. Validation also includes the assessment of stability of the protein biomarker in the biological matrix. It is recognized that the extent of the validation should be correlated to the intended use of the data and the assay acceptance criteria should take into consideration the study objective(s), nature of the methodology and the biological variability of the biomarker.     

Immunogenicity of therapeutics: a matter of efficacy and safety

Importance of the field: The unwanted immunogenicity of therapeutic proteins is a major concern regarding patient safety. Furthermore, pharmacokinetic, pharmacodynamic and clinical efficacy can be seriously affected by the immunogenicity of therapeutic proteins. Authorities have fully recognized this issue and demand appropriate and well-characterized assays to detect anti-drug antibodies (ADAs).

Areas covered in this review: We provide an overview of the immunogenicity topic in general, the regulatory background and insight into underlying immunological mechanisms and the limited ability to predict clinical immunogenicity a priori. Furthermore, we comment on the analytical testing approach and the status-quo of appropriate method validation.

What the reader will gain: The review provides insight regarding the analytical approach that is expected by regulatory authorities overseeing immunogenicity testing requirements. Additionally, the factors influencing immunogenicity are summarized and key references regarding immunogenicity testing approaches and method validation are discussed.

Take home message: The unwanted immunogenicity of protein therapeutics is of major concern because of its potential to affect patient safety and drug efficacy. Analytical testing is sophisticated and requires more than one assay. Because immunogenicity in humans is hardly predictable, assay development has to start in a timely fashion and for clinical studies immunogenicity assay validation is mandatory prior to analyzing patient serum samples. Regarding ADAs, the question remains as to when such antibodies are regarded of clinical relevance and what levels are, if at all, acceptable. In summary, the detection of ADAs should raise the awareness of the physician concerning patient safety and of the sponsor/manufacture concerning the immunogenic potential of the drug product.     

An Approach for Widening the Bioequivalence Acceptance Limits in the Case of Highly Variable Drugs

Purpose. Highly variable drugs pose a problem in bioequivalence assessment because they often fail to meet current regulatory acceptance criteria for average bioequivalence (80–125%). This paper examines alternative approaches to establishing bioequivalence. Methods. Suggested solutions have included alternate study designs, e.g., replicate and multiple dose studies, reducing the level of the confidence interval, and widening the acceptance limits. We focus on the latter approach. Results. A rationale is presented for defining wider acceptance limits for highly variable drugs. Two previously described methods are evaluated, and a new method having more desirable properties is proposed. Conclusions. We challenge the one size fits all current definition of bioequivalence acceptance limits for highly variable drugs, proposing alternative limits or goal posts which vary in accordance with the intrasubject variability of the reference product.     

Compatibility of arformoterol tartrate inhalation solution with three nebulized drugs

ABSTRACT

Objective: Arformoterol tartrate inhalation solution (15?μg/2?mL) is approved for the twice-daily, long-term maintenance treatment of bronchoconstriction in patients with chronic obstructive pulmonary diseases (COPD). This study assessed the chemical and physical compatibility of arformoterol (15?μg/2?mL) with ipratropium bromide (0.5?mg/2.5?mL), acetylcysteine (800?mg/4?mL), and budesonide (0.25?mg/2?mL and 0.5?mg/2?mL).

Methods: Immediately (T₀) and 30?min (T₃₀) after preparation, the admixtures were tested by visual inspection, pH measurement, and HPLC assay of each active component.

Results: For all admixtures, no visible signs of change were observed. The pH of all admixtures at T₀ ranged from 4.82 to 6.40, which was within the range of individual drugs. For all admixtures, no unacceptable changes (less than 1% or 0.1 pH unit) in the pH values were observed within 30?min compared with the initial pH values in the admixtures, which met acceptance criteria of not more than (NMT) 10.0%. At both T₀ and T₃₀, the assay of each active component in all admixtures ranged from 98.3% to 101.4% compared to the assay in control samples, which met acceptance criteria of NMT 10.0%. In addition, no changes (less than 8%) in the assay of each active component at T₃₀ were observed compared to the initial assay values, which met acceptance criteria of NMT 10.0%. This study did not evaluate the clinical efficacy or safety of mixing arformoterol in patients. Nor did the study assess the aerosol characteristics of these admixtures or any potential changes in drug output.

Conclusion: The results demonstrated that arformoterol was chemically and physically compatible with commercially available nebulized formulations of ipratropium bromide, acetylcysteine, and budesonide.     

Establishment and intra‐/inter‐laboratory validation of a standard protocol of reactive oxygen species assay for chemical photosafety evaluation

A reactive oxygen species (ROS) assay was previously developed for photosafety evaluation of pharmaceuticals, and the present multi‐center study aimed to establish and validate a standard protocol for ROS assay. In three participating laboratories, two standards and 42 coded chemicals, including 23 phototoxins and 19 nonphototoxic drugs/chemicals, were assessed by the ROS assay according to the standardized protocol. Most phototoxins tended to generate singlet oxygen and/or superoxide under UV–vis exposure, but nonphototoxic chemicals were less photoreactive. In the ROS assay on quinine (200 µm ), a typical phototoxic drug, the intra‐ and inter‐day precisions (coefficient of variation; CV) were found to be 1.5–7.4% and 1.7–9.3%, respectively. The inter‐laboratory CV for quinine averaged 15.4% for singlet oxygen and 17.0% for superoxide. The ROS assay on 42 coded chemicals (200 µm ) provided no false negative predictions upon previously defined criteria as compared with the in vitro/in vivo phototoxicity, although several false positives appeared. Outcomes from the validation study were indicative of satisfactory transferability, intra‐ and inter‐laboratory variability, and predictive capacity of the ROS assay. Copyright © 2012 John Wiley & Sons, Ltd.     

Adaptive control of theophylline therapy: Importance of blood sampling times

A two-observation protocol for estimating theophylline clearance during a constant-rate intravenous infusion is used to examine the importance of blood sampling schedules with regard to the information content of resulting concentration data. Guided by a theory for calculating maximally informative sample times, population simulations are used to assess the effect of specific sampling times on the precision of resulting clearance estimates and subsequent predictions of theophylline plasma concentrations. The simulations incorporated noise terms for intersubject variability, dosing errors, sample collection errors, and assay error. Clearance was estimated using Chiou 's method, least squares, and a Bayesian estimation procedure. The results of these simulations suggest that clinically significant estimation and prediction errors may result when using the above two-point protocol for estimating theophylline clearance if the time separating the two blood samples is less than one population mean elimination half-life.     

ICCVAM Evaluation of the Murine Local Lymph Node Assay: III. Data Analyses Completed by the National Toxicology Program Interagency Center for the Evaluation of Alternative Toxicological Methods

To evaluate the reliability of the murine local lymph node assay (LLNA), a test for allergic contact dermatitis activity, the inter- and intralaboratory consistency statistics (h and k, respectively) were calculated for validation studies testing multiple chemicals. The analysis indicated the absence of excessive variability in the dose calculated to induce a threefold or greater increase in the stimulation index (SI). To assess the appropriateness of using an SI of 3 as the decision criteria for identifying a sensitizing compound, LLNA results based on SI values of 2.0, 2.5, 3.0, 3.5, and 4.0 were compared with guinea pig or human results. The results supported the use of an SI of 3 as the decision criteria. Assay performance was determined by comparing LLNA results to results obtained for guinea pigs or humans. The accuracy of the LLNA was 89% when compared with results from the guinea pig maximization test (GPMT)/Buehler assay (BA). The performance of the LLNA and the GPMT/BA was similar when each was compared to human maximization test results plus substances included as human patch test allergens. The LLNA offered advantages over the GPMT in respect to both the time required to conduct the test and the assay cost.     

Protein–protein interaction modulator drug discovery: past efforts and future opportunities using a rich source of low- and high-throughput screening assays

Introduction: Historically, small-molecule drug discovery projects have largely focused on the G-protein-coupled receptor, ion-channel and enzyme target classes. More recently, there have been successes demonstrating that protein–protein interactions (PPIs) can be targeted by small-molecules and that this strategy has the potential to provide appropriate specificity and selectivity. However, a disadvantage is that compounds that modulate PPIs are often associated with relatively weak affinities as the targeted interaction surfaces are often relatively large. Moreover, from a small-molecule screening perspective, a large proportion of the initial screening Hits are often false positives and these need to be identified and excluded in order to focus on genuine modulators of the PPI being investigated.

Areas covered: The authors review previous efforts on PPI modulator drug discovery. Furthermore, they review assays that can be employed in small-molecule screening and/or Hit validation. The PPI assays are categorized as: i) low-throughput target-based biochemical assays, which are primarily employed for Hit validation at the post-screening stage; ii) high-throughput target-based biochemical assays that are suitable for screening campaigns; and iii) cell-based assays, which are suitable for high-throughput screening campaigns and/or Hit validation.

Expert opinion: Modulating the interaction of PPIs offers the potential to develop novel drugs to treat a wide range of diseases. New assay technologies are continually being developed and it is anticipated that these will be able to be directly used for small-molecule screening campaigns in the future.     

基于生物活性测定的注射用丹参多酚酸质量控制方法研究

目的 以注射用丹参多酚酸（SAFI）体外抗血小板聚集活性为基础,建立该制剂的生物活性测定方法,并对30批样品进行测定,从而探索一种反映其有效性的新质控方法。方法 以家兔全血为试验系,一定浓度的SAFI与富血小板血浆混合后,采用光学比浊法检测二磷酸腺苷（ADP）诱导的血小板聚集率,通过与丹酚酸B对照品的测定结果进行比较,运用斜率比例法计算相对效价,并进行方法学考察验证。结果 方法学验证结果表明,建立的体外抗血小板聚集生物活性测定法符合要求;30批次样品的相对效价均值为0.366 1,RSD为9.43%。结论 SAFI具有较强的抗血小板聚集活性,所建方法适用于SAFI体外抗血小板聚集活性测定,对于补充调整SAFI的全面质量控制具有重要意义。     

In Silico Prediction of Optimal in Vivo Delivery Properties Using Convolution-Based Model and Clinical Trial Simulation

Purpose. To develop a new strategy for the in silico evaluation of the optimal in vivo delivery properties of a drug, minimizing a cost function defined by the brain receptor occupancy obtained in positron-emission tomography experiments. Methods. A convolution-based model was formulated to link in vivo delivery rate to plasma concentrations whereas a second-stage model was used to link plasma concentrations to the pharmacodynamic effect. A feedback control approach was applied to identify the optimal in vivo delivery rate given an appropriate optimality criterion. Finally, clinical trial simulation was used as a supportive tool for decision-making by evaluating different scenarios accounting for pharmacokinetic/pharmacodynamic parameter uncertainty, inter-subject variability, and drug potency. Results. The results revealed that the mean in vivo delivery time significantly affects brain receptor occupancy whereas the fraction of the dose available for the systemic circulation shows the highest influence on brain receptor occupancy for a givenin vivo delivery rate. Finally, variability on receptor occupancy seems to be more affected by the inter-individual variability on the disposition PK parameters. Conclusion. The integration of convolution-based model, feedback control approach, and clinical trial simulation offers a unique tool for in ilico improvement of the drug development process by identifying critical issues on drug properties, optimal in vivo delivery rate, and potential problems related to the inter-individual variability.     

Quantification of CKD-501, lobeglitazone, in rat plasma using a liquid-chromatography/tandem mass spectrometry method and its applications to pharmacokinetic studies

CKD-501 (i.e., lobeglitazone), a potent agonist for both PPARα/γ, is a new drug that has potential clinical applications in the management of type-2 diabetes. The objective of this study was to develop a rapid and sensitive method for the determination of CKD-501 in rat plasma and to assess the applicability of the assay to pharmacokinetic studies. Rat plasma samples were processed using a fast flow protein precipitation (FF-PPT) method and then introduced onto an LC–MS/MS system for quantification. The analyte and rosiglitazone, an internal standard, were analyzed by multiple reactions monitoring (MRM) at m/z transitions of 482.0 → 258.0 for CKD-501 and 358.0 → 135.0 for the internal standard. The lower limit of quantification (LLOQ) was determined at 50 ng/mL, with an acceptable linearity in the range from 50 to 10,000 ng/mL (R > 0.999). Validation parameters such as accuracy, precision, dilution, recovery, matrix effect and stability were found to be within the acceptance criteria of the assay validation guidelines, indicating that the assay is applicable to estimating the concentration in the range studied. The concentration of CKD-501 was readily quantifiable in plasma samples up to 24 h post-dose in rats that had received an oral dose of 1 mg/kg. These observations suggest, therefore, that the validated assay can be used in pharmacokinetic studies of CKD-501 in small animals such as the rat.     

Discrimination Between Rival Dosing Histories

Purpose. In population pharmacokinetic studies, the dosing history is sometimes recorded in more than one way. The purpose of this study was to develop and evaluate a procedure for discriminating between rival dosing histories, i.e., for each individual in a data set, identify the dosing history that is the most plausible. Methods. The procedure consists of four steps. In the first step we identify individuals whose dosing histories produce predictions that are consistent. In the second step these individuals are used to build a population pharmacokinetic model which is used, in step three, to select the dosing history for the individuals not identified in step one. In step four the population model is refined using the best available dosing histories for all individuals. The proposed procedure was evaluated using both simulations and a real data set, in which two dosing histories, based on patient diaries and electronic monitoring devices (MEMS) were available. Results. In the real data set, estimated variabilities were almost always lower when the selected dosing histories were used compared to when no selection procedure was used. The diary dosing histories were selected more often than the MEMS dosing histories. In the simulations, the parameter estimates obtained using the selection procedure were closer to the true parameter values compared to when only one of the dosing histories was used. Conclusions. The proposed procedure appears to be robust and should be beneficial in at least two respects: improved parameter estimation of population pharmacokinetic and PK/PD models and objective information by which dosage recording methodologies can be compared and patient dose recording behavior can be assessed.     

Farmacocinética poblacional de doxorubicina aplicada a la personalización de su dosificación en pacientes oncológicos

ObjectiveTo develop and internally validate a population pharmacokinetic model for doxorubicin and to evaluate its predictive performance for dose individualization in cancer patients.MethodsDoxorubicin plasma concentrations were determined in thirty-three cancer patients treated with intravenous doxorubicin. A three-compartment pharmacokinetic model was implemented in the NONMEN VI programme to determine the doxorubicin pharmacokinetic parameters. The identifiability of the parameters was assessed by parametric bootstrap and model validation was performed using nonparametric bootstrap, visual predictive check, and numerical predictive check. The final model‘s predictive performance was evaluated in terms of accuracy and precision of plasma concentration predictions during the first and second cycles of chemotherapy.ResultsDoxorubicin clearance was 58.8L/h, with interpatient variability of 29.2% and intrapatient variability of 18.9%. The estimated volume of distribution at steady state was 2294L, with inter-and intrapatient variability of 7.3% and 26.1%, respectively. Internal validation confirmed that the population pharmacokinetic model is appropriate to describe the time course of the doxorubicin plasma concentrations and its variability in this population. The accuracy and precision of an a posteriori prediction of doxorubicin plasma concentrations improved by 63% and 41% compared to the a priori prediction.ConclusionThe Bayesian population pharmacokinetic model characterised the time course of doxorubicine plasma concentrations and can be accurately and precisely used to optimise doxorubicine dosing regimens in cancer patients.     

Evaluation of the Proposed FDA Pilot Dose-Response Methodology for Topical Corticosteroid Bioequivalence Testing

Purpose. The American FDA has recently released a Guidance document for topical corticosteroid bioequivalence testing. The purpose of this study was to evaluate the recommendations of this document for appropriateness. The new specifications require a dose-vasoconstriction response estimation by the use of a Minolta chromameter in a preliminary pilot study to determine the parameters for use in a pivotal bioequivalence study. Methods. The visually-assessed human skin blanching assay methodology routinely practiced in our laboratories was modified to comply with the requirements of the pilot study so that visual and chromameter data could be compared. Two different cream formulations, each containing 0.12% betamethasone 17-valerate, were used for this comparison. Results. Visual data showed the expected rank order of AUC values for most dose durations whereas the chromameter data did not show similar results. The expected rank order of AUC values for both chromameter and visual data was not observed at very short dose durations. In fitting the data to pharmacodynamic models, equivalent goodness of fit criteria were obtained when several different parameter estimates were used in the model definition, however the visual data were best described by the sigmoid E_max model while the chromameter data were best described by the simple E_max model. Conclusions. The E_max values predicted by the models were close to the observed values for both data sets and, in addition, excellent correlation between the AUC values and the maximum blanching response (R_max) (r > 0.95) was noted for both methods of assessment. The chromameter ED₅₀ values determined in this study were approximately 2 hours for both preparations. At this dose duration the instrument would not be sensitive enough to distinguish between weak blanching responses and normal skin for bioequivalence assessment purposes.     

用紫外导数分光光度法测定复方替米沙坦氨氯地平片的含量

目的 测定复方片中替米沙坦和氨氯地平的含量。方法 采用紫外导数分光光度法,替米沙坦测定波长为236 nm,氨氯地平测定波长为390 nm。结果 替米沙坦浓度在（4~20）×10^-3mg/ml范围内与一阶导数值有良好的线性关系,其标准曲线方程为Y=0.0043X-0.0005,R²=0.9993;氨氯地平浓度在（10~90）×10^-3 mg/ml范围内与一阶导数值有良好的线性关系,其标准曲线方程为Y=0.0003X+0.0002,R²=0.9995。结论 该方法具有较好的灵敏度、准确度和精密度;样品检测结果与HPLC法基本一致;是测定复方替米沙坦氨氯地平片含量的一种较好的分析方法。