Int J Pharm Pharm Sci, Vol 9, Issue 6, 80-84Original Article

 

SAFETY SIGNAL DETECTION OF CARDIAC DISORDERS ADVERSE DRUG EVENTS FOR AZITHROMYCIN IN PEDIATRIC POPULATION USING HEALTH CANADA ADVERSE EVENT REPORTING SYSTEM DATABASE

GAURAV KUMAR SHAHa*, MUKESH KUMAR PATELb, BHANWARLAL JATc

aBhagwant University, Sikar Road, Ajmer, Rajashthan, India, bB. M. Shah College of Pharmaceutical Education and Research, Modasa, Gujarat, India, cDepartment of Agricultural Biotechnology, Bhagwant University, Ajmer, Rajasthan, India
Email: rxgaurav@gmail.com

Received: 20 Feb 2017 Revised and Accepted: 20 Apr 2017

ABSTRACT

Objective: We conducted signal detection of adverse drug events reported in Health Canada adverse event reporting system database “MedEffect” for azithromycin, a macrolide derivative and the first azalide antimicrobial agent to review the cardiac disorders adverse drug events (ADEs) in pediatric population with the drug labels of selected countries including India, USA, UK, Canada, Switzerland, Australia, New Zealand.

Methods: We extracted data between January 1965 and June 2016 from the Canada adverse event reporting system database “MedEffect”. Frequentist and Bayesian methods were used to calculate disproportionality distribution of drug-adverse event (AE) pairs. The AE which was detected by all the three indices of proportional reporting ratio (PRR), reporting odds ratio (ROR), and information component (IC) was identified as a potential signal. AE reports for azithromycin, among which 3651 reports were attributed to paediatrics.

Results: The signal detected by PRR and ROR for tachycardia associated with azithromycin were found to be 1.3 and for cardiovascular disorder were 1.2. The IC for azithromycin by a Bayesian method was 0.3 for both, tachycardia and cardiovascular disorder. Both AEs of cardiovascular disorder and tachycardia were detected as potential signals of azithromycin for the paediatric population. Comparing drug labels of 7 countries in paediatric population, both adverse events were not listed on any of the labels of seven countries against the pediatric population.

Conclusion: We detected 2 new potential signals of azithromycin which were not listed on the labels of 7 countries. Therefore, it should be accompanied by a signal evaluation including causal association, clinical significance, and preventability.

Keywords: Signal detection, Drug Safety, Pediatrics, Medeffect, Pharmacovigilance, Azithromycin

© 2017 The Authors. Published by Innovare Academic Sciences Pvt Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)
DOI: http://dx.doi.org/10.22159/ijpps.2017v9i6.18020

INTRODUCTION

Azithromycin, a macrolide derivative, is the first, 15-membered ring azalide class of antimicrobials. The spectrum of activity for azithromycin includes atypical pathogens besides gram-positive and gram-negative organisms [1]. Azithromycin was approved by the US Food and Drug Administration (FDA) in late 1991, as the first once-daily antibiotic with a 5-day course of therapy. Azithromycin is widely indicated for a wide range of bacterial infections, including upper and lower respiratory infections, skin and soft-tissue infections, as well as some sexually transmitted infections. The FDA has approved azithromycin for the following indications in children: acute otitis media, acute bacterial sinusitis, community-acquired pneumonia, pharyngitis or tonsillitis [2]. For respiratory tract infections, it has better coverage of atypical organisms, including mycoplasma and legionella [3].

Azithromycin is generally well tolerated; relatively common adverse events (AEs) reported in 1% to 5% of patients include gastrointestinal upset, headache and dizziness. Transient increases in transaminases have also been reported in 1.5% of patients [4]. Although azithromycin has been shown to be relatively free of cardiotoxic effects, cardiac events, including QT prolongation and cardiac arrhythmias associated with azithromycin use is common [5, 6]. In 2012, the US FDA issued a warning to consider fatal heart rhythms associated with azithromycin use in the select population [7]. The warning indicated that people with pre-existing conditions are at particular risks, such as those with QT interval prolongation, low blood levels of potassium or magnesium, a slower than normal heart rate or those who use certain drugs to treat abnormal heart rhythms [8].

Pediatric population is always at high risk of experiencing AE [9]. It is the responsibility of product license holders and regulatory authorities to monitor the safety of licensed drugs to detect adverse reactions that are unexpected in nature and impact risk to benefit ratio of the drug. By performing routine pharmacovigilance activities, the identification and evaluation of safety signals can be done which eventually identifies risk. Risk minimization and mitigation plan is required for all potentially identified risk which can be termed as a signal as well. Formally, a signal is defined as “Reported information on a possible causal relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously” [10].

Number of spontaneous reports of suspected adverse drug events or ADEs are submitted by consumers, medical and health care professionals [11], and pharmaceutical companies (also known as market authorization holders), to the US FDA on an ongoing basis. Each such report includes one or more ADEs suspected to be associated with the administration of a drug for identified indications. Over a period of time, such reports constitute a large database at US FDA. Like US FDA, many regulatory agencies collect and maintain data that provide safety information for its regulated products which assist in the evaluation of many important drug safety signals. Systematic data-mining techniques—commonly known as signal detection algorithms—when applied to such large data sets, supplement the traditional expert review of the reports and facilitate expedited analysis of large volume of data to obtain meaningful inferences [12]. Ongoing quantitative evaluation of such data mining via signal detection algorithms has been recognized as an important tool in pharmacovigilance [12, 13]. We aimed to explore potential safety signal of cardiac disorders ADEs associated with azithromycin use in pediatric population using signal detection algorithm for Canadian adverse reaction monitoring program database.

MATERIALS AND METHODS

Database and study drug

The Canadian vigilance adverse reaction online database Med Effect Canada was developed by the Health Canada Health Agency to make it easier for consumers, patients, and health professionals to report adverse reactions and side effects, obtain new safety information on drugs and other health products, and learn and better understand the importance of reported side effects [14]. The Canada vigilance program has collected reports of suspected adverse reactions since 1965. Adverse reaction reports are submitted by health professionals and consumers on a voluntary basis either directly to Health Canada or via market authorization holders. The computerized ADE reporting system includes voluntary reporting by health care workers, the general public, and mandatory reporting by manufacturers for serious and unexpected events requiring a standardized form.

We retrieved reported cases of spontaneous reporting of ADEs for azithromycin from the Med Effect Canada database. ADEs were coded according to the adverse reaction term and system organ class (SOC) of the Medical Dictionary for Regulatory Activities (MedDRA) v 19.0. Cardiac disorders SOC cases of azithromycin in the pediatric population (0 to 18years) were chosen as the cases of interest for our study drug and all the other cardiac disorders SOC cases of the pediatric population cases were used for comparison.

Statistical analysis

We used frequentist approaches of signal detection methods for calculation of the proportional reporting ratio (PRR) and the report odds ratio (ROR) [15]. Bayesian-based approaches include Bayesian confidence propagation neural network (BCPNN) [6] is used to calculate information component [15, 16].

For the frequentist approach, we produced a 2×2 contingency table by defining as a row with azithromycin and all other antibiotics, a column with specific ADE and all other ADEs (table 1) [15].

Table 1: 2×2 contingency table for the ADE-drug pair

ADE with the drug of interest ADE of all other drug in database
Reports of ADE of Interest a b
Total Reports of ADE in database c d

ADE = adverse drug event. Whereas: a =No of events of interest with the drug of interest, b = No of events of interest with other drugs in the database, c =No of events with the drug of interest, d =Total no of events with all drugs.

Proportion reporting rate (PRR), the ratio of reporting rate of one specific AE among all events for a given drug, was calculated using the following formula; the comparator being reporting rate for all drugs in the database including the drug of interest [15].

Whereas: a =No of events of interest with the drug of interest, b = No of events of interest with other drugs in the database, c =No of events with the drug of interest, d =Total no of events with all drugs.

Reporting odds ratio (ROR), defined as the ratio of the odds of reporting one specific AE versus all other events for a given drug compared to this reporting odds for all other drugs in the database. ROR was calculated using the following formula [15].

Whereas: a =No of events of interest with drug of interest, b = No of events of interest with other drugs in the database, c =No of events with drug of interest, d =Total no of events with all drugs

The BCPNN method measures the association between a drug and an ADE by the IC defined as the logarithm of the ratio of the observed rate of a specific drug-AE combination to the expected rate of AE under the null hypothesis of no association between the drug and the event [16]. Thus, when a drug-AE combination is reported more often than expected relative to general reporting of the drugs and the AEs it results into positive values of IC.

Whereas: a =No of events of interest with drug of interest, b = No of events of interest with other drugs in the database, c =No of events with drug of interest, d =Total no of events with all drugs.

A signal is considered when the lower limit of the 95% confidence interval (CI) of the ROR is greater than one. The ADE which was detected by all the three indices PRR, ROR, and IC was defined as a signal in this study [15, 16].

Table 2: Characteristics of ADE for azithromycin from January 1965 till June 2016

Characteristic Pediatric cases Adult cases Total
n % n % n %
Gender
Male 2803 76.77 5870 46.25 8673 53.07
Female 689 18.87 6670 52.56 7359 45.03
Unknown 159 4.35 151 1.19 310 1.90
Report Type
Spontaneous 3310 90.7 5903 46.5 9213 56.4
Published 281 7.7 1396 11.0 1677 10.3
Other 60 1.6 0 0.0 60 0.4
Special Access Program 0 0.0 121 1.0 121 0.7
Study 0 0.0 5271 41.5 5271 32.3
Report source by person
Consumer or other non-health professional 185 5.1 1147 9.0 1332 8.2
Other health professional 425 11.6 1798 14.2 2223 13.6
Pharmacist 2622 71.8 3327 26.2 5949 36.4
Physician 359 9.8 2320 18.3 2679 16.4
Other 60 1.6 4099 32.3 4159 25.4
Report Outcome
Death 21 0.6 1602 13 1623 10
Not recovered/not resolved 2476 69.7 2123 17 4599 28
Recovered/resolved 253 7.1 2741 21 2994 18
Recovered/resolved with sequelae 91 2.6 45 0 136 1
Recovering/resolving 16 0.5 2893 23 2909 18
Unknown 695 19.6 3386 26 4081 25

The signals detected in the MedEffect Canada database were compared with drug label information issued by the marketing authorization of India and 6 other countries: USA, UK, Canada, Switzerland, Australia, New Zealand. All statistical analyses were performed by SAS 9.4 (SAS Institute Inc., Cary, NC, USA) and Microsoft Excel 2010.

RESULTS

In the MedEffect Canada database, a total of 16,342 reports of cardiac disorders system organ class were accumulated from January 1965 till June 2016 in which 3651 were for the age group of 0 to 18 y whereas 12691 were of 18+years. The characteristics of ADE reports of azithromycin are presented in table 2.

Among the pediatric reports of 3651, reports attributed to azithromycin, 76.8% (2803) were from male, and 18.9% (689) were from female whereas among the adult report of 12691, reports attributed to azithromycin, 46.3% (5870) were from male, and 52.6% (6670) were from female. The majority of the reports in pediatric (90.7%) and almost half of them in adults (46.5%) were reported via spontaneous reporting. The most frequent source of reports were doctors (505, 29.3%); and the affiliation for such source was regional pharmacovigilance centre (1,009, 58.6%), or a pharmaceutical company (599, 34.8%).

Regarding the AE outcome amongst the pediatric report of 3651, 69.7% (2476) AEs were not recovered or did not resolve, 19.6% (695) were with unknown outcome, 7.1% (253) events has got recovered or did not resolve, 2.6% (91) got recovered or resolved with sequels, 0.6%(21) events were fatal whereas 0.5% (16) events were recovering at the time of reporting.

In total 16,342 cases reported for azithromycin (78% adults and 22% for pediatric population), 3% (544) cardiovascular disorder and 1% (1623) fatal cases were reported. Among the cardiovascular disorder cases, the proportion of cases in adult and the pediatric population was similar (3% in both the populations. In pediatric population (3629 cases), total 21 fatal cases were reported, of which 57% (12 cases) were attributed to cardiovascular disease. Whereas, for the adult population, (12713 cases), the proportion of fatal cases attributed to cardiovascular disease was 6% (89 cases of the total 1602 fatal cases in adults).

Table 3 illustrates details of cardiac disorders events for all the cases with azithromycin. Overall, the most frequent events in total 544 reported cases of cardiac disorders for azithromycin included tachycardia (133 cases, 24%), Torsade pointes (90 cases, 16.5%), cardiovascular disorder (74 cases, 13.6%) and cardiac failure (56 cases, 10%). Tachycardia and cardiovascular disorder occurred at a higher frequency in pediatrics than adult population (tachycardia: 44% in pediatrics and 20% in adults; cardiovascular disorder: 35% in pediatrics and 8% in adults). The frequency of fatal events attributed to cardiac disorder system order class was 57% in pediatrics vs 6% in the adult population. In the pediatric population, the frequency of fatal events attributed to cardiac disorder system order class was higher than any other system organ class.

Few of the cardiac disorder events observed in the pediatric population were in higher frequency as compared to those reported in adults. The cardiovascular disorder was 35.2% of total pediatric events as compared to 8.4% of adult events (table 4). ADEs cases of tachycardia were highest in pediatrics was 41.9% (44) which is higher than in adult 20.3% (89).

Table 3: Overview of cardiac disorder ADE due to azithromycin

MedEffect %
Total no of cases 16342 100
Cases of Pedia (<18) 3629 22
Cases of 18+ 12713 78
Total no of CD cases 544 100
Cases of CD in Pedia (<18) 105 24
Cases of CD in 18+ 439 76
Fatal 1623 100
Cases of Pedia (<18) 21 1
Cases of 18+ 1602 99
CD Fatal 107 100
Fatal Cases of CD in Pedia (<18) 12 11
Fatal Cases of CD in 18+ 89 83

ADE = adverse drug event; CD = cardiac disorder.

Table 4: Details of cardiac disorder ADEs in pediatrics due to azithromycin

PT level event No of events in pediatrics % No of events in adult %
Arrhythmia 3 3 14 3
Bradycardia 8 8 10 2
Cardiac arrest 3 3 27 6
Cardiovascular disorder 37 35 37 8
Cyanosis 2 2 2 1
Mitral valve incompetence 2 2 2 1
Supraventricular tachycardia 3 3 6 1
Tachycardia 44 42 89 20
Ventricular fibrillation 3 3 3 1
Cardiac disorder 0 0 7 2
Cardiac failure 0 0 56 13
Cardiac failure congestive 0 0 3 1
Cyanosis 0 0 2 1
Long QT syndrome 0 0 14 3
Mitral valve incompetence 0 0 2 1
Myocardial infarction 0 0 12 3
Myocarditis 0 0 5 1
Palpitations 0 0 19 4
Supraventricular tachycardia 0 0 6 1
Torsade de pointes 0 0 90 21
Ventricular extrasystoles 0 0 12 3
Ventricular tachycardia 0 0 8 2
Angina pectoris 0 0 6 1
Atrioventricular block second degree 0 0 7 2
Total 105 100 439 100

ADE = adverse drug event.

The relevant details for quantitative calculations for signal detection (PRR, ROR, and IC) for our data are presented in table 5.

Table 5: Signal detection: cardiovascular events and azithromycin use 2×2 contingency table for tachycardia as an event of interest

AE with the drug of interest AE of all another drug in database
Reports of AE of Interest 44 7032
Total Reports of AE in database 2422 504150
 

Table 6: (a) 2×2 contingency table for cardiovascular disorder as an event of interest

AE with the drug of interest AE of all another drug in database
Reports of AE of Interest 37 7032
Total Reports of AE in database 2182 504150
 

Table 6: (b) Signal detection

AE PRR ORR IC0
Tachycardia 1.3 1.3 0.3
Cardiovascular disorder 1.2 1.2 0.3

The signal detected by PRR and ROR for tachycardia associated with azithromycin were found to be 1.3 and for cardiovascular disorder were 1.2. The IC by BCPNN for azithromycin was0.3 for both tachycardia and cardiovascular disorder. By the three data mining indices for PRR, ROR and IC, cardiovascular disorder and tachycardia can be considered as a potential signal for azithromycin.

Comparison of the information on potential cardiac disorder signals in azithromycin labels across 7 countries is presented in table 7.

Table 7: Information on potential cardiac disorder signals in azithromycin labels across select countries

AEs India USA UK Canada Australia New zealand Switzerland
Adult Population
Cardiovascular disorder N N N Y Y Y N
Tachycardia N N N Y Y Y N
Pediatric population
Cardiovascular disorder N N N N N N N
Tachycardia N N N N N N N

Y = adverse event is listed for the label; N = adverse event is not listed for the event, Both tachycardia and cardiac disorders are not listed on any of the drug labels among these 7 countries for pediatric population use; whereas only 3 out of 7 countries listed them for adult population use.

DISCUSSION

The importance of post-marketing pharmacovigilance process is accentuated because adverse events cannot be fully detected during the clinical trial phase. However, a signal is described as reported information on a possible causal relationship between an ADE and a drug, the relationship being unknown or incompletely documented, which is considered necessary to evaluate further. A considerable percentage of drugs is withdrawn from the market, or their use is curtailed by black box warnings, due to some adverse effects that were not discovered during preclinical and clinical assays [17].

Pharmacokinetics and pharmacodynamics of many commonly used drugs vary significantly between different age groups of patients and pediatric population is always at high risk of experiencing ADEs [18]. Adverse drug reactions in children result in relatively more severe effect when compared with adults. AEs can lead to significant morbidity among children. It has been found that AEs in children not only results in hospitalization or prolonged hospitalization but also may lead to permanent disability or even death [19].

Based on our study, tachycardia and cardiac disorders emerged as a potential ADE for azithromycin in the pediatric population. High proportionate of fatal events in the pediatric population (12 fatal events against a total number of 21 which 57.14% of total events) indicate the severity of events. This was further supported by the results observed for all the three indices PRR, ROR, and IC for tachycardia and cardiovascular disorder events, both events can be confirmed as a potential signal. This observation is of great importance given the fact that regulatory health authority has issued letters to consider Torsade de pointes, arrhythmia including ventricular tachycardia as uncommon (≥1/1,000 to<1/100) event under cardiac disorder for adults.

More importantly, the potential safety signal associated with azithromycin (tachycardia and cardiovascular disorder events) has been not listed on drug labels from the labels of the 7 countries for the pediatric population. Nonexistence of tachycardia and cardiovascular disorder events for the pediatric population in labels from all 7 countries and the same events for adults in 4 countries indicates a lack of awareness of the severity of the events to marketing authorization holder as well as regulatory authorities. Our finding suggests that due to the higher percentage of ADEs including fatal events in cardiac disorder for the pediatric population, tachycardia and cardiovascular disorder should be treated as common (≥1/1,00 to<1/100) event and common events.

LIMITATION

It would be important to take into account a few inherent limitations while interpreting the results of this study. Firstly, our findings are based on the database managed by the regulatory authority which lacks information about causality assessment of the drugs other than study drug for the detected AE. We focused only on the cardiac disorder. Possibly, other system organ class might also have unidentified AEs which would have been missed. Lastly, because of the use of data mining approach, we did not examine the onset of signals relative to the time an ADE is confirmed or distinguish ADEs associated with acute and chronic drug use. Although it is hard to accurately determine the confirmed ADE, studies have attempted to provide preliminary analysis. Such additional analysis would have strengthened the findings [20, 21]. Further, while we calculated the three indices PRR, ROR, and IC for safety signal, it may be instructive to compare the findings presented herein with other methods. For example, receiver operating characteristic analysis and other approaches such as those that focus on positive predictive value or false discovery rates would, if performed, complement our analysis [20, 22, 23].

CONCLUSION

Findings from this research suggest that risk of cardiac disorder ADE is more in the pediatric population as compared with adults and ADE of tachycardia and cardiovascular disorder are potential safety signal of azithromycin for the pediatric population. Marketing authorization holders and regulatory authorities should closely monitor the risk of tachycardia and cardiovascular disorder by including both events in the product label.

CONTRIBUTION

GS conceptualised and designed the study, collected the data and performed statistical analysis, interpreted the findings and drafted the content of paper. MP contributed in study design and statistical analysis, provided inputs for data interpretation. BJ supported statistical analysis, helped in data interpretation and writing of content. GS, MP, and BJ finalized the content and agreed to submit it in present form for publication.

CONFLICTS OF INTERESTS

All authors have none to declare forum

REFERENCES

  1. Williams JD. Spectrum of activity of azithromycin. Eur J Clin Microbiol Infect Dis 1991;10:813-20.

  2. AZITHROMYCIN label. Available from: http://www.accessdata. fda.gov/spl/data/2e0577d5-adb9-6c8d-e054-00144ff8d46c/2e0577d5-adb9-6c8d-e054-00144ff8d46c.xml. [Last accessed on 18 Feb 2017].

  3. Camilleri M, Parkman HP, Shafi MA, Abell TL, Gerson L. Clinical guideline: management of gastroparesis. Am J Gastroenterol 2013;108:18-37.

  4. Zuckerman JM, Qamar F, Bono BR. Macrolides, ketolides, and glycylcyclines: azithromycin, clarithromycin, telithromycin, tigecycline. Infect Dis Clin North A 2009;23:997-1026, ix-x.

  5. Matsunaga N, Oki Y, Prigollini A. A case of QT-interval prolongation precipitated by azithromycin. N Z Med J 2003; 116:U666.

  6. Huang BH, Wu CH, Hsia CP, Yin Chen C. Azithromycin‐induced torsade de points. Pacing Clin Electrophysiol 2007;30:1579-82.

  7. FDA Drug Safety Communication [AZITHROMYCIN]. Available from: https://www.fda.gov/Drugs/DrugSafety/ucm341822.htm. [Last accessed on 18 Feb 2017].

  8. Ray WA, Murray KT, Hall K, Arbogast PG, Stein CM. Azithromycin and the risk of cardiovascular death. N Engl J Med 2012;366:1881-90.

  9. Balaji V, Sivaraj R, Nirmala P. Comparative study of safety and efficacy of azithromycin and amoxicillin in treating children with lower respiratory tract infection. Int J Curr Pharm Res 2015;7:56-7.

  10. WHO Guideline: Setting Up and Running a Pharmacovigilance Centre. Available from: http://apps.who.int/medicinedocs/ en/d/Jh2934e/14.html. [Last accessed on 18 Feb 2017]

  11. Ramesh M, Parthasarathi G. Adverse drug reactions reporting: attitudes and perceptions of medical practitioners. Asian J Pharm Clin Res 2009;2:10-4.

  12. Harpaz R, DuMouchel W, LePendu P, Bauer‐Mehren A, Ryan P, Shah NH. The performance of pharmacovigilance signal‐detection algorithms for the FDA adverse event reporting system. Clin Pharmacol Ther 2013;93:539-46.

  13. Sakaeda T, Kadoyama K, Okuno Y. Adverse event profiles of platinum agents: data mining of the public version of the FDA adverse event reporting system, AERS, and reproducibility of clinical observations. Int J Med Sci 2011;8:487-91.

  14. MedEffect Canada. Available from: http://www.hc-sc.gc.ca/ dhp-mps/medeff/databasdon/index-eng.php. [Last accessed on 18 Feb 2017]

  15. Evans SJ, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 2001;10:483-6.

  16. Kajungu D, Speybroeck N. Implementation of signal detection methods in pharmacovigilance–a case for their application to safety data from developing countries. Ann Biom Biostat 2015;2:1017.

  17. Dykens JA, Will Y. The significance of mitochondrial toxicity testing in drug development. Drug Discovery Today 2007;12:777-85.

  18. Alomar MJ. Factors affecting the development of adverse drug reactions (Review article). Saudi Pharm J 2014;22:83-94.

  19. Priyadharsini R, Surendiran A, Adithan C, Sreenivasan S, Sahoo FK. A study of adverse drug reactions in pediatric patients. J Pharmacol Pharmacother 2011;2:277-80.

  20. Alvarez Y, Hidalgo A, Maignen F, Slattery J. Validation of statistical signal detection procedures in Eudra vigilance post-authorization data. Drug Saf 2010;33:475-87.

  21. Hochberg AM, Reisinger SJ, Pearson RK, O'Hara DJ, Hall K. Using data mining to predict safety actions from FDA adverse event reporting system data. Drug Inf J 2007;41:633-43.

  22. Almenoff JS, LaCroix KK, Yuen NA, Fram D, DuMouchel W. Comparative performance of two quantitative safety signalling methods. Drug Saf 2006;29:875-87.

  23. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837-45.

How to cite this article