Int J Pharm Pharm Sci, Vol 13, Issue 1, 20-22Original Article

A STUDY OF AGREEMENT BETWEEN WHO-UPPSALA MONITORING CENTRE CRITERIA, NARANJO ALGORITHM, AND LIVERPOOL ALGORITHM FOR CAUSALITY ASSESSMENT OF ADVERSE DRUG REACTIONS

MESSALINE SUNITHA1, SHOBHA PARVATHY2

Department of Pharmacology, Sree Gokulam Medical College and Research Foundation, Trivandrum, Kerala
Email: sobhaent@gmail.com

Received: 18 Sep 2020, Revised and Accepted: 02 Nov 2020

ABSTRACT

Objective: A standard causality assessment tool of an adverse drug reaction (ADR) is essential to compute the risk-benefit assessment of the medication taken by the patient and categorize its relationship likelihood. It should be reproducible and should not differ with the background and experience of the evaluator. Though there are a large number of causality assessment tools, none is unanimously accepted worldwide. So, this study was done to assess the agreement between three frequently used methods of causality assessment, the World Health Organisation-Uppsala Monitoring Centre (WHO-UMC) system, the Naranjo’s algorithm, and the Liverpool algorithm.

Methods: 172 ADR forms from the pharmacovigilance unit were randomly selected for the study. Causality assessment was done using three different methods, the WHO-UMC system, Naranjo’s algorithm, and the Liver pool algorithm. Cohen’s Kappa statistics was applied to look for agreement between the causality assessment methods.

Results: The agreement between the WHO-UMC criteria and Naranjo’s algorithm was the highest (136), with a Kappa value of 0.511, suggesting a moderate level of agreement. A maximum number of disagreements were noted between the WHO-UMC system and the Liverpool algorithm method (110).

Conclusion: A moderate agreement exists between the WHO-UMC system and the Naranjo algorithm. There is poor agreement between the Liverpool algorithm and the other two scales. Therefore, it is recommended that both the WHO-UMC system and the Naranjo algorithm be used for causality assessment of ADRs.

Keywords: ADR Causality Assessment, WHO-UMC criteria, Naranjo algorithm, Liverpool algorithm

© 2021 The Authors. Published by Innovare Academic Sciences Pvt Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/)
DOI: http://dx.doi.org/10.22159/ijpps.2021v13i1.39800. Journal homepage: https://innovareacademics.in/journals/index.php/ijpps.

INTRODUCTION

Causality assessment is needed to prove the relationship between the drug used and the adverse event following its use [1]. Adverse drug reactions (ADR) are recognized as the seventh recurrent cause of death [2] and it is also known that one in seven hospitalized patients develops an ADR [3]. The majority of the ADRs, though mild to moderate, need hospitalization and immediate treatment [4]. The various measurable risk factors for an ADR include polypharmacy, co-morbidities, length of stay at the hospital, age, previous history of ADRs and gender of the patient [5]. Therefore, a standard causality assessment tool is essential to compute the risk-benefit assessment of the medication taken and categorize the relationship likelihood. It should be reproducible in all circumstances and should not differ with the background and experience of the evaluator [6]. Methods of causality assessment can be broadly divided into expert judgment or global introspection, Bayesian methods and algorithms. Though there are a large number of causality assessment tools, none is unanimously accepted throughout the world [6]. Hence it is very essential to explore and analyze these methods to what degree they agree or differ.

The World Health Organisation-Uppsala monitoring center (WHO-UMC) causality assessment system was first introduced in 1994. It is a convenient tool for assessing individual case reports. But being a non-probabilistic method, it creates extensive unpredictability in evaluation. The Pharmacovigilance Program of India (PvPI) recommends the use of WHO-UMC criteria because of its simplicity and fast assessment properties [7]. Naranjo’s algorithm is preferred by many clinicians because it has a lower rate of inter-rater disagreement and uncertainty in ADR evaluation [8]. But, the main concern of Naranjo’s scale is its poor application in the pediatric age group and inclusion of queries regarding re-challenge and de-challenge, which makes it unethical and difficult. The exclusion of these queries will make the assessment output more limited to the maximum number of ADRs, turning to have a “possible” association [9]. Another causality assessment method which is more user friendly, devoid of uncertainty, and more suitable in children as well as in adults is the Liverpool algorithm. It is based on a robust binary decision where responses are redirected to precise queries rather than scores. This method gives more ADRs as “definite” than using Naranjo’s [10].

Though these three causality assessment tools are widely used, there is not enough exploration done to compare these tools. So, this study was done to assess the agreement between the three different methods of causality assessment scales, the WHO-UMC system, Naranjo’s algorithm, and Liverpool algorithm.

MATERIALS AND METHODS

The ADR CDSCO forms were obtained from the pharmacovigilance unit of the hospital. 172 ADR forms were randomly selected for the study. Causality assessment was done using three assessment criteria, the WHO-UMC system, Naranjo’s algorithm and the Liver pool algorithm. Cohen’s Kappa statistics was applied to look for agreement across the three causality assessment methods.

RESULTS

Among the 172 ADR forms collected, the mean age of the patients was found to be 48, with the minimum being 2 y and the maximum being 94. More number of females (108) reported with ADRs than males (64). 45% of ADRs were labeled as serious according to WHO criteria. The maximum number of ADRs was assessed to be “probable” by both the WHO-UMC system (123) and Naranjo’s (126). On the other hand, the Liverpool causality algorithm had more number of ADRs identified as “possible” (113). Overall agreement between the WHO-UMC system and Naranjo”s algorithm was the highest (136) with a Kappa value of 0.511, suggesting a moderate level of agreement. A maximum number of disagreements was noted between the WHO-UMC system and Liverpool algorithm methods (110).

Table 1: Category-wise distribution of ADRs using WHO-UMC system, Naranjo’s and Liverpool casualty assessment tools

WHO-UMC system No of ADRs (%) Naranjo’s No of ADRs (%) Liverpool algorithm No of ADRs (%)
Certain 13(7.6) Definite 9(5.2) Definite 8(5.8)
Probable 123(71.5) Probable 126(73.3) Probable 49(28.5)
Possible 35(20.3) Possible 37(21.5) Possible 113(65.7)
Unlikely 1(0.6) Doubtful 0 Unlikely 0
Unclassified 0
Unclassifiable 0

Table 2: Comparison of agreement between the three methods of causality assessment scales

Certain/definite Probable Possible Unlikely Total Kappa value (κ)
WHO UMC and Liverpool 10 32 21 0 63 .037
WHO-UMC and Naranjo 9 107 20 0 136 .511
Naranjo and Liverpool 9 39 27 0 75 .128

Kappa value (κ) was calculated using Cohen’s kappa statistics

Table 3: Number of disagreements across the three methods of causality assessment

Causality assessment method Total disagreements
WHO UMC and Liverpool 110
WHO-UMC and Naranjo 36
Naranjo and Liverpool 97

DISCUSSION

The majority of the ADR’s found by the WHO-UMC system is “Probable”(71.5%) category followed by Possible(20.3%), Certain(7.6%) and Unlikely(0.6%). A similar pattern of causality is seen with the Naranjo’s algorithm having 73.3% of ADRs in the “Probable” order, followed by Possible (21.5%) and the least in (5.2%) Certain. This parallelism in both the scales is in agreement with the earlier studies done by Tejas. A. Acharya et al. [11], Roy et al. [12] and Nitti Mittal et al. [13], where the probable category was the highest in both the tools. In contrast to this, many studies show that the “Possible” criteria to be prominent, as seen in Behelkar MN et al. [14] and Lei et al. [15]. This deviation can be due to the types of ADRs encountered in different hospital setups and also the subjectivity involved in analyzing the WHO-UMC system. Liverpool causality assessment results in this study were at variance from the other two tools used, with the highest number of ADRs falling in the “Possible” (113) category followed by Probable with 49 ADRs and 8 ADRs with Certain category. Though the maximum number of “Possible” aligns with a study done by Helene Theophile et al. [16], the same study had a high level of agreement between Naranjo’s and Liverpool’s, which is not evident in the current study.

Cohen’s kappa showed a moderate agreement between Naranjo’s and WHO_UMC system (.511). There is poor agreement between the other Causality tools. The moderate level agreement which was implicated in this study aligns with the studies done by Thaker SJ (kappa=0.69) [17] and Tejas. A. Acharya (kappa=0.60) [11]. But lower kappa values were found in other studies like Rehan et al. (kappa=0.214) [18], Belheker et al. [14] (kappa=0.145) and Rana et al. (kappa=0.014) [19]. The reason for this variability can be due to the high subjectivity, clinical knowledge, and experience of different evaluators. Other factors like drug re-challenge in the WHO-UMC system and subjectivity questions in Naranjo’s algorithm can be contributing to these different outcomes.

Assuming that a drug is the cause of an ADR, the decision is made whether to continue or stop the drug, which is a big drawback and can sometimes even endanger the patient’s treatment. In every patient, the outcome of a causality assessment algorithm can never replace a good clinical diagnosis. The clinicians must understand how the scales perform at assessing ADRs using to interpret a clinical diagnosis. Keeping in mind these practical issues, some areas need to be revised in the causality assessment methods.

However, our study is limited by the fact that only three ADR causality assessment scales were included and the number of ADRs used for analysis was also comparatively smaller. Further studies are required to establish an agreement between the different causality assessment methods.

CONCLUSION

We conclude that moderate agreement exists between the WHO-UMC causality system and the Naranjo algorithm. There is poor agreement between the Liverpool algorithm and the other two scales. Therefore, it is recommended that both the WHO-UMC causality system and the Naranjo algorithm be used for causality assessment of ADRs.

FUNDING

Nil

AUTHORS CONTRIBUTIONS

Messaline Sunitha MBBS MD: Conception and design of the study. Acquisition, analysis and interpretation of data. Drafting the article. Shobha Parvathy MBBS MD: Conception and design of the study. Acquisition, analysis and interpretation of data.

CONFLICT OF INTERESTS

Declared none

REFERENCES

  1. World Health Organisation (WHO), Uppsala monitoring center. The use of the WHO-UMC system for standardized case causality assessment. Available from: https://www.who.int/medicines/ areas/quality_safety/safety_efficacy/WHOcausality_assessment.pdf [Last accessed on10 Aug 2020].

  2. Wester K, Jonsson AK, Spigset O, Druid H, Hagg S. Incidence of fatal adverse drug reactions: a population-based study. Br J Clin Pharmacol 2008;65:573‐9.

  3. Davies EC, Rowe PH, James S. An investigation of disagreement in causality assessment of adverse drug reactions. Pharm Med 2011;25:17–24.

  4. Kulkarni GP, Patil LV. Analysis of adverse drug reactions spontaneously reported to adverse drug monitoring centre of a tertiary care hospital prospective study. Int J Curr Pharm Sci 2018;10:23-5.

  5. Mudigubba MK, Murthy MK, Swaroop AM, MN Dahiya S. A systematic review of risk factors of adverse drug reactions in hospitalized patients. Asian J Pharm Clin Res 2018.;11:25-9.

  6. Du W, Lehr VT, Lieh Lai M. An algorithm to detect adverse drug reactions in the neonatal intensive care unit. J Clin Pharmacol 2013;53:87‐95.

  7. Sharma S, Gupta AK, Reddy GJ. Inter-rater and intra-rater agreement in causality assessment of adverse drug reactions: a comparative study of WHO-UMC versus naranjo scale. Int J Res Med Sci 2017;5:4389-94.

  8. Martin J Doherty. Algorithms for assessing the probability of an adverse drug reaction. Respiratory Med 2009;2:63-7.

  9. Khan LM, Al-Harthi SE, Osman AM, Sattar MA, Ali AS. Dilemmas of the causality assessment tools in the diagnosis of adverse drug reactions. Saudi Pharm J 2016;24:485‐93.

  10. Gallagher RM, Mason JR, Bird KA. Adverse drug reactions causing admission to a pediatric hospital. PLoS One 2012;7:e50127.

  11. Acharya TA, Trivedi MD, Joshi KJ, Chhaiya SB, Mehta DS. A study of agreement between WHO-UMC causality assessment system and the naranjo algorithm for causality assessment of adverse drug reactions observed in medical ICU of a tertiary care teaching hospital. Biomed Pharmacol J 2020;13:79-83.

  12. Roy D, Purkayastha A, Tigga R. Analysis of adverse drug reaction in a tertiary care hospital: a retrospective study. Asian J Pharm Clin Res 2017;10:347-9.

  13. Mittal N, Gupta MC. Comparison of agreement and rational uses of the WHO and Naranjo adverse event causality assessment tools. J Pharmacol Pharmacother 2015;6:91-3.

  14. Belhekar MN, Taur SR, Munshi RP. A study of agreement between the naranjo algorithm and WHO-UMC criteria for causality assessment of adverse drug reactions. Indian J Pharmacol 2014;46:117‐20.

  15. Lei HS, Rahman AF, Haq AS. Adverse drug reaction reports in Malaysia: comparison of causality assessments. Malays J Pharm Sci 2007;5:7–17.

  16. Theophile H, Andre M, Miremont Salame G, Arimone Y, Begaud B. Comparison of three methods (an updated logistic probabilistic method, the naranjo and liverpool algorithms) for the evaluation of routine pharmacovigilance case reports using consensual expert judgment as reference. Drug Saf 2013;36:1033‐44.

  17. Thaker SJ, Sinha RS, Gogtay NJ, Thatte UM. Evaluation of inter-rater agreement between three causality assessment methods used in pharmacovigilance. J Pharmacol Pharmacother 2016;7:31-3.

  18. Rehan HS, Chopra D, Kakkar AK. Causality assessment of spontaneously reported adverse drug events: comparison of WHO-UMC criteria and naranjo probability scale. Int J Risk Saf Med 2007;19:223-7.

  19. Rana DA, Bhadiyadara SN, Shah HJ, Malhotra SD, Patel VJ. Consistency between causality assessments obtained with various scales and their agreement for adverse drug events reported in the pediatric population. J Young Pharm 2015;7:89-95.