Healthc Inform Res.  2020 Apr;26(2):104-111. 10.4258/hir.2020.26.2.104.

Analysis of Adverse Drug Reactions Identified in Nursing Notes Using Reinforcement Learning

• Jeon E
• Kim Y
• Park H
• Park RW
• Shin H
• Park HA

• KMID: 2501335
• DOI: http://doi.org/10.4258/hir.2020.26.2.104

Abstract

Electronic Health Records (EHRs)-based surveillance systems are being actively developed for detecting adverse drug reactions (ADRs), but this is being hindered by the difficulty of extracting data from unstructured records. This study performed the analysis of ADRs from nursing notes for drug safety surveillance using the temporal difference method in reinforcement learning (TD learning).

Nursing notes of 8,316 patients (4,158 ADR and 4,158 non-ADR cases) admitted to Ajou University Hospital were used for the ADR classification task. A TD(λ) model was used to estimate state values for indicating the ADR risk. For the TD learning, each nursing phrase was encoded into one of seven states, and the state values estimated during training were employed for the subsequent testing phase. We applied logistic regression to the state values from the TD(λ) model for the classification task.

The overall accuracy of TD-based logistic regression of 0.63 was comparable to that of two machine-learning methods (0.64 for a naïve Bayes classifier and 0.63 for a support vector machine), while it outperformed two deep learning-based methods (0.58 for a text convolutional neural network and 0.61 for a long short-term memory neural network). Most importantly, it was found that the TD-based method can estimate state values according to the context of nursing phrases.

TD learning is a promising approach because it can exploit contextual, time-dependent aspects of the available data and provide an analysis of the severity of ADRs in a fully incremental manner.

Keyword

Drug-Related Side Effects and Adverse Reactions; Electronic Health Records; Machine Learning; Deep Learning; Nursing Records

Figure

• 

  Figure 1 Our proposed model as two separable processes: (A) TD learning process of state values for the seven predefined states and (B) the entire procedure of our classification method. ADR: adverse drug reaction, TD: temporal difference, CNN: convolutional neural network.

• 

  Figure 2 The general process of reward shaping.

• 

  Figure 3 The concrete results obtained from the temporal difference-based predictions in Table 3.

Reference

References

1. Jensen PB, Jensen LJ, Brunak S. Mining electronic health records: towards better research applications and clinical care. Nat Rev Genet. 2012; 13(6):395–405.

2. Oliveira JL, Lopes P, Nunes T, Campos D, Boyer S, Ahlberg E, et al. The EU-ADR Web Platform: delivering advanced pharmacovigilance tools. Pharmacoepidemiol Drug Saf. 2013; 22(5):459–67.

3. Kho AN, Rasmussen LV, Connolly JJ, Peissig PL, Starren J, Hakonarson H, et al. Practical challenges in integrating genomic data into the electronic health record. Genet Med. 2013; 15(10):772–8.

4. Ahn HJ, Park HA. Adverse-drug-event surveillance using narrative nursing records in electronic nursing records. Comput Inform Nurs. 2013; 31(1):45–51.

5. Lee S, Choi J, Kim HS, Kim GJ, Lee KH, Park CH, et al. Standard-based comprehensive detection of adverse drug reaction signals from nursing statements and laboratory results in electronic health records. J Am Med Inform Assoc. 2017; 24(4):697–708.

6. Park MY, Yoon D, Lee K, Kang SY, Park I, Lee SH, et al. A novel algorithm for detection of adverse drug reaction signals using a hospital electronic medical record database. Pharmacoepidemiol Drug Saf. 2011; 20(6):598–607.

7. Jagannatha AN, Yu H. Structured prediction models for RNN based sequence labeling in clinical text. Proc Conf Empir Methods Nat Lang Process. 2016; 2016:856–65.

8. Hughes M, Li I, Kotoulas S, Suzumura T. Medical text classification using convolutional neural networks. Stud Health Technol Inform. 2017; 235:246–50.

9. Sutton RS. Temporal credit assignment in reinforcement learning [dissertation]. Amherst, MA: University of Massachusetts;Amherst: 1984.

10. Van Seijen H, Mahmood AR, Pilarski PM, Machado MC, Sutton RS. True online temporal-difference learning. J Mach Learn Res. 2016; 17(1):5057–96.

11. Kim Y. Convolutional neural networks for sentence classification. In : Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP); 2014 Oct 25–29; Doha, Qatar. p. 1746–51.

12. Ng AY, Harada D, Russell S. Policy invariance under reward transformations: theory and application to reward shaping. In : Proceedings of the Sixteenth International Conference on Machine Learning (ICML); 1999 Jun 27–30; Bled, Slovenia. p. 278–87.

13. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997; 9(8):1735–80.

14. Trinh TH, Dai AM, Luong MT, Le QV. Learning longer-term dependencies in RNNs with auxiliary losses. In : Proceedings of the 35th International Conference on Machine Learning (ICML); 2018 Jul 10–15; Stockholm, Sweden. p. 4972–81.

15. Bahdanau D, Brakel P, Xu K, Goyal A, Lowe R, Pineau J, et al. An actor-critic algorithm for sequence prediction. In : Proceedings of the 5th International Conference on Learning Representations (ICLR); 2017 Apr 24–26; Toulon, France.

16. Lopez-Gonzalez E, Herdeiro MT, Figueiras A. Determinants of under-reporting of adverse drug reactions: a systematic review. Drug Saf. 2009; 32(1):19–31.