Healthc Inform Res.  2023 Jul;29(3):218-227. 10.4258/hir.2023.29.3.218.

Simulation Method for the Physical Deformation of a Three-Dimensional Soft Body in Augmented Reality-Based External Ventricular Drainage

• Koo K
• Park T
• Jeong H
• Khang S
• Koh CS
• Park M
• Kim MJ
• Jung HH
• Shin J
• Kim KW
• Lee J

• KMID: 2544873
• DOI: http://doi.org/10.4258/hir.2023.29.3.218

Abstract

Objectives
Intraoperative navigation reduces the risk of major complications and increases the likelihood of optimal surgical outcomes. This paper presents an augmented reality (AR)-based simulation technique for ventriculostomy that visualizes brain deformations caused by the movements of a surgical instrument in a three-dimensional brain model. This is achieved by utilizing a position-based dynamics (PBD) physical deformation method on a preoperative brain image.
Methods
An infrared camera-based AR surgical environment aligns the real-world space with a virtual space and tracks the surgical instruments. For a realistic representation and reduced simulation computation load, a hybrid geometric model is employed, which combines a high-resolution mesh model and a multiresolution tetrahedron model. Collision handling is executed when a collision between the brain and surgical instrument is detected. Constraints are used to preserve the properties of the soft body and ensure stable deformation.
Results
The experiment was conducted once in a phantom environment and once in an actual surgical environment. The tasks of inserting the surgical instrument into the ventricle using only the navigation information presented through the smart glasses and verifying the drainage of cerebrospinal fluid were evaluated. These tasks were successfully completed, as indicated by the drainage, and the deformation simulation speed averaged 18.78 fps.
Conclusions
This experiment confirmed that the AR-based method for external ventricular drain surgery was beneficial to clinicians.

Keyword

Augmented Reality, Ventriculostomy, Surgical Navigation Systems, Computer Simulation, Biomechanical Phenomena

Figure

• 

  Figure 1 Augmented reality (AR)-based ventriculostomy simulation process. IR: infrared, 3D CT: three-dimensional computed tomography.

• 

  Figure 2 Augmented reality surgical environment. IR: infrared.

• 

  Figure 3 Setting and manual matching of response points. (A) Feature points in virtual space. (B) Feature points in real space.

• 

  Figure 4 Defining and distinguishing the forecasted deformation area based on the strain level.

• 

  Figure 5 Phantom model. (a) Rendered phantom model. (B) A 3D-printed phantom model.

• 

  Figure 6 Smart glasses screen. (A) Multiplanar reconstruction images, (B) zoom navigation, and (C) entry angle error of the surgical instrument.

• 

  Figure 7 Drainage and evaluation of artificial cerebrospinal fluid using the phantom model.

• 

  Figure 8 Performing augmented reality ventriculostomy in a real-world surgical environment.

Reference

References

1. Chen Y, Wang Q, Chen H, Song X, Tang H, Tian M. An overview of augmented reality technology. J Phys Conf Ser. 2019; 1237(2):022082. https://doi.org/10.1088/1742-6596/1237/2/022082.

2. Mishra R, Narayanan MD, Umana GE, Montemurro N, Chaurasia B, Deora H. Virtual reality in neurosurgery: beyond neurosurgical planning. Int J Environ Res Public Health. 2022; 19(3):1719. https://doi.org/10.3390/ijerph19031719.

3. Bhushan S, Anandasabapathy S, Shukla R. Use of augmented reality and virtual reality technologies in endoscopic training. Clin Gastroenterol Hepatol. 2018; 6(11):1688–91. https://doi.org/10.1016/j.cgh.2018.08.021.

4. Park SM, Kim HJ, Yeom JS, Shin YG. Spine surgery using augmented reality. J Korean Soc Spine Surg. 2019; 26(1):26–32. https://doi.org/10.4184/jkss.2019.26.1.26.

5. Chen Y, Wang Q, Chen H, Song X, Tang H, Tian M. An overview of augmented reality technology. J Phys Conf Ser. 2019; 1237(2):022082. https://doi.org/10.1088/1742-6596/1237/2/022082.

6. Cabrilo I, Bijlenga P, Schaller K. Augmented reality in the surgery of cerebral arteriovenous malformations: technique assessment and considerations. Acta Neurochir (Wien). 2014; 156(9):1769–74. https://doi.org/10.1007/s00701-014-2183-9.

7. Besharati Tabrizi L, Mahvash M. Augmented reality-guided neurosurgery: accuracy and intraoperative application of an image projection technique. J Neurosurg. 2015; 123(1):206–11. https://doi.org/10.3171/2014.9.JNS141001.

8. Watanabe E, Satoh M, Konno T, Hirai M, Yamaguchi T. The trans-visible navigator: a see-through neuronavigation system using augmented reality. World Neurosurg. 2016; 87:399–405. https://doi.org/10.1016/j.wneu.2015.11.084.

9. Gerard IJ, Kersten-Oertel M, Petrecca K, Sirhan D, Hall JA, Collins DL. Brain shift in neuronavigation of brain tumors: a review. Med Image Anal. 2017; 35:403–20. https://doi.org/10.1016/j.media.2016.08.007.

10. Vigo V, Tassinari A, Scerrati A, Cavallo MA, Rodriguez-Rubio R, Fernandez-Miranda JC, et al. Ideal trajectory for frontal ventriculostomy: radiological study and anatomical study. Clin Neurol Neurosurg. 2022; 217:107264. https://doi.org/10.1016/j.clineuro.2022.107264.

11. Amoo M, Henry J, Javadpour M. Common trajectories for freehand frontal ventriculostomy: a systematic review. World Neurosurg. 2021; 146:292–7. https://doi.org/10.1016/j.wneu.2020.11.065.

12. Hamze N, Peterlik I, Cotin S, Essert C. Preoperative trajectory planning for percutaneous procedures in deformable environments. Comput Med Imaging Graph. 2016; 47:16–28. https://doi.org/10.1016/j.compmedimag.2015.10.002.

13. Bayer S, Maier A, Ostermeier M, Fahrig R. Intraoperative imaging modalities and compensation for brain shift in tumor resection surgery. Int J Biomed Imaging. 2017; 2017:6028645. https://doi.org/10.1155/2017/6028645.

14. Fukuhara A, Tsujita T, Sase K, Konno A, Jiang X, Abiko S, et al. Proposition and evaluation of a collision detection method for real time surgery simulation of opening a brain fissure. ROBOMECH J. 2014; 1:6. https://doi.org/10.1186/s40648-014-0006-7.

15. Forte AE, Galvan S, Dini D. Models and tissue mimics for brain shift simulations. Biomech Model Mechanobiol. 2018; 17(1):249–61. https://doi.org/10.1007/s10237-017-0958-7.

16. Muller M, Heidelberger B, Hennix M, Ratcliff J. Position based dynamics. J Vis Commun Image Represent. 2007; 18(2):109–18. https://doi.org/10.1016/j.jvcir.2007.01.005.

17. Pan J, Yan S, Qin H, Hao A. Real-time dissection of organs via hybrid coupling of geometric metaballs and physics-centric mesh-free method. Vis Comput. 2018; 34:105–16. https://doi.org/10.1007/s00371-016-1317-x.

18. Pan J, Bai J, Zhao X, Hao A, Qin H. Real-time haptic manipulation and cutting of hybrid soft tissue models by extended position-based dynamics. Comput Anim Virtual Worlds. 2015; 26(3–4):321–35. https://doi.org/10.1002/cav.1655.

19. Romeo M, Monteagudo C, Sanchez-Quiros D. Muscle and fascia simulation with extended position based dynamics. Comput Graph Forum. 2020; 39(1):134–46. https://doi.org/10.1111/cgf.13734.

20. Mortensen EN, Barrett WA. Interactive segmentation with intelligent scissors. Graph Models Image Process. 1998; 60(5):349–84. https://doi.org/10.1006/gmip.1998.0480.

21. Lorensen WE, Cline HE. Marching cubes: a high resolution 3D surface construction algorithm. ACM SIGGRAPH Comput Graph. 1987; 21(4):163–9. https://doi.org/10.1145/37402.37422.

22. Allard J, Cotin S, Faure F, Bensoussan PJ, Poyer F, Duriez C, et al. SOFA: an open source framework for medical simulation. Stud Health Technol Inform. 2007; 125:13–8.

23. Kim J, Kwon K, Shin BS. Adaptive tetrahedral mesh generation for non-uniform soft tissue simulation. Hum Cent Comput Inf Sci. 2021; 11:29. https://doi.org/10.22967/HCIS.2021.11.029.

24. Muller M, Gross MH. Interactive virtual materials. In : Proceedings of the Graphics Interface 2004 Conference; 2004 May 17–19; London, Ontario, Canada. p. 239–46.

25. OptiTrack. Motive: optical motion capture software [Internet]. Corvallis (OR): NaturalPoint Inc;c2023. [cited at 2023 Jul 21]. Available from: https://optitrack.com/software/motive/.

26. Kim JD. Application of augmented reality for surgical guidance [thesis]. Seoul, Korea: Seoul University;2020.

27. Kelager M, Niebe S, Erleben K. A triangle bending constraint model for position-based dynamics. In : Proceedings of the Workshop on Virtual Reality Interaction and Physical Simulation; 2010 Nov 11–12; Copenhagen, Denmark. p. 31–7. https://doi.org/10.2312/PE/vriphys/vriphys10/031-037.

28. Palmer IJ, Grimsdale RL. Collision detection for animation using sphere-trees. Comput Graph Forum. 1995; 14(2):105–16. https://doi.org/10.1111/1467-8659.1420105.

29. Yang C, Li S, Wang L, Hao A, Qin H. Real-time physical deformation and cutting of heterogeneous objects via hybrid coupling of meshless approach and finite element method. Comput Anim Virtual Worlds. 2014; 25(3–4):421–33. https://doi.org/10.1002/cav.1594.

30. Xu L, Lu Y, Liu Q. Integrating viscoelastic mass spring dampers into position-based dynamics to simulate soft tissue deformation in real time. R Soc Open Sci. 2018; 5(2):171587. https://doi.org/10.1098/rsos.171587.