[3Yin-A] Poster 5

[3Yin-A-20] Development of a Motion Recording System and Data Collection Experiments Toward Realizing MR-Based Medical Interview Examinations

〇Yusei Hashimoto¹, Ryo Taguchi¹, Satoshi Arakawa², Shin Ishihara² (1. Nagoya Institute of Technology, 2. Fujita Health University)

Keywords:

Medical Interview,Mixed Reality,Motion Recognition,Temporal Convolutional Network

Medical interviewing is a clinical practice in which a physician communicates with a patient to obtain necessary information. In Objective Structured Clinical Examinations (OSCEs) for medical and dental students, the availability of standardized simulated patients and the burden on evaluators in medical interview examinations remain major problems. To address these problems, our research group has been developing an MR-based medical interview examination system using a CG simulated patient avatar. Because the examinee’s behavior toward the patient is evaluated in medical interview examinations, in addition to verbal information, it is necessary to recognize and assess actions such as nodding, gestures, and posture of the MR headset wearer. In this study, aiming to realize an automatic evaluation system for medical interview examinations, we develop a system that records the head and hand/finger motions of the headset wearer. This paper outlines the recording system and reports methods for analyzing the wearer’s behavior (e.g., gaze direction and note-taking motions) from the collected motion data. We also report the results of a data collection experiment involving four physicians affiliated with a medical school.