Challenge: While Home Blood Transfusion Is Becoming Common to Improve the Patient’s QoL, A Physician’s Immediate Response is Required if Adverse Events Occur
To ease the burden on patients suffering from hematological diseases such as myelodysplastic syndrome, aplastic anemia, leukemia, multiple myeloma, and malignant lymphoma, some medical institutions have begun to offer at-home blood transfusions, where a transfusion to supplement insufficient red blood cells and platelets is performed at a patient’s home. Although in-hospital blood transfusion is preferred, home transfusions aim to improve patients’ quality of life (QoL) for those who find it difficult to be hospitalized or come to the hospital on a regular basis.
However, due to the possibility of adverse events occurring during blood transfusion therapy (ranging from allergic reactions such as fever and itching to hypotension and dyspnea), patients must be accompanied by a physician, nurse, or an attendant (generally a family member) during and after the transfusion.1 If one of these adverse events is observed while the patient's attendant is present, they must immediately alert a physician.
Dr. Akinori Nishikawa, General Manager, Medical Information Department. Assistant Director, Division of Blood Transfusion (Associate Professor). Wakayama Medical University Hospital.
Wakayama Medical University Hospital's Dr. Akinori Nishikawa (Associate Professor, Division of Blood Transfusion) and his team are now utilizing technology in order to further facilitate the discovery and handling of such adverse events.
“Because physicians need to make house calls to other patients, it is unrealistic for them to stay with each patient until the blood transfusion is completed. Once the transfusion has started, the task of overseeing the patient is generally left to the attendant or a visiting nurse. However, since attendants do take their eyes off the patient to undertake household chores, etc., we decided to use technology to monitor the patient's condition and help improve safety,” Dr. Nishikawa said.
Wakayama Medical University Hospital.
The first initiative undertaken by Wakayama Medical University Hospital was remote monitoring of vital signs. In this project, the patient is fitted with an electrocardiograph on the chest and a pulse oximeter on the index finger. The system collects the data, and transmits it via Bluetooth to a bedside smartphone, which sends the data via a 4G network to the hospital where it can be monitored by a clinician.
Dr. Nishikawa explained the progress of his research. “We tested the system on several consenting patients and discovered some issues. Body movements such as turning over in bed created noise in the electrocardiograph, and occasionally cold fingertips prevented the collection of accurate data by the pulse oximeter. We therefore investigated whether we could combine AI with video to identify patient movements.”
Solution: Intel® Distribution of OpenVINO™ Toolkit Is Adopted for Video Recognition to Detect Dangerous Movements During Blood Transfusion
Dr. Nishikawa decided to use video recognition with deep learning to detect patient movements, such as those listed below, during transfusions.
- Getting up or standing up during a blood transfusion, which can lead to falls or accidental removal of blood transfusion tubes.
- Bending of an upper extremity (right or left arm in which the needle is inserted) that may lead to poor instillation or obstruction.
“Some patients of an advanced age with declining cognitive function may be unable to follow instructions such as ‘please do not get up during the transfusion’ or ‘please do not bend your arm.’ By detecting these harmful movements that cannot be determined by vital monitoring alone, we hope to further enhance safety during blood transfusions,” Dr. Nishikawa said. Dr. Nishikawa adopted the OpenVINO™ toolkit to detect harmful movements during home blood transfusions. “When meeting with staff from Intel at the Japan Association for Medical Informatics, we were given a demonstration of pose detection using the OpenVINO toolkit. We felt that the toolkit would be easy to use, even if one is not familiar with deep learning, as various pre-trained models are provided as samples.”
OpenVINO Toolkit with a Standard Model: A 2D Human Pose Estimation Model Is Adopted to Determine Body Position
The OpenVINO toolkit performs optimization so that inference models developed on other deep learning frameworks can be executed on hardware platforms such as Intel® processors. Standard pre-trained models are also provided for the purpose of demonstration or evaluation, and a framework is not required if using these models.
The toolkit provides object detection models for people, vehicles, and other objects, as well as object recognition models, human tracking models, semantic segmentation (classification) models, instance segmentation models, human pose estimation models, image processing models, character detection models, character recognition models including handwriting, and mathematical formulas and motion recognition models.
Of these, Dr. Nishikawa focused on a 2D human pose estimation model (model name: “human-pose-estimation-0001”), which detects the ears, eyes, nose, neck, shoulders, elbows, wrists, left and right pelvis, knees, and ankles, from a human’s image for a total of 18 key points.
Human-pose-estimation-0001 was developed using Carnegie Mellon University's OpenPose technology, which estimates human joints. It is based on a convolution technique called MobileNet v1, which greatly reduces computational complexity, allowing pose estimation to be performed at practical speeds on ordinary PCs.
Figure 1: Overview of detection system of dangerous movements of home blood transfusion patients developed by Wakayama Medical University Hospital.
How It Works: A Unique Algorithm Using Python Is Installed to Determine Body Position – Getting Up or Bending of An Upper Extremity Is Detected and the Physician Is Notified
An overview of an abnormal movement detection system for home blood transfusion patients is shown in Figure 1. A smartphone is placed at the bedside of a patient who has given consent, a PC at Wakayama Medical University Hospital receives video via Zoom, and the OpenVINO toolkit's human pose estimation model and the algorithm is used to detect any dangerous movements, which are then flagged for physician review. The physician can check the footage of the patient on their Zoom smartphone app and contact the patient's attendant as well as perform any other necessary measures, including another house call.
The first step required in detecting abnormal movements of a home blood transfusion patient is discerning the position of their body: whether it is decubitus, seated, or erect.
“The video taken by a camera placed at the bedside is sent to the OpenVINO toolkit's human pose estimation model via Zoom. The tilt of the head/neck, upper body/trunk and thighs are calculated from the coordinates of each key point output by the model, and are used to determine the body position,” Dr. Nishikawa said.
Figure 2: Flowchart of the algorithm that determines body position from the output (right) by 2D human pose estimation models of the OpenVINO toolkit.
An overview of the algorithm is shown in Figure 2. When a patient is under a blanket and only the angle of the neck can be observed, the flow is constructed so that the patient is determined to be in a “decubitus” position if the neck angle is at a certain value or below, and is determined to be in a “seated” position if the neck and trunk are erect and the thigh angle is close to horizontal. However, since the pose estimation model output may output “noise” (false estimation), the position is determined by taking the average of 500 data points (equivalent to approximately 60 seconds of monitoring).
When the body position changes (for example, from decubitus to seated), or when an upper extremity is bent for three minutes or more, the physician is notified with a message, as seen in the bottom left of Figure 1.
Mr. Yoshiki Kusumoto, Assistant Manager, Public Cloud Project Division, CYBERLINKS CO., LTD.
CYBERLINKS CO., LTD. was responsible for building the platform based on the OpenVINO toolkit and installing the algorithm with Python. Yoshiki Kusumoto of CYBERLINKS said, “10th Gen. Intel® Core™ processor family have been adopted by Wakayama Medical University Hospital for their abnormal movement detection system. Furthermore, there were no problems installing of the OpenVINO toolkit or using the human pose estimation model.” He also indicated, “We can see new possibilities for AI deployments that don’t require expensive GPU servers, but can be programmed on PCs with general-purpose CPUs.”
Figure 3: Results from the trial operation of the abnormal movement detection system in 14 instances of home blood transfusions.
Result: 14 Trials of Home Transfusions Revealed Approximately 90% Match Between Actual Body Position of a Patient and System Determination
A trial operation of the prototype system was conducted with the cooperation of Akasaka Clinic (Nada-ku, Kobe-shi), which is actively involved in home blood transfusions. The results of 14 home blood transfusions are shown in Figure 3. The blue graph shows the number of times a message was sent to a physician when the system determined a change in body position, and the orange graph shows the number of matches between the determination and the patient’s actual body position obtained when the physician received a message and checked the video.
The match rate between the determination and the actual body position regarding decubitus and seated positions was around 90%, confirming that determinations were made with almost problem-free accuracy. However, in the erect position, standing poses were not recognized, even when checking the video after notification. Dr. Nishikawa explains, “When the patient is sitting near the front of the installed camera, the angle of the thigh appears to be almost vertical. Therefore, the body position is determined to be erect, even if the patient isn't standing, due to the combination of the current human pose estimation model and determination algorithm.” As shown in the right column in Figure 3, the average number of messages sent during the 14 home blood transfusions was 7.5. Dr. Nishikawa believes that this frequency can be handled even by physicians making house calls to other patients. Although the maximum number of messages was 18, the system was set to notify the physician every three minutes when bending of an upper extremity was detected for three minutes or longer. The number of messages increased as the patient’s arm was bent for a sustained period.
Although some incorrect determinations were observed, Dr. Nishikawa noted that “the foundation of home blood transfusions is having an attendant present. This system and vital monitoring are only additional support. We believe that we have validated the basic functions and performance, without the need for split-second decisions.”
The research was undertaken with the support of the Clinical Research Promotion Project by the Japan Society of Transfusion Medicine and Cell Therapy, and the results were presented at the 69th Annual Meeting of the Japan Society of Transfusion Medicine and Cell Therapy in June 2021.
Going forward, Dr. Nishikawa will continue to work on detecting sudden changes in body position (getting up or standing) that are highly likely to result in falls. He is also implementing a process to identify adverse events using existing PCs at Wakayama Medical University Hospital. He also has plans to study so-called edge computing operations, where a small PC equipped with the OpenVINO toolkit, decision-making algorithms, and vital monitoring functions would be installed in the patient's home.
There is ongoing support for home blood transfusion, including safety and operation, although it remains uncommon due to the burden of treatment on medical services. Meanwhile, as Japan's population continues to age, the need to stay at home while receiving in-home medical care is expected to grow. More medical institutions may be encouraged to introduce home blood transfusion through Dr. Nishikawa’s research on improving patient care and safety, and reducing burden on patient attendants.
The Intel Distribution of OpenVINO toolkit optimizes the development of deep learning applications. The Intel Distribution of OpenVINO toolkit is an open-source platform designed for developers and data scientists to speed up the development of high-performance computing vision and deep learning applications. Based on convolutional neural networks (CNNs), it includes a model optimizer for frameworks like Caffe and TensorFlow, an inference engine for computing vision accelerators, and a common API for Intel® hardware.
The toolkit can run on Intel® Xeon® processors, the Intel® Core™ processor family, Intel® Movidius™ Myriad™ X Vision Projector Units (VPU), or Intel® Processor Graphics Units (GPU).2
Download the PDF ›