Patient Fall Monitoring and Prevention in Hospitals Use Case

Published: 04/21/2021  

Last Updated: 06/24/2021

Overview

Each year in the United States, avoidable patient falls number in the hundreds of thousands, with 30 to 50 percent of them resulting in new injuries and subsequent care issues for the patient involved.1 This use case enables you to create Artificial Intelligence (AI) based fall prevention to raise an alert when a patient is at risk of a fall.

Select Configure & Download to download the use case and the software listed below.  

Configure & Download



Time to Complete

Programming
Language

Available Software

Approximately 60 - 90 minutes

Python* 3.6

  • OpenNESS version 20.12 
  • Google* Cloud SDK
  • OpenVINO™ Model Server (OVMS) version 2021.2

Recommended Hardware
The hardware below is recommended for use with this use case. See the Recommended Hardware page for other suggestions. 


Target System Requirements

Controller and Node Systems

  • One of the following processors: 
    • Intel® Xeon® scalable processor. 
    • Intel® Xeon® processor D. 
  • At least 128 GB RAM. 
  • At least 256 GB hard drive. 
  • Intel® Ethernet Converged Network Adapter X710-DA4.
  • CentOS* 7.8.2003.
  • An Internet connection. 

Client System

  • One of the following processors: 
    • Intel® Core™ processor. 
    • Intel® Xeon® processor. 
  • At least 8 GB RAM. 
  • At least 256 GB hard drive. 
  • Intel® Ethernet Converged Network Adapter X710-DA4.
  • An Internet connection.

How it Works

Fall prevention in Hospital room includes the OpenVINO™ Model Server (OVMS) along with OpenNESS software. 

Open Network Edge Services Software (OpenNESS) toolkit
The Open Network Edge Services Software (OpenNESS) toolkit enables developers to port existing applications running in the cloud, provides components to build platform software, the ability to build and deploy E2E edge services in the field as well as perform benchmarking for RFPs for network/on-premise edge.  

Figure 1: Architecture Diagram

 

In this use case, the client is the IP camera that captures inputs of patient position. OpenNESS helps converge this input and transform it to a format that can be consumed by the OpenVINO™ model server.
This input is then fed to the OpenVINO™ pose estimation model, which generates inference if a patient is at risk of a fall.


Get Started

Prerequisites

Ensure that the following conditions are met properly to ensure smooth installation process.

  1. Hardware Requirements
    Make sure you have a fresh CentOS* 7.8.2003 installation with the hardware specified in Edge Node.
  2. Proxy Settings
    If you are behind a proxy network, please ensure that proxy addresses are configured in the system. 
    export http_proxy=<proxy-address>:<proxy-port> 
    export https_proxy=<proxy-address>:<proxy-port> 
  3. Date and Time
    Make sure that the Date and Time are in sync with current local time.
  4. OpenNESS supports two different modes:
    • For Single Device mode, you need one machine. (Both controller and edge node will be on same device.)
    • For Multi Device mode, make sure you have at least two machines (one for controller and other(s) for edge node).
      NOTE: Multi Device mode is not supported in this release.
  5. IP Address Conflict
    Make sure that the Edge Controller IP does not conflict with OpenNESS reserved IP addresses. For more details, refer to the Troubleshooting section. 

 

Install the Use Case 

Select Configure & Download to download the use case and then follow the steps below on the Controller to install it.  

Configure & Download

 

  1. Make sure that the Target System Requirements are met properly before proceeding further.  
    • For single-device mode, only one machine is needed. (Both controller and edge node will be on the same device.) 
    • For multi-device mode, make sure you have at least two machines (one for the controller and another for the edge node).
      NOTE: Multi-device mode is not supported in the current release. 
  2. Open a new terminal as a root user and move the downloaded zip package to the /root folder:  
    sudo passwd
    su
    mv <path-of-downloaded-directory>/converged_edge_insights.zip /root 
  3. Go to the /root directory using the following command and unzip the use case:  
    cd /root 
    unzip converged_edge_insights.zip 
  4. Go to the converged_edge_insights directory:  
    cd converged_edge_insights
  5. Change permission of the executable edgesoftware file: 
    chmod 755 edgesoftware 
  6. Run the command below to install the use case: 
    ./edgesoftware install 
  7. During the installation, you will be prompted for the Product Key. The Product Key is contained in the email you received from Intel confirming your download.  
    NOTE: Installation logs are available at: /var/log/esb-cli/converged_edge_insights//install.log  
     
    Figure 2: Product Key

     
  8. During the installation, you will be prompted to configure a few things before installing OpenNESS. Refer the screenshot below to configure.
    NOTE: Multi Device is not supported in this release. Select Single Device when prompted to select the type of installation. 
     
    Figure 3: OpenNESS Configuration

    NOTE: If you are using a Microsoft Azure* instance, please enter the Private IP address as the IP address of the controller.
  9. When the installation is complete, you see the message Installation of package complete and the installation status for each module. 
     
    Figure 4: Installation Successful

     
  10. If OpenNESS is installed, running the following command should show output similar to the image below. All the pods should be either in running or completed stage.
    kubectl get pods -A

     

When set up is complete, choose single human pose detection for a step-by-step, hands-on walkthrough of how to use and configure modules in Converged Edge Insights.


Fall Prevention in Hospital

This tutorial demonstrates deploying and executing OpenVINO™ pose detection on the OpenNESS edge platform. This tutorial uses the pre-trained OpenVINO™ pose detection model and covers OpenVINO™ applications with video streaming and video playback:

  1. Launch Kubernetes* Dashboard
    Kubernetes* Dashboard is a web-based user interface to manage cluster resources, deploy applications and troubleshoot it. Generate the access token and launch the dashboard.
  2. Deploy the OpenVINO™ Application
    Deploy and manage the OpenVINO™ edge application from Edge Controller to Edge Node.
  3. Simulate the IP Camera Stream
    Simulate the Hospital_Room.mp4 video file to work as an IP Camera stream and send it to the OpenVINO™ application in Edge Node.
  4. Start the Application
    Start the OpenVINO™ inference application for single human pose detection inside the pod.
  5. Verify Inference
    Start a video playback to visualize the inference video.
  6. Stop the Application
    Stop the application and delete the OpenVINO™ application.
Figure 5: Tutorial Activities

 

Step 1: Launch the Kubernetes* Dashboard

The Kubernetes Dashboard helps you manage the cluster and edge applications through the web-based UI.

  1. Open a terminal and login as a root user:
    su root

     

  2. Get the Kubernetes dashboard URL and token. Run the following script on Edge Controller system and wait for few seconds, as Token generation takes time:

    /root/get_controller_token.sh


    You are required to generate a new Token by running this script whenever you want to login to the Dashboard, as Token is valid only for the active session.
    You will see the Dashboard URL and Token generated in the terminal: 

    Figure 6: Output of get_controller_token.sh

     

  3. Use Firefox* browser to launch, and sign into the Kubernetes Dashboard using the URL generated on terminal and the token for authentication. Firefox* browser allows you to login to the Dashboard without additional configurations. You may require configuring browser settings in other browsers. The URL will be https://
    The Kubernetes Dashboard uses self-signed certificates. In the Firefox* browser, you might need to click on Advanced and Accept the Risk and Continue. 

    Figure 7: Kubernetes Dashboard Sign In

     

  4. You will see the following screen after a successful sign-in.
    You might see a different dashboard and different services after login, compared to the image below.

    Figure 8: Kubernetes Dashboard Overview

     

  5. Verify the status of Edge Controller that you installed in Dashboard. Select Cluster > Nodes tab in Dashboard, as shown in the following image:
    If the device is installed in Single-node installation, you will only see one device. In addition, you might see a different device name based on the configuration.

    Figure 9: Kubernetes Dashboard Overview - Single Node


    The tick marks in green beside the Edge Controller label (under Names column of Nodes screen) indicate that they are healthy.

 

Troubleshooting

If the status of the edge controller changes from: Ready (appears in Green in UI) to NotReady (appears in Gray in UI) after system reboot, then run the commands below on the respective systems and wait for a few seconds:

swapoff -a 
systemctl restart kubelet 

 

Step 2: Deploy the OpenVINO™ Application

In this step, you deploy the OpenVINO™ application from the Edge Controller. The application is containerized and stored in the local Harbor registry hosted at Edge Controller.

  1. Verify the OpenVINO™ application is available in the local Docker* registry:
    curl -X GET 'https://<Controller-IP>:30003/api/v2.0/projects/intel/repositories' -H 'accept: application/json' -k --cacert /etc/docker/certs.d/<Controller-IP>:30003/harbor.crt -u 'admin:Harbor12345' |jq | grep -w 'openvino-cons-app' 
     
    Figure 10: Verify OpenVINO™ Application Availability

    Check that the openvino-cons-app is available in the list of images. Other images are also available in the local Harbor registry targeted towards different use cases.
  2. Deploy a network policy allowing ingress traffic on port 5000 (tcp and udp) from 192.168.1.0/24 network to the OpenVINO consumer application pod as follows:
    1. Create a new file named network_policy.yml in the current working directory and add the following specification to the file:
      NOTE: Make sure to maintain the indentation for the following code snippet.
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: openvino-policy
        namespace: default
      spec:
        podSelector:
          matchLabels:
            name: openvino-cons-app
        policyTypes:
        - Ingress
        ingress:
        - from:
          - ipBlock:
              cidr: 192.168.1.0/24
          ports:
          - protocol: TCP
            port: 5000
          - protocol: UDP
            port: 5000 
      


       

    2. Create the network policy: 

      kubectl apply -f network_policy.yml

       

  3. Start the OpenVINO™ application. This command downloads the app named openvino-cons-app, which is around 2 GB in size and contains the OpenVINO™ application, libraries and inference model.
    cd /root/converged_edge_insights/Converged_Edge_Insights_<version>/Sample_OpenVINO_Application_for_OpenNESS/edgeapps/applications/openvino/consumer/
    
    kubectl apply -f openvino-cons-app.yaml
    kubectl certificate approve openvino-cons-app 


     

  4. Open the Kubernetes dashboard. View the status, log, and the device in which the pods are deployed.

    Figure 11: Kubernetes Dashboard Overview after successful application deployment


     

Troubleshooting

If the status of openvino-cons-app deployment appears Red, then run the following command to stop the deployment and continue again with the previous step providing the correct IP address.

kubectl delete deployment openvino-cons-app

If the deployment does not appear to be removed from the dashboard or the status shows as terminating, run the following commands to force the termination to complete:

swapoff -a 
systemctl restart kubelet 

Step 3: Simulate the IP Camera Stream

In this step, you use the Edge Controller for video stream simulation. This example uses a video file to simulate an IP camera video stream.

  1. In the terminal, run the following command to install ffmpeg server, download the video file and start simulating the video file: 
    cd /root/converged_edge_insights/Converged_Edge_Insights_<version>/Sample_OpenVINO_Application_for_OpenNESS/ 
    	
    chmod a+x simulate-video-stream.sh 
    ./simulate-video-stream.sh 
    
    Where  is the version selected before downloading the package.
     
  2. The Hospital_Room.mp4 file will start streaming to the OpenVINO™ pod. You will see output similar to the below. Let this continue in the terminal and proceed to the next step.
    Figure 12: Sample of Streaming Simulation Output

     

Step 4: Start the Application

In this step, you start the Single Human Pose Detection-based OpenVINO™ application in the pod. You must be logged on as the root user.

  1. Open a new terminal and login as a root user:
    su root

     
  2. Run the following command to update the OpenVINO™ Kubernetes pod to start the application and stream inference video back to the Edge Controller.
    cd /root/converged_edge_insights/Converged_Edge_Insights_<version>/Sample_OpenVINO_Application_for_OpenNESS/ 
    
    chmod a+x start-application.sh 
    ./start-application.sh  
    Where  is the version selected before downloading the package.
     
  3. Let this continue and proceed to the next step.
    Figure 13: Start Application and Stream Inference Video 

     

Step 5: Verify Inference

In this step you use a command-line media player to see the rendered data. You must be logged on as the root user.

  1. Open a new terminal and login as a root user: 
    su root
  2. Locally see the rendered data: 
    setenforce 0
    systemctl stop firewalld.service
    ffplay rtp://127.0.0.1:5001 
    The video playback of inferenced data begins. If your bandwidth is poor, a few data frames might be missed and you might see latency.

In this application, we have seen that the OpenVINO™ pod deployed in the Edge Controller is able to perform the inference on the video stream sent using IP camera simulation. The output video is streamed back from the OpenVINO™ application to the device for playback.

This tutorial also highlights that you can deploy and manage an OpenVINO™ application to the Edge Controller through a Kubernetes cluster.

Step 6: Stop the Application

  1. Close the video playback application by clicking the X icon on the top right of the window or by pressing Ctrl+C in the terminal from which it was launched.
  2. Stop the OpenVINO™ application by pressing Ctrl+C in the terminal where it was running.
  3. Close the simulate IP Camera application using Ctrl+Z command in the terminal.
  4. To delete the OpenVINO™ application from the Edge Controller and re-enable the firewall, run the following commands as root:
    kubectl delete deployment openvino-cons-app
    systemctl start firewalld.service 

    If the deployment does not appear to be removed from the dashboard or the status shows as terminating, run the following commands to force the termination to complete:
    swapoff -a 
    systemctl restart kubelet 

Summary and Next Steps

In this tutorial, you learned how the OpenVINO™ toolkit and OpenNESS work together to detect objects in video streams transmitted by the Edge Nodes.


Troubleshooting

IP Address Range Allocation Conflicts

The OpenNESS Experience kits deployment uses, allocates, and reserves a set of IP address ranges for different CNIs and interfaces. The server or host IP address should not conflict with the default address allocation. If there is a critical need for the server IP address used by the OpenNESS default deployment, you must modify the default addresses used by OpenNESS.
The following files specify the CIDR for CNIs and interfaces. These are the IP address ranges allocated and used by default. 

flavors/media-analytics-vca/all.yml:19:vca_cidr: "172.32.1.0/12"
group_vars/all/10-default.yml:90:calico_cidr: "10.243.0.0/16"
group_vars/all/10-default.yml:93:flannel_cidr: "10.244.0.0/16"
group_vars/all/10-default.yml:96:weavenet_cidr: "10.32.0.0/12"
group_vars/all/10-default.yml:99:kubeovn_cidr: "10.16.0.0/16,100.64.0.0/16,10.96.0.0/12"
roles/kubernetes/cni/kubeovn/controlplane/templates/crd_local.yml.j2:13:  cidrBlock: "192.168.{{ loop.index0 + 1 }}.0/24"

The address 192.168.x.y is used for SR-IOV and interface service IP address allocation in Kube-ovn CNI. Therefore, it is not allowed as a server IP address, which conflicts with this range. We recommend that you completely avoid the address range defined by the netmask because it may conflict with routing rules.

For example, if the server/host IP address must use 192.168.x.y, there is a conflict because this range is used for SR-IOV interfaces in OpenNESS by default. The IP address range for cidrBlock in the roles/kubernetes/cni/kubeovn/controlplane/templates/crd_local.yml.j2 file can be changed to 192.167.{{ loop.index0 + 1 }}.0/24 to use some other IP segment for SR-IOV interfaces.

Support Forum 

If you're unable to resolve your issues, contact the  Support Forum.  

 

1  Lyons, Maureen, “The Joint Commission launches educational campaign on preventing falls,” The Joint Commission, July 24, 2019: jointcommission.org/en/resources/news-and-multimedia/ news/2019/07/the-joint-commission-launches-educational-campaign-on-preventing-falls.

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.