The emerging role of Microsoft Kinect in physiotherapy rehabilitation for stroke patients
Original Editor - Kate Reidy, User:Sally Wood, Hanna Meredith, Nicole Bell and Karen Feeney as part of the Queen Margaret University's Current and Emerging Roles in Physiotherapy Practice Project Top Contributors - Sally Wood, Kate Reidy, Nicole Bell, Karen Feeney and Hannah Meredith
This article is currently under review and may not be up to date. Please come back soon to see the finished work! (14/05/2020)
- 1 Introduction
- 2 The role of Sensors in Healthcare
- 3 Advantages and Disadvantages
- 4 Cerebrovascular Accidents
- 5 Barriers to Stroke Physiotherapy Rehabilitation
- 5.1 The Role of Microsoft Kinect
- 5.2 The Need for Intensive Rehabilitation
- 5.3 The ICF model and Virtual Rehab
- 5.4 Individuals Perspective on Technology
- 5.5 Microsoft Kinect’s potential future in stroke rehabilitation: Emerging evidence
- 5.6 Physiotherapy Role
- 5.7 Patient Suitability
- 5.8 Protocol of Patient Assessment
- 5.9 Advantages & Disadvantages for Physiotherapists
- 5.10 Conclusion
- 5.11 Resources
- 5.12 References
Effective stroke rehabilitation is dependent on patients continuing their exercise programme in the home environment . Microsoft Kinect is a device which can offer innovative and exciting ways to rehabilitate, making treatment more enjoyable and thus increasing motivation and subsequent adherence. This is an important area for physiotherapists to develop their skills in, as technology is beginning to thrive in the health sector and is becoming a part of therapy treatment options. In addition, it reduces workload by utilising physiotherapy time effectively while still providing therapy.
The median age of the general population is predicted to significantly rise in upcoming years. This causes much added stress for clinics and hospitals. Age is a significant risk factor for astroke and the need for new rehabilitation is in demand. Technology and it's advanced methods are continuously being assessed so that they can be used in a private, home-based setting while still providing rehabilitation instructions and progress tracking. This area of rehabilitation technology is expected to expand and will do so over the coming years.
The Kinect allows the patient to interact with the system in a 3D environment, where they perform multiple movement combinations without the need of an attached device or a controller. It is tackling issues related to rehabilitation. Not only is it providing a rehabilitation service which improves adherence to treatment by adding fun and entertaining features, but it also reduces the high cost associated with traditional rehabilitation making it more affordable and accessible to everyone.
Ideally, all stroke rehabilitation exercises would be performed with a therapist in a clinical setting on a daily basis with a recommended session of 45 minutes per patient. However, this demand within hospitals is becoming increasing difficult and logistically and financially impractical.
Microsoft Kinect is the forerunner in commercially available hardware in which development of these new technologies can be built. Information on how it works, what is does, and an insight into Virtual Rehabilitation utilizing Microsoft Kinect for the use with stroke patients will be discussed in the following page.
The role of Sensors in Healthcare
Technology within rehabilitation has emerged dramatically over the past couple of years. Technology's capability and affordability are partly to blame for this. The future of health is about highly personalised care and putting the control of patient’s health back into their own hands. Sensors in healthcare are becoming more prevalent and their ability to detect even the smallest of movements more recently is making them very appealing. Sensors are capable of automating, large scale monitoring and measurement of health-related parameters. They are capable of diagnosing, treating and monitoring patients and providing clinicians with early warning signs for potential problems. They can also provide more objective feedback than a clinician stating subjectively how a patient’s body is positioned in space. Vinod Khosla stated that “In the next 10 years, data science and software will do more for medicine than all the biological sciences together”. Thus indicating that as allied health professionals there is a need for us to develop our skills in technology in order to keep up with the rapidly progressing rehabilitation environment. One device in particular that has been developing over the past few years is Microsoft Kinect and it is developing into a very feasible option for rehabilitation in the stroke population.
What is Microsoft Kinect?
The name Kinect is inspired by the words "kinetic", meaning to be in motion, and "connect" meaning it "connects you to the entertainment and friends you love". Microsoft Kinect is a motion sensing input device for Xbox 360, Xbox one video game consoles and Windows PCs. Kinect is a webcam based peripheral add-on that enables its users to control and interact with their console without the need for a controller. It is a general purpose and low-cost 3D input device that is popular in the home setting. The first generation of Kinect was introduced in November 2010 in an effort to increase the Xbox 360's audience beyond that of its normal gamer base. A version of Kinect for Windows was released on 1 February 2012 and subsequently for the Xbox One.
Microsoft Kinect is a recently advanced multimedia console that was created to revolutionize the gaming experience and the way people experience entertainment. Sensors are used with the device which can directly sense a players’ movement and their environment. The sensors utilize 3D depth cameras which enables people to interact with games using their own body in a natural way. A unique characteristic is that the games are played without the use of a controller as whole limb movements are monitored by the sensor. The impact the Kinect has made has ventured far beyond the gaming industry and has extended into creating new and exciting ways to interact such as helping doctors in operating rooms, helping children with autism and rehabilitating neurological patients. A team at Southampton have taken this technology to the next level and created an algorithm whereby the Microsoft Kinect sensors can track and measure joint angles of the hand and fine dexterity of individual finger movements making movements recorded even more precise.
The Kinect sensor is a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display. The device features an RGB camera, depth sensor, and multi-array microphone which are responsible for full-body 3D motion capture, facial recognition and voice recognition abilities. The RGB camera allows the Kinect to recognise up to 6 people and up to two people can be tracked. Skeletal tracking allows users to be tracked whilst sitting or standing and facing the Kinect. The user simply has to stand in front of the Kinect with head and upper limbs visible to be recognized, no specific calibrations need to be made for a user to be tracked. Skeletal tracking mode allows the sensor to track 20 joint coordinates in 3D space per player (see figure below). Users are provided with a natural user interface which allows them to control the game with a gesture or voice command. The ability of the Kinect to capture 3D data independent of lighting conditions strengthens the value of the console in rehabilitation settings. These features allows the Kinect device to identify players and detects movement from the body. Computer-based gaming systems such as Microsoft Kinect, can facilitate complex task practice, enhance sensory and motor feedback, and provide visually observed technique for the user.
The Kinect is currently the hardware device that provides developers with the greatest opportunities for innovative programmes - for both games and applications. Although Kinect was developed for playing games, the technology has evolved and is now been applied to real-world applications which vary from virtual shopping, education, telehealth service delivery and further regions of health IT.
The software programmes can be used in clinic and in patients homes where they can continue their rehabilitation programmes. Due to the games being more engaging, interesting and innovative, they have the ability to create improved motivation and compliance leading to better patient outcomes. Motivation is improved as patients can work against themselves and see improvements no matter how small. Compliance is maintained due to the more exciting nature of the games making patients more likely to rehabilitate everyday as maintaining a set routine of exercise for weeks on end can become tedious.
The Microsoft Research video below demonstrates the use of the Kinect and gives an insight into how it works.
Many different companies and organisations have utilised the unique technology of the Microsoft Kinect to create rehabilitation based software programmes. Creating new and innovative rehabilitation options for many different groups of patients but most notably those with neurological conditions such as stroke, Parkinson’s disease and multiple sclerosis. Two currently developing softwares for the rehabilitation of neurological conditions that we will explore further are VirtualRehab and Jintronix. The software product has been developed by a multidisciplinary team of experts in neurology and physiotherapy. This software enables patients with different affected functions to work on rehabilitation in an innovative and exciting way. It allows for the treatment of multiple different pathologies such as neurodegenerative diseases, neuromuscular disorders, neurovascular disorders/trauma and mobility for the elderly. This tool provides functional training so as to improve equilibrium, coordination, weakness, fatigue and spasticity. The exercises can be adapted to a patient’s disability levels so that the programme can be used with a wide range of ability levels.
Virtual Rehab utilises the unique characteristics of Microsoft Kinects motion technology to track and capture the movements of the patients so that patients become immersed in a 3D environment where they interact with the game. VirtualRehab Body programme allows patients to retain upper and lower limb motor functions. The structure of the exercise will be clearly defined and visual cues are provided for execution of the exercise eradicating the potential problem of forgetting a step in the exercise or the whole exercise completely. Furthermore, the Kinect motion technology will record the execution of the movement to provide feedback to the patient and the therapist on correct or incorrect movement performed. This can be used within clinics and hospitals as well as within the patient’s house so that rehabilitation can be continued at home. By providing highly motivating games patients are more likely to adhere to their rehabilitation programmes leading to improved outcomes.
- Bullseyes & barriers – static & dynamic balance, load transfer. The user must intercept the objects that appear on screen with their upper limbs and must avoid lower obstacles by raising their feet and step on indicated tiles
- Fit in the figure – flexibility, standing position, balance. The user must ensure the avatar shape matches their own
- Row the boat – bilateral upper limb training. The user must row and move their hands in a coordinated and parallel fashion
- Bail out the water – trunk control, body scheme, hand-eye coordination. The user must bail the water out of the boat using the pump which is activated by moving the hands in a parallel but inverted fashion.
Jintronix was set up in 2012 after the CEO, Justin Tan's father suffered a stroke. After standing alongside his father through the long rehabilitation process he became determined to find out whether technology could help this process. He eventually went on to create Jintronix with a few of his engineering friends. The purpose was to create an engaging, accessible and cost-effective stroke rehabilitation program which would motivate and improve patients' overall experience of rehabilitation.
One way Jintronix achieved this was by developing an exercise platform that provides immediate feedback so that patients and clinicians can participate in the sessions at home or in hospital. The plan of Jintronix technology was to make repetitive rehabilitation programs more engaging and turn them into games that patients would enjoy playing. Once released from hospital, 65 percent of patients either don't do the prescribed exercises or are only partially adherent because they get fatigued easily and the exercises themselves are tedious. Therefore, solutions like the Jintronix have taken on the task of improving client compliance and, ultimately, therapy outcomes.
The games were developed through researching which exercises and sports best fitted with conventional therapy. The rehab modules are adaptable to the level of a patient's functional and cognitive abilities and are designed to train balance & mobility, muscle strengthening & endurance, flexibility & range of motion, fall prevention, postural control, motor control & relearning and bilateral coordination . Jintronix is capable of tracking a players’ movements to see whether they are performing the activities correctly and relays this to the therapist who can then make adjustments. So far, there are seven games that have been created and work smoothly. See what patients had to say about their experience of using Jintronix in section 6.3.2.
Jintronix can be used in 3 different ways:
- Integrated model – Clinicians use Jintronix with their clients during their one-on-one session
- Kiosk model – Clinician sets up Jintronix rehab program for patient. Patient performs their exercises with the help of an aide.
- Telerehab model – Client uses Jintronix at home with remote supervision
- Ski game – weight transfer & squatting. Player will move their avatar down a slope through various obstacles and performing different balancing movement 
- Fish frenzy – controlled movement. Player must will control the movement of a fish through the game, performing different exercises such as drawing figures of 8, moving up and down whilst on the hunt for food .
- Pop clap & garden crab – bilateral coordination games
- Whack-a-mole – lower limb strengthening & dynamic balance. Player must walk or hop towards a specific part of the screen when a bunny when it pops up onto the screen 
- Trial using Jintronix software and Microsoft Kinect motion sensor for full body rehabilitation using a pull to stand aid and standby table
- 88 year old male, 3 months post-stroke
- Active-assisted upper limb activities are possible, weight shift and balance activities are possible and the tracking system is capable of giving feedback on performance all whilst using a stand aid.
- Increased total activity time in rehabilitation, increased repetition of movements of both limbs are possible in stroke patients who cannot stand independently.
- Beneficial evidence that assistive devices are able to be used with this software.
- This extremely important as following stroke many patients are left with debilitating effects and cannot stand or walk independently.
- It is valuable to know that patients with a range of physical limitations are capable of using Jintronix for rehabilitation
- 14 stroke participants took part in three 20 minutes practice sessions
- Participants arm impairment levels were measured using the Chedoke-McMasters (CM) scale
- Each participant ran through a range of difficultly levels on Jintronix, which has a total of 10, in order to find the level that suited their ability.
- Data received has given an insight into the relationship between arm impairment level of the participants and the difficulty level that participants were able to achieve on Jintronix.
- Clinicians have an idea of what can be expected from clients with varying arm impairment levels so that precise goals can be made in relation to achieving a specific difficulty level.
- The ultimate goal would be to create guidelines for prescription of UL training using Jintronix.
- Patients would not have to trial and error through the stages but would begin at their optimal level based on their CM score.
- Majority of participants enjoyed the activities provided by Jintronix and found them easy to use .
- Picture from: 
- Kinematic Validity of the Kinect camera for rehabilitation purposes
- Kinect was compared against the gold standard; the Optotrak 3D motion capture system.
- A series of hand reaching movements, trunk leaning and elbow flexion and extension were performed by one healthy participant over each camera position and repeated three times.
- Both systems tracked the participant’s movements simultaneously and the captured frames were compared for each movement.
- Data received from the Kinect closely matched Optotrak after bias was removed.
- Kinect is a valid tool monitoring hand position, trunk position and elbow angle .
- Usability of Jintronix for patients and therapists
- 12 therapists evaluated the Jintronix system based on usability and utility for rehabilitation.
- They navigated the home page, used the calibration sequence and played 5 of the systems games.
- Different aspects of the system such as involvement of specific movements e.g shoulder abduction, wrist extension, range of motion etc. were rated using a 5-point Likert scale.
- All games were reported as being capable of training active range of motion, speed, precision, and shoulder flexion and extension.
- Bilateral games required bilateral coordination and unilateral games trained elbow flexion and extension.
- The trunk game was capable of training trunk movement and balance.
- All therapists were interested in using Jintronix in the clinical setting
- Picture from: 
Advantages and Disadvantages
|Advantages of Microsoft Kinect||Disadvantages of Microsoft Kinect|
|Intensive therapy that can be carried out at home.||Many patients demonstrate significant levels of disability making it difficult to conduct rehabilitation using Microsoft Kinect.|
|Inexpensive tool compared to long term rehabilitation sessions- Microsoft Kinect is available as an add on for Xbox One and Xbox 360 sensor.||Recognition of patients- patients who present with hemiplegia, limited active movement in one arm or other similar postural deformities may find that the Kinect does not 'see' them. Similarly someone using a self propelled wheelchair will be recognized straight away, someone else using a powered wheelchair, with joystick control for example, may find themselves struggling to be noticed and registered by the sensors.|
|The accuracy of Kinect for clinical use is strong.||Kinect systems are usually not suitable for severely disabled patients, as gross movements that remain very small in their entirety are challenging for the Kinect to precisely capture.|
|Good test re-test reliability, and good repeatability.||Games targeted at rehabilitation may be prone to “cheating” (e.g. excessive, unnatural and counter-productive trunk-based compensation).|
|Can increase motivation levels to complete home rehabilitation programme since they can be viewed as a fun activity.||The current benefits of Kinect-based gaming have only tentatively been studied with mainly short term and small sample sized studies.|
|Does not require the use of attachments of sensor devices to the body or held onto by hand.||Software is still in development.|
|Simple to use.||Social problem – patients may become house bound with their rehabilitation programme and become excluded from society.|
A cerebrovascular accident or stroke occurs when there is interruption of the blood supply to a particular area of the brain, usually because a blood vessel bursts or is blocked by a clot. This cuts off the supply of oxygen and nutrients, ultimately leading to cell injury and cell death.
Oxford Stroke Classification
Adapted from: 
Types of stroke
|Ischemic stroke (85%)- infarction||Haemorrhagic stroke (15%)|
|Atherothromboembolism 50%||Primary intracerebral haemorrhage 10%|
|Intracranial small vessel disease 25%||Subarachnoid haemorrhage 5%|
|Embolism from heart 20%|
Barriers to Stroke Physiotherapy Rehabilitation
Stroke occurs approximately 152,000 times a year in the UK; that is one every 3 minutes 27 seconds. As people are living longer the number of sufferers will continue to increase, making cost effective community rehabilitation a higher priority. Stroke is the largest cause of complex disability with half of stroke survivors being left with residual disability. Globally the challenge of providing quality, affordable health care has never been so fundamental in stroke rehabilitation. The number of sessions that stroke survivors attend is restricted to the availability of a clinic near their residence and the amount of time friends and family can offer to help them commute. Other factors limiting access to rehabilitation services may include: a physical impairment preventing or restricting attendance at a local service, a lack of clinicians or specialists in a geographic area, or the insufficient provision of resources in a geographical region.
With the high prevalence rate of stroke this puts increased demand on physiotherapists to treat each patient both in hospital and after discharge. Although many people with stroke receive early rehabilitation, many are left dissatisfied with the way they are followed up and the level of rehabilitation provided in the year following the stroke. Due to time restraints and prioritisation of patients they can be pushed further down the rehabilitation follow up list. Another challenge to effective rehabilitation is insufficient completion of rehabilitation exercises by patient's which severely reduces the potential outcome of overall motor recovery. Exercises are often unpleasant and tedious leading to patient's tolerance for exercise to decline. Patients are often dependent on one-to-one rehabilitation services provided by physiotherapists to help regain their functional ability.It is widely recognized that a strong rehabilitation service using technologically advanced methods is the most effective and efficient way of addressing these growing issues.
Home-based rehabilitation is more flexible and convenient for the patient and allows more frequent repetition of exercises. In order to stimulate neural reactivation in the regions of the brain that controls movement, exercises must be repeated many times every day. While rehabilitation sessions alone often cannot fulfill the required frequency of practice, at home rehabilitation programmes using Microsoft Kinect can help achieve this goal.
The Role of Microsoft Kinect
Microsoft Kinect can: :
- Provide flexibility in delivering individualized physiotherapy in the patient's home.
- Involve carers more within the rehabilitative process (Stroke Association recommendation)
- Help the NHS meet National Clinical Guidelines on physiotherapy interventions for stroke (45 minutes per day of treatment during the acute phase)
- Improve patient engagement through stimulating rehabilitation.
- Be a low cost rehabilitation tool
- Reduce the need for expensive one-to-one clinical contact
- Provide a range of indirect savings due to improved rehabilitation of patient
- Reduce transportation cost and time savings from the perspective of both the physiotherapist and patient
- Provide continuity of patient care
The demand for technologically advanced home-based rehabilitation which allows therapists to track progression is expected to expand. Microsoft Kinect is the leading commercially available hardware which can be developed to meet these needs while simultaneously maintaining affordability.In an ideal world, stroke rehabilitation would take place with one to one therapist-patient interaction on a daily basis, however, demands on both the NHS and physiotherapists makes this dream difficult to achieve. Through guided interactive rehabilitation and the ability for therapists to track progress, the Kinect-based stroke rehabilitation programmes potentially reduce these difficulties on demand. The accuracy of the Kinect for clinical use in stroke rehabilitation is strong, allowing therapist assisted home-based rehab to become a reality. The Kinects ability to allow therapists to track progress and give feedback may not only enhance rehabilitation, but also improve medical record keeping.
The Need for Intensive Rehabilitation
Physiotherapy plays a key role in the rehabilitation of those affected by stroke and is needed in order to aid patients through their journey to recovery. Physiotherapy in stroke rehabilitation typically aims to strengthen and retrain muscles to regain lost limb function and improve quality of life.
Through studies of motor learning and motor control evidence shows that for effective learning and muscle re-education to take place practice of exercises needs to be intensive and frequent. The most rapid recovery occurs within the first 3 months post stroke and therefore during this time frame patients need intensive rehabilitation therapy. Unfortunately, currently in most NHS hospitals stroke patients receive approximately 7.5 hours of physiotherapy rehabilitation per week and when discharged patients receive on average 1-2 hours of community rehabilitation per week. Additionally, the current early supported discharge plan, for stroke patients with mild to moderate disability, only runs for on average 6 weeks. Due to the current economic status of the NHS this is not likely to improve therefore more intensive rehabilitation strategies need to be introduced in order to improve outcomes for stroke patients.
Patients are being discharged from hospital earlier and earlier as there is a need to free up hospital beds which often leaves patients with fewer hours of physiotherapy rehabilitation. There is therefore a need for cost effective, intensive home based physiotherapy programmes which retrain and re-educate muscles in stroke patients. Microsoft Kinect might be the answer as it allows intensive stroke rehabilitation to take place in the home environment as well as taking pressure off health providers as patients can perform this type of rehabilitation on their own with remote supervision.
The ICF model and Virtual Rehab
The International Classification of Functioning, Disability and Health (ICF) is a framework for describing and organising information on functioning and disability. It provides a standard language and a conceptual basis for the definition and measurement of health and disability.
Aims of the ICF are:
- “To provide a scientific basis for understanding and studying health and health-related states, outcomes and determinants.
- To provide a standard language and framework for the description of health and health-related states.
- It defines components of health and some health-related components of well-being (such as education and labour).”
Individuals Perspective on Technology
The diagram below represents individuals perspectives towards the use of technology. Patient's views can be very powerful in the healthcare market and their voice should be heard. Patient's feedback in delivering services and bringing about change is highly important. The patient's experience framework should be considered here as it highlights respect for patient-centred values, preferences, and expressed needs.
From the patient's viewpoint, decisions about whether or not to comply with rehabilitation programmes using technology are rational but often cannot be predicted by therapists. Understanding the patient will help with compliance to rehabilitation programmes that suits their needs and abilities. Educating the patient about the effectiveness of Microsoft Kinect as an alternative method to stroke rehabilitation will help relieve any concerns or worries that they may have.
Microsoft Kinect’s potential future in stroke rehabilitation: Emerging evidence
The Kinect has shown much potential for use in stroke rehabilitation but evolving evidence highlights its prospective use into specific areas of rehab for stroke. Below are examples of research into the effectivenss of Kinect and its potential use in different aspects of rehabilitation for stroke patients.
Facial Paralysis Rehabilitationstroke cases that occur in the UK each year, around 16% of them will have long standing facial weakness. There is an increasing need, by the NHS, to provide cost effective community rehabilitation to these patients. Facial weakness rehabilitation has been investigated using the Microsoft Kinect by a team from Nottingham Trent University and the evidence suggests that it can be a useful device for this use.
- Patients can use Kinect to complete physiotherapy regimes by copying on-screen exercises, while doing so the system will compare the patient’s facial expression with that of a range of preconfigured facial poses..
- The patient will be guided through the exercises by an on-screen pre-recorded voice of a real therapist and feedback will be provided to the patient on how well they completed the exercises.
- An exercise programme can be individualised for each patient, whereby exercises might include alternating between ‘oo’ and ‘ee’ mouth shapes.
- The Kinect will recognise differences between an undesirable one-sided unilateral movement and a desired symmetrical one, across both sides of the face.
- An exercise programme can be individualised for each patient, whereby exercises might include alternating between ‘oo’ and ‘ee’ mouth shapes.
- Shows where the patients needs to concentrate their efforts.
Upper Extremity Rehabilitationstroke due to upper limb deficits occurring in over 66% of patients early after stroke. The effects can be highly debilitating which is most evident when there is a loss of manual dexterity. The use of therapy using Kinect-based systems maintained or improved the patient's motor performance.
- Most studies have only investigated mild to moderately impaired patients and have emphasized that it was particularly beneficial to those who have recovered significant amounts of motor functionality leading to question it's effectiveness in the use with patients at different levels of recovery.
- However one study did include patients who were severely impaired and also mild to moderate cognitively impaired which still resulted in beneficial results from the majority of participants. By including a greater range of patients this increased the external validity and generalizability so assumptions can extend further beyond mild to moderately impaired populations.
- The Kinect has been found to be beneficial in promoting recovery for hemiplegic upper limbs of subacute stroke patients at 3 weeks and continued to significantly increase at 12 weeks after training.
- Kinect provides opportunities for gross arm movements exercises, but fine motor control movements of the hand and fingers are more difficult to track.
- Another study looked hemiplegic stroke survivors who were beyond 6 months since onset in the use of Kinect along with conventional therapy.
- Results showed that all participants who took part improvement in all aspects of arm function including range of motion, Fugl-Meyer assessment and the Box and Block test.
Balance and Posture Rehabilitationstroke survivors may suffer from static and/or dynamic balance abnormalities. Muscle weakness, impaired sensation, and ankle proprioception were associated with abnormal postural sway and balance difficulty after stroke affecting their independence. Balance rehabilitation is possible in the stroke population using the Kinect:
- The patient stands in front of a screen while his/her movements are monitored using Kinect.
- A pair of virtual shoes is displayed on the monitor following the patient's feet movement.
- The results of this study showed significant improvements in balance recovery as measured by the Berg Balance Scale (BBS).
- Virtual reality based telerehabilitation interventions are capable both in clinic and out of clinic (at a patient’s home) of achieving re-acquisition of loco-motor skills.
- In a group of chronic stroke patient improvements were made after 12 weeks of tele-rehabilitation using the Kinect in the BBS, the gait and balance components of the Tinetti’s scale and the Brunel Balance assessment.
- Thus indicating that balance rehabilitation using the Kinect in stroke patients is feasible and proving that at home rehabilitation is just as effective as in clinic.
- The use of Microsoft Kinect also demonstrated improvement in postural awareness and balance through the repetitive nature of the virtual rehabilitation exercise programme.
- Kinect can aid stroke patients to correct and improve their posture by playing games which require them to adapt to the posture on the screen.
Gait Rehabilitationgait identified by research observed individuals post-stroke and discovered spatial and temporal asymmetry during the gait cycle. This may result in high levels of energy expenditure and increase the risk of falls to the patients.
- Microsoft Kinect can help address these deficits in functional mobility via stepping and balance during the gaming rehabilitation programme.
- It has been shown to be useful tool for prescribing feedback-assisted gait retraining in both home and clinic settings.
- A 3D motion analysis was previously used to quantify movement patterns and joint range of motion, however it is expensive, time consuming and difficult to set up for a large number of clients.
- Weak descending neural pathways to ankle muscles can also be rehabilitated by stepping over objects during rehabilitation sessions using the virtual rehab which also will help patients with stride length and hip joint range of motion for sides to become more symmetrical helping improve balance and centre of mass.
- There is evidence to support the existence of an entrainment effect of visual rhythmic cuing on hemiparetic gait which could negatively impact on the quality of activities of daily living.
- Thus, restoration of gait symmetry is an important goal in stroke rehabilitation and can be facilitated using the Microsoft Kinect in stroke rehabilitation.
- Can examine the contributions of abnormalities of gait biomechanics and improve strength and balance during rehabilitation sessions.
- It is possible to analyse and give feedback on patient’s ability to utilize both sides of their body in virtual space.
- Traditionally neglect rehabilitation was possible through paper-and-pencil activities which only utilities static and 2D targets.
- Kinect is capable of providing a virtual environment which requires dynamic responses to the relevant stimuli (i.e the games) which are always changing, replicating real life.
- The use of moving stimuli is extremely important in modulating a patient’s visual attention.
- The objects provided in the Kinect games are capable of driving a patient’s attention to the area of space it is located in.
Proprioception Rehabilitationproprioception, sensori-motor virtual rehabilitation training programmes have been suggested to facilitate joint position sense and dynamic joint stability using rhythmic active motion and angle repositioning to stimulate muscular co-activation . The Kinect has been found to be beneficial in improving proprioception.
- Visual feedback of the virtual objects/exercise may be manipulated and are more suitable for the proprioception rehabilitation.
- A positive effect has been demonstrated on proprioception of upper limbs for stroke clients using the developed virtual rehabilitation proprioceptive therapy.
- Provides client interaction between virtual objects and motion using motion tracking.
- Client can confirm their own movement without the assistance of a therapist which will help improve proprioception.
Research suggests the following criteria should be taken into account prior to commencement of Microsoft Kinect based rehabilitation:
- Cognition test performed by the patient is a prerequisite to virtual rehabilitation programme.
- Participants with a mini-mental state examination score of > 20 were able to use Kinect effectively.
- Patients with mild to moderate cognitive deficits are capable of using Kinect. However, clinicians must be wary of any cognitive deficits that patients may have and severity.
- Patients do not need full lower limb function and strength as therapy can be conducted in sitting.
- Those who cannot stand independently are capable of using Kinect and is feasible whilst using a stand aid.
- Patients’ need a certain level of gross movement so that the sensor is able to detect any movements made.
- Having a Chedoke-McMasters Arm score of 3 and over is acceptable for using all Jintronix games.
- Patients with neglect are capable of using Kinect.
- Those suffering from visual impairments, dizziness and vertigo may not be suitable for safety reasons.
- Patients with extreme dyspraxia are not suitable due to major difficulties with coordinated movement.
As always informed clinical decisions need to made before selecting who should use Microsoft Kinect, as with all forms of treatment. Clinical reasoning is vital when deciding upon what games and rehabilitation options should be used to individualize treatment.
Protocol of Patient Assessment
A protocol on how Microsoft Kinect could be implemented for stroke rehabilitation has been proposed. A general understanding of this protocol on how Microsoft Kinect can be applied is outlined below.
For every new patient an assessment will be carried out by a physiotherapist who has received training in the use of the Microsoft Kinect rehabilitation programme. The assessment will involve inputting data about the patient's weight, height, the range of motion in his/her limbs and body parts lengths. The range of motion and length of the body parts can be assessed by getting the patient to perform different exercises while the Kinect detects the measurements in 3 planes- frontal, sagittal and transverse. Obtaining accurate measurements will help provide information about the maximum flexion and extension ranges for specific joints and will make sure the patient carries out these exercises in a safe range to optimize efficiency of the rehabilitation programme.
Rehabilitation programme design
An individualized rehabilitation programme will be designed by the physiotherapist by selecting exercises that are provided on the system. This programme will consist of a set of multiple exercises that will target specific joints or muscle groups. Due to the nature of stroke the left and right hands for example, can have different impairments which would lead to different exercises being provided to help achieve specific patient goals. The physiotherapist will choose the appropriate amount of sets and repetitions for each exercise to be performed depending on the impairments of the patient.
Once all the equipment has been set up in the patient's home, the physiotherapist will go through the exercises that they would like the patient to perform.The patient will be positioned at least 1.2m away from the camera. To begin the patient selects the “Start Rehab” button and the system prepares the individualized programme designed by the physiotherapist. As soon as the first exercise starts, there will be a message on the screen saying what to do. If it is an elbow extension exercise the message will say “Please straighten your (left/right) elbow as much as possible”. Instructions for each exercise will also be read out loud by the system (using Microsoft text to speech API). While performing the exercise the patient will get visual feedback on the progress. If the exercises is not performed correctly, the repetition counter will stop until the user starts doing the exercise correctly and instructions will be provided on how to perform the exercise appropriately. The physiotherapist will track the patients progress from the statistics that is stored on the system. Follow up will be done by phoning the patient on a weekly basis to keep up to date with their progression with the rehabilitation programme.
Advantages & Disadvantages for Physiotherapists
| Provides instant feedback to the patient. It is capable of identifying when the patient is carrying out the exercises correctly, warning him/her otherwise and also saving the statistics in a report for a further professional analysis.
|| The traditional physiotherapy approach involves significant physical contact with the patient and the physiotherapist making it difficult for some treatments to be applied through Microsoft Kinect.|
| Individual monitoring for each patient.
|| Physiotherapists will require training to learn how to use the Kinect.|
| Evaluation and graphical analysis of patient's progress - allowing for progression or adaptation.
|| More research needs to be done to realise the full potential of the Kinect with respect to rehabilitation in stroke patients. |
| Provides objective rehabilitation measurements.
|| Clinics will have to purchase the Kinect sensor if they do not have already have one.|
| Quick and efficient record keeping.
| Decreasing client need for clinic appointments - decreasing waiting list times
Skills and Knowledge Required
- Training will be necessary for the use of Microsoft Kinect software, this will include:
- Navigating the software screens
- Learning how to use Program Manager
- To create personalised exercise programmes to suit individual needs
- To analyse statistics received from the softwares
- Setting up the system in clients home
- Be able to clinically reason which patients are suitable for Microsoft Kinect rehabilitation
- Be competent in using Microsoft Kinect as a rehabilitation tool for stroke patients.
- The physiotherapist should be willing to adapt and use Microsoft Kinect as an alternative method of rehabilitation
- Keep up to date with continuous professional development.
As with all treatment options informed clinical decisions have to made as to the appropriate treatment selection. The Microsoft Kinect is no different and still requires therapists to utilize their knowledge and clinical reasoning skills when choosing who would be an appropriate candidate. This will most likely become easier once immersed in an environment that utilities Microsoft Kinect. Careful selection of appropriate games and a difficulty level appropriate for the individual would need to be made. Remember there are numerous games which aim to rehabilitate both upper and lower limbs, static & dynamic balance, weight transfer, coordination, flexibility, posture, trunk control and many more.
Microsoft Kinect is a device which has been utilized to allow virtual rehabilitation differing from more conventional therapy in its ability to provide more innovative and exciting ways to rehabilitate. It can also allow for remote rehabilitation where clinicians are still able to monitor patient progress. The Kinect is continuing to advance, update and develop into a very feasible option for rehabilitation in the stroke population.
Stroke occurs approximately 152,000 times a year in the UK and is the largest single cause of severe disability. As the aging population continues to increase so too will the incidence of strokes. This will cause an increased strain on the healthcare service to provide adequate rehabilitation to these people. A large majority of patients are discharged from hospital and fail to continue their home exercise programme leading to poorer patient outcomes and decreased quality of life. One way to tackle this is to introduce Microsoft Kinect which aims to increase motivation and adherence but still promotes repetitive functional movements. A number of patients could benefit from this not just in the UK, but all over the world.
Two of the major software companies utilizing Microsoft Kinect, VirtualRehab and Jintronix, have over 12 clinical trials between them that are currently underway, in order to find out the suitability and rehabilitation options of the software in the stroke population. Results will help to provide guidelines for clinicians on what difficulty levels (of each of the games) can be used for patients with differing levels of impairments and also guidelines on patient suitability. This will also provide more evidence backing the use of these software in the near future as they continue to develop.
- Haas D, Phommahavong S, Yu J, Krüger-Ziolek S, Möller K, Kretschmer J. Kinect based physiotherapy system for home use. Current Directions in Biomedical Engineering 2015;1(1):180-3.
- Microsoft. Skeletal Tracking. 2015. Available from: https://msdn.microsoft.com/en-us/library/hh973074.aspx (accessed 26 October 2015).
- Webster D, Celik O. Systematic review of Kinect applications in elderly care and stroke rehabilitation. Journal of neuroengineering and rehabilitation 2014;11(1):108.
- Microsoft. UK Developers. Available from: http://www.microsoft.com/en-gb/developers/articles/week05may14/how-the-kinect-azure-and-gamification-are-helping-virtualrehab-to-rehabilitate-patients-worldwide/ (accessed 5 November 2015).
- National Institute for Health and Care Excelance. Stroke rehabilitation in adults. Available from:https://www.nice.org.uk/guidance/CG162/chapter/Key-priorities-for-implementation#the-core-multidisciplinary-stroke-team (accessed 5 November 2015).
- Hondori, H., M. and Khademi, M. A Review on Technical and Clinical Impact of Microsoft Kinect on Physical Therapy and Rehabilitation. Journal of Medical Engineering 2014;16.
- Lardinois, F. Vinod Khosla. The Next 10 Years, Data Science Will Do More For Medicine Than All Biological Sciences Combined. Available from: http://techcrunch.com/2013/09/11/vinod-khosla-in-the-next-10-years-data-science-will-do-more-for-medicine-than-all-biological-sciences-combined/ (accessed 13 November 2013).
- Kristof, B. The power of sensors in health healthcare. Available from: http://www.slideshare.net/micvlaanderen/mic-sensors-1 (accessed 13 November 2014).
- TechTarget. Kinect. Available from: http://searchhealthit.techtarget.com/definition/Kinect (accessed 3 November 2015).
- Fekri, A. Microsoft makes Xbox One Compatible to PC. Available from: http://masterherald.com/microsoft-makes-kinect-for-xbox-one-compatible-to-pc/14107/ (accessed 3 December 2015).
- Kinect for windows blog. Available from: http://blogs.msdn.com/b/kinectforwindows/archive/2012/01/09/kinect-for-windows-commercial-program-announced.aspx (accessed 4 November 2012).
- Zhang Z. Microsoft kinect sensor and its effect. IEEE multimedia 2012;19(2):4-10.
- University of Southampton. Re-education after stroke. Available from: http://www.southampton.ac.uk/weareconnected/story/ re-education-after-stroke/ (accessed 27 October 2015).
- Fairhead.H All about Kinect. Available from: http://www.i-programmer.info/babbages-bag/2003-kinect-the-technology-.html (accessed 3 November 2015).
- Jintronix. Available from: http://www.jintronix.com (accessed 25 October 2015).
- Virtualware Group. VirtualRehab. Available from: http://www.virtualrehab.info/ (accessed 13 November 2015).
- Living Labs Global. Virtual and remote rehabilitation platforms. Available from: http://www.llga.org/SolutionHistory?sid=58 (accessed 26 October 2015).
- Digital AV. Machinery 3.0 adds new features and a robotic arm for rehabilitation of fine motor. Available from: http://www.digitalavmagazine.com/en/2015/06/12/machinery-3-0-adds-new-features-and-an-arm-robot-for-rehabilitation-of-fine-motor/ (accessed 29 November 2015).
- OrthoRent. VirtualRehab [internet] 2015 [cited 3rd December 2015] Available from: http://ortorent.ru/product/virtualrehab-virtualnaya-reabilitaciya
- Gilpin L. Gaming healthcare: how Microsoft Kinect is revolutionising the future of rehab. Available from: http://www.techrepublic.com/article/gaming-health-care-how-microsoft-kinect-is-revolutionizing-the-future-of-rehab/ (accessed 25 October 2015).
- Fitness Gaming. New stroke recovery study launched in Tasmania with Jintronix [Internet] 2015. [cited 14th November 2015] Available from http://www.fitness-gaming.com/news/markets/health-and-rehab/new-stroke-recovery-study-launched-in-tasmania-with-jintronix.html#.VkdIC7fhDIU
- Bass, D. For Stroke victims, Jintronix software turns rehab into a game [Internet] 2014. [cited 13th November 2015] Available from: http://www.bloomberg.com/bw/articles/2014-05-08/for-stroke-victims-jintronix-software-turns-rehab-into-a-game#r=nav-r-story
- Bird ML, Cannell J, Callisaya M, Moles E, Smith S. A single case study using Jintronix software for stroke rehabilitation and Kinect motion tracking for physical rehabilitation using a putt to stand aid and standby table. In: Smart Strokes 2014 Conference 2014;9:24.
- Archambault PS, Norouzi NG, Kairy D, Solomon JM, Levin MF. Towards Establishing Clinical Guidelines for an Arm Rehabilitation Virtual Reality System. In: Replace, Repair, Restore, Relieve–Bridging Clinical and Engineering Solutions in Neurorehabilitation 2014:263-270.
- Tao G, Archambault PS, Levin M. Evaluation of a virtual reality rehabilitation system for upper limb hemiparesis. Virtual Rehabilitation (ICVR) International Conference. IEEE Xplore 2013:163-165.
- Giles, J., Elevate your used experience [Internet]. 2014 [cited 03 Dcember 2015]. Available from: http://marketing-mob.com/elevate-user-experience/
- Lange B, Chang CY, Suma E, Newman B, Rizzo AS, Bolas M. Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor. In: 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society 2011:1831-1834.
- Microsoft. Kinect for Xbox One. Available from: http://www.xbox.com/en-GB/xbox-one/accessories/kinect-for-xbox-one#fbid=ixU-gozjLIq (accessed 28 November 2015).
- Scott, R. Kinect’in Therapy. Disabled access to the Xbox kinect - hardware and games. Available from: http://kinect-therapy.blogspot.co.uk/ (accessed 30 October 2015).
- Obdržálek Š, Kurillo G, Ofli F, Bajcsy R, Seto E, Jimison H, Pavel M. Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society 2012:1188-1193.
- Mobini A, Behzadipour S, Saadat M. Test–retest reliability of Kinect’s measurements for the evaluation of upper body recovery of stroke patients. Biomedical engineering online 2015;14(1):75.
- Bonnechere B, Sholukha V, Jansen B, Omelina L, Rooze M, Van Sint Jan S. Determination of repeatability of kinect sensor. Telemedicine and e-Health 2014;20(5):451-3.
- Chang YJ, Chen SF, Huang JD. A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in developmental disabilities 2011;32(6):2566-70.
- WHO. Stroke, cerebrovascular accident. Available from: http://www.who.int/topics/cerebrovascular_accident/en/ (accessed 12 October 2015).
- Medquarterly. Available from: http://www.medquarterly.com/mq88/MQPDF/MM/OxfordStrokeClassification.pdf (accessed 3 December 2015).
- Smith M. An introduction to stroke. [lecture] Edinburgh: Queen Margaret University 2015.
- Stroke Association. State of the Nation Stroke Statistics. Available from: https://www.stroke.org.uk/sites/default/files/stroke_statistics_2015.pdf (accessed 10 November 2015).
- Ekstam L, Johansson U, Guidetti S, Eriksson G, Ytterberg C. The combined perceptions of people with stroke and their carers regarding rehabilitation needs 1 year after stroke: a mixed methods study. BMJ open 2015;5(2):e006784.
- Dobkin BH. Rehabilitation after stroke. New England Journal of Medicine 2005;352(16):1677-84.
- Iosa M, Hesse S, Oliviero A, Paolucci S. New technologies for stroke rehabilitation. Stroke research and treatment 2013;2013.
- McBean D, Van Wijck F. Perceptuo-motor control. In: Applied neurosciences for the Allied Health Professions. Churchill Livingstone Elsevier. Britain 2013;65-109.
- National Institute for Health Research. Using the Microsoft ‘Kinect’ as a stroke rehabilitation tool. Re-appropriation of non-commodity electronics devices for use in facial paralysis rehabilitation. Available from: http://www.nihr.ac.uk/funding/funded-research/funded-research.htm?postid=1905 (accessed 16 November 2015).
- SIGN. Management of patients with stroke: Rehabilitation, prevention and management of complications, and discharge planning. A national clinical guideline. Available from: http://www.sign.ac.uk/pdf/sign118.pdf (accessed 12 November 2015).
- Wiemeyer J, Kliem A. Serious games in prevention and rehabilitation—a new panacea for elderly people? European Review of Aging and Physical Activity 2012;9(1):41.
- Australian Physiotherapy Rehabilitation. Telerehabilitation. Available from: https://www.physiotherapy.asn.au/DocumentsFolder/Advocacy_Background_Papers_Telerehabilitation.pdf (accessed 13 November 2015).
- Stroke Association. State of the Nation Stroke Statistics. Available from: https://www.stroke.org.uk/sites/default/files/stroke_statistics_2015.pdf (accessed 10 Novemver 2015).
- World Health Organization. International classification of functioning, disability and health: ICF. Geneva: World Health Organization; 2001.
- World Health Organisation (WHO). International Classification of Functioning, Disability and Health (ICF). Available from: http://www.who.int/classifications/icf/site/icftemplate.cfm?myurl=introduction.html (accessed 3 December 2015).
- Weiss, Patrice. Virtual reality in neurorehabilitation. Available from: http://ebooks.cambridge.org/chapter.jsf?bid=CBO9780511545078&cid=CBO9780511545078A028 (accessed 20 November 2015).
- CSP. Patient experience matters. Patient experience framework. Available from: http://www.csp.org.uk/frontline/article/patient-experience-matters (accessed 13 November 2015).
- Campbell R, Evans M, Tucker M, Quilty B, Dieppe P, Donovan JL. Why don't patients do their exercises? Understanding non-compliance with physiotherapy in patients with osteoarthritis of the knee. Journal of Epidemiology & Community Health 2001;55(2):132-8.
- Breedon P, Russell A, Logan P, Newell O, O'Brien B, Edmans J, Baines D, Hall P. First for Stroke: using the Microsoft'Kinect'as a facial paralysis stroke rehabilitation tool. International Journal of Integrated Care (IJIC) 2014;14.
- Blackburn PM. Stroke patients using video game technology to learn to talk again. Available from: http://www.nottinghampost.com/Pioneering-research-transform-recovery-stroke/story-23207091-detail/story.html (accessed 17 November 2015).
- NHS, National Institute for Health Research. 2014. Available from: http://www.nihr.ac.uk/funding/fundingdetails.htm?postid=1905 (accessed 12 November 2015).
- Cattaneo L, Pavesi G. The facial motor system. Neuroscience and Biobehavioral Reviews. 2014;38:135–159.
- Brunner I, Skouen JS, Hofstad H, Strand LI, Becker F, Sanders AM, Pallesen H, Kristensen T, Michielsen M, Verheyden G. Virtual reality training for upper extremity in subacute stroke (VIRTUES): study protocol for a randomized controlled multicenter trial. BMC neurology 2014;14(1):186.
- Acosta IP. Upper limb rehabilitation of stroke patients using kinect and computer games [dissertation]. School of Computing, University of Utah. 2012.
- Shires L, Brown DJ, Sherkat N, Lewis J, Standen PJ. Evaluating the Microsoft Kinect for use in upper extremity rehabilitation following stroke as a commercial off the shelf gaming system. In: Proc. 10th Intl Conf. Disability, Virtual Reality & Associated Technologies 2014:197-204.
- Laver K, George S, Thomas S, Deutsch JE, Crotty M. Cochrane review: virtual reality for stroke rehabilitation. European journal of physical and rehabilitation medicine 2012;48(3):523-30.
- Sin H, Lee G. Additional virtual reality training using Xbox Kinect in stroke survivors with hemiplegia. American journal of physical medicine & rehabilitation 2013;92(10):871-80.
- Clark RA, Pua YH, Fortin K, Ritchie C, Webster KE, Denehy L, Bryant AL. Validity of the Microsoft Kinect for assessment of postural control. Gait & posture 2012;36(3):372-7.
- Clark RA, Pua YH, Fortin K, Ritchie C, Webster KE, Denehy L, Bryant AL. Validity of the Microsoft Kinect for assessment of postural control. Gait & posture 2012;36(3):372-7.
- Wiederhold B, Riva G. Balance recovery through virtual stepping exercises using Kinect skeleton tracking: a followup study with chronic stroke patients. Annual review of cybertherapy and telemedicine 2012: Advanced technologies in the behavioral, social and neurosciences 2012;181:108-12.
- Lloréns R, Noé E, Colomer C, Alcañiz M. Effectiveness, usability, and cost-benefit of a virtual reality–based telerehabilitation program for balance recovery after stroke: A randomized controlled trial. Archives of physical medicine and rehabilitation 2015;96(3):418-25.
- Lee G. Effects of training using video games on the muscle strength, muscle tone, and activities of daily living of chronic stroke patients. Journal of physical therapy science 2013;25(5):595-7.
- Guerrero C, Uribe-Quevedo A. Kinect-based posture tracking for correcting positions during exercise. Studies in health technology and informatics 2013;184:158-60.
- Clark RA, Pua YH, Bryant AL, Hunt MA. Validity of the Microsoft Kinect for providing lateral trunk lean feedback during gait retraining. Gait & posture 2013;38(4):1064-6.
- Kitatani R, Ohata K, Aga Y, Mashima Y, Hashiguchi Y, Wakida M, Maeda A, Yamada S. Descending neural drives to ankle muscles during gait and their relationships with clinical functions in patients after stroke. Clinical Neurophysiology 2016;127(2):1512-20.
- Cho S, Ku J, Cho YK, Kim IY, Kang YJ, Jang DP, Kim SI. Development of virtual reality proprioceptive rehabilitation system for stroke patients. Computer methods and programs in biomedicine 2014;113(1):258-65.
- Giuseppe RI. A virtual reality platform for assessment and rehabilitation of neglect using a kinect. Medicine Meets Virtual Reality 21: NextMed/MMVR21 2014;196:66.
- Pedroli E, Serino S, Cipresso P, Pallavicini F, Riva G. Assessment and rehabilitation of neglect using virtual reality: a systematic review. Frontiers in behavioral neuroscience 2015;9:226.
- Smith, S.T., and Schoene, D. The use of exercise-based videogames for training and rehabilitation of physical function in older adults: current practice and guidelines for future research. Aging Health. 2012; 8:3, 243–252.
- Palade N. Autonomous Stroke Rehabilitation with Microsoft Kinect [dissertation]. KTH Computer Science and Communication Machine Learning. Stockholm. 2013
- Clark RA, Vernon S, Mentiplay BF, Miller KJ, McGinley JL, Pua YH, Paterson K, Bower KJ. Instrumenting gait assessment using the Kinect in people living with stroke: reliability and association with balance tests. Journal of neuroengineering and rehabilitation 2015;12(1):15.