The road less travelled

Algorithm detects hospital frequent flyers with over 90% accuracy

By Dr Eman Leung

Dr Eman Leung is a co-investigator of the CityU's first Theme-based Research Scheme project: Delivering 21st Century Healthcare in Hong Kong. He was an Assistant Professor of Management Sciences, College of Business, City University of Hong Kong until September 2018. Below, Eman describes his humbling journey across the elderly care continuum as he developed his research programme with the support of the Theme-based Research Scheme.

We constantly make assumptions about the world we live in. Every day we make assumptions about how people around us will act towards us and react to our actions. In science we make assumptions about the system we seek to study. To elaborate on Einstein's famous quote, in most cases these assumptions are wrong. But assumptions are to an extent useful, because they shape something potentially unknown into something with which we are familiar. They give us at least the illusion of a solid ground on which to stand and leap towards the future. Maybe even to plant a pivot that lets us move the earth.

Mathematical perfection vs messy reality

Nowadays we apply scientific models and algorithms to solve real-life problems. But here's the challenge: Whilst influential models are perfectly engineered by some of the best minds in history, real life is messy and in constant flux. To study reality is to peel through layers and layers of complexity – and in my humble opinion, there is no reality more complex than the one presented in our healthcare system.

In the simplest terms, a healthcare system needs to balance the care quality of the individual patient, the health of the population and total healthcare costs with the concerted effort of a continuum of service providers. But the picture is made complex because these providers serve different purposes and different mixes of health professionals, who are themselves credentialed with different kinds of qualifications and experiences.

"Assumptions are made and most assumptions are wrong"

Albert Einstein

The road less travelled

The temptation is to take a simple slice of this reality and supplement an oversimplified definition of the problem with pre-existing assumptions – rather than study the intricacy of the interdependent slices of reality.

"All models are wrong, but some are useful"

George E. P. Box

Under the leadership of Professor Frank Chen of the Department of Management Sciences, CityU's Theme-based Research Scheme (TRS) Delivering 21st Century Healthcare in Hong Kong – Building a Quality-and-Efficiency Driven System followed the road less travelled, and took on the daunting challenge of studying the multidimensionality of the healthcare system.

While most junior faculty would have been encouraged to focus on what was most immediately publishable, I had the good fortune of working under the mentorship of Professor Chen, and was encouraged to study what is scientifically and societally impactful. I am grateful to Professor Chen who, despite his dedication to mathematical rigour in a more theoretical pursuit, has shown me immense patience and trust in the long incubation period during which I struggled with the reality of engaging healthcare providers and making sense of healthcare data.

Kick-starting an academic career

Despite my experience as a data scientist in Canada and the US, and as a manager of the quality and safety portfolio for a university-affiliated hospital in Canada, I didn’t begin my career as an academic until 2014 when I relocated to Hong Kong and was recruited to the City University of Hong Kong. I was very fortunate to be able to kick-start my academic career by serving as a co-investigator on Hong Kong's first TRS in healthcare quality and efficiency. The TRS is a collaborative effort that transcends the boundaries of universities. All universities in Hong Kong take part, with the School of Public Health and Primary Care of the Chinese University of Hong Kong as our closest partner in this effort.

Leveraging data science

The TRS supported my programme of research, whose objective was to leverage different data science methodologies to address the following question: How do we assign an optimal ensemble of community, primary, acute, subacute, and outbound care services to each patient in accordance with the triple aim of our healthcare system, i.e. better care for individual patients, better health for the population, and lower cost per capita?

Together with Professor Chen and Dr Guan Jingjing, and with the help of a team of nurses and social workers managed by Ms Hera Leung, we embarked on a journey to apply data-driven operations research and analytics to study the interconnectedness of medical and social services that provide community-dwelling elderly and elderly residents of residential homes a continuum of care, and their concerted impacts on the elderly.

Sharing data and challenges

Much time and effort went into engaging the various medical and community service providers that make up the elderly care continuum. We were very privileged that different service providers along this continuum shared with us not only their data but also their challenges in providing person-centered care in a system that is under strain. Despite seemingly unsurmountable systematic obstacles, they still manage to advocate for their clients and patients. Such sharing helps us understand the context of their local clinical practices. In turn, we were committed to helping them better understand their data and operations, and to improve the quality of both. The outcome of this partnership has been fruitful, as reflected in the pipeline publications and the implementation initiatives that have been recognised by the Hospital Authority (HA) described below.

The challenge
How do we assign optimal care services to individual patients and meet the triple aim of our healthcare system?

Detecting ‘hospital frequent flyers’

In terms of research, we have submitted for publication studies addressing challenges identified by providers across the elderly care continuum. At one end of the care continuum where community-dwelling seniors require no active clinical intervention, I have applied a novel machine learning methodology to simultaneously identify through data and algorithm perturbations: 1) the cutoff that delineates high vs low hospitalisation risk given the sample and time span, 2) the highest performing machine learning algorithm in classifying individuals’ hospitalisation risk, and 3) the minimum number of indicators required to profile individuals who are ‘hospital frequent flyers’ in terms of hospitalisations. The result reveals that out of hundreds of features included in the initial pool, just 13 are sufficient to classify those who are likely to be frequently hospitalised with above 90% confidence.

AI-driven screening for fall risk

The most significant contributor to frequent hospitalisation is an event that is tragically commonplace for many elderly – falling down, and this resonates with the findings in the research literature. We therefore applied and compared the performance of different machine learning algorithms to profile elderly people with high fall risk through the identification of a minimum set of screening items. These could best predict which elderly in the community would most likely be screened positive by in-depth clinical fall risk assessments administered by health professionals, and should therefore receive appropriate intervention.

$1 invested in homecare for community-dwelling seniors can save $9 in medical costs

Moving down the care continuum, for those community-dwelling seniors who require homecare services, we foraged the data linkage between medical and social services to estimate how much community home care services can save on medical costs. Given the goal of informing real-life decision making, our economic evaluation was based on observational study conducted within the context of how services are actually delivered, rather than the controlled environment engendered in experimental trials. With the messiness that the reality brings to observational studies, however, we adjusted for the cohort's socio-demographical, clinical and functional statuses; health service utilisation history; and the individual elderly's frailty process over time in our model.

As different types of home care services simultaneously delivered to the same client impact on each other and on the medical system as a whole, our economic evaluation was conducted within the framework of generalised cost-effectiveness analysis. This enabled us to group patients according to the combination of home care services they received, and to estimate the risk-adjusted medical cost-reduction effect of each type of home care service in the presence or absence of other types of home care services, either individually or in combinations, to enable comparison with a counterfactual scenario and with each other.

Based on our analysis, $1 invest in home care services delivered in the community can save from $9 to $69 depending on age and what concurrent home care service or services they are receiving. The preliminary finding of this study was cited in the Our Hong Kong Foundation's report on the Hong Kong healthcare system, and was mentioned in a public address delivered by the Chief Executive of HKSAR as a piece of evidence supporting the government's policy in investing in community care.

Informing acute and postacute care assignments with an ensemble learning algorithm

In the acute care setting, we conducted extensive data mining on the Electronic Health Record to model the acute care cost using mixture models tailored to the clinical reality of "alternate level of care". In this model, even patients who share the same acute clinical statuses and demographic profiles are probabilistically associated with different length-of-stay alternatives as a function of population aging-specific factors. We then integrated this mixture model-based alternate level of care estimator with a dynamic joint estimator of hospital readmissions that we had developed, and whose accuracy in predicting readmission over a one-year period reached 85% in both retrospective and prospective patient samples. Finally, we created an ensemble learning algorithm that is able to recognise the patterns in what patient segment-specific ensemble of primary, acute, convalescent and outbound care services may yield the smallest (and the largest) 28-day readmission rate with above 80% accuracy.

Is residential care a revolving door to hospital?

Hong Kong ranks number one in the percentage of older people being institutionalised in residential care facilities or old age homes. At the very end of the care continuum, we are currently preparing for publication of an economic evaluation of outbound medical care to residential care home. The estimation is based on data extracted from a residential care institution to identify the utilisation of hospitals, clinics, and outbound medical services provided to the residential care institution as well as services provided locally by residential care homes.

Core members of the project: (from left) Dr Eman Leung, Dr Jingjing Guan, Phyllis Chau, Hera Leung, Alison Lee and Professor Frank Chen

An opportunity to make an impact on our aging population

The TRS has given me the mandate to make visible impacts in the community. It has been my privilege to have an opportunity to work with the statistician and clinicians at one of the largest hospitals in Hong Kong to begin implementing the ensemble learning algorithm introduced above. The goal of this collaboration is to achieve data-driven discharge planning at the point of admission to the hospital thereby addressing the grave challenges our aging population post to our healthcare system.

The chance to implement one's own algorithm in the clinical context to support actual clinical decisions is an immense privilege and a rare opportunity where the algorithms could be prospectively and repeatedly validated. Moreover, the implementation work at the hospital has caught the attention of senior management at the head office of the HA which has recently created two channels, the Data Collaboration Lab (HADCL) and the Data Competence Centre (DCC). The aim is to bring data science and AI to clinical service delivery and management to all public medical facilities in Hong Kong.

The ensemble learning algorithm we developed and piloted at the hospital got us selected by the Data Collaboration Lab as one of six inaugural project teams to access the Hong Kong-wide HA data. With the territory-wide data that are made available through HADCL, we can further improve our algorithm. It is our hope that the algorithm can eventually be implemented in selected health clusters across the whole of Hong Kong.

"Models are perfectly engineered, but real life is messy"

The true visionaries

We could never have overcome all the daunting challenges we faced in studying the complicated healthcare system and its imperfect data without the support of our medical and community partners and the conducive environment created by our Dean, Professor Houmin Yan. Professor Yan's vision and unwavering support got us where we are today. Also, we could never have achieved the balance between the scientific excellence and societal impact if not for the generous support of our donors who share both our vision and pioneering spirits in achieving an innovative, evidence-based and AI-driven solution for the healthcare problem. I feel extremely fortunate that our fundraising work has been supported by a very competent Development Office. I am particularly grateful to Ms Lolitta Wong, the development professional posted in the College, who not only provided us with valuable strategic assistance but managed to understand and communicate projects of academic complexity to our patrons, and teamed up with us to work on pitches to ensure the sustainability of the project.

Gratitude and goodbye

I am indeed immensely grateful for the generous support of the donors. Phase one of the donation project has yielded an AI-driven approach to the detection of "hospital frequent flyers" and a low-cost screening for fall risk, which are not only academically interesting but can also serve as a platform for fostering care coordination between medical and social services – a direction that my medical and social service partners and I are embarking on.

The second phase of my project was originally intended to provide intervention to elderly participants who were identified as high-risk during phase one of my project, and evaluate the impact of the intervention using a wait-list control study, but I had already left CityU before the study began. Nevertheless, I have no doubt that the new custodian of the project – and the many new directions that the project has taken in the short time since I left – will enable TRS to add value to the community.

Dr Eman Leung
Assistant Professor
Faculty of Medicine
The Chinese University of Hong Kong