Predictive Analytics: A Way to Determine Who Should or Shouldn't Have Spinal Deformity Surgery

It's a pleasure to be with you at this great, uh, great meeting, really, uh, wonderfully done. Uh, in the next 12 minutes, I'm gonna describe uh our journey, uh, both National Spine Study Group and our journey at UCSF into data and machine, uh, machine learning.

And how that's uh changing our care.

First of all, I think it's important to step back and, and go with me on this exercise of looking at Spine deformity Surgery really as a manufacturing process. And let's, let's study that manufacturing process as such. And this idea actually was given to me by one of my patients who wrote me this letter saying that I ran a God's Spine factory, which I thought was pretty funny.

Um, and this DMAC process is actually quite interesting to, to put our, uh, our care pathways through. So define, measure, analyze, improve, and control. And this is what's done all the time when you're manufacturing cars, widgets, doesn't matter.

So first, let's define our, our problem. We have an epidemic of adult deformity Surgery driven by greater expectations as we age and aging population, more surgeons being trained in these complex procedures. And this is true, especially in comparison to AIS Surgery. But, uh, the, one of the issues is we're in a severe competition and worsening competition for healthcare dollars. De Dementia is estimated to be a $1 trillion cost, cardiovascular and musculoskeletal care.

So we must innovate. Why?

Because we have nearly a 40% failure, rod Fracture failure, and the only NIH uh prospective trial. This is data that Justin Smith submitted this year to the SRS. Our PJK rates are up somewhere between 10 and 40%.

Our re-op rates are 20%.

Well, how are we doing in comparison? Well, we're doing pretty terribly. Uh, when you look at arthroplasty, they're doing a great job. You look at Other complex procedures, ASD Surgery is really failing.

When we look at the incremental cost-effectiveness ratio, we're really doing poorly. Uh, 10 year, it's only borderline cost effective in terms of its ICE, uh, at $80,000 if these cases are not re-operated on.

Technically, in the past, our ways to prevent this was to really spend more money. This is the UCSF special, we call it double pelvis, double rods, BMP, ligament repair, vertebroplasty, two surgeons, plastic Surgery. This just, we're spending more money and much of this is not reimbursed.

So let's look at how machine learning and process optimization may help us. Uh, first of all, when we compare quality, we can use predictive modeling for, uh, for more accurate benchmarking. We can eliminate human bias and human error in the decision-making process. We can supply missing structural information that can then be combined with machine learning to, to optimize accuracy. And then we have to accumulate more data. We have to figure out what, what is going on with unmeasured biologic variation.

And then we need to create large loop information infrastructure. So not just the guys that are part of the study groups can participate, but surgeons in the community can load data and learn from it. Let's look at comparative measurement error. This is from the International Spine Study Group. It used to be in the past when we benchmarked PJK for quality, we just drew an average line across the different groups, and you can see they're over and underperformers.

But when we use predictive modeling for benchmarking, we can see that when we predict based upon 50 different variables, we see that sites that seem to be underperformers were actually doing quite well, and sites that seem to be really good with their PGF rates should have actually had even lower rates. And this is done with predictive modeling to generate what the expected rates should be.

Now let's look at the elimination of human noise uh and bias. A couple of great books, those of you who know me know I read all the time, is that this book Noise about the human noise and bias in the decision-making process. This is certainly rampant in our decision making in Spine Surgery. Another great book is called Thinking in Bets. One of the things we're guilty of is this bias called resulting. So if we make a decision and we operate on someone and they do poorly, we think, oh, that was a bad decision.

But the best poker sharks in the world routinely lose hands in poker, but they did exactly, they made exactly the right bet. So we need to kind of remove this bias from our thought process. So this is where risk calculators come in. This is a combined effort between ISSG and ESSG, and essentially we looked at, gave surgeons a bunch of different cases, and we said, what is your estimate of the 90 day complication rate or the 2 year complication rate?

And you can see if you look at some of those cases, that the variation goes from 1 to 100% in terms of what the surgeon was predicting for the risk of that operation. So tremendous bias and, and even noise if you gave the same case to the same surgeon on different days, they would say different things. The index of agreement between the machines and humans is not very good.

But when we look at the machines, they were right on. This is a validation we did in 90-day complication at UCSF. The machines were perfect, really in their prediction of the complication rates.

Also, the utilization of machines can really enhance our decision making. So in the past, the surgeon was sort of left to his own to look at a bunch of different variables, and he would measure the patients. Now this is how we can do it with machines. We can use what's called unsupervised clustering to look at patient patterns. We can generate the most similar patients around our patient, and we can look at how those cases did. We can look at how surgeons around the world treated those patients in real time immediately. This is available today.

We also can predict complication rates very accurately using numerous data points, 50 to 100 different data points, depending upon, you know, what time frame you're looking at before Surgery or in the perioperative period, and we can give surgeons these estimates and patients these estimates of these complication rates to assist in their shared decision making process. At UCSF, they actually sign their Informed Consent, which contains the machine prediction of their estimated complication rates.

We do the same for outcomes. Look at the outcome graph of the combined database of ISSG and ESSG. It's all over the place. That's in the middle.

In the past, we quoted average values. How am I going to do, doctor? I don't know. Let me look at a paper that quotes an average outcome from ISSG all over this meeting, all over SRS, their average outcome values quoted, right? But what we really want to know is what is the specific outcome for that specific patient having that specific Surgery by that specific surgeon, and that's again available today through risk calculation. So this is how it works. We cluster the patient, we generate a similar patient.

And then we enter demographic, radiographic information.

And we generate a complication prediction from 0 to 2 years, re-operation, readmission. We generate HRQOL information, and uh we can generate quality information.

So one of the interesting studies that we did for me is one of the most powerful ones, is when we put the machine up against the human, and we said, OK, if we operate according to the human, which is what the surgeon recommended, versus we operate according to what the machine recommended as to who was going to obtain MCID.

The machines beat the humans by 684% in quality gained. So anyone that doesn't think that machine learning is coming to prediction of outcome and Surgery, you're sadly mistaken. At the level, level of the economies, Healthcare Spain, NHS, US healthcare system, Medicare, we can do this now because we can accurately predict complication and outcome. We can say, OK, draw your filter according to your disposable income level, uh, as a health care carere economy for your population.

And in the US, if we drew a 50/50 complication outcome MCID filter, we save $500 million.

So this is exponentially growing across the healthcare sector. We had a great meeting on that yesterday, and we see this as the FDA approvals for AI and machine learning in the last couple of years.

One of the things that is really changing is the sagittal plane. This is data we submitted this year. It's a podium at NASA showing that correlation to the sagittal plane was essentially something that's not really true anymore. Outcomes are not linked exactly to the sagittal plane. The R2d values are vanishing by 2-year follow up. What does it do though? It predicts mechanical complication.

It predicts how this patient's gonna fail. We know that mechanical complications drive outcome to some extent, uh, at the later time points, but it's not showing up anymore in our R squares. So this is another area where we're going to see machine learning and AI come in. We have to get better at our realignments to prevent mechanical failure, not necessarily to drive outcome, which is influence, the number one Influencer of outcome is where the patient started.

I know I kind of begged that question. It's not the sagittal plane, it's what their, what their HRQOL was prior to Surgery. So we, we see this a study Justin Smith did, where we look at surgeons' ability to accurately bend rods. Machines beat them, beat the surgeons completely in their accuracy of rod bend. This creates significant error in our realignments that results in over and undercorrection. We can talk more about that in the next section. To accurately realign a patient, you have to predict the patient's reciprocal changes.

As a surgeon, we're not good at doing that. We use average values, right? This is the integration of many, many different data points to do this accurately. It's something that's probably beyond the ability of the surgeon to do this. But machines that have access to 10,000 patient databases can do this really accurately. And this is gonna result, this is gonna be, I think, in the future, the standard realignments. We're going to eliminate variation with how we do this, potentially with navigation and Robotics and placing the screws, because there's still error.

One of the Other big sources of errors, as he talked about yesterday, is the lack of mechanical force information in doing this. This is again something where AI and force prediction can be, can be very helpful because the error that's still present as we move from architects to engineers to surgeons realigning these patients, we're still missing data. Even if you take 15,000 patients.

from big databases like for example, Unit database or something like that. It doesn't include the data on the forces and stresses on those levels. So you have to add that data in to get an accurate database that you can then use for machine learning. And missing variables is going to be the, the section of the next couple of minutes of my talk as I finish up.

So we analyzed missing variables and the impact on our accuracy of our predictive models in the last couple of years. We found was even taking everything that I just showed you, we're still missing about 70% of the variables that's a driving outcome in these patients. What are they? And this is one of my favorite quotes from Donald Rumsfeld. These are, some of them are known unknowns and some of them are unknown unknowns. And again, we can take a deeper dive on this in the next section.

One of which is biology. We saw a great discussion today of all the different new biologics and bone grafts, but there's no discussion of host factors. If you're going to build a precision bone cocktail, you have to have precise information on the host, where you're using those. So we're entering into an osteoindex trial at UCSF where we're trying to build an osteoindex score that includes resorption, mineralization, osteoblast information, and making this feeding this back in a machine learning loop combined with the frailty information.

We're also not measuring transcription, biologics, and genetics. And again, we can spend more time in this in the next section. Our group's done quite a bit of work in differential aging and telomeres, which we know are a big missing variable. Uh, telomere length was the most impactful in predicting risk of major complication. This was completely missing from all of the databases, all the predictions that I showed you earlier. Uh, it was 20 sixfold in terms of the odds ratio, so.

The Other thing that we're doing is trying to develop a biological test of frailty and transcription information methylation seems to be the key here. We've partnered with a company called True Diagnostic, which used to do performance analysis and prediction on racehorses. Using True Diagnostic, we do a methylation analysis of aging called Doon and pace. This is very predictive of risk or this is very correlated well with frailty and very predictive.

as well on risk of major complication. We're looking at composite omics scores that contain proteomics, metabolomics, transcriptomics, and genetics, and these, these composite scores seem to be even more impactful in prediction. And finally, that the last area where AI and machine learning is going to help us to optimize the care of these patients is in neurom monitoring.

Just think of how stupid it is in this modern age that in, in Surgery, we have someone pushing a button and looking at the different tracings. How that was ridiculous, right? It's ridiculous. So, uh, one of the guys in our group who has a background in mathematics has been using AI to improve the sensitivity and accuracy of nerve root monitoring in these cases. And using AI we've been able to double the sensitivity and specificity.

For nerve root monitoring, the standard MEP does great for a spinal cord, but it does a terrible job for nerve roots. So that's where we're working. And then finally, last slide, we're building this huge community around us. So, so every surgeon from the, from planning to execution to outcome to putting their data back into the cloud, everyone can participate from the young surgeon just had a training to the surgeons that are part of these registers.

It used to be very exclusive. If you wanted to add to the Research and all this, you had to be part of these exclusive registries which essentially don't let people in anymore. And now it's going to be much more democratic, egalitarian, where everyone who uses these planning platforms, who uses the navigation platforms, these outcome platforms, these structural engineering platforms, everyone can enter data and participate in the, in the outcome and improving the care of these patients.

So thank you very much. Much like baseball, uh, the improvement in performance of surgeons in the future is going to be predicted and controlled through data.