[00:05] spk_0: Welcome to Science Pods where researchers share the stories of their science. If you want to share your research with the world, just visit science pods dot com to create your podcast. Science pods is free and we have automated podcast creation. So you can create a podcast in just a few minutes and start sharing the stories behind your science.

[00:31] spk_1: In this podcast, we hear from Eleftherios Kolle about recent research on real world data in support of early outpatient treatment protocols for COVID-19. To get started. We were curious to hear Aleia story of becoming a researcher.

[00:51] spk_2: Hello. My name is a Gule. I'm a professor at the School of Mathematical and Statistical Sciences at the University of Texas Rio Grande Valley. I received my bachelor's degree in Applied Mathematics from the California Institute of Technology and my master's and phd degree at the University of Washington also in the area of applied mathematics. My research for most of my academic career has been in the area of applied mathematics and more specifically in fluid dynamics um with sub areas hydrodynamic turbulence research and research in geophysical fluid dynamics. I've also published a paper in theoretical physics and I have taken an interest in mathematics education. I have several publications also in mathematics education and with the COVID-19 outbreak, uh what happened is at the beginning of 2020 my research attention shifted towards uh understanding uh the disease and the treatment protocols for the disease. I was particularly intrigued by the work of uh Doctor Vladimir Zenko and also Doctor Peter Macala on um early outpatient treatment protocols for COVID-19. And particularly what really caught my attention was the initial paper that was published by Vladimir Zenko, co-authored with uh Roland Durwin and Martin Schulz, in which he presented the preliminary results of uh his um of uh his um treatment of COVID-19 patients. And he, he said in several interviews that he planned to do a follow up study. And unfortunately, he never really got around to doing that. Uh But what really caught my attention was that when I first read his papers, I realized I didn't, I didn't really have sufficient mathematical knowledge in the area of statistics to actually understand the results in depth. OK. So superficially was that I couldn't understand what was going on, but I had some blind spots and I thought to myself, well, hell, if you're a mathematics professor, you know, how come you're reading a paper like this on the, the part that you do not understand is the mathematics. So I, I took a deep dive into learning the statistical methodologies. And then I also At the same time, I took a deep dive into reading more papers on COVID-19 medical research. And as often happens when you immerse yourself in a, in a new area and you just absorb and soak up all this knowledge and just learn all those things. Eventually, some idea will click in your head. And that's what happened that led into the publication of uh a research paper titled Statistical Analysis Methods Applied to early outpatient COVID-19 treatment case series data, which I co-authored with uh Vladimir Zenko and Peter Macala. The work for that paper started in um at the beginning of 2021 we worked on it for the the, the whole year on and off because at the same time, I was busy recording lectures for my classes. The paper was published in August 2022. And um it was a very amazing experience. It really felt like being a graduate student again where you just dive in to a research project where you know a completely new thing. And um it was a very exciting experience.

[04:46] spk_0: Research studies rarely start out with the questions readily in hand from the science lab to the pub the paths that researchers take leading up to a study are as unique as the researchers themselves. So we asked, how did this study come about

[05:07] spk_2: the goal of early outpatient treatment of COVID-19 is to prevent hospitalizations and deaths? Now, Doctor Vladimir Zenko began treating patients with his Zenko protocol in March 2020. And what he did in particular was that he risk stratified his patients into high risk versus low risk. He did this empirically and the criteria that he used is that he classified as high risk patients that were older than 60 or patients with comorbidities or high body mass index, higher than 30 or uh patients with shortness of breath. So there were three categories of uh high risk patients and for patients in either one of those three categories, he treated them with a triple drug therapy that consisted of hydroxychloroquine ayin and zinc. And for low risk patients, he provided supportive care. So, so he found that low risk patients did fine uh without uh the treatment, but the treatment was necessary for the high risk patients. Now, what he observed in terms of results was that in April 28th, 2020 he had treated 1450 patients of which 405 were high risk patients. And the result uh the outcomes were he had two deaths and six hospitalizations. And then by June 15th, 2020 he had seen 2200 patients of those 800 were high risk patients. And he his outcomes were still uh cumulatively uh two deaths and 12 hospitalizations from the beginning from March 2020. Now, over time, of course, uh during this period of time, Dr Zenko improved his treatment protocol by introducing uh steroid medications like dexamethasone and budesonide. So he did this, uh, in the beginning of May 2020 he also introduced blood thinning medications, uh, in, um, the towards the end of May 2020 and beginning of June 2020. And uh parallel to Zenko Air Force. Uh, Doctor Peter Macala also introduced his Macao Protocol and he published two research papers on that in, uh June 2020. And his second paper was published on December 2020. And Doctor Macao's approach was that uh he recognized that the illness has three stages. The first stage is viral proliferation where the virus makes copies of itself, then hyperinflammation and cytokine injury. That's the second stage. And the third stage is thrombosis where you have micro clotting that leads to oxygen de saturation. So his approach was to provide uh to, to treat with medications each phase of the illness in the sequenced manner. Hopefully, if you treat early enough, you nip it in the bud and it doesn't go beyond viral proliferation. But if the case worsens and persists the 2nd and 3rd stage, you can keep, you know, treating it with different medications, addressing the the dynamic of each stage, uh hoping, aiming towards a good outcome. Now, the problem was that uh when Doctor Zenko initially contacted uh the US administration and tried to present the results to them, they had no way of assessing uh his uh real world data. They said we need a randomized controlled trial to assess whether or not this is working and they didn't really have any mechanism for assessing this type of real world evidence. So the research question that came to me was, can is it possible to develop a statistical methodology for doing that

[09:08] spk_1: to understand the findings of any research? You have to begin with the methods applied by the researcher. With this in mind. We asked the Laos how this study was conducted

[09:24] spk_2: at this time. There are no randomized controlled trials of the complete Zenko and Macao protocols. There are no plans to conduct any such trials in the future. And there's even questions of whether it is ethical to conduct such trials. Now, randomized controlled trials are appropriate when you are uh experimenting with a completely new treatment uh that has never been used before where your expectation is that it could have a negative outcome or a neutral outcome or a positive outcome. So that could be the case when you have created a completely new drug, a medicine that has never been used before or a new vaccine, for example, that has never been used before. Now, the situation we have is actually different. We're looking at treatment protocols that use repurposed medications. These are medications that have been used before. They have a known safety profile, the safety profile is acceptable, they have acceptable safety. So therefore what we're concerned with possibilities. And those are that the treatment could have a neutral effect, it could be just doing nothing or it will have a positive effect. So you have to distinguish positive versus neutral effect. And all you need to do is really make a binary choice. You have to choose to either use the treatment or not use it. So the question you really want to ask is uh does the treatment work? Ok. So you don't even care about how well does it work? What's the actual efficacy? You just want to know? Yes or no. Do we have a positive outcome if we use this treatment or do we have a neutral outcome if we distinguish between one and the other? And the way we did this is that, first of all, we argue in the paper based on prior experience uh with COVID-19 for the initial period of the year 2020 that the mortality risk for high risk patients using an age 60 threat threshold is 3.5% minimum. So, meaning it's greater than 3.5% for uh using the AIDS 50 as a threshold. It is 2.2% or greater uh hospitalization risk uh for high risk patients is greater than 10%. And uh if you use the entire population as a control group, going with a case fatality rate, you could say you could argue that the prob the mortality risk for high risk patients is at least greater than 1.7% which is actually very severe, underestimate. Now, the methodology, what we actually did in the natural is that we have the number of patients of high risk patients that receive treatment. We have the number of patients with a bad outcome. Uh the bad outcome being hospitalization or death. And using those two numbers, we calculate an efficacy threshold and we calculate what's called a random selection bias threshold. Now, the uh if the let's say, take the mortality risk, now if you are expecting at least 3.5% mortality risk for high risk patients, then if the number 3.5% exceeds your efficacy threshold, you have established by the preponderance of evidence that the treatment has a positive effect. Uh Meaning it is more likely than not that it's working, which is sufficient to recommend that emergency adoption. And then if uh the number 3.5% the the more the minimum lower bound mortality risk for high risk patients is above the random selection bias threshold, that means that the evidence is now clear and convincing that the treatment has a positive effect. Now clear and convincing means that random selection bias cannot completely overturn. Uh the positive result that, you know, worst thing that could happen, there could be some selection bias that minimizes the effect but does not overturn it. And if you can cross the clear and convincing threshold, that is sufficient to recommend uh adoption of the treatment as a standard of care uh until we have more data. And of course, you continually monitor and improve what you're doing.

[13:27] spk_0: Having collected the data, we were curious to hear more about how the data were analyzed and what they showed,

[13:40] spk_2: we analyze the data by calculating the efficacy threshold and the random selection bias threshold for several case series. Now, these include uh the case series from Dr Zenko, his high risk patients until April 28th, 2020. The also the more extended, uh, cases of high risk patients by Doctor Zenko until the end of June 15th, 2020. Also, we looked at cases on Doctor Brian Proctor until September 2020 and the larger case series of high risk patients until December 2020. And we also reanalyzed the results that were published by Doctor Deer Raul in France, uh, until the end of December 2020. Now, what is interesting about all these results is that first of all, these are all high risk patients. Second, uh, the vaccines did not exist at the time. So the results are not confounded by vaccination. Also, natural immunity was holding up and throughout 2020. So you don't have to worry about maybe that some of the patients have done well because that was their second infection as opposed to the first one. And, uh, so what we did then is once we calculated the corresponding thresholds for all those cases, we looked at what the results are and uh we calculate the thresholds using 95% and 99% confidence intervals. Um So, depending on how much confidence you want, you get different thresholds and the results are as follows. So as far as hospitalization rate reduction is concerned, we have found that with using 99% confidence intervals, all case series uh satisfy the clear and convincing threshold, meaning that we can have 99% confidence that uh at least 99% confidence that the treatment reduced the rate of hospitalizations. Uh likewise, using 99% confidence interval. Uh All case series satisfy the preponderance of evidence threshold with respect to mortality rate redaction. Now, what that means is that as long as you do not have random selection bias, overturning your results, you can have 99% confidence that there was mortality rate reduction. But with a caveat that those results are susceptible to being overturned by a random selection bias except that it is more likely than not that this is not the case, meaning that it is more likely than not that the results have not been overturned by random selection bias. Now, in particular, if we use 95% confidence intervals, which is the the standard in epidemiological statistics, uh we cross over the preponderance of evidence threshold with respect to mortality rate reduction using the case series from April 2020. Uh The, if we use the June 2020 C case series, uh with 99% confidence interval, we have exceeded the, the, the clear and convincing threshold. Uh Meaning that we have clear and convincing evidence from this case series uh by June 2020 that there is mortality rate reduction. And then of course, uh using 99% confidence intervals uh from the Proctor and Raul case series. Uh we have clear and convincing evidence of mortality rate reduction at the end of December 2020. So we can say that by the end of 2020 December 2020 we have cleared uh the the clear and convincing threshold with respect to both hospitalization and mortality rate reduction. For all of those uh case series,

[17:41] spk_1: research results routinely have both expected and unexpected implications. This led us to wonder what Eleftherios believes will be the influence of this study.

[17:56] spk_2: The main takeaway from this work is that by the end of December 2020 we had clear and convincing observational evidence from Dr Zenko. Doctor Brian Proctor and Doctor Didier Raul that the respective Zenko and Macao treatment protocols uh reduced the mortality risk and the hospitalization risk for high risk patients. Now, more specifically, we can say that uh with respect to hospitalization rate risk reduction for high risk patients, the evidence was clear and convincing by the end of April 2020. So Zen data alone uh at the end of April 2020 was clear and convincing that there was hospitalization rate reduction uh effect uh by June 2020. Uh Zenko data was clear and convincing that we had the mortality rate reduction effect. And by the end of the year, we had corroboration uh by the data from Dr Brian Proctor and Dr Raul. Now, in spite of all this, uh the official government position has been that these treatments are unproven because they have not been Vaid by a randomized control trial. Now, the problem is that to conduct a randomized control trial trial, ethically, you need to have pause, you need to think you need to have a prior belief that it is equally likely that uh the treatment can have a negative versus positive effect. And in this case, we don't even think that the treatment will have a negative effect because the medications have acceptable safety and we do have clear and convincing evidence that actually there is a positive effect. So, in spite of this, unfortunately, many governments have tried to obstruct access to early treatment. Uh but there are several other governments that actually adopted it uh at the national or regional level during 2020. Those includes the governments of Algeria, Argentina, Brazil, Bangladesh, Cameroon, China, Colombia, Egypt, France, Ghana, India, Korea, Mexico, Morocco, Mozambique, Nigeria, Peru, Senegal, Taiwan, and Uganda. So all of those nations adopted uh early outpatient treatment during 2020 either at the regional or at the national level. And uh now our mathematical methods provide a method that provide a methodology for statistically analyzing the observational evidence and determining the strength of the evidence. So we can distinguish whether or not the evidence is clear and convincing uh or whether it uh we only have preponderance of evidence that there's a positive effect or uh maybe we don't have preponderance of evidence, maybe we just need to gather more data. So, so we can make these distinctions now to make these distinctions. Of course, the underlying assumption is you need to assume that you have acceptable safety. So that means the, the the methodology can only be used on treatments using repurposed medications. You cannot use this on a completely new medication. You can also not use it on the medication where you know that there is severe safety issues like let's say, for example, severe. Uh and uh that is it, I mean, uh I just have to say in conclusion that it is a great honor for me to uh work with uh Doctor Peter Macala and the late Vladimir Zenko on this work. It has been one of the most meaningful things that I have done in my life so far. And um we hope that this can then be used also to validate data that may arise from future pandemics. Thank you.

[21:57] spk_1: That was Eleftherios Kolle discussing recent research on real world data in support of early outpatient treatment protocols for COVID-19, you can learn about this research, download a copy of this podcast or read the transcript at science pods dot com. We hope

[22:13] spk_0: that you enjoyed this science pods podcast. You can listen to other podcasts at science pods dot com where you can also subscribe to get new podcasts delivered to your podcast player. If you have recently conducted research and want to share it with the world, whether it is in press on a preprint service or already published, you can create your science pod podcast in just a few minutes, just visit science pods dot com to get started.