| Author | Affiliation | Location |
|---|---|---|
| Antonio Remiro Azocar | Novo Nordisk | Madrid, Spain |
| Gianluca Baio | University College London | London, UK |
| Andrea Berardi | PRECISIONheor | London, UK |
| Darren Burns | Delta Hat, Nottingham | Nottingham, UK |
| Federico Rodriguez Cairoli | Triangulate Health | Doncaster, UK |
| Mark Clements | Karolinska Institutet | Stockholm, Sweden |
| Koen Degeling | GSK | Amersfoort, Netherlands |
| Padraig Dixon | University of Oxford | Oxford, UK |
| Andrea Gabrio | Maastricht University | Maastricht, Netherlands |
| Manuel Gomes | University College London | London, UK |
| Nathan Green | University College London | London, UK |
| Mathias Harrar | Technical University Munich | Munich, Germany |
| Rose Hart | Lumanity, Sheffield | Sheffield, UK |
| Anna Heath | The Hospital for Sick Children and University of Toronto | Toronto, Canada |
| David Incerti | EntityRisk, Inc. | San Francisco, US |
| Christopher Jackson | MRC Biostatistics Unit, University of Cambridge | Cambridge, UK |
| Jeroen Jansen | University of California San Francisco | San Francisco, US |
| Mi Jun Keng | University of Oxford | Oxford, UK |
| Erik Koffijberg | Universiteit Twente | Enschede, Netherlands |
| Eline Krijkamp | Erasmus University Rotterdam | Rotterdam, Netherlands |
| Felicity Lamrock | Queens University, Belfast | Belfast, Northern Ireland |
| Nick Latimer | University of Sheffield | Sheffield, UK |
| Dawn Lee | University of Exeter | Exeter, UK |
| Baptiste Leurent | University College London | London, UK |
| Alexina Mason | London School of Hygiene and Tropical Medicine | London, UK |
| Antoinette Buhle Ndweni | University of Cape Town | Cape Town, South Africa |
| James O’Mahony | University College Dublin | Dublin, Ireland |
| Petros Pechlivanoglou | The Hospital for Sick Children and University of Toronto | Toronto, Canada |
| David Philippo | University of Bristol | Bristol, UK |
| Eleanor Pullenayegum | Sick Kids Hospital and University of Toronto | Toronto, Canada |
| Mohsen Sadatsafavi | University of British Columbia | Vancouver, Canada |
| Pedro Saramago | University of York | York, UK |
| Iryna Schlackow | University of Oxford | Oxford, UK |
| Marta Soares | University of York | York, UK |
| Joshoua Soboil | Cogentia Healthcare Consulting | Cambridge, UK |
| Mark Strong | University of Sheffield | Sheffield, UK |
| Howard Thom | University of Bristol | Bristol, UK |
| Nicky Welton | University of Bristol | Bristol, UK |
| Claire Williams | University of Oxford | Oxford, UK |
R for Health Technology Assessment
Preface
The idea for the R-HTA book has arisen from the very first meetings of the R-HTA consortium, back in 2017/2018. R-HTA is an academic consortium whose main objective is to explore the use of R chiefly for cost-effectiveness analysis (CEA) as an alternative to less efficient, less generalisable and much less powerful software such as spreadsheets.
This has been a long-standing conviction for us and we have been lobbying at conferences (such as the International Society for Pharmacoeconomics, ISPOR; or the Society for Medical Decision Making, SMDM), initially as lone and nerdy statisticians and economists who appeared to be on a mission to destroy MS Excel, for some reason… Interestingly, as time has passed by, we have had to do much less of that and have been amazed by the fact that by now a large number of people argue the same as we were doing a few years back, at those very conferences. And, the annual meetings and events of the R-HTA consortium do attract a large number of participants, from the “usual suspect” countries, where HTA is well established, as well as from other parts of the world that are maybe lagging behind, but very quickly catching up. It is not by accident that the consortium now has its own “Low and Middle Income Countries” (LMICs) chapter, who has contributed their views to this book, in 3 Why R? A Low- and Middle-Income Countries Perspective.
This book is written with the objective of establishing even further the use of R as the standard tool for HTA, for academics, consultancy, industry and the regulators. In fact, the focus on R is perhaps a red herring — the basic problem is that tools such as spreadsheets that are still very prevalent in the preparation of dossiers submitted to regulators and governmental agencies. We see these as not fit-for-purpose and prone to inefficiencies in the process — because, inevitably, there is a strong element of statistical modelling that is propaedeutic for the actual economic modelling (e.g. using complicated survival modelling, as discussed in 7 Introduction to survival analysis in HTA) that non-statistical software just cannot cope with.
Other tools would be just as good, for instance modern and hipster programming languages such as python, or software that is considered even superior in computer science circles, such as C++. We do not disagree, but feel that the learning barrier for the acceptance of these alternatives would be even steeper and therefore have long elected to continue flagging the R flag. The reasons for this are multiple:
Ris a freely available language and environment for statistical computing and graphics; this has implications in all jurisdictions, but perhaps even more so in LMICs.Rprovides a wide variety of statistical and excellent graphical techniques. There is a very, very large community of contributors and increasingly more working on specific topics that are relevant to the HTA community, as we showcase throughout the book.- The use of
Rnaturally embeds the principles of reproducibility, as opposed to spreadsheets, which are based on effectively dead programming languages (e.g.Basic) and make the process of reproducing research and modelling much harder.
Despite its many perks, we are fully aware that the learning curve is steep and that has been a powerful deterrent for a wider uptake of R as the standard modelling tool in HTA. And this is why, to put it like comedian Stuart Lee, we have decided to enlist the brightest and best of the HTA community, to write a comprehensive guide to the use of R for the most common types of modelling and activities we all face in our day-to-day jobs.
The book is structured in three parts: Part I goes through some preliminaries — 1 Introduction to Health Technology Assessment briefly reviews the basics of HTA, with particular reference to the all-important activities of uncertainty analysis and some exciting work in the area of value of information (another one of our preferred nagging topics…). 2 Introduction to R provides an introduction to R programming: of course, this will not be sufficient to learn all that there is to learn about it, but it will hopefully mitigate the fears of the unknown, for readers without prior experience. Finally, 4 Introduction to statistical modelling reviews the basics of statistical modelling, highlighting the differences and subtle distinctions among the three main schools of statistical philosophy. We keep the mathematical sophistication to a minimum, in this chapter — and again, it cannot be thought of as a replacement for a solid statistical background. Nonetheless, we hope that it will provide some clarity, even for readers who already feel they understand Statistics.
Part II goes through the most important modelling tools in HTA applications. We start with Chapters 5 Individual level data and 6 Missing Data, which deal with individual level data (typically alongside randomised studies) and the all important issues of missing data. 7 Introduction to survival analysis in HTA is, in many ways, the cornerstone of this book, as well as the HTA modellers’ activity, since survival modelling is prevalent and possibly the most prevalent in HTA. This links naturally to Chapters 8 Decision tree models and 9 Cohort Markov Models in Discrete Time, which describe the efficient programming of decision trees and cohort Markov models, modelling structures that often are fed the output of a survival analysis. 10 Network Meta-Analysis concludes this part by discussing the important topic of Network Meta-Analyis and evidence synthesis.
Part III concentrates on advanced modelling tools — including continuous time multistate modelling in 11 Continuous time multistate models, Discrete Event Simulation (DES) in 12 Discrete Event Simulation in R and the extremely popular and important problem of population adjustment in 13 Population-adjusted indirect comparisons. All these chapters can be perhaps viewed as much more technical, both from the underlying methodological sense and in terms of the computational skills required to perform them. However, they are increasingly important and prevalent in the toolkit of the economic modellers and therefore we have included them here. This section of the book is concluded by 14 R and shiny in HTA, which describes the use of powerful integrations to R, such as Shiny web-applications. In our view, this is somehow the closure of the circle in that they allow a fully integrated and self-sustainable working environment for the entire process of modelling and presentation of the HTA results. Shiny web-apps can be used to completely replace the appealing feature of spreadsheets in which a user/reviewer can “play” with the assumptions, modify a cell or two and see what happens. In a Shiny web-app, this can also happen, while maintaining integration with the underlying computational engine (which is R throughout the process) and, in our view, this has the potential to completely overhaul the way in which the HTA community works.
An online, open-access version of this book is available at the website https://gianluca.statistica.it/books/online/rhta — this will also include links and additional resources, such as annotated code, which the reader can use to run through many of the examples shown in the book.
Editors
Gianluca Baio is Professor of Statistics and Health Economics in the Department of Statistical Science at University College London (UK). Gianluca’s main interests are in Bayesian statistical modelling for cost effectiveness analysis and decision-making problems in the health systems, hierarchical/multilevel models and causal inference using the decision-theoretic approach. Gianluca leads the Statistics for Health Economic Evaluation research group within the department of Statistical Science and was the co-director of UCL MSc Programme in Health Economics and Decision Science. He is a founding member and former Scientific co-Director of the R-HTA consortium (https://r-hta.org/) and a founding member of the ConVOI (https://www.convoi-group.org/) network. He also served as Secretary (2014-2016) and then Programme Chair (2016-2018) in the Section on Biostatistics and Pharmaceutical Statistics of the International Society for Bayesian Analysis. He collaborates with the UK National Institute for Health and Care Excellence (NICE) as a Scientific Advisor on Health Technology Appraisal projects and has been the 18th Armitage Lecturer in November 2021. His research activity is now (almost) officially dead, since he has become the head of the department of Statistical Science at UCL, in 2021.
Howard Thom is Associate Professor in Health Economics at the University of Bristol, a health economics lead at the Bristol NICE Technology Assessment Group (TAG), and managing director of the consultancy Clifton Insight. At the University of Bristol he created the world’s first annual short course on Economic Evaluation Modelling in R in 2019 and teaches on R for Health Technology Assessment for the International Society for Pharmacoeconomics and Outcomes Research. With Professor Gianluca Baio he founded the R-HTA organisation in 2018. He has published more than 70 peer reviewer papers, including new methods for network meta-analysis, structural uncertainty in cost-effectiveness models, and value of information analysis. He has built and contributed to dozens of academic and commercial cost-effectiveness models across a wide range of indications, including oncology (e.g. NSCLC, prostate cancer, breast cancer, hepatocellular carcinoma, and melanoma), neurology, rheumatology, and cardiology. Many of these models have been in R, including decision trees, Markov models, multistate microsimulations and discrete event simulations. He is a founding member and current Co-Director of the R-HTA consortium (https://r-hta.org/), as well as a member of the ConVOI (https://www.convoi-group.org/) network.
Petros Pechlivanoglou PhD, is a Senior Scientist at the Hospital for Sick Children, an Associate Professor at the Institute of Health Policy, Management and Evaluation (IHPME) at the University of Toronto and an adjunct ICES Scientist. He completed an MSc in econometrics and a PhD in health econometrics at the University of Groningen, the Netherlands. His current research focuses on the integration of large real-world data, decision analysis and statistical modelling in estimating the long-term health economic consequences of disease or treatment exposure, with a focus in early childhood. He has been an R user for over 20 years and has taught decision modeling using R for the last 15 years. Together with an international group of researchers has formed the Decision Analysis in R for Technologies in Health (DARTH) workgroup.