The Power of Data Podcast
Episode 55: A Data-Driven Approach To Policymaking
Guest: Katherine Baicker, Dean of the University of Chicago Harris School of Public Policy
Interviewer: Sam Tidswell-Norrish, International CMO, Dun & Bradstreet
Hi, welcome back to The Power of Data Podcast. Today, I am delighted to be joined by Katherine Baicker, who is Dean of the University of Chicago Harris School of Public Policy. Welcome, Katherine.
Thanks for having me.
Well, it's an absolute pleasure and we first spoke, it can’t be far off a year ago, before all of this madness entailed. And actually, the small silver lining is that we've been able to get this podcast recording in without having to meet in person. Whilst that would have been a pleasure, it means that we can get it done far sooner than I would have been in Chicago, a small blessing. To kick off, we typically like to tell our listeners a little bit about the person that we're interviewing. And to many people, you're going to be no stranger, you've had a star-studded academic career. You came from Yale, to a PhD at Harvard, where you later went on to become a professor. And your speciality is in health economics, where you were also used your expertise on the White House Council of Economic Advisers, and specifically the Medicare Payment Advisory Commission. And then just a couple of years ago, became the dean at the Harris School of Public Policy of the University of Chicago. There's a lot crammed in there, and it would be great for you to unpack it a little bit. Would you mind just by starting off, telling our listeners a bit about your background, and career journey?
Sure, I've always been really interested in public policy and in government programs in particular, and the social safety net, how programs work well, or don't work well, and national state local levels, to support all sorts of needs of the population, and what's the most effective way to structure them. I took a lot of political science classes in college and I took my first economics class, I think, my third year of college, and was so excited to get a toolkit that finally matched the way that I thought about the world. It made so much sense to me as a powerful approach to making trade-offs in the real world where there are lots of competing interests and lots of things you might do with scarce resources. And economics to me seemed like such a powerful analytical tool for figuring out what you should actually do in the world. So I became an economist in my heart right away and then after more years of education I entered the profession. I've gone back and forth between academic research and policymaking in Washington, as you mentioned, I had a couple of stints of government service in between my academic research positions, and I had the chance to come be Dean of the Harris School of Public Policy and that was a wonderful opportunity to bridge those worlds because I care deeply about the data-driven, evidence-based approach to policymaking that economics can underpin, and that is held dear at the University of Chicago and Harris School in particular. I can continue my own academic research – although in a much smaller part with my administrative responsibilities – but I can also help facilitate the training of the next generation of policy leaders who are going to get Master's degrees or PhDs at the University of Chicago in public policy and go off into public service or the private sector tackling policy problems. And also a chance to talk with the public, I talk with a lot of people in the media with lots of policymakers with general audiences about the evidence based approach to policymaking in general, but of course, in particular, my own approach to health policy and health economics and we can talk later in this discussion about some of my own research and how I've tried to use that toolkit to better inform policymaking. But it's just been a wonderful career trajectory for me to both generate new evidence and help inject that evidence into actual policymaking.
So before we get stuck into more of the policy side, you're very well-known establishment academic institution in the US. And having spoken to a number of people on the UK side in similar types of roles, it's not an easy time, in the educational establishments. Things are being done virtually, new policies are having to be implemented on the ground level around hygiene with COVID-19. And of course, there's then the more political debate around student fees and costs and a working world that students will then be arriving in. What are some of your views now are the outset of a new term for the school about how this is impacting students both short term and longer term?
This is a very challenging time in academia, as well as in so many other sectors around the world, but we're really having to rethink a lot of the modes of education and scholarship that we pursue. Of course, first and foremost is everyone's health and safety. And like many other universities, the University of Chicago has worked very hard to adapt all of our on-campus activities to be sure that we are following all public health guidelines and keeping people safe. But what we do is so much more effective with in-person elements than it is completely virtually, that conditional on keeping everyone safe, we want to have as much of an opportunity for learning and scholarship as we can on campus. And so that means not being completely shut down, but rather having dramatically reduced capacity with lots of other safety measures in place to allow people to start to resume those activities that are best done on campus. For example, you know, a lot of research projects really can't be done in people's homes safely. And so getting labs back up and running, getting research opportunities back on track as quickly as we can safely has been a priority and then making sure that our students can get the education that they need to pursue their career paths - wherever they are. And for some students, that's going to be on campus, for a lot of students it's not. The Harris School, for example, is about 50% international students, students from outside the US. Now those students have a really hard time getting to the US right now, both because of immigration challenges and because of safety concerns. We want to make sure that that doesn't derail their opportunity to get the evidence based policymaking toolkit that we want them to be applying in the middle of an economic and public health crisis. You know the work that our faculty and students are doing has never been more important, I think it's obvious that we need really well trained, thoughtful people making life or death policy decisions. So it's that dual mission that we've been pursuing, and it is really challenging to turn on a dime, but there are a lot of talented people working hard to do it and I'm really pleased with how things are going so far.
And Kate tell me, how do you think these times and we find ourselves in some of the scenarios you’ve just talked through are going to affect the educational system longer term. Do you think we're going to start seeing some trends appear that have a knock-on effect for future generations?
I think there are some lessons we can take away from the adaptations that we have been forced to make that can be valuable going forward. You know, we're doing this interview via Zoom, this technology has been around for a long time, and people were slow to adopt it, it wasn't refined all the fine tuning that would make it more usable wasn't happening very quickly because there weren't critical mass of people using it. That clearly has permanently changed and telemedicine is another example and we'll get back to that, I hope when we talk more about health policy - but for education, I think we're learning how to harness the technology to have multiple modes of interaction with students to think about ways to engage students outside of physical classroom, as well as inside the classroom. I don't think this will ever replace the in-person interaction. I think video adds a dimension to audio only, that is quite valuable. But going from two dimensions to three dimensions adds more value still. There is something important about the dynamic in-person interactions that I hope we'll be able to reintroduce as soon as we can safely. I'm sure that we will be taking advantage of remote education opportunities in the future in ways that we wouldn't have had we not been forced to adopt the technology, but I hope that it won't be exactly like this forever.
Talking about technology, there's a ton of opportunity that has been born out of this situation, I think and I'm really trying to look at the positive side of life and that statement. But in doing some research ahead of speaking to you today I noticed that the Harris School of Public Policy is ranked in the top five policy analysis schools in the United States. And you're a huge believer in that policy making needs to do good in some respects, and that's something that we're firm believers of at Dun and Bradstreet too. Your motto is quite unique and something I really enjoyed reading is: Social impact, down to a science. So how are you going to use data analysis and technology to help design policies that work and do good for society?
There are very few policies that don't involve trade-offs between competing needs, competing goals, competing people. There are a few things you can do that save lives and save money or are free. There are a few things you can do that generate more resources while improving safety or reducing risk. Great, those are the easy things, we should definitely be doing all of those. But almost every policy option on the table involves choosing between two things, but you can't have both. Choosing between alternative goals, both of which are admirable, both of which you would like to pursue, but that need the same set of resources and you have to make a decision about how to allocate those resources. You can have philosophical or ideological debates about those decisions, but in the end, you need quantitative analysis, you need empirical analysis, to be able to say, here's how much it costs to save a life via this intervention, and here's how much it costs to save a life via that intervention, I'm going to choose the one that's more cost effective. Or, here's how much good I can do in improving housing for this population with this set of resources, here's how many people I can feed with that set of resources. Now, economics can't tell you which you care more about, housing the homeless or feeding the hungry. You have as a voter or as a policymaker, to look at your own priorities and decide what's most important to you. But policy analysis can tell you the most effective way to achieve one of those goals or what each dollar would do when you have a budget to spend and then you have to decide which of those things is most important to you. I think we can give policymakers a framework for thinking about tough decisions and data that can help them make those tough decisions. But the decisions are still going to be tough and that's one thing that I have tried very hard to hew to as an academic and as an analytical source of information and expertise for policymakers is: I can give you those input variables, I can give you that data, I can give you that analysis, but I can't tell you what your priorities are, that's between you and the voters. And that's a line that's hard to walk, I think of myself not as an advocate for a particular position or a particular set of people. I'm glad they're advocates out in the world. That's not my role, that's not my best value to add to the system. As a voter, I have my own priorities, I vote accordingly, but it's not up to me to decide what a policymaker should prioritize. It's up to me as an analytical expert to say, here's what happens if you do A, here's what happens if you do B, here's how you could achieve this goal with the most bang for the buck in terms of your resources, but I can't tell you what your goal is.
It's so funny to hear you talking about all these enormous truisms of public policy design and I was speaking earlier to Neeraj Sahai our International President who actually was kindly the person that introduced us originally, and we were talking about how hard it must be to advise on and design public policy in today's day and age. I mean you see in the political arena every day at the moment, it's incredibly challenging, as everyone has their own opinion that's subjective, ultimately, and based on their beliefs of what they think is best. And therefore, it becomes even more important to use data and analytics to help create insights and informed decisions. What are some of the ways that you're using data and analytics today to do that?
Well, let me give you an example from my own research, which is my prerogative as the person who's been given a microphone. My own work in health policy is largely within the US, you know, I study the US healthcare system. And we have this program called Medicaid that is intended to provide health care for certain parts of the population: low income people who fall into certain demographic categories; pregnant women, disabled people. It's a social safety net program for largely low-income populations. It is a partnership between the federal government and the state governments and so there's a lot of variation state to state in what the eligibility criteria are for the program, what benefits it covers, how it’s structured, how it pays health care providers. So it's a real patchwork, and in a lot of states before Obamacare or the Affordable Care Act in 2010, a lot of states chose not to cover poor people with Medicaid unless they fell into some very narrow demographic categories like being pregnant women, or nursing mothers or children under a certain age and income level. But if you were an able-bodied adult living in poverty, you weren't necessarily eligible for Medicaid and lots of states chose not to cover those populations. Then the Affordable Care Act came along and said states had to cover those populations under 100% of the poverty level. Then the Supreme Court said, well, no actually it's optional for the states and so there's been this back and forth between the federal government and the states and decision making at the state level about whether to cover poor people through this Medicaid program, whether they are disabled or not, to cover all people under the poverty level. We wanted to provide policymakers with information about what that does to people's health, their financial security, their healthcare utilization, all sorts of things that having health insurance might affect. There's a raging debate about universal health insurance in the US and different countries have made very different decisions about their health care system. Well, one US state had a waiting list for its Medicaid program. And this was before the Affordable Care Act, so a lot of states weren't covering these populations at all. The state was Oregon; it decided the best way to allocate a limited number of spots and its program was drawing names from the waiting list by lottery. They didn't do it because they wanted to generate information for policy makers, they did it because it seemed fair when they had a limited number of spots. But it generated a randomized controlled trial of a major public health insurance program, inadvertently. When they drew names by lottery, the people whose names were drawn were a treatment group. And the people whose names were not drawn were control group. And this gave us the chance to give policymakers the kind of robust information about what health insurance actually does that they hadn't had before. And you think health insurance has been around forever, Medicaid in the US has been around for 50 years or more, how do we not know what health insurance does? Surely this is known. But it's the kind of inference problem that is very hard to solve in the absence of an experiment like this, because the kind of person who has health insurance is fundamentally different from the kind of person who doesn't have health insurance in lots of ways besides just the health insurance. For example, I've mentioned that you get on Medicaid by being low income. Well, being poor is hard on your health in lots of ways. If you didn't take that into account properly, you might think that Medicaid was actually harming your health, because people on Medicaid are in worse health than the uninsured. But it's not that Medicaid is harming their health necessarily, it may be that being poor is harming your health, and poor people are eligible for Medicaid. So that gives you just some flavor of why it's hard to know in the absence of a randomized controlled trial, what Medicaid actually does, here, we had the opportunity to study that because of this waiting list in Oregon, and we generated a wealth of information that policymakers are still using to assess the consequences of expanding Medicaid in their state because lots of states are still wrestling with this in the US.
It just goes to show Kate how important I think having multiple data sources, high quality analytics and perspective is on this stuff. And really, that's the fuel for a lot of the work that you do, I suspect. I'm fascinated by this and I could talk to you all day, I think that's probably largely because my wife is in the healthcare profession. But we recently did a podcast with the US Food and Drug Administration and the University of St. Gallen in Switzerland, which is an Operational Excellence Research Center in the field of pharmaceutical, and we set out doing a study with them. The podcast was talking about the study where we were helping them understand pharmaceutical quality management practices globally. And combining Dun and Bradstreet unique data sources and an unmatched base of knowledge about those global manufacturers combined with the University of St. Gallen’s pharmaceutical expertise, combined with the US FDA requirements. But bringing together and fusing that expertise with an unrivaled data set was an essential component and is an essential component to getting the right output. So when you're looking at designing and policy, where are you drawing your data from? What studies are you conducting to ensure the richest possible output?
Well, there are a lot of issues embedded in your question. I think in some ways, we're the proverbial drunk person looking for keys under a lamppost because that's where the light is.
That's a great analogy, by the way, I've never heard that before.
Our ability to answer questions is inherently limited by the data that we have. I started off by saying how so many of the key policy questions are empirical. And of course, there are all sorts of ethical and philosophical questions that you can wrestle with. But in the end, when you're making policy, it is specific and often quantitative, and you need data to answer the questions about what that policy is likely to do. Now there's a dilemma because sometimes you will need to make policy decisions in the absence of any data at all. And that's really challenging, say when there's an emerging pandemic, and your data is lagging, what's happening in the real world in a novel situation. You never though have perfect data, you're always making policy decisions without all the data you wish you had. And so there is a challenge to policymakers to make decisions under uncertainty. And there's a challenge to analysts to give policymakers the correct sense of certainty that the data can provide. It's not helpful to say, well, you never know, because you never know anything with 100% certainty, and neglecting to give policymakers information until you're really, really, really sure just means they'll be making policy in a vacuum. So that's a little aside on the challenge of decision making under uncertainty. But we have more data available to us now than ever before. And there are more things that we can do with that data as computing power has been growing at the same exponential rate that data availability has. So we have the ability to do a lot more predicting than we could before given machine learning, artificial intelligence, the big data sources that are available to us that are necessary to feed into those algorithms, we can do a much better job at extracting signal from the sea of noise that data sets often present. I'm most familiar with data that’s available on the healthcare side, of course, but it's not unique to that realm, but we now have all of these claims data that show the services that people consume and then we have much more granular information about their health outcomes. And we can marry that data to information about the social determinants of health, people's living conditions, their histories, their families. So all of that gives us the opportunity to do much more targeted allocation of resources than we had in the past. But the problem is, with so much data and so much computing power, you can also generate a lot of garbage, with garbage-in garbage-out kind of algorithms. If you don't understand the underlying processes at work, it's easy to find some signal and misapply it to generate a policy that will not have the effects that you're positing that it will. And so there's a real value to bringing together those prediction tools with deep understanding of causal inference. And I do think that economists have taken causal inference seriously for a long time and there are lots of others who do too, but I think that we're ahead of some other social sciences in generating the mechanics that help you understand what a policy change is likely to do. And machine learning alone can't do that, even though it's this shiny, new, powerful tool that people are very excited about and has all sorts of value in targeting resources and in generating hypotheses and in understanding which populations we should be thinking about and which way, but understanding how changing one parameter is going to affect the downstream outcomes that you care about, that requires some very careful thinking about causal inference. And again, at the University of Chicago I think we have a real history of dedication to that analytical framework that can be layered on top of the nonparametric machine learning prediction models. Fundamentally, you need the parameters at some point to be able to make policy.
It's interesting, we talk about resource allocation and just recently I was having a debate with Dun and Bradstreet Chief Data Scientist is always fun to get in the debate with particularly if you don't enjoy winning debates because I never do with him. But we spoke about the impact that has happened in 2020 on public and private health care. And every healthcare system around the world has been put under crazy pressure from Coronavirus. And I think we've learned a lot through the process, but reading recent research, it said that 65% of Americans are more concerned about having access to affordable health insurance since the onset of Coronavirus. But if you look at recent figures, you've got over 12 million people and now unemployed and in many of those cases, they've lost their employer-sponsored health care. So how are you seeing this changing dynamic, and how do you think it's going to impact public-private healthcare?
One of the real challenges in the US healthcare system is the patchwork nature of where people get their health insurance. So I talked a little bit about Medicaid, which is for low income populations, we have Medicare for people who are over 65. But the vast majority of non-elderly, non-poor people in the US who have private insurance are getting it through their employer. There's a growing non-group market or health insurance exchange generated by Obamacare or the ACA or augmented by that law. But most people who are not covered by Medicare or Medicaid, who have health insurance, get it through their job or their spouse's job or their parents’ job. And that is a particular problem when you have a public health crisis in the middle of an economic crisis, where people are also losing their jobs. And so people are potentially losing their health insurance at a moment when it is most important for them to have access to affordable health care. This is a real challenge for people's health and wellbeing first and foremost, but it also highlights the frictions in the labor market that are generated by having health insurance tied to employment. People studied for many years before the pandemic the role that fear of losing your health insurance plays in inhibiting job mobility, you don't want to change jobs because you don't want to lose your health insurance. The mobility of the US labor market, the fluidity of it has been a real asset in moving people to growing industries and having a really dynamic economy and a dynamic labor market. So this bundling of health insurance and jobs can be problematic on multiple levels, but it's particularly problematic in the middle of a pandemic. Now, there are opportunities for people to enroll in other forms of health insurance, when they lose their jobs. You can enroll in these health insurance exchanges that were facilitated by Obamacare or the ACA, you may become eligible for Medicaid, you may be able to purchase private insurance outside of the exchanges. But it just highlights the challenges of the patchwork system and I think gives us some impetus to those who advocate for policies that make it easier to bring your health insurance with you when you change jobs.
I have 1000 questions, most of them are actually down the path of insurance, which is probably less appropriate for right now. But let's just bring it back to a little bit more of a generalized question. And it's something we've covered off a little bit, but I'm keen to get back to basics towards the end of this podcast. When it comes to informing and advising on policy, what type of data is most valuable to analyze for you? And how do you approach it?
I don't think there's a single answer to what type of data is most valuable, there are so many new data sources available that open up new opportunities for us. So there's huge return to the really granular data that we get from administrative records on health care or employment. The administrative records data that's collected for some other purpose has this advantage of being both more detailed than people can self-report when you survey them, and often more complete in terms of covering 100% of the relevant population, which never happens in a survey. But administrative data sources have the disadvantage that they don't always have all the questions that you want contained within or all the data elements that you want contained within them. So let me give you an example from our study of the effects of expanding Medicaid. We had administrative records from hospitals and emergency departments with incredibly granular information about when someone was admitted, for what condition, what were the diagnoses, what procedures, when was the person discharged, how much did the care cost for everyone who was hospitalized in Oregon. So hundreds of thousands of records. You can't get that kind of information just by asking people. That level of completeness, and granularity of each data element is fantastic, but we don't have any information about outpatient care in those records. And there was no administrative data source on outpatient care, meaning if you saw the doctor in a doctor's office, or what your long term health outcomes were, how you were feeling, what your level of pain was, none of that was in those administrative records. So we had to couple the administrative records with primary data collection, meaning we went out and we did surveys of tens of thousands of people where we actually collected blood samples and blood pressure and asked people a whole battery of questions about their mental health, their physical health, their social determinants of health, all of the things that aren't contained in the administrative records from the hospital. That brings me to my last point about what kind of data is most valuable, having the ability to merge datasets across silos, I think is where the greatest value to unlock lies, because people are whole people, they have experiences across settings and across domains that affect all of the outcomes that we care about as policymakers. So if you can get hospital records, that not only have detailed information about the services people received, but also their medical histories, and their access to food, and their family situation and what their living situation is, and what their long term health outcomes are, you can do much more than if you just have millions and millions and millions and millions of hospital records, it's great to have both. So to the extent that we are moving towards a world where we can merge more data sources together, we'll be able to do even better in refining policy. Of course, that opens up all sorts of really legitimate privacy concerns and data security concerns. And I think that that's what we're all wrestling with right now, is balancing the vital protections to people's privacy and confidentiality with the good that we can do by harnessing more data sources bundled together.
I'm delighted to hear you talk about that one issue of extracting value by merging datasets. And mashing datasets together is a complex business, it's also something at Dun & Bradstreet that we spend an awful lot of time thinking about. Our clients need to do a number of things, they need to understand insights that they can infer from multiple data sets, they need to then also be able to share those findings across different silos within an organization or a government group or an educational establishment. And so some of the things that we've been focused on to address those exact problems include one of our new products coming to market that allows you to do investigations, and to share that within an organization to break down silos and to ensure that those insights are maximized within an organization. And then one of the big initiatives that we've been focusing on over the last 18 months is building a market leading sandbox that we call the Analytics Studio. And that allows you to mash different data sets together in a way that wasn't possible before. It's a data scientists dream. And so I'm really pleased to talk about that, because those are areas that are of paramount importance at Dun & Bradstreet, and I'm sure areas that we'll be able to collaborate on in the future. But Kate, we're coming to the end of the podcast, and it would be great to finish on a little bit of forward looking. So I would love to know, depending on how much you can tell us, what's next for you and your team in the area of health and public policy, what are your near term focuses?
Well, I do a lot less research now than I did before I assumed this academic leadership role. So I can tell you a little bit about what I'm working on, but maybe more about where I think the field is going. I'm really interested in cross silo analysis to allow public resources to be deployed to greatest effect in improving well-being and making a robust safety net. So if we're going to capture those opportunities, policymakers need to be able to think across silos of traditional departments like housing, transportation, health, education, because resources invested in one may affect outcomes in the other. And the way that we budget in the US I think is a real barrier to thinking broadly about population wellbeing. If you spend money on education, you may improve health outcomes, the person in charge of education has neither the budget tools nor the incentives to follow up on that and focus on that. So if we're going to let policymakers think more broadly, we need to couple budget flexibility with accountability, or we're going to lose focus of those public resources on the outcomes we really care about. So I am really interested in some of the Pay for Success type contracts or social impact bonds, yeah, used to be a way of thinking about this that's a little less in favor right now, where government officials have a little more flexibility and how they spend their budgets, but they're held to account by rigorous evaluation of what their use of those public dollars is actually achieving. And I'm working on one project with a different state in the US, evaluating the effect of home visiting services for low income pregnant women on the health and wellbeing of the mother, of the child but also the economic outlook for the family. Whether they are engaging with the labor market, whether their kids are ready for school, whether their interactions with the criminal justice system, whether they're substance abuse problems that are being addressed more effectively. If you can evaluate those outcomes using a rigorous randomized control trial framework, you can spend health dollars on services that may improve education and vice versa. Thinking even more broadly about health care, I think there's been a lot of attention paid over the last decade to designing insurance to get incentives right for individual patients, but we are just beginning to experiment at scale with new payment models for providers and that's hospitals and doctors and nursing facilities and imaging services, making sure that we are paying for value and outcomes, not just volume and inputs. And that kind of innovative payment mechanism implemented at scale, ideally, through our federal and state public programs, as well as through private insurance programs, I think can really move the way the healthcare system delivers care in the US and coming back to what we were talking about before about the value of data, we have data available to us in these massive new data sets to design much more sophisticated payment models that really reward providers for managing risk in a way that we haven't been able to before because we haven't had the data to underpin the sophisticated payment model. So I hope that we will be making big advances in refining our healthcare system through marrying the really rich data with the reward for better patient management for overall patient risk and health outcomes.
That's been perhaps the most insightful part of today's conversation and I love it. It aligns perfectly to Dun & Bradstreet perspective on being valued outcome oriented rather than volume oriented. I think that's a really profound way to end this podcast. But before we do, I want to ask you one more question, I wasn't going to but then I always wonder when speaking to such a brilliant academic mind, I love to know who's influenced the people we're talking to, through their careers. Who are some of your mentors and greatest influences been through your career?
Well, I'm glad to have the opportunity to mention one of my thesis advisors and a wonderful mentor throughout my career, Marty Feldstein who unfortunately died about a year ago, and really was a model for so many people in the profession of cutting edge academic research that was incredibly influential on generations of economists, while also being a really influential adviser to policymakers. And Marty was a lifelong engaged advisor at national, state, and international levels, but also really move the field forward and built institutions. He was the president of the National Bureau of Economic Research for many years and was responsible for its growth in eminence in a way that defined the whole profession. And so I benefited enormously from his generosity of mentorship and time and attention, and always looked up to him as somebody who had such profound influence not only on how economists think about things, but how policymakers use those insights in generating policy. So I feel very lucky to have had Marty as a mentor and advisor and he is sorely missed by many.
Well, thank you so much for sharing that and yeah, sounds like Marty had a profound impact on the way you've conducted your career and I guess how you will conduct it going forwards. Kate, it's been an enormous pleasure having you on our podcast and thank you for sharing some truly fascinating snippets of what you do on a daily basis. It really was rather sobering when I sit here in an office in London, thinking about the incredible work that you and the Harris School of Public Policy do every day. Thanks for joining us on The Power of Data.
Well, thank you so much for having me and good luck to you and your work.