The Power of Data Podcast
Episode 43: Quality Benchmarking Study
Guests: Jennifer Maguire, Deputy Director of the Office of Quality Surveillance of the U.S. Food & Drug Administration, Marten Ritz, Head of Operational Excellence at the University of St.Gallen, and Mark Seiss, Director of Predictive Analytics at Dun & Bradstreet
Interviewer: Sam Tidswell-Norrish, International CMO, Dun & Bradstreet
Hi there, and welcome to The Power of Data Podcast. I am extremely excited today we've got a little bit of an unusual setup. Normally we interview one person, it's very direct back and forth, one to one. Today we're joined by two of our very special partners. We're joined by Jennifer Maguire, who's Deputy Director of the Office of Quality Surveillance of the FDA, which is the U.S. Food and Drug Administration. We're also joined by Marten Ritz, who's Head of Operational Excellence at the University of St.Gallen, in Switzerland, otherwise known as HSG, a renowned research university. And Mark Seiss, who's Director of Predictive Analytics at my firm, Dun and Bradstreet. To kick off I'd love you guys just to give us a little bit of background on your careers and your organizations to kind of demystify why we're all here today and what each of your individual roles are, and we're going to start with Jennifer if that's okay.
Sure, I'd be happy to provide a brief overview. In terms of my formal training, I have a bachelor's degree in chemical engineering from the University of Virginia, and I have my doctorate in industrial and physical pharmacy from Purdue University. I spent a number of years working in Big Pharma in a research and development environment where I gained experience manufacturing both drug substances and finished drug products. My dissertation work actually focused on vaccine formulations with adjuvants. I've been quite lucky to have very broad exposure in the pharma industry. I'm actually just shy of 10 years with the U.S, Food and Drug Administration. I actually started as a reviewer responsible for assessing the chemistry, manufacturing and controls information that's submitted in drug applications. I've transitioned through different leadership positions with the agency to my current role as Deputy Director of the Office of Quality Surveillance, which is in the super office, the Office of Pharmaceutical Quality. My office focuses on using information that's available both internally and in the public domain to try and quantitate the state of quality. As you can imagine, these days, the volume and velocity of information is so high, and we really focus on using innovative techniques for signal detection and data analytics to aid in resource allocation and to really try and be proactive with engagement with our stakeholders.
Wow, Jennifer, I've already got a ton of questions that could take us off-piste, but I'm going to save them for another time. Marten, it'd be great to hear a little bit from you.
It's my pleasure to be here today, so thanks for the invitation to talk about our pharma study. I'm heading a research team here at St.Gallen University. It's called Operational Excellence and part of an institute that deals with researching production management. In this role, I actually support companies with working on their operations, doing production management with a special focus on the pharmaceutical industry. My daily work is really about supporting companies in moving forward, so being more effective and efficient. Just a few words about my background: I'm actually an engineer and graduated with a bachelor's in mechanical engineering and a master's in production engineering, adding some industrial engineering to my previous studies. During that time, I was already very passionate about optimizing corporations and companies in the field of lean production in general. However, when coming here to St.Gallen the pharmaceutical industry was actually new to me. My previous work was mainly in the automotive industry and with chemical companies. That's actually me in a nutshell and what we are doing here in Switzerland.
Perfect. Thank you, Marten. And finally over to my colleague Mark.
My name is Mark Seiss. I've been at Dun and Bradstreet for about seven years now mostly dedicated to the government solutions office in Washington DC that focus on helping government agencies. My educational background, I spent a lot of time in Virginia Tech, getting various degrees. I got a Bachelor's in mathematics, master's in statistics and mathematics. And then, about six years ago, I got my PhD in statistics from Virginia Tech. My previous work experience after graduation, my bachelor's I worked at the US Census Bureau for about eight years working on the annual census doing estimation and coverage measurement, basically an assessment of the census where their potential under coverage or over coverage was. Since working at Dun & Bradstreet, I've been working on government accounts. FDA was my first one or actually my first two projects. So right from the start, I've been working on FDA projects, working with the Office of Surveillance to provide insights into how D&B data blended with FDA data can provide optimization and help them intelligently target facilities that may have risk behaviors. At Dun & Bradstreet I also lead a team of data scientists that work with numerous federal agencies. We collaborate with numerous federal agencies to identify solutions and develop solutions that both blend our data and their data, provide optimization and support the missions of the agencies. My team's focus recently has been on this work. So working with FDA, but also we've been working with numerous other agencies on COVID related work streams.
Great. Thanks, Mark. And that's a perfect segue to the next question. I'm not going to give you guys my educational background because I don't have a PhD and it was relatively light touch. So we're going to get straight into the nuts and bolts of this and actually, it's fascinating we had a call yesterday talking about some of the things we were going to discuss today. I've been really excited about this. I learned yesterday, I'm gonna learn today and I think this is going to be a fascinating topical and timely discussion for our listeners.
So when it comes to the production of pharma products, such as prescription or the over the counter drugs that we’re used to it's important that the pharmaceutical industry always strives to maintain mature quality management practices. And that's so that we can protect our patients, it’s so we can protect our consumers and it's to remain ultimately ethical. Missteps in the manufacturing of a drug, or the lack of robustness in a pharmaceutical quality system can prevent that. It can prevent access to critical medicines. And that's one of the reasons why Dun & Bradstreet has partnered with St.Gallen on an FDA funded study on quality management practices in the pharmaceutical industry. So to begin with, guys, could you please share some of the background of this study and some of the objectives and tell us what you're hoping and who you're hoping will participate in the study and what outcomes you hope to achieve from it? And given it's largely coordinated by D&B I'm going to start with you Mark, and then we'll hand over to Marten and Jennifer.
Yeah, so again, one of my first projects I've worked on with FDA was trying to blend our data and theirs to optimize FDA surveillance activities. The second or third project I worked on was called DUNS verified. D&B worked with FDA to leverage D&B in country resources to collect essential data on manufacturing facilities for FDA where we have in country resources there so it provides cost savings to FDA in that they don't have to pay resources to actually go to the sites. So that was the genesis of this project, the first iteration of this kind of study where we would provide that kind of support to FDA. As we were transitioning into the next evolution of the DUNS verified survey, but we're thinking about how we can find additional insights to FDA in terms of manufacturing practices and good quality metrics for those manufacture facilities. So that was the conversations we started having with St.Gallen who are industry leaders in this space, doing a lot of research with respect to quality metrics, and we started to see how we can leverage their methodologies and their survey and combine it with this DUNS verified methodology where we can use our resources to help them collect the data, we can leverage their methodology to provide the additional insights for FDA with respect to the survey. Marten could probably give you a little bit more insights into the other side of it and how they came into the project.
It is great having this opportunity to partner with Dun and Bradstreet here and to bring together two different angles. Let me just elaborate a little bit on the point that Mark just made. Where we are coming from, as I just said, is that we as researchers want to support companies in improving operation. And one of the basic requirements to do that is actually being aware of what are the things that need to be improved? A question that always shows up is how can you actually measure performance of a production system or of a quality system? In the pharmaceutical industry, but also beyond. What are the metrics that you can use to assess to know are you performing well? ... to then define actions to become better. And one of the levels we have there is our experience in benchmarking. We developed standardized measures to support companies in comparing themselves to other comparable establishments based on the standardized set of KPIs . And this is actually what we bring here, we bring a scientific methodology, on how to measure and how to assess the performance and the maturity of pharmaceutical production. And coming back to your initial question Sam, who we are hoping that will finally be participating in the study and what's the outcome we can achieve from that. We are looking for pharmaceutical establishments, and when I'm talking about an establishment I actually mean a manufacturing site (but we keep it a little bit broader to also include contract facilities and so on). We want to have them participating and providing on the one hand, their operational performance data and, on the other hand, their current state of maturity based on our assessment methodology to be able to get an idea about the current state, the current state across the industry and across the globe. Based on that we can then jointly perform research together with the FDA to help industry to become more effective and predict quality issues. That's the main outcome we are looking for.
That's perfect, super concise. Thank you. And Jennifer, over to you.
I would just say from my perspective, when we received this research proposal at FDA, it was very timely. FDA actually published a report to Congress in October of 2019 that examined the underlying factors responsible for drug shortage and it recommended some enduring solutions. And of note, one of the three root causes that was identified included the fact that the market does not recognize and reward manufacturers for mature quality systems that focus on continuous improvement and early detection of supply chain issues. One of the enduring solutions that was recommended in the report was to develop a rating system to incentivize drug manufacturers to invest in Quality Management Maturity for their facilities. You can see that anything we learn out of this study is actually foundational as the FDA works towards establishing our rating system for incentivizing manufacturers and incentivizing an investment in Quality Management Maturity.
Thanks, Jennifer. It's really great to hear the evolution of the October 2019 study and this is all heading in such a positive direction, it really is. Earlier on Jennifer mentioned the volume and velocity of data and it triggered a thought in my mind. At D&B, we talked about the five V's, volume and velocity and then also the veracity, variety and value. And really those are the five keys to truly understanding and getting the best out of data analysis. So Mark, I'd love to hear a little bit about what are you bringing in terms of data science and analytics expertise, to this program? How are you going to analyze the study's findings and what insights and key takeaways are you looking for? What's the D&B edge?
The first thing we're doing is bringing that data science we're looking to incorporate St.Gallen’s proven methodology and quality metrics and show how the facilities relate to those and assess how they relate to their peers in terms of those quality metrics. And hopefully give them a guide towards improvement in those manufacturing practices. We also bring the power of the D&B Data Cloud where we have numerous different ways to look at facility beyond the view that FDA has. So for example, we can overlay our current COVID-19 Index score, where if we had a hypothesis, where we want to determine are the facilities in areas that are most impacted by COVID-19 are they differing in terms of their assessment, in terms of their quality metrics than, say, an area that's less affected. We can create hypotheses like that and give added value and make additional statements about manufacturing and give additional color to that. And really we hope this is a baseline for an annual study. So this is the first go or first survey, giving an assessment and giving first insights of these facilities and how they measure up to their peer in terms of quality metrics. We hope that this is a time series analysis where we can eventually see over time that these facilities are getting better, are the median or the midpoint of these facilities getting better over time, as we're giving these insights as we're getting that feedback. And as mentioned before on the podcast, the incentives to improve their processes and get better relative to their peers.
Excellent. And we see this all the time at D&B, assessment is great, but actually, it's the monitoring and the metrics that show improvement that are really vital for progress. So using data is an ideal way to benchmark global manufacturers and to understand how companies are performing relative to their peers, you know, Marten earlier talked about operational excellence and that's really what we're striving towards. Jennifer, how will you use the study to really help the pharmaceutical manufacturers improve on their practices and shape the wider program that the FDA is working on today?
I think that the assessments are great opportunity for manufacturers because they receive a personalized benchmarking report that also outlines for them continual improvement opportunities. And it's free, who doesn't love free. It's a way for companies to see how they stack up against their peers but also how they stack up against industry overall. In terms of shaping the greater program at FDA, I think this study really builds upon our learnings from some of the earlier work we've done with St.Gallen and the benchmarking studies on identifying quality maturity attributes that really impact effective and efficient manufacturing. For example, process performance, process capability, this study uses the global reach of D&B to complement the St.Gallen’s research through the design of a statistically representative sample of regulated industry and it accounts for multiple factors. We'll have the ability to look across industry sectors. For example, do we see differences in the quality management practices between brand companies and generic companies and OTC companies? Or maybe there's even cultural differences, and we'll see an effect of location. And also you can imagine that there's going to be some learnings about the firm type. So for example, contract manufacturing organizations and API manufacturers or finished dosage form manufacturers.
So let's talk about them for a second. So we've obviously got cross industry standards. Let's talk about global standards and manufacturing quality for the moment. Obviously, different regions have wildly different standards. And it seems to my under educated mind, that global standards makes sense. I know there are efforts underway, like through our research, I learned about the ICH, which is providing guidelines to harmonize across multiple bodies. But do they do enough? And what are the barriers to entry in your mind, Jennifer, for global standards in manufacturing of pharmaceuticals?
I do think that there will be a benefit for a global standard. Each competent authority right now follows the appropriate laws and regulations for their region. But when you think about it, Current Good Manufacturing Practices really establish the minimum standard for systems that assure proper design, monitoring and control of manufacturing processes and facilities, but they really don't address above the bar behaviors that would assure sustainable compliance and that's where Quality Management Maturity really comes in. The ability to understand those above the bar behaviors. So one of the barriers, I think, is that there's a lot of good work being done right now by different professional associations and consortiums on different aspects of Quality Management Maturity. So there's work going on, in the area of quality metrics or quality culture and elements of a robust pharmaceutical quality system, for example, and you mentioned the work that's being done by the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use, which is a mouthful, abbreviated ICH, for really harmonizing quality guidelines and expectations regarding pharmaceutical quality system effectiveness, and even quality risk management principles. So there's this movement to begin to standardize above the bar behavior. I think the challenge is really going to be drawing from all these individual efforts, taking the best bits and pieces and developing a rubric or a model for assessing Quality Management Maturity that works well for the pharmaceutical industry overall, and that all of the different stakeholders buy into.
Got it, makes a lot of sense, I guess conceptually, but not an easy thing to do. Mark what's the perspective from a data science standpoint?
From a data standpoint is you want a globally consistent method of assessing quality. We've at Dun & Bradstreet, we have a number of scores that are global in nature. But we've recently built the global business ranking because we found a need to have a globally consistent score, where an apples to apples comparison of a firm in the United Kingdom versus a firm in China versus a firm in Australia. So a globally consistent score is necessary for firms to assess on a single metric scale one firm to another. So from data science standpoint, I think we should really be driving towards one standard where businesses can all assess on the same scale.
So I'm going to keep on the kind of the data track for now. I'm actually preparing this evening for a podcast later this week with a leadership member of Microsoft who was once famed for saying that every company is a technology company. And at D&B we believe that every company is a data company. We've seen recently that data analytics and insights have the power to transform companies and ultimately to reshape entire industries, and that's never been truer than today, Marten how's the study's findings going to help shape policy and practices in the pharmaceutical manufacturing industry going forwards?
That's a pretty good question, Sam. I think the good component here – and you already mentioned that – is when you have data available, that means you measured something, you can actually come to a point where you say, hey, we are doing the right thing now, or we can give guidance on what are the right things. And we have evidence: It's basically data driven or data proven decisions what we are looking for in general from a global perspective (and also across industries) when it comes to optimization and the question about the way forward. And – coming back to our study and what we do here in particular – the good thing is with having this broad range of pharmaceutical establishments participating and providing their input into the study, we will actually be enabled to look at various relations. Because what we managed when conceptualizing the study, is that we do not only look on the measures, like the performance indicators, something you can calculate based on the data you can just get from your machines, but we also found a way to quantify the level of maturity. So, to actually operationalize practices that are in place that potentially help to produce medicines in a better quality. That actually enables us then to do the research and to come to a point where we can say, hey, those are the practices that seem to be most effective, here are the right levers. It's also about the soft effects, so coming to more cultural aspects like sustaining a culture of continuous improvement and empowering your employees to speak up. One of the maturity attributes we have from the study is, just to give an example: Are your shop floor employees actually able to explain the impact their work, their tasks and the job they do, has on patients and on the quality of medicines? Assessing those things, and linking that back to the hard outcome performance, you can calculate based on numbers. That is actually where it starts to get really, really interesting from a managerial perspective. This is where we also come in as a business school and say, hey, managers out there, this is how we can get your buy-in. Invest in those practices, because this is how you can actually also become then more predictive and have these early indicators in place to see that there might be a potential risk from a quality perspective. And this is how you can set up capabilities to be more robust and more effective.
And Sam, I would just add, you mentioned reshaping industries; I would say that we've seen data analytics help transform companies from multiple different industries - banking, retail, insurance. I think that the pharma industry may be slightly behind the curve when it comes to realizing the power of big data. But I think that the future is quite bright and a study like the QBS really can show industry what is possible.
Awesome. And Jennifer, if you had to be predictive on where this study could go, do you see the study being the start of many and iterative monitoring and improvement for the FDA?
I do see this study as foundational and supporting a number of different initiatives at the agency as we work to better understand the behaviors that really separate immature and mature quality management practices and how that really feeds into what the agency is trying to do with building a rating system and really starting to incentivize industry to invest in Quality Management Maturity.
Jennifer, Marten, on behalf of Dun & Bradstreet, it's a huge privilege to be involved in something as important as this. And we're as excited as you are, I think about the possibilities and the improvements to industry. So, it really is a pleasure. Mark, I'm very conscious that we've had listeners tune in to this, who probably want to know more about how to get involved, perhaps that we have people from pharmaceutical businesses or people who know executives at pharmaceutical firms who see the inherent good and the value in participating, what's the process, how can they get involved?
There's all kinds of information available online. So, someone listening to this podcast interested to more information about what we're doing here. The website is dnb.com/pharmastudy, one word, they can also email us again email@example.com and we can get back to them. Again, if they're just interested or want to read more about what we're doing here. Or if they're actually interested in participating, in volunteering participation into the study, we can also arrange that if they're a pharmaceutical manufacturing facility.
Wonderful. That's great to know. Before we wrap up, any final words from you Jennifer?
Yeah, I’d just like to say that the FDA’s Quality Management Maturity program is really in its infancy stage and this project is going to help inform the future direction. I think that my office has demonstrated over the past number of years that we're a really collaborative office and we do seek engagement with our stakeholders. This really is industry's opportunity to get involved and contribute to really enable FDA to move forward with our data driven policies. And I just want to alleviate any fears and remind companies that FDA will not be receiving the raw data from this study. What we receive back is going to be anonymized and aggregated. I hope that we've really relieved any fears about participation. We really do want companies to feel comfortable sharing, so we have a very representative benchmark of quality management practices.
That's wonderful to hear. And I'm sure everyone listening is also very pleased to hear it. Guys, it's been a real pleasure. I said at the beginning, I was going to learn something, you didn't let me down. I certainly have. And I look forward to seeing the output and ultimately, the evolution as all three of our organizations continue to work together with the industry to create improvements. Jennifer, Marten, Mark, thank you so much, it's been a real pleasure.
Thank you very much.