Building AI-Native Life Sciences Organizations: From Outcomes to Impact

Danny (00:00):

The Life Sciences DNA podcast is sponsored by AgilisiumLabs, a collaborative space where Agilisium works with its clients toco-develop and incubate POCs, products, and solutions. To learn how AgilisiumLabs can use the power of its generative AI for life sciences analytics, visitthem at labs.agilisium.com. Sri, we've got Amrit Moola on the show today. Whois Amrit?

Sri (00:29):

Amrit is senior director at Vertex, who sits in theintersection of data science, AI, and drug development. He's been deeplyinvolved in how Vertex is actually operationalizing machine learning, not justas a buzzword, but as something that can have meaningful impact on howtherapies are discovered and developed. He brings both a technical depth and areal world execution side, which is a rare combination in the industry.

Danny (00:52):

What should people know about Vertex?

Sri (00:54):

Vertex is one of the most successful biotech companies inthe last couple of decades, especially in the therapeutic area of rarediseases. They built their reputation on cystic fibrosis where they didn't justdevelop a drug, they essentially transformed the standard of care. But what'sreally interesting now is how they're expanding beyond that into areas likegene editing, pain, and other serious diseases. And increasingly, they'releaning into data and AI to make drug development more efficient and more precise.

Danny (01:24):

Well, what are you hoping to learn from Amrit today?

Sri (01:27):

Amrit is a very practical and pragmatic person, and so Iwant to get beyond the hype. Everybody talks about AI in multiple different usecases. I want to explore what kind of frameworks can one use to select AI usecases? How do you then go about building it and building the right guardrailsfrom a people, process, and technology perspective?

Danny (01:49):

Well, before we begin, I want to remind our audience thatthey can stay up on the latest episodes of Life Sciences DNA by hitting thesubscribe button. If you enjoy the content, be sure to hit the like button andlet us know your thoughts in the comments section. And don't forget to listento us on the go by downloading an audio-only version of the show from yourpreferred podcast platform. With that, let's welcome Amrit to the show.

Sri (02:16):

Hi, Amrit. Wonderful to have you on the podcast,LifeSciences DNA. Amrit, why don't you tell us about a little bit of yourjourney before you joined Vertex, and that'd be great to get to know youbetter.

Amrit (02:30):

Absolutely. First of all, thank you for the opportunity.It's great to see you in person. I started my career on the consulting side. Iwas with Satyam Tech-- Mahindra now. Been with them for five years, a lot ofinnovative work that was done there. One thing that I would notice, we built anequivalent of Equifax for India. So back then when there was no internet, noidentifier and things like that. Went on to lead the supply chain managementtechnology for air products and chemicals. Another innovative solution therewas to build AERP system where SAP didn't work because we had 40 plus entitieswith low bandwidth. SAP wasn't feasible for us. So that's one innovative thing.I moved on from there, did some consulting roles, worked through GE, Ford MotorCompany, and others. And in 2007, I joined Merck. That was my first entry intolife sciences.

(03:41):

And one thing, I've been there like 10 years and I've hadprogressive growth, work from commercial to research, to manufacturing, took onenterprise roles. And one thing I learned there is not just building technologyfor technology's sake, but more about how do we ensure that we actually delivervalue and how do you measure the value in a way that that's tangible and thatactually gets baked into the LRP, right? So that's the important part. Andthat's the principles that I've implemented and took on, started applyingelsewhere. After that, I went on to Bristol-Myers Squib to lead their digitaltransformation effort, as well as their IT PMO. Those principles I've learnedat Merck and implemented came in very handy. We had to look at getting $125million of run rate savings across. That was a huge endeavor. From there, in2020, I came to our Vertex.

Sri (04:45):

Wonderful. It seems like a fascinating journey, but Ithink through the journey, you've learned a lot around ROIs and how do youreally break in executable and tangible use cases. We're on the topic of AI. Iknow it's front and center of several of things which are going on in lifesciences. So walk me through how within that journey of AI, how have youpersonally adopted to that journey and how are you using those frameworks whichyou will learn from BMS and Merck to apply to AI and AI use cases?

Amrit (05:17):

When the AI bus started coming out, right? So in terms ofAI can do this and that. One of the things, of course, I'm naturally curiousfrom a technology standpoint. I started looking at what is the potential interms of in the business context, what problems can we solve with it? Solooking at it from that standpoint, and there were a lot of false starts in thebeginning. So if you think about it, we started our AI journey. I mean, AI isfor someone going through computer science, you can date back to like 1950s or1970s, I think. But when you look at what's possible here, you can actuallystart looking at rethinking your business in terms of what AI can do, buteveryone doesn't start at the same place. So the important thing is to makesure the organization is aware. So if you think about the change managementconcept, the building that awareness, building that, the desire for gettingpeople engaged is important.

(06:32):

So as long as we're able to do that with baby steps,that's the first thing. And then comes the knowledge and how do you build thatskill, the ability for people to get involved. These are all the baby steps.Most of the companies are doing that now, right? So they're doing that, I wouldsay, very well. So if you think about ... I can't think of any life sciencescompany that doesn't have the Microsoft 365 and Copilot enabled, right? Sothat's the start. Now, a lot of companies might end up declaring success atthat point. So saying that, "Hey, we've enabled AI to the masses, let themgo use it. " But that's just the beginning. So you had to start buildingyour processes natively.

Sri (07:14):

You hit up on a very important topic that is, "Hey,don't automate what is broken. Start to rethink the process in the newworld." So give me some specific use cases. So let's pick a process,whether in the clinical side or commercial, whichever, and walk me through whatwas the process in its current state, what was the way to rethink it, and thenhow did you apply AI to make that better?

Amrit (07:38):

I mean, I can think of many examples, but let's say ifyou're looking at whether it's CDMOs or CROs, right? So when you're thinkingabout a situation where most of the life sciences companies, a lot of work isdone outside, so to really improve the pace of the execution. If you thinkabout it from that standpoint, and the time it takes for the data to come inand for you to make decisions, right? So the time for the insights to come tothe table, most of the times you are in a reactive mode already. Now, you canleverage AI to augment the work that you're doing to kind of pull together theinsights and try to help you focus on which ones to work on. Yeah, you made itfaster, but only incrementally. But if you think about reimagining the wholething, right? So this is where, let's say you have the right, whether it beagents or controls in place across the entire ecosystem, whether it's the CDMOswhere you're trying to get the relevant manufacturing data or with the CROs interms of, "Hey, can we get early insight into which CROs might fail tomeet their recruiting criteria or whatnot?" So when you start looking atit from that standpoint and designing the processes, as well as making it AIenabled, right?

(09:08):

So then that changes the game.

Sri (09:09):

That's a fascinating way. You almost not just looked atthe process from an internal standpoint, but also end to end from an externalstandpoint. Of course, that requires the CDMO or the CRO to agentically enablesome of their outputs and give you MCP access so that you can then query that.And so there's a push and a pull, right? So you as a sponsor are saying,"Hey, I want an end-to-end visibility on recruitment. If you are CRO orCDMO working with me, this is how you can give me that visibility that's partof the contracts or whatever." And then data starts to flow in and you canmake much more real-time aspects. Are you actually doing that right now or isthat a vision? Are these things being built in parts? Are there some CROs orCDMOs who are better at this than the other?

Amrit (10:04):

So, without getting into the specifics, right? So this isan evolution, right? The common mistakes people generally make is the dealswith the contract organizations are done without thinking about the technologyenablement. And that makes it incredibly hard later to be able to say,"Hey, you know what? For us to make this process happen, we need thisdata. Can you provide..." It's more done as a favor rather than acontractual obligation, and this is where thinking about building the AI nativeprocesses, it doesn't just extend to just the CROs, right, or contractorganizations in general. Every organization should be coming together. So inthis case, when the contracts are being made, so we should have the sourcingorganizations work with the technology organizations to be able to say,"What are things that we need to consider to make this successful?"It does take time. And the important part is everyone's learning in thisjourney.

(11:10):

So it's not like everyone has everything fullyimplemented, so thought through. So the important part is learn and adaptquickly. So at least if we experience that there is an issue with the currentCDMO setup or CRO setup, right? When we're signing a new contract, thislearning should be applied there. If we don't, then we've just missed theopportunity.

Sri (11:34):

This seems to be very similar to data quality standardsand contracts. So in my past, previously, if you did not have the data qualitystandards, garbage in and garbage out, so you were getting bad data from CDMOsor CROs or others, and then you were spending a lot of IT time rationalizingand standardizing it. Then the IT leadership realized that, "Hey, if I canfix in the source," they went to sourcing and said, "Put a dataquality as a part of the contract." Then what that did was they pushed theonus on the owners who are giving this data. What you're now saying is,"Hey, don't put just a data quality contract, now put an engagementtimeliness contract." So there's a quality, of course, they can't give yougarbage data. They had to give you good data quality, but also give the accessto that through much more of a real-time nature so that then you can make thisorchestration happen.

(12:29):

It's very fascinating, but it's building up upon whatpharma has done before, but it took a long time to bring in those data qualitystandards just within the contracting framework. Now, you're saying, "Hey,let's extend that in the agentic world and let's enable that from the get go." Really interesting. So that's one case where you're looking at anecosystem play, which is great, but let's start to say, control thecontrollables within the controllable walls of what you're doing within R&D.Are there certain areas which people who are listening which are low hangingfruits that they should absolutely be doing? Do you guys have a framework on,okay, which has a very compelling ROI and which doesn't? And can you walk methrough that?

Amrit (13:17):

That's a good point. And also, before we even get to theROI part, so we have to help the organization build trust in AI. So oftentimeswhat happens, even if they're leveraging Copilot or Cloud or other tools thatare given to them, they ask a question and maybe try it a couple of times. Theydon't get the right response. They just give it up. Many don't recognize thatyou almost are training it first to get the right answers. And these LLM modelsespecially work best when you're pushing them. So when you are challengingtheir responses, that's when the best output comes in. And then the second partis how do you repeat it? And in situations where repeatability is important,how do you kind of streamline all these constraints? So you don't have to do itin a conversational style, you can just upload it once.

(14:17):

So what that requires is some training on how do you coachthe masses on the right prompt engineering, for example. And also the otherthing is, what sort of input do you need to provide for that model tosynthesize the information and give you the information that you're lookingfor? And in all these things, one of the mistakes one could make is thinkingthat they can just trust the output. AI can do a lot with reasoning, withspeeding up the processes, computation and other things, but the judgment stillremains with the individual, right? So that's where their expertise, domainexpertise comes into play and they should be able to use that judgment to say,"This response doesn't look right. Let me probe further." So that'sthe kind of training that we need to instill, right? So that's number one. Andthen once we do it, the organization becomes AI savvy, right?

(15:22):

Then we can start looking at the ... And this could betried in some know-to-grid zones first before we get into some of the complexregulatory governed processes. And then we can start looking at how do we lookat the entire process, whether it's intelligence building around the clinicaloperations overall, right? So what are the systems that we have? How can weharness this data to get the right insights? And then should we install agentsto look up information from different things to take actions? How do weorchestrate the agents across. So all that comes next.

Sri (16:02):

You've kind of given a little bit of a playbook, right? Sowhat you're saying is, A, first, there are two parts to it. One is the peoplepart where you have to enable people to really build the right skillsets aswell as the right governance structure. That's on the people side. So that'sjust getting people skilled. And I'd love to probe on what kind of training yourecommend. How do you get that skillset built into the teams? You come to that.The second part of your framework is like, okay, let me look at where I havethe impact, but first start to map the process end to end, right? That's whatyou said. Then look for what tools or automation efforts can be done, thendecide on the agents and then design on the orchestration architecture versussaying, now I have an AI, let me go and solve for it.

(16:51):

You're like, fix the process, look for weak points orplaces where you have automation opportunity, then build the right type ofinfrastructure, orchestrate it, and then put the AI governance. Is that a fairframework?

Amrit (17:05):

Literally. Absolutely. And then we can't boil the ocean.So the one thing that's important is to ask whoever that leader of theorganization is, where do you see most value, most impact that can be made? Soit could be two or three critical areas of the hundred different businessprocesses across. Just focusing on that and the peripheral ecosystem aroundthat and working from there maybe the way to go.

Sri (17:31):

Just probing a little bit on that, are there certain keyprocesses or areas which have surfaced like, "Hey, recruitment is the bestarea for putting it or medical writing is the best way for going." Is thatdata management where there's a lot of bulk work or is it pharmacovigilance? Isthere any blue ocean areas where people should be saying, "Hey, we shouldbe applying things in this area, AI in this area."

Amrit (17:55):

I mean, it doesn't have to be one, right? So there aredifferent avenues that you--of course, happen.

(18:01):

If you're thinking about the prior ways of solving it, youkind of collated all the data, then put some dashboards on it to glean insightsand all that. How about making a conversational kind of creating available topeople on the data that they have access to? So that could be a simple use casethat can be replicated across different areas. That could be one thing. Now, ifyou think about from just our own timeline, from when we finish the clinicaltrials to go for regulatory submissions, how do you make sure we pull all theinformation together in a way that we provide the right documentation forregulatory submission and things like that? There's so much opportunity therein how we can pull all the information together, still with human in the loop,owning that judgment across. And how do we help shorten those cycles? So theimportant part, and most of the leaders I think are doing this now, is to beable to say, "Hey, this is the longest thing on my critical path, right?

(19:19):

So how can I cut it by half or whatever?" So what arethe challenges? Throwing that challenge out there first and having theorganizations brainstorm on it to see how we can solve it. So it could be howwe create the narratives, right? So for the CSRs and other things, there aremany opportunities that we can look at, but one thing though, the prioritycould be different for each organization depending on where they are.

Sri (19:47):

Yeah. Just to probe a little bit on that, so one part ofit is the classic, let me look at the critical path and then automate out ofthe critical path, right? But you had also said that, "Hey, the criticalpath of today may not be the critical path of tomorrow if you don't rethink theprocess." So there's like competing approaches like in some areas, let metake a look at the critical path, get a quick win, get it out and going. But inthe long run, would you recommend, and I don't want to put words in your mouth,would you recommend that, hey, you got to really take a step back and look atend to end because just automating on the critical cloud may not get you thelong-term benefit you need.

Amrit (20:26):

Automating on the critical path, of course, it addressesthat acute problem and then you use the critical path somewhere else, right?And to your point, making sure we have an eye on the end-to-end process isimportant. And then when you look at the holistic end-to-end process across theorganization and then break it down into each of the divisions, they have theirown, what you might call the end-to-end processes for that division, and makingsure we are looking at those prioritized use cases to solve is important, butthat's one part, that's solving the known problems. So other thing, this iswhere the AI native companies or people that just kind of come up withdesigning things in an AI native way will really win in the long run, is tolook at what are things that we could solve that we couldn't do before with AI.

(21:23):

So when you look at it from that standpoint, differentavenues open up. So if you think about, for example, batch disposition as oneof the challenges that many organizations have. So it's time we spend andmaking sure everything's ticked and tied and making sure we are making theright decision. If you look at it and say, how can we make sure we have theright controls in place in different places to, one, detect the deviations atissues beforehand, so proactively, and address them, and then shorten the timeline.So that's important. It becomes even more important, especially if you'rethinking about cell and gene therapies, right? So where your process actuallyis the product, right? So that's where it's even critical when you have to lookat each process step. So having the lean mindset is important as we look at it,but also being business savvy about, if I'm making this investment here, whatis the return I'm going to get?

(22:28):

And then how is it going to position me in terms ofcompetitive advantage?

Sri (22:33):

That goes into that ROI stuff, right? What you exactlytold us, "Hey, what is the outcome I'm trying to do?" In cell andgene therapy, the outcome is the process. If I can make the process much moreefficient, I'm getting therapy back to the patient and not waiting for a longtime. That improves adherence, it makes sure that they're on track and therapyand all of that stuff. So that's a good outcome. So how do you, as anorganization, start to work with the business side and the IT side altogetherto define that? Is it a top down? Is it a collaborative? Is it a bottom up? Andthen once you do that, how do you then define what the ROI is to make sureeverybody is going down that path?

Amrit (23:14):

So the important thing is to focus on the problem firstand prioritizing what problems to solve. So that's where the outcome mindsetcomes in. If we don't have it and if we say, "Hey, I've got this cool toolthat I can use," then we are just trying to find the use cases that fitsthe tool. And that goes back to my first example in terms of just buildingsomething like equivalent of Equifax when we didn't have anything, right? Wehad a problem to solve and that's what we went after. So the right solutions willcome into play, whether it's pure AI, combination of AI plus data solutions andother things, you are more focused on what is the optimal way to solvesomething. And to your point, and as we get into ROI, we can spend a littlemore time on this. It's not just business and IT looking at it together.

(24:11):

You need your finance partners, you need your HR partnersto look at it as well, right? So especially if it's a big business case thatyou're trying to solve for where your skillset needs to evolve, right? Or ifyou're looking at, "Hey, implementing this, I'm going to unlock this muchof headcount capacity." Who's the expert to kind of validate that? Someonein HR should be able to review that data to say, "It makes sense." Ifyou're calling out financial benefits, someone from finance should be able tovalidate the assumption, say, "Okay, it makes sense and here's a sponsorthat's going to benefit and let's make sure we factor that into the long rangeplan as well." So that's where the accountability comes in. Oftentimes wedon't do that job very well. So we just celebrate the success as soon as wedeploy a solution, but it's important to get to a point where this is actuallygiving value back and that that value back just manifests itself in the LAP.

Sri (25:12):

Yeah. You bring up kind of a good governance point, right?So saying, "Hey, if it is going to give you a financial benefit, justdon't claim it. Make sure that it's part of your long range plan. Make surethat it's actually brought back into your financials." Same way if it'srelieving headcount productivity, then bake that into either more work to bedone or tasks to be done differently. And so just making sure that you'rerealizing the benefits, not just claiming the benefits. I think that's a verygood thing. And in terms of these ROI settings, is it top down? Does it comedown from the CEO? Is it divisional? Is it project team focused? Is it studyfocused? How do you go about setting the-

Amrit (25:59):

The simple way to do it is the sponsor. So accountabilityshould be at the sponsor. I'll give you one example. When I was working withMerck Manufacturing, the head of MMD, here's the concept of, his name isWillie's Bank of LEDs, right? So he used to look at it, okay, I'm making thisdeposit, so I'm putting this money. What is the value I'm getting? Show me thevalue. So it's in simple terms. I mean, you have to take away the complexity.Many times we get into, when we talk about ROI, we are talking about, okay,NPV, IRR and all that stuff. Those are behind the scenes calculations. So foran expert that wants to validate it, for example, a finance person, then youmay have to show all those. But for communicating what we're trying to do, wedon't have to get into all those details.

(26:51):

So it's about you're making X million dollars ofinvestment. You're trying to realize that 5X, 10X, whatever over a period oftime, how do you measure it? How are you tracking to that? And what levers doyou have to pull to make sure you're still on track on getting those savings?

Sri (27:09):

I want to go back to that enabling the employee pointwhich you made, which is how do you get that new skillset mindset happening? Sowalk me through, you're already in an organization which has establishedplayers. So how do you enable an existing organization? Maybe we can talk alittle bit about that. And then how do you then bring up new things to do?

Amrit (27:34):

So there are a couple of ways. So one, the nice thingabout AI is you can use AI to learn AI. That's the very important thing. Youcan just start with, whether it's Copilot or any other similar tool, say,"Tell me what I need to know. How do I learn?" And it just gives youthe things. So the important thing is to create the energy around it. So ifthere are some early successes, make sure those are showcased, make sure thatthe people are able to share that success. That is contagious. And I've seenbeing part of workshops where some of those are shared and everyone's like,"I want that too." If you're able to create that energy, that's thefirst victory. Then it's a matter of getting them to get over that hump wherethey try and they're not getting the results and people want to give up, givethem the needed boost there.

(28:28):

And then of course there are many courses, trainingmaterials, other things that you can do through structured learning, butindividually I'm talking about how do you create that interest in them and justleverage the avenues that they have, right? So that's the way to go. There's somuch material out there, right? So if you think about consulting firms that arepublishing things, use cases, be curious. And then also, it's important toreach into their own network outside to see, "Hey, this is a problem. Howare you solving and learning from that? " I mean, they should probablywatch your podcast as well, right?That's a good way for them to learn too.

Sri (29:07):

No, definitely. And I think you brought up some very keythings, right? You've got to be curious, you got to be learning, but you haveto be self-actuating. I think you kept saying that if you have that desire tochange, then you'll learn from AI, you can reach your network in doing it. Butwhat you're saying is the onus is on the individual. They have to come aboutdoing it. But from an organization perspective, you're also providing guidanceand material, maybe even mentors or coaching. I like share the examples like abrown bag session so that when people start to see that success, they're reallygetting excited about it. So one is self-actuating, one is organizational waysin which you bring people together. How has the journey been in a normaldistribution of doing this? Do you see people getting highly motivated oryou're seeing a lot of resistance groups.

(30:05):

Walk me through. What are you seeing on the ground?

Amrit (30:08):

It doesn't matter which transformation you'll go through.You'll always see people and the early adopters, the fast followers and theresisters that never change. That's always a situation in any organization. Andas long as we're able to focus on creating the right momentum, that you getmost of the organization to follow through, that's the important part. I knowyou didn't ask this question, but just on the flip side of it, we have to makesure that there are right controls in place as well. So as we are excited aboutmoving forward, especially in the regulated space that we are in, making surewe have the right guardrails that will help in ensuring that we are not makingany mistakes that will increase the compliance risk. So those are things - thateducation awareness should be planned and administered as well. So we do havethe governance frameworks and governance established.

(31:12):

One of the common mistakes that happens with thegovernance is it kind of becomes a stage-gater or a blocker rather than anenabler or a catalyst. So we need to rethink that in terms of how we establishthe governance.

Sri (31:27):

I was going to come to governance and stuff because we arein a regulated environment and GXP compliance and use of AI. And a lot of thequality groups are saying, you can do this or you can't change. As you start torethink bringing these different tools, you're literally looking at everyprocess from a GXP standpoint and saying, is it human in the middle? Is itagentic? And then what kind of controls do you have? Walk me through thattransformation because many people are early stage in that transformation. Andas you said, that becomes a stage gate and a blocker like, "Hey, it's toomuch risk, so we shouldn't do it. " Versus the leading organizationsactually say, "Yeah, it is risk, but we know how to mitigate it and thisis kind of how we're going about it. " So walk me through thattransformation because it's a big transformation going on.

Amrit (32:14):

One thing that's important that has to be reallyreinforced in people is you can't blame AI for something that went wrong. It'sa human that should be accountable for it, right? As long as we have the humanin the loop that's making the decision or the judgment call, that's animportant thing. Rest of it, then when you look at it, yes, you've got agentsdoing the job, but there should be right oversight, right checks in place thatsometimes agents can just be away from what they're supposed to do. So havingthose checks in place is important. So that's the principle that everyoneshould be on board. So if something goes wrong, you can't just blame an agentor something. So of course that may be the cause, but that's where, as yousaid, the human in the loop, the oversight comes in. When we put the frameworksin place, that is an essential component.

(33:12):

As long as we're able to make it, put that there. Eventhink about, for example, translations. So we are leveraging AI fortranslations. For anything that is regulated, you should always have a humanthat's reviewing it, right? Making sure ... I mean, you're not doing everypiece of work in terms of translation, but someone's reading it to make surethat it is accurate, it makes sense. And then we actually have not just one,but two reviews happening because that's important. So based on the process areathat you're in and the compliance risk or the risk of reputation that you mayhave for the organization, the scrutiny should be proportionate as we do that.

Sri (33:58):

How do you work across people who say, "No, no, no,this cannot be done, or this is what the regulation says." You're having alot of SOPs written, we are an SOP industry, everything has an SOP, butsometimes when you have to redefine the process, you're redoing the SOP, whenyou're into using AI agents, you're redoing the SOP. So walk me through thatfine line and balance between, hey, I could just follow the SOP and get out ofjail or I need to redo it. And it's a lot of hard work. Is the juice worth thesqueeze, right? Yeah. That's the key question everybody asks.

Amrit (34:34):

AThat's an excellent question. So especially when you havepeople that are kind of resisting, it's important to reason with them, right?So depending on where we are, the other thing that you could do is look atdoing paddle runs to build the trust. So there's that. And of course, you'regoing to have some people that are just beyond any reason you can't change.That's when you have to use the top down reinforcement there, right? But ingeneral, you can win people with reasoning, with demonstrating the proof. Andactually, a lot of times these are the people that are very valuable to theorganization because if you have everyone that's just going gung-ho about this,you may not have the right controls in place, right checks in place. So thesepeople in this with this mindset will actually help shoot holes, make theprocess better and make it more robust.

(35:36):

So I think it's important to tap into that kind of amindset to see how we can make it better, but along the way with them, with thetrust.

Sri (35:47):

So we've gone past a few good frameworks, right? How doyou select the right use case, think about outcomes first model, redefine theprocess, look at it in an end-to-end, then orchestrate, put the AI governance.This thing is happening so fast. Amrit, where are you seeing this? If we arehere in six months, what do you see if you're here in 12 months? Previously, Iused to ask questions in three to five years, which is meaningless now in theface of change. So I'm asking it now in a much more smaller chunks and saying,what happens at six months, 12 months, 18 months? Where do you see this goingin the next 12 and 18 months?

Amrit (36:26):

I want to answer that question by looking back. So if youplace yourself in 2023, 24, it's like chatbots that are just a little moreintelligent and beyond rule-based and things like that because they can answerquestions based on context. Then you have the LLM models that just give yousome guidance on what to do. Agents came in and now we are looking atleveraging agents in different ways. In the near future, what I see happeningis the--proliferation may not be a right word, but use of agents in many differentcases. But then the important thing that really sets apart the winners is howeffectively you're able to orchestrate the agents and how effectively you areable to refine them. So if you think about it, the critical path analogy thatwe talked about, once we improve something, you're already shifting thebaseline, right?

(37:30):

So then you have to start thinking about, "Hey, thiswas efficient with this. Now we've moved to a new level. How do we rethinkthis?" So that's the constant evolution that is the important part. Sothat's one thing I see that's happening. I mean, if you think about things likethe document generation and other things, that will follow its own path interms of how it's going to evolve, building that trust. But the biggest value Isee is with the agents where that autonomous thinking happens, of course, withthe human oversight on top.

Sri (38:06):

Just on that, because everybody says human in the loop,but by the time you get to such a complexity and you talked about batch and youmay have three million records of the batches and now AI is looking at it andgiving you an anomaly, a human can verify that the algorithm was run correctlyor the anomaly is correct or not within certain educated boundaries, but it'sgoing to get to a state where the human would not be able to keep up with someof the insights which are going on, especially as you start to build thisautonomous agentic. Are you seeing champion challenger models where there aredifferent role-based agents, one, the doer, one the reviewer, which thenaugments the human from at least giving multiple points of view? How do you seethat progressing?

Amrit (38:55):

That's an excellent point. And thank you for calling thatout. So you have agents that are doing the job and some organizations arethinking already about agents that oversee the agents and making sure they arewithin that, operating within the stated constraints, right? So that'simportant part. When I talk about human in the loop, the human should be veryvigilant about how the context is changing for the agents. So if, of course,they're operating on different variables, if at least one of the variables changessignificantly, the responses of the agent could go awry. So the human should bevigilant about that. I mean, you can argue you can have agents monitor that aswell, but you have to make sure we strike the right balance there. That'simportant. How are the agents, if they are drifting away, being able to detectthat sooner than later is important.

(39:55):

Sometimes you may just do a parallel, check, sampling somethings, right? So detect things and ultimately the end product, how is itmeeting the criteria? So that's important.

Sri (40:07):

My last question is that this completely redefines whatthe workforce is because before I was doing tasks and activities and I feltgood about it, I wrote an SOP or I did a task to do data queries and managementor I did a task of taking a case intake or case input. That role will be doneby agents. My task is to verify that that job was done right and having, as yousaid, critical thinking on figuring out if it was done right or appropriate. Soas you start to look into this timeframe, 12, 18 months, what kind of skillsetsshould people be having? And as these roles change, how do you have totransform your own way of how you look at work?

Amrit (40:53):

So it doesn't matter where one sits in the organization,being AI savvy is very important. Knowing what tools are at their disposal andhow to use them is very important. What's replaceable is their domain andknowledge expertise, right? So in terms of how things work, the human judgmentas I kind of kept on saying, that's an important part. So the people behaviors,how do you look at that? So those are things that an agent may not get to it.It's a human sentiment part. These are the things that we need to ensure peopleare focused on, especially when we start working cross-functionally across manyareas. If the organization has shifted in terms of roles, responsibilities, howdo we make sure that we are cognizant of it and then retrain whatever we need todo with the agents. So those are things that won't change. The creativity of anindividual is never going to get replaced, right?

(41:59):

So those are things that one needs to continue to focus onin their roles. I mean, there are going to be certain roles that certainly willcompletely be shifted by what AI can do. So as you're learning about it, it'simportant to know where the opportunities get created and work towards thatrather than just trying to stick on to just your own old role.

Sri (42:25):

Amrit, thank you so much. This was such an engagingdiscussion. We could go on for another half an hour or so. Really appreciateit. I think you gave a lot of good takeaways. On a final note, any keytakeaways you want the audience to get from this conversation as they go downthe AI journey?

Amrit (42:42):

Don't just wait. So that's the important part. Explore,learn, pick noted record zones first to dive in, work on it. I mean, if youfail, there's no big impact and then slowly evolve from that. That would be mysuggestion. You cannot ignore AI.

Sri (43:04):

Yeah. Thank you so much.

Danny (43:12):

Sri, you talked about Amrit being very pragmatic. What didyou think?

Sri (43:16):

I think it was a really fascinating discussion. We reallybroke down the process of identifying AI use cases from how do you go aboutselecting it? And it was a very interesting approach which he took on don'tjust look at point in time automation, look at it end to end. And then wereally explored how would you go about doing that from first redefining theprocess, then making sure that you're aligning it to the outcomes, and thenreally re-architecting your infrastructure to meet those needs.

Danny (43:47):

So as a for instance, he talked about using AI forevaluating CROs and their likelihood to meet recruitment deadlines, but youraised the issue of having access to the necessary data to do that. Howlimiting is it for companies to use AI as they'd like because of not havingaccess to necessary data?

Sri (44:09):

I think that's a very important problem which we need tosolve in the future. As we start to build agentic workflows, which are not justinternal to our organization, but external, you need to create the rightframework for data accessibility, as you call it. We need to have access to thedata, that data has to be of high quality, and that it has to be some goodgovernance and provenance around how that data can be used and be used to makeeffective decisions. In the recruiting example, what he's outlining is that aCRO would have the recruiting data, they need to make sure that they'reexternalizing that so that somebody through a API can go and consume that, butthey also have to ensure that those are accurate in point in time. So you can'tgive me recruiting data from 30 days back for me to then use that to makecritical decisions within the company around drug supply or other criticalactivities.

(45:04):

So we're going into a very interesting phase where themore we stitch together the end-to-end process, the more problems we'll findbecause each of the processes don't function the same way with the same qualityas you would need.

Danny (45:20):

Amrit talked about the need for really a comprehensiveapproach, whether it's a matter of enabling employees all the way throughneeding to map processes. It's a rather three-dimensional problem when you wantto implement AI. How thoughtful are companies about that? Are they solving thisas they see problems one by one, or are they thinking in comprehensive ways?

Sri (45:47):

There are different schools of philosophy down this path.The successful ones, and we've interviewed them in the podcast very similar toAmrit are look for the outcomes, look at the process end to end, thenorchestrate it with humans and AI, and then go and implement it. But mostpeople are very comfortable in automating a process or getting quick wins,which is not a bad place. It gets your company to adapt and adapting to AI andyou need to get quick wins. And so I think we're going to see a two speed process.You're going to see a process where there are going to be lots of pointautomations, which people are using to get experienced, comfortable in the useof AI and AI teammates, but the broader organizational impact is going to comefrom those end-to-end outcome driven programs where you're not looking at eachof the micro segments of the process, but looking at it end-to-end andre-architecting it from an AI standpoint on what is the work or task done by AIand what is the work or test done by the human?

Danny (46:49):

One thing that really stood out to me was when he wastalking about not blaming AI when things go wrong, the fact that there's got tobe a human in the loop and they have to own what happens. What did you thinkabout that as a key part to any framework?

Sri (47:03):

I think accountability lives within the humans in aregulated environment with good clinical practices. Again, the emphasis of theregulation is on who is the human who's verifying it and putting their namesbehind it, either from a signature standpoint or accountability standpoint. Sothat's very important, but we also explored that at some point in time, theagents are going to be so super efficient and smart that the humans may not beable to catch up with every aspect of what it is doing. And therefore, youreally need to have good measures to find out how you can trust the model andhow you can do that. And he talked about it. It's very important to be complexaware. It's very important to know what the compliance and regulations are.It's very important to know when it's going away from what it's desiring to do.And these are all experiences which come out of domain and working on theseproblem sets over and over again.

Danny (47:56):

Well, Sri, it was a great conversation and thanks asalways.

Sri (48:00):

Thank you Daniel. Appreciate it.

Danny (48:05):

Thanks again to our sponsor, Agilisium Labs. Life SciencesDNA is a bimonthly podcast produced by the Levine Media Group with productionsupport from FullView Media. Be sure to follow us on your preferred podcastplatform. Music for this podcast is provided courtesy of the Jonah LevineCollective. We'd love to hear from you. Pop us a note atdanny@levinemediagroup.com. For Life Sciences DNA, I'm Daniel Levine. Thanksfor joining us.

‍