Patrick Leung, CTO of Faro Health, drives the corporate’s AI-enabled platform, which simplifies and quickens clinical trial protocol design. Faro Health’s tools enhance efficiency, standardization, and accuracy in trial planning, integrating data-driven insights and streamlined processes to cut back trial risks, costs, and patient burden.
Faro Health empowers clinical research teams to develop optimized, standardized trial protocols faster, advancing innovation in clinical research.
You spent a few years constructing AI at Google. What were among the most fun projects you worked on during your time at Google, and the way did those experiences shape your approach to AI?
I used to be on the team that built Google Duplex, a conversational AI system that called restaurants and other businesses on the user’s behalf. This was a top secret project that was stuffed with extremely talented people. The team was fast-moving, consistently trying out latest ideas, and there have been cool demos of the most recent things people were working on every week. It was very inspiring to be on a team like that.
Considered one of the various things I learned on this team is that even whenever you’re working with the most recent AI models, sometimes you continue to just need to be scrappy to get the user experience and value you would like. To be able to generate hyper-realistic verbal conversations, the team stitched together recordings interspersed with temporizers like “um” to make the conversation sound more natural. It was a lot fun reading what the press needed to say about why those “ums” were there after we launched!
Each you and the CEO of Faro come from large tech firms. How has your past experience influenced the event and strategy of Faro?
Several times in my profession I’ve built firms that sell various services to large firms. Faro too is targeting the world’s largest pharma firms so there may be loads of experience around what it takes to win over and partner with large enterprises that is extremely relevant here. Working at Two Sigma, a big algorithmic hedge fund based in Recent York City, really shaped how I approach data science. They’ve a rigorous hypothesis-driven process whereby all latest ideas go right into a research plan and are tested thoroughly. Additionally they have a really well-developed data engineering organization for onboarding latest data sets and performing feature engineering. As Faro deepens its AI capabilities to tackle more problems in clinical trial development, this approach will likely be highly relevant and applicable to what we’re doing.
Faro Health is built around simplifying the complexity of clinical trial design with AI. Coming from a non-clinical background, what was the “aha moment” that led you to grasp the precise pain points in protocol design that needed to be addressed?
My first “aha moment” happened once I encountered the concept of “Eroom’s Law”. Eroom isn’t an individual, it’s just “Moore” spelt backwards. This tongue-in-cheek name is a reference to the proven fact that over the past 50 years, inflation adjusted clinical drug development costs and timelines have roughly doubled every 9 years. This flies within the face of all the information technology revolution, and just boggled my mind. It really sold me on the actual fact there may be an unlimited problem to resolve here!
As I got deeper into this domain and commenced understanding the underlying problems more fully, there have been many more insights like this. A fundamental and really obvious one is that Word docs aren’t an excellent format to design and store highly complex clinical trials! It is a key commentary, borne of our CEO Scott’s clinical experience, that Faro was built upon. There may be also the commentary that over time, trials are likely to get increasingly more complex, as clinical study teams literally copy and paste past protocols, after which add latest assessments with the intention to gather more data. Providing users with as many precious insights as possible, as early as possible, within the study design process is a key value proposition for Faro.
What role does AI play in Faro’s platform to make sure faster and more accurate clinical trial protocol design? How does Faro’s “AI Co-Creator” tool differentiate from other generative AI solutions?
It’d sound obvious, but you possibly can’t just ask ChatGPT to generate a clinical trial protocol document. To start with, you could have highly specific, structured trial information equivalent to the Schedule of Activities represented intimately with the intention to surface the correct information within the highly technical sections of the protocol document. Second, there are various details and specific clauses that should be present within the documentation for certain sorts of trials, and a certain style and level of detail that is anticipated by medical writers and reviewers. At Faro, we built a proprietary protocol evaluation system to make sure the content that the massive language model (LLM) was coming up with will meet users’ and regulators’ exacting standards.
As trials for rare diseases and immuno-oncology grow to be more complex, how does Faro be certain that AI can meet these specialized demands without sacrificing accuracy or quality?
A model is simply pretty much as good as the information it’s trained on. So because the frontier of recent medicine advances, we’d like to maintain pace by training and testing our models with the most recent clinical trials. This requires that we continually expand our library of digitized clinical protocols – we’re extremely pleased with the amount of clinical trial protocols that we’ve already brought into our data library at Faro, and we’re all the time prioritizing the expansion of this dataset. It also requires us to lean heavily on our in-house team of clinical experts, who consistently evaluate the output of our model and supply any crucial changes to the “evaluation checklists” we use to make sure its accuracy and quality.
Faro’s partnership with Veeva and other leading firms integrates your platform into the broader clinical trial ecosystem. How do these collaborations help streamline all the trial process, from protocol design to execution?
The center of a clinical trial is the protocol, which Faro’s Study Designer helps our customers design and optimize. The protocol informs all the pieces downstream concerning the trial, but traditionally, protocols are designed and stored in Word documents. Thus, one in every of the massive challenges in operationalizing clinical development today is the constant transcription or “translation” of information from the protocol or other document-based sources to other systems and even other documents. As you possibly can imagine, having humans manually translate document-based information into various systems by hand is incredibly inefficient, and introduces many opportunities for errors along the best way.
Faro’s vision is a unified platform where the “definition” or elements of a clinical trial can flow from the design system where they’re first conceived, downstream to varied systems or needed through the operational phase of the trial. When this type of seamless information flow is in place, there’s a big opportunity for automation and improved quality, meaning we are able to dramatically reduce the time and price to design and implement a clinical trial. Our partnership with Veeva to attach our Study Designer to Veeva Vault EDC is only one step on this direction, with quite a bit more to return.
What are among the key challenges AI faces in simplifying clinical trials, and the way does Faro overcome them, particularly around ensuring transparency and avoiding issues like bias or hallucination in AI outputs?
There may be a much higher bar for clinical trial documents than in most other domains. These documents affect the lives of real people, and thus go through a highly-exacting regulatory review process. After we first began generating clinical documents using an LLM, it was clear that with off-the-shelf models, the output was nowhere near meeting expectations. Unsurprisingly, the tone, level of detail, formatting – all the pieces – was way off, and was far more oriented to general-purpose business communications, fairly than expert clinical grade documents. Needless to say hallucination and in addition straight up omission of crucial details were major challenges. To be able to develop a generative AI solution that might meet the high standard for domain specificity and quality that our users expect, we needed to spend loads of time collaborating with clinical experts to plot guidelines and evaluation checklists that ensured our output wasn’t hallucinating or just omitting key details, and had the correct tone. We also needed to supply the capability for end users to supply their very own guidance and corrections to the output, as different customers have differing templates and standards that guide their document authoring process.
There’s also the challenge that the detailed clinical data needed to totally generate the trial protocol documentation will not be available, often stored deep in other complex documents equivalent to the investigational brochure. We’re taking a look at using AI to assist extract such information and make it available to be used in generating clinical protocol document sections.
Looking forward, how do you see AI evolving within the context of clinical trials? What role will Faro play within the digital transformation of this space over the following decade?
As time goes on, AI will help improve and optimize increasingly more decisions and processes throughout the clinical development process. We’ll give you the chance to predict key outcomes based on protocol design inputs, like whether the study team can expect enrollment challenges, or whether the study would require an amendment as a consequence of operational challenges. With that sort of predictive insight, we are going to give you the chance to assist optimize the downstream operations of the trial, ensuring each sites and patients have the perfect experience, and that the trial’s likelihood of operational success is as high as possible. Along with exploring these possibilities, Faro also plans to proceed generating a variety of various clinical documentation so that each one of the filing and paperwork processes of the trial are efficient and far less error-prone. And we foresee a world where AI enables our platform to grow to be a real design partner, engaging clinical scientists in a generative dialog to assist them design trials that make the correct tradeoffs between patient burden, site burden, time, cost, and complexity.
How does Faro’s deal with patient-centric design impact the efficiency and success of clinical trials, particularly when it comes to reducing patient burden and improving study accessibility?
Clinical trials are sometimes caught between the competing needs of collecting more participant data – which implies more assessments or tests for the patient – and managing a trial’s operational feasibility, equivalent to its ability to enroll and retain participants. But patient recruitment and retention are among the most vital challenges to the successful completion of a clinical trial today – by some estimates, as many as 20-30% of patients who elect to take part in a clinical trial will ultimately drop out as a consequence of the burden of participation, including frequent visits, invasive procedures and complicated protocols. Although clinical research teams are aware of the impact of high burden trials on patients, actually doing anything concrete to cut back burden might be hard in practice. We imagine one in every of the barriers to reducing patient burden is commonly the lack to readily quantify it – it’s hard to measure the impact to patients when your design is in a Word document or a pdf.
Using Faro’s Study Designer, clinical development teams can get real-time insights into the impact of their specific protocol on patient burden through the protocol planning process itself. By structuring trials and providing analytical insights into their cost, patient burden, complexity early through the trials’ design stage, Faro provides clinical research teams with a really effective solution to optimize their trial designs by balancing these aspects against scientific needs to gather more data. Our customers love the actual fact we give them visibility into patient burden and related metrics at a degree in development where changes are easy to make, they usually could make informed tradeoffs where crucial. Ultimately, we’ve seen our customers save 1000’s of hours of collective patient time, which we all know can have an instantaneous positive impact for study participants, while also helping ensure clinical trials can each initiate and complete on time.
What advice would you give to startups or firms trying to integrate AI into their clinical trial processes, based in your experiences at each Google and Faro?
Listed below are the most important takeaways I’d offer thus far from our experience applying AI to this domain:
- Divide and evaluate your AI prompts. Large language models like GPT aren’t designed to output clinical grade documentation. So should you’re planning to make use of gen AI to automate clinical trial document authoring, you could have an evaluation framework that ensures the generated output is accurate, complete, has the correct level of detail and tone, and so forth. This requires loads of careful testing of the model guided by clinical experts.
- Use a structured representation of a trial. There isn’t any way you possibly can generate the required data analytics with the intention to design an optimal clinical trial with out a structured repository. Many firms today use Word docs – not even Excel! – to model clinical trials. This should be done with a structured domain model that accurately represents the complexity of a trial – its schema, objectives and endpoints, schedule of assessments, and so forth. This requires loads of input and feedback from clinical experts.
- Clinical experts are crucial for quality. As seen within the previous two points, having clinical experts directly involved within the design and testing of any AI based clinical development system is totally critical. That is far more so than another domain I’ve worked in, just because the knowledge required is so specialized, detailed, and pervades any product you try and construct on this space.
We’re consistently trying latest things and frequently share our findings to our blog to assist firms navigate this space.
