Research and experiments are at the guts of any exercise that involves AI. Constructing LLM applications is not any different. Unlike traditional web apps that follow a pre-decided design that has little to no variation, AI-based designs rely heavily on the experiments and might change depending on early outcomes. The success factor is experimenting on clearly defined expectations in iterations, followed by constantly evaluating each iteration. In LLM-native development, the success criteria is often the standard of the output, which implies that the main target is on producing accurate and highly relevant results. This will be either a response from chatbot, text summary, image generation and even an motion (Agentic approach) defined by LLM. Generating quality results consistently requires a deep understanding of the underlying language models, constant fine-tuning of the prompts, and rigorous evaluation to make sure that the applying meets the specified standards.
What form of tech skill set do you wish within the team?
You may assume that a team with only a handful of information scientists is sufficient to construct you an LLM application. But in point of fact, engineering skills are equally or more necessary to truly ‘deliver’ the goal product, as LLM applications don’t follow the classical ML approach. For each data scientists and software engineers, some mindset shifts are required to get accustomed to the event approach. I even have seen each roles making this journey, corresponding to data scientists getting accustomed to cloud infrastructure and application deployment and then again, engineers familiarizing themselves with the intricacies of model usage and evaluation of LLM outputs. Ultimately, you wish AI practitioners in team who are usually not there simply to ‘code’, somewhat research, collaborate and improve on the AI applicability.
Do I really want to ‘experiment’ since we’re going to use pre-trained language models?
Popular LLMs like GPT-4o are already trained on large set of information and able to recognizing and generating texts, images etc., hence you don’t want to ‘train’ some of these model. Only a few scenarios might require to fine-tune the model but that can also be achievable easily without having classical ML approach. Nonetheless, let’s not confuse the term ‘experiment’ with ‘model training’ methodology utilized in predictive ML. As I’ve mentioned above that quality of the applying output matters. organising iterations of experiments might help us to succeed in the goal quality of result. For instance — in the event you’re constructing a chatbot and you desire to control how the bot output should appear to be to finish user, an iterative and experimental approach on prompt improvement and fine-tuning hyper parameters will enable you find the proper technique to generate most accurate and consistent output.
Construct a prototype early in your journey
Construct a prototype (also known as MVP — minimum viable product) with only the core functionalities as early as possible, ideally inside 2–4 weeks. Should you’re using a knowledge base for RAG approach, use a subset of information to avoid extensive data pre-processing.
- Gaining quick feedback from a subset of goal users helps you to know whether the answer is meeting their expectations.
- Review with stakeholders to not only show the nice results, also discuss the restrictions and constraints your team came upon during prototype constructing. That is crucial to mitigate risks early, and likewise to make informed decision regarding delivery.
- The team can finalize the tech stack, security and scalability requirements to maneuver the prototype to totally functional product and delivery timeline.
Determine in case your prototype is prepared for constructing into the ‘product’
Availability of multiple AI-focused samples have made it super easy to create a prototype, and initial testing of such prototypes normally delivers promising results. By the point the prototype is prepared, the team may need more understanding on success criteria, market research, goal user base, platform requirements etc. At this point, considering following questions might help to make a decision the direction to which the product can move:
- Does the functionalities developed within the prototype serve the first need of the top users or business process?
- What are the challenges that team faced during prototype development that may come up in production journey? Are there any methods to mitigate these risks?
- Does the prototype pose any risk close to responsible AI principles? In that case, then what guardrails will be implemented to avoid these risks? (We’ll discuss more on this point partly 2)
- If the answer is to be integrated into an existing product, what is likely to be a show-stopper for that?
- If the answer handles sensitive data, are effective measures been taken to handle the info privacy and security?
- Do you might want to define any performance requirement for the product? Is the prototype results promising on this aspect or will be improved further?
- What are the safety requirements does your product need?
- Does your product need any UI? (A typical LLM-based use case is chatbot, hence UI requirements are crucial to be defined as early as possible)
- Do you will have a value estimate for the LLM usage out of your MVP? How does it appear to be considering the estimated scale of usage in production and your budget?
Should you can gain satisfactory answers to a lot of the questions after initial review, coupled with good results out of your prototype, then you definitely can move forward with the product development.
Stay tuned for part 2 where I’ll discuss what must be your approach to product development, how you’ll be able to implement responsible AI early into the product and price management techniques.
Please follow me if you desire to read more such content about latest and exciting technology. If you will have any feedback, please leave a comment. Thanks 🙂
