Start by defining the target for every agent or prompt. Keep on with one cognitive process type per agent, similar to: conceptualizing a landing page, choosing components, or generating content for specific sections.
Having clear boundaries maintains focus and clarity in your LLM interactions, aligning with the Engineering Techniques apex of the LLM Triangle Principle.
“Each step in our flow is a standalone process that must occur to realize our task.”
For instance, avoid combining different cognitive processes in the identical prompt, which could yield suboptimal results. As a substitute, break these into separate, focused agents:
def generate_landing_page_concept(input_data: LandingPageInput) -> LandingPageConcept:
"""
Generate a landing page concept based on the input data.
This function focuses on the creative technique of conceptualizing the landing page.
"""
passdef select_landing_page_components(concept: LandingPageConcept) -> List[LandingPageComponent]:
"""
Select appropriate components for the landing page based on the concept.
This function is responsible only for selecting components,
not for generating their content or layout.
"""
pass
def generate_component_content(component: LandingPageComponent, concept: LandingPageConcept) -> ComponentContent:
"""
Generate content for a selected landing page component.
This function focuses on creating appropriate content based on the component type and overall concept.
"""
pass
By defining clear boundaries for every agent, we are able to be sure that each step in our workflow is tailored to a selected mental task. This can improve the standard of outputs and make it easier to debug and refine.
Define clear input and output structures to reflect the objectives and create explicit data models. This practice touches on the LLM Triangle Principles’ Engineering Techniques and Contextual Data apexes.
class LandingPageInput(BaseModel):
brand: str
product_desc: str
campaign_desc: str
cta_message: str
target_audience: str
unique_selling_points: List[str]class LandingPageConcept(BaseModel):
campaign_desc_reflection: str
campaign_motivation: str
campaign_narrative: str
campaign_title_types: List[str]
campaign_title: str
tone_and_style: List[str]
These Pydantic models define the structure of our input and output data and define clear boundaries and expectations for the agent.
Place validations to make sure the standard and moderation of the LLM outputs. Pydantic is superb for implementing these guardrails, and we are able to utilize its native features for that.
class LandingPageConcept(BaseModel):
campaign_narrative: str = Field(..., min_length=50) # native validations
tone_and_style: List[str] = Field(..., min_items=2) # native validations# ...remainder of the fields... #
@field_validator("campaign_narrative")
@classmethod
def validate_campaign_narrative(cls, v):
"""Validate the campaign narrative against the content policy, using one other AI model."""
response = client.moderations.create(input=v)
if response.results[0].flagged:
raise ValueError("The provided text violates the content policy.")
return v
In this instance, ensuring the standard of our application by defining two varieties of validators:
- Using Pydanitc’s
Fieldto define easy validations, similar to a minimum of two tone/style attributes, or a minimum of fifty characters within the narrative - Using a custom
field_validatorthat ensures the generated narrative is complying with our content moderation policy (using AI)
Structure your LLM workflow to mimic human cognitive processes by breaking down complex tasks into smaller steps that follow a logical sequence. To try this, follow the SOP (Standard Operating Procedure) tenet of the LLM Triangle Principles.
“Without an SOP, even probably the most powerful LLM will fail to deliver consistently high-quality results.”
4.1 Capture hidden implicit cognition jumps
In our example, we expect the model to return LandingPageConcept in consequence. By asking the model to output certain fields, we guide the LLM much like how a human marketer or designer might approach making a landing page concept.
class LandingPageConcept(BaseModel):
campaign_desc_reflection: str # Encourages evaluation of the campaign description
campaign_motivation: str # Prompts desirous about the 'why' behind the campaign
campaign_narrative: str # Guides creation of a cohesive story for the landing page
campaign_title_types: List[str]# Promotes brainstorming different title approaches
campaign_title: str # The ultimate decision on the title
tone_and_style: List[str] # Defines the general feel of the landing page
The LandingPageConcept structure encourages the LLM to follow a human-like reasoning process, mirroring the subtle mental leaps (implicit cognition “jumps”) that an authority would make instinctively, just as we modeled in our SOP.
4.2 Breaking complex processes into multiple steps/agents
For complex tasks, break the method down into various steps, each handled by a separate LLM call or “agent”:
async def generate_landing_page(input_data: LandingPageInput) -> LandingPageOutput:
# Step 1: Conceptualize the campaign
concept = await generate_concept(input_data)# Step 2: Select appropriate components
selected_components = await select_components(concept)
# Step 3: Generate content for every chosen component
component_contents = {
component: await generate_component_content(input_data, concept, component)
for component in selected_components
}
# Step 4: Compose the ultimate HTML
html = await compose_html(concept, component_contents)
return LandingPageOutput(concept, selected_components, component_contents, html)
This multi-agent approach aligns with how humans tackle complex problems — by breaking them into smaller parts.
YAML is a popular human-friendly data serialization format. It’s designed to be easily readable by humans while still being easy for machines to parse — which makes it classic for LLM usage.
I discovered YAML is especially effective for LLM interactions and yields significantly better results across different models. It focuses the token processing on worthwhile content quite than syntax.
YAML can also be way more portable across different LLM providers and lets you maintain a structured output format.
async def generate_component_content(input_data: LandingPageInput, concept: LandingPageConcept,component: LandingPageComponent) -> ComponentContent:
few_shots = {
LandingPageComponent.HERO: {
"input": LandingPageInput(
brand="Mustacher",
product_desc="Luxurious mustache cream for grooming and styling",
# ... remainder of the input data ...
),
"concept": LandingPageConcept(
campaign_title="Have fun Dad's Dash of Distinction",
tone_and_style=["Warm", "Slightly humorous", "Nostalgic"]
# ... remainder of the concept ...
),
"output": ComponentContent(
motivation="The hero section captures attention and communicates the core value proposition.",
content={
"headline": "Honor Dad's Distinction",
"subheadline": "The Art of Mustache Care",
"cta_button": "Shop Now"
}
)
},
# Add more component examples as needed
}sys = "Craft landing page component content. Respond in YAML with motivation and content structure as shown."
messages = [{"role": "system", "content": sys}]
messages.extend([
message for example in few_shots.values() for message in [
{"role": "user", "content": to_yaml({"input": example["input"], "concept": example["concept"], "component": component.value})},
{"role": "assistant", "content": to_yaml(example["output"])}
]
])
messages.append({"role": "user", "content": to_yaml({"input": input_data, "concept": concept, "component": component.value})})
response = await client.chat.completions.create(model="gpt-4o", messages=messages)
raw_content = yaml.safe_load(sanitize_code_block(response.selections[0].message.content))
return ComponentContent(**raw_content)
Notice how we’re using few-shot examples to “show, don’t tell” the expected YAML format. This approach is simpler than explicit instructions in prompt for the output structure.
Fastidiously consider the best way to model and present data to the LLM. This tip is central to the Contextual Data apex of the LLM Triangle Principles.
“Even probably the most powerful model requires relevant and well-structured contextual data to shine.”
Don’t throw away all the info you may have on the model. As a substitute, inform the model with the pieces of data which might be relevant to the target you defined.
async def select_components(concept: LandingPageConcept) -> List[LandingPageComponent]:
sys_template = jinja_env.from_string("""
Your task is to pick probably the most appropriate components for a landing page based on the provided concept.
Select from the next components:
{% for component in components %}
- {{ component.value }}
{% endfor %}
You MUST respond ONLY in a sound YAML list of chosen components.
""")sys = sys_template.render(components=LandingPageComponent)
prompt = jinja_env.from_string("""
Campaign title: "{{ concept.campaign_title }}"
Campaign narrative: "{{ concept.campaign_narrative }}"
Tone and elegance attributes: { join(', ') }
""")
messages = [{"role": "system", "content": sys}] + few_shots + [
{"role": "user", "content": prompt.render(concept=concept)}]
response = await client.chat.completions.create(model="gpt-4", messages=messages)
selected_components = yaml.safe_load(response.selections[0].message.content)
return [LandingPageComponent(component) for component in selected_components]
In this instance, we’re using Jinja templates to dynamically compose our prompts. This creates focused and relevant contexts for every LLM interaction elegantly.
“Data fuels the engine of LLM-native applications. A strategic design of contextual data unlocks their true potential.”
Few-shot learning is essential technique in prompt engineering. Providing the LLM with relevant examples significantly improves its understanding of the duty.
Notice that in each approaches we discuss below, we reuse our Pydantic models for the few-shots — this trick ensures consistency between the examples and our actual task! Unfortunately, I learned it the hard way.
6.1.1 Examples Few-Shot Learning
Take a take a look at the few_shots dictionary in section 5. On this approach:
Examples are added to the messages list as separate user and assistant messages, followed by the actual user input.
messages.extend([
message for example in few_shots for message in [
{"role": "user", "content": to_yaml(example["input"])},
{"role": "assistant", "content": to_yaml(example["output"])}
]
])
# then we are able to add the user prompt
messages.append({"role": "user", "content": to_yaml(input_data)})
By placing the examples as messages, we align with the training methodology of instruction models. It allows the model to see multiple “example interactions” before processing the user input — helping it understand the expected input-output pattern.
As your application grows, you possibly can add more few-shots to cover more use-cases. For much more advanced applications, consider implementing dynamic few-shot selection, where probably the most relevant examples are chosen based on the present input.
6.1.2 Task-Specific Few-Shot Learning
This method uses examples directly related to the present task inside the prompt itself. As an example, this prompt template is used for generating additional unique selling points:
Generate {{ num_points }} more unique selling points for our {{ brand }} {{ product_desc }}, following this style:
{% for point in existing_points %}
- {{ point }}
{% endfor %}
This provides targeted guidance for specific content generation tasks by including the examples directly within the prompt quite than as separate messages.
While fancy prompt engineering techniques like “Tree of Thoughts” or “Graph of Thoughts” are intriguing, especially for research, I discovered them quite impractical and infrequently overkill for production. For real applications, give attention to designing a correct LLM architecture(aka workflow engineering).
This extends to using agents in your LLM applications. It’s crucial to grasp the excellence between standard agents and autonomous agents:
Agents: “Take me from A → B by doing XYZ.”
Autonomous Agents:“Take me from A → B by doing something, I don’t care how.”
While autonomous agents offer flexibility and quicker development, they can even introduce unpredictability and debugging challenges. Use autonomous agents rigorously — only when the advantages clearly outweigh the potential lack of control and increased complexity.
Continuous experimentation is significant to improving your LLM-native applications. Do not be intimidated by the thought of experiments — they might be as small as tweaking a prompt. As outlined in “Constructing LLM Apps: A Clear Step-by-Step Guide,” it’s crucial to establish a baseline and track improvements against it.
Like all the pieces else in “AI,” LLM-native apps require a research and experimentation mindset.
One other great trick is to try your prompts on a weaker model than the one you aim to make use of in production(similar to open-source 8B models) — an “okay” performing prompt on a smaller model will perform significantly better on a bigger model.
