Advanced prompt engineering for smarter AI assistants

Although many organizations have adopted generative artificial intelligence (GenAI) to streamline their internal operations, the simple question-answer chat experience is insufficient for the demanding tasks of the enterprise. Instead, many enterprises are turning to GenAI-based, smart AI assistants. Fully leveraging the capabilities of AI assistants, however, requires sophisticated prompts tailored to the task at hand. 

Prompt engineering is the art and science of crafting effective prompts to guide an AI model to provide the most useful results. This technique is especially important when using AI assistants. Most GenAI users are familiar with basic prompt engineering techniques. However, for enterprises that lean on AI assistants, certain advanced prompt engineering techniques will serve them well.

Understanding prompt engineering

Prompt engineering is the process of preparing an instruction (also known as a prompt) for a large language model (LLM) to perform a task. For example, consider this prompt: Write a haiku that defines prompt engineering.

Here is the response from GPT-4o:

Crafting clear inputs,
Guiding AI's bright response,
Answers shaped with care.

Prompt engineering is an important part of successfully interacting with GenAI systems. This is especially true when using AI assistants. The knowledge of LLMs—especially GPT (which stands for “generative pre-trained transformer”) models—is frozen. These models don’t learn on the fly. However, they can perform in-context learning by utilizing additional knowledge provided in the prompt.

Therefore, the prompt is your critical opportunity to customize the behavior of an AI assistant during your interaction.

Basic prompt engineering techniques

Many basic prompt engineering techniques have become second nature to users that interact with GenAI applications regularly. In this section, we’ll cover basic prompt engineering techniques that will help you to improve the quality of your results. .

Be specific and clear

LLMs, and even smart AI assistants, are not magical. If your prompts are vague, then the responses will probably not meet your expectations. Here is an example of a poor prompt:

python
web app

It’s difficult to tell what the response ought to be, or even what the user desired as a response.

On the other hand, here is a good example of a prompt that is specific and clear:

Write a skeleton for a Python web app using FastAPI. Divide it into multiple files. Include a README.md and a LICENSE file.

Provide context

It is a good practice to include some relevant context in the prompt before stating the actual task. The model will digest this additional content and use it to generate the best response.

For example:

Context: Here is a FastAPI project structure

my_fastapi_app/
├── app/
│   ├── main.py
│   ├── models.py
│   ├── schemas.py
│   ├── database.py
│   ├── crud.py
│   └── routers/
│       ├── __init__.py
│       └── items.py
├── tests/
│   ├── __init__.py
│   └── test_main.py
├── requirements.txt
├── README.md
└── LICENSE

Task: 
Write a skeleton for a Python web app using FastAPI. Divide it into multiple files. Include a README.md and a LICENSE file.

Ask direct questions

The adage, “If you want direct answers, then ask direct questions.” applies to LLMs as well. Remember that models have a training cutoff date, so they might provide out-of-date answers. However, some smart AI assistants may use web browsing tools to enrich their context. Nonetheless, responses should always be double-checked for accuracy.

Consider the following poor example:

I’m not sure who won. Do you know?

A better prompt would be:

Who won the NBA championship in 2020?

Use examples

LLMs and AI assistants are great at following examples. If you want your answer in a particular structure, provide an example as part of the prompt, like the one below.

What are reasonable times to eat different meals?

For example… Breakfast: 7am-10am

Given this kind of prompt, you can expect a response like the following:

Here are reasonable times to eat different meals:

Breakfast: 7am-10am
Lunch: 12pm-2pm
Afternoon Snack: 2pm-4pm
Dinner: 6pm-8pm

Specify the format

It is often useful to request your AI assistant to provide a response that follows a very specific format. This may be helpful if you plan to use the response for further processing downstream. To specify the format, you might provide a prompt like this:

What are reasonable times to eat different meals?

For example… Breakfast: 7am-10am

Respond with JSON only.

The response would likely take on a form, such as this:

{
  "breakfast": "7am-10am",
  "lunch": "12pm-2pm",
  "afternoon_snack": "2pm-4pm",
  "dinner": "6pm-8pm",
}

If you want even more granular control of the format, then you can even provide a JSON schema.

Use personas

One of the most useful (and fun) prompt engineering techniques is to ask your assistant to take on a persona or play a role. For example:

You are a laid-back, super-chill yoga teacher. You recently became an expert in implementing GenAI applications. In two sentences, which programming language is best for working with LLMs?

If you were to use this prompt in an application such as ChatGPT, you would likely receive a response similar to this:

Hey there, friend. Python is your best buddy for working with large language models because it's super easy to use and has awesome libraries like TensorFlow and PyTorch to help you get things done smoothly. Namaste!

These basic prompt techniques are the bread and butter of interacting effectively with LLMs and AI assistants. However, more advanced techniques can yield much more sophisticated interactions.

Advanced prompt engineering techniques

When you need precise and high-quality responses, especially for complex and critical workflows, certain advanced prompt engineering techniques can be extremely powerful.

Few-shot prompting

Few-shot prompting is similar to the basic technique of providing examples, except it provides more examples. For complex domains, a single example might not be sufficient for the assistant to pick up the pattern.

Chain of Thought (CoT) prompting

AI assistants may get lost if they are asked to perform a big task that requires multiple logical steps to complete. The Chain of Thought (CoT) technique guides the assistant by showing them how to break down an example task. After providing this guidance, you would then ask the assistant to perform the actual task.

In the following example, the prompt starts by showing how to solve a math problem with variables. Then, it poses another similar problem:

Q: 2x * 3 / 8 if x = 4
A:
First, substitute x = 4 into the expression:
 2x * 3 / 8 
So, it becomes:
 2 * 4 * 3 / 8 
Now, let’s simplify that step by step:
 2 * 4 = 8 
 8 * 3 = 24 
 24 / 8 = 3 
So, the result is:
 3 
Q: (x * 5555 + y * 6) / z when x = 2 * y and y = z + 8 and z = 0.5

The above CoT prompt would be considered a one-shot prompt, since only one example was provided. By providing multiple examples for CoT, you can combine few-shot prompting and CoT.

Contextual priming

The contextual priming technique involves providing additional context before instructing the assistant with the task itself. For example, if the user asks the AI assistant to create a demo web application in Python using the BlackSheep framework, the prompt can include some of the BlackSheep documentation as context. For example:

<context>
  the contents of https://www.neoteroi.dev/blacksheep/getting-started/
</context>

Build a TODO list web app using BlackSheep. Produce only code. No explanations.

Self-consistency

Although CoT prompting is a great technique, it doesn’t work as well on smaller models (such as Mistral AI or OpenLLaMA). If your AI assistant uses a smaller model, then self-consistency can be a useful and powerful technique. With self-consistency, the AI assistant queries the model multiple times and evaluates the responses to find the majority response.

For example, you can direct the model to generate three responses and then conclude with a final response based on the most consistently generated response.

Output 1:
When I was 6 my sister was half my age, so she was 3. Now I am 70, so she is 70 - 3 = 67. The answer is 67.

Output 2:
When the narrator was 6, his sister was half his age, which is 3. Now that the narrator is 70, his sister would be 70 - 3 = 67 years old. The answer is 67.

Output 3:
When I was 6 my sister was half my age, so she was 3. Now I am 70, so she is 70/2 = 35. The answer is 35.

Conclusion:
The most consistent answer is 67 (2 out of 3)

Reasoning and Acting (ReAct)

The techniques explored so far rely on the assistant’s intrinsic knowledge (from the underlying model) and the information in the prompt. The Reasoning and Acting (ReAct) technique expands the horizons of the AI assistant by guiding it through a structured problem-solving process: iterative reasoning and acting. With this technique, your AI assistant might also be able to leverage pre-configured tools, such as a browser, Python interpreter, or an API.

A ReAct approach to a prompt may look like this:

To perform the task I am about to request, I want you to iterate through a cycle that involves the following stages:

Articulate problem
Determine sub-problems
Search for solution to sub-problem
Combine sub-problem solutions to solve overall problem

Keep iterating through those stages until you arrive at an answer to my question. In your response, show me all of the stages for all of the iterations, so that I can know how you arrived at your answer.

Here is the task:
Map me a driving route from Salt Lake City, Utah to Glacier National Park in Montana.

By guiding the AI assistant to use ReAct in how it executes the task, you’ll have much greater insight into how that final response was composed.

Given the above prompt, a response may look like this:

### Iteration 1:

#### Articulate problem:

Map a driving route from Salt Lake City, Utah to Glacier National Park in Montana.
#### Determine sub-problems:

1. Identify the starting point and endpoint.
2. Determine major highways and interstates connecting the two locations.
3. Identify key stops or cities along the route for navigation.
4. Optimize the route for travel time and distance.

#### Search for solution to sub-problem:
**Sub-problem 1: Identify the starting point and endpoint.**
- Starting Point: Salt Lake City, Utah
- Endpoint: Glacier National Park, Montana
**Sub-problem 2: Determine major highways and interstates connecting the two locations.**
- Research the main highways and interstates connecting Utah and Montana.

#### Combine sub-problem solutions to solve overall problem:
- Starting Point: Salt Lake City, Utah
- Major Highways: Interstate 15 (I-15) and US-93
- Endpoint: Glacier National Park, Montana

### Iteration 2:

#### Articulate problem:
Map the specific driving route with key stops along major highways and optimize for travel time and distance.

#### Determine sub-problems:
1. Identify the best route from Salt Lake City to I-15 N.
2. Determine the route from I-15 to US-93.
3. Identify the best route from US-93 to Glacier National Park.
4. Include key stops for navigation and rest.

#### Search for solution to sub-problem:

**Sub-problem 1: Identify the best route from Salt Lake City to I-15 N.**
- Research the direct routes within Salt Lake City to reach I-15.

**Sub-problem 2: Determine the route from I-15 to US-93.**
- Research connecting highways and interstates.

**Sub-problem 3: Identify the best route from US-93 to Glacier National Park.**
- Research the final stretch of the journey.

**Sub-problem 4: Include key stops for navigation and rest.**
- Identify cities or major stops for breaks and refueling.

#### Combine sub-problem solutions to solve overall problem:

**Detailed Driving Route:**

1. **Starting Point: Salt Lake City, Utah**
   - Get on I-15 N from W 600 S.
   - Continue on I-15 N.

2. **I-15 N to US-93 N:**
   - Continue on I-15 N passing through Idaho Falls, ID.
   - Take exit 278 to merge onto US-287 N toward Ennis/Helena.
   - Continue on US-287 N.
   - Merge onto I-90 W toward Missoula.
   - Take exit 332 for US-93 N toward Kalispell/Whitefish.

3. **US-93 N to Glacier National Park:**
   - Continue on US-93 N to reach Glacier National Park.

4. **Key Stops for Navigation and Rest:**
   - Idaho Falls, ID
   - Helena, MT
   - Missoula, MT
   - Kalispell, MT

### Final Answer:
The driving route from Salt Lake City, Utah to Glacier National Park in Montana is as follows:

1. From Salt Lake City, get on I-15 N.
2. Continue on I-15 N, passing through Idaho Falls.
3. Take exit 278 to merge onto US-287 N toward Ennis/Helena.
4. Merge onto I-90 W toward Missoula.
5. Take exit 332 for US-93 N toward Kalispell/Whitefish.
6. Follow US-93 N to Glacier National Park.

Key stops for navigation and rest include Idaho Falls, Helena, Missoula, and Kalispell.

Advanced prompt engineering benefits

Advanced prompt engineering offers tremendous benefits, especially when used with AI assistants.

• Increased user satisfaction: Tailored responses from the AI assistant meet user needs more effectively by providing relevant and accurate information. This ensures more pleasant and efficient interaction, enhancing the overall user experience.
• Greater adaptability: By adapting to diverse user inputs and contexts, AI assistants can handle various scenarios and requests, even if the original training dataset didn’t have similar inputs. This flexibility allows for more dynamic and responsive interactions.
• More intuitive interactions: Creating natural and human-like conversations helps users feel more comfortable and understood. This fosters a sense of ease and familiarity, encouraging more frequent and meaningful engagements with the AI assistant.

Prompt development and the cycle of continuous improvement

When building an AI assistant, you might abstract the process of crafting prompts, handling all prompt engineering concerns on behalf of the end user. Continuously evaluating and refining prompts will help you improve in meeting the current needs and expectations of the user, especially in these scenarios.
 
Even if the assistant is perfectly aligned with user expectations and can accomplish all current tasks successfully, this doesn’t necessarily mean that the current prompts will remain perfect. Users may change their behavior or preferences. The user may ask the assistant to perform new tasks for which the current prompts are unsuitable.
 
In addition, AI assistants will likely be upgraded to work with newer generations of the underlying LLMs. Changes to the model will usually require a calibration phase for the prompts.
 
This continuous improvement cycle consists of feedback loops where the AI assistant (or the developer) evaluates the user’s reaction to various responses and adjusts prompts incrementally to improve satisfaction.

Leverage state-of-the-art technology and techniques

GenAI has become an essential technology for enterprises, but simple Q&A chat experiences are insufficient for performing complex, high-value tasks. However, AI assistants guided by sophisticated prompts can be tailored to handle specific tasks with precision and efficiency. Prompt engineering is the crafting of effective prompts to guide AI models in providing useful results.
 
Advanced prompt engineering techniques, such as few-shot prompting, Chain of Thought (CoT) prompting, and contextual priming enhance the AI’s capability to deliver high-quality responses. These techniques lead to increased user satisfaction, greater adaptability, and more intuitive interactions, ultimately driving the continuous improvement of AI assistant performance.
 
If your enterprise is taking steps toward GenAI innovation, whether for internal operations or customer-facing applications, understanding the transformative potential of AI assistants will be a key step in your journey.
 
Learn more about how AI personalization improves customer experience and how to assess your enterprise’s AI readiness.