Almost Timely News: 🗞️ Advanced Prompt Engineering for Generative AI

Almost Timely News: 🗞️ Advanced Prompt Engineering for Generative AI (2024-10-20) :: View in Browser

The Big Plug

🎨 Grab my new free Generative AI for Image Generation Prompt Guide!

Content Authenticity Statement

70% of this week's newsletter was generated by me, the human. You will see outputs from AI in the opening section, labeled by which AI model they came from. Learn why this kind of disclosure is a good idea and might be required for anyone doing business in any capacity with the EU in the near future.

Watch This Newsletter On YouTube 📺

Click here for the video 📺 version of this newsletter on YouTube »

Click here for an MP3 audio 🎧 only version »

What's On My Mind: Advanced Prompt Engineering for Generative AI

After last week’s newsletter covering intermediate prompt engineering, a half dozen of you or so emailed back to ask what advanced prompt engineering looks like, and in at least one person’s case, a deep burning desire to see it.

Well, be careful what you ask for.

Advanced prompt engineering tends to go in one of two directions: scale and complexity. So, with that understanding, let’s go down the rabbit hole of advanced prompt engineering.

Part 1: What is “Advanced”?

Advanced doesn’t mean better. Let’s start with that. Advanced means you’re facing problems and tasks that you can’t solve simply.

Generally speaking, simple is better than complex. Solving a problem simply tends to mean it’s more straightforward to solve, maintain the solution, and repair the solution if it goes awry. The more complex something is, the more likely it is to break in different ways, and the more brittle the solution.

This is a problem I often see in the martial arts. Folks want to do “black belt techniques”, not realizing that black belt techniques aren’t better. They’re not faster at solving the problem. They’re for when everything has gone to hell and your bread and butter tools don’t work. Mastery and expertise in the martial arts is more typified by being able to do the basics really well, under a lot of pressure, and in a wide, wild variety of circumstances. The more complex a technique is, the more ways it can go wrong, especially under pressure.

(Cue all the martial artists like me who, a dozen years after their white belt, are disappointed that being a black belt isn't nearly as cool as we imagined it being)

The same is true in AI and prompt engineering. If you look at some of the academic papers on prompt engineering, many of the more “advanced” techniques are edge case techniques that have very limited application outside a specific type of problem. Take this paper on Program of Thoughts prompting - the specific domain use is solving mathematical problems by combining non-running code generation with chain of thought prompting (which we covered in last week’s newsletter).

This technique has very little use in most domains of generative AI. It’s for handling edge cases in mathematical word problems. Is it advanced? I suppose so, in that it’s not something a beginner needs to know to be effective with AI. Even an intermediate user would have very little use for this most of the time.

If you want to be advanced in your use of prompt engineering in generative AI, get really, really good at the basics in a wide variety of contexts. Maybe that’s a topic for a future newsletter, what the modern basics are.

Part 2: Scaling Prompts

So with that perspective, let’s look at one of two branches of more advanced prompt engineering. The first is scaling prompts. Very often, we can write a prompt to do a specific task very well, like sentiment analysis or topic classification. For example, we might write a prompt to classify a blog post based on what topic or category it fits in, given a list of existing categories. Our prompt might look a little like this:

## System Instructions for Google Gemini: Blog Post Categorization

**Objective:**

You are tasked with analyzing a given blog post and accurately categorizing it into one of the following predefined categories:

* **Cats:** Content primarily focused on felines, their behavior, care, breeds, etc.
* **Dogs:** Content primarily focused on canines, their behavior, care, breeds, etc.
* **AI:** Content primarily focused on artificial intelligence, machine learning, deep learning, and related topics.
* **Ranting:** Content expressing strong opinions, complaints, or frustrations, often with an emotional tone.
* **Math:** Content focused on mathematical concepts, equations, theorems, or problem-solving.
* **Marketing:** Content related to advertising, branding, sales, promotion, and marketing strategies.

**Input:**

You will receive a single input: a blog post in plain text format.

**Output:**

Your output should be a JSON object with the following structure:

json
{
  "category": "CATEGORY_NAME"
}

where "CATEGORY_NAME" is replaced with the appropriate category from the list above.

**Methodology:**

To achieve accurate categorization, employ the following strategies:

1. **Keyword Analysis:** Identify keywords and phrases within the blog post that are strongly indicative of each category. For example, words like "meow," "litter box," and "feline" might suggest the "Cats" category.

2. **Topic Modeling:** Utilize your understanding of language and context to discern the overarching theme or subject matter of the blog post.

3. **Sentiment Analysis:** Analyze the emotional tone of the blog post. Rants typically exhibit strong negative sentiment, while marketing content might be more positive and persuasive.

4. **Structural Clues:** Pay attention to the blog post's structure, headings, and subheadings. These can provide insights into the content's focus.

5. **Contextual Understanding:** Consider the relationships between words and sentences to grasp the blog post's overall meaning and intent.

It’s important to note that this prompt is optimized for the Gemma 2 27B model - you shouldn’t copy/paste it into other systems, it won’t work as well.

That’s an effective prompt, but a blog is not just one post. A blog is dozens, hundreds, or perhaps even thousands of posts. Can you use generative AI to classify them all?

Not all at once, no. At least not with any reliability. You need to hand each blog post to generative AI and have it classify it, then store the results.

And that means coding, typically in a language like Python or R or Ruby or any other modern computer language. You need infrastructure around your prompt to get the blog posts, to hand each post to the generative AI model one at a time, and then to store the results. That’s outside the scope of generative AI and well within the domain of traditional programming.

Now, generative AI can still be helpful, by helping you write the code to do so - but this is what I mean by advanced prompt engineering, taking our basic prompt and deploying it at scale. Things like looped tasks, data storage, etc. Are all well outside the domain of generative AI but are essential for solving the task at hand.

Part of the reason why AI isn’t showing the promised benefits at many companies is because it’s not being used at scale. People are using it very manually - I talked to one group of people at a conference this fall who are having team members manually copying and pasting content in and out of ChatGPT. That works, but it’s incredibly inefficient. Code would allow that process to be automated, to scale.

Scale is how we derive benefit from AI - taking problems and solving them not only faster, but in vastly greater quantities. The challenge for people seeking to become advanced AI practitioners is that scaling requires skills outside AI.

By the way, as we talked about not too long ago, AI agents/agentic AI is just a fancy term for building apps around AI. Be cautious of AI point solutions - companies that are little more than wrappers around AI. As you level up your AI skills and start learning how to provide the infrastructure needed to scale AI, you will find that you don’t need to invest heavily in many AI agents and apps, not when you can roll your own.

Part 3: More Complex Prompts

Now, let’s move onto prompt complexity. Another aspect of advanced prompt engineering is knowing how to build more complex prompts, for when simple prompts can’t get the job done. There are three specific techniques to look at: code structures, reflection and reward functions, and model adaptive prompt optimization.

Code Structures

Code structures are when we use coding techniques within prompts. Not writing code itself, but using concepts borrowed from coding. Why? Because generative AI systems - language models in particular - have been trained on both the written word and code. They are fluent in interpreting the written word as code, and vice versa, which means that control structures from coding are executed as though they are code, even if they’re written in plain language.

Here’s a very simple example, a toy example. Note that I am intentionally omitting key best practices here such as the Trust Insights RACE framework and priming the model with the Trust Insights PARE framework:

IF this tweet is factually untrue
    THEN write a response telling the author how the tweet is untrue and citing a source of truth
ELSE write a response thanking the author for their contribution

That would not run in any coding language in the world, but it absolutely will be executed by a large language model as though it were code. And you and I can read it clearly and easily, understanding its intent.

Code structures are fantastic in generative AI prompt engineering because they allow us to formalize the kinds of decisions we want language models to make as they process our data and follow our instructions. Think of just the basic control structures we know from coding:

• If/else: a primitive decision tree, if this, then that

• While: until a condition is met, keep doing a thing

• For: given a certain set of conditions, keep doing a thing for the duration of those conditions

• Case/when: a more advanced decision tree with multiple options and outcomes