[VIP] Llamacpp Document exploration - The local GPT engine room #11

You may wonder why the main image for this post is about prompt engineering…

RAG, and generally speaking all instructions to Language Models, inherit their accuracy and coherence from the prompt.

So a crappy prompt, means a crappy result!

Before we proceed to a well structured LocalGPT Document processing application, today I’ll give you a simple, three-part framework you can use to get dramatically better results from any AI.

I call it the "CPR" Framework: Context, Persona, Request.

1. Context: Give the AI the background information it needs. Don't assume it knows anything.

   • Bad Prompt: "Write about AI."

   • Good Prompt: "I am writing a blog post for beginners who have just installed their first local AI model."

2. Persona: Tell the AI who it should be. This dramatically changes the tone and style of the response.

   • Bad Prompt: "...write about AI."

   • Good Prompt: "...Act as an encouraging and expert guide who wants to make complex topics easy to understand."

3. Request: Be crystal clear and specific about what you want it to do.

   • Bad Prompt: "...write about AI."

   • Good Prompt: "...Write a 200-word introduction explaining the difference between a local AI model and a powerful, cloud-based AI API like the ones from Google or OpenAI."

Putting it all together: "I am writing a blog post for beginners who have just installed their first local AI model. Act as an encouraging and expert guide who wants to make complex topics easy to understand. Write a 200-word introduction explaining the difference between a local AI model and a powerful, cloud-based AI API like the ones from Google or OpenAI."

Try it yourself with any AI you have access to. See the difference? This is just one of the foundational techniques to make your LocalAI truly useful and accurate!

Previously, on LocalGPT...

Last week we used RAG without any chunk strategy or embeddings. We exploited the LLM built-in capability to understand the context in the prompt, passing the entire document as the first interaction with the Chat AI.

Specifically we:

• open a pdf

• transform it into text

• pass it in the chat history

• use the conversation history as short memory context

So, what is the next natural step?

It should be to introduce embeddings, chunks and similarity search (or similar techniques like a re-ranker…).

But today I want to show you how to properly set up (with no complicated object oriented programming schemes) a python project that you can re-use in the future.

Divide and Conquer

Last week, if you followed the steps, you completed a fully functional python application. But… not so easy to be read, because there are 180+ lines of code, and the real logic behind the application is only at the end.

So, here is the tip: let’s segregate the functions (and if you have the classes) from the main project file.

Maybe for you this is nothing new… but maybe not right?

We will use the same setup of last week, so no more installations, downloads and dependencies.

Divide

Let’s create a new file: I called mine RAGLIB.py (again, not so creative…). In this file we will move all the functions and main calls.

Have a look here.. and don’t get scared: I will explain everything!