Over the last 6+ months I have been building with agents, and the way I use AI today is drastically different than when I first started using AI when ChatGPT came out and then eventually trying out agentic development At first, I felt I had to try to use an LLM whenever I could...forcing myself to use an AI even though I knew traditional approaches worked just fine.
I had to figure out what the AI hype was all about, so I started using AI as the solution to every problem. I forced myself to put on the AI glasses...It reminded me of the phrase: to the person who has a hammer everything looks like a nail. I would try to use AI every way I could.
It reminded me of when blockchain came out and everyone felt the need to throw blockchain at every single problem even if it made no sense.
The problem with AI and how to get Deterministic Results
The first and biggest problem - The output of an LLM is non-deterministic. This is a fancy way of saying that the same exact input can give a different output. This is a feature of LLMs, but using an LLM as a deterministic tool is the actual problem.
I started simple - Have an AI do code reviews of PRs (easy to use copilot in GitHub to achieve this).
I then continued with this asking AI to do things like test my code, scan it for security issues, etc.
This was a great learning experience, but is not the best way to get quality results.
The fix: Use AI to build deterministic results
We can instead have AI build out deterministic checks using traditional tools. The AI helps you design it once and then it runs the same way every time.
Try prompting your AI:
"Set up a pre-commit hook that blocks commits if TypeScript compilation fails, ESLint finds errors, or Prettier formatting is off."
"Write a GitHub Actions workflow that runs my test suite, type checks, and npm audit on every pull request — and fails the build if any of them fail."
"Add a CI step that scans for hardcoded secrets and API keys using gitleaks.
"Configure ESLint with a rule that bans the any type and fails on unused imports."
etc...
Or, as a brain storm:
"What are some issues you discover in our code base that could be converted to deterministic gates, preventing that issue from ever happening in the future?"
The idea here is we now have the ability to build out more extensive quality checks with ease. Don't rely on the LLM's intuition to judge the code, but instead have it help you build quality pipelines.
Even things that cannot be converted directly to linting rules can be less unpredictable by having the AI agent reference a check list and give it an example output structure. In prompt engineering this is called "few-shot prompting" and I like to tie it with an extra level of review. for example:
"Yo Claude, please write the mentioned tests using subagents, then have an independent subagent validate it according to our testing.md standards"
Please respond to this email and let me know if you find tips like these helpful and where you currently are in your agentic development journey.
Python Mastery and AI Features
In the new Python Mastery we build out an AI pipeline for user-facing chat bots. We do it all from hand learning how to utilize models and enhance them with our own data through RAG and vector databases. If you're new to the AI feature world, this is a great place to start.
You can use these skills to build your own AI SaaS, enhance your business, or thrive at your software development role. These skills are also increasingly important for software interviews.
You'll have a hard time finding modern applications that do not have some AI features: RAG, MCP, AI Chat bots, summarizations, generative tools, AI automations, etc. Failing to learn these things is just making your skills irrelevant.
In these sections we learn how to run models locally and talk about:
standardizations so you can easily swap to a platform API in production
Running local models in with ollama and the different model types
Connecting to models from Python and making prompts
parsing and chunking .md documents and storing these in a vector db
reading relevant data from the database for augmented prompting (RAG)
Having AI models support users and learning through relevant sourcing from docs
Building a simple chat bot that says where all the information comes from in the original markdown files.
Track and display token usage both up and down
We do most of this manually, learning the fundamental systems instead of just plugging in some AI chatbot library that hides everything from us. This will give you the fundamental skill to better understand libraries like LangChain or LlamaIndex or ANY other library (again, focusing on the core skills).
Learn more in Python Mastery. There are still some coupons available for 30% off, use code PYTHON at checkout.