As with most developers in the current era, I find myself constantly being asked to leverage Artificial Intelligence when programming. While the intentions are (hopefully) good, I think AI is clearly over hyped. AI developers are barely more than an executive pipe dream, but I do think working with AI tools presents a unique opportunity to learn a new skill. If we remove the bombastic messaging, AI tooling becomes just another puzzle to be solved. Similar to learning how to use a new language, library, framework, or build system, using AI tooling requires keeping an open mind while attempting to understand how best to extract the benefits while minimizing the pitfalls. In the same vein, the best engineers continues to be the ones who are most able to use the correct tool for the job. With the introduction of AI tooling, it becomes essential to know when and how to use AI for any given task.
To start off, I want to note that I still spend the majority of my time writing code by hand. While this might end up being only a small portion of the code that gets submitted, it still feels like the main part of my job is writing code manually. In addition, I generally force myself to write code by hand when working on personal projects to ensure that I continue to grow and improve my ability to identify good, maintainable code.
With that out of the way, I think there are 2 key skills required to extract the best results from AI tooling:
- Understand the AI tooling pyramid. No AI tool is “one size fits all” and different stages of development work best with different AI tools.
- Switch quickly. When the AI starts to bottom out or get lost, switch to a different tool or step in and make manual changes quickly.
The AI Pyramid
The diagram above showcases how I’ve come to think about AI tooling. The size of each step in the pyramid represents how much code I expect to generate with that tool. The actual tools used are largely interchangeable, but they fall into the same 3 broad categories:
- Roo is an agentic AI designed for working on multi step tasks across many files.
- Copilot can be replaced with essentially any AI tab complete style tool.
- Gemini is just one example of a chat bot style AI. In most cases they run in browsers, but some offerings also run in editors.
Let’s start at the base of the pyramid. Roo or other agent based AI tools are ideal for generating large amounts of code. I often set Roo to a task while attending a meeting, and allow it to churn out a whole CRs worth of code which I will come back and review after the fact. With this strategy, Roo is usually writing entire files, classes, and test cases in one execution without significant oversight. The ambiguity of most tasks can lead to Roo getting stuck down incorrect paths or winding up unable to correct mistakes once they become clear. To address this, I use a technique known as prompt chaining. This approach uses a simple prompt to generate a lager more complete prompt. In the case of Roo, I start with a prompt like “Write a design to DESIGN.md which outlines the changes required to do…” followed by a prompt like “execute the design outlined in DESIGN.md”. This helps to focus the direction Roo will take when implementing the change required while also providing a means to check the changes before they are made. In some cases, I will delete whole sections from DESIGN.md when it’s clear they won’t end up working. This may sound similar to the way more junior engineers are asked to approach projects which I suspect is not a coincidence. The possibility for error seems to decrease drastically when agentic AI tools are asked to first explain their thinking.
The next step of the pyramid contains CoPilot and Gemini. Both tools generate a similar amount of code, but for different purposes. CoPilot has an extremely limited context scope with which to provide suggested changes, usually only the currently open file or files. This makes it effective primarily for completing simple methods or lines. With CoPilot enabled, I often find myself writing a comment in the middle of a method like “extract distinct values from the map” and then wait for CoPilot to write the next line. This is great for offloading the creation of helper methods and simple logic when I already know what the code should look like, but I don’t want to write it myself.
Gemini fills a similar but distinct role in my AI toolkit. I use Gemini through the web chat interface so it has no context on my actual project or code files. Never the less, I find it useful for generating blocks of code that I might not know the specifics of. Gemini is great for “researching” and generating segments of Gradle or Maven build files. Another use case for Gemini is unique but simple code conversions. For example, I had a JSON string which showed the shape of a request object from another team. I wanted convert this JSON object into a Java class which could deserialize it. This is dead simple to do, but no tool that I know of could run this conversion quickly. Gemini was able to produce mostly accurate code and guess at the data types required for each field. This ability to read between the lines is an impressive feature of LLMs and can make mindless but non-automatable tasks significantly easier.
Last but not least is hand written code. While this is the smallest chunk of the pyramid, I place it at the top purposely as I still see it as the most important skill for a developer to have. When AI tools fail, writing code by hand always succeeds. Keeping the code writing skills sharp is still as important as ever and I firmly believe that in any AI future, the best engineers will always retain incredible manual coding skills.
Switch Quickly
Maintaining the productivity of a 10x engineer while leveraging AI tools requires honing a number of new and unique skills. As described above, a big part of this is knowing which AI tool will fit best in which situation. However, a more important skill is knowing when to change tools or techniques if things go wrong. The goal of leveraging AI is not to get it to write all the code in your project, but use it to produce accurate, high quality code quickly. If a certain technique isn’t producing the desired results, take whatever elements are salvageable, and move up the pyramid. Similarly, as soon as the amount of code needing to be written increases and complexity decreases, step down the pyramid and allow AI to write a larger portion of the code. At times, I even find myself swapping back and forth between manual changes and Roo generated code. Asking Roo to repeat partially completed changes can allow for fine tuned segments to be incorporated into broader strokes.
As with all things, the definition of “quickly” varies based on the situation. An important consideration to make when working with AI tools is the training set that was used to build them. AI works best when generating code that has already been written. This means with a little coaxing, Roo is far more likely to succeed at making changes to a Java Spring application than a custom built Haskell framework. If you are still new to using AI tools, try them out in a setting they are likely to succeed at. GitHub is full of React App websites and Java Apache projects. Because of this, AI is likely to excel when working with these languages and frameworks.
Conclusion
AI is like any other tool. Refusing to embrace it can be just as damaging as falling into the over hype. The goal of any professional software engineer should be to produce efficient, maintainable code as quickly as possible. I have been using AI in my development workflow for the past 2 years, and in that time I have come to the conclusion that the best engineers of the future will all be leveraging AI in multiple facets simultaneously. If we assume that this is correct, it becomes essential to learn how to extract as many of the benefits of AI as possible while side stepping the pitfalls. Shifting the image of AI from a monolithic one-stop-shop to a set of similar yet different tools helped me understand how to extract the greatest value and quickly shift focus when issue arise.