The use of artificial intelligence in tradecraft is a hot topic today and will remain so for the foreseeable future. It is already used widely in searching, structuring and narrowing down available data. The industry is competently navigating those capabilities. But there are two more levels of capability beyond that that are just emerging.
Primer and Carahsoft recently teamed up to present a webinar, “The 5 Levels of AI Tradecraft.” Led by Primer’s VP of Data Science, John Bohannon, the session covered the levels of AI tradecraft, along with graphics, technological explanations and more details to fuel the discussion. We’ve boiled that in-depth presentation down to an overview of the five levels here. The webinar contained many more valuable nuggets of information, so you may want to watch the full presentation once you’re done reading.
Level 1—Search. When an intelligence analyst uses traditional Boolean search, they are only going to get responses that contain the exact keywords they put into the search. If you want to know about Ukraine diplomatic visits, you can search for the keywords, “Ukraine,” “diplomatic,” and “visits.” But you’re only going to find documents that specifically mention those exact words. This eliminates many documents that may be relevant, but don’t contain the exact keyword.
AI enables the analyst to do a semantic search of what they really want to know. This might be, “what leaders have recently had diplomatic visits with Ukraine?” And, instead of a keyword-dependent list of returns, you’ll get returns that are in the same semantic family as the question, whether they contain those keywords or not. In addition, your returns will be ranked in terms of relevance to your query. It’s a life-changing development for analysts because they don’t have to struggle to find the right combination of keywords to search and they can cast a wider net. They just have to express what they want to know in a plain, simple sentence.
Level 2—Structure. Structure is created when we ingest documents we’ve been searching for and save them in a database that can be queried. Instead of annotating or tagging all those documents yourself, however, AI can be used for that purpose, simplifying the analyst’s job and enabling more in-depth, thorough searching of those documents. So your initial search returned documents about diplomatic visits to Ukraine, but now you can dive further and see if Zelenskyy attended or if the visitors were from NATO countries. You can use sentiment analysis to see how the visitors felt about Ukraine. So, search is just the beginning of a query. Structure adds dimension and keeps you from “boiling the whole ocean” just to find the information you need.
Level 3—RAG. Retrieval augmented generation (RAG) adds another layer of sophistication to your search for information. When you run your semantic search, you can just get the relevant pages of a document returned to you. That narrows things down. But you may still be left with tens of thousands of pages to put your eyeballs on. RAG allows you to crunch all those documents by putting them into a prompt, running it through a large language model, and extracting the most relevant information.
These results come with citations so you can double-check to make sure there are no “hallucinations”—errors or misunderstandings between the data and the AI. Analysts have a lot of data they have to put eyeballs on over the course of the day. While they could easily scan the top 25 results of their query the human way, letting the RAG do that for them significantly enhances the scope and depth of their work.
Level 4—Chat. So, your semantic search has returned results to the query, “which leaders have had diplomatic visits to Ukraine?” But then you want to drill down further, such as “how many were from NATO nations?” That will require an entirely new search with all the relevant keywords to clarify we are talking about Ukraine and leaders and diplomatic visits. With search and structure and RAG alone, you can’t have a conversation with the data. A new search is a new search.
But chat, as with ChatGPT, enables you to ask a question, then a follow up question, then another, all the while drilling deeper down into your data without creating a new search. It searches the searching of the searching, whereas other methods either require new queries or don’t progressively drill down into the same body of data. By making these queries more user friendly for the analyst, you’re not only helping them gain more relevant information, you’re lowering the cost of analysis by reducing human effort.
Level 5—Agents. This is an area that is still in its infancy in AI tradecraft. Every search you launch, every summary you generate, and every timeline you create is a separate action. A separate task, if you will. And agents enable an analyst to rapidly complete many tasks at once. You can generate a timeline of those diplomatic visits, explore the implications for these meetings and create an executive briefing on the topic all at once. These are projects that may have previously been farmed out to multiple analysts to complete. But the agent generates drafts of all those documents in one fell swoop. This capability is developing quickly so the full breath of this technology is not yet apparent.
As you can see, each level of AI tradecraft builds on the one before, unlocking new efficiencies and deeper insights at every stage. While search and structure are already transforming how analysts process information, RAG, chat, and agents are pushing the boundaries of what’s possible. To explore these levels in more depth and see AI tradecraft in action, watch the full webinar.
The use of AI is increasing rapidly in tradecraft. Primer can show you how to gain better information, create superior products, and practically eliminate hallucinations. Contact us for more information and advisement on the rapidly progressing world of AI tradecraft.
