Beyond Keywords: AI Classification For Forensic Email Review

When Keyword Searches Stop Working Picture a fraud investigation. The suspect is a finance director suspected of routing payments to shell companies owned by a family member. Your collection contains 60,000 emails spanning three years. The company operates across three countries, so a meaningful portion of the correspondence is in German and French. You build a keyword list: the names of the suspect entities, known account numbers, “invoice,” “wire transfer,” “please approve.” You run it. You get 4,200 hits. Half are genuine invoices with no connection to the scheme. A third reference the suspect companies in entirely routine contexts. The genuinely suspicious emails—the ones where payments are discussed obliquely, in shorthand the participants developed over years of collaboration—barely surface at all. This is not a failure of effort or skill. It is a structural limitation of keyword-based filtering. Keywords match text; they do not understand it. A finance director arranging a fraudulent transfer does not write “I am now arranging a fraudulent transfer.” The meaningful emails are the ones where context, tone, and relationships between statements carry the significance, and those properties are invisible to a search query. Research confirms the gap: in a foundational study (Blair & Maron, 1985), experienced legal professionals reported reaching recall of at least 75% while averaging just 20%—a finding replicated across four decades of subsequent evaluations. “Experienced legal professionals believed they had reached recall of at least 75%—while averaging just 20%.” The volume problem compounds the structural one. Email collections have grown relentlessly. A mid-size organization with long retention periods can produce hundreds of thousands of messages from a single custodian. Manual review of that volume, even after keyword reduction, is not a realistic option within typical investigation timelines. Why TAR Doesn’t Solve the Problem Technology-Assisted Review (TAR) platforms, developed primarily for large-scale civil litigation, were designed to address exactly this kind of volume challenge. They have a proven track record in the right context. But forensic investigations are not large-scale civil litigation, and TAR’s design assumptions reflect that origin. Get The Latest DFIR News Join the Forensic Focus newsletter for the best DFIR articles in your inbox every month. Unsubscribe any time. We respect your privacy - read our privacy policy. TAR systems learn from examples. Before automated processing begins, a subject matter expert manually reviews and classifies a seed set of documents—typically 200 to 2,000—so the system can learn what “responsive” looks like. Each new matter requires its own training cycle, creating a cold-start problem that can delay meaningful output by days or weeks. The more complex the case theory—multi-category forensic triage, evolving classification criteria, a mixed-language corpus—the more training the system demands before it can be trusted. For investigations with tight timelines, specific evidentiary requirements, or multilingual datasets, the TAR model creates friction at exactly the points where speed and flexibility are most needed. It also incurs specialist overhead: effective TAR deployment requires statistical validation expertise and continuous quality monitoring, often involving external consultants. LLM Classification: A Different Approach Aid4Mail Investigator and Enterprise take a different approach, one built on Large Language Models (LLMs) rather than statistical pattern matching—and the difference shows in what you can do from the moment processing starts. An LLM does not learn to classify emails by studying examples. It classifies them by evaluating their semantic content. When you instruct Aid4Mail to flag emails exhibiting signs of bribery or corrupt financial arrangements, it applies semantic language comprehension: it considers context, infers intent from phrasing, and handles linguistic variation—including multilingual correspondence—without any additional configuration. There is no training phase and no seed set. You write a natural language prompt, define your classification categories, and begin processing immediately. Aid4Mail’s AI classification setup panel, showing model selection, prompt field, and classification category configuration Choosing a Deployment Path Aid4Mail supports three distinct deployment modes, each suited to different operational and compliance requirements. All three provide the same classification workflows, prompt library, and output formats—the choice is determined by your data handling obligations and infrastructure. Consumer APIs.  Direct access to commercial providers—Anthropic, OpenAI, Google AI Studio, xAI, and others—is the fastest path to getting started. Create an account with the provider, enter your credentials in Aid4Mail, and processing begins immediately. Standard rate limits and shared infrastructure apply. This path suits practitioners without data residency requirements who want to evaluate models quickly or handle small processing jobs. Per-token charges are billed directly by the provider. Enterprise APIs.  Amazon Bedrock, Google Vertex AI, and Microsoft Azure AI Foundry provide access to the same frontier models through infrastructure designed for organizational use. All three offer regional deployment options—including EU regions—enterprise security controls, and managed throughput quotas that consumer APIs do not. For organizations with data residency requirements, contractual data handling obligations, or sustained high-volume workloads, this is the recommended path. Geographic data containment is enforceable: processed content does not leave the selected region. Offline (local) models.  With Ollama or LM Studio—both free, locally installable inference servers—classification runs entirely within your organization’s infrastructure with no internet connection required. Data never leaves the local environment. This is the only viable path for classified government investigations, organizations with blanket cloud prohibitions, or any context where transmitting evidence to an external API is not permissible. The recommended model for most deployments is Mistral Small 3.2 24B; the recommended hardware for optimal throughput is an NVIDIA RTX 4090 (24 GB VRAM). For organizations with less demanding hardware, Ministral 3 14B offers the best speed-to-accuracy ratio among offline models, with a lighter footprint (10 GB or more of VRAM or 20 GB or more of system RAM). There is a real accuracy trade-off between offline and cloud deployment—one that practitioners should factor into their workflow planning. The benchmark results in the next section provide the data to make that comparison. Benchmark Results: Accuracy, Speed, and Cost We tested this approach rigorously. The evaluation covered 18 AI models—12 cloud-based and six locally deployable—across four tests spanning 35,267 emails. Three structured tests with verified ground truth assessed forensic classification across nine categories (including illicit financial activity, bribery, threats, harassment, and online extremist recruitment), FOIA-style binary classification, and real-world business email triage. A fourth test processed 34,097 emails from the publicly available John Podesta corpus at production scale, validating throughput estimates and providing operational data on cost and model behavior. (Pre-filtering in Aid4Mail reduced the original 50,887-email archive by removing duplicates, bulk mail, and notifications.) The accuracy results were consistent with what LLMs have demonstrated in other text comprehension tasks. Claude Opus 4.5, tested via Amazon Bedrock (also available on Google Vertex AI), achieved a weighted composite score of 97.9% across the three structured tests. Grok 4.1 Fast+Reasoning reached 97.6%—at $0.20 per million input tokens, 25 times less expensive to run than Claude Opus 4.5. Gemini 2.5 Flash, the fastest cloud model in the evaluation, processed the business mailbox test at over 8,200 tokens per second—a throughput that projects to over 400,000 emails in a 62-hour unattended weekend run. “Gemini 2.5 Flash projects to over 400,000 emails in a 62-hour unattended weekend run.” To put these figures in context: peer-reviewed benchmarks place even the best automated alternative—CAL-based TAR 2.0—at 75%–96% recall and 80%–96% precision on large email collections, requiring a full training cycle before processing begins. In Aid4Mail’s binary FOIA-style classification test—structured as a direct responsive/non-responsive triage decision—top cloud models achieved 96%–100% recall and 91%–98% precision with no training phase. The recommended offline model, Mistral Small 3.2 24B, matched the best cloud results in this test: 98% recall, 96% precision. One methodological caveat applies to these figures: precision in binary classification is sensitive to corpus prevalence. Test 2 used a 50% responsive rate, which is substantially higher than typical investigative corpora, where responsive prevalence is often below 10%. Precision will vary with dataset composition, and practitioners should consult the Test Report for full methodology before extrapolating to their own use case. The large-scale test provided the clearest cost validation: processing 34,097 emails using Gemini 2.5 Flash via Google Vertex AI cost under $18 excluding attachment text, or under $23 when extracted document content was included. At that cost, running a full AI classification pass before committing to manual review is simply good practice. For offline deployment, the accuracy trade-off is real but manageable. Mistral Small 3.2 24B achieved a weighted composite score of 93.1% in the evaluation—meaningfully below the top cloud models but within the range practitioners typically cite as acceptable for first-pass triage. On mailboxes with heavy attachment content, it runs at approximately 0.78 emails per second at the recommended 64K context setting, projecting to around 175,000 emails over a 62-hour weekend window. On lighter corpora, such as the Podesta email archive, throughput rises to approximately 1.85 emails per second, projecting to around 410,000 emails over the same window. “In FOIA-style binary classification, Mistral Small 3.2 24B matched the best cloud results: 98% recall, 96% precision.” Multilingual capability requires no special configuration. The structured test included emails in English, German, and Korean, with an additional real-world mailbox containing French and German correspondence. Top-ranked cloud models showed no meaningful accuracy degradation across languages—German and Korean results were comparable to English in the forensic classification test. Addressing AI Concerns: Hallucination, Defensibility, and Structured Output AI hallucination is a legitimate and well-documented concern in legal and investigative contexts. As of early 2026, over 1,000 cases involving AI-generated fabrications have been documented in legal proceedings globally—a figure tracked in an ongoing database maintained by legal scholar Damien Charlotin. The consequences in investigative work are more acute than in general legal drafting: a fabricated date, name, or event that makes its way into a report or affidavit introduces claims with no evidentiary source. The key distinction is between generative AI—which produces free-form text and can fabricate fluent but false content—and classification AI, which assigns a label from a constrained set you define. Aid4Mail uses the latter. When classifying emails into categories such as Illicit Financial Activity, Bribery, or Unresponsive, the model does not generate facts. It reads the email and selects from the options you have specified. There is no mechanism by which it can invent a date, fabricate a name, or assert something not present in the source material. That structural difference is what makes AI classification defensible in high-stakes review. A useful framework for evaluating AI use in legal and investigative contexts is a risk-tiered approach: generative tasks that produce factual assertions carry high risk and demand rigorous verification; classification tasks that assign categories from a defined set carry substantially lower risk, because verification is sampling-based rather than document-by-document. Aid4Mail’s classification architecture sits firmly in the lower-risk category. A further safeguard is the explicit INCONCLUSIVE category available in every classification prompt. Rather than forcing a decision when evidence is ambiguous, Aid4Mail can be configured to flag uncertain emails for human review. Practical Starting Points Aid4Mail ships with a library of over 200 pre-written prompts organized by investigative context. The Digital Forensics category covers 34 specialized themes, including cybercrime, financial fraud, child exploitation, crypto fraud, human trafficking, cybersecurity threats, and state-sponsored espionage. The eDiscovery category provides 24 litigation-focused prompts covering antitrust, IP theft, harassment, insider threats, M&A due diligence, whistleblower investigations, and regulatory compliance. The FOIA and Public Records category adds 14 government-specific themes. Each prompt is a starting point rather than a finished configuration. We recommend reviewing any pre-written prompt against your specific case theory, refining it for your dataset, and testing on a representative sample before committing to large-scale processing. The evaluation data confirmed that prompt quality has a direct and measurable effect on classification accuracy—investing time in prompt refinement yields compounding returns across the processing run. Aid4Mail’s prompt library, showing the Digital Forensics classification categories Start Small: A Full Classification Pass for Less Than $23 The economics of AI-assisted email review have reached a point where the question is no longer whether organizations can afford it. For a dataset the size of the Podesta corpus—34,097 emails, a scale representative of many corporate investigations—a complete AI classification pass costs under $23 using Gemini 2.5 Flash. Replacing that with keyword iteration, followed by manual spot-checking of borderline hits, costs orders of magnitude more in examiner time. “A complete AI classification pass on 34,097 emails—including attachment content—costs under $23.” Aid4Mail’s AI integration is available in the Investigator and Enterprise editions. A fully functional free trial is available at aid4mail.com/free-trial. For complete technical details on the AI integration and the full methodology behind the benchmark figures cited in this article, the following resources are available directly from Fookes Software: Aid4Mail AI Integration User Guide AI Classification Test Report About Aid4Mail™ Aid4Mail is specialized email forensics and eDiscovery software developed by Fookes Software Ltd, a Swiss software company founded in 1996. In continuous development since 2005, it handles the complete email evidence workflow—collection from cloud and local sources, recovery of deleted and corrupted mail, AI-powered analysis, and export to industry-standard review formats—functioning as a specialist complement to general-purpose forensic platforms. Aid4Mail is trusted by Fortune 500 companies, the U.S. Department of Justice, the FBI, and professionals in over 100 countries.