Augmenting Tax Resolution with Agentic AI

Abstract

[Abstract]: This thesis investigates the application of Large Language Models (LLMs) to improve user assistance within PitBullTax Software, a professional Software as a Service (SaaS) platform used by tax practitioners in the United States for Internal Revenue Service (IRS) tax resolution. Practitioners must navigate complex IRS procedures, interpret regulatory publications, and execute multi-step workflows when resolving tax debts. These tasks generate repetitive support requests and require extensive training, presenting an opportunity for intelligent automation through modern natural language technologies. The work addresses three challenges inherent to deploying LLMs in regulated professional environments: ensuring factual accuracy through grounding in authoritative documentation, enabling objective comparison across multiple LLM providers with different cost and performance characteristics, and implementing security measures appropriate for handling sensitive taxpayer information. The proposed solution integrates retrieval-augmented generation (RAG) with the Internal Revenue Manual as its knowledge source, a multi-model adapter architecture supporting multiple commercial LLM providers, and a layered security framework addressing prompt injection and data protection concerns. The prototype was evaluated through quantitative benchmarking measuring reliability, latency, and cost across providers, complemented by qualitative validation from domain experts assessing response accuracy and practical utility. Results demonstrate the feasibility of deploying LLM-based conversational assistance in tax resolution workflows, while revealing important trade-offs between model capabilities, operational costs, and response quality that inform production deployment decisions.