Financial losses in Medicaid, from Fraud, Waste and Abuse (FWA), in the United States are estimated to be in the tens of billions of dollars each year. This results in escalating costs as well as limiting the funding available to worthy recipients of healthcare. The Centers for Medicare & Medicaid Services mandate thorough auditing, in which policy investigators manually research and interpret policy to validate the integrity of claims submitted by providers for reimbursement, a very time consuming process. We propose a system that aims to interpret un-structured policy text to semi-automatically audit provider claims. Guided by a domain ontology, our system extracts entities and relations to build benefit rules that can be executed on top of claims to identify improper payments, and often in turn payment policy and/or claims adjudication system vulnerabilities. We validate the automatic knowledge extraction from policies based on ground truth created by domain-experts. Lastly, we discuss how the system can co-reason with human investigators in order to increase thoroughness and consistency in review of claims and policy, to identify providers that systematically violate policy and to help in prioritizing investigations.