A $2.5M tool realizing a fifth of its value
When a hospital's insurance claim is denied, the RCM company writes an appeal to the insurer arguing why the claim should be paid. Appeals are labor-intensive: an analyst or clinician reads the patient's electronic medical record, often 130 to 700 pages, then constructs a written argument.
The product and engineering teams built a generative AI appeal generator: ingest the EMR, draft the appeal. The business case was roughly $2.5M in annual EBITDA from analyst time savings, and it was rolled out to the entire appeals team. Two quarters after launch, telemetry showed strong usage. Realized value was roughly $0.5M.
Run cost was consuming the savings
Every appeal required converting hundreds of pages of tables, forms, and clinical notes into model-readable text. That extraction step alone was roughly 80% of the per-appeal cost, and at production volume it offset about two thirds of the projected savings.
Usage was not adoption
Analysts opened the tool, then rewrote a large share of the drafts. Telemetry recorded a use; it could not see the rework. The time savings the business case depended on were not materializing.
Underneath both was a definitional problem: “appeals” had been treated as one task. Operations knew it was several.
From a generic pipeline to a persona-specific one
Cost diagnosis
Walked the architecture step by step and attributed cost per appeal to each stage. Isolated EMR extraction (OCR and layout parsing of tables and forms) as roughly 80% of run cost. The root cause was not the model; it was sending the entire record when only a fraction was relevant to any given appeal.
Operational task mapping
Sat with analysts across appeal types to document what each actually reads in the EMR. A medical necessity appeal requires the patient's clinical journey over time; a DRG downgrade appeal turns on one or two physician notes. Produced a relevance map: which EMR sections matter for which appeal type.
Re-architecture of ingestion
Translated the relevance map into a hierarchical chunking approach with engineering: read section headers first, retrieve only relevant sections, skip tables and images where not needed, then order clinical events chronologically before drafting. Benchmarked alternative extraction vendors for the remaining OCR workload against the incumbent on cost and accuracy.
Adoption measurement and refocus
Surveyed the operations team to identify which appeal types the tool handled well and which were being rewritten. Redirected engineering effort to the appeal types with low rework, and defined a persona-specific roadmap for the rest. Established rework rate, not usage, as the adoption metric.
Per-appeal run cost by pipeline stage
Illustrative shares shown; actual figures are confidential. Input and output tokens were the smallest cost line. Data extraction was the largest. The original case modeled the former and omitted the latter.
Hierarchical chunking; only relevant sections extracted; vendor benchmark
Second-pass agent orders retrieved sections
Unchanged
Unchanged
Rework by appeal type
Telemetry showed near-uniform usage. The operations survey showed the value was concentrated in a subset of appeal types.
Medical necessity
StrongFull clinical journey, chronological
Scale; primary focus
DRG downgrade
WeakOne or two physician notes; coding argument
Refactor per persona before re-release
Administrative / timely filing
Under reviewSmall set of dates and correspondence
Sequenced behind medical necessity
Legal / contractual
Under reviewContract terms, payer policy
Sequenced behind medical necessity
Rebuilding the ROI on four inputs
The original case used two inputs: population and time saved. The rebuilt case used four.
Population impacted
Productivity gain
Cost to run
Capacity plan
What transfers to any AI deployment
Extraction, not inference, is the cost driver
For any workflow built on long documents, the cost of making the document machine-readable dominates model cost. Business cases that model tokens and omit extraction will overstate ROI by a wide margin.
“One task” is usually several
The same job title concealed at least four distinct appeal types with different evidence requirements. A generic pipeline was expensive for all of them and good for only some.
Usage metrics hide rework
Telemetry cannot see an analyst rewriting a draft. Adoption must be measured as output accepted without rework, which requires going to the operators directly.
Narrow beats broad
A tool that removes 30-40% of the work for a defined population produces bankable savings. A tool that saves 5% of everyone's time produces a spreadsheet figure that never reaches the P&L.
Cost to run and capacity plan belong in every case
The two inputs most often omitted are the two that decide whether value is realized: what it costs to operate at volume, and what the organization will do with freed hours.
Run cost down ~60%; value re-baselined by appeal type
Re-architected ingestion reduced per-appeal run cost by roughly 60% while maintaining draft quality on analyst review. The tool was refocused on the appeal types where analysts accept the output, with a persona-specific roadmap for the remainder.
The next phase closes the loop: capturing appeal outcomes (overturned, upheld, partially paid) and feeding them back into draft quality, and ultimately into a model that predicts which denials are worth appealing at all.
The engagement also produced the operating framework now applied across the client's AI portfolio: four-input business cases, narrow pilots of 30-60 users of a single persona, and rework rate as the adoption metric.
Discuss a Similar Engagement