Start with the assessment question

The Minimum Data Set (MDS) is a standardized assessment of nursing home residents. Answering an item can mean piecing together a progress note, a diagnosis, and an administration record. The assessment reference date anchors the observation period, and each item has its own coding criteria. CMS MDS 3.0 manual

The same resident chart can support different answers at different assessment dates. A record may be relevant to one observation period and fall outside the next. Retrieval needs the question and its date context to decide which records belong in the evidence set.

Build retrieval around the chart

Our home-built retrieval pipeline combines item-specific keyword search with the structure of the resident chart. The ranking weighs text relevance, how close each record is to the assessment date, the type of source, and which field matched. That gives the model a focused set of records to interpret against the item’s coding criteria.

We use BM25 to score how well the search terms match each record, then combine that score with source and field weights and recency. Building this layer in-house lets us tune retrieval for the question being answered and inspect why one record ranked above another. The model receives a curated context with source identifiers that can be carried through to the explanation.

  1. Find relevant recordsUse the item and its observation period to retrieve candidate chart records.
  2. Rank the candidatesCombine keyword relevance with timing, source type, and the chart field that matched.
  3. Propose an answerUse the selected records and item criteria to produce a suggested value with an explanation.
  4. Show the sourceLink the suggestion to chart records so a reviewer can verify the supporting evidence.

The review screen keeps the explanation beside the source links. A clinician can open the underlying record to see what it says and decide whether it supports the proposed answer.

Evaluate the answer and its evidence

I built calibration tools for MDS recommendations and diagnosis coding, worked on retrieval ranking and item-specific lookbacks, and helped integrate a fine-tuned model for skilled nursing assessments. The calibration tools let us compare proposed values with reference assessments and inspect the records and scores behind each run.

When a proposed answer disagrees with the reference, I can trace the case back through the pipeline. I can check whether retrieval surfaced the right record, whether the record was in the observation window, and whether the model interpreted it correctly. Those checks tell me where to make the next change.

Visit Layr