I'm building conflict detection for a retrieval-augmented knowledge base containing facts such as fees, deadlines, and eligibility criteria from many institutions. I want to normalize statements into deterministic attribute paths, for example program=BHM, category=General, year=2026-27, fee=₹150000, so equivalent facts from different documents can be compared and contradictions can be flagged.
Tables, labeled fields, question-and-answer pairs, and headed sections are straightforward because their structure provides the relationships. The difficult case is unlabeled prose such as: "Students from the general category are required to pay ₹1,50,000 for the 2026-27 academic year for the BHM programme." The fee, category, year, and program are related semantically, but there is no explicit delimiter connecting them. Handwritten sentence-pattern rules become brittle when wording varies across institutions.
What classical or pre-LLM techniques work well for extracting a small set of known fact types and their qualifying attributes from varied prose? I'm considering dependency parsing, rule-based information extraction, OpenIE-style subject-predicate-object extraction, or a small supervised tagger/CRF. The pipeline must be free, fast, reproducible, and able to run locally without live LLM inference. Pointers to practical libraries, architectures, or techniques that hold up on messy real-world text would be especially useful.
1 Answer
Consider reframing the hard part as extractive question answering rather than free-form generation. Since your schema is known, ask targeted questions such as "What is the fee for the BHM program for the General category in 2026-27?" and have a local span-extraction model return the answer text from the relevant sentence or document. You can first identify candidate sentences with keywords, entity dictionaries, date and currency patterns, or a lightweight classifier, then run extraction only on those candidates.
A Hugging Face model fine-tuned for extractive question answering can run locally on CPU, and the answer is constrained to a span that appears in the source. Combine it with deterministic validation: verify that the returned span contains a valid amount, that the sentence contains the expected program/category/year qualifiers, and that the evidence text is retained. A small NER model or CRF can similarly label spans such as FEE, PROGRAM, CATEGORY, YEAR, and DEADLINE, while dependency parsing helps connect them in simple sentence structures.
This still has uncertainty, so treat the model as a candidate generator rather than an authority. Set confidence thresholds, store multiple candidates when needed, and send ambiguous or unsupported cases to review. It also avoids using a hosted chatbot or live generative model; the model is a local, fixed component used only for the messy remainder.
The verify-rather-than-generate framing is useful. I already know the kinds of fields I'm looking for, so checking whether a matching value appears near the required qualifiers is much easier than interpreting an arbitrary sentence from scratch. A local extractive QA model for only the leftover prose cases seems like a good compromise.