South East Central Railway · S&T (Telecom) — Phase-1 Pilot

Answers from the manual.
Never from the model.

A plain-language question, answered strictly from the official Signalling & Telecommunication manuals — cited to the exact clause, traceable to the printed page, and refused outright when the manuals do not contain the answer.

Indexed corpus — IRSEM 2021 · Indian Railways Telecom Manual 2021 · Model SOP 2018

Grounded answer

Every claim carries its clause

The answer is assembled only from passages actually retrieved from the manuals. Each numbered marker is a live citation naming the document and clause it came from — nothing in the text is unattributed.

A cited answer with numbered citation markers
Your question Shown back exactly as typed
Answer from the manuals The direct, cited answer
Also check the internet Optional, separately labelled
Every claim carries its clause The marker links to its source
Document & clause details Manual, clause number and page
Citations section Every source used, listed

Your question — asked, shown back as typed. Answer from the manuals — the direct, cited answer. Also check the internet — an optional web lookup. Every claim carries its clause — marker links to its source. Document & clause details — manual, clause, page. Citations section — every source used, listed.

What to notice The markers [1], [2] are not decoration — each resolves to a document, a clause reference and a page number. An answer that cannot be cited is never shown.

Conversation context

Follow-ups resolve against the conversation

"What about the pipes?" is not a question a search index can answer. The prior turns rewrite the elliptical follow-up into a standalone question before retrieval runs — so the follow-up is grounded just as strictly as the first question was.

A follow-up question resolved using earlier turns
The design constraint History feeds query rewriting only. It never reaches the answer prompt, so a long conversation can never smuggle an unsourced claim into a later answer.

The printed page

The citation chain ends at the paper

Tapping a citation opens the actual rendered page of the manual, with the exact lines the answer drew from highlighted on it. The reader verifies against the document itself rather than trusting the summary.

The original manual page with the cited lines highlighted
Why it matters here The manual remains authoritative. The portal assists retrieval — it does not replace the document, and it is built so a reader can always check it.

Optional web lookup

Clearly separated from the manuals

Where the corpus is silent, the reader may opt in to a web search. The result is fenced, labelled, and never mixed into the cited answer — and untrusted sources such as social media pages are filtered out before they can be quoted.

Web search results, visually fenced from the grounded answer
What to notice The dashed border and the footer line — "Not verified against the indexed manuals. The cited answer above is the authoritative one." Provenance stays visible at a glance.

Refusal

"Not found" is a feature, not a failure

Asked something the manuals do not cover, the portal declines and says so — naming the documents it searched and the nearest related clauses. It does not improvise, and it does not dress a guess up as an answer.

A refusal: not found in the knowledge base
Measured, not asserted Every out-of-corpus adversarial question in the evaluation suite was refused, including multi-turn traps that establish a legitimate topic and then pivot to one outside the manuals.

Saved answers

Keep an answer, keep its evidence

A saved answer stores its full citations — document, clause, page and passage — not a pointer back to the chat. It survives deleting the conversation it came from, and its citations stay fully interactive.

The saved answers library

Light and dark

Built for a signal cabin, not just a desk

Dark mode is a full theme, not an inverted filter — sidebar, source viewer and highlight colours are all restyled for low-light reading, with the choice remembered once a reader picks it.

The portal in dark mode, sidebar and source viewer open

In the field

The same answers, on a phone

S&T staff need the manuals at the equipment, not only at a desk. The interface is built for a one-handed phone viewport — the same engine, the same citations, the same refusals.

Portal on a phone — a cited answer with its citations block
Cited answer
Portal on a phone — the original manual page with the cited lines highlighted
Source page
Portal on a phone — the chats sidebar with saved answers and settings
Chats & saved

Engineering

What makes it more than a standard RAG

Retrieval plus a language model is the easy part. Almost all of the work went into the failure modes — the cases where a plausible-sounding answer is worse than no answer at all.

An unexpected finding Smaller instruction-following models outperformed larger reasoning models on this task. The stronger models produced more elaborate, more loosely-attributed answers — which the citation checks then rejected. Capability made them worse at grounded extraction, not better.

Evaluation

Measured, not asserted

The portal is scored against a standing evaluation harness of 221 questions and conversation turns across six suites, including a 43-question gold benchmark scored for retrieval, answer accuracy and citation exactness. Every defect ever found is locked by a regression case, so a fix cannot silently reopen.

Metric Result What it measures
Retrieval hit@5 100% The clause that answers the question is among the top five passages retrieved
Answer accuracy 97% The generated answer matches the expected substance of the clause
Citation exactness 97% The clause reference cited is the correct one
Not-found precision 100% Every refusal was genuinely outside the corpus — no real question wrongly refused
Not-found recall 100% Every out-of-corpus question was refused rather than answered
Uncited claims 0% No sentence asserted without a citation behind it
Latency (p50) 5.6s End to end, question asked to answer complete
What these numbers do not say Safety, collision and not-found handling are 100% of small case counts — that is no known failures, not a solved class of problem. The honest reading is that nothing has broken yet in the cases tested, not that nothing can.

Under the hood

How an answer is built

  1. RewriteFollow-ups resolved into standalone questions using recent turns.
  2. RetrieveSemantic and keyword search run together, then the combined results are re-scored for relevance.
  3. GateCalibrated thresholds decide answer or refuse — before any generation cost is spent.
  4. JudgeA sufficiency check asks whether the passages actually answer the question, not merely share its topic.
  5. Generate & verifyThe draft is checked back against its sources; unsupported sentences are repaired or the answer is discarded.
  6. CiteEvery claim is bound to a clause, a page, and the lines on it.