Building Evidence Guard: Preventing LLM Hallucinations in Automated Resume Tailoring

The Problem: Why Generative Resumes Fail

Large Language Models are probabilistic token predictors, not factual database engines. When tasked with matching a job specification that demands Kubernetes, eBPF, and Kafka, conventional prompt engineering like:

This naive constraint fails in three repeatable failure modes:

• Metric Inflation: Turning "Reduced latency by optimizing SQL queries" into "Achieved 87% p99 latency reduction" without empirical benchmark data.
• Technology Smuggling: Introducing adjacent technologies (e.g., if Docker was mentioned, the model casually adds Kubernetes and Helm).
• Scope Escalation: Elevating contributor roles into owner/architect designations (e.g., "contributed to PR review" becomes "headed organizational migration").

System prompts alone cannot prevent this reliably because the temperature and attention distribution prioritize fluent token sequences over strict factual containment. We needed an adversarial system that treats every model generation as untrusted input.

The Architecture: Two-Phase Generation & Audit

Rather than relying on a single monolithic generation step, Evidence Guard separates content generation from factual verification in an asymmetric pipeline:

By decoupling the creative phrasing engine from the factual verification engine, we can run high-throughput LLM generation at low cost while guaranteeing zero hallucinated claims.

Implementing Evidence Guard: Grounding & Verification

The core verification engine inspects three distinct claim types: Hard Technical Entities, Numeric Claims, and Action Scope Hierarchy.

Here is a simplified TypeScript implementation demonstrating how entity grounding and numeric containment checks are performed on generated candidates:

interface GroundTruthRecord {
  canonicalSkills: Set<string>;
  metrics: Array<{ rawValue: string; magnitude: number; unit: string }>;
  originalText: string;
}

interface AuditResult {
  passed: boolean;
  violations: Array<{
    type: 'ILLEGAL_SKILL' | 'FABRICATED_METRIC' | 'SCOPE_ESCALATION';
    token: string;
    explanation: string;
  }>;
}

export function auditBulletPoint(
  candidateDraft: string,
  groundTruth: GroundTruthRecord
): AuditResult {
  const violations: AuditResult['violations'] = [];

  // 1. Entity boundary extraction & whitelist validation
  const extractedSkills = extractTechEntities(candidateDraft);
  for (const skill of extractedSkills) {
    if (!groundTruth.canonicalSkills.has(skill.toLowerCase())) {
      violations.push({
        type: 'ILLEGAL_SKILL',
        token: skill,
        explanation: `Skill "${skill}" does not exist in candidate profile history.`,
      });
    }
  }

  // 2. Numeric delta containment
  const candidateNumbers = extractQuantifiableDeltas(candidateDraft);
  for (const num of candidateNumbers) {
    const isVerified = groundTruth.metrics.some(
      (m) => m.unit === num.unit && Math.abs(m.magnitude - num.magnitude) < 0.01
    );

    if (!isVerified) {
      violations.push({
        type: 'FABRICATED_METRIC',
        token: num.rawValue,
        explanation: `Metric "${num.rawValue}" was not present in the original experience evidence.`,
      });
    }
  }

  return {
    passed: violations.length === 0,
    violations,
  };
}

When an audit fails, the pipeline does not discard the candidate and retry blind. Instead, the violation array is converted into a structured negative feedback constraint: "The previous draft introduced unverified skill [Kubernetes]. Regenerate without mentioning Kubernetes." This convergence loop typically resolves in a single additional sub-200ms pass.

Benchmarking Accuracy & Guardrail Latency

Safety mechanisms often introduce unacceptable user-facing latency. To assess the cost of Evidence Guard, we benchmarked 500 bullet point transformations using Llama 3.3 70B hosted on Groq inference engines:

For an overhead of roughly 145ms (running fast Groq LPUs with deterministic regex/AST token filters), hallucination incidence dropped from nearly 1-in-4 bullets to essentially zero. In production, this was the difference between candidates feeling anxious about resume accuracy versus confidently exporting their LaTeX document.

Key Takeaways for Production LLM Pipelines

Building ResumeFit reinforced several core patterns that apply to any engineer building production LLM products: