Scout — Intelligence Engine for ProjectHub Recruiter Alpha
Scout is a portable intelligence engine that powers an AI recruiter assistant for Bradley Matera's portfolio. It uses Cloudflare Workers AI for generative inference, local BM25 retrieval for evidence gathering, server-owned session state for multi-turn coherence, and strict grounded validation — all within a 15-second response contract. The frontend widget runs on GitHub Pages with zero build step.
Bradley Matera · Recruiter assistant
Calm, concise, honest, and grounded in real project data — powered by Cloudflare Workers AI free-tier inference.
What Scout is
Scout is a portable intelligence engine — not a hard-coded FAQ bot. It retrieves from real project data, recruiter knowledge, and source material using local BM25 retrieval, then generates conversational replies through Cloudflare Workers AI. Every generated reply passes safety, entity, number, length, source-overlap, polarity, and overclaim validation before it reaches the visitor.
The frontend widget is vanilla JavaScript with no build step, embeddable via a single script tag on GitHub Pages. The backend runs on a free GCP e2-micro VM with Cloudflare Workers AI free-tier inference.
Live deployment status
Model: @cf/meta/llama-3.1-8b-instruct-fast. RAG evidence is primary; one factual repair is allowed if the first generated answer fails validation.
Every request has a hard 15,000 ms budget. A factually invalid answer gets at most one generative repair; if that still fails, Scout fails closed instead of returning known-invalid prose.
Node.js Express API with server-owned session state, BM25 retrieval, and Cloudflare inference routing.
Vanilla JavaScript, no build step. Embedded via a single script tag. Zero hosting cost.
Safety, entity grounding, number accuracy, source overlap, polarity, overclaim detection, and length checks on every reply.
Per-tab session ID with topic tracking, referent resolution, and per-topic stance consistency (60-min TTL, cap 12).
Production is live at bradleymatera.dev/recruiter. The Scout Engineering Lab (experimental dev surface) is at bradleymatera.github.io/ProjectHub-dev.
What Scout answers
- Project questions: Explain my projects, CodePens, GitHub repos, tech stack, and what each project proves.
- Recruiter questions: Summarize my skills, background, certifications, AWS internship, education, and job fit.
- Career-fit questions: Compare my real experience against roles like web developer, IT support, cloud support, DevOps, technical writing, QA, or data-focused work.
- Grounded replies: All conversational responses are generated by the inference layer, then validated against retrieved evidence before delivery. No deterministic prose reaches the visitor.
- Honest limits: If the evidence does not support an answer, Scout says so instead of inventing experience. Unknown tools fail closed; no public tool performs writes or arbitrary web access.
Scout pipeline
How it works now
ProjectHub first resolves the current conversation into a retrieval query and always runs local BM25/RRF search. The highest-value evidence blocks become the model's primary context; structured tools may add evidence but do not replace RAG. Cloudflare Workers AI generates the conversational answer from that scoped packet. Factual validation then checks unsupported entities, relationships, project-to-technology attribution, employment, certifications, temporal claims, and overclaiming. If the first generation is factually invalid, Scout gets one evidence-guided generative repair. If the repair is still invalid, the request fails closed with a technical error instead of exposing known-invalid prose.
Inference efficiency
Scout saves hosted-model work by shrinking the context before generation rather than trying to replace the model with hard-coded answers.
- Local retrieval first: BM25/RRF searches the verified knowledge without a model call.
- Small evidence packet: only the highest-value evidence blocks are sent to Cloudflare instead of the entire knowledge base.
- One normal generation: most successful requests use one provider call; a second call is reserved for factual repair.
- Visible accounting: each reply exposes provider calls, input/output tokens, neurons, model latency, RAG selection, and repairs.
- Fail closed: validation cannot silently return a known-invalid answer just to save a provider call.
Browser WebGPU direction
The inference adapter boundary is designed for future browser-side WebGPU generation. In this model, capable browsers would run inference locally on the visitor's GPU, eliminating cloud inference costs entirely. Incapable browsers would fall back to cloud generation transparently. The server remains authoritative for RAG, state, evidence, validation, and orchestration regardless of where generation occurs.
This is an active research direction, not a current production feature. The Scout Engineering Lab (ProjectHub-dev) includes a browser capability panel that probes WebGPU support, memory, and adapter info.
Free-tier stack
Hosts the widget, landing page, and knowledge JSON for free.
Generative inference with @cf/meta/llama-3.1-8b-instruct-fast. The included allocation is 10,000 neurons per day, resetting at 00:00 UTC; the page now exposes live estimated usage.
Runs the Node.js Express API, BM25 retrieval, session state, and validation.
Verified recruiter facts and BM25 retrieval require no external database or API key.
How to embed ProjectHub
This is the script tag used to load the floating ProjectHub chat widget:
<script src="https://bradleymatera.github.io/ProjectHub/ProjectHub.js?v=14"></script>
If the script is cached aggressively, bump the version query like ?v=14.
Important links
- GitHub repo: github.com/BradleyMatera/ProjectHub
- Live demo: bradleymatera.github.io/ProjectHub
- Recruiter Hub: bradleymatera.dev/recruiter
- Scout Engineering Lab: bradleymatera.github.io/ProjectHub-dev
- CodePen: codepen.io/student-account-bradley-matera/full/yyLmYKR
Development state
Active branch: feat/rag-primary-restoration. The dev surface is validating the RAG-first pipeline, factual validator restoration, privacy controls, and live token/neuron/cost telemetry against the separate dev backend.
Latest commit
9b5728f — Scout alpha production state: Cloudflare Workers AI inference, session state fixes,
lenient validation, GREETING mode relaxation, clarification fallback fix, cache-hit session state update.
667 unit tests passing, 29 retrieval tests passing.
Scout pipeline (detailed)
Stage details
| Stage | Module | What happens |
|---|---|---|
| Session state | lib/session-state.js | Look up per-tab session ID. Track current topic, projects, recent turns, stances. |
| Query understanding | lib/query-understanding.js | Normalize, typo-correct, classify intent, rewrite bare follow-ups with context. |
| Policy classification | lib/response-policy-classifier.js | Determine response mode: GREETING, VERIFIED_FACT, PROFILE, OUT_OF_SCOPE, REFUSAL, FALSE_CLAIM_DENIAL. |
| BM25 retrieval | lib/bm25.js, lib/rrf.js | Okapi BM25 with TF saturation, IDF weighting, doc-length normalization. RRF fuses literal, alias-expanded, and context-rewritten rankings (k=60). |
| Response contract | lib/response-contract.js | Build requiredFacts, optionalFacts, contextEntities, mustMentionEntities, evidenceEntities, forbiddenEntities, responseShape. |
| Inference | lib/local-model-router.js, lib/cloudflare-provider.js | Generate conversational reply from contract + evidence + conversation state. Cloudflare Workers AI in prod, Ollama in dev. |
| Validation | lib/grounding-validator.js | Entity grounding, number accuracy, source overlap, polarity, overclaim, safety, length, fabricated entity detection. |
| Recovery | lib/lite-agent.js | Up to 3 attempts with progressively lenient validation. Final attempt uses lenientValidate (safety-critical only). |
| Session update | lib/session-state.js | Record turn, update topic, projects, stances. 60-min TTL, cap 12 stances. |
Production vs. development
| Dimension | Production | Development |
|---|---|---|
| Backend URL | projecthub-chat.bradleymatera.dev | dev.projecthub-chat.bradleymatera.dev |
| Widget asset | bradleymatera.github.io/ProjectHub/ProjectHub.js | ./ProjectHub.js (local dev asset) |
| Inference provider | Cloudflare Workers AI | Ollama (qwen2.5:1.5b) |
| Model | @cf/meta/llama-3.1-8b-instruct-fast | qwen2.5:1.5b |
| Agent mode | Lite agent (production-safe) | Full agent (experimental tools enabled) |
| Validation | Strict + lenient recovery | Strict + lenient recovery (same) |
| Session state | Server-owned, in-memory | Server-owned, in-memory (same) |
| Response cache | Enabled (5-min TTL) | Enabled (5-min TTL) |
| Think Mode | Disabled | Enabled (20-min interval) |
| Deadline | 15,000 ms | 15,000 ms (same) |
Inference routing
The inference adapter (lib/local-model-router.js) abstracts the provider behind a unified
generate() interface. Provider selection is driven by environment variables:
| Env var | Production value | Dev value |
|---|---|---|
| SCOUT_INFERENCE_PROVIDER | cloudflare | ollama |
| CLOUDFLARE_ACCOUNT_ID | Set | Not set |
| CLOUDFLARE_API_TOKEN | Set | Not set |
| OLLAMA_BASE_URL | Not set | http://127.0.0.1:11434 |
| SCOUT_AGENT_ENGINE_ENABLED | true | true |
| SCOUT_AGENT_MODE | lite | full |
Future: browser WebGPU routing
The adapter is designed for a third provider: browser-webgpu. In this mode, the server would
send the contract + evidence to the browser, which runs inference locally via WebGPU. The server remains
authoritative for RAG, state, validation, and orchestration. This is not yet implemented.
WebGPU research
Browser-side WebGPU inference would eliminate cloud inference costs for capable browsers. The research direction involves:
- Adapter detection: Probe
navigator.gpu, request adapter, query device limits. - Model selection: Identify browser-runnable models (ONNX, GGUF via WebLLM) that fit GPU memory.
- Fallback chain: WebGPU → Cloudflare Workers AI → constrained grounded recovery.
- Validation parity: Server-side validation must still run on browser-generated text.
- Contract transport: Server sends JSON contract; browser generates; server validates.
The browser capability panel below probes your current browser for WebGPU support.
Neuron economics
Cloudflare Workers AI uses neuron-based pricing. Each generation call consumes neurons proportional to input + output tokens. The free tier provides a generous monthly allocation. Scout's efficiency strategies:
| Strategy | Impact |
|---|---|
| Response cache | Eliminates inference calls for repeated questions within 5-min window |
| Context budget (5 turns) | Reduces input tokens vs. full history |
| Policy-aware contracts | GREETING mode = smaller contract = fewer input tokens |
| Lenient recovery | Avoids 4th+ inference call when answer is "close enough" |
| Max 3 recovery attempts | Hard cap on inference calls per request |
| 15s deadline | Prevents runaway generation from consuming neurons |
15-second budget
Every request has a hard 15,000 ms deadline enforced by server-gemini.js. The budget is
allocated across pipeline stages:
| Stage | Budget | Notes |
|---|---|---|
| Session + query understanding | ~50 ms | In-memory lookup + sync processing |
| BM25 retrieval + RRF | ~100 ms | Local index, no network |
| Response contract | ~20 ms | Sync contract construction |
| Inference generation | ~5–12 s | Cloudflare Workers AI round-trip |
| Validation | ~50 ms | Sync validation checks |
| Recovery (if needed) | remaining budget | Up to 3 attempts, each with inference |
| Session update | ~10 ms | In-memory state update |
If the deadline fires during generation, Scout delivers a constrained grounded fallback — never an incomplete or unvalidated reply.
Retrieval
BM25 configuration
| Parameter | Value |
|---|---|
| k1 (TF saturation) | 1.2 |
| b (doc-length normalization) | 0.75 |
| RRF k | 60 |
| Views fused | Literal, alias-expanded, context-rewritten |
| Golden eval set | 40 queries |
| Recall@6 | 1.000 |
Query understanding pipeline
- Normalization: Lowercase, strip punctuation, collapse whitespace.
- Typo correction: Damerau-Levenshtein against vocabulary (edit distance ≤ 2).
- Intent classification: Contact, role-fit, experience, smalltalk, frustration, factual-lookup.
- Contextual rewriting: Bare follow-ups ("tell me more about that") rewritten using recent turns.
Response contracts
The response contract (lib/response-contract.js) defines what the generated reply must contain
and what it must not contain:
| Field | Purpose |
|---|---|
| requiredFacts | Facts that must appear in the reply |
| optionalFacts | Facts that may be included for context |
| contextEntities | Entities from the question context |
| mustMentionEntities | Entities that must appear in the reply |
| evidenceEntities | Entities from retrieved evidence |
| forbiddenEntities | Entities that must NOT appear (e.g., wrong person, wrong company) |
| responseShape | Min sentences, max sentences, expected structure |
| policyMode | GREETING, VERIFIED_FACT, REFUSAL, etc. — drives validation strictness |
Validation layers
| Check | Module | What it catches |
|---|---|---|
| Safety | Grounding validator | PII leakage, harmful content, private data |
| Entity grounding | Grounding validator | Entities in reply not supported by evidence |
| Fabricated entity | Grounding validator | Entities not in knowledge base |
| Number accuracy | Grounding validator | Numbers in reply not matching evidence |
| Source overlap | Grounding validator | Reply doesn't overlap with retrieved evidence |
| Polarity | Grounding validator | Affirming a false claim or denying a true one |
| Overclaim | Grounding validator | Claiming more than evidence supports |
| Cross-project provenance | Grounding validator | Attributing one project's facts to another |
| Technology claims | lib/claim-extractor.js | Unsupported tech stack claims with alias matching |
| Relationship validity | lib/relationship-validator.js | Unsupported entity relationships (e.g., "uses_tech" not in graph) |
| Completeness | lib/completeness-check.js | Reply too short or missing required content |
| Negation scope | lib/negation-scope.js | Negated claims incorrectly flagged or affirmed |
Policy-aware validation
GREETING mode skips entity grounding, fabricated entity, and relevance checks — greetings don't need topic overlap or knowledge-base entity support. All other modes run full validation.
Recovery
When the initial generation fails validation, Scout attempts generative recovery:
| Attempt | Validation | Prompt strategy |
|---|---|---|
| 1 (initial) | Full strict | Standard contract + evidence |
| 2 (recovery 1) | Lenient | Rejection reason + "fix this issue" instruction |
| 3 (recovery 2) | Lenient | Stricter constraint emphasis + evidence re-injection |
| 4 (recovery 3) | Lenient (safety-critical only) | Minimal prompt — accept any safe answer |
| 5 (fallback) | N/A | Constrained grounded recovery — no inference call |
Lenient validation only rejects on safety-critical issues (PII leakage, harmful content). It accepts minor grounding imperfections to avoid wasting inference cycles on near-misses from the small model.
Conversation state
Server-owned session state (lib/session-state.js) enables multi-turn coherence:
- currentTopic: Tracks what the conversation is about (e.g., "projects", "skills", "experience").
- currentProjects: Which projects were recently mentioned — enables "tell me more about that one."
- recentTurns: Last 5 turns for context window construction.
- topicStances: Per-topic positions Scout has taken (e.g., "Bradley's main stack is JavaScript") — prevents contradictions. 60-min TTL, cap 12.
- Cache-hit update: Even cached responses update session state so follow-ups work after a cache hit.
Referent resolution
When a visitor says "tell me more about that" or "what tech does it use", Scout resolves the referent
("that", "it") using lib/conversation-resolver.js:
- Build conversation state: Construct context from recent turns + session state.
- Resolve referent: Map pronouns to the most recent subject in session state.
- Rewrite query: Replace "it" → "ProjectHub", "that" → "the weather dashboard", etc.
- BM25 with rewritten query: Retrieval uses the rewritten query for better evidence matching.
Failure taxonomy
| Failure type | Cause | Recovery |
|---|---|---|
| entity_not_grounded | Reply mentions entity not in evidence | Recovery with entity constraint |
| fabricated_entity | Reply mentions entity not in knowledge base | Recovery with entity allowlist |
| not_relevant_to_question | Reply doesn't address the question topic | Recovery with topic emphasis |
| number_mismatch | Numbers in reply don't match evidence | Recovery with number constraint |
| polarity_violation | Affirming false claim or denying true one | Recovery with polarity correction |
| overclaim | Reply claims more than evidence supports | Recovery with scope limitation |
| cross_project_provenance | One project's facts attributed to another | Recovery with project disambiguation |
| unsupported_tech_claim | Tech mentioned not in evidence | Recovery with tech allowlist |
| relationship_violation | Unsupported entity relationship | Recovery with relationship graph |
| incomplete_answer | Reply too short or missing required content | Recovery with completeness emphasis |
| safety_violation | PII leakage or harmful content | Immediate fallback, no recovery |
| deadline_exceeded | 15s budget exhausted | Constrained grounded fallback |
| clarification_required | Question too ambiguous to answer | Scout asks for clarification |
Current experiments
Probe WebGPU adapter, identify runnable models, design contract transport protocol.
20-min loop: stash weak answers, generate improved wording, score, judge, retain validated improvements locally.
Graph-derived type words replace hardcoded lists. Entity-type validation uses is_type triples.
lib/negation-scope.js prevents negated tech claims from being flagged as unsupported.
Engine works with non-Bradley knowledge sets. Two-tenant test suite validates cross-tenant isolation.
docker-compose.yml provides Ollama-parity testing. Cloudflare production uses GCP VM direct deploy.
Latest benchmarks
| Metric | Value | Source |
|---|---|---|
| Unit tests | 667 pass, 0 fail | npm test |
| Retrieval tests | 29 pass, 0 fail | npm run test:retrieval |
| BM25 Recall@6 | 1.000 | 40-query golden set |
| Test suites | 11 | Node test runner |
| Production smoke | 10/10 pass | scripts/prod-smoke-test.js |
| Conversation regression | 132 inputs | 126 production + 6-turn unknown-tech repair |
| Local API eval | 61 requests | scripts/eval-local-api.js |
Known limitations
- Hosted model constraints:
@cf/meta/llama-3.1-8b-instruct-fast(prod) andqwen2.5:1.5b(dev) are general-purpose instruction-tuned models. They can hallucinate, miss nuances, or produce awkward phrasing. Validation and recovery compensate but cannot eliminate all errors. - No persistent storage: Session state is in-memory on the GCP VM. A VM restart loses all sessions. No external database.
- Free-tier rate limits: Cloudflare Workers AI free tier has neuron allocation limits. High traffic could exhaust the monthly allocation.
- Single VM: One GCP e2-micro instance. No horizontal scaling, no load balancer, no redundancy.
- No streaming: Responses are delivered as complete JSON, not streamed. The 15s budget includes full round-trip.
- WebGPU not implemented: Browser-side inference is a research direction, not a current feature.
- Think Mode is dev-only: The self-improvement loop runs only on the dev backend. It never writes to GitHub or external systems.
- No multi-language: Scout speaks English only. No internationalization layer.