Production source line for the public widget and released backend behavior.
Reusable conversational runtime
Scout finds the evidence
before it answers.
Scout retrieves relevant knowledge, keeps conversation context, generates a bounded response, and checks its claims before replying. ProjectHub Recruiter Alpha is the current reference application; Scout's design is intended to support other knowledge domains too.
Source audit · September 15, 2026 · Evidence and branch inventory · Machine-readable snapshot
releasemaster
modelLlama 3.1 8B Fast
retrievalBM25 + RRF
deadline15 s
Released behavior, integrated development behavior, historical measurements, and planned productization are shown separately. Implementation references link to the audited source revision.
Matches the September 15 production tree 92b4d149. The semantic-reliability and tenant-portability line is now released through both integration and production source.
Canonical identity, empty/no-KB operation, structured semantic planning, and cross-domain portability regressions now ship in the same runtime. Formal domain packages and the controlled extension/action platform remain productization work.
01 / Features
Capabilities in the September 15 released tree.
These implementations are present in production source 7d011708. The discourse-frame work described as unmerged in the previous snapshot shipped in this release.
RAG.01
BM25 retrieval + RRF
Okapi BM25 indexes verified fact chunks. Contextual follow-ups can combine literal, alias-expanded, and rewritten rankings using reciprocal rank fusion.
source ↗CTX.02
Server session state
Tracks topic, projects, job context, comparison context, intent, unresolved references, visitor name, and five recent compact turns. In-memory TTL: two hours.
source ↗CNT.03
Response contracts
Derives intent, fact state, direct-answer polarity, required entities, evidence boundaries, and answer-shape instructions before generation.
source ↗TOOL.04
Read-only evidence tools
Portfolio search, project lookup/comparison, role matching, candidate profile lookup, skill evidence, and recruiter brief construction.
source ↗VAL.05
Grounding validator
Entity and number grounding, relationship checks, technology/project provenance, negation-aware claims, polarity, and overclaim/seniority rules.
source ↗GEN.06
Inference router
Cloudflare and Ollama share one inference boundary. Cloudflare is the current production primary; Ollama is the development/evaluation path and a gated fallback architecture.
source ↗OBS.07
Runtime telemetry
Prose source, provider, model, model-call count, token usage, provider-reported neuron usage when supplied, exact-model estimates only when verified, latency, repairs, retrieval candidates, selected evidence, and accounting completeness.
runtime facts ↗CI.08
CI checks
Build freshness, JavaScript syntax, cost/retrieval tests, retrieval threshold, knowledge validation, common secret-pattern scans, and staging/inference invariants.
workflow ↗REL.09
Production/staging separation
ProjectHub-dev:main is a prepared staging tree derived from ProjectHub:develop and records its source commit.
02 / How it works
Current request path.
The backend separates query handling, retrieval, state, tool execution, model inference, and answer validation. The model produces conversational text; deterministic code prepares evidence and checks the result.
Greetings, thanks, user-profile updates, and related control turns can use a compact generation packet.
Structured tools can add evidence but do not replace the primary RAG evidence context in the integrated path.
Open-world claims such as undocumented employment or future capability are not supposed to collapse into unsupported negatives.
03 / Verification
Claims with inspection paths.
The table below points to source, recorded artifacts, or commands that can be used to check the implementation.
Documented result on the 40-query retrieval golden set.
Golden set used for BM25/query-understanding regression measurement.
Local acceptance harness described by the current engineering instructions.
126 retained production inputs plus a six-turn unknown-technology repair regression.
| Claim | Evidence | Retest / inspect | Source |
|---|---|---|---|
| Retrieval Recall@6 = 1.000 | 40-query golden set + evaluator | npm run eval-retrieval | golden set ↗ |
| CI rejects retrieval below 0.90 | Threshold implemented in workflow | Inspect test.yml | workflow ↗ |
| Server keeps five compact turns | STATE_MAX_TURNS = 5 | Inspect constants and addTurn() | session-state.js ↗ |
| Production primary is Cloudflare Llama 3.1 8B Fast | Runtime facts + inference router | Inspect master runtime facts and router defaults | runtime facts ↗ |
| Production Ollama fallback is gated | Requires explicit fallback enablement and qualification | Inspect fallbackToOllamaEnabled() | router ↗ |
| Staging records canonical source commit | STAGING-SOURCE.json | Compare marker with ProjectHub commit | marker ↗ |
| Generated answers are checked by validators | Validator implementation + tests | Inspect validator and run repository tests | validator ↗ |
| Current public tools are read-only | Allowlisted recruiter tool definitions | Inspect tool definitions and dispatcher | tools ↗ |
131/178Historical strict re-score
73.6% of preserved pre-strict artifacts passed the newer scorer after earlier 114/115 and 40/40 results were found to contain false positives.
This is not a current production accuracy number. It is retained because it records the scorer change and the specific failure classes that followed: closed-world employer denials, future-capability polarity errors, incorrect fact-state contracts, and project/technology provenance drift.
Current source state
Released tree + active branch.
The September 15 release promoted the exact qualified integration tree to production. The next productization feature — the Scout action/tool runtime substrate — has started on a separate unmerged branch.
Not a perfect-conversation claim: the dated DEV live gate on the qualified runtime
654d25e recorded 100/132 turns and 20/33 conversations passing, with validator rejections of model fabrications among the residual failures. That is dated qualification evidence, not a universal accuracy claim.Current active branch is not released:
feat/scout-action-runtime@ebed168 adds the Phase 11 capability-registry substrate (tool registry, executor, permission policy, action audit). It has not crossed integration, staging, or release gates.Cloudflare allocation and exact-model accounting
Scout uses @cf/meta/llama-3.1-8b-instruct-fast. Its exact token-to-neuron rate is unverified. Provider-reported usage is retained when available; unknown usage is not zero. Cloudflare includes 10,000 neurons/day, resetting at 00:00 UTC. Workers Paid usage above that allocation is $0.011 per 1,000 neurons.
Release / September 15, 2026
Tenant-portability and semantic reliability are now in the released Scout tree.
PR #31 integrated the long-running conversation/core hardening into protected integration, and PR #32 promoted the exact qualified tree to production source without drifting from the tested develop tree.
7d011708Production and integration share tree 92b4d149.
201beb9cGenerated from integration e74ac22b.
Recall@6 1.000; dated DEV gate 100/132 turns · 20/33 conversations.
This release adds structured semantic query planning, canonical identity precedence, proposition-scoped validation, answer obligations, empty-KB support, and tenant-neutral portability coverage. Formal domain packages and the action/extension platform remain roadmap work.
04 / Scope
Current reference app vs. reusable runtime.
The deployed application is scoped to Bradley Matera's professional information and Scout's own runtime facts. The released Core already runs tenant-neutral and empty-knowledge configurations; formal domain packaging is what remains unshipped.
Current application
ProjectHub Recruiter Alpha
Bundled verified knowledge plus recruiter-specific tools.
- Projects, skills, work history, education, certifications, career goals, and public contact information
- Project comparison, skill evidence, profile retrieval, recruiter briefs, and role matching
- Multi-turn context and referent handling
- Scout runtime/meta questions
- Cloudflare model generation with local retrieval and validation
Released Core foundation
Tenant-neutral Scout Core
The released engine now supports tenant-neutral identity, relationships, semantic planning, empty/no-KB operation, and portable entity/property behavior. Formal domain packages remain the next boundary.
- Shipped: supported and tested empty/no-knowledge mode
- Shipped: canonical tenant-neutral identity, relationship, and property handling in Core
- Shipped: synthetic unrelated-domain and empty-KB regression coverage
- Remaining: formal domain-package contract and validator
- Remaining: runnable unrelated-domain packages on the same Core
- Remaining: deployment, licensing, security, and operator handoff packaging
05 / Regression controls
Implemented controls tied to observed failure cases.
These rows describe current or historical mechanisms in the repository. They are not a global quality score.
Integrated
Conversation-aware BM25/RRF
Follow-ups can preserve explicit subjects and combine multiple retrieval views.
Integrated
RAG-first primary path
Retrieved evidence is built first; structured tools provide supplemental evidence.
Integrated
TRUE / FALSE / UNKNOWN claim state
Open-world facts can remain UNKNOWN instead of becoming unsupported negative claims.
Integrated
Generative repair / constrained recovery
Validation failure can trigger another model pass under tighter constraints.
Integrated
Negative claims and hypothetical-role framing
The September 15 release tightened unknown facts, future-capability roles, and documented-gap responses with semantic (not lexical) contract checks.
Integrated
Cross-domain portability regression
Synthetic unrelated-domain and empty-knowledge regressions run in the released suite; runnable packaged domains remain productization work.
06 / Development history
Selected repository milestones.
Dates and descriptions below are based on merged PRs, commits, and current documentation.
Earlier ProjectHub
Embeddable recruiter / portfolio chatbot
The application began around a browser widget and recruiter knowledge, with model/provider logic and analytics added around it.
July 17, 2026
Staging and release controls
Staging isolation, CI hardening, deploy rollback/health checks, and coordinated release work were merged. PR #4, #5, #6, #7.
August 12, 2026
Git source-of-truth and workspace guardrails
Repository protections were added after an accidental stale-desktop merge, followed by multi-PC/IDE/agent workspace checks. PR #13, #14, #15.
August 2026 · local-agent phase
Conversation, contract, tool, and grounding layers expanded
qwen2.5 local models were used while conversation resolution, completeness checks, response contracts, tool selection, and grounding validation were developed and evaluated.
August 19, 2026
Strict scorer replaced earlier release-gate interpretation
Stored outputs were re-scored after false positives were identified in the earlier scorer. strict scorer commit.
August 20, 2026
Validated develop release promoted
A 93-commit release PR moved the validated develop line to master. PR #19.
August 21, 2026
Cloudflare Llama 3.1 8B Fast configuration
Runtime configuration and documentation were updated for @cf/meta/llama-3.1-8b-instruct-fast. commit d32b75e.
August 21–22, 2026
Post-release develop hardening
develop continued changing evidence selection, role framing, negative claims, and UNKNOWN handling after the production release.
07 / Repository state
Two repositories, one canonical source.
Project development belongs in ProjectHub. ProjectHub-dev is a staging deployment mirror prepared from the integration branch.
Canonical source
BradleyMatera/ProjectHub
Scout backend, ProjectHub frontend, tests, eval artifacts, release workflows, Docker files, analytics, and development instructions.
master
Released production line.
develop
Integration line; same Git tree as production at this audit.
old branches
Most audited feature/chore/release branches have no unique product commits ahead of current develop and function as historical pointers.
Staging deploy target
BradleyMatera/ProjectHub-dev
main is prepared from ProjectHub develop. The generated tree records its canonical source commit in STAGING-SOURCE.json.
main
Prepared staging tree, not an independent source code line.
frontend
bradleymatera.github.io/ProjectHub-dev/
backend
dev.projecthub-chat.bradleymatera.dev
08 / Engineering controls
Review-driven engineering.
Scout development separates product direction, implementation, verification, and release authority. Changes advance only when source state, tests, deployment provenance, and human evaluation support them.
Bradley Matera
Defines product goals, architecture requirements, behavior constraints, and the standard a change has to meet before it moves forward.
- product direction and architecture requirements
- behavior constraints and acceptance criteria
- human conversation evaluation
- integration and production authorization
Local repository + Devin/Windsurf
Scoped engineering tasks are implemented against the real workspace, then exercised through tests, evaluation harnesses, staging, and development deployment where required.
- source edits in the working repository
- unit, regression, browser, and conversation tests
- development/staging deployment when required
- commits and implementation evidence for review
GitHub + ChatGPT review
Architecture decisions and reports are checked against repository state, branch heads, PR history, test evidence, staging markers, runtime behavior, and external technical sources when needed.
- source, branch, PR, and deployment inspection
- architecture and failure analysis
- implementation planning and review criteria
- independent verification before the next gate
Review loop
Requirement → scoped implementation → local code/test cycle → source + deployment review → human behavior review → accept / revise → integration or release gateEvidence gate: no completion report, test count, branch name, or model output advances code on its own. Source state, tests, deployment provenance, and human evaluation are checked before integration or release.
09 / Roadmap
Released source first. Active work stays separate.
The September 15 release merged the semantic-reliability and tenant-portability work into develop, staged it, and promoted the exact tree to production. New productization work — the action/tool runtime substrate — is on a separate unmerged branch.
CURRENT ENGINEERING STATE
Released tree + active productization work
Production 7d011708, protected integration e74ac22b, and the current staging marker all agree on the released source tree 92b4d149. Separately, feat/scout-action-runtime@ebed168 adds the generic tool/capability registry, executor, permission policy, and action audit described by Phase 11. That branch has not crossed integration, staging, or release gates.
production · 7d011708develop/staging source · e74ac22bactive branch · ebed168released-tree tests · 1452/1452
Phases 01–06 · engine/release truth
The current recruiter implementation crossed integration, staging, and production source gates again for the September 15 iteration. New productization work is on a separate branch, while Phase 06 keeps code, provenance, evaluation, and documentation claims aligned.
01 · released
Working Scout foundation
Preserve the RAG-first retrieval, hosted generation, validation, state, provider, telemetry, and release foundations already in the system.
InvariantLater work must not silently regress the working foundation.
02 · released iteration complete
Conversation quality gate
The semantic-reliability iteration — canonical identity precedence, structured semantic plans, answer obligations, and generative-freedom contracts — was merged through PR #31 and released. The dated DEV gate on the qualified runtime recorded 100/132 turns and 20/33 conversations.
Continuation ruleKeep residual failures classified, preserve open-world/grounding behavior, and require any new branch to cross normal test/integration/release gates.
03 · released iteration complete
Integrate accepted conversation work
The semantic-reliability iteration is integrated in develop@e74ac22b via PR #31. The new feat/scout-action-runtime branch has not crossed this gate.
Current boundaryActive branch holds the tool-runtime substrate and has not opened a release-bound PR yet.
04 · released iteration complete
Staging truth + parity
ProjectHub-dev records develop@e74ac22b in STAGING-SOURCE.json, and the staging frontend serves a ProjectHub.js byte-identical to the develop tree. The active branch is not represented by staging.
BoundaryA source marker proves provenance; browser/backend runtime behavior remains a separate check.
05 · released Sep 15
Production source release
PR #32 promoted an ancestry-preserving release commit whose Git tree exactly matches qualified develop@e74ac22b. Current master@7d011708 and develop share tree 92b4d149; the production backend reports that same source commit.
BoundaryThe newer tool-runtime branch is not production. Git source release and external production-host verification remain separate facts.
06 · substantially complete
System truth cleanup
The September 15 refresh aligned executable behavior, runtime self-knowledge, deployment provenance, public documentation, and product-site claims with the released tree.
Standing ruleKeep historical scores dated; keep active branch capabilities separate from released capabilities; re-verify product truth on each release.
Phases 07–12
A refactor only succeeds if Scout still works afterward. ProjectHub Recruiter Alpha must remain a real configuration of the same core, not become a discarded prototype beside a second rewrite.
07 · substantially complete
Scout Core extraction
Separate reusable orchestration from Bradley/recruiter knowledge, identity assumptions, policies, tools, and workflows while ProjectHub continues to run on that same core.
Exit gateRecruiter-specific behavior sits outside core boundaries; normal specialization needs no customer-specific core branches.
08 · released foundation
General Scout / no-KB mode
Prove Scout Core exists without a customer knowledge package. Knowledge specializes Scout; it does not create Scout.
Exit gate
Scout Core + no domain package → General Scout is an intentional tested configuration.09 · next
Domain package contracts
Define stable knowledge, configuration, identity, policy, workflow, tool, and controlled-extension interfaces.
Exit gateA new domain specializes Scout through defined contracts instead of scattered core edits.
10 · partial proof
Cross-domain portability proof
Run deliberately unrelated domains such as recruiter, inventory/fruit-store, and IT support against the same Scout Core.
Pass conditionSame Scout Core SHA, different domain packages. Core edits needed only to satisfy one domain mean the portability test failed.
11 · planned
Extension + agent platform
Add richer customer tools, workflows, integrations, and agent-to-agent capabilities behind permissioned contracts instead of turning Scout into an unrestricted tool shell.
Exit gateSchemas, permissions, timeouts, validation, side-effect classes, failure isolation, logging, and tests exist.
12 · planned
Commercial/operator handoff
Make Scout installable, configurable, testable, deployable, operable, troubleshootable, and extensible by another competent developer without hidden Bradley-only knowledge.
Exit gateDeployment docs, ADRs, operator runbooks, configuration validation, security/licensing review, limitations, and handoff docs are sufficient for an independent operator.
09 / Current limits
Claims not made by this page.
These boundaries are based on the current repository and runtime configuration.
The released Core operates correctly with no knowledge package, but the only deployed application remains the recruiter-scoped one; a standalone general-mode product is not offered.
Empty/no-KB operation is tested and supported in the released runtime, but no runnable unrelated domain package exists yet.
Production generation currently uses Cloudflare Workers AI.
The router requires explicit enablement and qualification.
Dockerfiles exist, while the release docs still identify the SCP-style GCP deployment as a migration target.
Early-access reference prices are published; a portable customer-neutral package remains productization work.
Verification targets
Production, staging, source, and audit.
The links on the right are the direct places to inspect the deployed application, integrated staging build, canonical repository, and the source audit used to build this page.