How Scout works, from strings and ranks to generated answers.

Scout is mostly ordinary software wrapped around one generative step.

A useful way to understand Scout is to stop thinking of it as one giant AI model. The application is a pipeline. JavaScript prepares facts, ranks evidence, tracks conversation state, applies rules, calls a language model for wording, and checks the result afterward.

The search engine does not search the raw JSON directly.

The knowledge file is structured for software, but a lexical search algorithm needs searchable text documents. buildRagChunks() turns the structured records into a small corpus of text chunks. A project becomes a sentence containing its name, description, technologies, and links. Experience becomes a sentence containing role, company, dates, and summary. Skill groups, certifications, FAQ records, rules, blog records, and Scout runtime facts become their own chunks.

This is what RAG means here: retrieval selects outside evidence first, then generation receives that evidence. The model does not need to memorize the portfolio.

Before BM25 sees text, Scout simplifies it.

For BM25, the tokenizer lowercases text, replaces non-alphanumeric characters with spaces, splits on whitespace, removes common stopwords, drops one-character tokens, and applies a small suffix stemmer. The stemmer removes endings such as ing, ed, ly, plural s/es, and sometimes a doubled consonant.

Build: O(total corpus tokens) The index counts every token once to build per-document term frequencies and global document frequencies.

Edit distance measures how many small edits separate two words.

Scout uses a dynamic-programming edit-distance function for vocabulary-based typo correction. The code calls it Damerau-Levenshtein. More precisely, the recurrence implemented is the adjacent-transposition/restricted variant commonly called Optimal String Alignment distance: insertions, deletions, substitutions, and one adjacent swap each cost 1.

Vocabulary terms come from the current RAG chunks. A protected set prevents absent but legitimate technology names such as COBOL, Rust, Kubernetes, Terraform, and others from being “corrected” into a nearby corpus word.

Per pair: O(m × n) for word lengths m and n. Because an unknown query word can be compared against many vocabulary terms, the rough correction cost is O(V × m × n) per unknown word, where V is vocabulary size. The corpus is small enough that this remains practical.

BM25 ranks documents by term usefulness, not just term count.

BM25 answers a practical question: “Given this query, which chunks contain useful matching terms?” Scout uses Okapi BM25 with k1 = 1.2 and b = 0.75. Three ideas matter: rare terms are more informative, repeated terms help but saturate, and unusually long documents get normalized.

If a term occurs in only 5 of 100 chunks, Scout's formula gives an IDF of about 2.9104. If it occurs in 50 of 100 chunks, IDF falls to about 0.6931. The rarer term contributes much more because it distinguishes documents better.

Saturation: if frequency rises from 2 to 4 in the same 80-token document, the factor rises from roughly 1.457 to 1.753, not to 2.914. Saying a term four times is not treated as four times the evidence.

Length normalization: if the same rare term occurs once in a 200-token chunk while average length is 100, the contribution is about 2.0654. A match buried in a very long chunk receives less weight than an equally rare match in a concise chunk.

The final document score is the sum of these contributions for query terms. It is a ranking score, not a probability and not a confidence percentage.

RRF combines rankings without pretending their raw scores are comparable.

Scout can search multiple views of the same question: normalized wording, alias-expanded wording, and conversation-aware rewritten wording. Each BM25 search produces its own ordered list. Reciprocal Rank Fusion combines those lists using only positions.

The constant 60 is the smoothing value used by the implementation. Rank 1 contributes 1/61 ≈ 0.016393; rank 2 contributes 1/62 ≈ 0.016129. A document appearing in several lists accumulates contributions.

A chunk that is rank 1 in only one list scores about 0.016393. A chunk that is rank 10 in three lists scores about 3/70 = 0.042857. This illustrates the point of fusion: consistent relevance across different query views can outrank a one-view spike.

Scout de-duplicates identical query strings before searching. Unless overridden, candidate depth is max(limit × 3, 12). The fused results are sorted by RRF score, with best individual rank as the tie-breaker.

RRF is not the final evidence order.

The RAG agent backfills needed evidence, applies an intent-specific allowlist where configured, scores the remaining evidence, deduplicates it, preserves pinned facts, and selects the final bounded evidence block. Ranking is one step in that selection process.

This is a hand-tuned heuristic. The multipliers are configuration encoded in JavaScript, not learned weights and not probabilities. After multiplying, evidence is sorted by boostedScore, deduplicated, capped to 8 items by default, and rendered into an evidence budget of 1,100 characters by default.

Intent-specific multipliers

Exact implementation details: PROFILE applies its own allowed-tag set before scoring. Skills-prefixed tags receive the family multiplier; certification receives an additional certification multiplier after the exact-tag boost, so the configured intent value can be applied twice. Pinned backfilled evidence is preserved separately from score order. These rules describe the audited implementation, not a universal retrieval formula.

A question is transformed into several search views before evidence reaches the model.

The current integrated chat path retrieves up to 10 BM25/RRF candidates for the RAG-primary agent. Structured tools can add evidence afterward; they are supplemental rather than replacements for retrieval.

Recall@6 asks “did the right evidence show up?” MRR asks “how high did it rank?”

The offline retrieval evaluator uses a golden set of questions. For each question it searches the BM25 index and checks whether one of the first six results matches an expected tag or keyword.

If 38 of 40 queries retrieve something relevant in the first six results, Recall@6 = 38/40 = 0.95.

The checked-in evaluator fails CI if Recall@6 is below 0.90. MRR is reported but is not the gate in that script.

Retrieved evidence still has to fit into a small prompt.

The context-packet code deliberately caps evidence and tool observations. It renders only fields needed by the model, removes duplicate rendered items, limits item counts, truncates text, and serializes compact conversation state.

This is a heuristic, not the model's real tokenizer. Four characters per token is a useful rough estimate for English-like text, but actual tokenization varies by language, punctuation, code, URLs, and model tokenizer.

These values are configurable through environment variables. The purpose is engineering, not statistical optimization: smaller packets reduce latency and keep irrelevant text away from a small model.

The model writes the prose; it does not choose the evidence corpus.

In the current production configuration, the provider adapter sends messages to Cloudflare Workers AI using @cf/meta/llama-3.1-8b-instruct-fast. Retrieval, state, response contracts, and validation run in Scout's backend process. The provider receives the prepared prompt and returns generated text.

The Cloudflare adapter defaults to temperature 0, top-p 0.9, and clamps requested output to at most 512 tokens. These are sampling/runtime parameters, not proof of factual correctness. Evidence limits and validation reduce unsupported claims; neither guarantees correctness.

Scout's current model has no published token-to-neuron rate.

Scout's hosted generation model remains @cf/meta/llama-3.1-8b-instruct-fast. Cloudflare documents that model as active, but its current pricing table does not publish an exact input/output neuron rate for that identifier. Scout therefore does not borrow token-to-neuron rates from a similarly named model.

Platform allocation and request accounting are different facts

Actual, estimated, and unknown are different states

Exact model identifiers matter

Accounting completeness also matters across several calls. If one billable call has unknown neuron usage, the complete session neuron total remains unknown instead of later becoming a misleading partial total just because a subsequent call has known usage.

Several simple clocks bound the runtime.

These are deterministic limits. There is no learned scheduler deciding them. The chat deadline uses an AbortController so outstanding inference can be cancelled when time expires.

Conversation state is structured data, and tools are ordinary functions.

Scout keeps server-owned state such as current topic, named projects, job/company context, active comparison, unresolved references, recent turns, and visitor identity state. This makes references such as “that project” resolvable without asking the language model to infer everything from a long transcript.

The current tool set is allowlisted and read-only:

Before prose is accepted, Scout represents what kind of claim the answer is allowed to make.

A response contract can carry fields such as intent, sub-intent, policy mode, direct answer, fact state, evidence strength, claim ceiling, requested role/topic, boundaries, and forbidden claims. The important concept is that the system tries to decide the semantic shape of the answer separately from the wording.

This is different from a closed-world database assumption. “Not found” does not automatically mean “false.” That distinction is why unknown-skill and future-capability questions need different handling from explicit false-premise questions.

Validation is a stack of checks, not one “confidence score.”

The grounding validator splits generated text into claims/sentences/clauses and applies multiple deterministic checks. The current code describes checks for overclaim language, entity grounding, number grounding, content-word overlap, question relevance, length/structure, claim upgrades, claim-level support, cross-project provenance, technology relations, negation scope, and relationship consistency.

When primary generated output fails validation, the RAG agent can attempt generative repair or constrained recovery. If reliable generation is unavailable, current code has typed failure paths rather than treating unsupported prose as valid.

The browser-local experiment separates evidence preparation from browser inference.

Current integrated server code exposes /api/client-packet, /api/client-validate, and /api/client-status. The server prepares a client-safe compact packet, keeps the full validation evidence server-side for 60 seconds, the browser can generate an answer locally, then the server validates that answer against the same evidence.

ONNX is a model exchange/runtime format. WebGPU exposes GPU compute capabilities to web applications. q4 generally denotes a four-bit quantized model representation, reducing model memory relative to higher-precision weights. This path is experimental and is not the normal production generation path.

Follow one question through the complete system.

Suppose the visitor asks: “Which project best demonstrates debugging?”

No individual step is especially mysterious. The behavior comes from the composition of small retrieval, state, rule, generation, and validation components.

The current design is optimized for a small local corpus, not internet-scale search.

If Scout's corpus grew from hundreds of chunks to millions, the current “score every chunk” BM25 search would stop being appropriate. A production search system at that scale would normally use an inverted index or external search engine so query work is proportional to postings for query terms rather than every document.

Do not attribute every Scout behavior to “AI.”

Terms used throughout the Scout site.

RAG

Retrieval-Augmented Generation. Retrieve relevant external evidence first, then provide it to a generative model for the answer.

Corpus

The collection of searchable documents/chunks. Scout's corpus is built from verified knowledge and runtime facts.

Chunk

A small searchable text record derived from structured knowledge, such as one project, experience record, certification, or runtime fact.

Token

A unit of text. In Scout's BM25 code, tokens are simple normalized words/stems. In an LLM, tokens are model-specific subword units. Those are not the same tokenizer.

Stopword

A common word removed from lexical retrieval because it usually carries little search value, such as “the” or “and.”

Stemming

Reducing related word forms toward a common stem, such as debugging → debug. Scout uses a lightweight hand-written suffix stemmer, not Porter/Snowball.

TF

Term Frequency. How many times a search term occurs in one document.

DF

Document Frequency. How many documents in the corpus contain a term at least once.

IDF

Inverse Document Frequency. A weight that makes rare terms count more than terms appearing everywhere.

BM25

Best Matching 25 / Okapi BM25. A lexical relevance formula combining IDF, saturated term frequency, and document-length normalization.

k1

BM25 parameter controlling term-frequency saturation. Scout uses 1.2.

b

BM25 parameter controlling document-length normalization. Scout uses 0.75.

RRF

Reciprocal Rank Fusion. Combines multiple ranked lists using reciprocal rank rather than their incompatible raw scores.

Candidate depth

How many results each individual search contributes before rank fusion. Scout defaults to at least 12 or three times the requested final limit.

Edit distance

The minimum number of allowed character edits required to transform one string into another.

Optimal String Alignment distance

The restricted adjacent-transposition edit-distance variant implemented by Scout's typo code, although the function is named Damerau-Levenshtein.

Anaphora

A reference such as “it,” “that project,” or “the first one” whose meaning depends on prior conversation.

Intent

A category describing what the user is trying to do, such as factual lookup, role fit, contact, weakness assessment, or small talk.

Evidence

Retrieved or tool-returned data that is allowed to support factual claims in the answer.

Response contract

A structured semantic plan describing the allowed answer state, evidence boundary, requested topic/role, and forbidden claims before or around generation.

Open-world assumption

Missing evidence does not automatically make a claim false. It can remain UNKNOWN unless authoritative negative evidence exists.

Grounding

Connecting generated factual claims to supplied evidence rather than allowing unsupported invention.

Provenance

Where a fact came from and which entity/project it belongs to.

Inference

Running a trained model to produce output. In Scout production, hosted Workers AI performs normal generative inference.

Provider

The runtime service or adapter that actually executes model inference, such as Cloudflare Workers AI or a local Ollama path.

Context / context packet

The bounded instructions, state, evidence, and tool observations supplied to the language model for one turn.

Temperature

A language-model sampling parameter that rescales logits before token sampling. Lower generally means a sharper, less variable distribution.

Top-p

Nucleus sampling parameter. Sampling is limited to the smallest high-probability set of tokens whose cumulative probability reaches the chosen threshold.

Neuron

Cloudflare Workers AI's usage-accounting unit. Scout preserves provider-reported actual neurons when available and only computes token-derived estimates when the exact model identifier has a verified rate. Unknown exact-model usage remains unknown rather than becoming zero.

TTL

Time To Live. How long cached or temporary data remains valid before expiring.

Rate limit

A cap on how many requests a client may make during a time window.

Liveness

A health check answering “is the process alive?”

Readiness

A health check answering “is the service initialized enough to receive real traffic?”

Golden set

A fixed set of test questions with expected retrieval/behavior used to measure regressions.

Recall@k

Fraction of test queries for which a relevant result appears somewhere in the first k results.

MRR@k

Mean Reciprocal Rank. Average of 1/rank for the first relevant result, with zero for misses inside the cutoff.

Regression test

A test retained because a behavior broke before or is important enough that future changes must not silently break it.

Fail closed

When a required safety/correctness condition fails, return an error or constrained result instead of pretending success.

WebGPU

Browser API exposing modern GPU rendering/compute capabilities. Scout has an experimental browser-local inference path that can use it.

ONNX

An open format/ecosystem for representing and running machine-learning models across runtimes.

Quantization / q4

Representing model weights at lower numeric precision to reduce memory and compute. “q4” generally denotes a four-bit quantized representation; exact packing/quantization scheme depends on the model/runtime artifact.

Embedding

A numeric vector representation of data, often produced by a neural model so semantically related items have nearby vectors. Scout’s active BM25/RRF retrieval path does not require embeddings.

Vector search

Retrieval by comparing numeric vectors with a similarity or distance function such as cosine similarity, dot product, or Euclidean distance. This is different from Scout’s current lexical BM25 ranking.

Vector database

A database/index specialized for storing vectors and retrieving nearest neighbors. A RAG system may use one, but RAG does not require one and Scout’s current retrieval path does not depend on one.

Semantic search

Search intended to match meaning rather than only exact words. Embedding-based vector retrieval is one approach. Scout instead improves lexical retrieval with normalization, aliases, context rewriting, BM25, RRF, and deterministic re-ranking.

Deterministic

Given the same inputs and configuration, ordinary code follows the same defined rules and calculations. BM25, RRF, tool functions, contracts, and most validators are deterministic.

Generative

Produces new output rather than only selecting stored values. In Scout, the language-model call is the main generative component.

Heuristic

A practical rule chosen by engineering judgment rather than learned from data or derived as an optimal theorem. Scout’s evidence multipliers and several validator thresholds are heuristics.

Re-ranking

Taking an already ranked candidate list and applying another scoring/ordering stage before final selection. Scout re-ranks retrieval evidence with type and intent multipliers.

Multiplier / boost

A numeric factor multiplied into a score. A factor above 1 increases relative priority; below 1 decreases it. It does not make the score a probability.

Logit

A model’s raw pre-softmax score for a possible next token. Sampling parameters such as temperature operate on logits/probabilities inside the model runtime, not in Scout’s BM25 code.

Reference library.

Implementation claims on this site are sourced to ProjectHub. The references below are independent material for the algorithms, evaluation methods, standards and provider behavior used in the teaching guide.