P1: Reflex External AI Assistance

Overview and Purpose

This document outlines the architecture and rationale behind an enhanced AI-integrated search and analysis capability within the Reflex platform. The capability is implemented as a modular, vendor-independent component and is the first deliverable from the Reflex spinoff, GRAYBELT Academy. For additional technical background and forward-looking information, see the GRAYBELT Academy site.

The feature is designed to solve a core problem: incident-related input from Reflex users (in the form of forum messages and informal observations) typically cannot be interpreted meaningfully by an AI engine alone. This is because there is no accumulated corpus of past incidents yet available, and because generalized AI models are trained on public, static datasets that do not include security-specific, live incident data. Submitting this input to an AI model yields superficial, textbook-like answers that do not address the specifics of the current situation.

Initial Problem and Structural Response

At the top of the diagram, user-generated data is represented as a FILO (First-In, Last-Out) stack. This reflects how observations and forum messages are collected and staged. The most recent content is always processed first. This mirrors the common FILO data handling approach used in legacy software systems, such as mechanical dish dispensers used in diners. This metaphor illustrates how Reflex retrieves the latest information first, without assuming the content itself is chronological or logically structured.

To overcome the limitations of general-purpose AI models, the system reframes the task. Rather than asking an AI to explain the data, the goal is to extract a targeted search string—a phrase that can be sent to a search engine to locate externally relevant data. If the AI were asked to interpret the meaning of the supplied strings directly, it would rely solely on its internal model—limited to the data it was originally trained on and unable to access current, real-time information available elsewhere.

The system introduces a black box containing two independent components:

• An inserted AI, which attempts to generate a search string based on the collected user content.
• A custom full-text retrieval engine, which also attempts to generate a search string, but is fundamentally different. It relies on the built-in Reflex AI, which is intentionally biased toward the platform creator’s reasoning and security background. It does not try to be neutral or general-purpose. It tries to approximate the kind of search that an experienced incident handler would construct.

A Results Evaluator then compares both generated search strings. This comparison is agnostic: it does not assume the full-text engine or the inserted AI will be better in all cases. Performance depends on many factors—including the capabilities of the AI model being used and the current availability and structure of information on the public internet. Whichever query is expected to return more relevant or actionable results is stored in a cache for the next processing stage.

It’s important to note that this process is asking more of the AI than it was designed for. The data being submitted—consisting of raw human dialogue and informal notes—does not explicitly state what is being asked. In contrast to typical prompts where a human is clearly requesting something, this system is essentially saying: “Make sense of this human conversation.” The input may include multiple discussion threads and fragmented observations, without any clear structure. The only thing tying them together is the fact that they all relate to the same ongoing incident.

In the early stages of an incident, there may be very little useful information to work with. The input can be vague or incomplete, and may not yet contain enough clues to suggest a meaningful direction. As the incident progresses and more material accumulates, the likelihood increases that a relevant search string can be derived. This phase is not about reaching a conclusion—it is about identifying the most promising starting point for external search.

Search and Analysis: A Modular, Vendor-Independent Black Box

Once a set of search strings has been collected in the cache, the next phase begins. The objective now is to use those strings to obtain timely, relevant external information that could help understand the incident in progress. This phase is handled by a second black box, which performs two functions in sequence:

1. Search Execution — The stored search strings are submitted to a designated search engine. This engine is not a free public search utility. It is a paid commercial engine chosen specifically for its lack of embedded bias, ad prioritization, or content filtering common to consumer-facing search platforms. Its ranking algorithms are not influenced by vendor preferences, promoted content, or internal agendas. This neutrality is critical in information security, where false emphasis can lead to wasted time or missed threats.
2. AI Interpretation — The results returned by the search engine—raw documents, summaries, and site snippets—are passed into an inserted AI engine. This AI is different from the one used earlier in the process; it has one task: to read only the search results and synthesize a meaningful explanation or analysis based solely on those results.

To assist with this, the orchestrator can insert a contextual prompt at this point. This is a structured piece of guidance that explains to the AI where the content came from and what it’s meant to inform. For example, the prompt might say: “This content is the result of a web search initiated during a live security incident. Your task is to identify any relevant patterns or threats that might explain the situation. Do not rely on your own model or training data. Respond only based on this input.”

This is the final, crucial human contribution. It ensures the AI doesn’t default to its internal model or inject assumptions. It frames the response in the language of the incident and reflects the orchestrator’s oversight.

A core feature of this black box is its interchangeability. The search engine and the AI interpreter are both modular—either can be replaced independently at any time. This design choice is not just about optimization; it reflects the reality of today’s AI development cycle. The best model or engine available today may be outclassed next week. The rapid pace of competition among AI vendors ensures constant improvement. By remaining agnostic and modular, this system passively benefits from that competition without becoming entangled in it.

This approach aligns with a core tenet of the platform: CISOware is not an AI company. CISOware produces Reflex, which is a security-first situation response platform. This feature fits directly into that mission. The platform remains focused on security outcomes, not the AI race. The final owner of the platform may choose to continue this modular strategy or substitute preferred vendors. But until security-specific LLMs become available, this implementation delivers the best combination of immediacy, relevance, and flexibility possible with current technology.