Why we modify open-weight models instead of training on data or renting a chatbot

When a security firm says it uses AI, a technical buyer usually has two immediate questions:

1. Are you just wrapping a commercial chatbot API and sending our application traffic to a third party?
2. If you customize your models, are you training them on data from client engagements?

Our answer to both is no. Here is how our setup actually works.

Why raw frontier chatbots fall short on security reviews

Out-of-the-box frontier models are built to be polite, general-purpose assistants. Ask one to look at a web page and it will summarize the layout or offer generic advice about HTTP headers. It is not wired to sign in as Contractor A, map every /api/orders route, compare the responses against Contractor B's session, and test whether a shared search widget leaks records across accounts.

Just as important, piping a client's authenticated portal responses through a third-party consumer AI API creates data-residencies and retention questions that most companies would rather avoid.

Editing open model weights + hand-built testing logic (with zero training data)

Instead of renting a generalist API or building a giant training dataset of past client work, we do two things:

• Direct weight modification on open-weight models: We run our own open-weight models in isolated environments and modify their weights directly so they stay focused on offensive and defensive web analysis rather than conversational small talk.
• Testing logic and prompts edited by hand from human findings: Whenever our reviewers uncover a new way permissions drift, multi-step checkouts fail, or RAG chatbots leak documents across tenants, we encode that logic directly into our system prompts and testing harness by hand.

Because we update our models through direct weight edits and human-written rules rather than dataset training runs, we have no training dataset at all. Your application traffic, credentials, files, and findings are never used to train a model.

Cages in the middle, humans at the ends

A capable security model still needs hard boundaries. Before a review starts, a human reviewer configures an outbound scope filter with the exact domains from your signed scope and sets rate and mutation guardrails. During the run, the models explore your predefined domains autonomously—and if they spot a link to an outside host (like an internal ERP or a payment provider), they log it under Observed, Not Tested without ever probing it.

Once the run finishes, a human reviewer checks every candidate finding. If we cannot reproduce it and attach clear proof, it does not go into your report. See our sample report for what the finished deliverable looks like.