Security · Evaluating the evaluator ·
Seven prompt-injection flags, two model problems
TickerTrac reads public documents to test an investor's saved reasoning. Those documents can contain instructions aimed at the model. We tested whether those instructions could affect answers or cross the application's action boundaries. The first live run returned seven flags. Reviewing the cases found two model-output problems and five errors in our detector.
What we measured
An offline suite contained 168 cases, including 24 benign controls. It exercised application boundaries and output checks against known inputs. We then ran 84 live model calls across question-answering and evidence assessment. All completed, at about $0.31 of measured usage.
No unauthorized action was observed in the tested paths. That is a scoped result: these paths did not expose an unrestricted agent to arbitrary actions, and the experiment cannot establish resistance to every possible attack.
The detector needed correction too
Two outputs repeated source-embedded authority claims or synthetic test content. These were answer-contamination problems. No customer secret was used in the fixtures, and the observed output did not establish transmission to an outside destination.
The other five flags alleged unsupported figures. Our detector had evaluated them against an incomplete view of the context supplied to the model. The disputed information was available to the model, so those flags were false positives.
Treating all seven as successful attacks would have exaggerated the findings. Treating the absence of actions as proof of clean answers would have missed the two real output problems.
Repair and regression checks
We tightened instruction handling and output validation, then corrected the evaluator's context handling. Scripted tests checked the repaired behavior. Those tests establish application responses to specified outputs; they do not measure how a live model will paraphrase future attacks.
We also replayed the checks against 62 previously recorded judgments. That review helped avoid suppressing legitimate text: the narrower figure check would have replaced two non-alerting outputs, while the corrected check replaced none. The sample is limited and cannot prove that future legitimate answers will never be affected.
The repairs shipped with regression checks. A live rerun after the changes has not been recorded in this study, so their effect on new model outputs remains unmeasured.
This exercise improved both the application and its evaluator. It also kept two conclusions separate: permissions constrain what an assistant can do, while evidence review and answer validation address what it says. Neither result establishes the other.