Engineering evidence
What I measured, what I changed, and what the evidence did not settle.
I'm Eduardo Moreno, TickerTrac's founder and sole engineer. The product checks public evidence against an investor's saved thesis. These studies follow my engineering decisions, including rejected approaches, quality and cost tradeoffs, and remaining limitations.
Evidence quality · Model routing
1,000 cases · 3,600 model comparisons
A production quality investigation separated failures in source coverage, model judgment, citations and delivery. It led to source-specific routing and durable supporting evidence, while a cleaner-looking prompt was rejected for missing useful alerts.
Source-first automated labels and selected cohorts support specific engineering decisions, not a general production accuracy claim.
Retrieval · Answer quality
100 cases · 20 companies
A comparison of dense, hybrid and reranked retrieval improved ranking quality and exposed a cross-corpus coverage miss. The study follows the selected approach through answer review, a targeted repair and production rollback checks.
The frozen, human-adjudicated corpus is bounded; nine answer cases still fail at least one review criterion.
Model selection · Rejected experiment
3,169 retained judgments · 1,200 articles
A fast, inexpensive decision-only model was tested for news screening and relevance judgments. Neither proposed role met the predeclared agreement gates, so the candidate was not adopted.
Agreement with retained judgments is not independent ground truth. The failed gates and measurement limits remain part of the result.