Open-source platform combining product analytics, session replay, A/B testing, and feature flags in one toolkit for product engineers.
Heros
$1.00Starting price
0Popularity
About Heros
Heros ingests a public GitHub repository and determines whether it contains an AI agent. If found, the tool assesses the agent across nine distinct axes, each probing a specific property such as the model called, system prompt, skills bound, tools offered, memory handling, spend limits, loop structure, and execution graph. The evaluation produces a report that lists what was measured, what was not, and why certain measurements could not be taken. Each finding is tied to a specific line of code, enabling direct navigation to the relevant source. The analyzer also provides a ranked list of code spans for each axis, indicating proximity to call sites and whether the file actually invokes a model. Users can ask follow-up questions about the findings or request changes to the agent’s configuration, and Heros will generate diffs or open pull requests accordingly.
Key features
- Model identification and sampling parameter detection
- System prompt and role message verification
- Skill binding and registry checks
- Tool declaration and calling configuration analysis
- Memory persistence and retrieval store inspection
- Spend ceiling and error handling validation
- Loop and iteration ceiling detection
- Execution graph construction and control flow analysis
Use cases
- Identifying weaknesses in an AI agent’s configuration
- Generating evaluation sets for agent performance comparison
- Validating changes between agent versions before deployment
Pros
- Reports findings with direct file and line references
- Assesses agents across nine predefined axes
- Provides ranked evidence for each measurement
- Supports follow-up questions and code modifications
- Works with any public GitHub repository
Cons
- No free tier beyond initial evaluation
- Requires a GitHub repository for analysis
- Limited to public repositories
- No API access described
Frequently asked questions about Heros
What does Heros do when analyzing an AI agent repository?
Heros reads a public GitHub repository to determine if it contains an AI agent, then assesses the agent across nine distinct axes such as model calls, system prompts, skills, tools, memory, spend limits, loop structure, and execution graph.
Who is Heros designed for?
Heros is designed for developers, researchers, or teams working with AI agents who need to evaluate agent behavior, identify weaknesses, or verify configurations in their codebases.
How does Heros provide evidence for its findings?
Every finding in Heros includes a specific line of code with a file path and line number, allowing users to directly navigate to the relevant source for verification.
Can Heros help modify an agent's configuration based on its findings?
Yes, users can ask Heros to generate diffs or open pull requests to implement changes to an agent’s configuration based on the evaluation results.
What happens if Heros cannot measure a specific axis in an agent?
Heros reports what was not measured and explains why certain measurements could not be taken, such as when the required code patterns are absent or the analysis hits its limits.
How does Heros handle potential false positives or false negatives in its analysis?
Heros uses deliberately generous patterns to minimize false negatives, as missing a real issue is costlier than a false positive. Findings are tied to specific code lines to help users distinguish real signals from coincidental matches.
