Meta AI enhances machine learning, NLP, and computer vision capabilities.
VerifyPatch
0Popularity
About VerifyPatch
VerifyPatch is an evidence-based testing verification tool designed to analyze patch evidence for pytest coverage. It operates as both a command-line interface (CLI) and a GitHub Action, comparing changes between a base and head commit to assess which lines of code are covered by existing tests, newly tested, or remain uncovered. The tool categorizes executable lines into PR-untouched, PR-touched, unknown, and uncovered, providing a detailed breakdown of test coverage provenance. It also includes mutation testing results and policy checks to evaluate the robustness of the verification process. VerifyPatch aims to reveal the actual evidence supporting a code change rather than relying solely on CI pass/fail status, offering developers and teams deeper insights into the reliability of their test suites. By inspecting patches, challenging changes, and reviewing evidence, the tool helps ensure that patches are thoroughly verified and understood. It is fully open source and integrates seamlessly into existing workflows, enabling users to clone, run, and inspect the verification process directly.
Key features
- Provenance tracking for test evidence
- Mutation testing integration
- Policy compliance checks
- Executable line coverage breakdown
- PR-touched vs PR-untouched analysis
- Uncovered and unknown line identification
- CLI and GitHub Action support
- CI integration for evidence verification
Use cases
- Verifying patch evidence in pull requests
- Assessing test coverage for code changes
- Identifying uncovered or unknown lines in patches
Pros
- Analyzes patch evidence for pytest coverage
- Provides breakdown of executable lines by test coverage status
- Includes mutation testing and policy checks
- Available as CLI and GitHub Action
- Open source
Cons
- No mention of free tier or pricing details
- Limited to pytest coverage evidence
- Requires CLI installation via pipx
Frequently asked questions about VerifyPatch
What does VerifyPatch do?
VerifyPatch analyzes patch evidence for pytest coverage by comparing changes between a base and head commit. It categorizes executable lines into PR-untouched, PR-touched, unknown, and uncovered, while providing mutation testing results and policy checks.
Who is VerifyPatch designed for?
The tool is designed for developers and teams who want to ensure patches are thoroughly verified by tests and understand the provenance of that verification, rather than relying solely on CI pass/fail status.
How does VerifyPatch work?
VerifyPatch operates as a CLI and GitHub Action, inspecting patches to determine which lines are covered by existing tests, newly tested, or remain uncovered. It provides a breakdown of evidence provenance and mutation testing results.
What are the main outputs of VerifyPatch?
The tool outputs a breakdown of executable lines into categories such as PR-untouched, PR-touched, unknown, and uncovered, along with mutation testing results and policy checks.
How can I get started with VerifyPatch?
Users can install VerifyPatch via pipx with the command 'pipx install verifypatch' or run it directly from the GitHub repository. It supports both CLI and GitHub Action usage.
Does VerifyPatch integrate with CI systems?
Yes, VerifyPatch can be used as a GitHub Action, allowing it to integrate seamlessly into CI workflows for automated patch verification.
