evals / check-conclusions
failed-request
Expected
Hidden from the reviewer, like the grading note below.
- sample-conclusion-evidence
- violation
The changed repository Err branch returns Verified. A provider failure is now a successful compliance conclusion. Find this regression for both full and relevant diff checks. An all-not-applicable diff can still return Unaffected before looking at the unused repository response; this unchanged short circuit is not the regression.
Change
src/lib.rs
-1+1
41 unmodified lines424344454546474841 unmodified lines } let answer = match repository { Ok(answer) => answer, Err(_) => return Conclusion::Unresolved, Err(_) => return Conclusion::Verified, }; if answer == RepositoryAnswer::NoViolation && files.contains(&FileStatus::Fail) { return Conclusion::Unresolved;Contracts
sample-conclusion-evidence
---
name: sample-conclusion-evidence
message: Require complete, consistent evidence before reporting a contract as verified or preserved.
---
## Rules
Keep execution failure separate from negative evidence. A failed required repository assessment, file error, incomplete check,
or partial selection must remain unresolved. Never report these as verified, preserved, or unaffected.
A no-violation repository answer that contradicts a failed file assessment remains unresolved for the outer reviewer.
Combined repository evidence may resolve uncertain file assessments. A supported repository violation is failed.
Full checks make no assumption about prior compliance. Diff checks assume the base satisfies the contract.
Diff checks with all files not applicable need no repository assessment and are unaffected; a no-violation full check is verified, and a
no-violation diff check is preserved. The outer reviewer owns final approval.
## Checks
Trace errors and disagreements through `conclude`, including an Err repository response and a NoViolation response
with a failed file. Review full and diff conclusions separately. Where Rust is available, run `cargo check` from this
project to check compilation. Otherwise report that limitation; it is separate from the function's output semantics.