evals / python-rules

except-return-empty

Expected

Hidden from the reviewer, like the grading note below.

python-let-errors-crash
violation
python-no-defensive-none-checks
holds
python-typed-models-over-dicts
holds

handle_order now catches every exception from place_order and returns an empty string, so an unknown SKU or OutOfStock becomes an empty successful response. The handler does not end by raising. None checks and typed models are unaffected; flagging them is an invented failure.

Change

Contracts

python-let-errors-crash
---
name: python-let-errors-crash
message: Let Python exceptions propagate; never catch one to continue, log, or return a fallback.
---

## Rules

Apply these rules to Python source files under `packages/`, including tests.
Do not write an exception handler that hides the error: after an `except` clause runs, the handler must end by raising.
A handler that re-raises is allowed, either with a bare `raise` or with `raise OtherError(...) from exc` that converts the exception into a domain or HTTP error.
`try`/`finally` blocks and context managers that release resources are allowed.
These handlers violate the rule: `pass`, `continue`, logging and then carrying on, returning `None`, an empty collection, or a default value, and setting a failure flag.
`contextlib.suppress(...)` hides errors the same way and violates the rule.
Catching `FileNotFoundError`, `KeyError`, or another exception to report that something is missing also violates the rule; check for the missing file or key before the operation instead, such as with `path.exists()` or `key in mapping`.
`pytest.raises` in a test asserts an exception and is allowed.

Allowed, because the handler raises:

```python
try:
    manifest = Manifest.model_validate_json(text)
except ValidationError as exc:
    raise ManifestRejected(path) from exc
```

Violation, because execution continues after the handler:

```python
try:
    shutil.rmtree(scratch_dir)
except OSError:
    logger.info("scratch cleanup failed")
```

## Checks

Search changed Python files for `except` and `suppress(` and confirm each handler ends by raising.
python-no-defensive-none-checks
---
name: python-no-defensive-none-checks
message: Trust non-optional annotations; check for None only where the type includes None.
---

## Rules

Apply these rules to Python source files under `packages/`.
A value annotated with a type that excludes `None`, such as `str`, `int`, `User`, or a model field without `| None`, is never `None`.
The result of calling a function whose return annotation excludes `None` is never `None` either.
Do not guard such a value with `is None`, `is not None`, `== None`, `assert value is not None`, `value or default`, or `getattr(obj, "field", None)` for a declared field.
A truthiness fallback such as `count or 0` on a non-optional `int`, or `items if items is not None else []` on a non-optional `list`, is a defensive check too.
If a value can really be `None`, annotate it `X | None` and handle `None` meaningfully, or fix the caller so it never passes `None`.

A `None` check is allowed when the checked value's type includes `None`:
a parameter, attribute, or field annotated `X | None` or `Optional[X]`, or the result of an API that returns `None` for absence, such as `re.match`, `next(iterator, None)`, or `dict.get`.

Allowed, because `timeout` is annotated `float | None`:

```python
def connect(host: str, timeout: float | None = None) -> Connection:
    if timeout is None:
        timeout = DEFAULT_TIMEOUT
    return Connection(host, timeout)
```

Violation, because `request` is annotated `Request`:

```python
def handle(request: Request) -> Response:
    if request is None:
        return Response.empty()
    return route(request)
```

## Checks

For each `None` check, find the annotation of the checked value and confirm that it includes `None`.
python-typed-models-over-dicts
---
name: python-typed-models-over-dicts
message: Validate structured records into Pydantic models instead of passing raw dicts and reading them with get.
---

## Rules

Apply these rules to Python source files under `packages/`.
A structured record is data with known field names, such as a JSON body, an API response, a YAML or TOML configuration, a webhook payload, or a bag of options.
Validate a structured record into a Pydantic model where it enters the code, then read its fields as attributes.
These violate the rule: a parameter or return value annotated `dict`, `dict[str, Any]`, or `Mapping[str, Any]` that carries a structured record; reading record fields with `.get("field")` or `.get("field", default)`; chained `.get(...)` calls on parsed JSON; and returning a dict literal with fixed keys as a record.

Dicts used as mappings are allowed: a dict whose keys are data and whose values share one type, such as a `dict[str, int]` counter or a `dict[str, Job]` index, including `.get(key, default)` on it.
Dumping a model with `model_dump()` or `json.dumps` at an output boundary is allowed.
Reading environment variables through `os.environ` is outside this contract.

Allowed:

```python
class Lease(BaseModel):
    holder: str
    expires_at: datetime

lease = Lease.model_validate_json(response.content)
return lease.holder
```

Violation:

```python
data = response.json()
holder = data.get("holder")
expires = data.get("expires_at", "")
```

## Checks

Search changed Python files for `.get(`, `dict[str, Any]`, and bare `dict` annotations, and decide whether each carries a structured record or a mapping.