Summary
--
While looking at #28887, I noticed a repeated ecosystem check failure
caused by a git error:
> Failed to commit: fatal: failed to encode
'tests/roots/test-root/wrongenc.inc' from iso-8859 to UTF-8
This is caused by a known upstream bug (sphinx-doc/sphinx#14364) with an
open PR (sphinx-doc/sphinx#14467), but the PR is waiting for the next
release milestone, I think, so it seemed easiest just to temporarily
remove the project to unblock CI checks.
Test Plan
--
CI on this PR
## Summary
Enable `truthiness-test-of-none-union` in the standalone scripts' inline
configuration and in the `py-fuzzer` and `ty_benchmark` projects,
following #28889.
Summary
--
This PR updates our ruff-pre-commit dependency to 0.16.5, which supports
the new category selectors,
and then enables `lint.preview` and the new default categories so that
we can dogfood them on our
code. There were 66 diagnostics initially, which I also fixed or
ignored:
```
28 FURB113 [ ] repeated-append
12 ASYNC240 [ ] blocking-path-method-in-async-function
6 RUF105 [*] noqa-comments
5 PLW1510 [ ] subprocess-run-without-check
3 PLW0108 [ ] unnecessary-lambda
2 EXE001 [ ] shebang-not-executable
2 RUF039 [*] unraw-re-pattern
1 B024 [ ] abstract-base-class-without-abstract-method
1 FURB142 [*] for-loop-set-mutations
1 PERF102 [ ] incorrect-dict-iterator
1 PLC0207 [*] missing-maxsplit-arg
1 PLW0127 [ ] self-assigning-variable
1 TRY004 [ ] type-check-without-type-error
1 PLW0120 [*] useless-else-on-loop
1 TRY201 [ ] verbose-raise
```
Of these, I ignored `repeated-append` and `blocking-path-method-in-async-function`. I think they are
still reasonable defaults, just less useful/important in our uses.
0.16.5 also includes https://github.com/astral-sh/ruff-pre-commit/pull/159, so I removed `markdown` from our main `ruff-format` hook, leaving it for the later mdtest formatting stage.
I also had to make a few changes related to config discovery in tests:
- move our cached ecosystem checkouts to `$RUNNER_TEMP` instead of inside of our checkout
- convert some CLI tests to use the `CliTest` helpers
- add `configurationPreference: editorOnly` to some notebook tests
In these cases, our own Ruff settings were being inherited by test code. This likely affects many other tests, but I only updated the obviously failing ones for now. I'm least confident about the notebook tests, but as far as I can tell from the history (https://github.com/astral-sh/ruff/pull/11944, https://github.com/astral-sh/ruff/pull/22810), the specific diagnostics are not the important parts being asserted.
Test Plan
--
CI on this PR
Summary
--
This PR allows adding rules that only belong to categories and only have
names rather than requiring
a linter group and code.
The first non-empty commit uses `Rule::name` instead of
`Rule::noqa_code` throughout the linter tests, causing a huge number of
file renames without changes.
The second commit updates `FixTable` from a mapping of `SecondaryCode ->
(name, fix_count)` to
`DiagnosticId -> (Option<SecondaryCode>, fix_count)`. This doesn't
really change its usage but
involves minor diffs to a substantial number of lines. It also allows us
to drop some of the special
`hashbrown` handling and the `hashbrown` dependency in `ruff_linter`,
which is nice.
The third commit contains the bulk of the change, including the macro
changes needed to support
`map_codes` entries like that for the `pytest-fixture-autouse` rule in
the fourth commit:
```rust
() => rules::ruff::rules::PytestFixtureAutouse,
```
These changes are pretty mechanical overall, just handling cases where
the linter or noqa code could
be missing, which fortunately seemed pretty straightforward in every
case.
The fourth commit takes advantage of these changes to reinstate `RUF076`
as our first purely
`pedantic` rule. I'm happy to drop that if it's too controversial or
opens too many questions about
the previously-removed rule and its code, but I thought it was nice to
have a motivating example. We
moved this rule from `preview` to `removed` because it was too pedantic
for the `RUF` category, even
in preview, so it's exactly the kind of rule we can support now with the
categories.
Finally, the fifth commit updates the contributing docs to reflect that
linters and codes are
optional. This may need to be adjusted if we back out the `RUF076`
change.
Test Plan
--
Existing tests, plus a few new ones for `pytest-fixture-autouse`
Python tooling can currently build third-party source distributions. Set
`no-build` in the maintained Python projects and PEP 723 scripts,
retaining named exceptions for Git-pinned tooling and the local
ecosystem runner. Pin the isolated build dependencies for those
exceptions, including their transitive requirements.
Follows merged astral-sh/ruff#27856, which gives the legacy `ruff-lsp`
suite a dedicated, locked Python 3.12 test environment.
Follow-up to astral-sh/ruff#27843. Replaces astral-sh/ruff#27840, which
closed when the merged docs branch was deleted.
Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
- Does this PR follow our AI policy
(https://github.com/astral-sh/.github/blob/main/AI_POLICY.md)?
-->
## Summary
Addresses all `uv audit` findings.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
NFC.
<!-- How was it tested? -->
---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
- Does this PR follow our AI policy
(https://github.com/astral-sh/.github/blob/main/AI_POLICY.md)?
-->
## Summary
Adds explicit 7d cooldowns to all of our PEP 723 scripts, as well as
similar cooldowns via `.npmrc`.
<!-- What's the purpose of the change? What does it do, and why? -->
## Test Plan
NFC.
<!-- How was it tested? -->
---
<sub>Stack created with <a
href="https://github.com/github/gh-stack">GitHub Stacks CLI</a> • <a
href="https://gh.io/stacks-feedback">Give Feedback 💬</a></sub>
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
- Does this PR follow our AI policy
(https://github.com/astral-sh/.github/blob/main/AI_POLICY.md)?
-->
## Summary
As a pywin32 maintainer and Ruff user, I'd like to add pywin32 to the
ecosystem checks.
I think that pywin32 would be a good fit given its age, size,
popularity, and importance to the Python on Windows ecosystem. As well
as my efforts in the past 3 years modernizing its Python codebase.
Most of the Ruff issues / requests I opened have come from using it in
setuptools and pywin32.
## Test Plan
N/A ?
## Summary
Reintroducing `demisto/content` back to `ruff-ecosystem`, it's a huge
repo with 4.5K py file and 2.5M loc, so may have catch some cases for
ecosystem report. It was previously commented out in
https://github.com/astral-sh/ruff/pull/12129 due to the use of removed
`E999`
Issue with the use of removed rules upstream is resolved, though
apparently `E999` was dropped from selection a while ago and new issue
with the use of removed `UP038` occurred, but it's resolved now too -
https://github.com/demisto/content/pull/45227
Syntax error in still present in
https://github.com/demisto/content/blob/master/Packs/ThreatQ/Integrations/ThreatQ/ThreatQ.py,
so keeping `exclude`. Though apparently it doesn't break
`ruff-ecosystem` anymore, but keeping it safe for now. I've submitted a
fix upstream https://github.com/demisto/content/pull/45275, so `exclude`
can be removed later too.
## Test Plan
Tested running `ruff-ecosystem` locally, no issues found.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
- Does this PR follow our AI policy
(https://github.com/astral-sh/.github/blob/main/AI_POLICY.md)?
-->
## Summary
Previously disabled in https://github.com/astral-sh/ruff/pull/9854
Not sure which preview rule was the blocker exactly, but apparently it's
now stabilized and we can use `sphinx` in `ruff-ecosystem`.
## Test Plan
I ran `ruff-ecosystem` locally with and without `--force-preview` and it
worked without issues.
<!--
Thank you for contributing to Ruff/ty! To help us out with reviewing,
please consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title? (Please prefix
with `[ty]` for ty pull
requests.)
- Does this pull request include references to any relevant issues?
- Does this PR follow our AI policy
(https://github.com/astral-sh/.github/blob/main/AI_POLICY.md)?
-->
## Summary
This reverts commit dd16d689ab (#23921),
since issue upstream is fixed in
https://github.com/apache/airflow/pull/64920 and workaround is no longer
needed.
https://patch-diff.githubusercontent.com/raw/apache/airflow/pull/64920.diff
```diff
diff --git a/task-sdk/src/airflow/sdk/_shared/AGENTS.md b/task-sdk/src/airflow/sdk/_shared/AGENTS.md
index 8d2c7cf9d3367..57a3a75046e45 120000
--- a/task-sdk/src/airflow/sdk/_shared/AGENTS.md
+++ b/task-sdk/src/airflow/sdk/_shared/AGENTS.md
@@ -1 +1 @@
-../../../../../airflow-core/src/airflow/_shared/AGENTS..md
\ No newline at end of file
+../../../../../airflow-core/src/airflow/_shared/AGENTS.md
\ No newline at end of file
```
So workaround is no longer needed.
## Test Plan
`ruff-ecosystem` runs without issues.
Summary
--
Now that we allow rule names in selectors in preview, we need to
normalize them back to codes to
allow running on stable. This PR implements this normalization as one of
our config overrides,
using the comparison executable to extract the rule mapping from `ruff
rule`.
Test Plan
--
CI on this PR and then #27154
Summary
--
This is the first in a series of migration rules to help users move from
`noqa` comments with rule
codes to `ruff:ignore` comments with names. This rule simply replaces
`noqa: codes` with
`ruff:ignore[codes]` (and the file-level variants), deferring the
transformation of codes to names
to a follow-up rule. This rule may be useful on its own without the
codes -> names rule, assuming we
stabilize `ruff:ignore` in the next minor release without stabilizing
human-readable names.
The rule skips any comment consisting entirely of known `external`
selectors, while still emitting diagnostics for totally unknown codes.
We also skip file-level `flake8: noqa` directives assuming they are
intended to be shared with flake8.
I initially wrote a small, standalone implementation in the rule file
itself, but it had to be
combined with the `RUF100` implementation to leave unused `noqa` codes
out of `RUF105` to be cleaned
up by `RUF100` instead, as I note in the mdtest for this case.
Test Plan
--
New mdtests
I also expect a huge number of ecosystem results on this PR, possibly so
many that the check times out.
---------
Co-authored-by: Micha Reiser <micha@reiser.io>
## Summary
Follow up to #26050 to clean up the ecosystem check script now that
diagnostics without severity are the baseline again. This reverts both
the temporary normalization code from that PR and the earlier regex
change from https://github.com/astral-sh/ruff/pull/23845.
## Test Plan
CI on this PR
Summary
--
This reverts #23845 since we're now prioritizing the human-readable
names and category work before getting back to severity.
I had to revert (the revert of) the changes to the ecosystem check
script and then add another normalization pass again to get CI to pass.
I can follow up immediately to revert these changes once this is the
baseline.
Test Plan
--
Existing tests updated following #25937
I said I didn't love diagnostics like this:
```
fib.py:1:8: unused-import: [*] `os` imported but unused
```
with both a human-readable name and a fix indicator on that PR, but I
think it's not actually that
bad, and I haven't thought of a better alternative, so I think it's fine
for now.
Summary
--
Reverts the name normalization part of #25937 now that human-readable
names are also present in the
baseline for future ecosystem check runs.
Test Plan
--
CI on this PR
## Summary
This PR updates all of our CLI output formats to use human-readable
names instead of rule codes in preview.
## Test Plan
The first commit adds a new CLI test and then each following commit
updates one output format and the associated snapshots.
The last commit also updates the ecosystem check to normalize rule names
back to codes to avoid a huge diff that causes the check to time out.
Alternatively, we may just want to accept the time out on this PR so
that we can see the un-normalized report in the future.
Summary
--
I noticed while extending the regex to match human-readable names that
it also didn't cover the
ASYNC rules, which have five letters in the rule code. For example,
#24644 revealed no new
diagnostics in the ecosystem, but a local run on my cached ecosystem
repos revealed many
diagnostics. This won't show up as a diff in CI on this PR but should
help with ecosystem checks on
the ASYNC rules in the future.
Close and reopen this PR to trigger CI
---------
Co-authored-by: typeshedbot <>
Co-authored-by: Micha Reiser <micha@reiser.io>
Co-authored-by: Alex Waygood <alex.waygood@gmail.com>
This is just the "aesthetic" part of implementing `warning` severities.
Configuration changes are in
https://github.com/astral-sh/ruff/pull/23846
It's expected that the `preview` ecosystem check times out here.
Summary
--
The huge number of changes in
https://github.com/astral-sh/ruff/pull/22205#issuecomment-3696660639
should have
obviously been a red flag, but I think it would be nice if CI failed
when new
ecosystem panics were introduced. This PR adds a check for diagnostic
lines that
start with `panic: Panicked at crates/`, raises a `ToolError` if any are
found
in the results from the comparison executable, and then ~~also exits
non-zero if
any errors are returned~~ fails the CI run if the corresponding error
message
was printed.
After trying this out in CI, I opted not to change the script's exit
code itself
because that suppressed the ecosystem comment. It feels a little hackier
this way but preserves the behavior I wanted of both failing CI and
still getting
the ecosystem comment to help with debugging.
Test Plan
--
Local testing on the 0.15.3 tag showing that ruff-ecosystem exited
non-zero and
some manual testing in CI, as you can see below.
Summary
--
I think Micha suggested this before, but I couldn't find the message.
This also
may have helped to catch #23125.
Test Plan
--
CI on this PR. I think this is a pretty large repo, so I want to make
sure it
doesn't cause too much of a slowdown, in addition to the job passing.
However,
it only takes ~4 seconds to check locally compared to ~22 for airflow,
despite
containing more files (16k vs 6.5k) and more lines of code (3 million vs
1.2
million).
Summary
--
I was firing up the fuzzer tonight and hit an assertion error here. We
now build with the `profiling` profile, so we need to use that
executable too.
This hasn't affected CI because we always set the `--test-executable`.
Test Plan
--
Ran the script again with the same arguments on this branch
## Summary
Infer a type of `Self` for unannotated `self` parameters in methods of
classes.
part of https://github.com/astral-sh/ty/issues/159
closes https://github.com/astral-sh/ty/issues/1081
## Conformance tests changes
```diff
+enums_member_values.py:85:9: error[invalid-assignment] Object of type `int` is not assignable to attribute `_value_` of type `str`
```
A true positive ✔️
```diff
-generics_self_advanced.py:35:9: error[type-assertion-failure] Argument does not have asserted type `Self@method2`
-generics_self_basic.py:14:9: error[type-assertion-failure] Argument does not have asserted type `Self@set_scale
```
Two false positives going away ✔️
```diff
+generics_syntax_infer_variance.py:82:9: error[invalid-assignment] Cannot assign to final attribute `x` on type `Self@__init__`
```
This looks like a true positive to me, even if it's not marked with `#
E` ✔️
```diff
+protocols_explicit.py:56:9: error[invalid-assignment] Object of type `tuple[int, int, str]` is not assignable to attribute `rgb` of type `tuple[int, int, int]`
```
True positive ✔️
```
+protocols_explicit.py:85:9: error[invalid-attribute-access] Cannot assign to ClassVar `cm1` from an instance of type `Self@__init__`
```
This looks like a true positive to me, even if it's not marked with `#
E`. But this is consistent with our understanding of `ClassVar`, I
think. ✔️
```py
+qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__`
+qualifiers_final_annotation.py:65:9: error[invalid-assignment] Cannot assign to final attribute `ID7` on type `Self@method1`
```
New true positives ✔️
```py
+qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__`
+qualifiers_final_annotation.py:57:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__`
+qualifiers_final_annotation.py:59:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__`
```
This is a new false positive, but that's a pre-existing issue on main
(if you annotate with `Self`):
https://play.ty.dev/3ee1c56d-7e13-43bb-811a-7a81e236e6ab❌ => reported
as https://github.com/astral-sh/ty/issues/1409
## Ecosystem
* There are 5931 new `unresolved-attribute` and 3292 new
`possibly-missing-attribute` attribute errors, way too many to look at
all of them. I randomly sampled 15 of these errors and found:
* 13 instances where there was simply no such attribute that we could
plausibly see. Sometimes [I didn't find it
anywhere](https://github.com/internetarchive/openlibrary/blob/8644d886c6579a5f49faadc4cd1ba9992e603d7e/openlibrary/plugins/openlibrary/tests/test_listapi.py#L33).
Sometimes it was set externally on the object. Sometimes there was some
[`setattr` dynamicness going
on](https://github.com/pypa/setuptools/blob/a49f6b927d83b97630b4fb030de8035ed32436fd/setuptools/wheel.py#L88-L94).
I would consider all of them to be true positives.
* 1 instance where [attribute was set on `obj` in
`__new__`](https://github.com/sympy/sympy/blob/9e87b44fd43572b9c4cc95ec569a2f4b81d56499/sympy/tensor/array/array_comprehension.py#L45C1-L45C36),
which we don't support yet
* 1 instance [where the attribute was defined via `__slots__`
](https://github.com/spack/spack/blob/e250ec0fc81130b708a8abe1894f0cc926880210/lib/spack/spack/vendor/pyrsistent/_pdeque.py#L48C5-L48C14)
* I see 44 instances [of the false positive
above](https://github.com/astral-sh/ty/issues/1409) with `Final`
instance attributes being set in `__init__`. I don't think this should
block this PR.
## Test Plan
New Markdown tests.
---------
Co-authored-by: Shaygan Hooshyari <sh.hooshyari@gmail.com>
## Summary
This PR Removes deprecated UP038 as per instructed in #18727closes#18727
## Test Plan
I have run tests non of them failing
One Question i have is do we have to document that UP038 is removed?
---------
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com>
## Summary
Fuzzer seed 208 seems to be timing out all fuzzer runs on PRs today.
This has happened on multiple unrelated PRs, as well as on an initial
version of this PR that made a comment-only change in ty and didn't skip
any seeds, so the timeout appears to be consistent in CI, on ty main
branch, as of today, but it started happening due to some change in a
factor outside ty; not sure what.
I checked the code generated for seed 208 locally, and it takes about
30s to check on current ty main branch. This is slow for a fuzzer seed,
but shouldn't be slow enough to make it time out after 20min in CI (even
accounting for GH runners being slower than my laptop.)
I tried to bisect the slowness of checking that code locally, but I
didn't go back far enough to find the change that made it slow. In fact
it seems like it became significantly faster in the last few days (on an
older checkout I had to stop it after several minutes.) So whatever the
cause of the slowness, it's not a recent change in ty.
I don't want to rabbit-hole on this right now (fuzzer-discovered issues
are lower-priority than real-world-code issues), and need a working CI,
so skip this seed for now until we can investigate it.
## Test Plan
CI. This PR contains a no-op (comment) change in ty, so that the fuzz
test is triggered in CI and we can verify it now works (as well as
verify, on the previous commit, that the fuzzer job is timing out on
that seed, even with just a no-op change in ty.)
Alex Waygood