Skip to content

Commit 589e8ac

Browse files
sharkdpGlyphack
andauthored
[ty] Infer type for implicit self parameters in method bodies (#20922)
## Summary Infer a type of `Self` for unannotated `self` parameters in methods of classes. part of astral-sh/ty#159 closes astral-sh/ty#1081 ## Conformance tests changes ```diff +enums_member_values.py:85:9: error[invalid-assignment] Object of type `int` is not assignable to attribute `_value_` of type `str` ``` A true positive :heavy_check_mark: ```diff -generics_self_advanced.py:35:9: error[type-assertion-failure] Argument does not have asserted type `Self@method2` -generics_self_basic.py:14:9: error[type-assertion-failure] Argument does not have asserted type `Self@set_scale ``` Two false positives going away :heavy_check_mark: ```diff +generics_syntax_infer_variance.py:82:9: error[invalid-assignment] Cannot assign to final attribute `x` on type `Self@__init__` ``` This looks like a true positive to me, even if it's not marked with `# E` :heavy_check_mark: ```diff +protocols_explicit.py:56:9: error[invalid-assignment] Object of type `tuple[int, int, str]` is not assignable to attribute `rgb` of type `tuple[int, int, int]` ``` True positive :heavy_check_mark: ``` +protocols_explicit.py:85:9: error[invalid-attribute-access] Cannot assign to ClassVar `cm1` from an instance of type `Self@__init__` ``` This looks like a true positive to me, even if it's not marked with `# E`. But this is consistent with our understanding of `ClassVar`, I think. :heavy_check_mark: ```py +qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__` +qualifiers_final_annotation.py:65:9: error[invalid-assignment] Cannot assign to final attribute `ID7` on type `Self@method1` ``` New true positives :heavy_check_mark: ```py +qualifiers_final_annotation.py:52:9: error[invalid-assignment] Cannot assign to final attribute `ID4` on type `Self@__init__` +qualifiers_final_annotation.py:57:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__` +qualifiers_final_annotation.py:59:13: error[invalid-assignment] Cannot assign to final attribute `ID6` on type `Self@__init__` ``` This is a new false positive, but that's a pre-existing issue on main (if you annotate with `Self`): https://play.ty.dev/3ee1c56d-7e13-43bb-811a-7a81e236e6ab ❌ => reported as astral-sh/ty#1409 ## Ecosystem * There are 5931 new `unresolved-attribute` and 3292 new `possibly-missing-attribute` attribute errors, way too many to look at all of them. I randomly sampled 15 of these errors and found: * 13 instances where there was simply no such attribute that we could plausibly see. Sometimes [I didn't find it anywhere](https://github.com/internetarchive/openlibrary/blob/8644d886c6579a5f49faadc4cd1ba9992e603d7e/openlibrary/plugins/openlibrary/tests/test_listapi.py#L33). Sometimes it was set externally on the object. Sometimes there was some [`setattr` dynamicness going on](https://github.com/pypa/setuptools/blob/a49f6b927d83b97630b4fb030de8035ed32436fd/setuptools/wheel.py#L88-L94). I would consider all of them to be true positives. * 1 instance where [attribute was set on `obj` in `__new__`](https://github.com/sympy/sympy/blob/9e87b44fd43572b9c4cc95ec569a2f4b81d56499/sympy/tensor/array/array_comprehension.py#L45C1-L45C36), which we don't support yet * 1 instance [where the attribute was defined via `__slots__` ](https://github.com/spack/spack/blob/e250ec0fc81130b708a8abe1894f0cc926880210/lib/spack/spack/vendor/pyrsistent/_pdeque.py#L48C5-L48C14) * I see 44 instances [of the false positive above](astral-sh/ty#1409) with `Final` instance attributes being set in `__init__`. I don't think this should block this PR. ## Test Plan New Markdown tests. --------- Co-authored-by: Shaygan Hooshyari <[email protected]>
1 parent 76a5531 commit 589e8ac

File tree

16 files changed

+325
-210
lines changed

16 files changed

+325
-210
lines changed

crates/ruff_benchmark/benches/ty.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -667,7 +667,7 @@ fn attrs(criterion: &mut Criterion) {
667667
max_dep_date: "2025-06-17",
668668
python_version: PythonVersion::PY313,
669669
},
670-
100,
670+
110,
671671
);
672672

673673
bench_project(&benchmark, criterion);
@@ -684,7 +684,7 @@ fn anyio(criterion: &mut Criterion) {
684684
max_dep_date: "2025-06-17",
685685
python_version: PythonVersion::PY313,
686686
},
687-
100,
687+
150,
688688
);
689689

690690
bench_project(&benchmark, criterion);

crates/ruff_benchmark/benches/ty_walltime.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -210,7 +210,7 @@ static TANJUN: Benchmark = Benchmark::new(
210210
max_dep_date: "2025-06-17",
211211
python_version: PythonVersion::PY312,
212212
},
213-
100,
213+
320,
214214
);
215215

216216
static STATIC_FRAME: Benchmark = Benchmark::new(
@@ -226,7 +226,7 @@ static STATIC_FRAME: Benchmark = Benchmark::new(
226226
max_dep_date: "2025-08-09",
227227
python_version: PythonVersion::PY311,
228228
},
229-
630,
229+
750,
230230
);
231231

232232
#[track_caller]

crates/ty_ide/src/completion.rs

Lines changed: 28 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1957,14 +1957,34 @@ class Quux:
19571957
",
19581958
);
19591959

1960-
// FIXME: This should list completions on `self`, which should
1961-
// include, at least, `foo` and `bar`. At time of writing
1962-
// (2025-06-04), the type of `self` is inferred as `Unknown` in
1963-
// this context. This in turn prevents us from getting a list
1964-
// of available attributes.
1965-
//
1966-
// See: https://github.com/astral-sh/ty/issues/159
1967-
assert_snapshot!(test.completions_without_builtins(), @"<No completions found>");
1960+
assert_snapshot!(test.completions_without_builtins(), @r"
1961+
bar
1962+
baz
1963+
foo
1964+
__annotations__
1965+
__class__
1966+
__delattr__
1967+
__dict__
1968+
__dir__
1969+
__doc__
1970+
__eq__
1971+
__format__
1972+
__getattribute__
1973+
__getstate__
1974+
__hash__
1975+
__init__
1976+
__init_subclass__
1977+
__module__
1978+
__ne__
1979+
__new__
1980+
__reduce__
1981+
__reduce_ex__
1982+
__repr__
1983+
__setattr__
1984+
__sizeof__
1985+
__str__
1986+
__subclasshook__
1987+
");
19681988
}
19691989

19701990
#[test]

crates/ty_ide/src/semantic_tokens.rs

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1798,23 +1798,23 @@ class BoundedContainer[T: int, U = str]:
17981798
"T" @ 554..555: TypeParameter
17991799
"value2" @ 557..563: Parameter
18001800
"U" @ 565..566: TypeParameter
1801-
"self" @ 577..581: Variable
1801+
"self" @ 577..581: TypeParameter
18021802
"value1" @ 582..588: Variable
18031803
"T" @ 590..591: TypeParameter
18041804
"value1" @ 594..600: Parameter
1805-
"self" @ 609..613: Variable
1805+
"self" @ 609..613: TypeParameter
18061806
"value2" @ 614..620: Variable
18071807
"U" @ 622..623: TypeParameter
18081808
"value2" @ 626..632: Parameter
18091809
"get_first" @ 642..651: Method [definition]
18101810
"self" @ 652..656: SelfParameter
18111811
"T" @ 661..662: TypeParameter
1812-
"self" @ 679..683: Variable
1812+
"self" @ 679..683: TypeParameter
18131813
"value1" @ 684..690: Variable
18141814
"get_second" @ 700..710: Method [definition]
18151815
"self" @ 711..715: SelfParameter
18161816
"U" @ 720..721: TypeParameter
1817-
"self" @ 738..742: Variable
1817+
"self" @ 738..742: TypeParameter
18181818
"value2" @ 743..749: Variable
18191819
"BoundedContainer" @ 798..814: Class [definition]
18201820
"T" @ 815..816: TypeParameter [definition]

crates/ty_python_semantic/resources/mdtest/annotations/self.md

Lines changed: 44 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -56,24 +56,38 @@ In instance methods, the first parameter (regardless of its name) is assumed to
5656

5757
```toml
5858
[environment]
59-
python-version = "3.11"
59+
python-version = "3.12"
6060
```
6161

6262
```py
6363
from typing import Self
6464

6565
class A:
6666
def implicit_self(self) -> Self:
67-
# TODO: This should be Self@implicit_self
68-
reveal_type(self) # revealed: Unknown
67+
reveal_type(self) # revealed: Self@implicit_self
6968

7069
return self
7170

72-
def a_method(self) -> int:
73-
def first_arg_is_not_self(a: int) -> int:
71+
def implicit_self_generic[T](self, x: T) -> T:
72+
reveal_type(self) # revealed: Self@implicit_self_generic
73+
74+
return x
75+
76+
def method_a(self) -> None:
77+
def first_param_is_not_self(a: int):
7478
reveal_type(a) # revealed: int
75-
return a
76-
return first_arg_is_not_self(1)
79+
reveal_type(self) # revealed: Self@method_a
80+
81+
def first_param_is_not_self_unannotated(a):
82+
reveal_type(a) # revealed: Unknown
83+
reveal_type(self) # revealed: Self@method_a
84+
85+
def first_param_is_also_not_self(self) -> None:
86+
reveal_type(self) # revealed: Unknown
87+
88+
def first_param_is_explicit_self(this: Self) -> None:
89+
reveal_type(this) # revealed: Self@method_a
90+
reveal_type(self) # revealed: Self@method_a
7791

7892
@classmethod
7993
def a_classmethod(cls) -> Self:
@@ -127,19 +141,16 @@ The name `self` is not special in any way.
127141
```py
128142
class B:
129143
def name_does_not_matter(this) -> Self:
130-
# TODO: Should reveal Self@name_does_not_matter
131-
reveal_type(this) # revealed: Unknown
144+
reveal_type(this) # revealed: Self@name_does_not_matter
132145

133146
return this
134147

135148
def positional_only(self, /, x: int) -> Self:
136-
# TODO: Should reveal Self@positional_only
137-
reveal_type(self) # revealed: Unknown
149+
reveal_type(self) # revealed: Self@positional_only
138150
return self
139151

140152
def keyword_only(self, *, x: int) -> Self:
141-
# TODO: Should reveal Self@keyword_only
142-
reveal_type(self) # revealed: Unknown
153+
reveal_type(self) # revealed: Self@keyword_only
143154
return self
144155

145156
@property
@@ -165,8 +176,7 @@ T = TypeVar("T")
165176

166177
class G(Generic[T]):
167178
def id(self) -> Self:
168-
# TODO: Should reveal Self@id
169-
reveal_type(self) # revealed: Unknown
179+
reveal_type(self) # revealed: Self@id
170180

171181
return self
172182

@@ -252,6 +262,20 @@ class LinkedList:
252262
reveal_type(LinkedList().next()) # revealed: LinkedList
253263
```
254264

265+
Attributes can also refer to a generic parameter:
266+
267+
```py
268+
from typing import Generic, TypeVar
269+
270+
T = TypeVar("T")
271+
272+
class C(Generic[T]):
273+
foo: T
274+
def method(self) -> None:
275+
reveal_type(self) # revealed: Self@method
276+
reveal_type(self.foo) # revealed: T@C
277+
```
278+
255279
## Generic Classes
256280

257281
```py
@@ -342,31 +366,28 @@ b: Self
342366

343367
# TODO: "Self" cannot be used in a function with a `self` or `cls` parameter that has a type annotation other than "Self"
344368
class Foo:
345-
# TODO: rejected Self because self has a different type
369+
# TODO: This `self: T` annotation should be rejected because `T` is not `Self`
346370
def has_existing_self_annotation(self: T) -> Self:
347371
return self # error: [invalid-return-type]
348372

349373
def return_concrete_type(self) -> Self:
350-
# TODO: tell user to use "Foo" instead of "Self"
374+
# TODO: We could emit a hint that suggests annotating with `Foo` instead of `Self`
351375
# error: [invalid-return-type]
352376
return Foo()
353377

354378
@staticmethod
355-
# TODO: reject because of staticmethod
379+
# TODO: The usage of `Self` here should be rejected because this is a static method
356380
def make() -> Self:
357381
# error: [invalid-return-type]
358382
return Foo()
359383

360-
class Bar(Generic[T]):
361-
foo: T
362-
def bar(self) -> T:
363-
return self.foo
384+
class Bar(Generic[T]): ...
364385

365386
# error: [invalid-type-form]
366387
class Baz(Bar[Self]): ...
367388

368389
class MyMetaclass(type):
369-
# TODO: rejected
390+
# TODO: reject the Self usage. because self cannot be used within a metaclass.
370391
def __new__(cls) -> Self:
371392
return super().__new__(cls)
372393
```

crates/ty_python_semantic/resources/mdtest/attributes.md

Lines changed: 18 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,7 @@ class C:
2626
c_instance = C(1)
2727

2828
reveal_type(c_instance.inferred_from_value) # revealed: Unknown | Literal[1, "a"]
29-
30-
# TODO: Same here. This should be `Unknown | Literal[1, "a"]`
31-
reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown
29+
reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown | Literal[1, "a"]
3230

3331
# There is no special handling of attributes that are (directly) assigned to a declared parameter,
3432
# which means we union with `Unknown` here, since the attribute itself is not declared. This is
@@ -177,8 +175,7 @@ c_instance = C(1)
177175

178176
reveal_type(c_instance.inferred_from_value) # revealed: Unknown | Literal[1, "a"]
179177

180-
# TODO: Should be `Unknown | Literal[1, "a"]`
181-
reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown
178+
reveal_type(c_instance.inferred_from_other_attribute) # revealed: Unknown | Literal[1, "a"]
182179

183180
reveal_type(c_instance.inferred_from_param) # revealed: Unknown | int | None
184181

@@ -399,9 +396,19 @@ class TupleIterable:
399396

400397
class C:
401398
def __init__(self) -> None:
399+
# TODO: Should not emit this diagnostic
400+
# error: [unresolved-attribute]
402401
[... for self.a in IntIterable()]
402+
# TODO: Should not emit this diagnostic
403+
# error: [unresolved-attribute]
404+
# error: [unresolved-attribute]
403405
[... for (self.b, self.c) in TupleIterable()]
406+
# TODO: Should not emit this diagnostic
407+
# error: [unresolved-attribute]
408+
# error: [unresolved-attribute]
404409
[... for self.d in IntIterable() for self.e in IntIterable()]
410+
# TODO: Should not emit this diagnostic
411+
# error: [unresolved-attribute]
405412
[[... for self.f in IntIterable()] for _ in IntIterable()]
406413
[[... for self.g in IntIterable()] for self in [D()]]
407414

@@ -598,6 +605,8 @@ class C:
598605
self.c = c
599606
if False:
600607
def set_e(self, e: str) -> None:
608+
# TODO: Should not emit this diagnostic
609+
# error: [unresolved-attribute]
601610
self.e = e
602611

603612
# TODO: this would ideally be `Unknown | Literal[1]`
@@ -685,7 +694,7 @@ class C:
685694
pure_class_variable2: ClassVar = 1
686695

687696
def method(self):
688-
# TODO: this should be an error
697+
# error: [invalid-attribute-access] "Cannot assign to ClassVar `pure_class_variable1` from an instance of type `Self@method`"
689698
self.pure_class_variable1 = "value set through instance"
690699

691700
reveal_type(C.pure_class_variable1) # revealed: str
@@ -885,11 +894,9 @@ class Intermediate(Base):
885894
# TODO: This should be an error (violates Liskov)
886895
self.redeclared_in_method_with_wider_type: object = object()
887896

888-
# TODO: This should be an `invalid-assignment` error
889-
self.overwritten_in_subclass_method = None
897+
self.overwritten_in_subclass_method = None # error: [invalid-assignment]
890898

891-
# TODO: This should be an `invalid-assignment` error
892-
self.pure_overwritten_in_subclass_method = None
899+
self.pure_overwritten_in_subclass_method = None # error: [invalid-assignment]
893900

894901
self.pure_undeclared = "intermediate"
895902

@@ -1839,6 +1846,7 @@ def external_getattribute(name) -> int:
18391846

18401847
class ThisFails:
18411848
def __init__(self):
1849+
# error: [invalid-assignment] "Implicit shadowing of function `__getattribute__`"
18421850
self.__getattribute__ = external_getattribute
18431851

18441852
# error: [unresolved-attribute]

crates/ty_python_semantic/resources/mdtest/call/dunder.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -205,7 +205,7 @@ class C:
205205
return str(key)
206206

207207
def f(self):
208-
# TODO: This should emit an `invalid-assignment` diagnostic once we understand the type of `self`
208+
# error: [invalid-assignment] "Implicit shadowing of function `__getitem__`"
209209
self.__getitem__ = None
210210

211211
# This is still fine, and simply calls the `__getitem__` method on the class

crates/ty_python_semantic/resources/mdtest/class/super.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -163,14 +163,13 @@ class A:
163163

164164
class B(A):
165165
def __init__(self, a: int):
166-
# TODO: Once `Self` is supported, this should be `<super: <class 'B'>, B>`
167-
reveal_type(super()) # revealed: <super: <class 'B'>, Unknown>
166+
reveal_type(super()) # revealed: <super: <class 'B'>, B>
168167
reveal_type(super(object, super())) # revealed: <super: <class 'object'>, super>
169168
super().__init__(a)
170169

171170
@classmethod
172171
def f(cls):
173-
# TODO: Once `Self` is supported, this should be `<super: <class 'B'>, <class 'B'>>`
172+
# TODO: Once `cls` is supported, this should be `<super: <class 'B'>, <class 'B'>>`
174173
reveal_type(super()) # revealed: <super: <class 'B'>, Unknown>
175174
super().f()
176175

@@ -358,15 +357,15 @@ from __future__ import annotations
358357

359358
class A:
360359
def test(self):
361-
reveal_type(super()) # revealed: <super: <class 'A'>, Unknown>
360+
reveal_type(super()) # revealed: <super: <class 'A'>, A>
362361

363362
class B:
364363
def test(self):
365-
reveal_type(super()) # revealed: <super: <class 'B'>, Unknown>
364+
reveal_type(super()) # revealed: <super: <class 'B'>, B>
366365

367366
class C(A.B):
368367
def test(self):
369-
reveal_type(super()) # revealed: <super: <class 'C'>, Unknown>
368+
reveal_type(super()) # revealed: <super: <class 'C'>, C>
370369

371370
def inner(t: C):
372371
reveal_type(super()) # revealed: <super: <class 'B'>, C>
@@ -616,7 +615,7 @@ class A:
616615
class B(A):
617616
def __init__(self, a: int):
618617
super().__init__(a)
619-
# TODO: Once `Self` is supported, this should raise `unresolved-attribute` error
618+
# error: [unresolved-attribute] "Object of type `<super: <class 'B'>, B>` has no attribute `a`"
620619
super().a
621620

622621
# error: [unresolved-attribute] "Object of type `<super: <class 'B'>, B>` has no attribute `a`"

crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,7 @@ def f1(flag: bool):
170170
attr = DataDescriptor()
171171

172172
def f(self):
173+
# error: [invalid-assignment] "Invalid assignment to data descriptor attribute `attr` on type `Self@f` with custom `__set__` method"
173174
self.attr = "normal"
174175

175176
reveal_type(C1().attr) # revealed: Unknown | Literal["data", "normal"]

crates/ty_python_semantic/resources/mdtest/named_tuple.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -208,8 +208,7 @@ class SuperUser(User):
208208
def now_called_robert(self):
209209
self.name = "Robert" # fine because overridden with a mutable attribute
210210

211-
# TODO: this should cause us to emit an error as we're assigning to a read-only property
212-
# inherited from the `NamedTuple` superclass (requires https://github.com/astral-sh/ty/issues/159)
211+
# error: 9 [invalid-assignment] "Cannot assign to read-only property `nickname` on object of type `Self@now_called_robert`"
213212
self.nickname = "Bob"
214213

215214
james = SuperUser(0, "James", 42, "Jimmy")

0 commit comments

Comments
 (0)