Annotation of args and kwargs in Python

Annotation of args and kwargs in Python

When I try to do without *args and **kwargs in function signatures, this cannot always be done without compromising the usability of the API. Especially when you need to write functions that refer to auxiliary functions with the same signatures.

Typification *args and **kwargs always frustrated me because they couldn’t be provided with precise instructions in advance. For example, if both positional and named arguments to a function could only contain values ​​of the same type, you could do something like this:

def foo(*args: int, **kwargs: bool) -> None:

The use of such a construction indicates that args is a tuple whose elements are all integers, a kwargs – This is a dictionary whose keys are strings and values ​​are of boolean type.

But it was not possible to adequately annotate *args and **kwargs in a situation where the values ​​that can be passed as positional and named arguments may, in different circumstances, refer to different types. In such cases, it was necessary to resort to Anywhich was against the purpose of typing function arguments.

Take a look at the following example:

def foo(*args: tuple[int, str], **kwargs: dict[str, bool | None]) -> None:

Here, the type checking system accepts each of the positional arguments as a tuple of an integer and a string. In addition, it treats each named argument as a dictionary whose keys are strings and whose values ​​are either Boolean entities or objects. None.

When using the instructions above, mypy will not skip the following code:

foo(*(1, "hello"), **{"key1": 1, "key2": False})
error: Argument 1 to "foo" has incompatible type "*tuple[int, str]";
expected "tuple[int, str]"  [arg-type]

error: Argument 2 to "foo" has incompatible type "**dict[str, int]";
expected "dict[str, bool | None]"  [arg-type]

But this will be recognized as normal:

foo((1, "hello"), kw1={"key1": 1, "key2": False})

A programmer would probably want to use the first version of the code, while the type checking system would need the second version.

In order to correctly annotate the second example, you need to use the tools from PEP-589, PEP-646, PEP-655 and PEP-692. Namely, we will use it Unpack and TypedDict from the module typing. Here’s how to do it:

from typing import TypedDict, Unpack  # Python 3.12+

# from typing_extensions import TypedDict, Unpack # < Python 3.12

class Kw(TypedDict):
    key1: int
    key2: bool

def foo(*args: Unpack[tuple[int, str]], **kwargs: Unpack[Kw]) -> None:

args = (1, "hello")
kwargs: Kw = {"key1": 1, "key2": False}

foo(*args, **kwargs)  # Ok

Type TypedDict appeared in Python 3.8. It allows you to annotate dictionaries that support values ​​of different types. If all dictionary values ​​have the same type, you can simply use the construct to annotate it dict[str, T]. And here is the type TypedDict oriented to situations where all dictionary keys are strings and values ​​can have different types.

The following example shows how a dictionary containing values ​​of different types can be annotated:

from typing import TypedDict

class Movie(TypedDict):
    name: str
    year: int

movies: Movie = {"name": "Mad Max", "year": 2015}

Using the operator Unpack objects can be shown to be unpacked.

Using TypedDict with Unpack allows you to tell the type checking system that it should err on the side of treating each positional and named argument as a tuple and a dictionary, respectively.

The type checking system does not mind when *args and **kwargs transmitted in the following form:

foo(*args, **kwargs)

But she is not satisfied when the named arguments are transmitted in full:

foo(*args, key1=1)  # error: Missing named argument "key2" for "foo"

You can turn off the flag to make all named arguments optional total in the definition of the class in which it is used TypedDict:

# ...
class Kw(TypedDict, total=False):
    key1: int
    key2: str

# ...

Or you can, using typing.NotRequiredIndicate the optionality of individual named arguments:

# ...
class Kw(TypedDict):
    key1: int
    key2: NotRequired[str]

# ...

This will allow you to pass incomplete sets of optional named arguments without offending the type checking system.

That’s all!

Oh, and come work for us? 🤗 💰

We at have been engaged in high-frequency algo trading since 2014. High-frequency trading is a continuous competition of the best programmers and mathematicians around the world. By joining us, you will be a part of this exciting battle.

We offer interesting and challenging tasks in data analysis and low latency development for enthusiastic researchers and programmers. Flexible schedule and no bureaucracy, decisions are quickly made and implemented.

We are currently looking for pros, pythonists, data engineers and ML researchers.

Join our team

Related posts