1. Type Annotations And Hints

By Bernd Klein. Last modified: 13 Jul 2023.

This chapter of our Python tutorial is about Type hints or type annotations. Both terms are synonyms and can be used interchangeably. Both terms refer to the practice of adding type information to variables, function parameters, and return values in Python code.

But let's start by looking at how Python is designed: Python is both a strongly typed and a dynamically typed language. Strong typing means that variables do have a type and that the type matters when performing operations on a variable. Dynamic typing means that the type of the variable is determined only during runtime. This means that types don't have to be declared in the program.

Yet, with Python 3.5 Type Annotations have been introduced. Are they really necessary? Do we have to use them?

The Python language itself doesn't care. The Python compiler itself does not enforce or check type annotations. Python remains a dynamically typed language, and type annotations are considered optional metadata. The Python interpreter does not perform any type checking based on these annotations during runtime.

Sitution in C and C++

If you know another programming language such as C or C++, you are used to declaring what data type you are working with. For example, this is how you would declare an integer variable in C or C++ like this.

int a;

This is known as type declaration. From this moment on, we - and the C/C++ compiler - know that "a" is of type integer. We can assign integers to a:

a = 3;

However, this is completely different in Python. Python doesn't know type declaration. Variables are just references to objects, as we have seen in chapter our chapter on Data Types and Variables

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here

Adding Type Hints to Variables

If you know another programming language such as C or C++, you are used to declaring what data type you are working with. For example, this is how you would declare an integer in C.

Yet, Python is a strictly typed programming language, whereas C and C++ are weakly typed. When we assign a "value" to a variable, Python automatically creates an object of the corresponding class:

programming_language = 'Python'
print(type(programming_language))

<class 'str'>

Guessing by the variable name 'programming_language', we assume that the one who wrote the Python code intended this variable to reference strings. Yet, Python doesn't "care". All kind of data types can be assigned to this variable name:

programming_language = 42

programming_language = ('Python', 'C', 'C++')

We will now demonstrate what Python offers to take care of these "type intentions", or as it is called in Python jargon "type hints", aka "type annotations". It's possible to define a variable with a type hint using the following syntax in Python:

variable_name: type = value

We can change our previous variable declaration accordingly with a Python type hint:

programming_language: str = 'Python'

Alternatively, we could have written this code like this:

programming_language: str
programming_language = 'Python'

Even though this looks now very similar to C or C++, one shouldn't be mislead. The behaviour of Python hasn't changed. We can still assign any data type to this variable. Python doesn't care, as we see in the following code:

programming_language: str
programming_language = 'Python'
programming_language = 42
print(programming_language)

42

Who Cares About Type Hints / Annotations

It's important to note that type annotations in Python are purely optional and do not affect the runtime behavior of your code. They are simply a way to provide additional information to tools that can help improve the quality and maintainability of your code.

However, there are external tools like

• mypy
• pyright
• pydantic
• IDEs like PyCharm, Spyder, VisualCode

that can analyze your code and perform static type checking based on the type annotations. These tools parse the code, interpret the type hints, and provide feedback on potential type-related errors and inconsistencies.

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here

mypy

mypy is a static type checker and checks for annotated code in Python. It emits warnings if annotated types are inconsistently used. It allows gradual typing, this means you can add type hints as you like.

mypy is an external program, which needs to be installed with for example

$ python3 -m pip install mypy

After this it can be ran in a shell (e.g. bash under Linux) with the Python program to be checked as an argument:

$ mypy example_prog.py

We will demonstrate with the following Python code how to use mypy. You have to know a few things about the Jupyter-Notebooks cells (ipython shell): With the shell magic %%writefile example1.py we can write the content of a cell into a file with the name example1.py. In IPython syntax, the exclamation mark (!) allows users to run shell commands (from your operating system) from inside a Jupyter Notebook code cell. Simply start a line of code with ! and it will run the command in the shell. We use this to call mypy on the Python file.

programming_language: str
programming_language = 'Python'
programming_language = 42
print(programming_language)

42

%%writefile example1.py
programming_language: str
programming_language = 'Python'
programming_language = 42
print(programming_language)

Overwriting example1.py

Let's test this annotated code now with mypy:

!mypy example1.py

example1.py:3: error: Incompatible types in assignment (expression has type "int", variable has type "str")
Found 1 error in 1 file (checked 1 source file)

!python example1.py

42

Displaying the type of an expression

reveal_type

If you find yourself unsure about how mypy handles a specific section of code, you can utilize reveal_type(expr) to request mypy to exhibit the inferred static type of an expression, offering helpful insights.

The mypy documentation says:

"reveal_type is only understood by mypy and doesn’t exist in Python, if you try to run your program. You’ll have to remove any reveal_type calls before you can run your code. reveal_type is always available and you don’t need to import it."

This means that you will get an exception, if you run a Python program containing a reveal_type:

%%writefile example.py
t = (1, 'hello')
reveal_type((1, 'hello'))

Overwriting example.py

!python example.py

Traceback (most recent call last):
  File "/home/bernd/Bodenseo Dropbox/Bernd Klein/notebooks/server/type-annotations/example.py", line 2, in <module>
    reveal_type((1, 'hello'))
NameError: name 'reveal_type' is not defined

It makes only sense, if you use it with a mypy call:

!mypy example.py

example.py:2: note: Revealed type is 'Tuple[Literal[1]?, Literal['hello']?]'

There is a way to use it in Python programs, if you set reveal_type to a function 'doing nothing', in case we are not in TYPE_CHECKING mode.

%%writefile example.py
from typing import TYPE_CHECKING

if not TYPE_CHECKING:
    def do_nothing(*args, **kwargs):
        pass
    #reveal_type = print
    reveal_type = do_nothing

x = 'hello'
x = 'hi'        # without this line x will be revealed as Literal['hello']?
reveal_type((1, 'hello', x))

Overwriting example.py

!python example.py

!mypy example.py

example.py:11: note: Revealed type is 'Tuple[Literal[1]?, Literal['hello']?, builtins.str]'

Reveal Locals

At any line in a file, you have the option to employ reveal_locals() to view the types of all local variables simultaneously.

%%writefile example.py
a: int = 1
b = 'one'
reveal_locals()

Overwriting example.py

!mypy example.py

example.py:3: note: Revealed local types are:
example.py:3: note:     b: builtins.str

pyright can also be used instead of mypy.

It's important to note that both pyright and mypy are actively maintained and have a strong community backing. They share many common features and can both provide valuable static type checking for Python projects. The choice between them depends on your specific needs, preferences, and the particular characteristics of your project.

!pyright example.py

WARNING: there is a new pyright version available (v1.1.306 -> v1.1.316).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`

/home/bernd/Bodenseo Dropbox/Bernd Klein/notebooks/server/type-annotations/example.py
  /home/bernd/Bodenseo Dropbox/Bernd Klein/notebooks/server/type-annotations/example.py:3:1 - information: Type of "a" is "int"
  Type of "b" is "str"
0 errors, 0 warnings, 1 information 

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here

Difference between pyright and mypy

Pyright implements its own parser, which recovers gracefully from syntax errors and continues parsing the remainder of the source file. By comparison, mypy uses the parser built in to the Python interpreter, and it does not support recovery after a syntax error.

From the README: Pyright is typically 5x or more faster than mypy and other type checkers that are written in Python. It is meant for large Python source bases. It can run in a “watch” mode and performs fast incremental updates when files are modified.

Type Comments

PEP484:

No first-class syntax support for explicitly marking variables as being of a specific type is added by this PEP. To help with type inference in complex cases, a comment of the following format may be used:

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here

Types

• int
• float
• str
• bool
• None ...

Variables

Type annotations for variables in Python are a way to provide explicit type information about the expected type of a variable. They can help improve code readability, provide documentation, and enable static type checking with tools like mypy. Here's how you can use type annotations for variables:

%%writefile example.py
firstname: str 
firstname = 'David'
surname: str = 'Miller'
min_value: int = 0
max_value: int = 100
temperature: float = 17.9

Overwriting example.py

!mypy example.py

Success: no issues found in 1 source file

As we have mentioned before: It's important to note that type annotations for variables in Python are optional and do not enforce the type at runtime. Python remains a dynamically typed language, so the actual type of a variable can still change during runtime.

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here

Tuples and Lists

Type annotations for tuples and lists in Python allow you to specify the expected types of the elements within these data structures. Here's how you can use type annotations for tuples and lists:

%%writefile example.py
from typing import List, Tuple

fibonacci: List[int] = [1, 1, 2, 3, 5, 8, 13]

person: Tuple[str, str, int] = ('Sarah', 'Brown', 42)

persons: List[Tuple[str, str, int]] = [('Sarah', 'Brown', 42),
                                       ('Edgar', 'Miller', 32),
                                       ('Donald', 'Brown', 55)]

Overwriting example.py

!mypy example.py

Success: no issues found in 1 source file

!python example.py

%%writefile example.py
from typing import List
lst: List[int]
lst = [3, 4, 5, 6]

Overwriting example.py

!mypy example.py

Success: no issues found in 1 source file

To make code more readable we can

%%writefile example.py
from typing import List, Tuple

Person =  Tuple[str, str, int]
persons: List[Person] = [('Sarah', 'Brown', 42),
                         ('Edgar', 'Miller', 32),
                         ('Donald', 'Brown', 55)]

Overwriting example.py

!python example.py

!mypy example.py

Success: no issues found in 1 source file

Since Python3.9+ we can write:

import sys
sys.version

'3.9.16 (main, Mar  8 2023, 14:00:05) \n[GCC 11.2.0]'

%%writefile example.py
import sys
print(f"{sys.version=}")

Person = tuple[str, str, int]

persons: list[Person] = [('Sarah', 'Brown', 42),
                         ('Donald', 'Brown', 45)]

Overwriting example.py

!python example.py

sys.version='3.9.16 (main, Mar  8 2023, 14:00:05) \n[GCC 11.2.0]'

!mypy --version

mypy 0.761

!/home/bernd/anaconda3/envs/py3.10/bin/mypy --version

mypy 0.981 (compiled: yes)

!/home/bernd/anaconda3/envs/py3.10/bin/python example.py

sys.version='3.10.10 (main, Mar 21 2023, 18:45:11) [GCC 11.2.0]'

!/home/bernd/anaconda3/envs/py3.10/bin/mypy example.py

Success: no issues found in 1 source file

Literal Ellipsis

In Python type annotations, the literal ellipsis (...) is a special type hint called "ellipsis" or "ellipsis type". It represents an unspecified or unknown type. The ellipsis type hint is often used when the specific type of a value or a part of a type is not known or is intentionally left unspecified.

%%writefile example.py
from typing import Tuple

x: tuple[int, ...] = (1, 2, 4, 6)
x = ()

Overwriting example.py

x: tuple[int, ...] = (1, 2, 4, 6)
x = ()
type(x)

tuple

!/home/bernd/anaconda3/envs/py3.10/bin/mypy example.py

Success: no issues found in 1 source file

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here

Type Aliases

PEP 613 summarizes Type Aliases like this:

Type aliases are user-specified types which may be as complex as any type hint, and are specified with a simple variable assignment on a module top level.

It's recommended to capitalize type aliases.! They are user-defined types like classes. Classes are also usually capitalized!

Necessity for TypeVars:

Readability Counts (Zen of Python)

Type aliases can be used in annotated function definitions. By using the alias Url in the following example, we clearly improve the readability of the code:

Url = str

def retry(url: Url, retry_count: int) -> None:
    pass

Another example:

from typing import List

Vector = List[float]

Type aliases shouldn't be confused with "untyped global expressions" and "typed global" expressions:

x = 1  # untyped global expression
x: int = 1  # typed global expression

I = int  # type alias

TypeVar

TypeVar is a Python construct used in type hints to indicate a placeholder for a generic type, allowing for flexible and reusable code.

Note that alias names should be uppercased by convention! They are user-defined types like classes. Classes are also usually uppercased!

Imagine, we would like to define three functions with the same name but a different signature (type annotation. We would like to do the following, which is not possible in Python.

We will start with an extremely simple function. This is a function which takes one object as an input and returns the object without any changes.

def identity(obj):
    return obj

The above function definition is untyped. We can annotate it by using Any:

from typing import Any

def identity(obj: Any) -> Any:
    return obj

There is a problem in this way of annotation. Any can be really anything, which is fine but not in the case of this function. The nature of the function is like this: The argument can be any type, but the return type depends on the input type, or more precise: It has to be the same type as the input argument.

%%writefile example.py
from typing import Any

def identity(obj: Any) -> Any:
    return obj


x: int

x = identity('hello')

Overwriting example.py

!mypy example.py

Success: no issues found in 1 source file

In the following code snippet we use a TypeVar. The function identity takes now an argument obj of type T and returns an object of the same type T. The T in the function signature is a type variable, which serves as a placeholder for a specific type that will be determined when the function is used.

By using the type variable T in the function signature and as the return type, the function maintains type safety and allows for a wide range of types to be handled without sacrificing type checking. It provides flexibility and code reusability, as the function can be used with different types while ensuring consistency in the return type.

%%writefile example.py
from typing import TypeVar

T = TypeVar('T')

def identity(obj: T) -> T:
    return obj

x: int

x = identity('hello')

Overwriting example.py

!mypy example.py

example.py:10: error: Incompatible types in assignment (expression has type "str", variable has type "int")
Found 1 error in 1 file (checked 1 source file)

We define now another simple function with a TypeVar:

%%writefile example.py

from typing import TypeVar

T = TypeVar("T")

def mul42(x: T) -> T:  
    return 42 * x

for x in [1, 1.1]:
    print(f"{x=}, {mul42(x)=}")

Overwriting example.py

!mypy example.py

example.py:7: error: Incompatible return value type (got "int", expected "T")
example.py:7: error: Unsupported operand types for * ("int" and "T")
Found 2 errors in 1 file (checked 1 source file)

mypy cannot find out if 42 * x has the same type as x. We can use a constraint for the types.

If we define

T = TypeVar('T')

The T can stand for anything, as we have seen.

If we write

A = TypeVar('A', str, bytes)

the types must be str or bytes

%%writefile example.py

from typing import TypeVar

T = TypeVar("T", int, float)

def mul42(x: T) -> T:  
    return 42 * x

for x in [1, 1.1]:
    print(f"{x=}, {mul42(x)=}")

Overwriting example.py

!python example.py

x=1, mul42(x)=42
x=1.1, mul42(x)=46.2

!mypy example.py

Success: no issues found in 1 source file

A TypeVar() expression must always directly be assigned to a variable (it should not be used as part of a larger expression). The argument to TypeVar() must be a string equal to the variable name to which it is assigned. Type variables must not be redefined.

Type variables created using the TypeVar() expression should always be assigned directly to a variable and not used as part of a larger expression. The argument passed to TypeVar() must be a string that matches the variable name to which it is assigned.

T = TypeVar('T')    # correct
S = TypeVar('U')    # not correct

It is important to note that type variables should not be redefined.

%%writefile example.py
from collections.abc import Sequence
from typing import TypeVar

T = TypeVar('T')      # Declare type variable

def first(lst: Sequence[T]) -> T:   # Generic function
    return lst[0]

lst1: Sequence[int] = [3, 5, 34]
lst2: Sequence[float] = [3.4, 1.8, 3.6]
#lst3: Sequence[str] = ['abs', 'abc']

for lst in [lst1, lst2]:
    print(first(lst))

Overwriting example.py

!python example.py

3
3.4

!mypy example.py

Success: no issues found in 1 source file

TypeVar supports constraining parametric types to a fixed set of possible types (note: those types cannot be parameterized by type variables). For example, we can define a type variable that ranges over just str and bytes. By default, a type variable ranges over all possible types. PEP 484 Type Hints

Examples:

%%writefile example.py
from typing import TypeVar

AnyStr = TypeVar('AnyStr', str, bytes)

def concat(x: AnyStr, y: AnyStr) -> AnyStr:
    return x + y

s: str = "hello "
t: str = "world "
print(concat(s, t))

bs: bytes = b"hello "
bt: bytes = b"world "
print(concat(bs, bt))

Overwriting example.py

!python example.py

hello world 
b'hello world '

!mypy example.py

Success: no issues found in 1 source file

We called the function concat two str arguments and two bytes arguments, but not with a mix of str and bytes arguments. A mix is not possible, as we can see in the following code example:

%%writefile example.py
from typing import TypeVar

AnyStr = TypeVar('AnyStr', str, bytes)

def concat(x: AnyStr, y: AnyStr) -> AnyStr:
    return x + y

s: str = "hello "
bt: bytes = b"world "
print(concat(s, bt))

Overwriting example.py

!mypy example.py

example.py:10: error: Value of type variable "AnyStr" of "concat" cannot be "object"
Found 1 error in 1 file (checked 1 source file)

%%writefile example.py
xs = [3.4, 4]
for x in xs:
    reveal_type(x)  # note: Revealed type is 'builtins.object*'

Overwriting example.py

!mypy example.py

example.py:3: note: Revealed type is 'builtins.float*'

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here

Using Union

Union[X, Y] is equivalent to X | Y and means either X or Y.

To define a union, use e.g. Union[int, str] or the shorthand int | str. Using that shorthand is recommended.

The following rules apply:

• The arguments must be types and there must be at least one.
• Unions of unions are flattened, e.g.: Union[Union[int, str], float] == Union[int, str, float]
• Unions of a single argument vanish, e.g.: Union[int] == int # The constructor actually returns int
• Redundant arguments are skipped, e.g.: Union[int, str, int] == Union[int, str] == int | str
• When comparing unions, the argument order is ignored, e.g.: Union[int, str] == Union[str, int]
• You cannot subclass or instantiate a Union.

Let's rewrite the previous example with unions:

%%writefile example.py

from typing import Union

def mul3(x: Union[int, float, str]) -> Union[int, float, str]:
    return 3 * float(x)

for x in [3, 3.4]:
    print(f"{x=}, {mul3(x)=}, {type(mul3(x))=}")

Overwriting example.py

!mypy example.py

Success: no issues found in 1 source file

Have you noticed the difference between TypeVar and Union? In the case of TypeVar, we had "The type getting in has to get out" whereas in Union they can be different!

%%writefile example.py
from typing import List, Tuple

def append_42(in_list: List[int]) -> List[int]:
    """ appends 42 to a copy of in_list """
    out_list = in_list + [42]
    return out_list

x: int = 42
values: List[float] = [4, 5, 9, x]
#append_42(values)

Overwriting example.py

!mypy example.py

Success: no issues found in 1 source file

Creating New Types

%%writefile example.py
from typing import NewType

UserId = NewType('UserId', int)
some_id = UserId(524313)

reveal_type(some_id)

Overwriting example.py

!mypy example.py

example.py:6: note: Revealed type is 'example.UserId'

The new types will be treated by the type checker as if they were subclasses of the original types.

By using NewType you can declare a type without actually creating new class instances. In the type checker, NewType('UserId', int) creates a subclass of int named "UserId"

NewType('UserId', int) is not a class but the identity function, so x is NewType('NewType', int)(x) is always true.

from typing import NewType

UserId = NewType('UserId', int)
some_id = UserId(524313)
some_id is NewType('NewType', int)(some_id)

True

type(some_id) == int

True

UserId is similiar as if it had been created with

class UserId(int): 
    pass

from typing import NewType

UserId = NewType('UserId', int)

def get_user_name(user_id: UserId) -> str:
    # Implementation of getting user name from user_id
    return "Bruce"

user_id = UserId(123)
user_name = get_user_name(user_id)

Note:

Recall that the use of a type alias declares two types to be equivalent to one another. Doing

Alias = Original

will make the static type checker treat Alias as being exactly equivalent to Original in all cases. This is useful when you want to simplify complex type signatures.

In contrast, NewType declares one type to be a subtype of another. Doing

Derived = NewType('Derived', Original)

will make the static type checker treat Derived as a subclass of Original, which means a value of type Original cannot be used in places where a value of type Derived is expected. This is useful when you want to prevent logic errors with minimal runtime cost.

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here

Casts

The cast function is a utility provided by the typing module in Python. It allows you to explicitly specify the type of an expression or variable, providing a hint to the type checker without affecting the runtime behavior of the code.

The cast function has the following signature:

def cast(typ: Type[T], val: T) -> T:
    ...

The first argument, typ, is the type that you want to cast the value to. The second argument, val, is the value that you want to cast.

The following is a useful example illustrating the usage of cast:

%%writefile example.py
from typing import cast

def calculate_average(numbers: list) -> float:
    total = sum(numbers)
    count = len(numbers)
    average = cast(float, total) / count
    return average

data = [1, 2, 3, 4, 5]
result = calculate_average(data)
print(result)

Overwriting example.py

!mypy example.py

Success: no issues found in 1 source file

In this example, the function calculate_average takes a list of numbers as input and calculates the average value. The variable total represents the sum of the numbers, and count represents the number of elements in the list. To calculate the average, we divide total by count.

The use of cast(float, total) is an explicit type cast annotation. It tells mypy to treat total as a float type in the context of the division, even though it was originally calculated as the sum of integers. This helps to avoid potential type mismatch warnings or errors from mypy.

Note that cast is a runtime no-op, meaning it has no effect on the actual execution of the code. Its purpose is to provide a hint to the type checker (e.g., mypy) about the intended type of a value in a specific context.

Live Python training

Enjoying this page? We offer live Python training courses covering the content of this site.

See: Live Python courses overview

Enrol here