---
jupytext:
  formats: content_md///md:myst,content///ipynb
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.13.8
kernelspec:
  display_name: Python 3 (ipykernel)
  language: python
  name: python3
---

(python:strings:special-characters)=
# Special Characters

Strings may contain characters not available on some or all keyboards, like umlauts ä, ö, ü on US keyboards. To use such special characters in string literals Python provides escape sequences `\x`, `\u`, and `\U`.

## Character Sets

The relation between characters in strings and their numerical representation in memory will be considered in detail in the chapter on [](python:accessing-data:text-files). For the moment we content ourselves with the observation that there are several such mappings, the most prominent ones known as [ASCII](https://en.wikipedia.org/wiki/ASCII), the [ISO 8859](https://en.wikipedia.org/wiki/ISO/IEC_8859) family and several [Unicode](https://en.wikipedia.org/wiki/Unicode) variants.

ASCII knows 128 different characters, the ISO 8859 family some hundred, and Unicode several million ones. Nowadays, Unicode in its [UTF-8](https://en.wikipedia.org/wiki/UTF-8) variant is the standard mapping for numerical representations of characters. Even Windows is adopting UTF-8 more and more after backing the wrong horse for two decades.

There are lots of searchable lists of Unicode characters in the web. Wikipedia's [List of Unicode characters](https://en.wikipedia.org/wiki/List_of_Unicode_characters) is a good starting point.

## Unicode Characters in String Literals

To use Unicode characters in a string literal we either may type or copy them to the source code file or we may specify the character numerically. If the numercial representation has two hexadecimal digits, use `\x` followed by the two digits. In case of 4 digits use `\u` and for 8 digit characters use `\U`.

```{code-cell} ipython3
print('umlauts: \xe4, \xf6, \xfc')
print('Greek: \u03b1, \u03b2, \u03b3')
print('Chinese (?): \U0002070e, \U00028cd2')
```

```{note}
Not all Unicode characters are available in each font.
```

Some Unicode codes do not represent concrete characters, but control reading direction (left to right or right to left) and other properties like spacing.

```{figure} xkcd1137.png
---
alt: line drawing showing two persons discussing while reading direction flips due to unicode character
---
Collaborative editing can quickly become a textual rap battle fought with increasingly convoluted invocations of U+202a to U+202e. Source: Randall Munroe, [xkcd.com/1137](https://xkcd.com/1137)
```
