peps/pep-0655/index.html

753 lines
73 KiB
HTML
Raw Permalink Blame History

This file contains invisible Unicode characters!

This file contains invisible Unicode characters that may be processed differently from what appears below. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to reveal hidden characters.

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="color-scheme" content="light dark">
<title>PEP 655 Marking individual TypedDict items as required or potentially-missing | peps.python.org</title>
<link rel="shortcut icon" href="../_static/py.png">
<link rel="canonical" href="https://peps.python.org/pep-0655/">
<link rel="stylesheet" href="../_static/style.css" type="text/css">
<link rel="stylesheet" href="../_static/mq.css" type="text/css">
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light">
<link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark">
<link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss">
<meta property="og:title" content='PEP 655 Marking individual TypedDict items as required or potentially-missing | peps.python.org'>
<meta property="og:type" content="website">
<meta property="og:url" content="https://peps.python.org/pep-0655/">
<meta property="og:site_name" content="Python Enhancement Proposals (PEPs)">
<meta property="og:image" content="https://peps.python.org/_static/og-image.png">
<meta property="og:image:alt" content="Python PEPs">
<meta property="og:image:width" content="200">
<meta property="og:image:height" content="200">
<meta name="description" content="Python Enhancement Proposals (PEPs)">
<meta name="theme-color" content="#3776ab">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all">
<title>Following system colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="9"></circle>
<path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all">
<title>Selected dark colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all">
<title>Selected light colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
</svg>
<script>
document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto"
</script>
<section id="pep-page-section">
<header>
<h1>Python Enhancement Proposals</h1>
<ul class="breadcrumbs">
<li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li>
<li><a href="../pep-0000/">PEP Index</a> &raquo; </li>
<li>PEP 655</li>
</ul>
<button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())">
<svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg>
<span class="visually-hidden">Toggle light / dark / auto colour theme</span>
</button>
</header>
<article>
<section id="pep-content">
<h1 class="page-title">PEP 655 Marking individual TypedDict items as required or potentially-missing</h1>
<dl class="rfc2822 field-list simple">
<dt class="field-odd">Author<span class="colon">:</span></dt>
<dd class="field-odd">David Foster &lt;david at dafoster.net&gt;</dd>
<dt class="field-even">Sponsor<span class="colon">:</span></dt>
<dd class="field-even">Guido van Rossum &lt;guido at python.org&gt;</dd>
<dt class="field-odd">Discussions-To<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="https://mail.python.org/archives/list/typing-sig&#64;python.org/thread/53XVOD5ZUKJ263MWA6AUPEA6J7LBBLNV/">Typing-SIG thread</a></dd>
<dt class="field-even">Status<span class="colon">:</span></dt>
<dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd>
<dt class="field-odd">Type<span class="colon">:</span></dt>
<dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd>
<dt class="field-even">Topic<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="../topic/typing/">Typing</a></dd>
<dt class="field-odd">Created<span class="colon">:</span></dt>
<dd class="field-odd">30-Jan-2021</dd>
<dt class="field-even">Python-Version<span class="colon">:</span></dt>
<dd class="field-even">3.11</dd>
<dt class="field-odd">Post-History<span class="colon">:</span></dt>
<dd class="field-odd">31-Jan-2021, 11-Feb-2021, 20-Feb-2021, 26-Feb-2021, 17-Jan-2022, 28-Jan-2022</dd>
<dt class="field-even">Resolution<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="https://mail.python.org/archives/list/python-dev&#64;python.org/message/AJEDNVC3FXM5QXNNW5CR4UCT4KI5XVUE/">Python-Dev message</a></dd>
</dl>
<hr class="docutils" />
<section id="contents">
<details><summary>Table of Contents</summary><ul class="simple">
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#motivation">Motivation</a></li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#interaction-with-total-false">Interaction with <code class="docutils literal notranslate"><span class="pre">total=False</span></code></a></li>
<li><a class="reference internal" href="#interaction-with-annotated">Interaction with <code class="docutils literal notranslate"><span class="pre">Annotated[]</span></code></a></li>
<li><a class="reference internal" href="#runtime-behavior">Runtime behavior</a><ul>
<li><a class="reference internal" href="#interaction-with-get-type-hints">Interaction with <code class="docutils literal notranslate"><span class="pre">get_type_hints()</span></code></a></li>
<li><a class="reference internal" href="#interaction-with-get-origin-and-get-args">Interaction with <code class="docutils literal notranslate"><span class="pre">get_origin()</span></code> and <code class="docutils literal notranslate"><span class="pre">get_args()</span></code></a></li>
<li><a class="reference internal" href="#interaction-with-required-keys-and-optional-keys">Interaction with <code class="docutils literal notranslate"><span class="pre">__required_keys__</span></code> and <code class="docutils literal notranslate"><span class="pre">__optional_keys__</span></code></a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li>
<li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a><ul>
<li><a class="reference internal" href="#usage-in-python-3-11">Usage in Python &lt;3.11</a></li>
</ul>
</li>
<li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li>
<li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul>
<li><a class="reference internal" href="#special-syntax-around-the-key-of-a-typeddict-item">Special syntax around the <em>key</em> of a TypedDict item</a></li>
<li><a class="reference internal" href="#marking-required-or-potentially-missing-keys-with-an-operator">Marking required or potentially-missing keys with an operator</a></li>
<li><a class="reference internal" href="#marking-absence-of-a-value-with-a-special-constant">Marking absence of a value with a special constant</a><ul>
<li><a class="reference internal" href="#misalignment-with-how-unions-apply-to-values">Misalignment with how unions apply to values</a></li>
<li><a class="reference internal" href="#misalignment-with-how-unions-are-subdivided">Misalignment with how unions are subdivided</a></li>
<li><a class="reference internal" href="#difficult-to-implement">Difficult to implement</a></li>
<li><a class="reference internal" href="#introduces-a-second-null-like-value-into-python">Introduces a second null-like value into Python</a></li>
</ul>
</li>
<li><a class="reference internal" href="#replace-optional-with-nullable-repurpose-optional-to-mean-optional-item">Replace Optional with Nullable. Repurpose Optional to mean “optional item”.</a></li>
<li><a class="reference internal" href="#change-optional-to-mean-optional-item-in-certain-contexts-instead-of-nullable">Change Optional to mean “optional item” in certain contexts instead of “nullable”</a></li>
<li><a class="reference internal" href="#various-synonyms-for-potentially-missing-item">Various synonyms for “potentially-missing item”</a></li>
</ul>
</li>
<li><a class="reference internal" href="#references">References</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
</details></section>
<div class="pep-banner canonical-typing-spec sticky-banner admonition attention">
<p class="admonition-title">Attention</p>
<p>This PEP is a historical document. The up-to-date, canonical spec, <a class="reference external" href="https://typing.readthedocs.io/en/latest/spec/typeddict.html#required-notrequired" title="(in typing)"><span>Required and NotRequired</span></a>, is maintained on the <a class="reference external" href="https://typing.readthedocs.io/en/latest/spec/">typing specs site</a>.</p>
<p class="close-button">×</p>
<p>See the <a class="reference external" href="https://typing.readthedocs.io/en/latest/spec/meta.html">typing specification update process</a> for how to propose changes.</p>
</div>
<section id="abstract">
<h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2>
<p><a class="pep reference internal" href="../pep-0589/" title="PEP 589 TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys">PEP 589</a> defines notation
for declaring a TypedDict with all required keys and notation for defining
a TypedDict with <a class="pep reference internal" href="../pep-0589/#totality" title="PEP 589 TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys § Totality">all potentially-missing keys</a>, however it
does not provide a mechanism to declare some keys as required and others
as potentially-missing. This PEP introduces two new notations:
<code class="docutils literal notranslate"><span class="pre">Required[]</span></code>, which can be used on individual items of a
TypedDict to mark them as required, and
<code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code>, which can be used on individual items
to mark them as potentially-missing.</p>
<p>This PEP makes no Python grammar changes. Correct usage
of required and potentially-missing keys of TypedDicts is intended to be
enforced only by static type checkers and need not be enforced by
Python itself at runtime.</p>
</section>
<section id="motivation">
<h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2>
<p>It is not uncommon to want to define a TypedDict with some keys that are
required and others that are potentially-missing. Currently the only way
to define such a TypedDict is to declare one TypedDict with one value
for <code class="docutils literal notranslate"><span class="pre">total</span></code> and then inherit it from another TypedDict with a
different value for <code class="docutils literal notranslate"><span class="pre">total</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">_MovieBase</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span> <span class="c1"># implicitly total=True</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">_MovieBase</span><span class="p">,</span> <span class="n">total</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="n">year</span><span class="p">:</span> <span class="nb">int</span>
</pre></div>
</div>
<p>Having to declare two different TypedDict types for this purpose is
cumbersome.</p>
<p>This PEP introduces two new type qualifiers, <code class="docutils literal notranslate"><span class="pre">typing.Required</span></code> and
<code class="docutils literal notranslate"><span class="pre">typing.NotRequired</span></code>, which allow defining a <em>single</em> TypedDict with
a mix of both required and potentially-missing keys:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">year</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span>
</pre></div>
</div>
<p>This PEP also makes it possible to define TypedDicts in the
<a class="pep reference internal" href="../pep-0589/#alternative-syntax" title="PEP 589 TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys § Alternative Syntax">alternative functional syntax</a>
with a mix of required and potentially-missing keys,
which is not currently possible at all because the alternative syntax does
not support inheritance:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Actor</span> <span class="o">=</span> <span class="n">TypedDict</span><span class="p">(</span><span class="s1">&#39;Actor&#39;</span><span class="p">,</span> <span class="p">{</span>
<span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
<span class="c1"># &quot;in&quot; is a keyword, so the functional syntax is necessary</span>
<span class="s1">&#39;in&#39;</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">]],</span>
<span class="p">})</span>
</pre></div>
</div>
</section>
<section id="rationale">
<h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2>
<p>One might think it unusual to propose notation that prioritizes marking
<em>required</em> keys rather than <em>potentially-missing</em> keys, as is
customary in other languages like TypeScript:</p>
<div class="highlight-typescript notranslate"><div class="highlight"><pre><span></span><span class="kd">interface</span><span class="w"> </span><span class="nx">Movie</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="nx">title</span><span class="o">:</span><span class="w"> </span><span class="kt">string</span><span class="p">;</span>
<span class="w"> </span><span class="nx">year?</span><span class="o">:</span><span class="w"> </span><span class="kt">number</span><span class="p">;</span><span class="w"> </span><span class="c1">// ? marks potentially-missing keys</span>
<span class="p">}</span>
</pre></div>
</div>
<p>The difficulty is that the best word for marking a potentially-missing
key, <code class="docutils literal notranslate"><span class="pre">Optional[]</span></code>, is already used in Python for a completely
different purpose: marking values that could be either of a particular
type or <code class="docutils literal notranslate"><span class="pre">None</span></code>. In particular the following does not work:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="o">...</span>
<span class="n">year</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span> <span class="c1"># means int|None, not potentially-missing!</span>
</pre></div>
</div>
<p>Attempting to use any synonym of “optional” to mark potentially-missing
keys (like <code class="docutils literal notranslate"><span class="pre">Missing[]</span></code>) would be too similar to <code class="docutils literal notranslate"><span class="pre">Optional[]</span></code>
and be easy to confuse with it.</p>
<p>Thus it was decided to focus on positive-form phrasing for required keys
instead, which is straightforward to spell as <code class="docutils literal notranslate"><span class="pre">Required[]</span></code>.</p>
<p>Nevertheless it is common for folks wanting to extend a regular
(<code class="docutils literal notranslate"><span class="pre">total=True</span></code>) TypedDict to only want to add a small number of
potentially-missing keys, which necessitates a way to mark keys that are
<em>not</em> required and potentially-missing, and so we also allow the
<code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> form for that case.</p>
</section>
<section id="specification">
<h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2>
<p>The <code class="docutils literal notranslate"><span class="pre">typing.Required</span></code> type qualifier is used to indicate that a
variable declared in a TypedDict definition is a required key:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">,</span> <span class="n">total</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="n">title</span><span class="p">:</span> <span class="n">Required</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>
<span class="n">year</span><span class="p">:</span> <span class="nb">int</span>
</pre></div>
</div>
<p>Additionally the <code class="docutils literal notranslate"><span class="pre">typing.NotRequired</span></code> type qualifier is used to
indicate that a variable declared in a TypedDict definition is a
potentially-missing key:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span> <span class="c1"># implicitly total=True</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">year</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span>
</pre></div>
</div>
<p>It is an error to use <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> or <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> in any
location that is not an item of a TypedDict.
Type checkers must enforce this restriction.</p>
<p>It is valid to use <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> and <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> even for
items where it is redundant, to enable additional explicitness if desired:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">title</span><span class="p">:</span> <span class="n">Required</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="c1"># redundant</span>
<span class="n">year</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span>
</pre></div>
</div>
<p>It is an error to use both <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> and <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> at the
same time:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">year</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="n">Required</span><span class="p">[</span><span class="nb">int</span><span class="p">]]</span> <span class="c1"># ERROR</span>
</pre></div>
</div>
<p>Type checkers must enforce this restriction.
The runtime implementations of <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> and <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code>
may also enforce this restriction.</p>
<p>The <a class="pep reference internal" href="../pep-0589/#alternative-syntax" title="PEP 589 TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys § Alternative Syntax">alternative functional syntax</a>
for TypedDict also supports
<code class="docutils literal notranslate"><span class="pre">Required[]</span></code> and <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Movie</span> <span class="o">=</span> <span class="n">TypedDict</span><span class="p">(</span><span class="s1">&#39;Movie&#39;</span><span class="p">,</span> <span class="p">{</span><span class="s1">&#39;name&#39;</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="s1">&#39;year&#39;</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">]})</span>
</pre></div>
</div>
<section id="interaction-with-total-false">
<h3><a class="toc-backref" href="#interaction-with-total-false" role="doc-backlink">Interaction with <code class="docutils literal notranslate"><span class="pre">total=False</span></code></a></h3>
<p>Any <a class="pep reference internal" href="../pep-0589/" title="PEP 589 TypedDict: Type Hints for Dictionaries with a Fixed Set of Keys">PEP 589</a>-style TypedDict declared with <code class="docutils literal notranslate"><span class="pre">total=False</span></code> is equivalent
to a TypedDict with an implicit <code class="docutils literal notranslate"><span class="pre">total=True</span></code> definition with all of its
keys marked as <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code>.</p>
<p>Therefore:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">_MovieBase</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span> <span class="c1"># implicitly total=True</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">_MovieBase</span><span class="p">,</span> <span class="n">total</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="n">year</span><span class="p">:</span> <span class="nb">int</span>
</pre></div>
</div>
<p>is equivalent to:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">_MovieBase</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">_MovieBase</span><span class="p">):</span>
<span class="n">year</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span>
</pre></div>
</div>
</section>
<section id="interaction-with-annotated">
<h3><a class="toc-backref" href="#interaction-with-annotated" role="doc-backlink">Interaction with <code class="docutils literal notranslate"><span class="pre">Annotated[]</span></code></a></h3>
<p><code class="docutils literal notranslate"><span class="pre">Required[]</span></code> and <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> can be used with <code class="docutils literal notranslate"><span class="pre">Annotated[]</span></code>,
in any nesting order:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">year</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="n">Annotated</span><span class="p">[</span><span class="nb">int</span><span class="p">,</span> <span class="n">ValueRange</span><span class="p">(</span><span class="o">-</span><span class="mi">9999</span><span class="p">,</span> <span class="mi">9999</span><span class="p">)]]</span> <span class="c1"># ok</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">year</span><span class="p">:</span> <span class="n">Annotated</span><span class="p">[</span><span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">],</span> <span class="n">ValueRange</span><span class="p">(</span><span class="o">-</span><span class="mi">9999</span><span class="p">,</span> <span class="mi">9999</span><span class="p">)]</span> <span class="c1"># ok</span>
</pre></div>
</div>
<p>In particular allowing <code class="docutils literal notranslate"><span class="pre">Annotated[]</span></code> to be the outermost annotation
for an item allows better interoperability with non-typing uses of
annotations, which may always want <code class="docutils literal notranslate"><span class="pre">Annotated[]</span></code> as the outermost annotation.
<a class="footnote-reference brackets" href="#id6" id="id1">[3]</a></p>
</section>
<section id="runtime-behavior">
<h3><a class="toc-backref" href="#runtime-behavior" role="doc-backlink">Runtime behavior</a></h3>
<section id="interaction-with-get-type-hints">
<h4><a class="toc-backref" href="#interaction-with-get-type-hints" role="doc-backlink">Interaction with <code class="docutils literal notranslate"><span class="pre">get_type_hints()</span></code></a></h4>
<p><code class="docutils literal notranslate"><span class="pre">typing.get_type_hints(...)</span></code> applied to a TypedDict will by default
strip out any <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> or <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> type qualifiers,
since these qualifiers are expected to be inconvenient for code
casually introspecting type annotations.</p>
<p><code class="docutils literal notranslate"><span class="pre">typing.get_type_hints(...,</span> <span class="pre">include_extras=True)</span></code> however
<em>will</em> retain <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> and <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> type qualifiers,
for advanced code introspecting type annotations that
wishes to preserve <em>all</em> annotations in the original source:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">year</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span>
<span class="k">assert</span> <span class="n">get_type_hints</span><span class="p">(</span><span class="n">Movie</span><span class="p">)</span> <span class="o">==</span> \
<span class="p">{</span><span class="s1">&#39;title&#39;</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="s1">&#39;year&#39;</span><span class="p">:</span> <span class="nb">int</span><span class="p">}</span>
<span class="k">assert</span> <span class="n">get_type_hints</span><span class="p">(</span><span class="n">Movie</span><span class="p">,</span> <span class="n">include_extras</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span> <span class="o">==</span> \
<span class="p">{</span><span class="s1">&#39;title&#39;</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="s1">&#39;year&#39;</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">]}</span>
</pre></div>
</div>
</section>
<section id="interaction-with-get-origin-and-get-args">
<h4><a class="toc-backref" href="#interaction-with-get-origin-and-get-args" role="doc-backlink">Interaction with <code class="docutils literal notranslate"><span class="pre">get_origin()</span></code> and <code class="docutils literal notranslate"><span class="pre">get_args()</span></code></a></h4>
<p><code class="docutils literal notranslate"><span class="pre">typing.get_origin()</span></code> and <code class="docutils literal notranslate"><span class="pre">typing.get_args()</span></code> will be updated to
recognize <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> and <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">assert</span> <span class="n">get_origin</span><span class="p">(</span><span class="n">Required</span><span class="p">[</span><span class="nb">int</span><span class="p">])</span> <span class="ow">is</span> <span class="n">Required</span>
<span class="k">assert</span> <span class="n">get_args</span><span class="p">(</span><span class="n">Required</span><span class="p">[</span><span class="nb">int</span><span class="p">])</span> <span class="o">==</span> <span class="p">(</span><span class="nb">int</span><span class="p">,)</span>
<span class="k">assert</span> <span class="n">get_origin</span><span class="p">(</span><span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">])</span> <span class="ow">is</span> <span class="n">NotRequired</span>
<span class="k">assert</span> <span class="n">get_args</span><span class="p">(</span><span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">])</span> <span class="o">==</span> <span class="p">(</span><span class="nb">int</span><span class="p">,)</span>
</pre></div>
</div>
</section>
<section id="interaction-with-required-keys-and-optional-keys">
<h4><a class="toc-backref" href="#interaction-with-required-keys-and-optional-keys" role="doc-backlink">Interaction with <code class="docutils literal notranslate"><span class="pre">__required_keys__</span></code> and <code class="docutils literal notranslate"><span class="pre">__optional_keys__</span></code></a></h4>
<p>An item marked with <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> will always appear
in the <code class="docutils literal notranslate"><span class="pre">__required_keys__</span></code> for its enclosing TypedDict. Similarly an item
marked with <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> will always appear in <code class="docutils literal notranslate"><span class="pre">__optional_keys__</span></code>.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">assert</span> <span class="n">Movie</span><span class="o">.</span><span class="n">__required_keys__</span> <span class="o">==</span> <span class="nb">frozenset</span><span class="p">({</span><span class="s1">&#39;title&#39;</span><span class="p">})</span>
<span class="k">assert</span> <span class="n">Movie</span><span class="o">.</span><span class="n">__optional_keys__</span> <span class="o">==</span> <span class="nb">frozenset</span><span class="p">({</span><span class="s1">&#39;year&#39;</span><span class="p">})</span>
</pre></div>
</div>
</section>
</section>
</section>
<section id="backwards-compatibility">
<h2><a class="toc-backref" href="#backwards-compatibility" role="doc-backlink">Backwards Compatibility</a></h2>
<p>No backward incompatible changes are made by this PEP.</p>
</section>
<section id="how-to-teach-this">
<h2><a class="toc-backref" href="#how-to-teach-this" role="doc-backlink">How to Teach This</a></h2>
<p>To define a TypedDict where most keys are required and some are
potentially-missing, define a single TypedDict as normal
(without the <code class="docutils literal notranslate"><span class="pre">total</span></code> keyword)
and mark those few keys that are potentially-missing with <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code>.</p>
<p>To define a TypedDict where most keys are potentially-missing and a few are
required, define a <code class="docutils literal notranslate"><span class="pre">total=False</span></code> TypedDict
and mark those few keys that are required with <code class="docutils literal notranslate"><span class="pre">Required[]</span></code>.</p>
<p>If some items accept <code class="docutils literal notranslate"><span class="pre">None</span></code> in addition to a regular value, it is
recommended that the <code class="docutils literal notranslate"><span class="pre">TYPE|None</span></code> notation be preferred over
<code class="docutils literal notranslate"><span class="pre">Optional[TYPE]</span></code> for marking such item values, to avoid using
<code class="docutils literal notranslate"><span class="pre">Required[]</span></code> or <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> alongside <code class="docutils literal notranslate"><span class="pre">Optional[]</span></code>
within the same TypedDict definition:</p>
<p>Yes:</p>
<div class="good highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">annotations</span> <span class="c1"># for Python 3.7-3.9</span>
<span class="k">class</span> <span class="nc">Dog</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">name</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">owner</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">str</span><span class="o">|</span><span class="kc">None</span><span class="p">]</span>
</pre></div>
</div>
<p>Okay (required for Python 3.5.3-3.6):</p>
<div class="maybe highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Dog</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">name</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">owner</span><span class="p">:</span> <span class="s1">&#39;NotRequired[str|None]&#39;</span>
</pre></div>
</div>
<p>No:</p>
<div class="bad highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Dog</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">name</span><span class="p">:</span> <span class="nb">str</span>
<span class="c1"># ick; avoid using both Optional and NotRequired</span>
<span class="n">owner</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
</pre></div>
</div>
<section id="usage-in-python-3-11">
<h3><a class="toc-backref" href="#usage-in-python-3-11" role="doc-backlink">Usage in Python &lt;3.11</a></h3>
<p>If your code supports Python &lt;3.11 and wishes to use <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> or
<code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code> then it should use <code class="docutils literal notranslate"><span class="pre">typing_extensions.TypedDict</span></code> rather
than <code class="docutils literal notranslate"><span class="pre">typing.TypedDict</span></code> because the latter will not understand
<code class="docutils literal notranslate"><span class="pre">(Not)Required[]</span></code>. In particular <code class="docutils literal notranslate"><span class="pre">__required_keys__</span></code> and
<code class="docutils literal notranslate"><span class="pre">__optional_keys__</span></code> on the resulting TypedDict type will not be correct:</p>
<p>Yes (Python 3.11+ only):</p>
<div class="good highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">NotRequired</span><span class="p">,</span> <span class="n">TypedDict</span>
<span class="k">class</span> <span class="nc">Dog</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">name</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">owner</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">str</span><span class="o">|</span><span class="kc">None</span><span class="p">]</span>
</pre></div>
</div>
<p>Yes (Python &lt;3.11 and 3.11+):</p>
<div class="good highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">annotations</span> <span class="c1"># for Python 3.7-3.9</span>
<span class="kn">from</span> <span class="nn">typing_extensions</span> <span class="kn">import</span> <span class="n">NotRequired</span><span class="p">,</span> <span class="n">TypedDict</span> <span class="c1"># for Python &lt;3.11 with (Not)Required</span>
<span class="k">class</span> <span class="nc">Dog</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">name</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">owner</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">str</span><span class="o">|</span><span class="kc">None</span><span class="p">]</span>
</pre></div>
</div>
<p>No (Python &lt;3.11 and 3.11+):</p>
<div class="bad highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">TypedDict</span> <span class="c1"># oops: should import from typing_extensions instead</span>
<span class="kn">from</span> <span class="nn">typing_extensions</span> <span class="kn">import</span> <span class="n">NotRequired</span>
<span class="k">class</span> <span class="nc">Movie</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">title</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">year</span><span class="p">:</span> <span class="n">NotRequired</span><span class="p">[</span><span class="nb">int</span><span class="p">]</span>
<span class="k">assert</span> <span class="n">Movie</span><span class="o">.</span><span class="n">__required_keys__</span> <span class="o">==</span> <span class="nb">frozenset</span><span class="p">({</span><span class="s1">&#39;title&#39;</span><span class="p">,</span> <span class="s1">&#39;year&#39;</span><span class="p">})</span> <span class="c1"># yikes</span>
<span class="k">assert</span> <span class="n">Movie</span><span class="o">.</span><span class="n">__optional_keys__</span> <span class="o">==</span> <span class="nb">frozenset</span><span class="p">()</span> <span class="c1"># yikes</span>
</pre></div>
</div>
</section>
</section>
<section id="reference-implementation">
<h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2>
<p>The <a class="reference external" href="http://www.mypy-lang.org/">mypy</a>
<a class="reference external" href="https://mypy-lang.blogspot.com/2021/12/mypy-0930-released.html">0.930</a>,
<a class="reference external" href="https://github.com/Microsoft/pyright">pyright</a>
<a class="reference external" href="https://github.com/microsoft/pyright/commit/7ed245b1845173090c6404e49912e8cbfb3417c8">1.1.117</a>,
and <a class="reference external" href="https://github.com/quora/pyanalyze">pyanalyze</a>
<a class="reference external" href="https://pyanalyze.readthedocs.io/en/latest/changelog.html#version-0-4-0-november-18-2021">0.4.0</a>
type checkers support <code class="docutils literal notranslate"><span class="pre">Required</span></code> and <code class="docutils literal notranslate"><span class="pre">NotRequired</span></code>.</p>
<p>A reference implementation of the runtime component is provided in the
<a class="reference external" href="https://github.com/python/typing/tree/master/typing_extensions">typing_extensions</a>
module.</p>
</section>
<section id="rejected-ideas">
<h2><a class="toc-backref" href="#rejected-ideas" role="doc-backlink">Rejected Ideas</a></h2>
<section id="special-syntax-around-the-key-of-a-typeddict-item">
<h3><a class="toc-backref" href="#special-syntax-around-the-key-of-a-typeddict-item" role="doc-backlink">Special syntax around the <em>key</em> of a TypedDict item</a></h3>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>class MyThing(TypedDict):
opt1?: str # may not exist, but if exists, value is string
opt2: Optional[str] # always exists, but may have None value
</pre></div>
</div>
<p>This notation would require Python grammar changes and it is not
believed that marking TypedDict items as required or potentially-missing
would meet the high bar required to make such grammar changes.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyThing</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">Optional</span><span class="p">[</span><span class="n">opt1</span><span class="p">]:</span> <span class="nb">str</span> <span class="c1"># may not exist, but if exists, value is string</span>
<span class="n">opt2</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span> <span class="c1"># always exists, but may have None value</span>
</pre></div>
</div>
<p>This notation causes <code class="docutils literal notranslate"><span class="pre">Optional[]</span></code> to take on different meanings depending
on where it is positioned, which is inconsistent and confusing.</p>
<p>Also, “lets just not put funny syntax before the colon.” <a class="footnote-reference brackets" href="#id4" id="id2">[1]</a></p>
</section>
<section id="marking-required-or-potentially-missing-keys-with-an-operator">
<h3><a class="toc-backref" href="#marking-required-or-potentially-missing-keys-with-an-operator" role="doc-backlink">Marking required or potentially-missing keys with an operator</a></h3>
<p>We could use unary <code class="docutils literal notranslate"><span class="pre">+</span></code> as shorthand to mark a required key, unary
<code class="docutils literal notranslate"><span class="pre">-</span></code> to mark a potentially-missing key, or unary <code class="docutils literal notranslate"><span class="pre">~</span></code> to mark a key
with opposite-of-normal totality:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyThing</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">,</span> <span class="n">total</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="n">req1</span><span class="p">:</span> <span class="o">+</span><span class="nb">int</span> <span class="c1"># + means a required key, or Required[]</span>
<span class="n">opt1</span><span class="p">:</span> <span class="nb">str</span>
<span class="n">req2</span><span class="p">:</span> <span class="o">+</span><span class="nb">float</span>
<span class="k">class</span> <span class="nc">MyThing</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">req1</span><span class="p">:</span> <span class="nb">int</span>
<span class="n">opt1</span><span class="p">:</span> <span class="o">-</span><span class="nb">str</span> <span class="c1"># - means a potentially-missing key, or NotRequired[]</span>
<span class="n">req2</span><span class="p">:</span> <span class="nb">float</span>
<span class="k">class</span> <span class="nc">MyThing</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">req1</span><span class="p">:</span> <span class="nb">int</span>
<span class="n">opt1</span><span class="p">:</span> <span class="o">~</span><span class="nb">str</span> <span class="c1"># ~ means a opposite-of-normal-totality key</span>
<span class="n">req2</span><span class="p">:</span> <span class="nb">float</span>
</pre></div>
</div>
<p>Such operators could be implemented on <code class="docutils literal notranslate"><span class="pre">type</span></code> via the <code class="docutils literal notranslate"><span class="pre">__pos__</span></code>,
<code class="docutils literal notranslate"><span class="pre">__neg__</span></code> and <code class="docutils literal notranslate"><span class="pre">__invert__</span></code> special methods without modifying the
grammar.</p>
<p>It was decided that it would be prudent to introduce long-form notation
(i.e. <code class="docutils literal notranslate"><span class="pre">Required[]</span></code> and <code class="docutils literal notranslate"><span class="pre">NotRequired[]</span></code>) before introducing
any short-form notation. Future PEPs may reconsider introducing this
or other short-form notation options.</p>
<p>Note when reconsidering introducing this short-form notation that
<code class="docutils literal notranslate"><span class="pre">+</span></code>, <code class="docutils literal notranslate"><span class="pre">-</span></code>, and <code class="docutils literal notranslate"><span class="pre">~</span></code> already have existing meanings in the Python
typing world: covariant, contravariant, and invariant:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">&gt;&gt;&gt; </span><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">TypeVar</span>
<span class="gp">&gt;&gt;&gt; </span><span class="p">(</span><span class="n">TypeVar</span><span class="p">(</span><span class="s1">&#39;T&#39;</span><span class="p">,</span> <span class="n">covariant</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">TypeVar</span><span class="p">(</span><span class="s1">&#39;U&#39;</span><span class="p">,</span> <span class="n">contravariant</span><span class="o">=</span><span class="kc">True</span><span class="p">),</span> <span class="n">TypeVar</span><span class="p">(</span><span class="s1">&#39;V&#39;</span><span class="p">))</span>
<span class="go">(+T, -U, ~V)</span>
</pre></div>
</div>
</section>
<section id="marking-absence-of-a-value-with-a-special-constant">
<h3><a class="toc-backref" href="#marking-absence-of-a-value-with-a-special-constant" role="doc-backlink">Marking absence of a value with a special constant</a></h3>
<p>We could introduce a new type-level constant which signals the absence
of a value when used as a union member, similar to JavaScripts
<code class="docutils literal notranslate"><span class="pre">undefined</span></code> type, perhaps called <code class="docutils literal notranslate"><span class="pre">Missing</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyThing</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">):</span>
<span class="n">req1</span><span class="p">:</span> <span class="nb">int</span>
<span class="n">opt1</span><span class="p">:</span> <span class="nb">str</span><span class="o">|</span><span class="n">Missing</span>
<span class="n">req2</span><span class="p">:</span> <span class="nb">float</span>
</pre></div>
</div>
<p>Such a <code class="docutils literal notranslate"><span class="pre">Missing</span></code> constant could also be used for other scenarios such
as the type of a variable which is only conditionally defined:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyClass</span><span class="p">:</span>
<span class="n">attr</span><span class="p">:</span> <span class="nb">int</span><span class="o">|</span><span class="n">Missing</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">set_attr</span><span class="p">:</span> <span class="nb">bool</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
<span class="k">if</span> <span class="n">set_attr</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">attr</span> <span class="o">=</span> <span class="mi">10</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">set_attr</span><span class="p">:</span> <span class="nb">bool</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
<span class="k">if</span> <span class="n">set_attr</span><span class="p">:</span>
<span class="n">attr</span> <span class="o">=</span> <span class="mi">10</span>
<span class="n">reveal_type</span><span class="p">(</span><span class="n">attr</span><span class="p">)</span> <span class="c1"># int|Missing</span>
</pre></div>
</div>
<section id="misalignment-with-how-unions-apply-to-values">
<h4><a class="toc-backref" href="#misalignment-with-how-unions-apply-to-values" role="doc-backlink">Misalignment with how unions apply to values</a></h4>
<p>However this use of <code class="docutils literal notranslate"><span class="pre">...|Missing</span></code>, equivalent to
<code class="docutils literal notranslate"><span class="pre">Union[...,</span> <span class="pre">Missing]</span></code>, doesnt align well with what a union normally
means: <code class="docutils literal notranslate"><span class="pre">Union[...]</span></code> always describes the type of a <em>value</em> that is
present. By contrast missingness or non-totality is a property of a
<em>variable</em> instead. Current precedent for marking properties of a
variable include <code class="docutils literal notranslate"><span class="pre">Final[...]</span></code> and <code class="docutils literal notranslate"><span class="pre">ClassVar[...]</span></code>, which the
proposal for <code class="docutils literal notranslate"><span class="pre">Required[...]</span></code> is aligned with.</p>
</section>
<section id="misalignment-with-how-unions-are-subdivided">
<h4><a class="toc-backref" href="#misalignment-with-how-unions-are-subdivided" role="doc-backlink">Misalignment with how unions are subdivided</a></h4>
<p>Furthermore the use of <code class="docutils literal notranslate"><span class="pre">Union[...,</span> <span class="pre">Missing]</span></code> doesnt align with the
usual ways that union values are broken down: Normally you can eliminate
components of a union type using <code class="docutils literal notranslate"><span class="pre">isinstance</span></code> checks:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Packet</span><span class="p">:</span>
<span class="n">data</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">bytes</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">send_data</span><span class="p">(</span><span class="n">packet</span><span class="p">:</span> <span class="n">Packet</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
<span class="k">if</span> <span class="nb">isinstance</span><span class="p">(</span><span class="n">packet</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="nb">str</span><span class="p">):</span>
<span class="n">reveal_type</span><span class="p">(</span><span class="n">packet</span><span class="o">.</span><span class="n">data</span><span class="p">)</span> <span class="c1"># str</span>
<span class="n">packet_bytes</span> <span class="o">=</span> <span class="n">packet</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">&#39;utf-8&#39;</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">reveal_type</span><span class="p">(</span><span class="n">packet</span><span class="o">.</span><span class="n">data</span><span class="p">)</span> <span class="c1"># bytes</span>
<span class="n">packet_bytes</span> <span class="o">=</span> <span class="n">packet</span><span class="o">.</span><span class="n">data</span>
<span class="n">socket</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">packet_bytes</span><span class="p">)</span>
</pre></div>
</div>
<p>However if we were to allow <code class="docutils literal notranslate"><span class="pre">Union[...,</span> <span class="pre">Missing]</span></code> youd either have to
eliminate the <code class="docutils literal notranslate"><span class="pre">Missing</span></code> case with <code class="docutils literal notranslate"><span class="pre">hasattr</span></code> for object attributes:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Packet</span><span class="p">:</span>
<span class="n">data</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Missing</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">send_data</span><span class="p">(</span><span class="n">packet</span><span class="p">:</span> <span class="n">Packet</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">packet</span><span class="p">,</span> <span class="s1">&#39;data&#39;</span><span class="p">):</span>
<span class="n">reveal_type</span><span class="p">(</span><span class="n">packet</span><span class="o">.</span><span class="n">data</span><span class="p">)</span> <span class="c1"># str</span>
<span class="n">packet_bytes</span> <span class="o">=</span> <span class="n">packet</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">&#39;utf-8&#39;</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">reveal_type</span><span class="p">(</span><span class="n">packet</span><span class="o">.</span><span class="n">data</span><span class="p">)</span> <span class="c1"># Missing? error?</span>
<span class="n">packet_bytes</span> <span class="o">=</span> <span class="sa">b</span><span class="s1">&#39;&#39;</span>
<span class="n">socket</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">packet_bytes</span><span class="p">)</span>
</pre></div>
</div>
<p>or a check against <code class="docutils literal notranslate"><span class="pre">locals()</span></code> for local variables:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">send_data</span><span class="p">(</span><span class="n">packet_data</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">packet_bytes</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Missing</span><span class="p">]</span>
<span class="k">if</span> <span class="n">packet_data</span> <span class="ow">is</span> <span class="ow">not</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">packet_bytes</span> <span class="o">=</span> <span class="n">packet</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s1">&#39;utf-8&#39;</span><span class="p">)</span>
<span class="k">if</span> <span class="s1">&#39;packet_bytes&#39;</span> <span class="ow">in</span> <span class="nb">locals</span><span class="p">():</span>
<span class="n">reveal_type</span><span class="p">(</span><span class="n">packet_bytes</span><span class="p">)</span> <span class="c1"># bytes</span>
<span class="n">socket</span><span class="o">.</span><span class="n">send</span><span class="p">(</span><span class="n">packet_bytes</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">reveal_type</span><span class="p">(</span><span class="n">packet_bytes</span><span class="p">)</span> <span class="c1"># Missing? error?</span>
</pre></div>
</div>
<p>or a check via other means, such as against <code class="docutils literal notranslate"><span class="pre">globals()</span></code> for global
variables:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">warning</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Missing</span><span class="p">]</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="k">if</span> <span class="n">sys</span><span class="o">.</span><span class="n">version_info</span> <span class="o">&lt;</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">6</span><span class="p">):</span>
<span class="n">warning</span> <span class="o">=</span> <span class="s1">&#39;Your version of Python is unsupported!&#39;</span>
<span class="k">if</span> <span class="s1">&#39;warning&#39;</span> <span class="ow">in</span> <span class="nb">globals</span><span class="p">():</span>
<span class="n">reveal_type</span><span class="p">(</span><span class="n">warning</span><span class="p">)</span> <span class="c1"># str</span>
<span class="nb">print</span><span class="p">(</span><span class="n">warning</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">reveal_type</span><span class="p">(</span><span class="n">warning</span><span class="p">)</span> <span class="c1"># Missing? error?</span>
</pre></div>
</div>
<p>Weird and inconsistent. <code class="docutils literal notranslate"><span class="pre">Missing</span></code> is not really a value at all; its
an absence of definition and such an absence should be treated
specially.</p>
</section>
<section id="difficult-to-implement">
<h4><a class="toc-backref" href="#difficult-to-implement" role="doc-backlink">Difficult to implement</a></h4>
<p>Eric Traut from the Pyright type checker team has stated that
implementing a <code class="docutils literal notranslate"><span class="pre">Union[...,</span> <span class="pre">Missing]</span></code>-style notation would be
difficult. <a class="footnote-reference brackets" href="#id5" id="id3">[2]</a></p>
</section>
<section id="introduces-a-second-null-like-value-into-python">
<h4><a class="toc-backref" href="#introduces-a-second-null-like-value-into-python" role="doc-backlink">Introduces a second null-like value into Python</a></h4>
<p>Defining a new <code class="docutils literal notranslate"><span class="pre">Missing</span></code> type-level constant would be very close to
introducing a new <code class="docutils literal notranslate"><span class="pre">Missing</span></code> value-level constant at runtime, creating
a second null-like runtime value in addition to <code class="docutils literal notranslate"><span class="pre">None</span></code>. Having two
different null-like constants in Python (<code class="docutils literal notranslate"><span class="pre">None</span></code> and <code class="docutils literal notranslate"><span class="pre">Missing</span></code>) would
be confusing. Many newcomers to JavaScript already have difficulty
distinguishing between its analogous constants <code class="docutils literal notranslate"><span class="pre">null</span></code> and
<code class="docutils literal notranslate"><span class="pre">undefined</span></code>.</p>
</section>
</section>
<section id="replace-optional-with-nullable-repurpose-optional-to-mean-optional-item">
<h3><a class="toc-backref" href="#replace-optional-with-nullable-repurpose-optional-to-mean-optional-item" role="doc-backlink">Replace Optional with Nullable. Repurpose Optional to mean “optional item”.</a></h3>
<p><code class="docutils literal notranslate"><span class="pre">Optional[]</span></code> is too ubiquitous to deprecate, although use of it
<em>may</em> fade over time in favor of the <code class="docutils literal notranslate"><span class="pre">T|None</span></code> notation specified by <a class="pep reference internal" href="../pep-0604/" title="PEP 604 Allow writing union types as X | Y">PEP 604</a>.</p>
</section>
<section id="change-optional-to-mean-optional-item-in-certain-contexts-instead-of-nullable">
<h3><a class="toc-backref" href="#change-optional-to-mean-optional-item-in-certain-contexts-instead-of-nullable" role="doc-backlink">Change Optional to mean “optional item” in certain contexts instead of “nullable”</a></h3>
<p>Consider the use of a special flag on a TypedDict definition to alter
the interpretation of <code class="docutils literal notranslate"><span class="pre">Optional</span></code> inside the TypedDict to mean
“optional item” rather than its usual meaning of “nullable”:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyThing</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">,</span> <span class="n">optional_as_missing</span><span class="o">=</span><span class="kc">True</span><span class="p">):</span>
<span class="n">req1</span><span class="p">:</span> <span class="nb">int</span>
<span class="n">opt1</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>
</pre></div>
</div>
<p>or:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">MyThing</span><span class="p">(</span><span class="n">TypedDict</span><span class="p">,</span> <span class="n">optional_as_nullable</span><span class="o">=</span><span class="kc">False</span><span class="p">):</span>
<span class="n">req1</span><span class="p">:</span> <span class="nb">int</span>
<span class="n">opt1</span><span class="p">:</span> <span class="n">Optional</span><span class="p">[</span><span class="nb">str</span><span class="p">]</span>
</pre></div>
</div>
<p>This would add more confusion for users because it would mean that in
<em>some</em> contexts the meaning of <code class="docutils literal notranslate"><span class="pre">Optional[]</span></code> is different than in
other contexts, and it would be easy to overlook the flag.</p>
</section>
<section id="various-synonyms-for-potentially-missing-item">
<h3><a class="toc-backref" href="#various-synonyms-for-potentially-missing-item" role="doc-backlink">Various synonyms for “potentially-missing item”</a></h3>
<ul class="simple">
<li>Omittable too easy to confuse with optional</li>
<li>OptionalItem, OptionalKey two words; too easy to confuse with
optional</li>
<li>MayExist, MissingOk two words</li>
<li>Droppable too similar to Rusts <code class="docutils literal notranslate"><span class="pre">Drop</span></code>, which means something
different</li>
<li>Potential too vague</li>
<li>Open sounds like applies to an entire structure rather then to an
item</li>
<li>Excludable</li>
<li>Checked</li>
</ul>
</section>
</section>
<section id="references">
<h2><a class="toc-backref" href="#references" role="doc-backlink">References</a></h2>
<aside class="footnote-list brackets">
<aside class="footnote brackets" id="id4" role="doc-footnote">
<dt class="label" id="id4">[<a href="#id2">1</a>]</dt>
<dd><a class="reference external" href="https://mail.python.org/archives/list/typing-sig&#64;python.org/message/4I3GPIWDUKV6GUCHDMORGUGRE4F4SXGR/">https://mail.python.org/archives/list/typing-sig&#64;python.org/message/4I3GPIWDUKV6GUCHDMORGUGRE4F4SXGR/</a></aside>
<aside class="footnote brackets" id="id5" role="doc-footnote">
<dt class="label" id="id5">[<a href="#id3">2</a>]</dt>
<dd><a class="reference external" href="https://mail.python.org/archives/list/typing-sig&#64;python.org/message/S2VJSVG6WCIWPBZ54BOJPG56KXVSLZK6/">https://mail.python.org/archives/list/typing-sig&#64;python.org/message/S2VJSVG6WCIWPBZ54BOJPG56KXVSLZK6/</a></aside>
<aside class="footnote brackets" id="id6" role="doc-footnote">
<dt class="label" id="id6">[<a href="#id1">3</a>]</dt>
<dd><a class="reference external" href="https://bugs.python.org/issue46491">https://bugs.python.org/issue46491</a></aside>
</aside>
</section>
<section id="copyright">
<h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2>
<p>This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.</p>
</section>
</section>
<hr class="docutils" />
<p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0655.rst">https://github.com/python/peps/blob/main/peps/pep-0655.rst</a></p>
<p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0655.rst">2024-02-16 06:27:36 GMT</a></p>
</article>
<nav id="pep-sidebar">
<h2>Contents</h2>
<ul>
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#motivation">Motivation</a></li>
<li><a class="reference internal" href="#rationale">Rationale</a></li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#interaction-with-total-false">Interaction with <code class="docutils literal notranslate"><span class="pre">total=False</span></code></a></li>
<li><a class="reference internal" href="#interaction-with-annotated">Interaction with <code class="docutils literal notranslate"><span class="pre">Annotated[]</span></code></a></li>
<li><a class="reference internal" href="#runtime-behavior">Runtime behavior</a><ul>
<li><a class="reference internal" href="#interaction-with-get-type-hints">Interaction with <code class="docutils literal notranslate"><span class="pre">get_type_hints()</span></code></a></li>
<li><a class="reference internal" href="#interaction-with-get-origin-and-get-args">Interaction with <code class="docutils literal notranslate"><span class="pre">get_origin()</span></code> and <code class="docutils literal notranslate"><span class="pre">get_args()</span></code></a></li>
<li><a class="reference internal" href="#interaction-with-required-keys-and-optional-keys">Interaction with <code class="docutils literal notranslate"><span class="pre">__required_keys__</span></code> and <code class="docutils literal notranslate"><span class="pre">__optional_keys__</span></code></a></li>
</ul>
</li>
</ul>
</li>
<li><a class="reference internal" href="#backwards-compatibility">Backwards Compatibility</a></li>
<li><a class="reference internal" href="#how-to-teach-this">How to Teach This</a><ul>
<li><a class="reference internal" href="#usage-in-python-3-11">Usage in Python &lt;3.11</a></li>
</ul>
</li>
<li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li>
<li><a class="reference internal" href="#rejected-ideas">Rejected Ideas</a><ul>
<li><a class="reference internal" href="#special-syntax-around-the-key-of-a-typeddict-item">Special syntax around the <em>key</em> of a TypedDict item</a></li>
<li><a class="reference internal" href="#marking-required-or-potentially-missing-keys-with-an-operator">Marking required or potentially-missing keys with an operator</a></li>
<li><a class="reference internal" href="#marking-absence-of-a-value-with-a-special-constant">Marking absence of a value with a special constant</a><ul>
<li><a class="reference internal" href="#misalignment-with-how-unions-apply-to-values">Misalignment with how unions apply to values</a></li>
<li><a class="reference internal" href="#misalignment-with-how-unions-are-subdivided">Misalignment with how unions are subdivided</a></li>
<li><a class="reference internal" href="#difficult-to-implement">Difficult to implement</a></li>
<li><a class="reference internal" href="#introduces-a-second-null-like-value-into-python">Introduces a second null-like value into Python</a></li>
</ul>
</li>
<li><a class="reference internal" href="#replace-optional-with-nullable-repurpose-optional-to-mean-optional-item">Replace Optional with Nullable. Repurpose Optional to mean “optional item”.</a></li>
<li><a class="reference internal" href="#change-optional-to-mean-optional-item-in-certain-contexts-instead-of-nullable">Change Optional to mean “optional item” in certain contexts instead of “nullable”</a></li>
<li><a class="reference internal" href="#various-synonyms-for-potentially-missing-item">Various synonyms for “potentially-missing item”</a></li>
</ul>
</li>
<li><a class="reference internal" href="#references">References</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
<br>
<a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0655.rst">Page Source (GitHub)</a>
</nav>
</section>
<script src="../_static/colour_scheme.js"></script>
<script src="../_static/wrap_tables.js"></script>
<script src="../_static/sticky_banner.js"></script>
</body>
</html>