peps/pep-0698/index.html

610 lines
49 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="color-scheme" content="light dark">
<title>PEP 698 Override Decorator for Static Typing | peps.python.org</title>
<link rel="shortcut icon" href="../_static/py.png">
<link rel="canonical" href="https://peps.python.org/pep-0698/">
<link rel="stylesheet" href="../_static/style.css" type="text/css">
<link rel="stylesheet" href="../_static/mq.css" type="text/css">
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light">
<link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark">
<link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss">
<meta property="og:title" content='PEP 698 Override Decorator for Static Typing | peps.python.org'>
<meta property="og:type" content="website">
<meta property="og:url" content="https://peps.python.org/pep-0698/">
<meta property="og:site_name" content="Python Enhancement Proposals (PEPs)">
<meta property="og:image" content="https://peps.python.org/_static/og-image.png">
<meta property="og:image:alt" content="Python PEPs">
<meta property="og:image:width" content="200">
<meta property="og:image:height" content="200">
<meta name="description" content="Python Enhancement Proposals (PEPs)">
<meta name="theme-color" content="#3776ab">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all">
<title>Following system colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="9"></circle>
<path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all">
<title>Selected dark colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all">
<title>Selected light colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
</svg>
<script>
document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto"
</script>
<section id="pep-page-section">
<header>
<h1>Python Enhancement Proposals</h1>
<ul class="breadcrumbs">
<li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li>
<li><a href="../pep-0000/">PEP Index</a> &raquo; </li>
<li>PEP 698</li>
</ul>
<button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())">
<svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg>
<span class="visually-hidden">Toggle light / dark / auto colour theme</span>
</button>
</header>
<article>
<section id="pep-content">
<h1 class="page-title">PEP 698 Override Decorator for Static Typing</h1>
<dl class="rfc2822 field-list simple">
<dt class="field-odd">Author<span class="colon">:</span></dt>
<dd class="field-odd">Steven Troxler &lt;steven.troxler&#32;&#97;t&#32;gmail.com&gt;,
Joshua Xu &lt;jxu425&#32;&#97;t&#32;fb.com&gt;,
Shannon Zhu &lt;szhu&#32;&#97;t&#32;fb.com&gt;</dd>
<dt class="field-even">Sponsor<span class="colon">:</span></dt>
<dd class="field-even">Jelle Zijlstra &lt;jelle.zijlstra at gmail.com&gt;</dd>
<dt class="field-odd">Discussions-To<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="https://discuss.python.org/t/pep-698-a-typing-override-decorator/20839">Discourse thread</a></dd>
<dt class="field-even">Status<span class="colon">:</span></dt>
<dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd>
<dt class="field-odd">Type<span class="colon">:</span></dt>
<dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd>
<dt class="field-even">Topic<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="../topic/typing/">Typing</a></dd>
<dt class="field-odd">Created<span class="colon">:</span></dt>
<dd class="field-odd">05-Sep-2022</dd>
<dt class="field-even">Python-Version<span class="colon">:</span></dt>
<dd class="field-even">3.12</dd>
<dt class="field-odd">Post-History<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="https://mail.python.org/archives/list/typing-sig&#64;python.org/thread/V23I4D6DEOFW4BBPWBMYTHZUOMKR7KQE/" title="Typing-SIG thread">20-May-2022</a>,
<a class="reference external" href="https://mail.python.org/archives/list/typing-sig&#64;python.org/thread/7JDW2PKGF6YTERUJGWM3BRP3GDHRFP4O/" title="Typing-SIG thread">17-Aug-2022</a>,
<a class="reference external" href="https://mail.python.org/archives/list/typing-sig&#64;python.org/thread/TOIYZ3SNPBJZDBRU3ZSBREXV2NNHF4KW/" title="Typing-SIG thread">11-Oct-2022</a>,
<a class="reference external" href="https://discuss.python.org/t/pep-698-a-typing-override-decorator/20839" title="Discourse thread">07-Nov-2022</a></dd>
<dt class="field-even">Resolution<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="https://discuss.python.org/t/pep-698-a-typing-override-decorator/20839/11">Discourse message</a></dd>
</dl>
<hr class="docutils" />
<section id="contents">
<details><summary>Table of Contents</summary><ul class="simple">
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#motivation">Motivation</a><ul>
<li><a class="reference internal" href="#safe-refactoring">Safe Refactoring</a></li>
</ul>
</li>
<li><a class="reference internal" href="#rationale">Rationale</a><ul>
<li><a class="reference internal" href="#subclass-implementations-become-more-explicit">Subclass Implementations Become More Explicit</a></li>
<li><a class="reference internal" href="#precedent-in-other-languages-and-runtime-libraries">Precedent in Other Languages and Runtime Libraries</a><ul>
<li><a class="reference internal" href="#static-override-checks-in-other-languages">Static Override Checks in Other Languages</a></li>
<li><a class="reference internal" href="#runtime-override-checks-in-python">Runtime Override Checks in Python</a></li>
</ul>
</li>
<li><a class="reference internal" href="#disadvantages">Disadvantages</a></li>
</ul>
</li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#no-new-rules-for-override-compatibility">No New Rules for Override Compatibility</a></li>
</ul>
</li>
<li><a class="reference internal" href="#strict-enforcement-per-project">Strict Enforcement Per-Project</a><ul>
<li><a class="reference internal" href="#id2">Motivation</a></li>
<li><a class="reference internal" href="#precedent">Precedent</a></li>
</ul>
</li>
<li><a class="reference internal" href="#backward-compatibility">Backward Compatibility</a></li>
<li><a class="reference internal" href="#runtime-behavior">Runtime Behavior</a><ul>
<li><a class="reference internal" href="#set-override-true-when-possible">Set <code class="docutils literal notranslate"><span class="pre">__override__</span> <span class="pre">=</span> <span class="pre">True</span></code> when possible</a></li>
<li><a class="reference internal" href="#limitations-of-setting-override">Limitations of setting <code class="docutils literal notranslate"><span class="pre">__override__</span></code></a></li>
</ul>
</li>
<li><a class="reference internal" href="#rejected-alternatives">Rejected Alternatives</a><ul>
<li><a class="reference internal" href="#rely-on-integrated-development-environments-for-safety">Rely on Integrated Development Environments for safety</a></li>
<li><a class="reference internal" href="#runtime-enforcement">Runtime enforcement</a></li>
<li><a class="reference internal" href="#mark-a-base-class-to-force-explicit-overrides-on-subclasses">Mark a base class to force explicit overrides on subclasses</a></li>
<li><a class="reference internal" href="#include-the-name-of-the-ancestor-class-being-overridden">Include the name of the ancestor class being overridden</a></li>
</ul>
</li>
<li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
</details></section>
<div class="pep-banner canonical-typing-spec sticky-banner admonition attention">
<p class="admonition-title">Attention</p>
<p>This PEP is a historical document. The up-to-date, canonical spec, <a class="reference external" href="https://typing.readthedocs.io/en/latest/spec/class-compat.html#override" title="(in typing)"><span>&#64;override</span></a>, is maintained on the <a class="reference external" href="https://typing.readthedocs.io/en/latest/spec/">typing specs site</a>.</p>
<p class="close-button">×</p>
<p>See the <a class="reference external" href="https://typing.readthedocs.io/en/latest/spec/meta.html">typing specification update process</a> for how to propose changes.</p>
</div>
<section id="abstract">
<h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2>
<p>This PEP proposes adding an <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> decorator to the Python type system.
This will allow type checkers to prevent a class of bugs that occur when a base
class changes methods that are inherited by derived classes.</p>
</section>
<section id="motivation">
<h2><a class="toc-backref" href="#motivation" role="doc-backlink">Motivation</a></h2>
<p>A primary purpose of type checkers is to flag when refactors or changes break
pre-existing semantic structures in the code, so users can identify and make
fixes across their project without doing a manual audit of their code.</p>
<section id="safe-refactoring">
<h3><a class="toc-backref" href="#safe-refactoring" role="doc-backlink">Safe Refactoring</a></h3>
<p>Pythons type system does not provide a way to identify call sites that need to
be changed to stay consistent when an overridden function API changes. This
makes refactoring and transforming code more dangerous.</p>
<p>Consider this simple inheritance structure:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Parent</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="n">x</span>
<span class="k">class</span> <span class="nc">Child</span><span class="p">(</span><span class="n">Parent</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="mi">1</span>
<span class="k">def</span> <span class="nf">parent_callsite</span><span class="p">(</span><span class="n">parent</span><span class="p">:</span> <span class="n">Parent</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">parent</span><span class="o">.</span><span class="n">foo</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">child_callsite</span><span class="p">(</span><span class="n">child</span><span class="p">:</span> <span class="n">Child</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
<span class="n">child</span><span class="o">.</span><span class="n">foo</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
</pre></div>
</div>
<p>If the overridden method on the superclass is renamed or deleted, type checkers
will only alert us to update call sites that deal with the base type directly.
But the type checker can only see the new code, not the change we made, so it
has no way of knowing that we probably also needed to rename the same method on
child classes.</p>
<p>A type checker will happily accept this code, even though we are likely
introducing bugs:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Parent</span><span class="p">:</span>
<span class="c1"># Rename this method</span>
<span class="k">def</span> <span class="nf">new_foo</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="n">x</span>
<span class="k">class</span> <span class="nc">Child</span><span class="p">(</span><span class="n">Parent</span><span class="p">):</span>
<span class="c1"># This (unchanged) method used to override `foo` but is unrelated to `new_foo`</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="mi">1</span>
<span class="k">def</span> <span class="nf">parent_callsite</span><span class="p">(</span><span class="n">parent</span><span class="p">:</span> <span class="n">Parent</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
<span class="c1"># If we pass a Child instance well now run Parent.new_foo - likely a bug</span>
<span class="n">parent</span><span class="o">.</span><span class="n">new_foo</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">child_callsite</span><span class="p">(</span><span class="n">child</span><span class="p">:</span> <span class="n">Child</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
<span class="c1"># We probably wanted to invoke new_foo here. Instead, we forked the method</span>
<span class="n">child</span><span class="o">.</span><span class="n">foo</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
</pre></div>
</div>
<p>This code will type check, but there are two potential sources of bugs:</p>
<ul class="simple">
<li>If we pass a <code class="docutils literal notranslate"><span class="pre">Child</span></code> instance to the <code class="docutils literal notranslate"><span class="pre">parent_callsite</span></code> function, it will
invoke the implementation in <code class="docutils literal notranslate"><span class="pre">Parent.new_foo</span></code>. rather than <code class="docutils literal notranslate"><span class="pre">Child.foo</span></code>.
This is probably a bug - we presumably would not have written <code class="docutils literal notranslate"><span class="pre">Child.foo</span></code> in
the first place if we didnt need custom behavior.</li>
<li>Our system was likely relying on <code class="docutils literal notranslate"><span class="pre">Child.foo</span></code> behaving in a similar way to
<code class="docutils literal notranslate"><span class="pre">Parent.foo</span></code>. But unless we catch this early, we have now forked the
methods, and in future refactors it is likely no one will realize that major
changes to the behavior of <code class="docutils literal notranslate"><span class="pre">new_foo</span></code> likely require updating <code class="docutils literal notranslate"><span class="pre">Child.foo</span></code> as
well, which could lead to major bugs later.</li>
</ul>
<p>The incorrectly-refactored code is type-safe, but is probably not what we
intended and could cause our system to behave incorrectly. The bug can be
difficult to track down because our new code likely does execute without
throwing exceptions. Tests are less likely to catch the problem, and silent
errors can take longer to track down in production.</p>
<p>We are aware of several production outages in multiple typed codebases caused by
such incorrect refactors. This is our primary motivation for adding an <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code>
decorator to the type system, which lets developers express the relationship
between <code class="docutils literal notranslate"><span class="pre">Parent.foo</span></code> and <code class="docutils literal notranslate"><span class="pre">Child.foo</span></code> so that type checkers can detect the problem.</p>
</section>
</section>
<section id="rationale">
<h2><a class="toc-backref" href="#rationale" role="doc-backlink">Rationale</a></h2>
<section id="subclass-implementations-become-more-explicit">
<h3><a class="toc-backref" href="#subclass-implementations-become-more-explicit" role="doc-backlink">Subclass Implementations Become More Explicit</a></h3>
<p>We believe that explicit overrides will make unfamiliar code easier to read than
implicit overrides. A developer reading the implementation of a subclass that
uses <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> can immediately see which methods are overriding
functionality in some base class; without this decorator, the only way to
quickly find out is using a static analysis tool.</p>
</section>
<section id="precedent-in-other-languages-and-runtime-libraries">
<h3><a class="toc-backref" href="#precedent-in-other-languages-and-runtime-libraries" role="doc-backlink">Precedent in Other Languages and Runtime Libraries</a></h3>
<section id="static-override-checks-in-other-languages">
<h4><a class="toc-backref" href="#static-override-checks-in-other-languages" role="doc-backlink">Static Override Checks in Other Languages</a></h4>
<p>Many popular programming languages support override checks. For example:</p>
<ul class="simple">
<li><a class="reference external" href="https://en.cppreference.com/w/cpp/language/override">C++ has</a> <code class="docutils literal notranslate"><span class="pre">override</span></code>.</li>
<li><a class="reference external" href="https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/override/">C# has</a> <code class="docutils literal notranslate"><span class="pre">override</span></code>.</li>
<li><a class="reference external" href="https://docs.hhvm.com/hack/attributes/predefined-attributes#__override">Hack has</a> <code class="docutils literal notranslate"><span class="pre">&lt;&lt;__Override&gt;&gt;</span></code>.</li>
<li><a class="reference external" href="https://docs.oracle.com/javase/tutorial/java/IandI/override.html">Java has</a> <code class="docutils literal notranslate"><span class="pre">&#64;Override</span></code>.</li>
<li><a class="reference external" href="https://kotlinlang.org/docs/inheritance.html#overriding-methods">Kotlin has</a> <code class="docutils literal notranslate"><span class="pre">override</span></code>.</li>
<li><a class="reference external" href="https://www.javatpoint.com/scala-method-overriding">Scala has</a> <code class="docutils literal notranslate"><span class="pre">override</span></code>.</li>
<li><a class="reference external" href="https://docs.swift.org/swift-book/documentation/the-swift-programming-language/inheritance/#Overriding">Swift has</a> <code class="docutils literal notranslate"><span class="pre">override</span></code>.</li>
<li><a class="reference external" href="https://www.typescriptlang.org/docs/handbook/release-notes/typescript-4-3.html#override-and-the---noimplicitoverride-flag">TypeScript has</a> <code class="docutils literal notranslate"><span class="pre">override</span></code>.</li>
</ul>
</section>
<section id="runtime-override-checks-in-python">
<h4><a class="toc-backref" href="#runtime-override-checks-in-python" role="doc-backlink">Runtime Override Checks in Python</a></h4>
<p>Today, there is an <a class="reference external" href="https://pypi.org/project/overrides/">Overrides library</a>
that provides decorators <code class="docutils literal notranslate"><span class="pre">&#64;overrides</span></code> [sic] and <code class="docutils literal notranslate"><span class="pre">&#64;final</span></code> and will enforce
them at runtime.</p>
<p><a class="pep reference internal" href="../pep-0591/" title="PEP 591 Adding a final qualifier to typing">PEP 591</a> added a <code class="docutils literal notranslate"><span class="pre">&#64;final</span></code> decorator with the same semantics as those in the
Overrides library. But the override component of the runtime library is not
supported statically at all, which has added some confusion around the
mix/matched support.</p>
<p>Providing support for <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> in static checks would add value because:</p>
<ul class="simple">
<li>Bugs can be caught earlier, often in-editor.</li>
<li>Static checks come with no performance overhead, unlike runtime checks.</li>
<li>Bugs will be caught quickly even in rarely-used modules, whereas with runtime
checks these might go undetected for a time without automated tests of all
imports.</li>
</ul>
</section>
</section>
<section id="disadvantages">
<h3><a class="toc-backref" href="#disadvantages" role="doc-backlink">Disadvantages</a></h3>
<p>Using <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> will make code more verbose.</p>
</section>
</section>
<section id="specification">
<h2><a class="toc-backref" href="#specification" role="doc-backlink">Specification</a></h2>
<p>When type checkers encounter a method decorated with <code class="docutils literal notranslate"><span class="pre">&#64;typing.override</span></code> they
should treat it as a type error unless that method is overriding a compatible
method or attribute in some ancestor class.</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">typing</span> <span class="kn">import</span> <span class="n">override</span>
<span class="k">class</span> <span class="nc">Parent</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="mi">1</span>
<span class="k">def</span> <span class="nf">bar</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
<span class="k">return</span> <span class="n">x</span>
<span class="k">class</span> <span class="nc">Child</span><span class="p">(</span><span class="n">Parent</span><span class="p">):</span>
<span class="nd">@override</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="mi">2</span>
<span class="nd">@override</span>
<span class="k">def</span> <span class="nf">baz</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span> <span class="c1"># Type check error: no matching signature in ancestor</span>
<span class="k">return</span> <span class="mi">1</span>
</pre></div>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> decorator should be permitted anywhere a type checker
considers a method to be a valid override, which typically includes not only
normal methods but also <code class="docutils literal notranslate"><span class="pre">&#64;property</span></code>, <code class="docutils literal notranslate"><span class="pre">&#64;staticmethod</span></code>, and <code class="docutils literal notranslate"><span class="pre">&#64;classmethod</span></code>.</p>
<section id="no-new-rules-for-override-compatibility">
<h3><a class="toc-backref" href="#no-new-rules-for-override-compatibility" role="doc-backlink">No New Rules for Override Compatibility</a></h3>
<p>This PEP is exclusively concerned with the handling of the new <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> decorator,
which specifies that the decorated method must override some attribute in
an ancestor class. This PEP does not propose any new rules regarding the type
signatures of such methods.</p>
</section>
</section>
<section id="strict-enforcement-per-project">
<h2><a class="toc-backref" href="#strict-enforcement-per-project" role="doc-backlink">Strict Enforcement Per-Project</a></h2>
<p>We believe that <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> is most useful if checkers also allow developers
to opt into a strict mode where methods that override a parent class are
required to use the decorator. Strict enforcement should be opt-in for backward
compatibility.</p>
<section id="id2">
<h3><a class="toc-backref" href="#id2" role="doc-backlink">Motivation</a></h3>
<p>The primary reason for a strict mode that requires <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> is that developers
can only trust that refactors are override-safe if they know that the <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code>
decorator is used throughout the project.</p>
<p>There is another class of bug related to overrides that we can only catch using a strict mode.</p>
<p>Consider the following code:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Parent</span><span class="p">:</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">Child</span><span class="p">(</span><span class="n">Parent</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="mi">2</span>
</pre></div>
</div>
<p>Imagine we refactor it as follows:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Parent</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span> <span class="c1"># This method is new</span>
<span class="k">return</span> <span class="mi">1</span>
<span class="k">class</span> <span class="nc">Child</span><span class="p">(</span><span class="n">Parent</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span> <span class="c1"># This is now an override!</span>
<span class="k">return</span> <span class="mi">2</span>
<span class="k">def</span> <span class="nf">call_foo</span><span class="p">(</span><span class="n">parent</span><span class="p">:</span> <span class="n">Parent</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="n">parent</span><span class="o">.</span><span class="n">foo</span><span class="p">()</span> <span class="c1"># This could invoke Child.foo, which may be surprising.</span>
</pre></div>
</div>
<p>The semantics of our code changed here, which could cause two problems:</p>
<ul class="simple">
<li>If the author of the code change did not know that <code class="docutils literal notranslate"><span class="pre">Child.foo</span></code> already
existed (which is very possible in a large codebase), they might be surprised
to see that <code class="docutils literal notranslate"><span class="pre">call_foo</span></code> does not always invoke <code class="docutils literal notranslate"><span class="pre">Parent.foo</span></code>.</li>
<li>If the codebase authors tried to manually apply <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> everywhere when
writing overrides in subclasses, they are likely to miss the fact that
<code class="docutils literal notranslate"><span class="pre">Child.foo</span></code> needs it here.</li>
</ul>
<p>At first glance this kind of change may seem unlikely, but it can actually
happen often if one or more subclasses have functionality that developers later
realize belongs in the base class.</p>
<p>With a strict mode, we will always alert developers when this occurs.</p>
</section>
<section id="precedent">
<h3><a class="toc-backref" href="#precedent" role="doc-backlink">Precedent</a></h3>
<p>Most of the typed, object-oriented programming languages we looked at have an
easy way to require explicit overrides throughout a project:</p>
<ul class="simple">
<li>C#, Kotlin, Scala, and Swift always require explicit overrides</li>
<li>TypeScript has a
<a class="reference external" href="https://www.typescriptlang.org/docs/handbook/release-notes/typescript-4-3.html#override-and-the---noimplicitoverride-flag/">no-implicit-override</a>
flag to force explicit overrides</li>
<li>In Hack and Java the type checker always treats overrides as opt-in, but
widely-used linters can warn if explicit overrides are missing.</li>
</ul>
</section>
</section>
<section id="backward-compatibility">
<h2><a class="toc-backref" href="#backward-compatibility" role="doc-backlink">Backward Compatibility</a></h2>
<p>By default, the <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> decorator will be opt-in. Codebases that do not
use it will type-check as before, without the additional type safety.</p>
</section>
<section id="runtime-behavior">
<h2><a class="toc-backref" href="#runtime-behavior" role="doc-backlink">Runtime Behavior</a></h2>
<section id="set-override-true-when-possible">
<h3><a class="toc-backref" href="#set-override-true-when-possible" role="doc-backlink">Set <code class="docutils literal notranslate"><span class="pre">__override__</span> <span class="pre">=</span> <span class="pre">True</span></code> when possible</a></h3>
<p>At runtime, <code class="docutils literal notranslate"><span class="pre">&#64;typing.override</span></code> will make a best-effort attempt to add an
attribute <code class="docutils literal notranslate"><span class="pre">__override__</span></code> with value <code class="docutils literal notranslate"><span class="pre">True</span></code> to its argument. By “best-effort”
we mean that we will try adding the attribute, but if that fails (for example
because the input is a descriptor type with fixed slots) we will silently
return the argument as-is.</p>
<p>This is exactly what the <code class="docutils literal notranslate"><span class="pre">&#64;typing.final</span></code> decorator does, and the motivation
is similar: it gives runtime libraries the ability to use <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code>. As a
concrete example, a runtime library could check <code class="docutils literal notranslate"><span class="pre">__override__</span></code> in order
to automatically populate the <code class="docutils literal notranslate"><span class="pre">__doc__</span></code> attribute of child class methods
using the parent method docstring.</p>
</section>
<section id="limitations-of-setting-override">
<h3><a class="toc-backref" href="#limitations-of-setting-override" role="doc-backlink">Limitations of setting <code class="docutils literal notranslate"><span class="pre">__override__</span></code></a></h3>
<p>As described above, adding <code class="docutils literal notranslate"><span class="pre">__override__</span></code> may fail at runtime, in which
case we will simply return the argument as-is.</p>
<p>In addition, even in cases where it does work, it may be difficult for users to
correctly work with multiple decorators, because successfully ensuring the
<code class="docutils literal notranslate"><span class="pre">__override__</span></code> attribute is set on the final output requires understanding the
implementation of each decorator:</p>
<ul class="simple">
<li>The <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> decorator needs to execute <em>after</em> ordinary decorators
like <code class="docutils literal notranslate"><span class="pre">&#64;functools.lru_cache</span></code> that use wrapper functions, since we want to
set <code class="docutils literal notranslate"><span class="pre">__override__</span></code> on the outermost wrapper. This means it needs to
go <em>above</em> all these other decorators.</li>
<li>But <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> needs to execute <em>before</em> many special descriptor-based
decorators like <code class="docutils literal notranslate"><span class="pre">&#64;property</span></code>, <code class="docutils literal notranslate"><span class="pre">&#64;staticmethod</span></code>, and <code class="docutils literal notranslate"><span class="pre">&#64;classmethod</span></code>.</li>
<li>As discussed above, in some cases (for example a descriptor with fixed
slots or a descriptor that also wraps) it may be impossible to set the
<code class="docutils literal notranslate"><span class="pre">__override__</span></code> attribute at all.</li>
</ul>
<p>As a result, runtime support for setting <code class="docutils literal notranslate"><span class="pre">__override__</span></code> is best effort
only, and we do not expect type checkers to validate the ordering of
decorators.</p>
</section>
</section>
<section id="rejected-alternatives">
<h2><a class="toc-backref" href="#rejected-alternatives" role="doc-backlink">Rejected Alternatives</a></h2>
<section id="rely-on-integrated-development-environments-for-safety">
<h3><a class="toc-backref" href="#rely-on-integrated-development-environments-for-safety" role="doc-backlink">Rely on Integrated Development Environments for safety</a></h3>
<p>Modern Integrated Development Environments (IDEs) often provide the ability to
automatically update subclasses when renaming a method. But we view this as
insufficient for several reasons:</p>
<ul class="simple">
<li>If a codebase is split into multiple projects, an IDE will not help and the
bug appears when upgrading dependencies. Type checkers are a fast way to catch
breaking changes in dependencies.</li>
<li>Not all developers use such IDEs. And library maintainers, even if they do use
an IDE, should not need to assume pull request authors use the same IDE. We
prefer being able to detect problems in continuous integration without
assuming anything about developers choice of editor.</li>
</ul>
</section>
<section id="runtime-enforcement">
<h3><a class="toc-backref" href="#runtime-enforcement" role="doc-backlink">Runtime enforcement</a></h3>
<p>We considered having <code class="docutils literal notranslate"><span class="pre">&#64;typing.override</span></code> enforce override safety at runtime,
similarly to how <code class="docutils literal notranslate"><span class="pre">&#64;overrides.overrides</span></code>
<a class="reference external" href="https://pypi.org/project/overrides/">does today</a>.</p>
<p>We rejected this for four reasons:</p>
<ul class="simple">
<li>For users of static type checking, it is not clear this brings any benefits.</li>
<li>There would be at least some performance overhead, leading to projects
importing slower with runtime enforcement. We estimate the
<code class="docutils literal notranslate"><span class="pre">&#64;overrides.overrides</span></code> implementation takes around 100 microseconds, which
is fast but could still add up to a second or more of extra initialization
time in million-plus line codebases, which is exactly where we think
<code class="docutils literal notranslate"><span class="pre">&#64;typing.override</span></code> will be most useful.</li>
<li>An implementation may have edge cases where it doesnt work well (we heard
from a maintainer of one such closed-source library that this has been a
problem). We expect static enforcement to be simple and reliable.</li>
<li>The implementation approaches we know of are not simple. The decorator
executes before the class is finished evaluating, so the options we know of
are either to inspect the bytecode of the caller (as <code class="docutils literal notranslate"><span class="pre">&#64;overrides.overrides</span></code>
does) or to use a metaclass-based approach. Neither approach seems ideal.</li>
</ul>
</section>
<section id="mark-a-base-class-to-force-explicit-overrides-on-subclasses">
<h3><a class="toc-backref" href="#mark-a-base-class-to-force-explicit-overrides-on-subclasses" role="doc-backlink">Mark a base class to force explicit overrides on subclasses</a></h3>
<p>We considered including a class decorator <code class="docutils literal notranslate"><span class="pre">&#64;require_explicit_overrides</span></code>, which
would have provided a way for base classes to declare that all subclasses must
use the <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> decorator on method overrides. The
<a class="reference external" href="https://pypi.org/project/overrides/">Overrides library</a> has a mixin class,
<code class="docutils literal notranslate"><span class="pre">EnforceExplicitOverrides</span></code>, which provides similar behavior in runtime checks.</p>
<p>We decided against this because we expect owners of large codebases will benefit
most from <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code>, and for these use cases having a strict mode where
explicit <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> is required (see the Backward Compatibility section)
provides more benefits than a way to mark base classes.</p>
<p>Moreover we believe that authors of projects who do not consider the extra type
safety to be worth the additional boilerplate of using <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> should not
be forced to do so. Having an optional strict mode puts the decision in the
hands of project owners, whereas the use of <code class="docutils literal notranslate"><span class="pre">&#64;require_explicit_overrides</span></code> in
libraries would force project owners to use <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> even if they prefer
not to.</p>
</section>
<section id="include-the-name-of-the-ancestor-class-being-overridden">
<h3><a class="toc-backref" href="#include-the-name-of-the-ancestor-class-being-overridden" role="doc-backlink">Include the name of the ancestor class being overridden</a></h3>
<p>We considered allowing the caller of <code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> to specify a specific
ancestor class where the overridden method should be defined:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">class</span> <span class="nc">Parent0</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="mi">1</span>
<span class="k">class</span> <span class="nc">Parent1</span><span class="p">:</span>
<span class="k">def</span> <span class="nf">bar</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="mi">1</span>
<span class="k">class</span> <span class="nc">Child</span><span class="p">(</span><span class="n">Parent0</span><span class="p">,</span> <span class="n">Parent1</span><span class="p">):</span>
<span class="nd">@override</span><span class="p">(</span><span class="n">Parent0</span><span class="p">)</span> <span class="c1"># okay, Parent0 defines foo</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="mi">2</span>
<span class="nd">@override</span><span class="p">(</span><span class="n">Parent0</span><span class="p">)</span> <span class="c1"># type error, Parent0 does not define bar</span>
<span class="k">def</span> <span class="nf">bar</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">int</span><span class="p">:</span>
<span class="k">return</span> <span class="mi">2</span>
</pre></div>
</div>
<p>This could be useful for code readability because it makes the override
structure more explicit for deep inheritance trees. It also might catch bugs by
prompting developers to check that the implementation of an override still makes
sense whenever a method being overridden moves from one base class to another.</p>
<p>We decided against it because:</p>
<ul class="simple">
<li>Supporting this would add complexity to the implementation of both
<code class="docutils literal notranslate"><span class="pre">&#64;override</span></code> and type checker support for it, so there would need to
be considerable benefits.</li>
<li>We believe that it would be rarely used and catch relatively few bugs.<ul>
<li>The author of the
<a class="reference external" href="https://pypi.org/project/overrides/">Overrides package</a> has
<a class="reference external" href="https://discuss.python.org/t/pep-698-a-typing-override-decorator/20839/4">noted</a>
that early versions of his library included this capability but it was
rarely useful and seemed to have little benefit. After it was removed, the
ability was never requested by users.</li>
</ul>
</li>
</ul>
</section>
</section>
<section id="reference-implementation">
<h2><a class="toc-backref" href="#reference-implementation" role="doc-backlink">Reference Implementation</a></h2>
<p>Pyre: A proof of concept is implemented in Pyre:</p>
<ul class="simple">
<li>The decorator
<a class="reference external" href="https://github.com/facebook/pyre-check/blob/f4d3f676d17b2e59c4c55d09dfa3caead8ec2e7c/pyre_extensions/__init__.py#L95/">&#64;pyre_extensions.override</a>
can mark overrides</li>
<li>Pyre can <a class="reference external" href="https://github.com/facebook/pyre-check/blob/ae68c44f4e5b263ce0e175f0798272d9318589af/source/analysis/test/integration/methodTest.ml#L2515-L2638/">type-check this decorator</a>
as specified in this PEP</li>
</ul>
</section>
<section id="copyright">
<h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2>
<p>This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.</p>
</section>
</section>
<hr class="docutils" />
<p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0698.rst">https://github.com/python/peps/blob/main/peps/pep-0698.rst</a></p>
<p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0698.rst">2024-02-15 19:03:31 GMT</a></p>
</article>
<nav id="pep-sidebar">
<h2>Contents</h2>
<ul>
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#motivation">Motivation</a><ul>
<li><a class="reference internal" href="#safe-refactoring">Safe Refactoring</a></li>
</ul>
</li>
<li><a class="reference internal" href="#rationale">Rationale</a><ul>
<li><a class="reference internal" href="#subclass-implementations-become-more-explicit">Subclass Implementations Become More Explicit</a></li>
<li><a class="reference internal" href="#precedent-in-other-languages-and-runtime-libraries">Precedent in Other Languages and Runtime Libraries</a><ul>
<li><a class="reference internal" href="#static-override-checks-in-other-languages">Static Override Checks in Other Languages</a></li>
<li><a class="reference internal" href="#runtime-override-checks-in-python">Runtime Override Checks in Python</a></li>
</ul>
</li>
<li><a class="reference internal" href="#disadvantages">Disadvantages</a></li>
</ul>
</li>
<li><a class="reference internal" href="#specification">Specification</a><ul>
<li><a class="reference internal" href="#no-new-rules-for-override-compatibility">No New Rules for Override Compatibility</a></li>
</ul>
</li>
<li><a class="reference internal" href="#strict-enforcement-per-project">Strict Enforcement Per-Project</a><ul>
<li><a class="reference internal" href="#id2">Motivation</a></li>
<li><a class="reference internal" href="#precedent">Precedent</a></li>
</ul>
</li>
<li><a class="reference internal" href="#backward-compatibility">Backward Compatibility</a></li>
<li><a class="reference internal" href="#runtime-behavior">Runtime Behavior</a><ul>
<li><a class="reference internal" href="#set-override-true-when-possible">Set <code class="docutils literal notranslate"><span class="pre">__override__</span> <span class="pre">=</span> <span class="pre">True</span></code> when possible</a></li>
<li><a class="reference internal" href="#limitations-of-setting-override">Limitations of setting <code class="docutils literal notranslate"><span class="pre">__override__</span></code></a></li>
</ul>
</li>
<li><a class="reference internal" href="#rejected-alternatives">Rejected Alternatives</a><ul>
<li><a class="reference internal" href="#rely-on-integrated-development-environments-for-safety">Rely on Integrated Development Environments for safety</a></li>
<li><a class="reference internal" href="#runtime-enforcement">Runtime enforcement</a></li>
<li><a class="reference internal" href="#mark-a-base-class-to-force-explicit-overrides-on-subclasses">Mark a base class to force explicit overrides on subclasses</a></li>
<li><a class="reference internal" href="#include-the-name-of-the-ancestor-class-being-overridden">Include the name of the ancestor class being overridden</a></li>
</ul>
</li>
<li><a class="reference internal" href="#reference-implementation">Reference Implementation</a></li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
<br>
<a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0698.rst">Page Source (GitHub)</a>
</nav>
</section>
<script src="../_static/colour_scheme.js"></script>
<script src="../_static/wrap_tables.js"></script>
<script src="../_static/sticky_banner.js"></script>
</body>
</html>