StarMirror
/
peps
mirror of https://github.com/python/peps
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
peps/pep-0517/index.html

1098 lines
96 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="color-scheme" content="light dark">
<title>PEP 517 – A build-system independent format for source trees | peps.python.org</title>
<link rel="shortcut icon" href="../_static/py.png">
<link rel="canonical" href="https://peps.python.org/pep-0517/">
<link rel="stylesheet" href="../_static/style.css" type="text/css">
<link rel="stylesheet" href="../_static/mq.css" type="text/css">
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" media="(prefers-color-scheme: light)" id="pyg-light">
<link rel="stylesheet" href="../_static/pygments_dark.css" type="text/css" media="(prefers-color-scheme: dark)" id="pyg-dark">
<link rel="alternate" type="application/rss+xml" title="Latest PEPs" href="https://peps.python.org/peps.rss">
<meta property="og:title" content='PEP 517 – A build-system independent format for source trees | peps.python.org'>
<meta property="og:type" content="website">
<meta property="og:url" content="https://peps.python.org/pep-0517/">
<meta property="og:site_name" content="Python Enhancement Proposals (PEPs)">
<meta property="og:image" content="https://peps.python.org/_static/og-image.png">
<meta property="og:image:alt" content="Python PEPs">
<meta property="og:image:width" content="200">
<meta property="og:image:height" content="200">
<meta name="description" content="Python Enhancement Proposals (PEPs)">
<meta name="theme-color" content="#3776ab">
</head>
<body>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-sun-half" viewBox="0 0 24 24" pointer-events="all">
<title>Following system colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="9"></circle>
<path d="M12 3v18m0-12l4.65-4.65M12 14.3l7.37-7.37M12 19.6l8.85-8.85"></path>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24" pointer-events="all">
<title>Selected dark colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path stroke="none" d="M0 0h24v24H0z" fill="none"></path>
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z"></path>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24" pointer-events="all">
<title>Selected light colour scheme</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none"
stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
</svg>
<script>
document.documentElement.dataset.colour_scheme = localStorage.getItem("colour_scheme") || "auto"
</script>
<section id="pep-page-section">
<header>
<h1>Python Enhancement Proposals</h1>
<ul class="breadcrumbs">
<li><a href="https://www.python.org/" title="The Python Programming Language">Python</a> &raquo; </li>
<li><a href="../pep-0000/">PEP Index</a> &raquo; </li>
<li>PEP 517</li>
</ul>
<button id="colour-scheme-cycler" onClick="setColourScheme(nextColourScheme())">
<svg aria-hidden="true" class="colour-scheme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg aria-hidden="true" class="colour-scheme-icon-when-light"><use href="#svg-sun"></use></svg>
<span class="visually-hidden">Toggle light / dark / auto colour theme</span>
</button>
</header>
<article>
<section id="pep-content">
<h1 class="page-title">PEP 517 – A build-system independent format for source trees</h1>
<dl class="rfc2822 field-list simple">
<dt class="field-odd">Author<span class="colon">:</span></dt>
<dd class="field-odd">Nathaniel J. Smith &lt;njs&#32;&#97;t&#32;pobox.com&gt;,
Thomas Kluyver &lt;thomas&#32;&#97;t&#32;kluyver.me.uk&gt;</dd>
<dt class="field-even">BDFL-Delegate<span class="colon">:</span></dt>
<dd class="field-even">Alyssa Coghlan &lt;ncoghlan&#32;&#97;t&#32;gmail.com&gt;</dd>
<dt class="field-odd">Discussions-To<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="https://mail.python.org/archives/list/distutils-sig&#64;python.org/">Distutils-SIG list</a></dd>
<dt class="field-even">Status<span class="colon">:</span></dt>
<dd class="field-even"><abbr title="Accepted and implementation complete, or no longer active">Final</abbr></dd>
<dt class="field-odd">Type<span class="colon">:</span></dt>
<dd class="field-odd"><abbr title="Normative PEP with a new feature for Python, implementation change for CPython or interoperability standard for the ecosystem">Standards Track</abbr></dd>
<dt class="field-even">Topic<span class="colon">:</span></dt>
<dd class="field-even"><a class="reference external" href="../topic/packaging/">Packaging</a></dd>
<dt class="field-odd">Created<span class="colon">:</span></dt>
<dd class="field-odd">30-Sep-2015</dd>
<dt class="field-even">Post-History<span class="colon">:</span></dt>
<dd class="field-even">01-Oct-2015, 25-Oct-2015, 19-May-2017, 11-Sep-2017</dd>
<dt class="field-odd">Resolution<span class="colon">:</span></dt>
<dd class="field-odd"><a class="reference external" href="https://mail.python.org/pipermail/distutils-sig/2017-September/031548.html">Distutils-SIG message</a></dd>
</dl>
<hr class="docutils" />
<section id="contents">
<details><summary>Table of Contents</summary><ul class="simple">
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#terminology-and-goals">Terminology and goals</a></li>
<li><a class="reference internal" href="#source-trees">Source trees</a><ul>
<li><a class="reference internal" href="#build-requirements">Build requirements</a></li>
</ul>
</li>
<li><a class="reference internal" href="#build-backend-interface">Build backend interface</a><ul>
<li><a class="reference internal" href="#mandatory-hooks">Mandatory hooks</a><ul>
<li><a class="reference internal" href="#build-wheel">build_wheel</a></li>
<li><a class="reference internal" href="#build-sdist">build_sdist</a></li>
</ul>
</li>
<li><a class="reference internal" href="#optional-hooks">Optional hooks</a><ul>
<li><a class="reference internal" href="#get-requires-for-build-wheel">get_requires_for_build_wheel</a></li>
<li><a class="reference internal" href="#prepare-metadata-for-build-wheel">prepare_metadata_for_build_wheel</a></li>
<li><a class="reference internal" href="#get-requires-for-build-sdist">get_requires_for_build_sdist</a></li>
</ul>
</li>
<li><a class="reference internal" href="#config-settings">Config settings</a></li>
<li><a class="reference internal" href="#build-environment">Build environment</a><ul>
<li><a class="reference internal" href="#recommendations-for-build-frontends-non-normative">Recommendations for build frontends (non-normative)</a></li>
</ul>
</li>
<li><a class="reference internal" href="#in-tree-build-backends">In-tree build backends</a></li>
</ul>
</li>
<li><a class="reference internal" href="#source-distributions">Source distributions</a></li>
<li><a class="reference internal" href="#evolutionary-notes">Evolutionary notes</a></li>
<li><a class="reference internal" href="#rejected-options">Rejected options</a></li>
<li><a class="reference internal" href="#summary-of-changes-to-pep-517">Summary of changes to PEP 517</a></li>
<li><a class="reference internal" href="#appendix-a-comparison-to-pep-516">Appendix A: Comparison to PEP 516</a><ul>
<li><a class="reference internal" href="#other-differences">Other differences</a></li>
</ul>
</li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
</details></section>
<section id="abstract">
<h2><a class="toc-backref" href="#abstract" role="doc-backlink">Abstract</a></h2>
<p>While <code class="docutils literal notranslate"><span class="pre">distutils</span></code> / <code class="docutils literal notranslate"><span class="pre">setuptools</span></code> have taken us a long way, they
suffer from three serious problems: (a) they’re missing important
features like usable build-time dependency declaration,
autoconfiguration, and even basic ergonomic niceties like <a class="reference external" href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself">DRY</a>-compliant
version number management, and (b) extending them is difficult, so
while there do exist various solutions to the above problems, they’re
often quirky, fragile, and expensive to maintain, and yet (c) it’s
very difficult to use anything else, because distutils/setuptools
provide the standard interface for installing packages expected by
both users and installation tools like <code class="docutils literal notranslate"><span class="pre">pip</span></code>.</p>
<p>Previous efforts (e.g. distutils2 or setuptools itself) have attempted
to solve problems (a) and/or (b). This proposal aims to solve (c).</p>
<p>The goal of this PEP is get distutils-sig out of the business of being
a gatekeeper for Python build systems. If you want to use distutils,
great; if you want to use something else, then that should be easy to
do using standardized methods. The difficulty of interfacing with
distutils means that there aren’t many such systems right now, but to
give a sense of what we’re thinking about see <a class="reference external" href="https://github.com/takluyver/flit">flit</a> or <a class="reference external" href="https://cournape.github.io/Bento/">bento</a>. Fortunately, wheels have now
solved many of the hard problems here – e.g. it’s no longer necessary
that a build system also know about every possible installation
configuration – so pretty much all we really need from a build system
is that it have some way to spit out standard-compliant wheels and
sdists.</p>
<p>We therefore propose a new, relatively minimal interface for
installation tools like <code class="docutils literal notranslate"><span class="pre">pip</span></code> to interact with package source trees
and source distributions.</p>
</section>
<section id="terminology-and-goals">
<h2><a class="toc-backref" href="#terminology-and-goals" role="doc-backlink">Terminology and goals</a></h2>
<p>A <em>source tree</em> is something like a VCS checkout. We need a standard
interface for installing from this format, to support usages like
<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">some-directory/</span></code>.</p>
<p>A <em>source distribution</em> is a static snapshot representing a particular
release of some source code, like <code class="docutils literal notranslate"><span class="pre">lxml-3.4.4.tar.gz</span></code>. Source
distributions serve many purposes: they form an archival record of
releases, they provide a stupid-simple de facto standard for tools
that want to ingest and process large corpora of code, possibly
written in many languages (e.g. code search), they act as the input to
downstream packaging systems like Debian/Fedora/Conda/…, and so
forth. In the Python ecosystem they additionally have a particularly
important role to play, because packaging tools like <code class="docutils literal notranslate"><span class="pre">pip</span></code> are able
to use source distributions to fulfill binary dependencies, e.g. if
there is a distribution <code class="docutils literal notranslate"><span class="pre">foo.whl</span></code> which declares a dependency on
<code class="docutils literal notranslate"><span class="pre">bar</span></code>, then we need to support the case where <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">bar</span></code> or
<code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">foo</span></code> automatically locates the sdist for <code class="docutils literal notranslate"><span class="pre">bar</span></code>,
downloads it, builds it, and installs the resulting package.</p>
<p>Source distributions are also known as <em>sdists</em> for short.</p>
<p>A <em>build frontend</em> is a tool that users might run that takes arbitrary
source trees or source distributions and builds wheels from them. The
actual building is done by each source tree’s <em>build backend</em>. In a
command like <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">wheel</span> <span class="pre">some-directory/</span></code>, pip is acting as a build
frontend.</p>
<p>An <em>integration frontend</em> is a tool that users might run that takes a
set of package requirements (e.g. a requirements.txt file) and
attempts to update a working environment to satisfy those
requirements. This may require locating, building, and installing a
combination of wheels and sdists. In a command like <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span>
<span class="pre">lxml==2.4.0</span></code>, pip is acting as an integration frontend.</p>
</section>
<section id="source-trees">
<h2><a class="toc-backref" href="#source-trees" role="doc-backlink">Source trees</a></h2>
<p>There is an existing, legacy source tree format involving
<code class="docutils literal notranslate"><span class="pre">setup.py</span></code>. We don’t try to specify it further; its de facto
specification is encoded in the source code and documentation of
<code class="docutils literal notranslate"><span class="pre">distutils</span></code>, <code class="docutils literal notranslate"><span class="pre">setuptools</span></code>, <code class="docutils literal notranslate"><span class="pre">pip</span></code>, and other tools. We’ll refer
to it as the <code class="docutils literal notranslate"><span class="pre">setup.py</span></code>-style.</p>
<p>Here we define a new style of source tree based around the
<code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file defined in <a class="pep reference internal" href="../pep-0518/" title="PEP 518 – Specifying Minimum Build System Requirements for Python Projects">PEP 518</a>, extending the
<code class="docutils literal notranslate"><span class="pre">[build-system]</span></code> table in that file with one additional key,
<code class="docutils literal notranslate"><span class="pre">build-backend</span></code>. Here’s an example of how it would look:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">build</span><span class="o">-</span><span class="n">system</span><span class="p">]</span>
<span class="c1"># Defined by PEP 518:</span>
<span class="n">requires</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;flit&quot;</span><span class="p">]</span>
<span class="c1"># Defined by this PEP:</span>
<span class="n">build</span><span class="o">-</span><span class="n">backend</span> <span class="o">=</span> <span class="s2">&quot;flit.api:main&quot;</span>
</pre></div>
</div>
<p><code class="docutils literal notranslate"><span class="pre">build-backend</span></code> is a string naming a Python object that will be
used to perform the build (see below for details). This is formatted
following the same <code class="docutils literal notranslate"><span class="pre">module:object</span></code> syntax as a <code class="docutils literal notranslate"><span class="pre">setuptools</span></code> entry
point. For instance, if the string is <code class="docutils literal notranslate"><span class="pre">&quot;flit.api:main&quot;</span></code> as in the
example above, this object would be looked up by executing the
equivalent of:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">flit.api</span>
<span class="n">backend</span> <span class="o">=</span> <span class="n">flit</span><span class="o">.</span><span class="n">api</span><span class="o">.</span><span class="n">main</span>
</pre></div>
</div>
<p>It’s also legal to leave out the <code class="docutils literal notranslate"><span class="pre">:object</span></code> part, e.g.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">build</span><span class="o">-</span><span class="n">backend</span> <span class="o">=</span> <span class="s2">&quot;flit.api&quot;</span>
</pre></div>
</div>
<p>which acts like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">flit.api</span>
<span class="n">backend</span> <span class="o">=</span> <span class="n">flit</span><span class="o">.</span><span class="n">api</span>
</pre></div>
</div>
<p>Formally, the string should satisfy this grammar:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>identifier = (letter | &#39;_&#39;) (letter | &#39;_&#39; | digit)*
module_path = identifier (&#39;.&#39; identifier)*
object_path = identifier (&#39;.&#39; identifier)*
entry_point = module_path (&#39;:&#39; object_path)?
</pre></div>
</div>
<p>And we import <code class="docutils literal notranslate"><span class="pre">module_path</span></code> and then lookup
<code class="docutils literal notranslate"><span class="pre">module_path.object_path</span></code> (or just <code class="docutils literal notranslate"><span class="pre">module_path</span></code> if
<code class="docutils literal notranslate"><span class="pre">object_path</span></code> is missing).</p>
<p>When importing the module path, we do <em>not</em> look in the directory containing the
source tree, unless that would be on <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> anyway (e.g. because it is
specified in PYTHONPATH). Although Python automatically adds the working
directory to <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> in some situations, code to resolve the backend should
not be affected by this.</p>
<p>If the <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file is absent, or the <code class="docutils literal notranslate"><span class="pre">build-backend</span></code>
key is missing, the source tree is not using this specification, and
tools should revert to the legacy behaviour of running <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> (either
directly, or by implicitly invoking the <code class="docutils literal notranslate"><span class="pre">setuptools.build_meta:__legacy__</span></code>
backend).</p>
<p>Where the <code class="docutils literal notranslate"><span class="pre">build-backend</span></code> key exists, this takes precedence and the source tree follows the format and
conventions of the specified backend (as such no <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> is needed unless the backend requires it).
Projects may still wish to include a <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> for compatibility with tools that do not use this spec.</p>
<p>This PEP also defines a <code class="docutils literal notranslate"><span class="pre">backend-path</span></code> key for use in <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code>, see
the “In-Tree Build Backends” section below. This key would be used as follows:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">[</span><span class="n">build</span><span class="o">-</span><span class="n">system</span><span class="p">]</span>
<span class="c1"># Defined by PEP 518:</span>
<span class="n">requires</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;flit&quot;</span><span class="p">]</span>
<span class="c1"># Defined by this PEP:</span>
<span class="n">build</span><span class="o">-</span><span class="n">backend</span> <span class="o">=</span> <span class="s2">&quot;local_backend&quot;</span>
<span class="n">backend</span><span class="o">-</span><span class="n">path</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;backend&quot;</span><span class="p">]</span>
</pre></div>
</div>
<section id="build-requirements">
<h3><a class="toc-backref" href="#build-requirements" role="doc-backlink">Build requirements</a></h3>
<p>This PEP places a number of additional requirements on the “build requirements”
section of <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code>. These are intended to ensure that projects do
not create impossible to satisfy conditions with their build requirements.</p>
<ul class="simple">
<li>Project build requirements will define a directed graph of requirements
(project A needs B to build, B needs C and D, etc.) This graph MUST NOT
contain cycles. If (due to lack of co-ordination between projects, for
example) a cycle is present, front ends MAY refuse to build the project.</li>
<li>Where build requirements are available as wheels, front ends SHOULD use these
where practical, to avoid deeply nested builds. However front ends MAY have
modes where they do not consider wheels when locating build requirements, and
so projects MUST NOT assume that publishing wheels is sufficient to break a
requirement cycle.</li>
<li>Front ends SHOULD check explicitly for requirement cycles, and terminate
the build with an informative message if one is found.</li>
</ul>
<p>Note in particular that the requirement for no requirement cycles means that
backends wishing to self-host (i.e., building a wheel for a backend uses that
backend for the build) need to make special provision to avoid causing cycles.
Typically this will involve specifying themselves as an in-tree backend, and
avoiding external build dependencies (usually by vendoring them).</p>
</section>
</section>
<section id="build-backend-interface">
<h2><a class="toc-backref" href="#build-backend-interface" role="doc-backlink">Build backend interface</a></h2>
<p>The build backend object is expected to have attributes which provide
some or all of the following hooks. The common <code class="docutils literal notranslate"><span class="pre">config_settings</span></code>
argument is described after the individual hooks.</p>
<section id="mandatory-hooks">
<h3><a class="toc-backref" href="#mandatory-hooks" role="doc-backlink">Mandatory hooks</a></h3>
<section id="build-wheel">
<h4><a class="toc-backref" href="#build-wheel" role="doc-backlink">build_wheel</a></h4>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">build_wheel</span><span class="p">(</span><span class="n">wheel_directory</span><span class="p">,</span> <span class="n">config_settings</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">metadata_directory</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
</div>
<p>Must build a .whl file, and place it in the specified <code class="docutils literal notranslate"><span class="pre">wheel_directory</span></code>. It
must return the basename (not the full path) of the <code class="docutils literal notranslate"><span class="pre">.whl</span></code> file it creates,
as a unicode string.</p>
<p>If the build frontend has previously called <code class="docutils literal notranslate"><span class="pre">prepare_metadata_for_build_wheel</span></code>
and depends on the wheel resulting from this call to have metadata
matching this earlier call, then it should provide the path to the created
<code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory as the <code class="docutils literal notranslate"><span class="pre">metadata_directory</span></code> argument. If this
argument is provided, then <code class="docutils literal notranslate"><span class="pre">build_wheel</span></code> MUST produce a wheel with identical
metadata. The directory passed in by the build frontend MUST be
identical to the directory created by <code class="docutils literal notranslate"><span class="pre">prepare_metadata_for_build_wheel</span></code>,
including any unrecognized files it created.</p>
<p>Backends which do not provide the <code class="docutils literal notranslate"><span class="pre">prepare_metadata_for_build_wheel</span></code> hook may
either silently ignore the <code class="docutils literal notranslate"><span class="pre">metadata_directory</span></code> parameter to <code class="docutils literal notranslate"><span class="pre">build_wheel</span></code>,
or else raise an exception when it is set to anything other than <code class="docutils literal notranslate"><span class="pre">None</span></code>.</p>
<p>To ensure that wheels from different sources are built the same way, frontends
may call <code class="docutils literal notranslate"><span class="pre">build_sdist</span></code> first, and then call <code class="docutils literal notranslate"><span class="pre">build_wheel</span></code> in the unpacked
sdist. But if the backend indicates that it is missing some requirements for
creating an sdist (see below), the frontend will fall back to calling
<code class="docutils literal notranslate"><span class="pre">build_wheel</span></code> in the source directory.</p>
<p>The source directory may be read-only. Backends should therefore be
prepared to build without creating or modifying any files in the source
directory, but they may opt not to handle this case, in which case
failures will be visible to the user. Frontends are not responsible for
any special handling of read-only source directories.</p>
<p>The backend may store intermediate artifacts in cache locations or
temporary directories. The presence or absence of any caches should not
make a material difference to the final result of the build.</p>
</section>
<section id="build-sdist">
<h4><a class="toc-backref" href="#build-sdist" role="doc-backlink">build_sdist</a></h4>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">build_sdist</span><span class="p">(</span><span class="n">sdist_directory</span><span class="p">,</span> <span class="n">config_settings</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
</div>
<p>Must build a .tar.gz source distribution and place it in the specified
<code class="docutils literal notranslate"><span class="pre">sdist_directory</span></code>. It must return the basename (not the full path) of the
<code class="docutils literal notranslate"><span class="pre">.tar.gz</span></code> file it creates, as a unicode string.</p>
<p>A .tar.gz source distribution (sdist) contains a single top-level directory called
<code class="docutils literal notranslate"><span class="pre">{name}-{version}</span></code> (e.g. <code class="docutils literal notranslate"><span class="pre">foo-1.0</span></code>), containing the source files of the
package. This directory must also contain the
<code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> from the build directory, and a PKG-INFO file containing
metadata in the format described in
<a class="pep reference internal" href="../pep-0345/" title="PEP 345 – Metadata for Python Software Packages 1.2">PEP 345</a>. Although historically
zip files have also been used as sdists, this hook should produce a gzipped
tarball. This is already the more common format for sdists, and having a
consistent format makes for simpler tooling.</p>
<p>The generated tarball should use the modern POSIX.1-2001 pax tar format, which
specifies UTF-8 based file names. This is not yet the default for the tarfile
module shipped with Python 3.6, so backends using the tarfile module need to
explicitly pass <code class="docutils literal notranslate"><span class="pre">format=tarfile.PAX_FORMAT</span></code>.</p>
<p>Some backends may have extra requirements for creating sdists, such as version
control tools. However, some frontends may prefer to make intermediate sdists
when producing wheels, to ensure consistency.
If the backend cannot produce an sdist because a dependency is missing, or
for another well understood reason, it should raise an exception of a specific
type which it makes available as <code class="docutils literal notranslate"><span class="pre">UnsupportedOperation</span></code> on the backend object.
If the frontend gets this exception while building an sdist as an intermediate
for a wheel, it should fall back to building a wheel directly.
The backend does not need to define this exception type if it would never raise
it.</p>
</section>
</section>
<section id="optional-hooks">
<h3><a class="toc-backref" href="#optional-hooks" role="doc-backlink">Optional hooks</a></h3>
<section id="get-requires-for-build-wheel">
<h4><a class="toc-backref" href="#get-requires-for-build-wheel" role="doc-backlink">get_requires_for_build_wheel</a></h4>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_requires_for_build_wheel</span><span class="p">(</span><span class="n">config_settings</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
</div>
<p>This hook MUST return an additional list of strings containing <a class="pep reference internal" href="../pep-0508/" title="PEP 508 – Dependency specification for Python Software Packages">PEP 508</a>
dependency specifications, above and beyond those specified in the
<code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file, to be installed when calling the <code class="docutils literal notranslate"><span class="pre">build_wheel</span></code> or
<code class="docutils literal notranslate"><span class="pre">prepare_metadata_for_build_wheel</span></code> hooks.</p>
<p>Example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_requires_for_build_wheel</span><span class="p">(</span><span class="n">config_settings</span><span class="p">):</span>
<span class="k">return</span> <span class="p">[</span><span class="s2">&quot;wheel &gt;= 0.25&quot;</span><span class="p">,</span> <span class="s2">&quot;setuptools&quot;</span><span class="p">]</span>
</pre></div>
</div>
<p>If not defined, the default implementation is equivalent to <code class="docutils literal notranslate"><span class="pre">return</span> <span class="pre">[]</span></code>.</p>
</section>
<section id="prepare-metadata-for-build-wheel">
<h4><a class="toc-backref" href="#prepare-metadata-for-build-wheel" role="doc-backlink">prepare_metadata_for_build_wheel</a></h4>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">prepare_metadata_for_build_wheel</span><span class="p">(</span><span class="n">metadata_directory</span><span class="p">,</span> <span class="n">config_settings</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
</div>
<p>Must create a <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory containing wheel metadata
inside the specified <code class="docutils literal notranslate"><span class="pre">metadata_directory</span></code> (i.e., creates a directory
like <code class="docutils literal notranslate"><span class="pre">{metadata_directory}/{package}-{version}.dist-info/</span></code>). This
directory MUST be a valid <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code> directory as defined in the
wheel specification, except that it need not contain <code class="docutils literal notranslate"><span class="pre">RECORD</span></code> or
signatures. The hook MAY also create other files inside this
directory, and a build frontend MUST preserve, but otherwise ignore, such files;
the intention
here is that in cases where the metadata depends on build-time
decisions, the build backend may need to record these decisions in
some convenient format for re-use by the actual wheel-building step.</p>
<p>This must return the basename (not the full path) of the <code class="docutils literal notranslate"><span class="pre">.dist-info</span></code>
directory it creates, as a unicode string.</p>
<p>If a build frontend needs this information and the method is
not defined, it should call <code class="docutils literal notranslate"><span class="pre">build_wheel</span></code> and look at the resulting
metadata directly.</p>
</section>
<section id="get-requires-for-build-sdist">
<h4><a class="toc-backref" href="#get-requires-for-build-sdist" role="doc-backlink">get_requires_for_build_sdist</a></h4>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">get_requires_for_build_sdist</span><span class="p">(</span><span class="n">config_settings</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
</div>
<p>This hook MUST return an additional list of strings containing <a class="pep reference internal" href="../pep-0508/" title="PEP 508 – Dependency specification for Python Software Packages">PEP 508</a>
dependency specifications, above and beyond those specified in the
<code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file. These dependencies will be installed when calling the
<code class="docutils literal notranslate"><span class="pre">build_sdist</span></code> hook.</p>
<p>If not defined, the default implementation is equivalent to <code class="docutils literal notranslate"><span class="pre">return</span> <span class="pre">[]</span></code>.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Editable installs</p>
<p>This PEP originally specified another hook, <code class="docutils literal notranslate"><span class="pre">install_editable</span></code>, to do an
editable install (as with <code class="docutils literal notranslate"><span class="pre">pip</span> <span class="pre">install</span> <span class="pre">-e</span></code>). It was removed due to the
complexity of the topic, but may be specified in a later PEP.</p>
<p>Briefly, the questions to be answered include: what reasonable ways existing
of implementing an ‘editable install’? Should the backend or the frontend
pick how to make an editable install? And if the frontend does, what does it
need from the backend to do so.</p>
</div>
</section>
</section>
<section id="config-settings">
<h3><a class="toc-backref" href="#config-settings" role="doc-backlink">Config settings</a></h3>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">config_settings</span>
</pre></div>
</div>
<p>This argument, which is passed to all hooks, is an arbitrary
dictionary provided as an “escape hatch” for users to pass ad-hoc
configuration into individual package builds. Build backends MAY
assign any semantics they like to this dictionary. Build frontends
SHOULD provide some mechanism for users to specify arbitrary
string-key/string-value pairs to be placed in this dictionary.
For example, they might support some syntax like <code class="docutils literal notranslate"><span class="pre">--package-config</span> <span class="pre">CC=gcc</span></code>.
In case a user provides duplicate string-keys, build frontends SHOULD
combine the corresponding string-values into a list of strings.
Build frontends MAY also provide arbitrary other mechanisms
for users to place entries in this dictionary. For example, <code class="docutils literal notranslate"><span class="pre">pip</span></code>
might choose to map a mix of modern and legacy command line arguments
like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pip</span> <span class="n">install</span> \
<span class="o">--</span><span class="n">package</span><span class="o">-</span><span class="n">config</span> <span class="n">CC</span><span class="o">=</span><span class="n">gcc</span> \
<span class="o">--</span><span class="k">global</span><span class="o">-</span><span class="n">option</span><span class="o">=</span><span class="s2">&quot;--some-global-option&quot;</span> \
<span class="o">--</span><span class="n">build</span><span class="o">-</span><span class="n">option</span><span class="o">=</span><span class="s2">&quot;--build-option1&quot;</span> \
<span class="o">--</span><span class="n">build</span><span class="o">-</span><span class="n">option</span><span class="o">=</span><span class="s2">&quot;--build-option2&quot;</span>
</pre></div>
</div>
<p>into a <code class="docutils literal notranslate"><span class="pre">config_settings</span></code> dictionary like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="p">{</span>
<span class="s2">&quot;CC&quot;</span><span class="p">:</span> <span class="s2">&quot;gcc&quot;</span><span class="p">,</span>
<span class="s2">&quot;--global-option&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;--some-global-option&quot;</span><span class="p">],</span>
<span class="s2">&quot;--build-option&quot;</span><span class="p">:</span> <span class="p">[</span><span class="s2">&quot;--build-option1&quot;</span><span class="p">,</span> <span class="s2">&quot;--build-option2&quot;</span><span class="p">],</span>
<span class="p">}</span>
</pre></div>
</div>
<p>Of course, it’s up to users to make sure that they pass options which
make sense for the particular build backend and package that they are
building.</p>
<p>The hooks may be called with positional or keyword arguments, so backends
implementing them should be careful to make sure that their signatures match
both the order and the names of the arguments above.</p>
<p>All hooks are run with working directory set to the root of the source
tree, and MAY print arbitrary informational text on stdout and
stderr. They MUST NOT read from stdin, and the build frontend MAY
close stdin before invoking the hooks.</p>
<p>The build frontend may capture stdout and/or stderr from the backend. If the
backend detects that an output stream is not a terminal/console (e.g.
<code class="docutils literal notranslate"><span class="pre">not</span> <span class="pre">sys.stdout.isatty()</span></code>), it SHOULD ensure that any output it writes to that
stream is UTF-8 encoded. The build frontend MUST NOT fail if captured output is
not valid UTF-8, but it MAY not preserve all the information in that case (e.g.
it may decode using the <em>replace</em> error handler in Python). If the output stream
is a terminal, the build backend is responsible for presenting its output
accurately, as for any program running in a terminal.</p>
<p>If a hook raises an exception, or causes the process to terminate,
then this indicates an error.</p>
</section>
<section id="build-environment">
<h3><a class="toc-backref" href="#build-environment" role="doc-backlink">Build environment</a></h3>
<p>One of the responsibilities of a build frontend is to set up the
Python environment in which the build backend will run.</p>
<p>We do not require that any particular “virtual environment” mechanism
be used; a build frontend might use virtualenv, or venv, or no special
mechanism at all. But whatever mechanism is used MUST meet the
following criteria:</p>
<ul>
<li>All requirements specified by the project’s build-requirements must
be available for import from Python. In particular:<ul class="simple">
<li>The <code class="docutils literal notranslate"><span class="pre">get_requires_for_build_wheel</span></code> and <code class="docutils literal notranslate"><span class="pre">get_requires_for_build_sdist</span></code> hooks are
executed in an environment which contains the bootstrap requirements
specified in the <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file.</li>
<li>The <code class="docutils literal notranslate"><span class="pre">prepare_metadata_for_build_wheel</span></code> and <code class="docutils literal notranslate"><span class="pre">build_wheel</span></code> hooks are
executed in an environment which contains the
bootstrap requirements from <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> and those specified by the
<code class="docutils literal notranslate"><span class="pre">get_requires_for_build_wheel</span></code> hook.</li>
<li>The <code class="docutils literal notranslate"><span class="pre">build_sdist</span></code> hook is executed in an environment which contains the
bootstrap requirements from <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> and those specified by the
<code class="docutils literal notranslate"><span class="pre">get_requires_for_build_sdist</span></code> hook.</li>
</ul>
</li>
<li>This must remain true even for new Python subprocesses spawned by
the build environment, e.g. code like:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">sys</span><span class="o">,</span> <span class="nn">subprocess</span>
<span class="n">subprocess</span><span class="o">.</span><span class="n">check_call</span><span class="p">([</span><span class="n">sys</span><span class="o">.</span><span class="n">executable</span><span class="p">,</span> <span class="o">...</span><span class="p">])</span>
</pre></div>
</div>
<p>must spawn a Python process which has access to all the project’s
build-requirements. This is necessary e.g. for build backends that
want to run legacy <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> scripts in a subprocess.</p>
</li>
<li>All command-line scripts provided by the build-required packages
must be present in the build environment’s PATH. For example, if a
project declares a build-requirement on <a class="reference external" href="https://flit.readthedocs.org/en/latest/">flit</a>, then the following must
work as a mechanism for running the flit command-line tool:<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">subprocess</span>
<span class="kn">import</span> <span class="nn">shutil</span>
<span class="n">subprocess</span><span class="o">.</span><span class="n">check_call</span><span class="p">([</span><span class="n">shutil</span><span class="o">.</span><span class="n">which</span><span class="p">(</span><span class="s2">&quot;flit&quot;</span><span class="p">),</span> <span class="o">...</span><span class="p">])</span>
</pre></div>
</div>
</li>
</ul>
<p>A build backend MUST be prepared to function in any environment which
meets the above criteria. In particular, it MUST NOT assume that it
has access to any packages except those that are present in the
stdlib, or that are explicitly declared as build-requirements.</p>
<p>Frontends should call each hook in a fresh subprocess, so that backends are
free to change process global state (such as environment variables or the
working directory). A Python library will be provided which frontends can use
to easily call hooks this way.</p>
<section id="recommendations-for-build-frontends-non-normative">
<h4><a class="toc-backref" href="#recommendations-for-build-frontends-non-normative" role="doc-backlink">Recommendations for build frontends (non-normative)</a></h4>
<p>A build frontend MAY use any mechanism for setting up a build
environment that meets the above criteria. For example, simply
installing all build-requirements into the global environment would be
sufficient to build any compliant package – but this would be
sub-optimal for a number of reasons. This section contains
non-normative advice to frontend implementors.</p>
<p>A build frontend SHOULD, by default, create an isolated environment
for each build, containing only the standard library and any
explicitly requested build-dependencies. This has two benefits:</p>
<ul class="simple">
<li>It allows for a single installation run to build multiple packages
that have contradictory build-requirements. E.g. if package1
build-requires pbr==1.8.1, and package2 build-requires pbr==1.7.2,
then these cannot both be installed simultaneously into the global
environment – which is a problem when the user requests <code class="docutils literal notranslate"><span class="pre">pip</span>
<span class="pre">install</span> <span class="pre">package1</span> <span class="pre">package2</span></code>. Or if the user already has pbr==1.8.1
installed in their global environment, and a package build-requires
pbr==1.7.2, then downgrading the user’s version would be rather
rude.</li>
<li>It acts as a kind of public health measure to maximize the number of
packages that actually do declare accurate build-dependencies. We
can write all the strongly worded admonitions to package authors we
want, but if build frontends don’t enforce isolation by default,
then we’ll inevitably end up with lots of packages on PyPI that
build fine on the original author’s machine and nowhere else, which
is a headache that no-one needs.</li>
</ul>
<p>However, there will also be situations where build-requirements are
problematic in various ways. For example, a package author might
accidentally leave off some crucial requirement despite our best
efforts; or, a package might declare a build-requirement on <code class="docutils literal notranslate"><span class="pre">foo</span> <span class="pre">&gt;=</span>
<span class="pre">1.0</span></code> which worked great when 1.0 was the latest version, but now 1.1
is out and it has a showstopper bug; or, the user might decide to
build a package against numpy==1.7 – overriding the package’s
preferred numpy==1.8 – to guarantee that the resulting build will be
compatible at the C ABI level with an older version of numpy (even if
this means the resulting build is unsupported upstream). Therefore,
build frontends SHOULD provide some mechanism for users to override
the above defaults. For example, a build frontend could have a
<code class="docutils literal notranslate"><span class="pre">--build-with-system-site-packages</span></code> option that causes the
<code class="docutils literal notranslate"><span class="pre">--system-site-packages</span></code> option to be passed to
virtualenv-or-equivalent when creating build environments, or a
<code class="docutils literal notranslate"><span class="pre">--build-requirements-override=my-requirements.txt</span></code> option that
overrides the project’s normal build-requirements.</p>
<p>The general principle here is that we want to enforce hygiene on
package <em>authors</em>, while still allowing <em>end-users</em> to open up the
hood and apply duct tape when necessary.</p>
</section>
</section>
<section id="in-tree-build-backends">
<h3><a class="toc-backref" href="#in-tree-build-backends" role="doc-backlink">In-tree build backends</a></h3>
<p>In certain circumstances, projects may wish to include the source code for the
build backend directly in the source tree, rather than referencing the backend
via the <code class="docutils literal notranslate"><span class="pre">requires</span></code> key. Two specific situations where this would be expected
are:</p>
<ul class="simple">
<li>Backends themselves, which want to use their own features for building
themselves (“self-hosting backends”)</li>
<li>Project-specific backends, typically consisting of a custom wrapper around a
standard backend, where the wrapper is too project-specific to be worth
distributing independently (“in-tree backends”)</li>
</ul>
<p>Projects can specify that their backend code is hosted in-tree by including the
<code class="docutils literal notranslate"><span class="pre">backend-path</span></code> key in <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code>. This key contains a list of
directories, which the frontend will add to the start of <code class="docutils literal notranslate"><span class="pre">sys.path</span></code> when
loading the backend, and running the backend hooks.</p>
<p>There are two restrictions on the content of the <code class="docutils literal notranslate"><span class="pre">backend-path</span></code> key:</p>
<ul class="simple">
<li>Directories in <code class="docutils literal notranslate"><span class="pre">backend-path</span></code> are interpreted as relative to the project
root, and MUST refer to a location within the source tree (after relative
paths and symbolic links have been resolved).</li>
<li>The backend code MUST be loaded from one of the directories specified in
<code class="docutils literal notranslate"><span class="pre">backend-path</span></code> (i.e., it is not permitted to specify <code class="docutils literal notranslate"><span class="pre">backend-path</span></code> and
<em>not</em> have in-tree backend code).</li>
</ul>
<p>The first restriction is to ensure that source trees remain self-contained,
and cannot refer to locations outside of the source tree. Frontends SHOULD
check this condition (typically by resolving the location to an absolute path
and resolving symbolic links, and then checking it against the project root),
and fail with an error message if it is violated.</p>
<p>The <code class="docutils literal notranslate"><span class="pre">backend-path</span></code> feature is intended to support the implementation of
in-tree backends, and not to allow configuration of existing backends. The
second restriction above is specifically to ensure that this is how the feature
is used. Front ends MAY enforce this check, but are not required to. Doing so
would typically involve checking the backend’s <code class="docutils literal notranslate"><span class="pre">__file__</span></code> attribute against
the locations in <code class="docutils literal notranslate"><span class="pre">backend-path</span></code>.</p>
</section>
</section>
<section id="source-distributions">
<h2><a class="toc-backref" href="#source-distributions" role="doc-backlink">Source distributions</a></h2>
<p>We continue with the legacy sdist format, adding some new restrictions.
This format is mostly
undefined, but basically comes down to: a file named
<code class="docutils literal notranslate"><span class="pre">{NAME}-{VERSION}.{EXT}</span></code>, which unpacks into a buildable source tree
called <code class="docutils literal notranslate"><span class="pre">{NAME}-{VERSION}/</span></code>. Traditionally these have always
contained <code class="docutils literal notranslate"><span class="pre">setup.py</span></code>-style source trees; we now allow them to also
contain <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code>-style source trees.</p>
<p>Integration frontends require that an sdist named
<code class="docutils literal notranslate"><span class="pre">{NAME}-{VERSION}.{EXT}</span></code> will generate a wheel named
<code class="docutils literal notranslate"><span class="pre">{NAME}-{VERSION}-{COMPAT-INFO}.whl</span></code>.</p>
<p>The new restrictions for sdists built by <a class="pep reference internal" href="../pep-0517/" title="PEP 517 – A build-system independent format for source trees">PEP 517</a> backends are:</p>
<ul class="simple">
<li>They will be gzipped tar archives, with the <code class="docutils literal notranslate"><span class="pre">.tar.gz</span></code> extension. Zip
archives, or other compression formats for tarballs, are not allowed at
present.</li>
<li>Tar archives must be created in the modern POSIX.1-2001 pax tar format, which
uses UTF-8 for file names.</li>
<li>The source tree contained in an sdist is expected to include the
<code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file.</li>
</ul>
</section>
<section id="evolutionary-notes">
<h2><a class="toc-backref" href="#evolutionary-notes" role="doc-backlink">Evolutionary notes</a></h2>
<p>A goal here is to make it as simple as possible to convert old-style
sdists to new-style sdists. (E.g., this is one motivation for
supporting dynamic build requirements.) The ideal would be that there
would be a single static <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> that could be dropped into any
“version 0” VCS checkout to convert it to the new shiny. This is
probably not 100% possible, but we can get close, and it’s important
to keep track of how close we are… hence this section.</p>
<p>A rough plan would be: Create a build system package
(<code class="docutils literal notranslate"><span class="pre">setuptools_pypackage</span></code> or whatever) that knows how to speak
whatever hook language we come up with, and convert them into calls to
<code class="docutils literal notranslate"><span class="pre">setup.py</span></code>. This will probably require some sort of hooking or
monkeypatching to setuptools to provide a way to extract the
<code class="docutils literal notranslate"><span class="pre">setup_requires=</span></code> argument when needed, and to provide a new version
of the sdist command that generates the new-style format. This all
seems doable and sufficient for a large proportion of packages (though
obviously we’ll want to prototype such a system before we finalize
anything here). (Alternatively, these changes could be made to
setuptools itself rather than going into a separate package.)</p>
<p>But there remain two obstacles that mean we probably won’t be able to
automatically upgrade packages to the new format:</p>
<ol class="arabic simple">
<li>There currently exist packages which insist on particular packages
being available in their environment before setup.py is
executed. This means that if we decide to execute build scripts in
an isolated virtualenv-like environment, then projects will need to
check whether they do this, and if so then when upgrading to the
new system they will have to start explicitly declaring these
dependencies (either via <code class="docutils literal notranslate"><span class="pre">setup_requires=</span></code> or via static
declaration in <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code>).</li>
<li>There currently exist packages which do not declare consistent
metadata (e.g. <code class="docutils literal notranslate"><span class="pre">egg_info</span></code> and <code class="docutils literal notranslate"><span class="pre">bdist_wheel</span></code> might get different
<code class="docutils literal notranslate"><span class="pre">install_requires=</span></code>). When upgrading to the new system, projects
will have to evaluate whether this applies to them, and if so they
will need to stop doing that.</li>
</ol>
</section>
<section id="rejected-options">
<h2><a class="toc-backref" href="#rejected-options" role="doc-backlink">Rejected options</a></h2>
<ul class="simple">
<li>We discussed making the wheel and sdist hooks build unpacked directories
containing the same contents as their respective archives. In some cases this
could avoid the need to pack and unpack an archive, but this seems like
premature optimisation. It’s advantageous for tools to work with archives
as the canonical interchange formats (especially for wheels, where the archive
format is already standardised). Close control of archive creation is
important for reproducible builds. And it’s not clear that tasks requiring an
unpacked distribution will be more common than those requiring an archive.</li>
<li>We considered an extra hook to copy files to a build directory before invoking
<code class="docutils literal notranslate"><span class="pre">build_wheel</span></code>. Looking at existing build systems, we found that passing
a build directory into <code class="docutils literal notranslate"><span class="pre">build_wheel</span></code> made more sense for many tools than
pre-emptively copying files into a build directory.</li>
<li>The idea of passing <code class="docutils literal notranslate"><span class="pre">build_wheel</span></code> a build directory was then also deemed an
unnecessary complication. Build tools can use a temporary directory or a cache
directory to store intermediate files while building. If there is a need, a
frontend-controlled cache directory could be added in the future.</li>
<li>For <code class="docutils literal notranslate"><span class="pre">build_sdist</span></code> to signal a failure for an expected reason, various
options were debated at great length, including raising
<code class="docutils literal notranslate"><span class="pre">NotImplementedError</span></code> and returning either <code class="docutils literal notranslate"><span class="pre">NotImplemented</span></code> or <code class="docutils literal notranslate"><span class="pre">None</span></code>.
Please do not attempt to reopen this discussion without an <em>extremely</em> good
reason, because we are quite tired of it.</li>
<li>Allowing the backend to be imported from files in the source tree would be
more consistent with the way Python imports often work. However, not allowing
this prevents confusing errors from clashing module names. The initial
version of this PEP did not provide a means to allow backends to be
imported from files within the source tree, but the <code class="docutils literal notranslate"><span class="pre">backend-path</span></code> key
was added in the next revision to allow projects to opt into this behaviour
if needed.</li>
</ul>
</section>
<section id="summary-of-changes-to-pep-517">
<h2><a class="toc-backref" href="#summary-of-changes-to-pep-517" role="doc-backlink">Summary of changes to PEP 517</a></h2>
<p>The following changes were made to this PEP after the initial reference
implementation was released in pip 19.0.</p>
<ul class="simple">
<li>Cycles in build requirements were explicitly prohibited.</li>
<li>Support for in-tree backends and self-hosting of backends was added by
the introduction of the <code class="docutils literal notranslate"><span class="pre">backend-path</span></code> key in the <code class="docutils literal notranslate"><span class="pre">[build-system]</span></code>
table.</li>
<li>Clarified that the <code class="docutils literal notranslate"><span class="pre">setuptools.build_meta:__legacy__</span></code> <a class="pep reference internal" href="../pep-0517/" title="PEP 517 – A build-system independent format for source trees">PEP 517</a> backend is
an acceptable alternative to directly invoking <code class="docutils literal notranslate"><span class="pre">setup.py</span></code> for source trees
that don’t specify <code class="docutils literal notranslate"><span class="pre">build-backend</span></code> explicitly.</li>
</ul>
</section>
<section id="appendix-a-comparison-to-pep-516">
<h2><a class="toc-backref" href="#appendix-a-comparison-to-pep-516" role="doc-backlink">Appendix A: Comparison to PEP 516</a></h2>
<p><a class="pep reference internal" href="../pep-0516/" title="PEP 516 – Build system abstraction for pip/conda etc">PEP 516</a> is a competing proposal to specify a build system interface, which
has now been rejected in favour of this PEP. The primary difference is
that our build backend is defined via a Python hook-based interface
rather than a command-line based interface.</p>
<p>This appendix documents the arguments advanced for this PEP over <a class="pep reference internal" href="../pep-0516/" title="PEP 516 – Build system abstraction for pip/conda etc">PEP 516</a>.</p>
<p>We do <em>not</em> expect that specifying Python hooks rather than command line
interfaces will, by itself, reduce the
complexity of calling into the backend, because build frontends will
in any case want to run hooks inside a child – this is important to
isolate the build frontend itself from the backend code and to better
control the build backends execution environment. So under both
proposals, there will need to be some code in <code class="docutils literal notranslate"><span class="pre">pip</span></code> to spawn a
subprocess and talk to some kind of command-line/IPC interface, and
there will need to be some code in the subprocess that knows how to
parse these command line arguments and call the actual build backend
implementation. So this diagram applies to all proposals equally:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">+-----------+</span> <span class="o">+---------------+</span> <span class="o">+----------------+</span>
<span class="o">|</span> <span class="n">frontend</span> <span class="o">|</span> <span class="o">-</span><span class="n">spawn</span><span class="o">-&gt;</span> <span class="o">|</span> <span class="n">child</span> <span class="n">cmdline</span> <span class="o">|</span> <span class="o">-</span><span class="n">Python</span><span class="o">-&gt;</span> <span class="o">|</span> <span class="n">backend</span> <span class="o">|</span>
<span class="o">|</span> <span class="p">(</span><span class="n">pip</span><span class="p">)</span> <span class="o">|</span> <span class="o">|</span> <span class="n">interface</span> <span class="o">|</span> <span class="o">|</span> <span class="n">implementation</span> <span class="o">|</span>
<span class="o">+-----------+</span> <span class="o">+---------------+</span> <span class="o">+----------------+</span>
</pre></div>
</div>
<p>The key difference between the two approaches is how these interface
boundaries map onto project structure:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">.-=</span> <span class="n">This</span> <span class="n">PEP</span> <span class="o">=-.</span>
<span class="o">+-----------+</span> <span class="o">+---------------+</span> <span class="o">|</span> <span class="o">+----------------+</span>
<span class="o">|</span> <span class="n">frontend</span> <span class="o">|</span> <span class="o">-</span><span class="n">spawn</span><span class="o">-&gt;</span> <span class="o">|</span> <span class="n">child</span> <span class="n">cmdline</span> <span class="o">|</span> <span class="o">-</span><span class="n">Python</span><span class="o">-&gt;</span> <span class="o">|</span> <span class="n">backend</span> <span class="o">|</span>
<span class="o">|</span> <span class="p">(</span><span class="n">pip</span><span class="p">)</span> <span class="o">|</span> <span class="o">|</span> <span class="n">interface</span> <span class="o">|</span> <span class="o">|</span> <span class="o">|</span> <span class="n">implementation</span> <span class="o">|</span>
<span class="o">+-----------+</span> <span class="o">+---------------+</span> <span class="o">|</span> <span class="o">+----------------+</span>
<span class="o">|</span>
<span class="o">|</span><span class="n">______________________________________</span><span class="o">|</span> <span class="o">|</span>
<span class="n">Owned</span> <span class="n">by</span> <span class="n">pip</span><span class="p">,</span> <span class="n">updated</span> <span class="ow">in</span> <span class="n">lockstep</span> <span class="o">|</span>
<span class="o">|</span>
<span class="o">|</span>
<span class="n">PEP</span><span class="o">-</span><span class="n">defined</span> <span class="n">interface</span> <span class="n">boundary</span>
<span class="n">Changes</span> <span class="n">here</span> <span class="n">require</span> <span class="n">distutils</span><span class="o">-</span><span class="n">sig</span>
<span class="o">.-=</span> <span class="n">Alternative</span> <span class="o">=-.</span>
<span class="o">+-----------+</span> <span class="o">|</span> <span class="o">+---------------+</span> <span class="o">+----------------+</span>
<span class="o">|</span> <span class="n">frontend</span> <span class="o">|</span> <span class="o">-</span><span class="n">spawn</span><span class="o">-&gt;</span> <span class="o">|</span> <span class="n">child</span> <span class="n">cmdline</span> <span class="o">|</span> <span class="o">-</span><span class="n">Python</span><span class="o">-&gt;</span> <span class="o">|</span> <span class="n">backend</span> <span class="o">|</span>
<span class="o">|</span> <span class="p">(</span><span class="n">pip</span><span class="p">)</span> <span class="o">|</span> <span class="o">|</span> <span class="o">|</span> <span class="n">interface</span> <span class="o">|</span> <span class="o">|</span> <span class="n">implementation</span> <span class="o">|</span>
<span class="o">+-----------+</span> <span class="o">|</span> <span class="o">+---------------+</span> <span class="o">+----------------+</span>
<span class="o">|</span>
<span class="o">|</span> <span class="o">|</span><span class="n">____________________________________________</span><span class="o">|</span>
<span class="o">|</span> <span class="n">Owned</span> <span class="n">by</span> <span class="n">build</span> <span class="n">backend</span><span class="p">,</span> <span class="n">updated</span> <span class="ow">in</span> <span class="n">lockstep</span>
<span class="o">|</span>
<span class="n">PEP</span><span class="o">-</span><span class="n">defined</span> <span class="n">interface</span> <span class="n">boundary</span>
<span class="n">Changes</span> <span class="n">here</span> <span class="n">require</span> <span class="n">distutils</span><span class="o">-</span><span class="n">sig</span>
</pre></div>
</div>
<p>By moving the PEP-defined interface boundary into Python code, we gain
three key advantages.</p>
<p><strong>First</strong>, because there will likely be only a small number of build
frontends (<code class="docutils literal notranslate"><span class="pre">pip</span></code>, and… maybe a few others?), while there will
likely be a long tail of custom build backends (since these are chosen
separately by each package to match their particular build
requirements), the actual diagrams probably look more like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">.-=</span> <span class="n">This</span> <span class="n">PEP</span> <span class="o">=-.</span>
<span class="o">+-----------+</span> <span class="o">+---------------+</span> <span class="o">+----------------+</span>
<span class="o">|</span> <span class="n">frontend</span> <span class="o">|</span> <span class="o">-</span><span class="n">spawn</span><span class="o">-&gt;</span> <span class="o">|</span> <span class="n">child</span> <span class="n">cmdline</span> <span class="o">|</span> <span class="o">-</span><span class="n">Python</span><span class="o">+&gt;</span> <span class="o">|</span> <span class="n">backend</span> <span class="o">|</span>
<span class="o">|</span> <span class="p">(</span><span class="n">pip</span><span class="p">)</span> <span class="o">|</span> <span class="o">|</span> <span class="n">interface</span> <span class="o">|</span> <span class="o">|</span> <span class="o">|</span> <span class="n">implementation</span> <span class="o">|</span>
<span class="o">+-----------+</span> <span class="o">+---------------+</span> <span class="o">|</span> <span class="o">+----------------+</span>
<span class="o">|</span>
<span class="o">|</span> <span class="o">+----------------+</span>
<span class="o">+&gt;</span> <span class="o">|</span> <span class="n">backend</span> <span class="o">|</span>
<span class="o">|</span> <span class="o">|</span> <span class="n">implementation</span> <span class="o">|</span>
<span class="o">|</span> <span class="o">+----------------+</span>
<span class="p">:</span>
<span class="p">:</span>
<span class="o">.-=</span> <span class="n">Alternative</span> <span class="o">=-.</span>
<span class="o">+-----------+</span> <span class="o">+---------------+</span> <span class="o">+----------------+</span>
<span class="o">|</span> <span class="n">frontend</span> <span class="o">|</span> <span class="o">-</span><span class="n">spawn</span><span class="o">+&gt;</span> <span class="o">|</span> <span class="n">child</span> <span class="n">cmdline</span> <span class="o">|</span> <span class="o">-</span><span class="n">Python</span><span class="o">-&gt;</span> <span class="o">|</span> <span class="n">backend</span> <span class="o">|</span>
<span class="o">|</span> <span class="p">(</span><span class="n">pip</span><span class="p">)</span> <span class="o">|</span> <span class="o">|</span> <span class="o">|</span> <span class="n">interface</span> <span class="o">|</span> <span class="o">|</span> <span class="n">implementation</span> <span class="o">|</span>
<span class="o">+-----------+</span> <span class="o">|</span> <span class="o">+---------------+</span> <span class="o">+----------------+</span>
<span class="o">|</span>
<span class="o">|</span> <span class="o">+---------------+</span> <span class="o">+----------------+</span>
<span class="o">+&gt;</span> <span class="o">|</span> <span class="n">child</span> <span class="n">cmdline</span> <span class="o">|</span> <span class="o">-</span><span class="n">Python</span><span class="o">-&gt;</span> <span class="o">|</span> <span class="n">backend</span> <span class="o">|</span>
<span class="o">|</span> <span class="o">|</span> <span class="n">interface</span> <span class="o">|</span> <span class="o">|</span> <span class="n">implementation</span> <span class="o">|</span>
<span class="o">|</span> <span class="o">+---------------+</span> <span class="o">+----------------+</span>
<span class="p">:</span>
<span class="p">:</span>
</pre></div>
</div>
<p>That is, this PEP leads to less total code in the overall
ecosystem. And in particular, it reduces the barrier to entry of
making a new build system. For example, this is a complete, working
build backend:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># mypackage_custom_build_backend.py</span>
<span class="kn">import</span> <span class="nn">os.path</span>
<span class="kn">import</span> <span class="nn">pathlib</span>
<span class="kn">import</span> <span class="nn">shutil</span>
<span class="kn">import</span> <span class="nn">tarfile</span>
<span class="n">SDIST_NAME</span> <span class="o">=</span> <span class="s2">&quot;mypackage-0.1&quot;</span>
<span class="n">SDIST_FILENAME</span> <span class="o">=</span> <span class="n">SDIST_NAME</span> <span class="o">+</span> <span class="s2">&quot;.tar.gz&quot;</span>
<span class="n">WHEEL_FILENAME</span> <span class="o">=</span> <span class="s2">&quot;mypackage-0.1-py2.py3-none-any.whl&quot;</span>
<span class="c1">#################</span>
<span class="c1"># sdist creation</span>
<span class="c1">#################</span>
<span class="k">def</span> <span class="nf">_exclude_hidden_and_special_files</span><span class="p">(</span><span class="n">archive_entry</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Tarfile filter to exclude hidden and special files from the archive&quot;&quot;&quot;</span>
<span class="k">if</span> <span class="n">archive_entry</span><span class="o">.</span><span class="n">isfile</span><span class="p">()</span> <span class="ow">or</span> <span class="n">archive_entry</span><span class="o">.</span><span class="n">isdir</span><span class="p">():</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">basename</span><span class="p">(</span><span class="n">archive_entry</span><span class="o">.</span><span class="n">name</span><span class="p">)</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s2">&quot;.&quot;</span><span class="p">):</span>
<span class="k">return</span> <span class="n">archive_entry</span>
<span class="k">def</span> <span class="nf">_make_sdist</span><span class="p">(</span><span class="n">sdist_dir</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Make an sdist and return both the Python object and its filename&quot;&quot;&quot;</span>
<span class="n">sdist_path</span> <span class="o">=</span> <span class="n">pathlib</span><span class="o">.</span><span class="n">Path</span><span class="p">(</span><span class="n">sdist_dir</span><span class="p">)</span> <span class="o">/</span> <span class="n">SDIST_FILENAME</span>
<span class="n">sdist</span> <span class="o">=</span> <span class="n">tarfile</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">sdist_path</span><span class="p">,</span> <span class="s2">&quot;w:gz&quot;</span><span class="p">,</span> <span class="nb">format</span><span class="o">=</span><span class="n">tarfile</span><span class="o">.</span><span class="n">PAX_FORMAT</span><span class="p">)</span>
<span class="c1"># Tar up the whole directory, minus hidden and special files</span>
<span class="n">sdist</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">getcwd</span><span class="p">(),</span> <span class="n">arcname</span><span class="o">=</span><span class="n">SDIST_NAME</span><span class="p">,</span>
<span class="nb">filter</span><span class="o">=</span><span class="n">_exclude_hidden_and_special_files</span><span class="p">)</span>
<span class="k">return</span> <span class="n">sdist</span><span class="p">,</span> <span class="n">SDIST_FILENAME</span>
<span class="k">def</span> <span class="nf">build_sdist</span><span class="p">(</span><span class="n">sdist_dir</span><span class="p">,</span> <span class="n">config_settings</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;PEP 517 sdist creation hook&quot;&quot;&quot;</span>
<span class="n">sdist</span><span class="p">,</span> <span class="n">sdist_filename</span> <span class="o">=</span> <span class="n">_make_sdist</span><span class="p">(</span><span class="n">sdist_dir</span><span class="p">)</span>
<span class="k">return</span> <span class="n">sdist_filename</span>
<span class="c1">#################</span>
<span class="c1"># wheel creation</span>
<span class="c1">#################</span>
<span class="k">def</span> <span class="nf">get_requires_for_build_wheel</span><span class="p">(</span><span class="n">config_settings</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;PEP 517 wheel building dependency definition hook&quot;&quot;&quot;</span>
<span class="c1"># As a simple static requirement, this could also just be</span>
<span class="c1"># listed in the project&#39;s build system dependencies instead</span>
<span class="k">return</span> <span class="p">[</span><span class="s2">&quot;wheel&quot;</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">build_wheel</span><span class="p">(</span><span class="n">wheel_directory</span><span class="p">,</span>
<span class="n">metadata_directory</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">config_settings</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;PEP 517 wheel creation hook&quot;&quot;&quot;</span>
<span class="kn">from</span> <span class="nn">wheel.archive</span> <span class="kn">import</span> <span class="n">archive_wheelfile</span>
<span class="n">path</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">wheel_directory</span><span class="p">,</span> <span class="n">WHEEL_FILENAME</span><span class="p">)</span>
<span class="n">archive_wheelfile</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="s2">&quot;src/&quot;</span><span class="p">)</span>
<span class="k">return</span> <span class="n">WHEEL_FILENAME</span>
</pre></div>
</div>
<p>Of course, this is a <em>terrible</em> build backend: it requires the user to
have manually set up the wheel metadata in
<code class="docutils literal notranslate"><span class="pre">src/mypackage-0.1.dist-info/</span></code>; when the version number changes it
must be manually updated in multiple places… but it works, and more features
could be added incrementally. Much experience suggests that large successful
projects often originate as quick hacks (e.g., Linux – “just a hobby,
won’t be big and professional”; <a class="reference external" href="https://en.wikipedia.org/wiki/IPython#Grants_and_awards">IPython/Jupyter</a> – <a class="reference external" href="http://blog.fperez.org/2012/01/ipython-notebook-historical.html">a grad
student’s $PYTHONSTARTUP file</a>),
so if our goal is to encourage the growth of a vibrant ecosystem of
good build tools, it’s important to minimize the barrier to entry.</p>
<p><strong>Second</strong>, because Python provides a simpler yet richer structure for
describing interfaces, we remove unnecessary complexity from the
specification – and specifications are the worst place for
complexity, because changing specifications requires painful
consensus-building across many stakeholders. In the command-line
interface approach, we have to come up with ad hoc ways to map
multiple different kinds of inputs into a single linear command line
(e.g. how do we avoid collisions between user-specified configuration
arguments and PEP-defined arguments? how do we specify optional
arguments? when working with a Python interface these questions have
simple, obvious answers). When spawning and managing subprocesses,
there are many fiddly details that must be gotten right, subtle
cross-platform differences, and some of the most obvious approaches –
e.g., using stdout to return data for the <code class="docutils literal notranslate"><span class="pre">build_requires</span></code> operation
– can create unexpected pitfalls (e.g., what happens when computing
the build requirements requires spawning some child processes, and
these children occasionally print an error message to stdout?
obviously a careful build backend author can avoid this problem, but
the most obvious way of defining a Python interface removes this
possibility entirely, because the hook return value is clearly
demarcated).</p>
<p>In general, the need to isolate build backends into their own process
means that we can’t remove IPC complexity entirely – but by placing
both sides of the IPC channel under the control of a single project,
we make it much cheaper to fix bugs in the IPC interface than if
fixing bugs requires coordinated agreement and coordinated changes
across the ecosystem.</p>
<p><strong>Third</strong>, and most crucially, the Python hook approach gives us much
more powerful options for evolving this specification in the future.</p>
<p>For concreteness, imagine that next year we add a new
<code class="docutils literal notranslate"><span class="pre">build_sdist_from_vcs</span></code> hook, which provides an alternative to the current
<code class="docutils literal notranslate"><span class="pre">build_sdist</span></code> hook where the frontend is responsible for passing
version control tracking metadata to backends (including indicating when all
on disk files are tracked), rather than individual backends having to query that
information themselves. In order to manage the transition, we’d want it to be
possible for build frontends to transparently use <code class="docutils literal notranslate"><span class="pre">build_sdist_from_vcs</span></code> when
available and fall back onto <code class="docutils literal notranslate"><span class="pre">build_sdist</span></code> otherwise; and we’d want it to be
possible for build backends to define both methods, for compatibility
with both old and new build frontends.</p>
<p>Furthermore, our mechanism should also fulfill two more goals: (a) If
new versions of e.g. <code class="docutils literal notranslate"><span class="pre">pip</span></code> and <code class="docutils literal notranslate"><span class="pre">flit</span></code> are both updated to support
the new interface, then this should be sufficient for it to be used;
in particular, it should <em>not</em> be necessary for every project that
<em>uses</em> <code class="docutils literal notranslate"><span class="pre">flit</span></code> to update its individual <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> file. (b)
We do not want to have to spawn extra processes just to perform this
negotiation, because process spawns can easily become a bottleneck when
deploying large multi-package stacks on some platforms (Windows).</p>
<p>In the interface described here, all of these goals are easy to
achieve. Because <code class="docutils literal notranslate"><span class="pre">pip</span></code> controls the code that runs inside the child
process, it can easily write it to do something like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">command</span><span class="p">,</span> <span class="n">backend</span><span class="p">,</span> <span class="n">args</span> <span class="o">=</span> <span class="n">parse_command_line_args</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
<span class="k">if</span> <span class="n">command</span> <span class="o">==</span> <span class="s2">&quot;build_sdist&quot;</span><span class="p">:</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">backend</span><span class="p">,</span> <span class="s2">&quot;build_sdist_from_vcs&quot;</span><span class="p">):</span>
<span class="n">backend</span><span class="o">.</span><span class="n">build_sdist_from_vcs</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
<span class="k">elif</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">backend</span><span class="p">,</span> <span class="s2">&quot;build_sdist&quot;</span><span class="p">):</span>
<span class="n">backend</span><span class="o">.</span><span class="n">build_sdist</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="c1"># error handling</span>
</pre></div>
</div>
<p>In the alternative where the public interface boundary is placed at
the subprocess call, this is not possible – either we need to spawn
an extra process just to query what interfaces are supported (as was
included in an earlier draft of <a class="pep reference internal" href="../pep-0516/" title="PEP 516 – Build system abstraction for pip/conda etc">PEP 516</a>, an alternative to this), or
else we give up on autonegotiation entirely (as in the current version
of that PEP), meaning that any changes in the interface will require
N individual packages to update their <code class="docutils literal notranslate"><span class="pre">pyproject.toml</span></code> files before
any change can go live, and that any changes will necessarily be
restricted to new releases.</p>
<p>One specific consequence of this is that in this PEP, we’re able to
make the <code class="docutils literal notranslate"><span class="pre">prepare_metadata_for_build_wheel</span></code> command optional. In our design,
this can be readily handled by build frontends, which can put code in
their subprocess runner like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">dump_wheel_metadata</span><span class="p">(</span><span class="n">backend</span><span class="p">,</span> <span class="n">working_dir</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Dumps wheel metadata to working directory.</span>
<span class="sd"> Returns absolute path to resulting metadata directory</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="k">if</span> <span class="nb">hasattr</span><span class="p">(</span><span class="n">backend</span><span class="p">,</span> <span class="s2">&quot;prepare_metadata_for_build_wheel&quot;</span><span class="p">):</span>
<span class="n">subdir</span> <span class="o">=</span> <span class="n">backend</span><span class="o">.</span><span class="n">prepare_metadata_for_build_wheel</span><span class="p">(</span><span class="n">working_dir</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">wheel_fname</span> <span class="o">=</span> <span class="n">backend</span><span class="o">.</span><span class="n">build_wheel</span><span class="p">(</span><span class="n">working_dir</span><span class="p">)</span>
<span class="n">already_built</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">working_dir</span><span class="p">,</span> <span class="s2">&quot;ALREADY_BUILT_WHEEL&quot;</span><span class="p">)</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">already_built</span><span class="p">,</span> <span class="s2">&quot;w&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">wheel_fname</span><span class="p">)</span>
<span class="n">subdir</span> <span class="o">=</span> <span class="n">unzip_metadata</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">working_dir</span><span class="p">,</span> <span class="n">wheel_fname</span><span class="p">))</span>
<span class="k">return</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">working_dir</span><span class="p">,</span> <span class="n">subdir</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">ensure_wheel_is_built</span><span class="p">(</span><span class="n">backend</span><span class="p">,</span> <span class="n">output_dir</span><span class="p">,</span> <span class="n">working_dir</span><span class="p">,</span> <span class="n">metadata_dir</span><span class="p">):</span>
<span class="w"> </span><span class="sd">&quot;&quot;&quot;Ensures built wheel is available in output directory</span>
<span class="sd"> Returns absolute path to resulting wheel file</span>
<span class="sd"> &quot;&quot;&quot;</span>
<span class="n">already_built</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">working_dir</span><span class="p">,</span> <span class="s2">&quot;ALREADY_BUILT_WHEEL&quot;</span><span class="p">)</span>
<span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">exists</span><span class="p">(</span><span class="n">already_built</span><span class="p">):</span>
<span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">already_built</span><span class="p">,</span> <span class="s2">&quot;r&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">wheel_fname</span> <span class="o">=</span> <span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">()</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
<span class="n">working_path</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">working_dir</span><span class="p">,</span> <span class="n">wheel_fname</span><span class="p">)</span>
<span class="n">final_path</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">output_dir</span><span class="p">,</span> <span class="n">wheel_fname</span><span class="p">)</span>
<span class="n">os</span><span class="o">.</span><span class="n">rename</span><span class="p">(</span><span class="n">working_path</span><span class="p">,</span> <span class="n">final_path</span><span class="p">)</span>
<span class="n">os</span><span class="o">.</span><span class="n">remove</span><span class="p">(</span><span class="n">already_built</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">wheel_fname</span> <span class="o">=</span> <span class="n">backend</span><span class="o">.</span><span class="n">build_wheel</span><span class="p">(</span><span class="n">output_dir</span><span class="p">,</span> <span class="n">metadata_dir</span><span class="o">=</span><span class="n">metadata_dir</span><span class="p">)</span>
<span class="k">return</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">output_dir</span><span class="p">,</span> <span class="n">wheel_fname</span><span class="p">)</span>
</pre></div>
</div>
<p>and thus expose a totally uniform interface to the rest of the frontend,
with no extra subprocess calls, no duplicated builds, etc. But
obviously this is the kind of code that you only want to write as part
of a private, within-project interface (e.g. the given example requires that
the working directory be shared between the two calls, but not with any
other wheel builds, and that the return value from the metadata helper function
will be passed back in to the wheel building one).</p>
<p>(And, of course, making the <code class="docutils literal notranslate"><span class="pre">metadata</span></code> command optional is one piece
of lowering the barrier to entry for developing new backends, as discussed
above.)</p>
<section id="other-differences">
<h3><a class="toc-backref" href="#other-differences" role="doc-backlink">Other differences</a></h3>
<p>Besides the key command line versus Python hook difference described
above, there are a few other differences in this proposal:</p>
<ul class="simple">
<li>Metadata command is optional (as described above).</li>
<li>We return metadata as a directory, rather than a single METADATA
file. This aligns better with the way that in practice wheel metadata
is distributed across multiple files (e.g. entry points), and gives us
more options in the future. (For example, instead of following the PEP
426 proposal of switching the format of METADATA to JSON, we might
decide to keep the existing METADATA the way it is for backcompat,
while adding new extensions as JSON “sidecar” files inside the same
directory. Or maybe not; the point is it keeps our options more open.)</li>
<li>We provide a mechanism for passing information between the metadata
step and the wheel building step. I guess everyone probably will
agree this is a good idea?</li>
<li>We provide more detailed recommendations about the build environment,
but these aren’t normative anyway.</li>
</ul>
</section>
</section>
<section id="copyright">
<h2><a class="toc-backref" href="#copyright" role="doc-backlink">Copyright</a></h2>
<p>This document has been placed in the public domain.</p>
</section>
</section>
<hr class="docutils" />
<p>Source: <a class="reference external" href="https://github.com/python/peps/blob/main/peps/pep-0517.rst">https://github.com/python/peps/blob/main/peps/pep-0517.rst</a></p>
<p>Last modified: <a class="reference external" href="https://github.com/python/peps/commits/main/peps/pep-0517.rst">2023-10-11 12:05:51 GMT</a></p>
</article>
<nav id="pep-sidebar">
<h2>Contents</h2>
<ul>
<li><a class="reference internal" href="#abstract">Abstract</a></li>
<li><a class="reference internal" href="#terminology-and-goals">Terminology and goals</a></li>
<li><a class="reference internal" href="#source-trees">Source trees</a><ul>
<li><a class="reference internal" href="#build-requirements">Build requirements</a></li>
</ul>
</li>
<li><a class="reference internal" href="#build-backend-interface">Build backend interface</a><ul>
<li><a class="reference internal" href="#mandatory-hooks">Mandatory hooks</a><ul>
<li><a class="reference internal" href="#build-wheel">build_wheel</a></li>
<li><a class="reference internal" href="#build-sdist">build_sdist</a></li>
</ul>
</li>
<li><a class="reference internal" href="#optional-hooks">Optional hooks</a><ul>
<li><a class="reference internal" href="#get-requires-for-build-wheel">get_requires_for_build_wheel</a></li>
<li><a class="reference internal" href="#prepare-metadata-for-build-wheel">prepare_metadata_for_build_wheel</a></li>
<li><a class="reference internal" href="#get-requires-for-build-sdist">get_requires_for_build_sdist</a></li>
</ul>
</li>
<li><a class="reference internal" href="#config-settings">Config settings</a></li>
<li><a class="reference internal" href="#build-environment">Build environment</a><ul>
<li><a class="reference internal" href="#recommendations-for-build-frontends-non-normative">Recommendations for build frontends (non-normative)</a></li>
</ul>
</li>
<li><a class="reference internal" href="#in-tree-build-backends">In-tree build backends</a></li>
</ul>
</li>
<li><a class="reference internal" href="#source-distributions">Source distributions</a></li>
<li><a class="reference internal" href="#evolutionary-notes">Evolutionary notes</a></li>
<li><a class="reference internal" href="#rejected-options">Rejected options</a></li>
<li><a class="reference internal" href="#summary-of-changes-to-pep-517">Summary of changes to PEP 517</a></li>
<li><a class="reference internal" href="#appendix-a-comparison-to-pep-516">Appendix A: Comparison to PEP 516</a><ul>
<li><a class="reference internal" href="#other-differences">Other differences</a></li>
</ul>
</li>
<li><a class="reference internal" href="#copyright">Copyright</a></li>
</ul>
<br>
<a id="source" href="https://github.com/python/peps/blob/main/peps/pep-0517.rst">Page Source (GitHub)</a>
</nav>
</section>
<script src="../_static/colour_scheme.js"></script>
<script src="../_static/wrap_tables.js"></script>
<script src="../_static/sticky_banner.js"></script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink