address review comments: better split and organize rationale/motivation

Author: rickeylev

peps/pep-0796.rst

@@ -24,55 +24,24 @@ become more portable.
 Motivation
 ==========
 
-There are two main motivations for allowing relative paths in ``pyvenv.cfg``.
-
-First, it is currently prescribed that the ``home`` value in ``pyvenv.cfg`` be
-an absolute path (`gh-135773`). The behavior of relative paths is unspecified. While
-techniques exist to work around this for every other sub-part of a virtual
-environment, the one remaining part without a tenable solution is how the
-Python runtime itself finds ``PYTHONHOME``. This is because, currently, the
-startup process requires absolute paths be used for the ``home`` key in
-``pyvenv.cfg``. If a relative path is used, behavior is unspecified (the
-current implementation ends up making it relative to the process's current
-working directory, making it untenable to use).
-
-This requirement is overly proscriptive and restrictive because, given a known
-anchor point, it's easy to transform a relative path to an absolute path and
-still retain predictable and reliable behavior. Thus, the absolute path
-requirement should be relaxed and relative path behavior allowed and defined.
-
-Second, such relative paths are a building block to enable portable virtual
-environments, i.e. copying a virtual environment as-is between hosts of
-compatible platforms. For example, by pointing to a parent directory, the
-virtual environment becomes independent of path prefix differences between
-hosts (e.g. ``/usr/local/`` in a container vs
-``/home/user/.pyenv/versions/3.12.0/bin`` in a user's dev environment).
-
-Portable virtual environments are appealing because virtual environments are a
-popular mechanism for running Python applications. This provides several
-benefits:
-
-* The closer the development environment is to the non-development environment,
-  environment-specific issues are less likely, and the easier it is to
-  reproduce issues.
-* The simpler the process of re-creating the environment, environment-specific
-  issues are less likely, and the faster the process can be.
-
-Making it simpler to copy a virtual environment from one host to another
-mitigates these categories of problems. Additionally, the development tools to
-create a virtual environment and install its dependencies aren't needed on the
-host that intends to run the program.
-
-When the virtual environment doesn't require modifications to be usable, it
-also allows more advanced deployment mechanisms, e.g. remote mounting and
-caching of artifacts. While this PEP on its own isn't sufficient to enable
-that, it allows tools like Bazel or venvstacks to more easily prepare
-constrained environments that allow for such use cases.
-
-Rationale
-=========
-
-The reason support for relative virtual environments needs to be
+The ``home`` field in :file:`pyvenv.cfg` is used on interpreter startup to
+determine the actual Python interpreter installation that is used to execute
+code in that virtual environment. Currently, this path is required to be
+absolute for correct virtual environment operation because the original
+`PEP 405 <https://peps.python.org/pep-0405/>`__
+specifying virtual environments didn't cover any specific way of processing
+relative paths, their behaviour is implementation dependent. CPython releases
+up to and including CPython 3.14 resolve them relative to the current process
+working directory, making them too unreliable to use in practice.
+
+The reason to support a relative path is to support portable virtual
+environments, which rely on using a host-agnostic relative path to point to
+``PYTHONHOME``.
+A portable virtual environment is one that can be moved between
+platform-compatible hosts, which is an important feature for some projects (see
+"Why portable environments matter").
+
+The reason support for a relative ``home`` path needs to be
 in the interpreter itself is because locating ``PYTHONHOME`` happens
 very early in the interpreter startup process, which limits the options for
 customizing how it's computed. Without the ability to specify where the
@@ -84,7 +53,66 @@ machines do so either by relying on undocumented interpreter behaviour
 (Bazel, omitting the ``home`` key entirely to trigger an implementation
 dependent fallback to resolving via a symlinked interpreter binary on
 non-Windows systems, see `gh-135773`) or by requiring a post-installation script to be executed
-after the environment is placed in its target location (venvstacks).
+after the environment is placed in its target location (
+`venvstacks <https://lmstudio.ai/blog/venvstacks#publishing-environment-layer-archives>`__
+).
+
+While this PEP on its own isn't sufficient to enable portable virtual
+environments, it allows tools like Bazel or venvstacks to more easily prepare
+constrained environments that allow for such use cases.
+
+Why portable virtual environments matter
+----------------------------------------
+
+Portable virtual environments are important for the efficiency and
+reproducibility benefits they bring from being created once and reused multiple
+times later in different locations. For example, a build farm can build a
+virtual environment once, cache it, and then re-use it as-is to CI jobs.
+
+
+Rationale
+=========
+
+Defining semantics for a relative ``home`` path is the chosen design for the
+following reasons.
+
+First, it is a small change to interpreter startup, in particular of an
+unreliable behavior that isn't specified. Currently, relative paths are
+resolved to the process's current working directory, which makes them
+unreliable for use in practice.
+
+Second, for portable virtual environments, relative paths allow more
+efficient, simple, and correct reproduction of virtual environments between
+hosts. This is because they can be copied as-is to different locations. Some
+example capabilities this allows are:
+
+* A build farm creating (and caching) a virtual environment, which is then
+  served to developers (e.g. Bazel).
+* Composing virtual environments together (e.g. venvstacks).
+* Installing multiple arbitrary virtual environments into a container to
+  save disk space.
+* Layering a virtual environment atop a container image for faster image
+  building.
+* Not needing virtual environment creation tools on the host that uses a
+  virtual environment.
+* Exact reproduction of an application's virtual environment between a
+  developer's host and a production host.
+
+Third, requiring an absolute path is inherently overly proscriptive. The
+interpreter itself doesn't care whether paths are relative or absolute, merely
+that they point to valid locations, so users should be given the ability to use
+a path of their choosing. Given a known anchor point, it's easy to transform a
+relative path to an absolute path and still retain predictable and reliable
+behavior that produces correct values.
+
+Fullying designing portable virtual environments
+------------------------------------------------
+
+This PEP purposely only focuses on the interpreter startup behavior to limit
+its scope. There are multiple implementations and many design questions for how
+to implement portable virtual environments work (e.g. what installers should
+do), but they are separate from the Python runtime initialization phase.
+
 
 Specification
 =============
@@ -162,6 +190,22 @@ relative virtual environment paths will typically be aware of the underlying
 base runtime Python version, and hence able to update the emitted relative path
 accordingly.
 
+Security Implications
+=====================
+
+A relative path in :file:`pyvenv.cfg` may resolve differently depending on the
+location of the virtual environment. This *could* point to a surprising,
+potentially malicious, location.
+
+However, this risk already exists today because a relative path isn't
+_rejected_, but resolved relative to the current working directory. This PEP
+just changes the anchor point to ``pyvenv.cfg`` itself.
+
+Similarly, the same concern exists for absolute paths. The two are
+fundamentally the same because they both rely on trusting whoever created
+the ``pyvenv.cfg`` file, which requires having run another tool or downloaded
+something from elsewhere.
+
 
 How to Teach This
 =================
@@ -193,6 +237,11 @@ them. These questions are best addressed separately by tool owners.
 References
 ==========
 
+portable virtual environment
+    A portable virtual environment is one that can be copied from
+    one host to another that is platform compatible (e.g. same OS, CPU
+    architecture, etc), with little or no modification or post processing.
+
 * `rules_python <https://github.com/bazel-contrib/rules_python>`__: implements
   host-relocatable virtual environments.
 * `rules_py <https://github.com/aspect-build/rules_py>`__: implements
@@ -265,6 +314,7 @@ Code generally assumes that any virtual environment will be
 automatically detected and activated by the presence of ``pyvenv.cfg``, so
 things work better when alterations to the environment aren't a concern.
 
+
 Copyright
 =========