Pipcl

Contents

• Overview
• Locations
• Basic usage
• Documentation
• API overview
  • The pipcl.Package class
  • Other functions and classes
  • Environment variables
  • Other
• Changelog

Overview

The pipcl package provides Python build backend operations (including PEP 517 support), for use by a setup.py script.

• Designed to help build complex Python extension packages.
• Can also be used to build simple pure-python packages.
• Works on Linux, Windows, MacOS and OpenBSD.
• Is a python module, not a framework, so does not impose any restrictions on usage.
• Can be used with any external build system by running commands, for example with subprocess.run() or enhanced wrapper pipcl.run().

The intention is to allow a setup.py script to use the full power of Python to do everything that is specific to the package, without having to worry about generic Python packaging issues such wheel formats etc (see https://packaging.python.org/en/latest/specifications/).

Locations

• https://pypi.org/project/pipcl
• https://github.com/ArtifexSoftware/pipcl

Basic usage

To build a pure-python package:

• Create a setup.py file that imports pipcl and uses it to specify and build a package:

  import pipcl
  
  def build():
      # Creates a module called `mymodule`.
      return [
              ('src/foo.py', 'mymodule/__init__.py'),
              ]
  def sdist():
      return pipcl.git_items('.')
  p = pipcl.Package(
          'mypackage',    # Name of package.
          '1.2.3',        # Package version.
          pure=True,      # Pure python package.
          summary = 'My package',
          description = 'README.md',
          fn_build = build,
          fn_sdist = sdist,
          )
  # PEP 517 support.
  build_wheel = p.build_wheel
  build_sdist = p.build_sdist
  

• Create a pyproject.toml that tells pip (and other build frontends) to use setup.py:

  [build-system]
      requires = ['pipcl']
      build-backend = 'setup' # Use setup.py
      backend-path = ['.']    # in top-level of the checkout.
  

To build a package containing a SWIG extension module, the build() function can be modified to use pipcl.build_extension():

import pipcl

def build():
    so_leaf = pipcl.build_extension(
            name = 'mymodule',      # Name of extension module.
            path_i = 'src/foo.i',   # SWIG input file.
            outdir = 'build',
            )
    # <so_leaf> will be '_mymodule.so' or similar.
    return [
            ('build/mymodule.py', 'mymodule/__init__.py'),
            (f'build/{so_leaf}', 'mymodule/'),
            ]
def sdist():
    return pipcl.git_items('.')
p = pipcl.Package(
        'mypackage',    # Name of package.
        '1.2.3',        # Package version.
        summary = 'My package',
        description = 'README.md',
        fn_build = build,
        fn_sdist = sdist,
        )
# PEP 517 support.
build_wheel = p.build_wheel
build_sdist = p.build_sdist

For more details, see doctest examples in file:src/pipcl.py.

Documentation

• Detailed documentation is in doc comments in Python code.

  • Convert into HTML with sphinx:

    pip install sphinx
    sphinx-build -M html docs docs/_build
    

    View at: file:docs/_build/html/index.html

• Convert README.rst to html with:

  pip install docutils
  docutils -gdst --halt=3 --pep-references README.rst README.rst.html
  

  View at: file:README.rst.html

API overview

The pipcl.Package class

This is similar in approach to setuptools/distutils in that one creates an instance of the class, passing the package name, version etc.

• Most of the constructor arguments correspond exactly to metadata items in https://packaging.python.org/specifications/core-metadata/.
• An explicit function fn_build() should be provided for building the package. This should return a list of files to be included in a wheel or install, typically as (from, to) pairs, where <from> is the path to a file and <to> is the path within a wheel or install.
• An explicit function fn_sdist() can be provided for building an sdist, returning a list of files to include.
• Unlike setuptools, pipcl does not use heuristics for finding files, instead everything must be specified exactly.

Other functions and classes

• pipcl.build_extension() - build a Python extension using swig. Typically used by pipcl.Package's fn_build() callback.

  • Support for building with the Limited C API.

• pipcl.git_get(): Create/update a clean git checkout for a particular branch, tag or sha of a remote repository.

• pipcl.git_items(): Get list of files within a git checkout.

• pipcl.git_info*(): Get git information such as sha, comment, diff, branch name, author etc.

• pipcl.macos_add_brew_path(): on MacOS, support for adding package binaries to PATH.

• pipcl.macos_patch(): patching of MacOS shared libraries to avoid use of absolute paths.

• Class pipcl.NewFiles: Support for simple detection of changed files, for example around running of an external command.

  This can be useful when building a wheel with pip, as the leafname of the wheel is not known in advance.

• pipcl.run(): enhances subprocess.run(), providing dynamic output along with capture of output, prefixing of output, timeouts etc.

  • Also allows commands to be specified as multiple lines, which improves readability in logs.

• pipcl.run_if(): simple dependency checking to only run commands if prerequisites are newer than the output.

  • This can use information in Makefile-style .d dependency files.

• pipcl.swig*(): build and use swig from source; this can be useful on MacOS where some swig versions appear to generate incorrect code.

• version_to_tuple(): get numerical tuple from '.'-separated version string.

• Class pipcl.wdev.WindowsVS: on Windows, search for specific/latest versions of Visual Studio compiler (cl.exe) and linker (link.exe).

  • Has a command string that includes running of an appropriate vcvars, which can be used directly in compile and link command.

Environment variables

• PIPCL_CHANGE_VERSIONS

  This can be used to override package versions specified in various places:

  • The package version specified when creating a pipcl.Package().
  • Packages specified in the requires_dist specified when creating a pipcl.Package.
  • Packages returned by setup.py's get_requires_for_build_wheel().
    • Note that this only works if get_requires_for_build_wheel() is defined in setup.py before it calls pipcl.Package().

  For more information see file:src/pipcl.py, class Package, method __init__(), args version and requires_dist.

• PIPCL_PREBUILT_WHEEL_<packagae-name>

  Specifies a location of wheel to be used instead of building a wheel for <package-name>.

  This can be used to get a wheel from an external location instead of building it. The value should be something that can be passed to an internal call to pip wheel, for example a local path or a URL.

  <package-name> will be as used in wheel filenames - all lower-case with special characters replaced by underscores.

Other

• Support for Pyodide.
• Experimental support for Graal.

Changelog

Version 13

• Added missing <cpu> arg to pipcl.wdev.windows_vs().
• pipcl.version_to_tuple() now also accepts a sequence as well as a string.
• build_extension(): optionally return (py, lib) instead of library leafname.
• Minor improvements to indentation of compiler/linker commands.
• Extend PIPCL_CHANGE_VERSIONS to also modify return from get_requires_for_build_wheel().
• git_items(): avoid occasional problem with windows / vs \.
• pipcl.run(): added <cwd> arg.
• Add support for getting wheel from external location - see Environment variables PIPCL_PREBUILT_WHEEL_<packagae-name>.

Version 12 (2026-07-09)

• Fixed handling of None in Package.__init__()'s <requires_dist> arg.
• On Windows, improved searching for Visual Studio.
  • We now find Visual Studio 2026 correctly.
  • We now use Microsoft's vswhere.exe executable to find installations of Visual Studio.
  • Instances of class pipcl.wdev.WindowsVS are now primarily obtained using new functions pipcl.wdev.windows_vs() and pipcl.wdev.windows_vs_multiple() instead of direct construction.
• Fixed bug in pipcl.build_extension() example.
• Fixed compile/link commands on windows-arm - use vcvarsarm.bat or vcvarsarm64.bat.

Version 11 (2026-07-01)

• In Package class, make metadata version explicit.
• Use license classifier to clarify that pipcl is GNU Affero v3 only.

Version 10 (2026-06-26)

• Fixed pipcl.git_info_py() in a non-git-checkout.

Version 9 (2026-06-18)

• Added added git_info_py().
• Added version_to_tuple().

Version 8 (2026-06-17)

• Improve support for PIPCL_CHANGE_VERSIONS:
  • Convert package name to lower case before matching with PIPCL_CHANGE_VERSIONS.
  • Provide pipcl.version_override() for use by a setup.py's get_requires_for_build_wheel().

Version 7 (2026-06-05)

• Override matching package version numbers if PIPCL_CHANGE_VERSIONS is set.

Version 6 (2026-05-21)

• In build_extension(), added new arg compiler_extra_cpp.

Version 5 (2026-05-18)

• Automatically compile extension source with cc or c++ depending on suffix.

Version 4 (2026-05-08)

• Ignore encoding errors in pipcl.log(), e.g. on Windows if default is not utf8.

Version 3 (2026-05-02):

• Avoid unhelpful assert failures on Windows if paths differ only in upper/lower case.

Version 2 (2026-04-18):

• Fixed bug in zip file generation on python<3.13.
• Moved Python code into src/.
• Don't attempt to be usable in raw checkout.
• Fixed doctest's to work on Windows.
• Added python_version_tuple().
• Avoid spurious differences between wheels built on different systems:
  • Sort lines in generated RECORD file.
  • Use --global core.autocrlf input when running git clone.

Version 1 (2026-04-16):

• First release to pypi.org