actions/setup-python
3
0
Fork 0
Code Issues Actions 2 Packages Projects Releases Wiki Activity
setup-python/docs/advanced-usage.md
IvanZosimov 6dd8ff72eb Change tool cache wording
2022-07-12 17:32:40 +02:00

440 lines
22 KiB
Markdown
Raw Blame History

# Table of contents
- [Using python-version input](#using-python-version-file-input)
- [Specifying a Python version](#specifying-a-python-version)
- [Specifying a PyPy version](#specifying-a-pypy-version)
- [Matrix Testing](#matrix-testing)
- [Using python-version-file input](#using-python-version-file-input)
- [Check latest version](#check-latest-version)
- [Caching packages data](#caching-packages-data)
- [Environment variables and action's outputs](#environment-variables-and-actions-outputs)
- [Available versions of Python and PyPy](#available-versions-of-python-and-pypy)
- [Python](#python)
- [PyPy](#pypy)
- [Hosted tool cache](#hosted-tool-cache)
- [Using `setup-python` with a self hosted runner](#using-setup-python-with-a-self-hosted-runner)
- [Windows](#windows)
- [Linux](#linux)
- [MacOS](#macos)
- [Using `setup-python` on GHES](#using-setup-python-on-ghes)
# Using python-version input
The `python-version` input is used to specify the required version of Python or PyPy.
## Specifying a Python version
If there is a specific version of Python that you need and you don't want to worry about any potential breaking changes due to patch updates (going from `3.7.5` to `3.7.6` for example), you should specify the **exact major, minor, and patch version** (such as `3.7.5`):
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.7.5'
- run: python my_script.py
```
- The only downside to this is that set up will take a little longer since the exact version will have to be downloaded if the exact version is not already installed on the runner due to more recent versions.
- MSI installers are used on Windows for this, so runs will take a little longer to set up vs MacOS and Linux.
You should specify **only a major and minor version** if you are okay with the most recent patch version being used:
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.7'
- run: python my_script.py
```
- There will be a single patch version already installed on each runner for every minor version of Python that is supported.
- The patch version that will be preinstalled, will generally be the latest and every time there is a new patch released, the older version that is preinstalled will be replaced.
- Using the most recent patch version will result in a very quick setup since no downloads will be required since a locally installed version Python on the runner will be used.
You can specify version with **prerelease tag** to download and set up an accurate pre-release version of Python:
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.11.0-alpha.1'
- run: python my_script.py
```
It's also possible to use **x.y-dev syntax** to download and set up the latest patch version of Python, alpha and beta releases included. (for specified major & minor versions):
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.11-dev'
- run: python my_script.py
```
You can also use several types of ranges that are specified in [Semantic Versioning Specification](https://github.com/npm/node-semver#ranges), for instance:
- **[hyphen ranges](https://github.com/npm/node-semver#hyphen-ranges-xyz---abc)** to download and set up the latest available version of Python (includes both pre-release and stable versions):
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.11.0-alpha - 3.11.0'
- run: python my_script.py
```
- **[x-ranges](https://github.com/npm/node-semver#x-ranges-12x-1x-12-)** to specify the latest stable version of Python (for specified major version):
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.x'
- run: python my_script.py
```
Please refer to the [Advanced range syntax section](https://github.com/npm/node-semver#advanced-range-syntax) of the [Semantic Versioning Specification](https://github.com/npm/node-semver) to check other available range syntaxes.
## Specifying a PyPy version
The version of PyPy should be specified in the format `pypy<python_version>[-v<pypy_version>]` or `pypy-<python_version>[-v<pypy_version>]`.
The `-v<pypy_version>` parameter is optional and can be skipped. The latest PyPy version will be used in this case.
```
pypy3.7 or pypy-3.7 # the latest available version of PyPy that supports Python 3.7
pypy3.8 or pypy-3.8 # the latest available version of PyPy that supports Python 3.8
pypy2.7 or pypy-2.7 # the latest available version of PyPy that supports Python 2.7
pypy3.7-v7.3.3 or pypy-3.7-v7.3.3 # Python 3.7 and PyPy 7.3.3
pypy3.7-v7.x or pypy-3.7-v7.x # Python 3.7 and the latest available PyPy 7.x
pypy3.7-v7.3.3rc1 or pypy-3.7-v7.3.3rc1 # Python 3.7 and preview version of PyPy
pypy3.7-nightly or pypy-3.7-nightly # Python 3.7 and nightly PyPy
```
Download and set up PyPy:
```yaml
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- 'pypy3.7' # the latest available version of PyPy that supports Python 3.7
- 'pypy3.7-v7.3.3' # Python 3.7 and PyPy 7.3.3
- 'pypy3.8' # the latest available version of PyPy that supports Python 3.8
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- run: python my_script.py
```
More details on PyPy syntax and examples of using preview / nightly versions of PyPy can be found in the [Available versions of PyPy](#pypy) section.
## Matrix Testing
Using `setup-python` it's possible to use [matrix syntax](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix) to install several versions of Python/PyPy:
```yaml
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ '2.x', '3.x', 'pypy2.7', 'pypy3.7', 'pypy3.8' ]
name: Python ${{ matrix.python-version }} sample
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
architecture: x64
- run: python my_script.py
```
Exclude a specific Python version:
```yaml
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
python-version: ['2.7', '3.7', '3.8', '3.9', '3.10', 'pypy2.7', 'pypy3.8']
exclude:
- os: macos-latest
python-version: '3.8'
- os: windows-latest
python-version: '3.6'
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Display Python version
run: python --version
```
# Using python-version-file input
`setup-python` action has ability to read Python/PyPy version from a version file. `python-version-file` input is used for specifying path to the version file. If version file at the specified path doesn't exist action will try to find `.python-version` file implying that it as a default type of version files. If `.python-version` file doesn't exist also, action will fail with error.
>In case both `python-version` and `python-version-file` inputs are supplied, `python-version-file` input will be ignored due to its lower priority.
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version-file: '.python-version' # Read python version from a file .python-version
- run: python my_script.py
```
# Check latest version
The `check-latest` flag defaults to `false`. Use the default or set `check-latest` to `false` if you prefer stability and if you want to ensure a specific `Python/PyPy` version is always used.
If `check-latest` is set to `true`, the action first checks if the cached version is the latest one. If the locally cached version is not the most up-to-date, a `Python/PyPy` version will then be downloaded. Set `check-latest` to `true` if you want the most up-to-date `Python/PyPy` version to always be used.
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v3
with:
python-version: '3.7'
check-latest: true
- run: python my_script.py
```
> Setting `check-latest` to `true` has performance implications as downloading `Python/PyPy` versions is slower than using cached versions.
# Caching packages data
**Caching pipenv dependencies:**
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.9'
cache: 'pipenv'
- name: Install pipenv
run: curl https://raw.githubusercontent.com/pypa/pipenv/master/get-pipenv.py | python
- run: pipenv install
```
**Caching poetry dependencies:**
```yaml
steps:
- uses: actions/checkout@v3
- name: Install poetry
run: pipx install poetry
- uses: actions/setup-python@v4
with:
python-version: '3.9'
cache: 'poetry'
- run: poetry install
- run: poetry run pytest
```
**Using wildcard patterns to cache dependencies**
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.9'
cache: 'pip'
cache-dependency-path: '**/requirements-dev.txt'
- run: pip install -r subdirectory/requirements-dev.txt
```
**Using a list of file paths to cache dependencies**
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.9'
cache: 'pipenv'
cache-dependency-path: |
server/app/Pipfile.lock
__test__/app/Pipfile.lock
- name: Install pipenv
run: curl https://raw.githubusercontent.com/pypa/pipenv/master/get-pipenv.py | python
- run: pipenv install
```
**Using a list of wildcard patterns to cache dependencies**
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.10'
cache: 'pip'
cache-dependency-path: |
**/setup.cfg
**/requirements*.txt
- run: pip install -e . -r subdirectory/requirements-dev.txt
```
# Environment variables and action's outputs
## Action's outputs
### `python-version`
Using **python-version** output it's possible to get the installed by action python's version. This output is useful when the input `python-version` given as a range (e.g. 3.8.0 - 3.10.0 ), but down in a workflow you need to operate with exact installed version (e.g. 3.10.1).
```yaml
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
id: cp310
with:
python-version: "3.8.0 - 3.10.0"
- run: echo '${{ steps.cp310.outputs.python-version }}'
```
### `python-path`
**python-path** output is available with the absolute path of the python interpreter executable if you need it:
```yaml
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
id: cp310
with:
python-version: "3.10"
- run: pipx run --python '${{ steps.cp310.outputs.python-path }}' nox --version
```
## Evironment variables
These environment variables become available after setup-python action execution:
| Env.Variable | Description |
| ----------- | ----------- |
| pythonLocation |Contains the absolute path to the folder where the requested version of Python or PyPy is installed|
| Python_ROOT_DIR | https://cmake.org/cmake/help/latest/module/FindPython.html#module:FindPython |
| Python2_ROOT_DIR |https://cmake.org/cmake/help/latest/module/FindPython2.html#module:FindPython2|
| Python3_ROOT_DIR |https://cmake.org/cmake/help/latest/module/FindPython2.html#module:FindPython3|
## Using `update-environment` flag
The `update-environment` flag defaults to `true`.
With this setting, the action will add/update environment variables (e.g. `PATH`, `PKG_CONFIG_PATH`, `pythonLocation`) for `python` to just work out of the box.
If `update-environment` is set to `false`, the action will not add/update environment variables.
This can prove useful if you want the only side-effect to be to ensure python is installed and rely on the `python-path` output to run python.
Such a requirement on side-effect could be because you don't want your composite action messing with your user's workflows.
```yaml
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
id: cp310
with:
python-version: '3.10'
update-environment: false
- run: ${{ steps.cp310.outputs.python-path }} my_script.py
```
# Available versions of Python and PyPy
## Python
`setup-python` is able to configure **Python** from two sources:
- Preinstalled versions of Python in the tool cache on GitHub-hosted runners.
- For detailed information regarding the available versions of Python that are installed, see [Supported software](https://docs.github.com/en/actions/reference/specifications-for-github-hosted-runners#supported-software).
- For every minor version of Python, expect only the latest patch to be preinstalled.
- If `3.8.1` is installed for example, and `3.8.2` is released, expect `3.8.1` to be removed and replaced by `3.8.2` in the tool cache.
- If the exact patch version doesn't matter to you, specifying just the major and minor version will get you the latest preinstalled patch version. In the previous example, the version spec `3.8` will use the `3.8.2` Python version found in the cache.
- Use `-dev` instead of a patch number (e.g., `3.11-dev`) to install the latest patch version release for a given minor version, *alpha and beta releases included*.
- Downloadable Python versions from GitHub Releases ([actions/python-versions](https://github.com/actions/python-versions/releases)).
- All available versions are listed in the [version-manifest.json](https://github.com/actions/python-versions/blob/main/versions-manifest.json) file.
- If there is a specific version of Python that is not available, you can open an issue here
>**Note:** Python versions used in this action are generated in the [python-versions](https://github.com/actions/python-versions) repository. For macOS and Ubuntu images python versions are built from the source code. For Windows the python-versions repository uses installation executable. For more information please refer to the [python-versions](https://github.com/actions/python-versions) repository.
## PyPy
`setup-python` is able to configure **PyPy** from two sources:
- Preinstalled versions of PyPy in the tool cache on GitHub-hosted runners
- For detailed information regarding the available versions of PyPy that are installed, see [Supported software](https://docs.github.com/en/actions/reference/specifications-for-github-hosted-runners#supported-software).
- For the latest PyPy release, all versions of Python are cached.
- Cache is updated with a 1-2 week delay. If you specify the PyPy version as `pypy3.7` or `pypy-3.7`, the cached version will be used although a newer version is available. If you need to start using the recently released version right after release, you should specify the exact PyPy version using `pypy3.7-v7.3.3` or `pypy-3.7-v7.3.3`.
- Downloadable PyPy versions from the [official PyPy site](https://downloads.python.org/pypy/).
- All available versions that we can download are listed in [versions.json](https://downloads.python.org/pypy/versions.json) file.
- PyPy < 7.3.3 are not available to install on-flight.
- If some versions are not available, you can open an issue in https://foss.heptapod.net/pypy/pypy/
# Hosted tool cache
GitHub hosted runners have a tool cache that comes with a few versions of Python + PyPy already installed. This tool cache helps speed up runs and tool setup by not requiring any new downloads. There is an environment variable called `RUNNER_TOOL_CACHE` on each runner that describes the location of this tool cache and there is where you will find Python and PyPy installed. `setup-python` works by taking a specific version of Python or PyPy in this tool cache and adding it to PATH.
|| Location |
|------|-------|
|**Tool cache Directory** |`RUNNER_TOOL_CACHE`|
|**Python tool cache**|`RUNNER_TOOL_CACHE/Python/*`|
|**PyPy tool cache**|`RUNNER_TOOL_CACHE/PyPy/*`|
GitHub virtual environments are set up in [actions/virtual-environments](https://github.com/actions/virtual-environments). During the setup, the available versions of Python and PyPy are automatically downloaded, set up and documented.
- Tool cache setup for Ubuntu: [Install-Toolset.ps1](https://github.com/actions/virtual-environments/blob/main/images/linux/scripts/installers/Install-Toolset.ps1) [Configure-Toolset.ps1](https://github.com/actions/virtual-environments/blob/main/images/linux/scripts/installers/Configure-Toolset.ps1)
- Tool cache setup for Windows: [Install-Toolset.ps1](https://github.com/actions/virtual-environments/blob/main/images/win/scripts/Installers/Install-Toolset.ps1) [Configure-Toolset.ps1](https://github.com/actions/virtual-environments/blob/main/images/win/scripts/Installers/Configure-Toolset.ps1)
# Using `setup-python` with a self hosted runner
Python distributions are only available for the same [environments](https://github.com/actions/virtual-environments#available-environments) that GitHub Actions hosted environments are available for. If you are using an unsupported version of Ubuntu such as `19.04` or another Linux distribution such as Fedora, `setup-python` will not work. If you have a supported self-hosted runner and you would like to use `setup-python`, there are a few extra things you need to make sure are set up so that new versions of Python can be downloaded and configured on your runner.
If you are experiencing problems while configuring Python on your self-hosted runner, turn on [step debugging](https://github.com/actions/toolkit/blob/main/docs/action-debugging.md#step-debug-logs) to see addition logs.
### Windows
- Your runner needs to be running with administrator privileges so that the appropriate directories and files can be set up when downloading and installing a new version of Python for the first time.
- If your runner is configured as a service, make sure the account that is running the service has the appropriate write permissions so that Python can get installed. The default `NT AUTHORITY\NETWORK SERVICE` should be sufficient.
- You need `7zip` installed and added to your `PATH` so that the downloaded versions of Python files can be extracted properly during first-time setup.
- MSI installers are used when setting up Python on Windows. A word of caution as MSI installers update registry settings.
- The 3.8 MSI installer for Windows will not let you install another 3.8 version of Python. If `setup-python` fails for a 3.8 version of Python, make sure any previously installed versions are removed by going to "Apps & Features" in the Settings app.
### Linux
- The Python packages that are downloaded from `actions/python-versions` are originally compiled from source in `/opt/hostedtoolcache/` with the [--enable-shared](https://github.com/actions/python-versions/blob/94f04ae6806c6633c82db94c6406a16e17decd5c/builders/ubuntu-python-builder.psm1#L35) flag, which makes them non-relocatable.
- By default runner downloads and install the tools to `/opt/hostedtoolcache`. The environment variable called `AGENT_TOOLSDIRECTORY` can be set to change this location.
- In the same shell that your runner is using, type `export AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache`.
- A more permanent way of setting the environment variable is to create a `.env` file in the same directory as your runner and to add `AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache`. This ensures the variable is always set if your runner is configured as a service.
- Create a directory called `hostedtoolcache` inside `/opt`.
- The user starting the runner must have write permission to the `/opt/hostedtoolcache` directory. It is not possible to start the Linux runner with `sudo` and the `/opt` directory usually requires root privileges to write to. Check the current user and group that the runner belongs to by typing `ls -l` inside the runners root directory.
- The runner can be granted write access to the `/opt/hostedtoolcache` directory using a few techniques:
- The user starting the runner is the owner, and the owner has write permission.
- The user starting the runner is in the owning group, and the owning group has write permission.
- All users have write permission.
- One quick way to grant access is to change the user and group of `/opt/hostedtoolcache` to be the same as the runners using `chown`.
- `sudo chown runner-user:runner-group /opt/hostedtoolcache/`.
- If your runner is configured as a service and you run into problems, make sure the user that the service is running as is correct. For more information, you can [check the status of your self-hosted runner](https://help.github.com/en/actions/hosting-your-own-runners/configuring-the-self-hosted-runner-application-as-a-service#checking-the-status-of-the-service).
### MacOS
- The same setup that applies to `Linux` also applies to `MacOS`, just with a different tools cache directory.
- Create a directory called `/Users/runner/hostedtoolcache`.
- Set the `AGENT_TOOLSDIRECTORY` environment variable to `/Users/runner/hostedtoolcache`.
- Change the permissions of `/Users/runner/hostedtoolcache` so that the runner has write access.
# Using `setup-python` on GHES
`setup-python` comes pre-installed on the appliance with GHES if Actions is enabled. When dynamically downloading Python distributions, `setup-python` downloads distributions from [`actions/python-versions`](https://github.com/actions/python-versions) on github.com (outside of the appliance). These calls to `actions/python-versions` are made via unauthenticated requests, which are limited to [60 requests per hour per IP](https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting). If more requests are made within the time frame, then you will start to see rate-limit errors during download that read `##[error]API rate limit exceeded for...`.
To avoid hitting rate-limit problems, we recommend [setting up your own runner tool cache](https://docs.github.com/en/enterprise-server@2.22/admin/github-actions/managing-access-to-actions-from-githubcom/setting-up-the-tool-cache-on-self-hosted-runners-without-internet-access#about-the-included-setup-actions-and-the-runner-tool-cache).
Reference in New Issue View Git Blame Copy Permalink