What

This is where we store standalone Python builds for distributing with the built application. These will get unpacked into a temp directory on desktop platforms so that we can have an extremely specific, isolated Python environment.

Standalone Python builds from:

https://github.com/indygreg/python-build-standalone/releases

.tar.zst vs .tar.zip vs .tar.bz2 vs tar.gz, etc

We're going to unpack these at runtime so they need to exist in a way that Godot can load. It's fairly simple for us to write our own .tar format parser, but for the compression format (zst, zip, gz, bz2, etc) it's better to rely on the engine's built-in decompression code.

Of these formats, the only one that can be read by Godot (without Godot-specific headers being attached by saving the file from Godot) is the .zip format. Unfortunately, .zip format doesn't include a lot of the file permissions that the original .tar.whatever archive includes.

So we're splitting the difference in an slightly unusual way: Use .zip as the compression around the .tar file instead of bzip2, gzip, zstd, it whatever else, then write a .tar parser to load the internal .tar at runtime. What we get from this is a slightly worse compression format that Godot can actually read at runtime, which preserves file permissions and other attributes the way a .tar would.

For format reference on .tar:

https://www.gnu.org/software/tar/manual/html_node/Standard.html https://www.ibm.com/docs/en/zos/2.4.0?topic=formats-tar-format-tar-archives

The Process

Automated way

Run update.bsh.

Obsolete, manual way

1. Grab latest archives

To update the archives here, grab the latest archive from:

https://github.com/indygreg/python-build-standalone/releases

There's a huge list of files there, so here's how to determine the latest version for this project:

cpython-PYTHONVERSION+RELEASE-ARCHITECTURE-PLATFORM-OPTIMIZATIONS.tar.zst

PYTHONVERSION: The Python version. Unless there's a good reason to, you probably want the latest version of this.
RELEASE: Should correspond to the latest release. Formatted as a date (YYYYMMDD).
ARCHITECTURE: CPU architecture. This is going to be funky for Macs, but for desktop Linux/Windows PCs we usually just want x86_64. x86_64_v2 and up include instructions found in newer and newer architectures. This may change if we start supporting stuff like Linux on ARM or whatever.
PLATFORM:
- For Windows we want windows-msvc-shared.
- For Linux we want unknown-linux-gnu.
OPTIMIZATIONS: pgo+lto for fully optimized builds.

Examples:

Linux Python 3.12.3, release 20240415: cpython-3.12.3+20240415-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst
Windows Python 3.12.3, release 20240415: cpython-3.12.3+20240415-x86_64-pc-windows-msvc-shared-pgo-full.tar.zst

See here for more info:

https://gregoryszorc.com/docs/python-build-standalone/20240415/running.html

2. Stick them in this directory

3. Run the conversion script

Run ./convert_zsts.bsh in this directory.

3.0 KiB Raw Blame History