The piw-import script is used to inject the specified file(s) manually into the piwheels database and file-system. This script must be run on the same node as the piw-master script.
usage: piw-import [-h] [--version] [-c FILE] [-q] [-v] [-l FILE] [--package PACKAGE] [--package-version VERSION] [--abi ABI] [--duration DURATION] [--output FILE] [-y] [-d] [--import-queue ADDR] files [files ...]
show this help message and exit
show program’s version number and exit
specify a configuration file to load
produce less console output
produce more console output
log messages to the specified file
the name of the package to import; if omitted this will be derived from the file(s) specified
the version of the package to import; if omitted this will be derived from the file(s) specified
the ABI of the package to import; if omitted this will be derived from the file(s) specified
the time taken to build the package (default: 0s)
the filename containing the build output to insert into the database; if this is omitted an appropriate message will be inserted instead
run non-interactively; never prompt during operation
remove the specified file(s) after a successful import; if the import fails, no files will be removed
the address of the queue used by piw-import (default: (ipc:///tmp/piw-import); this should always be an ipc address
The following section documents the protocol used between the importer and the tasks that it talks to in the piw-master. Each protocol operates over a separate queue. All protocols in the piwheels system follow a similar structure:
- Each message is a list of Python objects.
- The first element in the list is a string indicating the type of message.
- Additional elements depend on the type of the message.
- A given message type always contains the same number of elements (there are no variable length messages).
6.3.1. Mr Chase¶
- The importer sends
["IMPORT", abi_tag, package, version, status, duration, output, files]:
- abi_tag is either
None, indicating that the master should use the “default” (minimum) build ABI registered in the system, or is a string indicating the ABI that the build was attempted for.
- package is the name of the package that the build is for.
- version is the version of the package that the build is for.
- status is
Trueif the build succeeded and
- duration is a
floatvalue indicating the length of time it took to build in seconds.
- output is a string containing the complete build log.
- files is a list of file state tuples containing the following fields
in the specified order:
- filename is the filename of the wheel.
- filesize is the size in bytes of the wheel.
- filehash is the SHA256 hash of the wheel contents.
- package_tag is the package tag extracted from the filename.
- package_version_tag is the version tag extracted from the filename.
- py_version_tag is the python version tag extracted from the filename.
- abi_tag is the ABI tag extracted from the filename (sanitized).
- platform_tag is the platform tag extracted from the filename.
- abi_tag is either
- If the import information is insufficient or incorrect, the master will
["ERROR", args, ...]where args and any further fields are the arguments of the exception that was raised.
- If the import information is okay, the master will send
["SEND", filename]for each file mentioned in the build.
- At this point the importer should use the File Juggler
protocol to transmit the contents of the specified file to the master. When
the file transfer is complete, the importer sends
["SENT"]to the master.
- If the file transfer fails to verify, or if there are more files to send the
master will repeat the “SEND” message. Otherwise, if all transfers have
completed and have been verified, the master replies with
- The importer is now free to remove all files associated with the build, if requested to.
This utility is used to import wheels manually into the system. This is useful with packages which have no source available on PyPI, or binary-only packages from third parties. If invoked with multiple files, all files will be associated with a single “build” and the build will be for the package and version of the first file specified. No checks are made for equality of package name or version (as several packages on PyPI would violate such a rule!).
The utility can be run in a batch mode with
--yes but still requires
invoking once per build required (you cannot register multiple builds in a
The return code will be 0 if the build was registered and all files were
uploaded successfully. Additionally the
--delete option can be
specified to remove the source files once all uploads are completed
successfully. If anything fails, the return code will be non-zero and no files
will be deleted.
The utility should only ever be run directly on the master node (opening the import queue to other machines is a potential security risk).