2. piw-master¶
The piw-master script is intended to be run on the database and file-server machine. It is recommended you do not run piw-slave on the same machine as the piw-master script. The database specified in the configuration must exist and have been configured with the piw-initdb script. It is recommended you run piw-master as an ordinary unprivileged user, although obviously it will need write access to the output directory.
2.1. Synopsis¶
piw-master [-h] [--version] [-c FILE] [-q] [-v] [-l FILE] [-d DSN]
[-o PATH] [--dev-mode] [--pypi-xmlrpc URL]
[--pypi-simple URL] [--status-queue ADDR]
[--control-queue ADDR] [--import-queue ADDR]
[--log-queue ADDR] [--slave-queue ADDR] [--file-queue ADDR]
[--web-queue ADDR] [--builds-queue ADDR] [--db-queue ADDR]
[--fs-queue ADDR] [--stats-queue ADDR]
2.2. Description¶
-
-h
,
--help
¶
Show this help message and exit
-
--version
¶
Show program’s version number and exit
-
-c
FILE
,
--configuration
FILE
¶ Specify a configuration file to load
-
-q
,
--quiet
¶
Produce less console output
-
-v
,
--verbose
¶
Produce more console output
-
-l
FILE
,
--log-file
FILE
¶ Log messages to the specified file
-
-d
DSN
,
--dsn
DSN
¶ The database to use; this database must be configured with piw-initdb and the user must not be a PostgreSQL superuser (default: postgres:///piwheels)
-
-o
PATH
,
--output-path
PATH
¶ The path under which the website should be written; must be writable by the current user
-
--dev-mode
¶
Run the master in development mode, which reduces some timeouts and tweaks some defaults
-
--pypi-xmlrpc
URL
¶ The URL of the PyPI XML-RPC service (default: https://pypi.python.org/pypi)
-
--pypi-simple
URL
¶ The URL of the PyPI simple API (default: https://pypi.python.org/simple)
-
--status-queue
ADDR
¶ The address of the queue used to report status to monitors (default: ipc:///tmp/piw-status); this is usually an ipc address
-
--control-queue
ADDR
¶ The address of the queue a monitor can use to control the master (default: ipc:///tmp/piw-control); this is usually an ipc address
-
--import-queue
ADDR
¶ The address of the queue used by piw-import (default: ipc:///tmp/piw-import); this should always be an ipc address
-
--log-queue
ADDR
¶ The address of the queue used by piw-logger (default: ipc:///tmp/piw-logger); this should always be an ipc address
-
--slave-queue
ADDR
¶ The address of the queue used to talk to the build slaves (default: tcp://*:5555); this is usually a tcp address
-
--file-queue
ADDR
¶ The address of the queue used to transfer files from slaves (default: tcp://*:5556); this is usually a tcp address
-
--builds-queue
ADDR
¶ The address of the queue used to store pending builds (default: inproc://builds)
-
--db-queue
ADDR
¶ The address of the queue used to talk to the database server (default: inproc://db)
-
--fs-queue
ADDR
¶ The address of the queue used to talk to the file- system server (default: inproc://fs)
-
--stats-queue
ADDR
¶ The address of the queue used to send statistics to the collator task (default: inproc://stats)
2.3. Deployment¶
A typical deployment of the master service on a Raspbian server goes something like this (each step assumes you start as root):
Install the pre-requisite software:
# apt install postgresql-9.6 apache2 python3-psycopg2 python3-geoip # apt install python3-sqlalchemy python3-urwid python3-zmq python3-voluptuous python3-chameleon # pip install piwheels[monitor,master,logger]
Set up the (unprivileged) piwheels user and the output directory:
# groupadd piwheels # useradd -g piwheels -m piwheels # mkdir /var/www/piwheels # chown piwheels:piwheels /var/www/piwheels
Set up the database:
# su - postgres $ createuser piwheels $ createdb -O postgres piwheels $ piw-initdb
Set up the web server:
- Point the document root to the output path (
/var/www/piwheels
above, but it can be anywhere your piwheels user has write access to; naturally you want to make sure your web-server’s user only has read access to the location). - Set up SSL for the web server (e.g. with Let’s Encrypt; the dehydrated utility is handy for getting and maintaining the SSL certificates).
- Point the document root to the output path (
Start the master running (it’ll take quite a while to populate the list of packages and versions from PyPI on the initial run so get this going before you start bringing up build slaves):
# su - piwheels $ piw-master -v
Deploy some build slaves; see piw-slave for deployment instructions.
2.4. Automatic start¶
If you wish to ensure that the master starts on every boot-up, you may wish to define a systemd unit for it. Example units can be also be found in the root of the piwheels repository:
# wget https://raw.githubusercontent.com/bennuttall/piwheels/master/piwheels-master.service
# cp piwheels-master.service /etc/systemd/system/
# systemctl daemon-reload
# systemctl enable piwheels-master
# systemctl start piwheels-master
2.5. Upgrades¶
The master will check that build slaves have the same version number and will reject them if they do not. Furthermore, it will check the version number in the database’s configuration table matches its own and fail if it does not. Re-run the piw-initdb script as the PostgreSQL super-user to upgrade the database between versions (downgrades are not supported, so take a backup first!).