2.8. The pi-manage Script

pi-manage is the script that is used during the installation process to setup the database and do many other tasks.

Note

The interesting thing about pi-manage is, that it does not need the server to run as it acts directly on the database. Therefor you need read access to /etc/privacyidea/pi.cfg and the encryption key.

If you want to use a config file other than /etc/privacyidea/pi.cfg, you can set an environment variable:

PRIVACYIDEA_CONFIGFILE=/home/user/pi.cfg pi-manage

pi-manage always takes a command and sometimes a sub command:

pi-manage <command> [<subcommand>] [<parameters>]

For a complete list of commands and sub commands use the -h parameter.

You can do the following tasks.

2.8.1. Encryption Key

You can create an encryption key and encrypt the encryption key.

Create encryption key:

pi-manage setup create_enckey [--enckey_b64=BASE64_ENCODED_ENCKEY]

Note

The filename of the encryption key is read from the configuration. The key will not be created, if it already exists. Optionally, enckey can be passed via –enckey_b64 argument, but it is not recommended. –enckey_b64 must be a string with 96 bytes, encoded in base 64 in order to avoid ambiguous chars.

The encryption key is a plain file on your hard drive. You need to take care, to set the correct access rights.

You can also encrypt the encryption key with a passphrase. To do this do:

pi-manage setup encrypt_enckey /etc/privacyidea/enckey

and pipe the encrypted enckey to a new file.

Read more about the database encryption and the enckey in Security Modules.

2.8.2. Backup and Restore

You can create a backup which will be save to /var/lib/privacyidea/backup/.

The backup will contain the database dump and the complete directory /etc/privacyidea. You may choose if you want to add the encryption key to the backup or not.

Warning

If the backup includes the database dump and the encryption key all seeds of the OTP tokens can be read from the backup.

As the backup contains the etc directory and the database you only need this tar archive backup to perform a complete restore.

2.8.2.1. Supported databases

SQLite, MySQL/MariaDB and PostgreSQL are supported. The database is dumped and restored with the command line tools of the respective database, which have to be installed on the privacyIDEA machine:

Database	Commands used	Debian/Ubuntu package
SQLite	none, the database file is copied	–
MySQL/MariaDB	mysqldump, mysql	mariadb-client or mysql-client
PostgreSQL	pg_dump, psql	postgresql-client

For PostgreSQL the client has to be at least as new as the server it connects to. Dumping a PostgreSQL 17 server with the pg_dump of an older major version fails, so install the postgresql-client-<version> package that matches your server.

2.8.2.2. Restoring

The restore overwrites the contents of the database the restored pi.cfg points to:

pi-manage backup restore /var/lib/privacyidea/backup/privacyidea-backup-<date>.tgz

It does not create the database: the database and the database user have to exist already, as they do on a machine that has been set up before. Only the contents are replaced.

Note

The archive also contains the pi.cfg of the machine the backup was taken on, including its SQLALCHEMY_DATABASE_URI. If you restore onto a machine whose database is reached under a different URI, use --keep-db-uri to keep the URI of the running installation instead of the one from the backup.

Warning

On MySQL/MariaDB the dump contains the name of the database it was taken from and creates that database if it is missing, so the restore always writes into a database of that name - also with --keep-db-uri, which only changes the server, the credentials and the port that are connected to. Restore a MySQL/MariaDB backup only into an installation that uses the same database name, otherwise the data ends up in a newly created copy of the original database while the configured one stays untouched.

A backup can only be restored into the database it was taken from: a dump written by one database cannot be read by another one. Restoring an archive onto an installation using a different database aborts with an error.

2.8.3. Rotate Audit Log

Audit logs are written to the database. You can use pi-manage to perform a log rotation:

pi-manage audit rotate

You can specify a highwatermark and a lowwatermark, age or a config file. Read more about it at cleaning up audit entries.

2.8.4. Clean up challenges

The challenges of challenge-response tokens are stored in a database table. Each challenge has a validity time. Challenges which haven’t been answered, persist in the database and must be cleaned up manually. To clean up all expired challenges use:

pi-manage config challenge cleanup

To clean up challenges older than a certain age (in minutes), use the parameter --age:

pi-manage config challenge cleanup --age 10

This will clean up challenges that were created more than 10 minutes ago.

Use --chunksize to avoid deadlocks when cleaning up a large challenge table. To get only the number of challenges which would be deleted, use --dryrun.

2.8.5. API Keys

Deprecated since version 3.14: The pi-manage api createtoken JWT API keys described here (and the api_key_required policy) are deprecated and will be removed in a future release. The API clients and “remember this device” feature (X-API-Key) is intended to replace them — API clients are stored, individually revocable and rotatable, and auditable — but it does not yet cover every use of these JWTs.

You can use pi-manage to create API keys. API keys can be used to

1. secure the access to the /validate/check API or

2. to access administrative tasks via the REST API.

You can create API keys for /validate/check using the command:

pi-manage api createtoken -r validate

If you want to secure the access to /validate/check you also need to define a policy in scope authorizaion. See api_key_required.

If you wan to use the API key to automate administrative REST API calls, you can use the command:

pi-manage api createtoken -r admin

This command also generates an admin account name. But it does not create this admin account. You need to do so using pi-manage admin. You can now use this API key to enroll tokens as administrator.

Note

These API keys are not persistent. They are not stored in the privacyIDEA server. The API key is connected to the username, that is also generated. This means you have to create an administrative account with this very username to use this API key for this admin user. You also should set policies for this admin user, so that this API key has only restricted rights!

Note

The API key is valid for 365 days.

2.8.6. Policies

You can use pi-manage config policy to enable, disable, create and delete policies.

2.8.7. Exporting and Importing the Configuration

Using pi-manage config export and pi-manage config import you can export and import parts of the server configuration like policies, resolvers, realms, events, periodic tasks, CA connectors, SMS, SMTP and RADIUS server definitions and the global configuration. Run pi-manage config export -h to see the list of available configuration types on your installation.

This can be used to keep a versionable, human-readable copy of single configuration objects, or to transfer a configuration from one privacyIDEA instance to another - for example from a staging to a production system.

Export the complete configuration to a file:

pi-manage config export -o backup.json

The export can be restricted to certain types with -t (which can be given multiple times) and to a single object with -n. The output format is chosen with -f and can be json (default) or yaml. To export only one policy as YAML:

pi-manage config export -t policy -n my_policy -f yaml -o my_policy.yaml

Importing works the other way round. The input format (JSON, YAML or a Python dictionary) is detected automatically and is read from a file given with -i or from standard input:

pi-manage config import -i backup.json

Existing configuration objects with the same name are overwritten, all other existing configuration is kept as is. As with the export, -t and -n restrict the import to certain types or to a single object.

Note

In contrast to pi-manage backup, the configuration export does not contain any tokens and is not a full disaster-recovery backup. It exports the logical configuration, which is portable between instances and - in most cases - between versions. Use pi-manage backup if you want a complete database dump to restore the very same instance.

Warning

By default the exported data contains decrypted secrets - for example the bind password of an LDAP resolver, the password of an SQL resolver, a RADIUS secret or an SMTP password - in clear text, so that it can be imported into an instance with a different encryption key. Store the exported files in a secure location or use the --censor option described below.

2.8.7.1. Censoring secrets on export

If you do not want the secrets to leave the server in clear text, use the --censor option. Every secret - resolver and CA connector passwords, the RADIUS secret, the SMTP password and private key password and secret-looking SMS gateway options and headers - is then replaced with the placeholder __CENSORED__:

pi-manage config export --censor -o config.json

A censored export is useful to

• share or review the configuration (for example in a version control system or attached to a support ticket) without disclosing any secret, and

• take a configuration snapshot of an instance that you can re-import into the same instance later, for example to roll back a change.

On import, __CENSORED__ means “keep the stored secret unchanged”. This only works on the instance the data was exported from, because that is where the original secrets still exist:

• When importing into the same instance, the existing secret is kept and the rest of the configuration is updated.

• When importing into a different or fresh instance, there is no stored secret to keep, so the affected object is created without the secret. You then have to set the passwords manually afterwards.

In other words: use the default (clear text) export to migrate a configuration including its secrets to another instance, and use --censor to produce a shareable artifact or a same-instance snapshot.

Importing a configuration that was exported from a different privacyIDEA version may fail if it contains options that are no longer available - for example a policy action of a token type that has been removed. In this case the affected object is not imported while the rest of the configuration still is, and the command exits with a non-zero status. You can use the --skip-invalid option to drop the parts that are not valid for the running version and import the remaining configuration:

pi-manage config import -i backup.json --skip-invalid

Currently --skip-invalid is evaluated for policies, where it removes policy actions that are not available in the running version. A policy that has no valid action left after this is skipped.

This can also be used to transfer the policies from one privacyIDEA instance to another.