Table of contents

Administering CloudBees Jenkins Enterprise


CLI Reference

Most of the CloudBees Jenkins Enterprise Cluster initialization and maintenance actions can be accomplished by executing various sub-commands of the cje CLI. Here is a top-level summary of the available sub-commands.

$ cje -h
usage: cje [-h] COMMAND ...

optional arguments:
  -h, --help       show this help message and exit

  COMMAND
    init-project     Initializes a CloudBees Jenkins Enterprise project skeleton
    init             Initialize a CloudBees Jenkins Enterprise environment
    check            Run tests against a CloudBees Jenkins Enterprise environment
    status           Show CloudBees Jenkins Enterprise cluster status
    list-operations  List supported operations
    prepare          Prepare an operation
    verify           Verify a staged operation
    apply            Apply a staged operation
    cancel           Cancel a staged operation
    upgrade-project  Upgrade CloudBees Jenkins Enterprise project to the current version
    upgrade          Upgrade CloudBees Jenkins Enterprise to the latest version
    reinit           Reinitialize a CloudBees Jenkins Enterprise resource
    run              Run a CloudBees Jenkins Enterprise command
    destroy          Destroy a CloudBees Jenkins Enterprise cluster
    version          Print CloudBees Jenkins Enterprise version
    lock-project     Lock a CloudBees Jenkins Enterprise project
    unlock-project   Unlock a CloudBees Jenkins Enterprise project

For more information about a particular command, use:

$ cje COMMAND -h

where COMMAND is one of the commands above.

Cluster Operations

Upgrading CloudBees Jenkins Enterprise

During the upgrade of components, Operations Center might become unavailable for a few minutes. Several internal services will be upgraded and restarted. Services such as the routing service might make some components of the cluster inaccessible for a few seconds.

Upgrading CloudBees Jenkins Enterprise consists of the following steps:

  1. Download a new CloudBees Jenkins Enterprise release

  2. Install the new release locally

  3. Upgrade the CloudBees Jenkins Enterprise project: run cje upgrade-project within the CloudBees Jenkins Enterprise project to upgrade the project

  4. Upgrade CloudBees Jenkins Enterprise components: run cje upgrade

Updating configuration only

Updating the configuration of an existing cluster:

  • Update the configuration file (e.g. $CJE-PROJECT/.dna/project.config)

  • Update CloudBees Jenkins Enterprise project configuration: run cje upgrade --config-only

  • Restart those workers where the changes need to take effect

Upgrading a Managed Master

See Upgrading Managed Masters section in "Operating CJE.".

Managing Workers

List Workers

To list all workers use the command: cje run list-workers.

$ cje run list-workers
worker-1 ( master ) ec2-155-88-200-66.compute-1.amazonaws.com > ACTIVE
worker-2 ( master ) ec2-152-77-190-55.compute-1.amazonaws.com > ACTIVE
worker-3 ( build ) ec2-254-88-160-11.compute-1.amazonaws.com > ACTIVE

Status of Workers

You can get the current status of the cluster with the command: cje status.

$ cje status
CloudBees Jenkins Operation Center
  URL: https://yourdomain.com/cjoc/
  cjoc: OK
Server Status
  worker-1: OK
  worker-2: OK
  controller-1: OK

You can get the current status of a specific worker with the command: cje status <WORKER-NAME>. Note that the worker name syntax is always worker-ID

For example, to get the status of worker-1

$ cje status worker-1
Worker worker-1:
hostname:        ec2-01-01-01-01.compute-1.amazonaws.com
instance_type:   m4.xlarge

Connectivity to worker-1: OK

Applications running on worker-1:
jce_cjoc
jce_castle
masters_eval-master

Management Operations

Managing workers is done with the CloudBees Jenkins Enterprise CLI "operations" commands. You can get a list of operations by executing the following command:

$ cje list-operations
usage: tiger prepare OPERATION

  OPERATION
    access-port-update             Update admin and user access ports.
    castle-logging-level-update    Specify logging level for Volume Managers
    cjoc-update                    Update CJOC settings such as memory, disk and evaluation mode.
    cluster-destroy                Destroys a CloudBees Jenkins Enterprise cluster.
    cluster-init                   Initialize the CloudBees Jenkins Enterprise cluster using the provided configuration.
    cluster-recover                Recover a CloudBees Jenkins Enterprise cluster.
    controller-restart             Restart a controller. This operation will destroy and recreate the controller.
    domain-name-change             Change domain settings such domain name and SSL settings.
    elasticsearch-backup           Backup Elasticsearch indices.
    elasticsearch-indices-delete   Delete indices older than specified days.
    elasticsearch-restore          Restore Elasticsearch latest backup.
    elasticsearch-update           Update Elasticsearch settings such as memory, sharding, instance count, ...
    pse-support                    Generates support bundle containing information to help debugging.
    worker-add                     Add one or more workers to the cluster.
    worker-disable                 Operation not supported.
    worker-enable                  Operation not supported.
    worker-remove                  Delete a specific worker.
    worker-restart                 Restart one or more workers. This operation will destroy and recreate the worker(s).

CloudBees Jenkins Enterprise "operations" commands have a lifecycle. In order to execute an operation, you first need to stage/prepare the command. Next the operation parameters can be edited, reviewed and verified. Then the operation can be applied or cancelled.

The various operations commands are:

list-operations   List supported operations
prepare           Prepare/stage an operation
verify            Verify a staged operation
apply             Apply a staged operation
cancel            Cancel a staged operation
Adding Workers

The 'worker-add' operation is used during the initialization process to set up dedicated workers. Also, if you run low in capacity for Jenkins masters or agents, you can add additional workers with this operation.

$ cje prepare worker-add
worker-add is staged - review worker-add.config and edit as needed - then run 'cje apply' to perform the operation.

worker-add.config lets you define the number of workers to add and the workload type of the worker. It also allows you to set a specific size if the default size is not adequate.

Note
On AWS, the work_volume_size is by default 50 GB. For instances with dedicated disk space, set the proper value to use the entire disk allocated to that the instance.

Workload type

## Type of workload that the worker will handle
# master          : (default) master/main workload: Jenkins masters, {CJOC} and {PSE} components
# build           : dedicated to Jenkins build agents
# elasticsearch   : dedicated to Elasticsearch nodes
# workload_type =

Workers created with the 'master' workload type will handle workload of the Jenkins masters plus Operations Center and CloudBees Jenkins Enterprise components such as Elasticsearch if not deployed on a dedicated 'elasticsearch' worker node.

Apply the operation with cje apply

Removing Workers

You can remove/delete a specific worker with the operation: worker-remove.

You can get the list of workers with the cje status. Note that worker names are always in the format: worker-ID

For example to remove worker-2

$ cje prepare worker-remove
worker-remove is staged - review worker-remove.config and edit as needed - then run 'cje apply' to perform the operation.

You must specify the worker name in the 'worker-remove.config' file before executing the operation.

[server]

# The name of the server to remove (required)
#
name = worker-2

Then apply the operation with cje apply

Restarting Workers

You can restart a specific worker when unhealthy or unresponsive with the operation: worker-restart

For example to restart worker-2

$ cje prepare worker-restart
worker-restart is staged - review worker-restart.config and edit as needed - then run 'cje apply' to perform the operation.

You must specify the worker name in the worker-restart.config file before executing the operation. Then apply the operation with cje apply

This operation will in essence replace a worker with a new one. All applications running on the restarted worker will be restarted on other workers of the cluster.

Changing Domain Name and Enabling SSL

Initial Domain Names or IPs After Installation

After the CloudBees Jenkins Enterprise cluster is running take note of the generated domain name or IP addresses, depending on the infrastructure where it is running.

AWS

On Amazon AWS it will look like this:

Controllers: ec2-52-91-242-61.compute-1.amazonaws.com,ec2-52-90-204-3.compute-1.amazonaws.com,ec2-54-173-172-182.compute-1.amazonaws.com
Workers    : ec2-52-90-206-206.compute-1.amazonaws.com,ec2-54-88-252-108.compute-1.amazonaws.com

{CJOC}    : http://pse-controller-1009100199.us-east-1.elb.amazonaws.com.elb.cloudbees.net/cjoc/
Mesos   : http://mesos.pse-controller-1009100199.us-east-1.elb.amazonaws.com.elb.cloudbees.net
Marathon: http://marathon.pse-controller-1009100199.us-east-1.elb.amazonaws.com.elb.cloudbees.net
...

The name of the ELB will be the section after http://cjoc. (example: pse-controller-1009100199.us-east-1.elb.amazonaws.com.elb.cloudbees.net) and that is the name the DNS records have to point to.

OpenStack

In OpenStack, you can take the IP addresses for the controllers and create one or more A records for them instead of CNAME records.

Controllers: 192.0.2.109,192.0.2.110,192.0.2.111
Workers    : 192.168.2.44,192.168.2.42

{CJOC}    : http://192.0.2.109.nip.io/cjoc/
Mesos   : http://mesos.192.0.2.109.nip.io
Marathon: http://marathon.192.0.2.109.nip.io

The controller IPs in this example would be 192.0.2.109, 192.0.2.110 and 192.0.2.111

Change Domain Name and/or Enable SSL

When the domain-name-change operation is carried out, it is assumed that the new domain name is operational and is being served by the local DNS server. Please work with your Operations department to create a new domain.

If enabling SSL, it is assumed that the certificates are valid for the machine running the installation.

The DNS domain can be changed with the operation: domain-name-change

$ cje prepare domain-name-change
domain-name-change is staged - review domain-name-change.config and edit as needed - then run 'cje apply' to perform the operation.

Edit the domain-name-change.config file to specify the new domain name and related parameters. For more information about the configuration parameters, see the Change Domain Name and/or Enable SSL section. Then apply the operation with cje apply.

Note
The DNS records need to be set up before executing cje apply.
Warning
This operation requires down-time because it re-configures and re-starts several applications.
Domain options guide

You want to run your CloudBees Jenkins Enterprise service under your own domain. CloudBees Jenkins Enterprise uses URL names both for user-facing Jenkins instances, Operations Center and for internal use.

To do this, you need to have access to your own domain name and access to the DNS settings.

The following options allow you to customize the URLs that will be used to access your cluster services. Depending on what your network administrator lets you do on your network, you may be in one of the following situations.

The target for the DNS records can be:

  • a domain name, in providers such as AWS where an Elastic Load Balancer (ELB) is created, for example, cje-controller-1009100199.us-east-1.elb.amazonaws.com. In this case a CNAME record needs to be created.

  • one or more IPs, in other providers such as OpenStack, for example, 192.0.2.109,192.0.2.110,192.0.2.111. In this case one or more A records need to be created.

You can create subdomains beyond 1 level

All the cluster services will be exposed as subdomains of the domain name you provided.

Note
These snippets need to be tuned to fit your own domain name.

To use the domain cje.example.com, you would use

cluster-init.config excerpt
[tiger]
...
domain_name = cje.example.com
domain_separator = .
DNS record

AWS

cje           IN CNAME  <name of the ELB>.
mesos.cje     IN CNAME  <name of the ELB>.
marathon.cje  IN CNAME  <name of the ELB>.

OpenStack

mesos.pse         IN A <IP of the lbaas instance>.
marathon.pse      IN A <IP of the lbaas instance>.
pse               IN A <IP of the lbaas instance>.

Then your cluster will be available with the following URLs :

A master named master-1 will be available as http://cje.example.com/master-1.

You cannot create subdomains beyond 1 level

Infrastructure services will be registered using the provided domain as a suffix.

Note
These snippets need to be tuned to fit your own domain name.

For domain cje.example.com, you would use

cluster-init.config excerpt
[tiger]
...
domain_name = cje.example.com
domain_separator = -
DNS records

AWS

  mesos-cje         IN CNAME  <name of the ELB>.
  marathon-cje      IN CNAME  <name of the ELB>.
  cje               IN CNAME  <name of the ELB>.

OpenStack

  mesos-cje         IN A <IP of the lbaas instance>.
  marathon-cje      IN A <IP of the lbaas instance>.
  cje               IN A <IP of the lbaas instance>.

Then your cluster will be available with the following URLs:

A master named master-1 will be available as http://cje.example.com/master-1.

SSL Configuration

SSL termination can be configured at controller level by configuring the NGINX proxy server with the SSL certificates or in AWS at the ELB level.

It is configured by setting protocol = https and one of the following options.

Controller Termination

Set router_ssl = yes and provide key and certificate files as nginx.key and nginx.cert respectively in the project directory.

AWS ELB Termination

SSL certificates will need to be configured in EC2 and provided via ssl_certificate_id using Amazon Resource Names (ARN) syntax.

CloudBees Jenkins Enterprise requires a certificate with multiple names : cje.example.com, mesos.cje.example.com, marathon.cje.example.com for the base domain cje.example.com.

AWS IAM certificate example
ssl_certificate_id = arn:aws:iam::123456789012:certificate/some-certificate-name
AWS ACM certificate example
ssl_certificate_id = arn:aws:acm:us-east-1:123456789012:certificate/12345678-aaaa-bbbb-cccc-012345678901

If it is not possible to provide a certificate with multiple names, it is possible to provide multiple certificates. The additional certificates can be set up using ssl_certificate_id_mesos and ssl_certificate_id_marathon options using the AWS ARN syntax.

SSL certificate example (three certificates with one name each)
ssl_certificate_id = arn:aws:acm:us-east-1:123456789012:certificate/12345678-aaaa-bbbb-cccc-012345678901
ssl_certificate_id_mesos = arn:aws:acm:us-east-1:123456789012:certificate/12345678-aaaa-bbbb-cccc-012345678902
ssl_certificate_id_marathon = arn:aws:acm:us-east-1:123456789012:certificate/12345678-aaaa-bbbb-cccc-012345678903
Restart Controller

In the event of a controller failure, the controller can be replaced with the 'controller-restart' operation.

$ cje prepare controller-restart
controller-restart is staged - review controller-restart.config and edit as needed - then run 'cje apply' to perform the operation.

Edit the controller-restart.config file and enter the controller name as the [server] name.

Then carry out the operation with cje apply.

Note
This operation will terminate the specified controller and restart a new one. To avoid loss of data, perform this operation only on a multi-controller setup.

Restore a CloudBees Jenkins Enterprise Cluster

If an entire cluster fails or you need to re-create a destroyed cluster, you can use the 'cluster-recover' operation to recover the cluster as long as you still have the PROJECT directory. See the Backups section on how to back up your PROJECT directory.

Note
If the cluster fails and needs recovery, it is recommended to first attempt to destroy the cluster (See Destroy a CloudBees Jenkins Enterprise Project in order to cleanup/reclaim resources, then recover as a destroyed cluster.
Note
If you do NOT have the PROJECT directory anymore, you will have to re-create a new cluster following the standard initialization steps. If using EBS storage on AWS, use the same cluster_name to recover Operations Center and masters data.
$ cje prepare cluster-recover
cluster-recover is staged - review cluster-recover.config and edit as needed - then run 'cje apply' to perform the operation.

Edit the cluster-recover.config file before executing the operation to specify the configuration directory path to recover.

[pse]

## Cluster configuration directory path to recover
# path relative to the PROJECT directory
dna_path=.dna
...
  • In the case of a cluster failure, the configuration directory path (dna_path) will be the default .dna hidden path.

  • In the case of a recovery of a destroyed cluster, specify the destroyed dna path. The destroyed path is usually a hidden path like .dna-destroyed-DATESTAMP. You can list the hidden path with ls -alt

  • If the recovery is done from a different machine or bastion host, the access to the administrative ports need to be updated via the tiger_admin_port_access parameter.

Then apply the operation with cje apply

Migrate an Entire EC2 Cluster

When you use the AWS/EC2 version of CloudBees Jenkins Enterprise, you should use the EBS service for persistence:

[castle]
storage_server = ebs://

This means that all long-term state information is stored in EBS volumes and snapshots. When you run cje destroy, these volumes and snapshots are left in place.

You can use the EBS persistence feature to migrate between Amazon regions. You can run cje destroy in one region, and then tell Amazon (through the console, or CLI) to copy all snapshots to your new region. You can then run the cje cluster-init operation with cluster-init.config changed to reflect the new region you want to run your cluster in (note that you will have to follow the initialization steps to setup DNS records).

Generate a CloudBees Jenkins Enterprise Support Bundle

A CloudBees Support Engineer may ask you to generate a CloudBees Jenkins Enterprise Support Bundle. This is a common step in triaging problems with your CloudBees Jenkins Enterprise instance. This bundle will include information that will be helpful in troubleshooting problems. To do so, run the following command:

$ cje prepare pse-support
pse-support is staged - review pse-support.config and edit as needed - then
run `cje apply` to perform the operation.

Edit the pse-support.config file before executing the operation to choose what packages to include in the bundle.

Note
By default, this operation will generate a bundle with all of the options included. If you wish to exclude a package from the bundle, indicate 'no' instead of 'yes'.
[tiger]

## Include the Support Core Plugin Bundle.
cjoc_support_bundle = yes

## Include logs of this cluster's controllers and workers.
cluster_task_logs = yes

## Include PSE Workspace Specific Files
pse_workspace = yes

Then apply the operation with cje apply.

The Support Bundle archive file will be saved to the current working directory to a time-stamped file called cloudbees-pse-support-YYYY-MM-DD-HH-MM-SS.tgz.

You can then submit this Support Bundle archive file to a CloudBees Support Engineer.

Note
If errors occur in generating the Support Bundle, the pse-support operation will either write a message to stdout indicating the error or indicate inside the support bundle itself if a resource could not be reached.
Note
The CloudBees Jenkins Enterprise Support Bundle is a simple tgz file containing mostly plaintext files. We prefer that you do not modify this archive, but, if necessary, you may redact any sensitive data in this archive.

Destroy a CloudBees Jenkins Enterprise Cluster

When you are finished evaluating CloudBees Jenkins Enterprise or you want to terminate the resources used by a CloudBees Jenkins Enterprise cluster, run the following command:

$ cje prepare cluster-destroy
cluster-destroy is staged - review cluster-destroy.config and edit as needed - then
run `cje apply` to perform the operation.

Edit the cluster-destroy.config file before executing the operation.

Because this is a destructive operation, you must explicitly enter the cluster name in the configuration file. This requirement helps prevent accidental cluster manipulation or destruction.

In the cluster-destroy.config file, available options (operations on clusters and storage buckets, plus — in releases prior to 1.6.0 — operations on Elasticsearch snapshots) are commented out by default.

To invoke any of the three parameters associated with AWS cluster destruction and cleanup (destroying a cluster, storage bucket, or Elasticsearch snapshot), you must first uncomment that parameter in the configuration file by removing the pound sign (#) prefix that by default nullifies the power of these destructive operations by treating them as comments rather than operators.

By uncommenting cluster_name = [your cluster name], destroy_storage_bucket = yes, or destroy_elasticsearch_snapshots = yes, you explicitly permit those operations to be invoked.

{cluster-destroy.config-file} sample excerpt:
[pse]
...
## Cluster name
# Uncomment the next line after verifying that this is really the cluster you want to destroy.
# cluster_name = cobra

(AWS Only) You can also decide whether storage buckets should be deleted. The data stored in the storage buckets can be used to recover a cluster after it has been destroyed. However, the data will prevent creating a new cluster of the same name.

Warning
Destroying a CloudBees Jenkins Enterprise cluster deletes all the resources associated with that cluster. Depending on the data storage backend used, you may not be able to retrieve data after these resources are deleted.

Destroying all data

Depending on the persistent storage used during installation, data may be left over after destroying the cluster.

rsync

The rsync server and all data is deleted after the cluster is destroyed.

NFS

The NFS directories created for Operations Center and Managed Master are not deleted.

AWS Elastic Block Storage

Volumes, snapshots and S3 buckets with Operations Center and Managed Master Jenkins masters data are not deleted after destroy.

In order to do a full deletion the following script can be used, provided the AWS cli is already configured for the same account used for CloudBees Jenkins Enterprise:

CLUSTER_NAME=myclustername
AWS_REGION=us-east-1

ec2-snapshots-for-cluster() {
    aws ec2 describe-snapshots \
        --region $AWS_REGION \
        --owner-ids self \
        --filters Name=tag:cluster,Values=$CLUSTER_NAME \
        --query 'Snapshots[*].{ID:SnapshotId}' \
        --output text
}

ec2-volumes-for-cluster() {
    aws ec2 describe-volumes \
        --region $AWS_REGION \
        --filters Name=tag:cluster,Values=$CLUSTER_NAME \
        --query 'Volumes[*].{ID:VolumeId}' \
        --output text
}

s3-buckets-for-cluster() {
    for id in $(aws s3api list-buckets --query 'Buckets[].Name' --output text); do
        if [ -n "$CLUSTER_NAME" ] && [ "$CLUSTER_NAME" == "$(aws s3api get-bucket-tagging --bucket $id --query 'TagSet[?Key==`cloudbees:pse:cluster`].Value[]' --output text 2> /dev/null)" ]; then
            echo $id
        fi
    done
}

ec2-volumes-for-cluster | xargs -P 20 -I {} bash -c "echo {}; aws ec2 delete-volume --region $AWS_REGION --volume-id {}"
ec2-snapshots-for-cluster | xargs -P 20 -I {} bash -c "echo {}; aws ec2 delete-snapshot --region $AWS_REGION --snapshot-id {}"
s3-buckets-for-cluster | xargs -P 20 -I {} bash -c "echo {}; aws s3 rb s3://{} --force"
Note
There may be some snapshots and volumes being created for some time after the cluster is destroyed so the command may need to be run again.

Recreating a Destroyed CloudBees Jenkins Enterprise Cluster

The CloudBees Jenkins Enterprise configuration created when you ran cje will not be deleted when you run cje destroy. You can re-initialize by going through the installation process again.

Removing CloudBees Jenkins Enterprise

To remove all traces of CloudBees Jenkins Enterprise:

  1. Destroy the CloudBees Jenkins Enterprise cluster, if it’s still running

  2. Delete stored data.

  3. Delete $PROJECT (created by cje init-project …​)

  4. Delete CloudBees Jenkins Enterprise cli

Updating access from selected IPs

Both the admin_port_access parameter, controlling the admin access from selected IPs, and the user_port_access parameter, controlling the user access from selected IPs, can be updated with the 'access-port-update' operation.

$ cje prepare access-port-update
access-port-update is staged - review access-port-update.config and edit as needed - then run 'cje apply' to perform the operation.

Both access port parameters can contain one or more IP address ranges in CIDR notation separated by commas (for example, 192.0.2.0/24,198.51.100.1/32).

Use 0.0.0.0/0 to allow connections from any IP, or IP/32 (for example 198.51.100.1/32) to allow access from a single IP (for example: 198.51.100.1). Other CIDR network masks can be used to control wider ranges of IPs.

Then carry out the operation with cje apply.

Updating Operations Center parameters

To update Operations Center parameters, use the 'cjoc-update' operation.

$ cje prepare cjoc-update
cjoc-update is staged - review cjoc-update.config and edit as needed - then run 'cje apply' to perform the operation.

Edit the cjoc-update.config file to define the parameters you want to change. Only uncomment the parameters you want to change.

This operation allows to: (see cjoc-update.config file for a complete list of parameters)

  • Enable/disable the evaluation mode

  • Set the Operations Center container memory

  • Set the Operations Center application JVM options

  • Set the Operations Center workspace disk size

  • Set a custom Operations Center docker image

Enabling/updating EC2 Container Registry (ECR) configuration

To update the ECR configuration, use the 'ecr-update' operation.

$ cje prepare ecr-update
ecr-update is staged - review ecr-update.config and edit as needed - then run 'cje apply' to perform the operation.

Edit the ecr-update.config file to define the parameters you want to change.

This operation allows to: (see ecr-update.config file for a complete list of parameters)

  • Enable usage of the default AWS EC2 Container Registry

  • Enable AWS EC2 Container Registry for specific accounts

Scripting cluster operations

CloudBees Jenkins Enterprise "operations" commands have a life-cycle. In order to execute an operation, you first need to stage/prepare the command. This stage lays down a configuration file that contains the input parameters of the operation.

By default the config file is edited by the admin user. In order to facilitate the scripting of cli operations, there are two ways to specify operations values via the cje prepare command:

  1. Config/secrets file arguments

  2. Operation parameter arguments

Note that both type of arguments can be used together if necessary. In this case, the parameter value arguments will overwrite the values specified in the config file. Also, only config file parameters can be specified as arguments. If the OPERATION requires secrets, a secrets file can be specified with the --secrets-file SECRETS-FILE argument.

Config/Secrets file arguments

To use a specific config file and/or secrets file for the operation, use the following options of the cje prepare command. See cje prepare -h for all options

  • --config-file CONFIG_FILE

    • cje prepare --config-file CONFIG_FILE OPERATION

  • --secrets-file SECRETS-FILE for operations that require secrets inputs

    • cje prepare --secrets-file SECRETS-FILE OPERATION

Operation parameter arguments

Operation parameters can also be specified as arguments to the cje prepare command. Use cje prepare OPERATION -h to get the list of parameters available for the specified OPERATION.

For example for the worker-add (Add worker(s) to the cluster) operation, the arguments available are:

$ cje prepare worker-add -h
usage: tiger prepare [-h] [-p DIR] [--config-file CONFIG-FILE]
                     [--secrets-file SECRETS-FILE]
                     [--aws.worker_instance_type VAL]
                     [--aws.worker_volume_size VAL]
                     [--worker.count VAL]
                     worker-add

Prepares an operation.

Prepared operations must be configured and then applied using the apply command.

  worker-add            Prepare worker-add.

optional arguments:
  -h, --help            show this help message and exit
  -p DIR, --project DIR
                        Directory containing the CloudBees Jenkins Enterprise project files
  --config-file CONFIG-FILE
                        Use the specified config file
  --secrets-file SECRETS-FILE
                        Use the specified secrets file
  --aws.worker_instance_type VAL
                        The instance type of the worker to create.
  --aws.worker_volume_size VAL
                        The instance root volume size
  --worker.count VAL    Number of workers to add

For example, to add 2 workers using a m4.xlarge instance type on AWS, you can specify the worker count and the instance type as arguments:

$ cje prepare worker-add --worker.count 2 --aws.worker_instance_type m4.xlarge
worker-add is staged - review worker-add.config and edit as needed - then run 'cje apply' to perform the operation.

The worker-add.config file would be pre-populated with the values specified as arguments and ready for 'cje apply'.

worker-add.config file content after the prepare with arguments command:

[worker]

## Number of workers to add
count = 2

[aws]

## The instance type of the worker to create.
# Leave empty to use default value
#
worker_instance_type = m4.xlarge

## The instance root volume size
# worker_volume_size =

Backup a CloudBees Jenkins Enterprise 1.x Cluster Project

A CloudBees Jenkins Enterprise Cluster project contains all configuration parameters and state information of the CloudBees Jenkins Enterprise Cluster. This information is needed to administer a cluster. It is recommended to backup this information to prevent loss of information and administrative access to the cluster.

The project information contains sensitive information such as credentials to manage the underlying instances of the clusters (cloud API credentials, ssh keys) and credentials to access the various CloudBees Jenkins Enterprise components (mesos, marathon, Elasticsearch, Operations Center, etc…​). This information should be backed up with care.

Setting up a CloudBees Jenkins Enterprise Project for backup

CloudBees Jenkins Enterprise CLI contains a set of commands to lock a project by encrypting sensitive information and to unlock a previously locked project by decrypting the sensitive information. These operations rely on GPG private/public keys to secure/encrypt information and the presence of a SCM ignore file (currently only a Git ignore file is provided) that will prevent sensitive information to be accidentally saved into a SCM repository.

As a pre-requisite to use the 'cje lock-project' and cje unlock-project" commands:

  1. Configure a GPG private/public key on the system used to manage the CloudBees Jenkins Enterprise Cluster

  2. Create an ACL file referencing the GPG key to use for encryption.

ACL file

The ACL file 'secrets.acl' needs to be placed under the project directory and contain a reference to the GPG key UID. For example if the key user name is 'John Doe' and the email address of the user is "john.doe@acme.com", the secrets.acl content will be:

John Doe <john.doe@acme.com>

You can add multiple keys to the ACL file to share the project with others. See Sharing a CloudBees Jenkins Enterprise Project section.

Sharing a CloudBees Jenkins Enterprise Project

In order to share the operation of a CloudBees Jenkins Enterprise cluster, the project needs to be made available to the respective operators. This can be accomplished by locking the project and sharing the saved project with all operators.

To avoid sharing your GPG private key with all operators so they can unlock the project, it is preferable to lock the project using each operator’s GPG public key. To do this, you import all GPG public keys into your GPG key chain ( See GPG keys how-to for more information on GPG key import) and add all key UIDs to the 'secrets.acl' (one per line) prior to locking the project. Note that all GPG public keys need to be trusted.

Lock a CloudBees Jenkins Enterprise Project

In order to save a project, first lock the project with the 'cje lock-project' command. This will encrypt the sensitive information of the project and place the project into a lock state. The lock state will prevent further management commands to execute.

Once the project is locked, you can save/backup your project with your favorite tool.

Note that if using Git as your backup repository, a '.gitignore' file is provided to prevent sensitive information to be accidentally saved into a Git repository.

Unlock a CloudBees Jenkins Enterprise Project

To unlock a 'locked' project use the 'cje unlock-project' command. This will decrypt the previously encrypted information and unlock the project state to allow management commands to execute again.