iib.workers.tasks¶

Submodules¶

iib.workers.tasks.build module¶

iib.workers.tasks.build._adjust_csv_annotations(operator_csvs, package_name, organization)¶

Annotate ClusterServiceVersion objects based on an organization configuration.

Parameters:	operator_csvs (list) – the list of `OperatorCSV` objects to examine. package_name (str) – the operator package name. organization (str) – the organization this bundle is for. This determines what annotations to make.

iib.workers.tasks.build._adjust_operator_bundle(manifests_path, metadata_path, organization=None)¶

Apply modifications to the operator manifests at the given location.

For any container image pull spec found in the Operator CSV files, replace floating tags with pinned digests, e.g. image:latest becomes image@sha256:….

If spec.relatedImages is not set, it will be set with the pinned digests. If it is set but there are also RELATED_IMAGE_* environment variables set, an exception will be raised.

This method relies on the OperatorManifest class to properly identify and apply the modifications as needed.

Parameters:	manifests_path (str) – the full path to the directory containing the operator manifests. metadata_path (str) – the full path to the directory containing the bundle metadata files. organization (str) – the organization this bundle is for. If no organization is provided, no custom behavior will be applied.
Raises:	IIBError – if the operator manifest has invalid entries
Returns:	a dictionary of labels to set on the bundle
Return type:	dict

iib.workers.tasks.build._apply_package_name_suffix(metadata_path, organization=None)¶

Add the package name suffix if configured for this organization.

This adds the suffix to the value of annotations['operators.operatorframework.io.bundle.package.v1'] in metadata/annotations.yaml.

The final package name value is returned as part of the tuple.

Parameters:	metadata_path (str) – the path to the bundle’s metadata directory. organization (str) – the organization this customization is for.
Raises:	IIBError – if the `metadata/annotations.yaml` file is in an unexpected format.
Returns:	a tuple with the package name and a dictionary of labels to set on the bundle.
Return type:	tuple(str, dict)

iib.workers.tasks.build._build_image(dockerfile_dir, dockerfile_name, request_id, arch)¶

Build the index image for the specified architecture.

Parameters:	dockerfile_dir (str) – the path to the directory containing the data used for building the container image dockerfile_name (str) – the name of the Dockerfile in the dockerfile_dir to be used when building the container image request_id (int) – the ID of the IIB build request arch (str) – the architecture to build this image for
Raises:	IIBError – if the build fails

iib.workers.tasks.build._cleanup()¶

Remove all existing container images on the host.

This will ensure that the host will not run out of disk space due to stale data, and that all images referenced using floating tags will be up to date on the host.

Raises:	IIBError – if the command to remove the container images fails

iib.workers.tasks.build._copy_files_from_image(image, src_path, dest_path)¶

Copy a file from the container image into the given destination path.

The file may be a directory.

Parameters:	image (str) – the pull specification of the container image. src_path (str) – the full path within the container image to copy from. dest_path (str) – the full path on the local host to copy into.

iib.workers.tasks.build._create_and_push_manifest_list(request_id, arches)¶

Create and push the manifest list to the configured registry.

Parameters:	request_id (int) – the ID of the IIB build request arches (iter) – an iterable of arches to create the manifest list for
Returns:	the pull specification of the manifest list
Return type:	str
Raises:	IIBError – if creating or pushing the manifest list fails

iib.workers.tasks.build._finish_request_post_build(output_pull_spec, request_id, arches, from_index=None, overwrite_from_index=False, overwrite_from_index_token=None)¶

Finish the request after the manifest list has been pushed.

This function was created so that code didn’t need to be duplicated for the add and rm request types.

Parameters:

output_pull_spec (str) – pull spec of the index image generated by IIB
request_id (int) – the ID of the IIB build request
arches (set) – the set of arches that were built as part of this request
from_index (str) – the pull specification of the container image containing the index that the index image build was based from.
overwrite_from_index (bool) – if True, overwrite the input from_index with the built index image.
overwrite_from_index_token (str) – the token used for overwriting the input from_index image.

Raises:

IIBError – if the manifest list couldn’t be created and pushed

iib.workers.tasks.build._get_external_arch_pull_spec(request_id, arch, include_transport=False)¶

Get the pull specification of the single arch image in the external registry.

Parameters:	request_id (int) – the ID of the IIB build request arch (str) – the specific architecture of the container image include_transport (bool) – if true, docker:// will be prefixed in the returned pull specification
Returns:	the pull specification of the single arch image in the external registry
Return type:	str

iib.workers.tasks.build._get_image_arches(pull_spec)¶

Get the architectures this image was built for.

Parameters:	pull_spec (str) – the pull specification to a v2 manifest list
Returns:	a set of architectures of the container images contained in the manifest list
Return type:	set
Raises:	IIBError – if the pull specification is not a v2 manifest list

iib.workers.tasks.build._get_local_pull_spec(request_id, arch)¶

Get the local pull specification of the architecture specfic index image for this request.

Parameters:	request_id (int) – the ID of the IIB build request arch (str) – the specific architecture of the container image.
Returns:	the pull specification of the index image for this request.
Return type:	str

iib.workers.tasks.build._get_resolved_bundles(bundles)¶

Get the pull specification of the bundle images using their digests.

Determine if the pull spec refers to a manifest list. If so, simply use the digest of the first item in the manifest list. If not a manifest list, it must be a v2s2 image manifest and should be used as it is.

Parameters:	bundles (list) – the list of bundle images to be resolved.
Returns:	the list of bundle images resolved to their digests.
Return type:	list
Raises:	IIBError – if unable to resolve a bundle image.

iib.workers.tasks.build._get_resolved_image(pull_spec)¶

Get the pull specification of the container image using its digest.

Parameters:	pull_spec (str) – the pull specification of the container image to resolve
Returns:	the resolved pull specification
Return type:	str

iib.workers.tasks.build._opm_index_add(base_dir, bundles, binary_image, from_index=None)¶

Add the input bundles to an operator index.

This only produces the index.Dockerfile file and does not build the container image.

Parameters:

base_dir (str) – the base directory to generate the database and index.Dockerfile in.
bundles (list) – a list of strings representing the pull specifications of the bundles to add to the index image being built.
binary_image (str) – the pull specification of the container image where the opm binary gets copied from. This should point to a digest or stable tag.
from_index (str) – the pull specification of the container image containing the index that the index image build will be based from.

Raises:

IIBError – if the opm index add command fails.

iib.workers.tasks.build._opm_index_rm(base_dir, operators, binary_image, from_index)¶

Remove the input operators from the operator index.

This only produces the index.Dockerfile file and does not build the container image.

Parameters:

base_dir (str) – the base directory to generate the database and index.Dockerfile in.
operators (list) – a list of strings representing the names of the operators to remove from the index image.
binary_image (str) – the pull specification of the container image where the opm binary gets copied from.
from_index (str) – the pull specification of the container image containing the index that the index image build will be based from.

Raises:

IIBError – if the opm index rm command fails.

iib.workers.tasks.build._prepare_request_for_build(binary_image, request_id, from_index=None, add_arches=None, bundles=None)¶

Prepare the request for the index image build.

All information that was retrieved and/or calculated for the next steps in the build are returned as a dictionary.

This function was created so that code didn’t need to be duplicated for the add and rm request types.

Parameters:	binary_image (str) – the pull specification of the container image where the opm binary gets copied from. request_id (int) – the ID of the IIB build request from_index (str) – the pull specification of the container image containing the index that the index image build will be based from. add_arches (list) – the list of arches to build in addition to the arches `from_index` is currently built for; if `from_index` is `None`, then this is used as the list of arches to build the index image for bundles (list) – the list of bundles to create the bundle mapping on the request
Returns:	a dictionary with the keys: arches, binary_image_resolved, and from_index_resolved.
Raises:	IIBError – if the container image resolution fails or the architectures couldn’t be detected.

iib.workers.tasks.build._push_image(request_id, arch)¶

Push the single arch container image to the configured registry.

Parameters:	request_id (int) – the ID of the IIB build request arch (str) – the architecture of the container image to push
Raises:	IIBError – if the push fails

iib.workers.tasks.build._skopeo_copy(source, destination, copy_all=False, dest_token=None, exc_msg=None)¶

Wrap the skopeo copy command.

Parameters:	source (str) – the source to copy destination (str) – the destination to copy the source to copy_all (bool) – if True, it passes `--all` to the command dest_token (str) – the token to pass to the --dest-token` parameter of the command. If not provided, ``--dest-token parameter is also not provided. exc_msg (str) – a custom exception message to provide
Raises:	IIBError – if the copy fails

iib.workers.tasks.build._verify_index_image(resolved_prebuild_from_index, unresolved_from_index)¶

Verify if the index image has changed since the IIB build request started.

Parameters:	resolved_prebuild_from_index (str) – resolved index image before starting the build unresolved_from_index (str) – unresolved index image provided as API input
Raises:	IIBError – if the index image has changed since IIB build started.

iib.workers.tasks.build._verify_labels(bundles)¶

Verify that the required labels are set on the input bundles.

Parameters:	bundles (list) – a list of strings representing the pull specifications of the bundles to add to the index image being built.
Raises:	IIBError – if one of the bundles does not have the correct label value.

iib.workers.tasks.build.get_image_label(pull_spec, label)¶

Get a specific label from the container image.

Parameters:	label (str) – the label to get
Returns:	the label on the container image or None
Return type:	str

iib.workers.tasks.build.get_rebuilt_image_pull_spec(request_id)¶

Generate the pull specification of the container image rebuilt by IIB.

Parameters:	request_id (int) – the ID of the IIB build request
Returns:	pull specification of the rebuilt container image
Return type:	str

(task)iib.workers.tasks.build.handle_add_request(bundles, binary_image, request_id, from_index=None, add_arches=None, cnr_token=None, organization=None, overwrite_from_index=False, overwrite_from_index_token=None, greenwave_config=None)¶

Coordinate the the work needed to build the index image with the input bundles.

Parameters:

bundles (list) – a list of strings representing the pull specifications of the bundles to add to the index image being built.
binary_image (str) – the pull specification of the container image where the opm binary gets copied from.
request_id (int) – the ID of the IIB build request
from_index (str) – the pull specification of the container image containing the index that the index image build will be based from.
add_arches (list) – the list of arches to build in addition to the arches from_index is currently built for; if from_index is None, then this is used as the list of arches to build the index image for
cnr_token (str) – the token required to push backported packages to the legacy app registry via OMPS.
organization (str) – organization name in the legacy app registry to which the backported packages should be pushed to.
overwrite_from_index (bool) – if True, overwrite the input from_index with the built index image.
overwrite_from_index_token (str) – the token used for overwriting the input from_index image. This is required for non-privileged users to use overwrite_from_index. The format of the token must be in the format “user:password”.
greenwave_config (dict) – the dict of config required to query Greenwave to gate bundles.

Raises:

IIBError – if the index image build fails or legacy support is required and one of cnr_token or organization is not specified.

(task)iib.workers.tasks.build.handle_regenerate_bundle_request(from_bundle_image, organization, request_id)¶

Coordinate the work needed to regenerate the operator bundle image.

Parameters:	from_bundle_image (str) – the pull specification of the bundle image to be regenerated. organization (str) – the name of the organization the bundle should be regenerated for. request_id (int) – the ID of the IIB build request
Raises:	IIBError – if the regenerate bundle image build fails.

(task)iib.workers.tasks.build.handle_rm_request(operators, binary_image, request_id, from_index, add_arches=None, overwrite_from_index=False, overwrite_from_index_token=None)¶

Coordinate the work needed to remove the input operators and rebuild the index image.

Parameters:

operators (list) – a list of strings representing the name of the operators to remove from the index image.
binary_image (str) – the pull specification of the container image where the opm binary gets copied from.
request_id (int) – the ID of the IIB build request
from_index (str) – the pull specification of the container image containing the index that the index image build will be based from.
add_arches (list) – the list of arches to build in addition to the arches from_index is currently built for.
overwrite_from_index (bool) – if True, overwrite the input from_index with the built index image.
overwrite_from_index_token (str) – the token used for overwriting the input from_index image. This is required for non-privileged users to use overwrite_from_index. The format of the token must be in the format “user:password”.

Raises:

IIBError – if the index image build fails.

iib.workers.tasks.celery module¶

iib.workers.tasks.general module¶

(task)iib.workers.tasks.general.failed_request_callback(context, exc, traceback, request_id)¶

Wrap set_request_state for task error callbacks.

Parameters:	context (celery.app.task.Context) – the context of the task failure exc (Exception) – the exception that caused the task failure request_id (int) – the ID of the IIB request

iib.workers.tasks.legacy module¶

iib.workers.tasks.legacy._get_base_dir_and_pkg_name(package_dir)¶

Get the base directory and the package name from package directory.

Parameters:	package_dir (str) – path to the exported package directory
Returns:	base_dir, package name
Return type:	str, str

iib.workers.tasks.legacy._opm_index_export(rebuilt_index_image, package, temp_dir)¶

Export the package that needs to be backported.

Parameters:	rebuilt_index_image (str) – the pull specification of the index image rebuilt by IIB. package (set) – a string representing the name of the package to be exported. temp_dir (str) – path to the temporary directory where the package will be exported to.
Raises:	IIBError – if the export of packages fails.

iib.workers.tasks.legacy._push_package_manifest(package_dir, cnr_token, organization)¶

Push manifests.zip file created for an exported package to OMPS.

Parameters:	package_dir (str) – path to the exported package directory. cnr_token (str) – the token required to push backported packages to the legacy app registry via OMPS. organization (str) – the organization name in the legacy app registry to which the backported packages should be pushed to.
Raises:	IIBError – if the push is unsucessful

iib.workers.tasks.legacy._verify_package_info(package_dir, from_index)¶

Verify if the exported package info is generated correctly.

Parameters:	package_dir (str) – path to the exported package directory from_index (str) – the pull specification of the image image
Raises:	IIBError – if the generated package info is missing

iib.workers.tasks.legacy._zip_package(package_dir)¶

Zip content of exported package to a manifests.zip file.

Parameters:	package_dir (str) – path to the exported package directory
Raises:	IIBError – if unable to zip the exported package

iib.workers.tasks.legacy.export_legacy_packages(packages, request_id, rebuilt_index_image, cnr_token, organization)¶

Export packages to be backported and push them via OMPS.

Parameters:

packages (set) – a set of strings representing the names of the packages to be exported.
request_id (int) – the ID of the IIB build request.
rebuilt_index_image (str) – the pull specification of the index image rebuilt by IIB.
cnr_token (str) – the token required to push backported packages to the legacy app registry via OMPS.
organization (str) – the organization name in the legacy app registry to which the backported packages should be pushed to.

Raises:

IIBError – if the export of packages fails.

iib.workers.tasks.legacy.get_legacy_support_packages(bundles)¶

Get the packages that must be pushed to the legacy application registry.

Parameters:	bundles (list<str>) – a list of strings representing the pull specifications of the bundles to add to the index image being built.
Returns:	a set of packages that require legacy support
Return type:	set

iib.workers.tasks.legacy.validate_legacy_params_and_config(packages, bundles, cnr_token, organization)¶

Valiate parameters and config variables required for legacy support.

Parameters:

packages (set) – a set of strings representing the names of the packages to be exported.
bundles (list) – a list of strings representing the bundles to be added to the index image.
cnr_token (str) – the token required to push backported packages to the legacy app registry via OMPS.
organization (str) – organization name in the legacy app registry to which the backported packages should be pushed to.

Raises:

IIBError – if legacy support is required and necessary params are missing.

iib.workers.tasks.utils module¶

iib.workers.tasks.utils.get_image_labels(pull_spec)¶

Get the labels from the image.

Parameters:	labels (list<str>) – the labels to get
Returns:	the dictionary of the labels on the image
Return type:	dict

iib.workers.tasks.utils.podman_pull(*args)¶

Wrap the podman pull command.

Parameters:	args – any arguments to pass to `podman pull`
Raises:	IIBError – if the command fails

iib.workers.tasks.utils.retry(attempts=5, wait_on=<class 'Exception'>, logger=None)¶

Retry a section of code until success or max attempts are reached.

Parameters:	attempts (int) – the total number of attempts to make before erroring out wait_on (Exception) – the exception on encountering which the function will be retried logger (logging) – the logger to log the messages on
Raises:	IIBError – if the maximum attempts are reached

iib.workers.tasks.utils.run_cmd(cmd, params=None, exc_msg=None, cmd_repr=None)¶

Run the given command with the provided parameters.

Parameters:	cmd (iter) – iterable representing the command to be executed params (dict) – keyword parameters for command execution exc_msg (str) – an optional exception message when the command fails cmd_repr (iter) – an optional representation of the command to be executed. If not provided, this is derived from the `cmd` parameter. This is useful when the command contains sensitive information that must not be logged.
Returns:	the command output
Return type:	str
Raises:	IIBError – if the command fails

iib.workers.tasks.utils.skopeo_inspect(*args)¶

Wrap the skopeo inspect command.

Parameters:	args – any arguments to pass to `skopeo inspect`
Returns:	a dictionary of the JSON output from the skopeo inspect command
Return type:	dict
Raises:	IIBError – if the command fails

iib.workers.tasks¶

Submodules¶

iib.workers.tasks.build module¶

iib.workers.tasks.celery module¶

iib.workers.tasks.general module¶

iib.workers.tasks.legacy module¶

iib.workers.tasks.utils module¶

Module contents¶