ksconf.app package

Submodules

ksconf.app.deploy module

class ksconf.app.deploy.DeployAction(action: 'DeployActionType')

Bases: object

action: DeployActionType
classmethod from_dict(data: dict) → DeployAction
to_dict() → dict
class ksconf.app.deploy.DeployActionType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

EXTRACT_FILE = 'extract_file'
REMOVE_FILE = 'remove'

Implement in future phase SET_SYMLINK = “link” UPDATE_META = “meta”

SET_APP_NAME = 'app'
SOURCE_REFERENCE = 'source'
class ksconf.app.deploy.DeployAction_ExtractFile(subtype: 'str', path: 'PurePosixPath', mode: 'Optional[int]' = None, mtime: 'Optional[int]' = None, hash: 'Optional[str]' = None, rel_path: 'Optional[str]' = None)

Bases: DeployAction

action: DeployActionType = 'extract_file'
hash: str | None = None
mode: int | None = None
mtime: int | None = None
path: PurePosixPath
rel_path: str | None = None
subtype: str
class ksconf.app.deploy.DeployAction_RemoveFile(path: 'PurePosixPath')

Bases: DeployAction

action: DeployActionType = 'remove'
path: PurePosixPath
class ksconf.app.deploy.DeployAction_SetAppName(name: 'str')

Bases: DeployAction

action: DeployActionType = 'app'
name: str
class ksconf.app.deploy.DeployAction_SourceReference(archive_path: 'str', hash: 'str')

Bases: DeployAction

action: DeployActionType = 'source'
archive_path: str
hash: str
class ksconf.app.deploy.DeployApply(dest: Path)

Bases: object

apply_sequence(deployment_sequence: DeploySequence)

Apply a pre-calculated deployment sequence to the local file system.

Note that we implicitly trust paths contained within deployment_sequence as all constructors run the check_paths() method on all input manifests. Deployment sequences are created locally and never persisted or transmitted.

resolve_source(source, hash)
class ksconf.app.deploy.DeployPlanner

Bases: object

class ksconf.app.deploy.DeploySequence

Bases: object

add(action: str | DeployAction, *args, **kwargs)
classmethod from_dict(data: dict) → DeploySequence
classmethod from_manifest(manifest: AppManifest) → DeploySequence

Fresh deploy of an app from scratch.

(There should probably be a new op-code for this, eventually instead of listing every single file.)

classmethod from_manifest_transformation(base: AppManifest, target: AppManifest) → DeploySequence
to_dict() → dict
ksconf.app.deploy.expand_archive_by_manifest(archive: Path, dest: Path, manifest: AppManifest, dir_mode=504)

Expand an tarball to a local file system including only the files referenced by the files within the app manifest.

This function assumes that safety checks on manifest have already been performed, such as eliminating any absolute paths.

ksconf.app.deploy.get_deploy_action_class(action: DeployActionType | str) → Type[DeployAction]

ksconf.app.facts module

Splunk Application facts:

Easily collect Splunk app name, version, label, and other nuggets from app.conf

class ksconf.app.facts.AppFacts(name: str, label: str | None = None, id: str | None = None, version: str | None = None, author: str | None = None, description: str | None = None, state: str | None = None, build: str | None = None)

Bases: object

Basic Splunk application info container. A majority of these facts are extracted from app.conf

allows_disable: bool | None = None
author: str | None = None
build: str | None = None
check_for_updates: bool | None = None
deployer_lookups_push_mode: str | None = None
deployer_push_mode: str | None = None
description: str | None = None
classmethod from_app_dir(app_path: Path) → AppFacts

Create an AppFacts from a local file system. This expects a standard (non-layered) installed or extracted app folder. Both default and local are considered.

classmethod from_archive(archive: Path)

Returns list of app names, merged app_conf and a dictionary of extra facts that may be useful

classmethod from_conf(name: str, conf: Dict[str, Dict[str, str]]) → AppFacts

Create AppFacts from an app.conf configuration content.

id: str | None = None
install_source_checksum: str | None = None
install_source_local_checksum: str | None = None
is_configured: bool | None = None
is_visible: bool | None = None
label: str | None = None
name: str
state: str | None = None
state_change_requires_restart: bool | None = None
to_dict() → dict
to_tiny_dict(*keep_attrs: str) → dict[str, Any]

Return dict representation, discarding the Nones

version: str | None = None

ksconf.app.manifest module

Splunk App content inventory and signature management

exception ksconf.app.manifest.AppArchiveContentError

Bases: Exception

Problem with the contents of an archive

exception ksconf.app.manifest.AppArchiveError

Bases: Exception

class ksconf.app.manifest.AppManifest(name: str | None = None, source: str | ~os.PathLike | None = None, hash_algorithm: str = 'sha256', files: list[~ksconf.app.manifest.AppManifestFile] = <factory>)

Bases: object

Manifest of a Splunk app. It contains the signatures of contained files and optionally a hash signature of app content.

This is quite very different than a tarball hash, which includes “noise”, like file modification time and possibly tarball creation time. These factors make comparison more expensive. The goal of this class is the ability to capture an app’s content “fingerprint” and quickly determine if another app is the same or not. And to compare apps across equally between tarballs, expanded folders, or a serialized capture at a point in time.

Build instances using:

check_paths()

Check for dangerous paths in the archive.

drop_ds_autogen()

Remove place-holder files created by the deployment server from the manifest for the purpose of consistent hash creation.

These files always live in local/app.conf and contain the literal text # Autogenerated file. Any other forms of this file are preserved.

files: list[AppManifestFile]
filter_files(filter: Callable[[AppManifestFile], bool])

Apply a filter function to files safely.

Note that unlike the filter_file argument on from_filesystem() and from_tarball(), the filter function is given an entire AppManifestFile object not just the file path.

find_local() → Iterable[AppManifestFile]
classmethod from_archive(archive: Path, calculate_hash=True, *, filter_file: Callable[[PurePosixPath], bool] | None = None) → AppManifest

Create as new AppManifest from a tarball. Set calculate_hash as False when only a file listing is needed.

classmethod from_dict(data: dict) → AppManifest
classmethod from_filesystem(path: str | PathLike, name: str | None = None, follow_symlinks=False, calculate_hash=True, *, filter_file: Callable[[PurePosixPath], bool] | None = None) → AppManifest

Create as new AppManifest from an existing directory structure. Set calculate_hash as False when only a file listing is needed.

property hash

Return hash, either from deserialization or local calculation.

hash_algorithm: str = 'sha256'
name: str | None = None
recalculate_hash() → bool

Recalculate hash and indicate if hash has changed.

source: str | PathLike | None = None
to_dict()
class ksconf.app.manifest.AppManifestFile(path: PurePosixPath, mode: int, size: int, hash: str | None = None)

Bases: object

Manifest entry for a single file contained within an app.

You probably don’t want this class. Use AppManifest instead.

content_match(other: AppManifestFile)
classmethod from_dict(data: dict) → AppManifestFile
hash: str | None = None
mode: int
path: PurePosixPath
size: int
to_dict()
exception ksconf.app.manifest.AppManifestStorageError

Bases: Exception

exception ksconf.app.manifest.AppManifestStorageInvalid

Bases: AppManifestStorageError

class ksconf.app.manifest.StoredArchiveManifest(archive: Path, size: int, mtime: float, hash: str)

Bases: object

Stored manifest for a tarball. Typically the manifest file lives in the same directory as the archive. Details around the naming, storage, and clean up of these persistent manifest files are managed by the caller.

archive: Path
classmethod from_dict(data: dict) → StoredArchiveManifest
classmethod from_file(archive: Path, manifest: AppManifest) → StoredArchiveManifest

Construct instance from a tarball.

classmethod from_json_manifest(archive: Path, stored_file: Path, *, permanent_archive: Path | None = None) → StoredArchiveManifest

Attempt to load an archive stored manifest from archive and stored_file paths. If the archive has changed since the manifest was stored, then an exception will be raised indicating the reason for invalidation.

hash: str
property manifest: AppManifest
mtime: float
classmethod read_json_manifest(manifest_file: Path) → StoredArchiveManifest
size: int
to_dict()
write_json_manifest(manifest_file: Path)
ksconf.app.manifest.create_manifest_from_archive(archive_file: Path, manifest_file: Path, manifest: AppManifest) → StoredArchiveManifest

Create a new stored manifest file based on a given archive.

ksconf.app.manifest.get_stored_manifest_name(archive: Path) → Path

Calculate the name of the stored manifest file based on archive.

ksconf.app.manifest.load_manifest_for_archive(archive: ~pathlib.Path, manifest_file: ~pathlib.Path | None = None, *, read_manifest=True, write_manifest=True, permanent_archive: ~pathlib.Path | None = None, log_callback=<built-in function print>) → AppManifest

Load manifest for archive and create a stored copy of the manifest in manifest_file. On subsequent calls the manifest data stored to disk will be reused assuming manifest_file is up-to-date.

File modification time and size are used to determine if archive has been changed since the manifest_file was written.

If no manifest_file is provided, the default manifest naming convention will be applied where the manifest_file is stored in the same directory as archive.

If permanent_archive is provided, then we assume it is the persistent name and archive is a temporary resource. In this mode, the default manifest_file is also based on permanent_archive not archive.

Module contents

Splunk App helper classes

Note that these representations are for native Splunk apps that use ‘default’ and ‘local’ and have not built-in concept of ksconf layers.

ksconf.app.get_facts_manifest_from_archive(archive: Path, calculate_hash=True, check_paths=True) → tuple[AppFacts, AppManifest]

Get both AppFacts and AppManifest from a single archive. If calculate_hash is True, then the manifest will contain checksums for all files in the archive. Without this, it’s not possible to calculate a hash for the combined manifest.

Use this function to collect both metadata about the app and a full listing of the app’s contents.