ksconf namespace



ksconf.archive module

class ksconf.archive.GenArchFile(path, mode, size, payload)

Bases: NamedTuple

mode: int

Alias for field number 1

path: str

Alias for field number 0

payload: ByteString | None

Alias for field number 3

size: int

Alias for field number 2

ksconf.archive.extract_archive(archive_name, extract_filter: Callable | None = None) Iterable[GenArchFile]
ksconf.archive.gen_arch_file_remapper(iterable: Iterable[GenArchFile], mapping: Sequence[Tuple[str, str]]) Iterable[GenArchFile]
ksconf.archive.sanity_checker(iterable: Iterable[GenArchFile]) Iterable[GenArchFile]

ksconf.cli module

KSCONF - Ksconf Splunk CONFig tool

Optionally supports argcomplete for commandline argument (tab) completion.

Install & register with:

pip install argcomplete activate-global-python-argcomplete (in ~/.bashrc)


Run a simple python environment sanity check. Here’s the scenario, if Splunk’s python is called but not all the correct environment variables have been set, then ksconf can fail in unclear ways.

ksconf.cli.cli(argv=None, _unittest=False)
ksconf.cli.handle_cmd_failed(subparser, ep)

Build a bogus subparser for a cmd that can’t be loaded, with the only purpose of providing a more consistent user experience.

ksconf.combine module

class ksconf.combine.LayerCombiner(follow_symlink: bool = False, banner: str = '', dry_run: bool = False, quiet: bool = False)

Bases: object

Class to recursively combine layers (directories) into a single rendered output target directory. This is heavily used by the ksconf combine command as well as by the package command.

Typical class use case:


lc = LayerCombiner()

# Setup source, either
  1. lc.set_source_dirs() OR

  2. lc.set_layer_root()

Call hierarch:

lc.combine()                    Entry point
    -> prepare()                Directory, layer prep
        -> prepare_target_dir() Make dir; subclass handles marker here (combine CLI)
    -> pre_combine_inventory()  Hook for pre-processing (or alerting) the set of files to combine
    -> combine_files()          Main worker function
    -> post_combine()           Optional, cleanup leftover files
add_layer_filter(action, pattern)
combine(target: Path | str, *, hook_label='')

Combine layers into target directory. Any hook_label given will be passed to the plugin system via the usage field.

combine_files(target: Path, src_files: list[LayerFile])
conf_file_re = re.compile('([a-z_-]+\\.conf|(default|local)\\.meta)$')
filetype_handlers: list[tuple[Callable, Callable]] = [(<function LayerCombiner.register_handler.<locals>.match_f>, <function handle_merge_conf_files>), (<function LayerCombiner.register_handler.<locals>.match_f>, <function handle_spec_concatenate>)]

Hook point for post-processing after all copy/merge operations have been completed.

pre_combine_inventory(target: Path, src_files: list[LayerFile]) Sequence[LayerFile]

Hook point for pre-processing before any files are copied/merged

prepare(target: Path)

Start the combine process. This includes directory checking, applying layer filtering, and marker file handling.

prepare_target_dir(target: Path)

Hook to ensure destination directory is ready for use. This can be overridden to adder marker file handling for use cases that need it (e.g., the ‘combine’ command)

classmethod register_handler(regex_match)

Decorator that registers a new file type handler. The handler is used if a file name matches a regex. Regex ‘search’ mode is used.

set_layer_root(root: Layer)
set_source_dirs(sources: list[Path])
spec_file_re = re.compile('\\.conf\\.spec$')
exception ksconf.combine.LayerCombinerException

Bases: Exception

ksconf.combine.handle_merge_conf_files(combiner: LayerCombiner, dest_path: Path, sources: list[LayerFile], dry_run)

Handle merging two or more .conf files.

ksconf.combine.handle_spec_concatenate(combiner: LayerCombiner, dest_path: Path, sources: list[LayerFile], dry_run)

Concatenate multiple .spec files. Likely a README.d situation.


Decorator that registers a new file type handler. The handler is used if a file name matches a regex. Regex ‘search’ mode is used.

ksconf.command module


Helpers functions and classes in support of the actual commands that live under ksconf.commands.*.

Note that ksconf.commands is a namespace package, which can be contributed to by multiple python packages (technically called “distributions”). Because of this, there can be no __init__.py, which is where this content logically belongs.

class ksconf.command.ConfDirProxy(name, mode, parse_profile=None)

Bases: object

class ksconf.command.ConfFileProxy(name: str, mode: str, stream: TextIO | None = None, *, parse_profile: Dict | None = None, is_file: bool | None = None)

Bases: object

property data
dump(data, **kwargs) SmartEnum
property mtime

Setting a key to None will remove that setting.

property stream
class ksconf.command.ConfFileType(mode='r', action='open', parse_profile: Dict | None = None, accept_dir: bool = False)

Bases: object

Factory for creating conf file object types; returns a lazy-loader ConfFile proxy class

Started from FileType() and then changed everything. With our use case, it’s often necessary to delay writing, or read before writing to a conf file (depending on whether or not –dry-run mode is enabled, for example.)

Instances of FileType are typically passed as type= arguments to the ArgumentParser add_argument() method.

  • mode (str) – How the file is to be opened. Accepts “r”, “w”, and “r+”.

  • action (str) – Determine how much work should be handled during argument parsing vs handed off to the caller. Supports ‘none’, ‘open’, ‘load’. Full descriptions below.

  • parse_profile – parsing configuration settings passed along to the parser

  • accept_dir (bool) – Should the CLI accept a directory of config files instead of an individual file. Defaults to False.

Values for action




No preparation or testing is done on the filename.


Ensure the file exists and can be opened.


Ensure the file can be opened and parsed successfully.

Once invoked, instances of this class will return a ConfFileProxy object, or a ConfDirProxy object if a directory is passed in via the CLI.

class ksconf.command.KsconfCmd(name)

Bases: object

Ksconf command specification base class.

description: str | None = None

Allow overriding for unittesting or other high-level functionality, like an interactive interface.

format = 'default'
help: str | None = None

Handle flow control between pre_run() / run() / post_run()

maturity = 'alpha'
parse_conf(path: str, mode: str = 'r', profile: Dict | None = None, raw_exec: bool = False) ConfFileProxy
parse_extra_vars(vars: str, arg_name='argument') dict

Argument can be either a string, or a @file

post_run(args, exec_info=None)

Optional custom clean up method. Always called if run() was. The presence of exc_info indicates failure.


Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.

register_args(parser: ArgumentParser)

This function in passed the


Actual works happens here. Return code should be an EXIT_CODE_* from consts.

version_extra: str | None = None
ksconf.command.add_splunkd_access_args(parser: ArgumentParser) ArgumentParser
ksconf.command.add_splunkd_namespace(parser: ArgumentParser) ArgumentParser

Remove any common leading whitespace from every line in text.

This can be used to make triple-quoted strings line up with the left edge of the display, while still presenting them in the source code in indented form.

Note that tabs and spaces are both treated as whitespace, but they are not equal: the lines “ hello” and “thello” are considered to have no common leading whitespace.

Entirely blank lines are normalized to a newline character.

ksconf.command.get_entrypoints(group, name=None) Mapping

ksconf.compat module

Silly simple Python version compatibility items


alias of dict


alias of list


alias of set


alias of tuple

ksconf.compat.cache(user_function, /)

Simple lightweight unbounded cache. Sometimes called “memoize”.

ksconf.consts module

class ksconf.consts.SmartEnum(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

CREATE = 'created'
NOCHANGE = 'unchanged'
UPDATE = 'updated'

ksconf.filter module

class ksconf.filter.FilteredList(flags: int = 0, default: bool = True)

Bases: object

feed(item: str, filter: Callable[[str], str] | None = None)

Feed a new pattern into the rule set.

Use filter to enable pre-processing on patterns expressions. This is handled, after checking for specially values. Specifically, the file://... syntax is used to feed additional patterns from a file.

feedall(iterable: Sequence[str], filter: Callable[[str], str] | None = None)
property has_rules: bool
init_counter() Counter
match(item: str) bool

See if given item matches any of the given patterns. If no patterns were provided, default: will be returned.

match_path(path) bool

Same as match() except with special handling of path normalization. Patterns must be given with unix-style paths.

match_stanza(stanza) bool

Same as match(), but handle GLOBAL_STANZA gracefully.


Prepare for matching activities.

Called automatically by match(), but it could helpful to call directly to ensure there are no user input errors (which is accomplished by calling _pre_match()).

class ksconf.filter.FilteredListRegex(flags: int = 0, default: bool = True)

Bases: FilteredList

Regular Expression support

init_counter() Counter
class ksconf.filter.FilteredListSplunkGlob(flags: int = 0, default: bool = True)

Bases: FilteredListRegex

Classic wildcard support (‘*’ and ?’) plus ‘…’ or ‘**’ for multiple-path components with some (non-advertised) pass-through regex behavior

class ksconf.filter.FilteredListString(flags: int = 0, default: bool = True)

Bases: FilteredList

Handle simple string comparisons

init_counter() Counter
class ksconf.filter.FilteredListWildcard(flags: int = 0, default: bool = True)

Bases: FilteredListRegex

Wildcard support (handling ‘*’ and ?’) Technically fnmatch also supports [] and [!] character ranges, but we don’t advertise that

ksconf.filter.create_filtered_list(match_mode: str, flags: int = 0, default=True) FilteredList

ksconf.hook module

exception ksconf.hook.BadPluginWarning

Bases: UserWarning

Issue with one or more plugins

ksconf.hook.get_plugin_manager() _plugin_manager

Return the shared pluggy PluginManager (singleton) instance.

This is for backwards compatibility. This was only added in v0.11.6; and replaced immediately after.

ksconf.hookspec module

This module contains all the plugin definitions (or hook “specifications”) for various customization or integration points with ksconf. Not all of these have been fully tested so please let us know if something is not working as expected, or if additional arguments are needed.

See ksconf plugins on pypi for a list of currently available plugins.

class ksconf.hookspec.KsconfHookSpecs(*args, **kwargs)

Bases: Protocol

Ksconf plugin specifications for all known supported functions.

Grouping these functions together in a single class allows for type support it supports typing. This adds a level of validation to the code base where a hook is invoked via plugin_manger.hook.<hook_name>().

If you are implementing one of these hooks, please note that you can simple make top-level function, no need to implement a class.

static ksconf_cli_init()

Simple hook that is run before CLI initialization. This can be use to modify the runtime environment.

This can be used to register additional handlers, such as:

static ksconf_cli_modify_argparse(parser: Any, name: str)

Manipulate argparse rules. This could be used to add additional CLI options for other hook-added features added features

Note that this hook is called for both the top-level argparse instance as well as each subparser. The name argument should be inspected to determine if the parse instances is the parent (top-level) parser, or some other named subcommands.

static ksconf_cli_process_args(args: Any)

Hook to capture all parsed arguments, includes any custom arguments added to the CLI via the the ksconf_cli_modify_argparse() hook. args can be mutated directly, if needed.

static modify_jinja_env(env: Any)

Modify the Jinja2 environment object. This can be used to add custom filters or tests, for example.

Invoked by LayerFile_Jinja2 immediately after initial Environment creation. env should be mutated in place.

static package_pre_archive(app_dir: Path, app_name: str)

Modify, inventory, or test the contents of an app before the final packaging commands. This can be triggered from the ksconf package command or via the API.

During a ksconf package process, this hook executes right before the final archive is created. All local merging, app version or build updates, and so on are completed before this hook is executed.

From an API perspective, this hook is called from ksconf.package.AppPackager whenever a content freeze occurs, which is typically when make_archive() or make_manifest() is invoked.

static post_combine(target: Path, usage: str)

Trigger a custom action after a layer combining operation. This is used by multiple ksconf subcommands and the API.

This trigger could be used to modify the file system, trigger external operations, track/audit behaviors, and so on.

When using CLI commands, usage should be either “combine” or “package” depending on which ksconf command was invoked. Direct invocation of LayerCombiner can pass along a custom usage label and avoid impacting CLI, when desirable.

If your goal is to only trigger an action during the app packaging process, also consider the package_pre_archive() hook, which may be more appropriate.

exception ksconf.hookspec.KsconfPluginWarning

Bases: Warning

ksconf.layer module

class ksconf.layer.DirectLayerRoot(context: LayerContext | None = None)

Bases: LayerRootBase

A very simple direct LayerRoot implementation that relies on all layer paths to be explicitly given without any automatic detection mechanisms. You can think of this as the legacy implementation.

add_layer(path: Path)
class ksconf.layer.DotDLayerRoot(context=None)

Bases: LayerRootBase

class Layer(name: str, root: Path, physical: PurePath, logical: PurePath, context: LayerContext, file_factory: Callable, prune_points: set[Path] = None)

Bases: Layer

prune_points: set[Path]
walk() Iterator[tuple[Path, list[str], list[str]]]
apply_filter(layer_filter: LayerFilter)

Apply a destructive filter to all layers. layer_filter(layer) will be called one for each layer, if the filter returns True than the layer is kept. Root layers are always kept.

Returns True if layers were removed

layer_regex = re.compile('(?P<layer>\\d\\d-[\\w_.-]+)')
list_layers() List[Layer]
mount_regex = re.compile('(?P<realname>[\\w_.-]+)\\.d$')
set_root(root: Path, follow_symlinks=None)

Set a root path, and auto discover all ‘.d’ directories.

Note: We currently only support .d/<layer> directories, a file like default.d/10-props.conf won’t be handled here. A valid name would be default.d/10-name/props.conf.

class ksconf.layer.FileFactory

Bases: object

enable(name, _enabled=True)
list_available_handlers() list[str]
register_handler(name: str, **kwargs)
class ksconf.layer.LayerContext(follow_symlink: 'bool' = False, block_files: 'Pattern' = re.compile('\\.(bak|swp)$'), block_dirs: 'set' = <factory>, template_variables: 'dict' = <factory>)

Bases: object

block_dirs: set
block_files: Pattern = re.compile('\\.(bak|swp)$')
template_variables: dict
exception ksconf.layer.LayerException

Bases: Exception

class ksconf.layer.LayerFile(layer: Layer, relative_path: PurePath, stat: stat_result | None = None)

Bases: PathLike

Abstraction of a file within a Layer

Path definitions


Conceptual file path. This is the final path after all layers are resolved. Think of this as the ‘destination’ file.


Actual file path. The location of the physical file found within a source layer. Most of the time this is the ‘source’ file, however this doesn’t take into considerations layer combining or template expansion requirements. (In the case of a template, this would be the template file)


Content location. Often this the physical_path, but in the case of abstracted layers (like templates, or archived layers), this would be the location of a temporary resource that contains the expanded/rendered content.

property logical_path: Path
static match(path: PurePath)
property mtime
property physical_path: Path
property resource_path: Path
property size
property stat: stat_result
class ksconf.layer.LayerFile_Jinja2(*args, **kwargs)

Bases: LayerRenderedFile

property jinja2_env
static match(path: PurePath)
render(template_path: Path) str
static transform_name(path: PurePath)
class ksconf.layer.LayerFilter

Bases: object

add_rule(action, pattern)
evaluate(layer: Layer) bool
class ksconf.layer.LayerRenderedFile(*args, **kwargs)

Bases: LayerFile

Abstract LayerFile for rendered scenarios, such as template scenarios. A subclass really only needs to implement match() render()

property logical_path: Path
property physical_path: Path
render(template_path: Path) str
property resource_path: Path
static transform_name(path: PurePath)
use_secure_delete = False
class ksconf.layer.LayerRootBase(context: LayerContext | None = None)

Bases: object

All ‘path’s here are relative to the ROOT.

class Layer(name: str, root: Path, physical: PurePath, logical: PurePath, context: LayerContext, file_factory: Callable)

Bases: object

Basic layer Container: Connects logical and physical paths.

get_file(path: Path) LayerFile | None

Return file object (by logical path), if it exists in this layer.

iter_files() Iterator[LayerFile]
list_files() list[LayerFile]
walk() Iterator[tuple[Path, list[str], list[str]]]
add_layer(layer: Layer, do_sort=True)
apply_filter(layer_filter: LayerFilter) bool

Apply a destructive filter to all layers. layer_filter(layer) will be called one for each layer, if the filter returns True than the layer is kept. Root layers are always kept.

Returns True if layers were removed

get_file(path) Iterator[LayerFile]

return all layers associated with the given relative path.

get_layers_by_name(name: str) Iterator[Layer]
iter_all_files() Iterator[LayerFile]

Iterator over all physical files.

list_files() list[LayerFile]

Return a list of logical paths.

list_layer_names() list[str]
list_layers() List[Layer]
list_logical_files() list[LayerFile]

Return a list of logical paths.

list_physical_files() list[LayerFile]
exception ksconf.layer.LayerUsageException

Bases: LayerException

ksconf.layer.register_file_handler(name: str, **kwargs)

ksconf.package module

class ksconf.package.AppPackager(src_path, app_name: str, output: TextIO, template_variables: dict | None = None, predictable_mtime: bool = True)

Bases: object


Run safety checks prior to building archive:

  1. Set app name based on app.conf [package] id, if set. Otherwise confirm that the package id and top-level folder names align.

  2. Check for files or directories starting with ., makes AppInspect very grumpy!

combine(src, filters, layer_method='dir.d', allow_symlink=False)
expand_new_only(value: str) str | bool

Expand a variable but return False if no substitution occurred


value (str) – String that may contain {{variable}} substitution.


Expanded value if variables were expanded, else False

Return type:


expand_var(value: str) str

Expand a variable, if present


value (str) – String that main contain {{variable}} substitution.


Expanded value

Return type:



Initiate a content freeze by restricting mutable methods. The “package_pre_archive” hook is invoked before freeze operation. Such hooks may choose to mutate the filesystem at app_dir, the only assumption is that all work is done before the hook returns.

Freeze can be safely called multiple times. caller_name is simply a label used in an exception message if the programmer screwed up.

make_archive(filename: str, temp_suffix: str = '.tmp') str

Create a compressed tarball of the build directory.

make_manifest(calculate_hash=True) AppManifest

Create a manifest of the app’s contents.


Find everything in local, if it has a corresponding file in default, merge.


Decorator to mark member functions that cannot be used until the context manager has been activated.

update_app_conf(version: str | None = None, build: str | None = None)

Update version and/or build in apps.conf

class ksconf.package.AppVarMagic(src_dir, build_dir, meta=None)

Bases: object

A lazy loading dict-like object to fetch things like app version and such on demand.

expand(value: str) str

A simple Jinja2 like {{VAR}} substitution mechanism.


Splunk app package id from app.conf


Splunk app build fetched from app.conf


Git HEAD rev abbreviated


Git abbreviated rev of the last change of the app. This may not be the same as HEAD.


Git version tag using the git describe --tags command


Build a unique hash representing the combination of ksconf layers used.


List of ksconf layers used.


Splunk app version fetched from app.conf


Return a list of (variable, description) available in this class.

exception ksconf.package.AppVarMagicException

Bases: KeyError

exception ksconf.package.PackagingException

Bases: Exception

ksconf.package.find_conf_in_layers(app_dir, conf, *layers)
ksconf.package.get_merged_conf(app_dir, conf, *layers)

Walk a tree and update the directory modification times to match the newest time of the children. This results in a more predictable behavior over multiple executions.

ksconf.setup_entrypoints module

Defines all command prompt entry points for CLI actions

This is a silly hack allows for fallback mechanism when
  1. running unit tests (can happen before install)

  2. unexpected issues with importlib.metadata or backport

class ksconf.setup_entrypoints.Ep(name, module_name, object_name)

Bases: NamedTuple

property formatted
module_name: str

Alias for field number 1

name: str

Alias for field number 0

object_name: str | None

Alias for field number 2

class ksconf.setup_entrypoints.LocalEntryPoint(data)

Bases: object

Bare minimum stand-in for entrypoints.EntryPoint

ksconf.setup_entrypoints.get_entrypoints_fallback(group) dict

Build entry point text descriptions for ksconf packaging

ksconf.types module

ksconf.version module

ksconf.version: Version and release info for the core ksconf package

ksconf.xmlformat module

class ksconf.xmlformat.FileReadlinesCache

Bases: object

Silly workaround for CDATA detection…

static convert_filename(filename)
class ksconf.xmlformat.SplunkSimpleXmlFormatter

Bases: object

static cdata_tags(elem: Any, tags: List[str])

Expand text to CDATA, if it isn’t already.

classmethod expand_tags(elem: Any, tags: set)

Keep <elem></elem> instead of shortening to <elem/>

classmethod format_json(elem: Any, indent=2)

Format JSON data within a Dashboard Studio dashboard. This is still pretty limited (for example, long searches still show up on a single line), but this give you at least a fighting change to figure out what’s different.

classmethod format_xml(src, dest, default_indent=2)
static guess_indent(elem: Any, default=2)
classmethod indent_tree(elem: Any, level=0, indent=2)
keep_tags = {'default', 'earliest', 'fieldset', 'label', 'latest', 'option', 'search', 'set'}