ksconf.commands namespace

Submodules

ksconf.commands.attr module

r SUBCOMMAND: ksconf attr-get <CONF> --stanza STANZA --attribute ATTR

ksconf attr-get $SPLUNK_HOME/etc/apps/Splunk_TA_aws/default/app.conf --stanza launcher --attribute version

SUBCOMMAND: ksconf attr-set <CONF> --stanza STANZA --attribute ATTR --value VALUE

ksconf attr-set $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/app.conf         --stanza launcher --attribute version --value 9.9.9

echo "9.9.9" > /tmp/new_version
ksconf attr-set $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/app.conf         --stanza launcher --attribute version  -t file /tmp/new_version

export NEW_VERSION=1.2.3
ksconf attr-set $SPLUNK_HOME/etc/apps/Splunk_TA_aws/local/app.conf         --stanza launcher --attribute version -t env NEW_VERSION

class ksconf.commands.attr.AttrGetCmd(name)

Bases: KsconfCmd

description: str | None = 'Get a specific stanza and attribute value from a Splunk .conf file.\n'

format = 'manual'

help: str | None = 'Get the value from a specific stanzas and attribute'

maturity = 'beta'

pre_run(args)

Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.

register_args(parser)

This function in passed the

run(args)

For a given conf file, get the ‘value’ from [stanza] attribute = value

class ksconf.commands.attr.AttrSetCmd(name)

Bases: KsconfCmd

description: str | None = 'Set a specific stanza and attribute value of a Splunk .conf file.\nThe value can be provided as a command line argument, file, or\nenvironment variable\n\nThis command does not support preserving leading or trailing whitespace.\nNormally this is desireable.\n'

format = 'manual'

get_value(value, value_type)

help: str | None = 'Set the value of a specific stanzas and attribute'

maturity = 'beta'

register_args(parser)

This function in passed the

run(args)

For a given conf file, set [stanza] attribute = value

set_conf_value(conf_file: Path, stanza: str, attribute: str, value: str, create_missing: bool, no_overwrite: bool)

ksconf.commands.check module

SUBCOMMAND: ksconf check <CONF>

Usage example: (Nice pre-commit script)

find . -name '*.conf' | ksconf check -

class ksconf.commands.check.CheckCmd(name)

Bases: KsconfCmd

description: str | None = "\nProvides basic syntax and sanity checking for Splunk's .conf\nfiles.  Use Splunk's built-in ``btool check`` for a more robust\nvalidation of attributes and values.\n\nConsider using this utility as part of a pre-commit hook."

help: str | None = 'Perform basic syntax and sanity checks on .conf files'

maturity = 'stable'

pre_run(args)

Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.

register_args(parser)

This function in passed the

run(args)

Actual works happens here. Return code should be an EXIT_CODE_* from consts.

ksconf.commands.combine module

SUBCOMMAND: combine --target=<DIR> <SRC1> [ <SRC-n> ]

Usage example:

cd MY_APP
ksconf combine default.d/* --target=default

class ksconf.commands.combine.CombineCmd(name)

Bases: KsconfCmd

description: str | None = "Merge .conf settings from multiple source directories into a combined target\ndirectory.  Configuration files can be stored in a ``/etc/*.d`` like directory\nstructure and consolidated back into a single 'default' directory.\n\nThis command supports both one-time operations and recurring merge jobs.  For\nexample, this command can be used to combine all users' knowledge objects (stored\nin 'etc/users') after a server migration, or to merge a single user's settings\nafter their account has been renamed.  Recurring operations assume some type\nof external scheduler is being used.  A best-effort is made to only write to\ntarget files as needed.\n\nThe 'combine' command takes your logical layers of configs (upstream, corporate,\nSplunk admin fixes, and power user knowledge objects, ...) expressed as\nindividual folders and merges them all back into the single ``default`` folder\nthat Splunk reads from.  One way to keep the 'default' folder up-to-date is\nusing client-side git hooks.\n\nNo directory layout is mandatory, but taking advantages of the native-support\nfor 'dir.d' layout works well for many uses cases.  This idea is borrowed from\nthe Unix System V concept where many services natively read their config files\nfrom ``/etc/*.d`` directories.\n\nVersion notes:  dir.d was added in ksconf 0.8.  Starting in 1.0 the default will\nswitch to 'dir.d', so if you need the old behavior be sure to update your scripts.\n"

format = 'manual'

help: str | None = 'Combine configuration files across multiple source directories into a single\ndestination directory.  This allows for an arbitrary number of Splunk\nconfiguration layers to coexist within a single app.  Useful in both ongoing\nmerge and one-time ad-hoc use.\n'

maturity = 'beta'

register_args(parser)

This function in passed the

run(args)

Actual works happens here. Return code should be an EXIT_CODE_* from consts.

exception ksconf.commands.combine.LayerCombinerExceptionCode(msg, return_code=None)

Bases: LayerCombinerException

class ksconf.commands.combine.RepeatableCombiner(*args, disable_marker: bool = False, disable_cleanup: bool = False, keep_existing: bool = False, **kwargs)

Bases: LayerCombiner

Re-runable combiner class. Beyond the reusable layer combining functionality, this class enables the use of a marker file for added safety. Removed files will cleanup.

post_combine(target: Path)

Handle cleanup of extra files

pre_combine_inventory(target: Path, src_files: Sequence[LayerFile]) → set[LayerFile]

Find a set of files that exist in the target folder, but in NO source folder (for cleanup)

prepare_target_dir(target: Path)

Handle marker file and ensure that target directory gets created safely.

ksconf.commands.diff module

SUBCOMMAND: ksconf diff <CONF> <CONF>

Usage example:

ksconf diff default/props.conf default/props.conf

class ksconf.commands.diff.DiffCmd(name)

Bases: KsconfCmd

description: str | None = "Compares the content differences of two .conf files\n\nThis command ignores textual differences (like order, spacing, and comments) and\nfocuses strictly on comparing stanzas, keys, and values.  Note that spaces within\nany given value, will be compared. Multi-line fields are compared in a more traditional\n'diff' output so that long saved searches and macros can be compared more easily.\n"

format = 'manual'

help: str | None = 'Compare settings differences between two .conf files ignoring spacing and sort order'

maturity = 'stable'

register_args(parser)

This function in passed the

run(args)

Compare two configuration files.

ksconf.commands.filter module

SUBCOMMAND: ksconf filter <CONF>

Usage example:

ksconf filter default/savedsearches.conf --stanza "My Special Search" -o my-special-search.conf

Future things to support:

• SED-like rewriting for stanza name or key values.

• Mini eval/query language for simple data manipulations supporting mixed used of matching modes on a case-by-base basis, custom logic (AND,OR,arbitrary groups), projections, and content rewriting. (Should leverage custom ‘combine’ mini-language where possible.)

class ksconf.commands.filter.FilterCmd(*args, **kwargs)

Bases: KsconfCmd

description: str | None = '\nFilter the contents of a conf file in various ways.  Stanzas can be included\nor excluded based on a provided filter or based on the presence or value of a key.\n\nWhere possible, this command supports GREP-like arguments to bring a familiar feel.\n'

filter_attrs(content: dict) → dict

help: str | None = 'A stanza-aware GREP tool for conf files'

maturity = 'alpha'

output(args, matches: dict, filename)

Process output for a single input file.

prep_filters(args)

register_args(parser: ArgumentParser)

This function in passed the

run(args)

Filter configuration files.

ksconf.commands.filter.is_disabled(attributes: dict) → bool

ksconf.commands.merge module

SUBCOMMAND: ksconf merge --target=<TARGET_CONF> <CONF> [ <CONF-n> ... ]

Usage example:

ksconf merge --target=master-props.conf /opt/splunk/etc/apps/*TA*/{default,local}/props.conf

class ksconf.commands.merge.MergeCmd(name)

Bases: KsconfCmd

description: str | None = 'Merge two or more .conf files into a single combined .conf file.\nThis is similar to the way that Splunk logically combines the ``default`` and ``local``\nfolders at runtime.\n'

help: str | None = 'Merge two or more .conf files'

maturity = 'stable'

pre_run(args)

Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.

register_args(parser)

This function in passed the

run(args)

Merge multiple configuration files into one

ksconf.commands.minimize module

SUBCOMMAND: ksconf minimize --target=<CONF> <CONF> [ <CONF-n> ... ]

Usage example:

ksconf minimize --target=local/inputs.conf default/inputs.conf

Example workflow:

1. cp default/props.conf local/props.conf

2. vi local/props.conf (edit JUST the lines you want to change)

3. ksconf minimize --target=local/props.conf default/props.conf (You could take this a step further by appending “$SPLUNK_HOME/system/default/props.conf” and removing any SHOULD_LINEMERGE = true entries (for example)

class ksconf.commands.minimize.MinimizeCmd(name)

Bases: KsconfCmd

description: str | None = "Minimize a conf file by removing any duplicated default settings.\n\nReduce a local conf file to only your intended changes without manually tracking\nwhich entries you've edited.  Minimizing local conf files makes your local\ncustomizations easier to read and often results in cleaner upgrades.\n"

help: str | None = 'Minimize the target file by removing entries duplicated in the default conf(s)'

maturity = 'beta'

register_args(parser)

This function in passed the

run(args)

Actual works happens here. Return code should be an EXIT_CODE_* from consts.

ksconf.commands.minimize.explode_default_stanza(conf, default_stanza=None)

Take the GLOBAL stanza, (aka [default]) and apply it’s settings underneath ALL other stanzas. This is mostly only useful in minimizing and other comparison operations.

ksconf.commands.package module

SUBCOMMAND: ksconf package -f <SPL> <DIR>

Usage example:

ksconf package -f myapp.tgz MyApp/

Build system example:

ksconf package -f release/myapp-{{version}}.tgz \
        --block-local \
        --set-version={{git_tag}} \
        --set-build=${TRAVIS_BUILD_NUMBER:-0}

class ksconf.commands.package.PackageCmd(name)

Bases: KsconfCmd

default_blocklist = ['.git*', '*.py[co]', '__pycache__', '.DS_Store']

description: str | None = 'Create a Splunk app or add on tarball (``.spl``) file from an app directory.\n\n``ksconf package`` can do useful things like, exclude unwanted files, combine layers, set the\napplication version and build number, drop or promote the ``local`` directory into ``default``.\n\nNote that some arguments, like the ``FILE`` support special values that can be automatically\nevaluated at runtime.  For example the placeholders ``{{version}}`` or ``{{git_tag}}`` can be\nexpanded into the output tarball filename.\n\nIf both layering and templating are in use at the same time, be aware that templates are\nrendered prior to layering operations.  This allows, for example, one layer to include a simple\n``indexes.conf`` file and another layer to include an ``indexes.conf.j2`` template.\n'

help: str | None = 'Create a Splunk app .spl file from a source directory'

static load_blocklist(path: str) → Iterable[str]

maturity = 'beta'

pre_run(args)

Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.

register_args(parser: ArgumentParser)

This function in passed the

run(args)

Create a Splunk app/add-on .spl file from a directory

ksconf.commands.promote module

SUBCOMMAND: ksconf promote <SOURCE> <TARGET>

Usage example: Promote local props changes (made via the UI) to the ‘default’ folder

ksconf local/props.conf default/props.conf

class ksconf.commands.promote.PromoteCmd(name)

Bases: KsconfCmd

apply_filters(delta, invert_match=False)

static combine_stanza(a, b)

description: str | None = 'Propagate .conf settings applied in one file to another.  Typically this is used\nto move ``local`` changes (made via the UI) into another layer, such as the\n``default`` or a named ``default.d/50-xxxxx``) folder.\n\nPromote has two modes:  batch and interactive.  In batch mode, all changes are\napplied automatically and the (now empty) source file is removed.  In interactive\nmode, the user is prompted to select stanzas to promote.  This way local changes\ncan be held without being promoted.\n\nNOTE: Changes are *MOVED* not copied, unless ``--keep`` is used.\n'

format = 'manual'

help: str | None = 'Promote .conf settings between layers using either batch or interactive mode.\n\nFrequently this is used to promote conf changes made via the UI (stored in\nthe ``local`` folder) to a version-controlled directory, such as ``default``.\n'

maturity = 'beta'

prep_filters(args)

register_args(parser: ArgumentParser)

This function in passed the

run(args)

Actual works happens here. Return code should be an EXIT_CODE_* from consts.

ksconf.commands.promote.empty_dict(d)

ksconf.commands.restexport module

SUBCOMMAND: ksconf rest-export --output=script.sh <CONF>

Usage example:

ksconf rest-export --output=apply_props.sh /opt/splunk/etc/app/Splunk_TA_aws/local/props.conf

NOTE:

If we add support for Windows CURL, then we’ll need to also support proper quoting for the ‘%’ character. This can be done with ‘%^’, wonky, I know…

class ksconf.commands.restexport.CurlCommand

Bases: object

extend_args(args)

get_command()

classmethod quote(s)

class ksconf.commands.restexport.Literal(value)

Bases: object

class ksconf.commands.restexport.RestExportCmd(name)

Bases: KsconfCmd

static build_rest_url(base, owner, app, conf)

description: str | None = "Build an executable script of the stanzas in a configuration file that can be later applied to\na running Splunk instance via the Splunkd REST endpoint.\n\nThis can be helpful when pushing complex props and transforms to an instance where you only have\nUI access and can't directly publish an app.\n\n"

format = 'manual'

help: str | None = 'Export .conf settings as a curl script to apply to a Splunk instance later (via REST)'

maturity = 'deprecated'

register_args(parser: ArgumentParser)

This function in passed the

run(args)

Convert a conf file into a bunch of CURL commands

ksconf.commands.restpublish module

SUBCOMMAND: ksconf rest-publish <ENDPOINT> <CONF>

Usage example:

ksconf rest-publish MyApp/local/props.conf

class ksconf.commands.restpublish.RestPublishCmd(*args, **kwargs)

Bases: KsconfCmd

connect_splunkd(args: Namespace)

delete_conf(stanza_name: str, stanza_data: Dict[str, Dict[str, str]], config_file)

description: str | None = "Publish stanzas in a .conf file to a running Splunk instance via REST.  This requires access to\nthe HTTPS endpoint of Splunk.  By default, ksconf will handle both the creation of new stanzas\nand the update of existing stanzas.\n\nThis can be used to push full configuration stanzas where you only have REST access and can't\ndirectly publish an app.\n\nOnly attributes present in the conf file are pushed.  While this may seem obvious, this fact can\nhave profound implications in certain situations, like when using this command for continuous\nupdates.  This means that it's possible for the source .conf to ultimately differ from what ends\nup on the server's .conf file.  One way to avoid this, is to explicitly remove an object using\n``--delete`` mode first, and then insert a new copy of the object.  Of course, this means that\nthe object will be unavailable.  The other impact is that diffs only compares and shows a subset\nof attribute.\n\nBe aware, that for consistency, the configs/conf-TYPE endpoint is used for this command.\nTherefore, a reload may be required for the server to use the published config settings.\n"

handle_conf_file(args: Namespace, conf_proxy: ConfFileProxy)

help: str | None = 'Publish .conf settings to a live Splunk instance via REST'

static make_boolean(stanza: Dict[str, Dict[str, str]], attr: str = 'disabled')

maturity = 'alpha'

publish_conf(stanza_name: str, stanza_data: Dict[str, Dict[str, str]], config_file)

register_args(parser: ArgumentParser)

This function in passed the

run(args: Namespace)

Actual works happens here. Return code should be an EXIT_CODE_* from consts.

ksconf.commands.snapshot module

SUBCOMMAND: ksconf snapshot --output=FILE.json <PATH> [ ... <PATH-n> ]

Usage example:

ksconf snapshot --output=daily.json /opt/splunk/etc/app/

class ksconf.commands.snapshot.ConfSnapshot(config)

Bases: object

schema_version = 0.2

snapshot_dir(path)

snapshot_file_conf(path)

write_snapshot(stream, **kwargs)

class ksconf.commands.snapshot.ConfSnapshotConfig

Bases: object

max_file_size = 10485760

class ksconf.commands.snapshot.SnapshotCmd(name)

Bases: KsconfCmd

description: str | None = 'Build a static snapshot of various configuration files stored within a structured json export\nformat.  If the .conf files being captured are within a standard Splunk directory structure,\nthen certain metadata and namespace information is assumed based on typical path locations.\nIndividual apps or conf files can be collected as well, but less metadata may be extracted.\n'

help: str | None = 'Snapshot .conf file directories into a JSON dump format'

register_args(parser)

This function in passed the

run(args)

Snapshot multiple configuration files into a single json snapshot.

ksconf.commands.sort module

SUBCOMMAND: ksconf sort <CONF>

Usage example: To recursively sort all files (in-place):

find . -name '*.conf' | xargs ksconf sort -i

class ksconf.commands.sort.SortCmd(name)

Bases: KsconfCmd

description: str | None = 'Sort a Splunk .conf file.  Sort has two modes:  (1) by default, the sorted\nconfig file will be echoed to the screen.  (2) the config files are updated\nin-place when the ``-i`` option is used.\n\nManually managed conf files can be protected against changes by adding a comment containing the\nstring ``KSCONF-NO-SORT`` to the top of any .conf file.\n'

format = 'manual'

help: str | None = 'Sort a Splunk .conf file creating a normalized format appropriate for version control'

maturity = 'stable'

pre_run(args)

Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.

register_args(parser)

This function in passed the

run(args)

Sort one or more configuration file.

ksconf.commands.unarchive module

SUBCOMMAND: ksconf unarchive <tarball>

Usage example:

ksconf unarchive splunk-add-on-for-amazon-web-services_111.tgz

class ksconf.commands.unarchive.UnarchiveCmd(name)

Bases: KsconfCmd

description: str | None = "\nInstall or overwrite an existing app in a git-friendly way.\nIf the app already exists, steps will be taken to upgrade it safely.\n\nThe ``default`` folder can be redirected to another path (i.e., ``default.d/10-upstream`` or\nother desirable path if you're using the ``ksconf combine`` tool to manage extra layers).\n"

format = 'manual'

help: str | None = 'Install or upgrade an existing app in a git-friendly and safe way'

maturity = 'beta'

register_args(parser)

This function in passed the

run(args)

Install / upgrade a Splunk app from an archive file

ksconf.commands.unarchive.fixup_pattern_bw(patterns, prefix=None)

ksconf.commands.xmlformat module

SUBCOMMAND: ksconf xml-format <XML>

Usage example: (Nice pre-commit script)

find default/data/ui -name '*.xml' | ksconf xml-format -

class ksconf.commands.xmlformat.XmlFormatCmd(name)

Bases: KsconfCmd

description: str | None = "\nNormalize and apply consistent XML indentation and CDATA usage for XML dashboards and\nnavigation files.\n\nTechnically this could be used on *any* XML file, but certain element names specific to Splunk's\nsimple XML dashboards are handled specially, and therefore could result in unusable results.\n\nThe expected indentation level is guessed based on the first element indentation, but can be\nexplicitly set if not detectable.\n"

help: str | None = 'Normalize XML view and nav files'

maturity = 'alpha'

pre_commit_repo_migration_warning(args)

Issue migration warning if (1) running hooks from the old repo (missing arg), and (2) parent process is from pre-commit (env var).

Another workaround is to use:

- repo: https://github.com/Kintyre/ksconf
rev: v0.11.8
hooks:
    - id: ksconf-check
    - id: ksconf-sort
    exclude: logging\.conf
    - id: ksconf-xml-format
      args: --disable-pre-commit-migration-check
additional_dependencies: [lxml]

But honestly, isn’t it just easy to add -pre-commit to the repo?

Remove this after Dec 2024 or v0.13.0

register_args(parser)

This function in passed the

run(args)

Actual works happens here. Return code should be an EXIT_CODE_* from consts.