ksconf.commands package¶
Submodules¶
ksconf.commands.check module¶
SUBCOMMAND: ksconf check <CONF>
Usage example: (Nice pre-commit script)
find . -name '*.conf' | ksconf check -
-
class
ksconf.commands.check.
CheckCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= "\nProvides basic syntax and sanity checking for Splunk's .conf\nfiles. Use Splunk's built-in ``btool check`` for a more robust\nvalidation of attributes and values.\n\nConsider using this utility as part of a pre-commit hook."¶
-
help
= 'Perform basic syntax and sanity checks on .conf files'¶
-
maturity
= 'stable'¶
-
pre_run
(args)¶ Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Actual works happens here. Return code should be an EXIT_CODE_* from consts.
-
ksconf.commands.combine module¶
SUBCOMMAND: combine --target=<DIR> <SRC1> [ <SRC-n> ]
Usage example:
cd MY_APP
ksconf combine default.d/* --target=default
-
class
ksconf.commands.combine.
CombineCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= "Merge .conf settings from multiple source directories into a combined target\ndirectory. Configuration files can be stored in a ``/etc/*.d`` like directory\nstructure and consolidated back into a single 'default' directory.\n\nThis command supports both one-time operations and recurring merge jobs. For\nexample, this command can be used to combine all users' knowledge objects (stored\nin 'etc/users') after a server migration, or to merge a single user's settings\nafter their account has been renamed. Recurring operations assume some type\nof external scheduler is being used. A best-effort is made to only write to\ntarget files as needed.\n\nThe 'combine' command takes your logical layers of configs (upstream, corporate,\nSplunk admin fixes, and power user knowledge objects, ...) expressed as\nindividual folders and merges them all back into the single ``default`` folder\nthat Splunk reads from. One way to keep the 'default' folder up-to-date is\nusing client-side git hooks.\n\nNo directory layout is mandatory, but taking advantages of the native-support\nfor 'dir.d' layout works well for many uses cases. This idea is borrowed from\nthe Unix System V concept where many services natively read their config files\nfrom ``/etc/*.d`` directories.\n\nVersion notes: dir.d was added in ksconf 0.8. Starting in 1.0 the default will\nswitch to 'dir.d', so if you need the old behavior be sure to update your scripts.\n"¶
-
format
= 'manual'¶
-
help
= 'Combine configuration files across multiple source directories into a single\ndestination directory. This allows for an arbitrary number of Splunk\nconfiguration layers to coexist within a single app. Useful in both ongoing\nmerge and one-time ad-hoc use.\n'¶
-
maturity
= 'beta'¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Actual works happens here. Return code should be an EXIT_CODE_* from consts.
-
ksconf.commands.diff module¶
SUBCOMMAND: ksconf diff <CONF> <CONF>
Usage example:
ksconf diff default/props.conf default/props.conf
-
class
ksconf.commands.diff.
DiffCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= "Compares the content differences of two .conf files\n\nThis command ignores textual differences (like order, spacing, and comments) and\nfocuses strictly on comparing stanzas, keys, and values. Note that spaces within\nany given value, will be compared. Multi-line fields are compared in a more traditional\n'diff' output so that long saved searches and macros can be compared more easily.\n"¶
-
format
= 'manual'¶
-
help
= 'Compare settings differences between two .conf files ignoring spacing and sort order'¶
-
maturity
= 'stable'¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Compare two configuration files.
-
ksconf.commands.filter module¶
SUBCOMMAND: ksconf filter <CONF>
Usage example:
ksconf filter default/savedsearches.conf --stanza "My Special Search" -o my-special-search.conf
Future things to support:
- SED-like rewriting for stanza name or key values.
- Mini eval/query language for simple data manipulations supporting mixed used of matching modes on a case-by-base basis, custom logic (AND,OR,arbitrary groups), projections, and content rewriting. (Should leverage custom ‘combine’ mini-language where possible.)
-
class
ksconf.commands.filter.
FilterCmd
(*args, **kwargs)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= '\nFilter the contents of a conf file in various ways. Stanzas can be included\nor excluded based on a provided filter or based on the presence or value of a key.\n\nWhere possible, this command supports GREP-like arguments to bring a familiar feel.\n'¶
-
filter_attrs
(content)¶
-
help
= 'A stanza-aware GREP tool for conf files'¶
-
maturity
= 'alpha'¶
-
output
(args, matches, filename)¶
-
prep_filters
(args)¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Filter configuration files.
-
ksconf.commands.merge module¶
SUBCOMMAND: ksconf merge --target=<CONF> <CONF> [ <CONF-n> ... ]
Usage example:
ksconf merge --target=master-props.conf /opt/splunk/etc/apps/*TA*/{default,local}/props.conf
-
class
ksconf.commands.merge.
MergeCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= 'Merge two or more .conf files into a single combined .conf file.\nThis is similar to the way that Splunk logically combines the ``default`` and ``local``\nfolders at runtime.\n'¶
-
help
= 'Merge two or more .conf files'¶
-
maturity
= 'stable'¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Merge multiple configuration files into one
-
ksconf.commands.minimize module¶
SUBCOMMAND: ksconf minimize --target=<CONF> <CONF> [ <CONF-n> ... ]
Usage example:
ksconf minimize --target=local/inputs.conf default/inputs.conf
- Example workflow:
cp default/props.conf local/props.conf
vi local/props.conf
(edit JUST the lines you want to change)ksconf minimize --target=local/props.conf default/props.conf
(You could take this a step further by appending “$SPLUNK_HOME/system/default/props.conf” and removing any SHOULD_LINEMERGE = true entries (for example)
-
class
ksconf.commands.minimize.
MinimizeCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= "Minimize a conf file by removing any duplicated default settings.\n\nReduce a local conf file to only your intended changes without manually tracking\nwhich entries you've edited. Minimizing local conf files makes your local\ncustomizations easier to read and often results in cleaner upgrades.\n"¶
-
help
= 'Minimize the target file by removing entries duplicated in the default conf(s)'¶
-
maturity
= 'beta'¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Actual works happens here. Return code should be an EXIT_CODE_* from consts.
-
-
ksconf.commands.minimize.
explode_default_stanza
(conf, default_stanza=None)¶ Take the GLOBAL stanza, (aka [default]) and apply it’s settings underneath ALL other stanzas. This is mostly only useful in minimizing and other comparison operations.
ksconf.commands.package module¶
SUBCOMMAND: ksconf package -f <SPL> <DIR>
Usage example:
ksconf package -f myapp.tgz MyApp/
Build system example:
ksconf package -f release/myapp-{{version}}.tgz \
--block-local \
--set-version={{git_tag}} \
--set-build=${TRAVIS_BUILD_NUMBER:-0}
-
class
ksconf.commands.package.
PackageCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
default_blocklist
= ['.git*', '*.py[co]', '__pycache__', '.DS_Store']¶
-
description
= 'Create a Splunk app or add on tarball (``.spl``) file from an app directory.\n\n``ksconf package`` can do useful things like, exclude unwanted files, combine layers, set the\napplication version and build number, drop or promote the ``local`` directory into ``default``.\n\nNote that some arguments, like the ``FILE`` support special values that can be automatically\nevaluated at runtime. For example the placeholders ``{{version}}`` or ``{{git_tag}}`` can be\nexpanded into the output tarball filename.\n'¶
-
help
= 'Create a Splunk app .spl file from a source directory'¶
-
static
load_blocklist
(path)¶
-
maturity
= 'alpha'¶
-
pre_run
(args)¶ Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Create a Splunk app/add-on .spl file from a directory
-
ksconf.commands.promote module¶
SUBCOMMAND: ksconf promote <SOURCE> <TARGET>
Usage example: Promote local props changes (made via the UI) to the ‘default’ folder
ksconf local/props.conf default/props.conf
-
class
ksconf.commands.promote.
PromoteCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
apply_filters
(delta, invert_match=False)¶
-
static
combine_stanza
(a, b)¶
-
description
= 'Propagate .conf settings applied in one file to another. Typically this is used\nto move ``local`` changes (made via the UI) into another layer, such as the\n``default`` or a named ``default.d/50-xxxxx``) folder.\n\nPromote has two modes: batch and interactive. In batch mode, all changes are\napplied automatically and the (now empty) source file is removed. In interactive\nmode, the user is prompted to select stanzas to promote. This way local changes\ncan be held without being promoted.\n\nNOTE: Changes are *MOVED* not copied, unless ``--keep`` is used.\n'¶
-
format
= 'manual'¶
-
help
= 'Promote .conf settings between layers using either batch or interactive mode.\n\nFrequently this is used to promote conf changes made via the UI (stored in\nthe ``local`` folder) to a version-controlled directory, such as ``default``.\n'¶
-
maturity
= 'beta'¶
-
prep_filters
(args)¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Actual works happens here. Return code should be an EXIT_CODE_* from consts.
-
-
ksconf.commands.promote.
empty_dict
(d)¶
ksconf.commands.restexport module¶
SUBCOMMAND: ksconf rest-export --output=script.sh <CONF>
Usage example:
ksconf rest-export --output=apply_props.sh /opt/splunk/etc/app/Splunk_TA_aws/local/props.conf
NOTE:
If we add support for Windows CURL, then we’ll need to also support proper quoting for the ‘%’ character. This can be done with ‘%^’, wonky, I know…
-
class
ksconf.commands.restexport.
CurlCommand
¶ Bases:
object
-
extend_args
(args)¶
-
get_command
()¶
-
classmethod
quote
(s)¶
-
-
class
ksconf.commands.restexport.
Literal
(value)¶ Bases:
object
-
class
ksconf.commands.restexport.
RestExportCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
static
build_rest_url
(base, owner, app, conf)¶
-
description
= "Build an executable script of the stanzas in a configuration file that can be later applied to\na running Splunk instance via the Splunkd REST endpoint.\n\nThis can be helpful when pushing complex props and transforms to an instance where you only have\nUI access and can't directly publish an app.\n\n"¶
-
format
= 'manual'¶
-
help
= 'Export .conf settings as a curl script to apply to a Splunk instance later (via REST)'¶
-
maturity
= 'beta'¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Convert a conf file into a bunch of CURL commands
-
static
ksconf.commands.restpublish module¶
SUBCOMMAND: ksconf rest-publish <ENDPOINT> <CONF>
Usage example:
ksconf rest-publish MyApp/local/props.conf
-
class
ksconf.commands.restpublish.
RestPublishCmd
(*args, **kwargs)¶ Bases:
ksconf.commands.KsconfCmd
-
connect_splunkd
(args)¶
-
delete_conf
(stanza_name, stanza_data, config_file)¶
-
description
= "Publish stanzas in a .conf file to a running Splunk instance via REST. This requires access to\nthe HTTPS endpoint of Splunk. By default, ksconf will handle both the creation of new stanzas\nand the update of existing stanzas.\n\nThis can be used to push full configuration stanzas where you only have REST access and can't\ndirectly publish an app.\n\nOnly attributes present in the conf file are pushed. While this may seem obvious, this fact can\nhave profound implications in certain situations, like when using this command for continuous\nupdates. This means that it's possible for the source .conf to ultimately differ from what ends\nup on the server's .conf file. One way to avoid this, is to explicitly remove an object using\n``--delete`` mode first, and then insert a new copy of the object. Of course, this means that\nthe object will be unavailable. The other impact is that diffs only compares and shows a subset\nof attribute.\n\nBe aware, that for consistency, the configs/conf-TYPE endpoint is used for this command.\nTherefore, a reload may be required for the server to use the published config settings.\n"¶
-
handle_conf_file
(args, conf_proxy)¶
-
help
= 'Publish .conf settings to a live Splunk instance via REST'¶
-
static
make_boolean
(stanza, attr='disabled')¶
-
maturity
= 'alpha'¶
-
publish_conf
(stanza_name, stanza_data, config_file)¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Actual works happens here. Return code should be an EXIT_CODE_* from consts.
-
ksconf.commands.snapshot module¶
SUBCOMMAND: ksconf snapshot --output=FILE.json <PATH> [ ... <PATH-n> ]
Usage example:
ksconf snapshot --output=daily.json /opt/splunk/etc/app/
-
class
ksconf.commands.snapshot.
ConfSnapshot
(config)¶ Bases:
object
-
schema_version
= 0.2¶
-
snapshot_dir
(path)¶
-
snapshot_file_conf
(path)¶
-
write_snapshot
(stream, **kwargs)¶
-
-
class
ksconf.commands.snapshot.
SnapshotCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= 'Build a static snapshot of various configuration files stored within a structured json export\nformat. If the .conf files being captured are within a standard Splunk directory structure,\nthen certain metadata and namespace information is assumed based on typical path locations.\nIndividual apps or conf files can be collected as well, but less metadata may be extracted.\n'¶
-
help
= 'Snapshot .conf file directories into a JSON dump format'¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Snapshot multiple configuration files into a single json snapshot.
-
ksconf.commands.sort module¶
SUBCOMMAND: ksconf sort <CONF>
Usage example: To recursively sort all files (in-place):
find . -name '*.conf' | xargs ksconf sort -i
-
class
ksconf.commands.sort.
SortCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= 'Sort a Splunk .conf file. Sort has two modes: (1) by default, the sorted\nconfig file will be echoed to the screen. (2) the config files are updated\nin-place when the ``-i`` option is used.\n\nManually managed conf files can be protected against changes by adding a comment containing the\nstring ``KSCONF-NO-SORT`` to the top of any .conf file.\n'¶
-
format
= 'manual'¶
-
help
= 'Sort a Splunk .conf file creating a normalized format appropriate for version control'¶
-
maturity
= 'stable'¶
-
pre_run
(args)¶ Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Sort one or more configuration file.
-
ksconf.commands.unarchive module¶
SUBCOMMAND: ksconf unarchive <tarball>
Usage example:
ksconf unarchive splunk-add-on-for-amazon-web-services_111.tgz
-
class
ksconf.commands.unarchive.
UnarchiveCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= "\nInstall or overwrite an existing app in a git-friendly way.\nIf the app already exists, steps will be taken to upgrade it safely.\n\nThe ``default`` folder can be redirected to another path (i.e., ``default.d/10-upstream`` or\nother desirable path if you're using the ``ksconf combine`` tool to manage extra layers).\n"¶
-
format
= 'manual'¶
-
help
= 'Install or upgrade an existing app in a git-friendly and safe way'¶
-
maturity
= 'beta'¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Install / upgrade a Splunk app from an archive file
-
ksconf.commands.xmlformat module¶
SUBCOMMAND: ksconf xml-format <XML>
Usage example: (Nice pre-commit script)
find default/data/ui -name '*.xml' | ksconf xml-format -
-
class
ksconf.commands.xmlformat.
XmlFormatCmd
(name)¶ Bases:
ksconf.commands.KsconfCmd
-
description
= "\nNormalize and apply consistent XML indentation and CDATA usage for XML dashboards and\nnavigation files.\n\nTechnically this could be used on *any* XML file, but certain element names specific to Splunk's\nsimple XML dashboards are handled specially, and therefore could result in unusable results.\n\nThe expected indentation level is guessed based on the first element indentation, but can be\nexplicitly set if not detectable.\n"¶
-
help
= 'Normalize XML view and nav files'¶
-
maturity
= 'alpha'¶
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Actual works happens here. Return code should be an EXIT_CODE_* from consts.
-
Module contents¶
-
class
ksconf.commands.
KsconfCmd
(name)¶ Bases:
object
Ksconf command specification base class.
-
add_parser
(subparser)¶
-
description
= None¶
-
format
= 'default'¶
-
help
= None¶
-
launch
(args)¶ Handle flow control between pre_run() / run() / post_run()
-
maturity
= 'alpha'¶
-
parse_conf
(path, mode='r', profile=None, raw_exec=False)¶
-
post_run
(args, exec_info=None)¶ Optional custom clean up method. Always called if run() was. The presence of exc_info indicates failure.
-
pre_run
(args)¶ Optional pre-run hook. Any exceptions or non-0 return code, will prevent run()/post_run() from being called.
-
register_args
(parser)¶ This function in passed the
-
run
(args)¶ Actual works happens here. Return code should be an EXIT_CODE_* from consts.
-
version_extra
= None¶
-
-
class
ksconf.commands.
ConfDirProxy
(name, mode, parse_profile=None)¶ Bases:
object
-
get_file
(relpath)¶
-
-
class
ksconf.commands.
ConfFileProxy
(name, mode, stream=None, parse_profile=None, is_file=None)¶ Bases:
object
-
close
()¶
-
data
¶
-
dump
(data, **kwargs)¶
-
exists
()¶
-
is_file
()¶
-
load
(profile=None)¶
-
mtime
¶
-
readable
()¶
-
reset
()¶
-
set_parser_option
(**kwargs)¶ Setting a key to None will remove that setting.
-
stream
¶
-
unlink
()¶
-
writable
()¶
-
-
class
ksconf.commands.
ConfFileType
(mode='r', action='open', parse_profile=None, accept_dir=False)¶ Bases:
object
Factory for creating conf file object types; returns a lazy-loader ConfFile proxy class
Started from argparse.FileType() and then changed everything. With our use case, it’s often necessary to delay writing, or read before writing to a conf file (depending on whether or not –dry-run mode is enabled, for example.)
Instances of FileType are typically passed as type= arguments to the ArgumentParser add_argument() method.
Parameters: - mode (str) – How the file is to be opened. Accepts “r”, “w”, and “r+”.
- action (str) – Determine how much work should be handled during argument parsing vs handed off to the caller. Supports ‘none’, ‘open’, ‘load’. Full descriptions below.
- parse_profile – parsing configuration settings passed along to the parser
- accept_dir (bool) – Should the CLI accept a directory of config files instead of an individual file. Defaults to False.
Values for action
Action Description none
No preparation or testing is done on the filename. open
Ensure the file exists and can be opened. load
Ensure the file can be opened and parsed successfully. Once invoked, instances of this class will return a
ConfFileProxy
object, or aConfDirProxy
object if a directory is passed in via the CLI.
-
ksconf.commands.
dedent
(text)¶ Remove any common leading whitespace from every line in text.
This can be used to make triple-quoted strings line up with the left edge of the display, while still presenting them in the source code in indented form.
Note that tabs and spaces are both treated as whitespace, but they are not equal: the lines ” hello” and “thello” are considered to have no common leading whitespace.
Entirely blank lines are normalized to a newline character.
-
ksconf.commands.
get_all_ksconf_cmds
(on_error='warn')¶
-
ksconf.commands.
get_entrypoints
¶
-
ksconf.commands.
add_splunkd_access_args
(parser)¶
-
ksconf.commands.
add_splunkd_namespace
(parser)¶