ksconf.commands package

Submodules

ksconf.commands.check module

SUBCOMMAND: ksconf check <CONF>

Usage example: (Nice pre-commit script)

find . -name ‘*.conf’ | ksconf check -
class ksconf.commands.check.CheckCmd(name)

Bases: ksconf.commands.KsconfCmd

description = u"\nProvide basic syntax and sanity checking for Splunk's .conf\nfiles. Use Splunk's builtin 'btool check' for a more robust\nvalidation of keys and values.\n\nConsider using this utility as part of a pre-commit hook."
help = u'Perform basic syntax and sanity checks on .conf files'
maturity = u'stable'
register_args(parser)
run(args)

ksconf.commands.combine module

SUBCOMMAND: combine –target=<DIR> <SRC1> [ <SRC-n> ]

Usage example:

cd MY_APP ksconf combine default.d/* –target=default
class ksconf.commands.combine.CombineCmd(name)

Bases: ksconf.commands.KsconfCmd

description = u"Merge .conf settings from multiple source directories into a combined target\ndirectory. Configuration files can be stored in a '/etc/*.d' like directory\nstructure and consolidated back into a single 'default' directory.\n\nThis command supports both one-time operations and recurring merge jobs. For\nexample, this command can be used to combine all users knowledge objects (stored\nin 'etc/users') after a server migration, or to merge a single user's settings\nafter an their account has been renamed. Recurring operations assume some type\nof external scheduler is being used. A best-effort is made to only write to\ntarget files as needed.\n\nThe 'combine' command takes your logical layers of configs (upstream, corporate,\nsplunk admin fixes, and power user knowledge objects, ...) expressed as\nindividual folders and merges them all back into the single 'default' folder\nthat Splunk reads from. One way to keep the 'default' folder up-to-date is\nusing client-side git hooks.\n\nNo directory layout is mandatory, but but one simple approach is to model your\nlayers using a prioritized 'default.d' directory structure. (This idea is\nborrowed from the Unix System V concept where many services natively read their\nconfig files from '/etc/*.d' directories.)\n\n\nTHE PROBLEM:\n\nIn a typical enterprise deployment of Splunk, a single app can easily have\nmultiple logical sources of configuration: (1) The upstream app developer, (2)\nlocal developer app-developer adds organization-specific customizations or\nfixes, (3) splunk admin tweaks the inappropriate 'indexes.conf' settings, and\n(4) custom knowledge objects added by your subject matter experts. Ideally we'd\nlike to version control these, but doing so is complicated because normally you\nhave to manage all 4 of these logical layers in one 'default' folder. (Splunk\nrequires that app settings be located either in 'default' or 'local'; and\nmanaging local files with version control leads to merge conflicts; so\neffectively, all version controlled settings need to be in 'default', or risk\nmerge conflicts.) So when a new upstream version is released, someone has to\nmanually upgrade the app being careful to preserve all custom configurations.\nThe solution provided by the 'combine' functionality is that all of these\nlogical sources can be stored separately in their own physical directories\nallowing changes to be managed independently. (This also allows for different\nlayers to be mixed-and-matched by selectively including which layers to\ncombine.) While this doesn't completely remove the need for a human to review\napp upgrades, it does lower the overhead enough that updates can be pulled in\nmore frequently, thus reducing the divergence potential. (Merge frequently.)\n\n\nNOTES:\n\nThe 'combine' command is similar to running the 'merge' subcommand recursively\nagainst a set of directories. One key difference is that this command will\ngracefully handle non-conf files intelligently too.\n\nEXAMPLE:\n\n Splunk_CiscoSecuritySuite/\n \u251c\u2500\u2500 README\n \u251c\u2500\u2500 default.d\n \u2502\xa0\xa0 \u251c\u2500\u2500 10-upstream\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 app.conf\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 data\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 ui\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 nav\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 default.xml\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 views\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 authentication_metrics.xml\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 cisco_security_overview.xml\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 getting_started.xml\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 search_ip_profile.xml\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 upgrading.xml\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 user_tracking.xml\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 eventtypes.conf\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 macros.conf\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 savedsearches.conf\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 transforms.conf\n \u2502\xa0\xa0 \u251c\u2500\u2500 20-my-org\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 savedsearches.conf\n \u2502\xa0\xa0 \u251c\u2500\u2500 50-splunk-admin\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 indexes.conf\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 macros.conf\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 transforms.conf\n \u2502\xa0\xa0 \u2514\u2500\u2500 70-firewall-admins\n \u2502\xa0\xa0 \u251c\u2500\u2500 data\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 ui\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 views\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 attacks_noc_bigscreen.xml\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u251c\u2500\u2500 device_health.xml\n \u2502\xa0\xa0 \u2502\xa0\xa0 \u2514\u2500\u2500 user_tracking.xml\n \u2502\xa0\xa0 \u2514\u2500\u2500 eventtypes.conf\n\nCommands:\n\n cd Splunk_CiscoSecuritySuite\n ksconf combine default.d/* --target=default\n"
format = u'manual'
help = u"Combine configuration files across multiple source directories into a single\ndestination directory. This allows for an arbitrary number of splunk\nconfiguration layers to coexist within a single app. Useful in both ongoing\nmerge and one-time ad-hoc use.\n\nFor example, combine can consolidate 'users' directory across several instances\nafter a phased server migration.\n"
maturity = u'beta'
register_args(parser)
run(args)

ksconf.commands.diff module

SUBCOMMAND: ksconf diff <CONF> <CONF>

Usage example:

ksconf diff default/props.conf default/props.conf
class ksconf.commands.diff.DiffCmd(name)

Bases: ksconf.commands.KsconfCmd

description = u"\nCompares the content differences of two .conf files\n\nThis command ignores textual differences (like order, spacing, and comments) and\nfocuses strictly on comparing stanzas, keys, and values. Note that spaces\nwithin any given value will be compared. Multiline fields are compared in are\ncompared in a more traditional 'diff' output so that long savedsearches and\nmacros can be compared more easily.\n"
format = u'manual'
help = u'Compare settings differences between two .conf files ignoring spacing and sort order'
maturity = u'stable'
register_args(parser)
run(args)

Compare two configuration files.

ksconf.commands.merge module

SUBCOMMAND: ksconf merge –target=<CONF> <CONF> [ <CONF-n> … ]

Usage example:

ksconf merge –target=master-props.conf /opt/splunk/etc/apps/TA/{default,local}/props.conf
class ksconf.commands.merge.MergeCmd(name)

Bases: ksconf.commands.KsconfCmd

description = u'Merge two or more .conf files into a single combined .conf file. This could be\nused to merge the props.conf file from ALL technology addons into a single file:\n\nksconf merge --target=all-ta-props.conf etc/apps/*TA*/{default,local}/props.conf\n\n'
format = u'manual'
help = u'Merge two or more .conf files'
maturity = u'stable'
register_args(parser)
run(args)

Merge multiple configuration files into one

ksconf.commands.minimize module

SUBCOMMAND: ksconf minimize –target=<CONF> <CONF> [ <CONF-n> … ]

Usage example:

ksconf minimize –target=local/inputs.conf default/inputs.conf
Example workflow:
  1. cp default/props.conf local/props.conf
  2. vi local/props.conf (edit JUST the lines you want to change)
  3. splconf minimize –target=local/props.conf default/props.conf (You could take this a step further by appending “$SPLUNK_HOME/system/default/props.conf” and removing any SHOULD_LINEMERGE = true entries (for example)
class ksconf.commands.minimize.MinimizeCmd(name)

Bases: ksconf.commands.KsconfCmd

description = u'Minimize a conf file by removing the default settings\n\nReduce local conf file to only your indented changes without manually tracking\nwhich entries you\'ve edited. Minimizing local conf files makes your local\ncustomizations easier to read and often results in cleaner add-on upgrades.\n\nA typical scenario & why does this matter:\n\nTo customizing a Splunk app or add-on, start by copying the conf file from\ndefault to local and then applying your changes to the local file. That\'s good.\nBut stopping here may complicated future upgrades, because the local file\ndoesn\'t contain *just* your settings, it contains all the default settings too.\nFixes published by the app creator may be masked by your local settings. A\nbetter approach is to reduce the local conf file leaving only the stanzas and\nsettings that you indented to change. This make your conf files easier to read\nand makes upgrades easier, but it\'s tedious to do by hand.\n\nFor special cases, the \'--explode-default\' mode reduces duplication between\nentries normal stanzas and global/default entries. If \'disabled = 0\' is a\nglobal default, it\'s technically safe to remove that setting from individual\nstanzas. But sometimes it\'s preferable to be explicit, and this behavior may be\ntoo heavy-handed for general use so it\'s off by default. Use this mode if your\nconf file that\'s been fully-expanded. (i.e., conf entries downloaded via REST,\nor the output of "btool list"). This isn\'t perfect, since many apps push their\nsettings into the global namespace, but it can help.\n\n\nExample usage:\n\n cd Splunk_TA_nix\n cp default/inputs.conf local/inputs.conf\n\n # Edit \'disabled\' and \'interval\' settings in-place\n vi local/inputs.conf\n\n # Remove all the extra (unmodified) bits\n ksconf minimize --target=local/inputs.conf default/inputs.conf\n'
format = u'manual'
help = u'Minimize the target file by removing entries duplicated in the default conf(s)'
maturity = u'beta'

Make sure this works before advertising (same file as target and source????) # Note: Use the ‘merge’ command to “undo” ksconf merge –target=local/inputs.conf default/inputs local/inputs.conf

register_args(parser)
run(args)
ksconf.commands.minimize.explode_default_stanza(conf, default_stanza=None)

Take the GLOBAL stanza, (aka [default]) and apply it’s settings underneath ALL other stanzas. This is mostly only useful in minimizing and other comparison operations.

ksconf.commands.promote module

SUBCOMMAND: ksconf promote –target=<CONF> <CONF>

Usage example: Promote local props changes (made via the UI) to the ‘default’ folder

ksconf –target=default/props.conf local/props.conf
class ksconf.commands.promote.PromoteCmd(name)

Bases: ksconf.commands.KsconfCmd

description = u"Propagate .conf settings applied in one file to another. Typically this is used\nto take local changes made via the UI and push them into a default (or\ndefault.d/) location.\n\nNOTICE: By default, changes are *MOVED*, not just copied.\n\nPromote has two different modes: batch and interactive. In batch mode all\nchanges are applied automatically and the (now empty) source file is removed.\nIn interactive mode the user is prompted to pick which stanzas and keys to\nintegrate. This can be used to push changes made via the UI, which are stored\nin a 'local' file, to the version-controlled 'default' file. Note that the\nnormal operation moves changes from the SOURCE file to the TARGET, updating both\nfiles in the process. But it's also possible to preserve the local file, if\ndesired.\n\nIf either the source file or target file is modified while a promotion is under\nprogress, changes will be aborted. And any custom selections you made will be\nlost. (This needs improvement.)\n"
format = u'manual'
help = u'Promote .conf settings from one file into another either in batch mode (all\nchanges) or interactively allowing the user to pick which stanzas and keys to\nintegrate.\n\nChanges made via the UI (stored in the local folder) can be promoted (moved) to\na version-controlled directory.\n'
maturity = u'beta'
register_args(parser)
run(args)

ksconf.commands.sort module

SUBCOMMAND: ksconf sort <CONF>

Usage example: To recursively sort all files (in-place):

find . -name ‘*.conf’ | xargs ksconf sort -i
class ksconf.commands.sort.SortCmd(name)

Bases: ksconf.commands.KsconfCmd

description = u"Sort a Splunk .conf file. Sort has two modes: (1) by default, the sorted\nconfig file will be echoed to the screen. (2) the config files are updated\ninplace when the '-i' option is used.\n\nManually managed conf files can be blacklisted by add a comment containing the\nstring 'KSCONF-NO-SORT' to the top of any .conf file.\n\nTo recursively sort all files:\n\n find . -name '*.conf' | xargs ksconf sort -i\n"
format = u'manual'
help = u'Sort a Splunk .conf file creating a normalized format appropriate for version control'
maturity = u'stable'
register_args(parser)
run(args)

Sort one or more configuration file.

ksconf.commands.unarchive module

SUBCOMMAND: ksconf upgrade tarball

Usage example:

ksconf upgrade tarball
class ksconf.commands.unarchive.UnarchiveCmd(name)

Bases: ksconf.commands.KsconfCmd

description = u"\nInstall or overwrite an existing app in a git-friendly way.\nIf the app already exist, steps will be taken to upgrade it safely.\n\nThe 'default' folder can be redirected to another path (i.e., 'default.d/10-upstream' or\nwhatever which is helpful if you're using the ksconf 'combine' mode.)\n\nSupports tarballs (.tar.gz, .spl), and less-common zip files (.zip)\n"
format = u'manual'
help = u'Install or upgrade an existing app in a git-friendly and safe way'
maturity = u'beta'
register_args(parser)
run(args)

Install / upgrade a Splunk app from an archive file

Module contents

class ksconf.commands.ConfDirProxy(name, mode, parse_profile=None)

Bases: object

get_file(relpath)
class ksconf.commands.ConfFileProxy(name, mode, stream=None, parse_profile=None, is_file=None)

Bases: object

backup(bkname=None)
checksum(hash=u'sha256')
close()
data
dump(data)
is_file()
load(profile=None)
reset()
set_parser_option(**kwargs)

Setting a key to None will remove that setting.

stream
class ksconf.commands.ConfFileType(mode=u'r', action=u'open', parse_profile=None, accept_dir=False)

Bases: object

Factory for creating conf file object types; returns a lazy-loader ConfFile proxy class

Started from argparse.FileType() and then changed everything. With our use case, it’s often necessary to delay writing, or read before writing to a conf file (depending on weather or not –dry-run mode is enabled, for example.)

Instances of FileType are typically passed as type= arguments to the ArgumentParser add_argument() method.

Keyword Arguments:
  • mode A string indicating how the file is to be opened. Accepts “r”, “w”, and “r+”.
  • action ‘none’, ‘open’, ‘load’. ‘none’ means no preparation or tests; ‘open’ means
    make sure the file exists/openable; ‘load’ means make sure the file can be opened and parsed successfully.
class ksconf.commands.KsconfCmd(name)

Bases: object

Ksconf command specification base class.

add_parser(subparser)
description = None
exit(exit_code)

Allow overriding for unittesting or other high-level functionality, like an interactive interface.

format = u'default'
help = None
launch(args)

Handle flow control between pre_run() / run() / post_run()

maturity = u'alpha'
post_run(args, exec_info=None)

Any custom clean up work that needs done. Allways called if run() was. Presence of exc_info indicates failure.

pre_run(args)

Pre-run hook. Any exceptions here prevent run() from being called.

redirect_io(stdin=None, stdout=None, stderr=None)
register_args(parser)

This function in passed the

run(args)

Actual works happens here. Return code should be an EXIT_CODE_* from consts.

class ksconf.commands.MyDescriptionHelpFormatter(prog, indent_increment=2, max_help_position=24, width=None)

Bases: argparse.HelpFormatter

ksconf.commands.get_entrypoints(*args, **kwargs)