ksconf.conf package

Submodules

ksconf.conf.delta module

class ksconf.conf.delta.DiffGlobal(type)

Bases: NamedTuple

type: DiffLevel

Alias for field number 0

class ksconf.conf.delta.DiffHeader(name, mtime=None)

Bases: object

detect_mtime()
mtime: float | None = None
name: str
class ksconf.conf.delta.DiffLevel(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

GLOBAL = 'global'
KEY = 'key'
STANZA = 'stanza'
class ksconf.conf.delta.DiffOp(tag, location, a, b)

Bases: NamedTuple

a: Dict[str, Dict[str, str]] | Dict[str, str] | str | None

Alias for field number 2

b: Dict[str, Dict[str, str]] | Dict[str, str] | str | None

Alias for field number 3

location: DiffGlobal | DiffStanza | DiffStzKey

Alias for field number 1

tag: DiffVerb

Alias for field number 0

class ksconf.conf.delta.DiffStanza(type, stanza)

Bases: NamedTuple

stanza: str

Alias for field number 1

type: DiffLevel

Alias for field number 0

class ksconf.conf.delta.DiffStzKey(type, stanza, key)

Bases: NamedTuple

key: str

Alias for field number 2

stanza: str

Alias for field number 1

type: DiffLevel

Alias for field number 0

class ksconf.conf.delta.DiffVerb(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

DELETE = 'delete'
EQUAL = 'equal'
INSERT = 'insert'
REPLACE = 'replace'
ksconf.conf.delta.compare_cfgs(a: Dict[str, Dict[str, str]], b: Dict[str, Dict[str, str]], replace_level: DiffLevel = DiffLevel.GLOBAL) → List[DiffOp]

Calculate a set of deltas which describes how to transform a into b.

Parameters:

  • a (dict) – the first/original configuration entity

  • b (dict) – the second/target configuration entity

  • replace_level (str: global, stanza, or key) –

    The highest level ‘replace’ event that can be returned. Acceptable values are global, stanza, and key. These examples may help:

    • Using ‘global’ with identical inputs will report a single global-level equal op.

    • Using ‘stanza’ with identical inputs will return all stanzas as equal.

    • Using ‘key’ will ensure that two stanzas with no common keys will be reported in terms of key changes. Whereas ‘global’ or ‘stanza’ would result in a single giant replace op.

Returns:

a sequence of differences in tuples

Return type:

[DiffOp]

Note

The DiffOp output idea was borrowed from SequenceMatcher class in the difflib in the standard Python module.

This function returns a sequence of 5 element tuples describing the transformation based on the detail level specified in replace_level.

Each DiffOp (named tuple) takes the form:

(tag, location, a, b)

tag:

Value

Meaning

‘replace’

same element in both, but different values.

‘delete’

remove value b

‘insert’

insert value a

‘equal’

same values in both

location is a namedtuple that can take the following forms:

Tuple form

Type

Description

(“global”)

DiffGlobal

Global file level context (e.g., both files are the same)

(“stanza”, stanza)

DiffStanza

Stanzas are the same, or completely different (no shared keys)

(“key”, stanza, key)

DiffStzKey

Key level change

Changed in version v0.8.8: The preserve_empty argument was originally introduced to preserve backwards compatibility, but it ended up introducing new bugs. Additionally, no use cases were found where better to automatically discarding empty stanzas.

Changed in version v0.8.8: The allow_level0 argument was replaced with replace_level. Instead of using allow_level0=False use replace_level="stanza". At the same time a new feature was added to support replace_level="key". The default behavior remains the same.

ksconf.conf.delta.compare_stanzas(a: Dict[str, str], b: Dict[str, str], stanza_name: str, replace_level: DiffLevel = DiffLevel.GLOBAL) → List[DiffOp]
Parameters:

replace_level (bool) – If a and b have no common keys, is a single stanza-level ‘replace’ is issue unless replace_level="key"

ksconf.conf.delta.diff_obj_json_format(o)
ksconf.conf.delta.is_equal(delta: List[DiffOp]) → bool

Is the delta output show that the compared objects are identical

ksconf.conf.delta.reduce_stanza(stanza: Dict[str, str], keep_attrs: Sequence[str]) → dict

Pre-process a stanzas so that only a common set of keys will be compared.

Parameters:

  • stanza (dict) – Stanzas containing attributes and values

  • keep_attrs ((list, set, tuple, dict)) – Listing of attributes to preserve

Returns:

a reduced copy of stanza.

ksconf.conf.delta.show_diff(stream: TextIO, diffs: List[DiffOp], headers=None) → int
ksconf.conf.delta.show_text_diff(stream: TextIO, a: PathLike, b: PathLike)
ksconf.conf.delta.summarize_cfg_diffs(delta: List[DiffOp], stream: TextIO)

Summarize a delta into a human-readable format. The input delta is in the format produced by the compare_cfgs() function.

ksconf.conf.delta.write_diff_as_json(delta: List[DiffOp], stream, **dump_args)

ksconf.conf.merge module

ksconf.conf.merge.merge_app_local(app_folder: str, cleanup: bool = True) → None

Find everything in local, if it has a corresponding file in default, merge. This function assumes standard Splunk app path names.

ksconf.conf.merge.merge_conf_dicts(*dicts: Dict[str, Dict[str, str]]) → Dict[str, Dict[str, str]]
ksconf.conf.merge.merge_conf_files(dest: ConfFileProxy, configs: List[ConfFileProxy], dry_run: bool = False, banner_comment: str | None = None) → SmartEnum
ksconf.conf.merge.merge_update_any_file(dest: str, sources: List[str], remove_source: bool = False) → None
ksconf.conf.merge.merge_update_conf_file(dest: str, sources: List[str], remove_source: bool = False) → None

Dest is treated as both the output, and the highest priority source.

ksconf.conf.meta module

Incomplete documentation available here:

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Defaultmetaconf

Specifically, attribute-level ACls aren’t discussed nor is the magic “import” directive.

LEVELS:

0 - global (or 1 stanza=”default”) 1 - conf 2 - stanzas 3 - attribute

class ksconf.conf.meta.MetaData

Bases: object

static expand_layers(layers: List[MetaLayer]) → dict
Parameters:

layers (list(dict)) – layer of stanzas, starting with the global ending with conf/stanza/attr

Returns:

Expanded layer

Return type:

dict

feed_conf(conf)
feed_file(stream)
get(*names)
get_layer(*names)
iter_raw()

RAW

classmethod parse_meta(stanza)

Split out the values of ‘access’ (maybe more someday) :param stanza: content of a meta stanza :return: extended meta data :rtype: dict

regex_access = '(?:^|\\s*,\\s*)(?P<action>read|write)\\s*:\\s*\\[\\s*(?P<roles>[^\\]]+?)\\s*\\]'
walk()
write_stream(stream: TextIO, sort=True)
class ksconf.conf.meta.MetaLayer(name)

Bases: object

property data
items(prefix=None)

Helpful when rebuilding the input file.

resolve(name)
update(*args, **kwargs)
walk(_prefix=())

ksconf.conf.parser module

Parse and write Splunk’s .conf files

According to this doc:

https://docs.splunk.com/Documentation/Splunk/7.2.3/Admin/Howtoeditaconfigurationfile

  1. Comments must start at the beginning of a line (#)

  2. Comments may not be after a stanza name or on an attribute’s value

  3. Supporting encoding is UTF-8 (and therefore ASCII too)

exception ksconf.conf.parser.ConfParserException

Bases: Exception

class ksconf.conf.parser.DuplicateEnum(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

EXCEPTION = 'exception'
MERGE = 'merge'
OVERWRITE = 'overwrite'
exception ksconf.conf.parser.DuplicateKeyException

Bases: ConfParserException

exception ksconf.conf.parser.DuplicateStanzaException

Bases: ConfParserException

class ksconf.conf.parser.Token

Bases: object

Immutable token object. deepcopy returns the same object

ksconf.conf.parser.conf_attr_boolean(value: str | bool | int) → bool
ksconf.conf.parser.cont_handler(iterable: TextIO | Iterable[str], continue_re: Pattern = re.compile('^(.*)\\\\$'), breaker: str = '\n') → Iterator[str]

Look for trailing backslashes (”\”) which indicate a value for an attribute is split across multiple lines. This function will group such lines together, and pass all other lines through as-is. Note that the continuation character must be the very last character on the line, trailing whitespace is not allowed.

Parameters:

  • iterable (iter) – lines from a configuration file

  • continue_re – regular expression to detect the continuation character

  • breaker – joining string when combining continued lines into a single string. Default ‘\n’

Returns:

lines of text

Return type:

str

ksconf.conf.parser.detect_by_bom(path: str | PathLike) → str
ksconf.conf.parser.inject_section_comments(section: Dict[str, str], prepend: Sequence[str] | None = None, append: Sequence[str] | None = None)

Extract existing comments from section dict (in order; and remove them) Add in any prepend/append comments (if that comment isn’t already present) Re-inject comments back into the section dict with fresh numbering

ksconf.conf.parser.parse_conf(stream: str | PathLike | TextIO | Iterable[str], profile: Dict = {'dup_key': DuplicateEnum.OVERWRITE, 'dup_stanza': DuplicateEnum.EXCEPTION, 'keep_comments': True, 'strict': True}, encoding: str | None = None) → Dict[str, Dict[str, str]]

Parse a .conf file. This is a wrapper around parse_conf_stream() that allows filenames or stream to be passed in.

Parameters:

  • stream (str, file) – the path to a configuration file or open file-like object to be parsed

  • profile – parsing configuration settings

  • encoding – Defaults to the system default, (Often “utf-8”)

Returns:

a mapping of the stanza and attributes. The resulting output is accessible as [stanza][attribute] -> value

Return type:

dict

ksconf.conf.parser.parse_conf_stream(stream: TextIO | Iterable[str], keys_lower: bool = False, handle_conts: bool = True, keep_comments: bool = False, dup_stanza: DuplicateEnum = DuplicateEnum.EXCEPTION, dup_key: DuplicateEnum = DuplicateEnum.OVERWRITE, strict: bool = False) → Dict[str, Dict[str, str]]

Low-level conf parsing functionality.

Most often, either parse_conf() or parse_conf_string() are better options.

ksconf.conf.parser.parse_conf_string(s: str, name: str | None = None, profile: Dict = {'dup_key': DuplicateEnum.OVERWRITE, 'dup_stanza': DuplicateEnum.EXCEPTION, 'keep_comments': True, 'strict': True}) → Dict[str, Dict[str, str]]

Parse a .conf file that’s already in memory, as a string.

ksconf.conf.parser.parse_string(*args, **kwargs)

Deprecated wrapper around parse_conf_string(). Use that instead.

ksconf.conf.parser.section_reader(stream: TextIO | Iterable[str], section_re: Pattern = re.compile('^[\\s\\t]*\\[(.*)\\]\\s*$')) → Iterator[Tuple[str | None, List[str]]]

This generator break a configuration file stream into sections. Each section contains a name and a list of text lines held within that section.

Sections that have no entries must be preserved. Any lines before the first section are send back with the section name of None.

Parameters:

  • stream (file) – configuration file input stream

  • section_re – regular expression for detecting stanza headers

Returns:

sections in the form of (section_name, lines_of_text)

Return type:

tuple

ksconf.conf.parser.smart_write_conf(filename: str | PathLike, conf: Dict[str, Dict[str, str]], stanza_delim: str = '\n', sort: bool = True, temp_suffix: str = '.tmp', mtime: float | None = None) → SmartEnum

Write conf data to a specific file, but only when necessary. This function is essentially the same as write_conf(), except that it avoids updating the file if it already exists and has the desired content.

ksconf.conf.parser.splitup_kvpairs(lines: Iterable[str], comments_re: Pattern = re.compile('^\\s*[#;]'), keep_comments: bool = False, strict: bool = False) → Iterator[Tuple[str, str]]

Break up ‘attribute=value’ entries in a configuration file.

Parameters:

  • lines (iter) – the body of a stanza containing associated attributes and values

  • comments_re – Regular expression used to detect comments.

  • keep_comments (bool, optional) – Should comments be preserved in the output. Defaults to False.

  • strict (bool, optional) – Should unknown content in the stanza stop processing. Defaults to False allowing “junk” to be silently ignored for a best-effort parse.

Returns:

iterable of (attribute,value) tuples

class ksconf.conf.parser.update_conf(conf_path: str | PathLike, profile: Dict = {'dup_key': DuplicateEnum.OVERWRITE, 'dup_stanza': DuplicateEnum.EXCEPTION, 'keep_comments': True, 'strict': True}, encoding: str | None = None, make_missing: bool = False)

Bases: object

Context manager that allows for simple in-place updates to conf files. This provides a simple dict-like interface for easy updates.

Usage example:

with update_conf("app.conf") as conf:
    conf["launcher"]["version"] = "1.0.2"
    conf["install"]["build"] = 33
Parameters:

  • conf_path (str) – Path to .conf file to be edited.

  • profile (dict) – Parsing settings and strictness profile.

  • encoding (str) – encoding to use for file operations.

  • make_missing (bool) – When true, a new blank configuration file will be created if conf_path is missing, otherwise an exception will be raised.

cancel()

Indicate that no updates were made and all processing is complete. An error will occur if additional read/writes are attempted.

keys() → List[str]
update(*args, **kwargs)
ksconf.conf.parser.write_conf(stream: str | PathLike | TextIO, conf: Dict[str, Dict[str, str]], stanza_delim: str = '\n', sort: bool = True, temp_suffix: str = '.tmp', mtime: float | None = None)
ksconf.conf.parser.write_conf_stream(stream: TextIO, conf: Dict[str, Dict[str, str]], stanza_delim: str = '\n', sort: bool = True)
ksconf.conf.parser.write_conf_string(conf: Dict[str, Dict[str, str]], stanza_delim: str = '\n', sort: bool = True) → str

Write conf data to a string.

Module contents