Note

Key concepts

Before diving into the combine command, it may be helpful to brush up on the concept of configuration layers.

ksconf combine¶

Merge .conf settings from multiple source directories into a combined target directory. Configuration files can be stored in a /etc/*.d like directory structure and consolidated back into a single ‘default’ directory.

This command supports both one-time operations and recurring merge jobs. For example, this command can be used to combine all users knowledge objects (stored in ‘etc/users’) after a server migration, or to merge a single user’s settings after an their account has been renamed. Recurring operations assume some type of external scheduler is being used. A best-effort is made to only write to target files as needed.

The ‘combine’ command takes your logical layers of configs (upstream, corporate, splunk admin fixes, and power user knowledge objects, …) expressed as individual folders and merges them all back into the single default folder that Splunk reads from. One way to keep the ‘default’ folder up-to-date is using client-side git hooks.

No directory layout is mandatory, but but one simple approach is to model your layers using a prioritized ‘default.d’ directory structure. (This idea is borrowed from the Unix System V concept where many services natively read their config files from /etc/*.d directories.)

usage: ksconf combine [-h] [--target TARGET] [--dry-run] [--banner BANNER]
                      source [source ...]

Positional Arguments¶

source The source directory where configuration files will be merged from. When multiple sources directories are provided, start with the most general and end with the specific; later sources will override values from the earlier ones. Supports wildcards so a typical Unix conf.d/##-NAME directory structure works well.

Named Arguments¶

`--target, -t`	Directory where the merged files will be stored. Typically either ‘default’ or ‘local’
`--dry-run, -D`	Enable dry-run mode. Instead of writing to TARGET, preview changes as a ‘diff’. If TARGET doesn’t exist, then show the merged file.
`--banner, -b`	A warning banner to discourage manual editing of conf files.

You may have noticed similarities between the combine and merge subcommands. That’s because under the covers they are using much of the same code. The combine operations essentially does a recursive merge between a set of directories. One big difference is that combine command will gracefully handle non-conf files intelligently, not just conf files.

Note

Mixing layers

Just like all layers can be managed independently, they can also be combined in any way you’d like. While this workflow is out side the scope of the examples provided here, it’s very doable. This also allows for different layers to be mixed-and-matched by selectively including which layers to combine.

Examples¶

Merging a multilayer app¶

Let’s assume you have a directory structure that looks like the following. This example features the Cisco Security Suite.

Splunk_CiscoSecuritySuite/
├── README
├── default.d
│   ├── 10-upstream
│   │   ├── app.conf
│   │   ├── data
│   │   │   └── ui
│   │   │       ├── nav
│   │   │       │   └── default.xml
│   │   │       └── views
│   │   │           ├── authentication_metrics.xml
│   │   │           ├── cisco_security_overview.xml
│   │   │           ├── getting_started.xml
│   │   │           ├── search_ip_profile.xml
│   │   │           ├── upgrading.xml
│   │   │           └── user_tracking.xml
│   │   ├── eventtypes.conf
│   │   ├── macros.conf
│   │   ├── savedsearches.conf
│   │   └── transforms.conf
│   ├── 20-my-org
│   │   └── savedsearches.conf
│   ├── 50-splunk-admin
│   │   ├── indexes.conf
│   │   ├── macros.conf
│   │   └── transforms.conf
│   └── 70-firewall-admins
│       ├── data
│       │   └── ui
│       │       └── views
│       │           ├── attacks_noc_bigscreen.xml
│       │           ├── device_health.xml
│       │           └── user_tracking.xml
│       └── eventtypes.conf

In this structure, you can see several layers of configurations at play:

The 10-upstream layer appears to be the version of the default folder that shipped with the Cisco app.

The 20-my-org layer is small and only contains tweaks to a few savedsearch entires.

The 50-splunk-admin layer represents local settings changes to specify index configurations, and to augment the macros and transformations that ship with the default app.

And finally, 70-firewall-admins contains some additional view (2 new, and 1 existing). Note that since user_tracking.xml is not a .conf file it will fully replace the upstream default version (that is, the file in 10-upstream)

Here’s are the commands that could be used to generate a new (merged) default folder from all these layers shown above.

cd Splunk_CiscoSecuritySuite
ksconf combine default.d/* --target=default

Consolidating ‘users’ directories¶

combine can consolidate ‘users’ directory across several instances after a phased server migration.