Note
Key concepts
Before diving into the combine
command, it may be helpful to brush up on the concept of
configuration layers.
ksconf combine¶
Merge .conf settings from multiple source directories into a combined target
directory. Configuration files can be stored in a /etc/*.d
like directory
structure and consolidated back into a single ‘default’ directory.
This command supports both one-time operations and recurring merge jobs. For example, this command can be used to combine all users’ knowledge objects (stored in ‘etc/users’) after a server migration, or to merge a single user’s settings after their account has been renamed. Recurring operations assume some type of external scheduler is being used. A best-effort is made to only write to target files as needed.
The ‘combine’ command takes your logical layers of configs (upstream, corporate,
Splunk admin fixes, and power user knowledge objects, …) expressed as
individual folders and merges them all back into the single default
folder
that Splunk reads from. One way to keep the ‘default’ folder up-to-date is
using client-side git hooks.
No directory layout is mandatory, but one simple approach is to model your
layers using a prioritized ‘default.d’ directory structure. This idea is
borrowed from the Unix System V concept where many services natively read their
config files from /etc/*.d
directories.
usage: ksconf combine [-h] [--target TARGET] [--dry-run] [--follow-symlink]
[--banner BANNER] [--disable-marker]
source [source ...]
Positional Arguments¶
source | The source directory where configuration files will be merged from.
When multiple source directories are provided, start with the most general and end
with the specific; later sources will override values from the earlier ones.
Supports wildcards so a typical Unix conf.d/##-NAME directory structure works well. |
Named Arguments¶
--target, -t | Directory where the merged files will be stored. Typically either ‘default’ or ‘local’ |
--dry-run, -D | Enable dry-run mode. Instead of writing to TARGET, preview changes as a ‘diff’. If TARGET doesn’t exist, then show the merged file. |
--follow-symlink, -l | |
Follow symbolic links pointing to directories. Symlinks to files are followed. | |
--banner, -b | A banner or warning comment added to the top of the TARGET file. Used to discourage Splunk admins from editing an auto-generated file. For other on-going combine operations, it’s helpful to inform any .conf file readers or potential editors that the file is automatically generated and therefore could be overwritten again.
For one-time combine operations, the default banner can be suppressed by passing in an empty string ( |
--disable-marker | |
Prevents the creation of or checking for the ‘.ksconf_controlled’ marker file safety check. This file is typically used indicate that the destination folder is managed by ksconf. This option should be reserved for well-controlled batch processing scenarios. |
You may have noticed similarities between the combine
and merge
subcommands. That’s because under the covers they are using much of the same code. The combine
operation essentially does a recursive merge between a set of directories. One big difference is
that combine
command will handle non-conf files intelligently, not just conf files.
Note
Mixing layers
Just like all layers can be managed independently, they can also be combined in any way you would like. While this workflow is outside of the scope of the examples provided here, it’s certainly feasible. This also allows for different layers to be mixed-and-matched by selectively including which layers to combine.
Examples¶
Merging a multilayer app¶
Let’s assume you have a directory structure that looks like the following. This example features the Cisco Security Suite.
Splunk_CiscoSecuritySuite/
├── README
├── default.d
│ ├── 10-upstream
│ │ ├── app.conf
│ │ ├── data
│ │ │ └── ui
│ │ │ ├── nav
│ │ │ │ └── default.xml
│ │ │ └── views
│ │ │ ├── authentication_metrics.xml
│ │ │ ├── cisco_security_overview.xml
│ │ │ ├── getting_started.xml
│ │ │ ├── search_ip_profile.xml
│ │ │ ├── upgrading.xml
│ │ │ └── user_tracking.xml
│ │ ├── eventtypes.conf
│ │ ├── macros.conf
│ │ ├── savedsearches.conf
│ │ └── transforms.conf
│ ├── 20-my-org
│ │ └── savedsearches.conf
│ ├── 50-splunk-admin
│ │ ├── indexes.conf
│ │ ├── macros.conf
│ │ └── transforms.conf
│ └── 70-firewall-admins
│ ├── data
│ │ └── ui
│ │ └── views
│ │ ├── attacks_noc_bigscreen.xml
│ │ ├── device_health.xml
│ │ └── user_tracking.xml
│ └── eventtypes.conf
In this structure, you can see several layers of configurations at play:
- The
10-upstream
layer appears to be the version of the default folder that shipped with the Cisco app.- The
20-my-org
layer is small and only contains tweaks to a few saved search entires.- The
50-splunk-admin
layer represents local settings changes to specify index configurations, and to augment the macros and transformations that ship with the default app.- And finally,
70-firewall-admins
contains some additional view (2 new, and 1 existing). Note that sinceuser_tracking.xml
is not a.conf
file it will fully replace the upstream default version (that is, the file in10-upstream
)
Here are the commands that could be used to generate a new (merged) default
folder from all
of the layers shown above.
cd Splunk_CiscoSecuritySuite
ksconf combine default.d/* --target=default
See also
The unarchive command can be used to install or upgrade apps stored in a version controlled system in a layer-aware manor.
Consolidating ‘users’ directories¶
The combine
command can consolidate ‘users’ directory across several instances after a phased server migration.
See Migrating the ‘users’ folder.