ksconf.builder package


ksconf.builder.cache module

class ksconf.builder.cache.CachedRun(root: Path)

Bases: object

STATE_DISABLED = 'disabled'
STATE_EXISTS = 'exists'
STATE_NEW = 'new'
STATE_TAINT = 'taint'
property cached_inputs
property cached_outputs
property exists
inputs_identical(inputs: FileSet) bool
property is_disabled
property is_expired
property is_new
set_cache_info(type: str, data: FileSet)
class ksconf.builder.cache.FileSet(*, fingerprint=<function fingerprint_hash>)

Bases: object

A collection of fingerprinted files.

Currently the fingerprint is only a SHA256 hash.

Two constructors are provided for building an instance from either files that live on the filesystem, via from_filesystem() or from a persisted cached record available from the from_cache(). The filesystem version actively reads all inputs files at object creation time, so this can be costly, especially if repeated.

add_file(root: Path, relative_path: str)

Add a simple relative path to a file to the FileSet.

add_glob(root: Path, pattern: str)

Recursively add all files matching glob pattern.

copy_all(src_dir: Path, dest_dir: Path)

Copy a the given set of files from one location to another.

classmethod from_cache(data: dict, **kw_init_args) FileSet
classmethod from_filesystem(root: Path, files: List[str] | None = None, **kw_init_args) FileSet

Create a new FileSet instance based on a filesystem location. If files is None, then the entire directory is added recursively.

get_fingerprint: Callable[[Path], dict]
to_cache() dict

Return a json-friendly dict

ksconf.builder.cache.fingerprint_hash(path: Path) dict

Build a fingerprint based a file’s content hash.

ksconf.builder.cache.fingerprint_stat(path: Path) dict

Build fingerprint based on a file’s stats.

ksconf.builder.core module

Cache build requirements:

  • Caching mechanism should inspect ‘inputs’ (collect file hashes) to determine if any content has changed. If input varies, then command should be re-run.

  • Command (decorated function) should be generally unaware of all other details of build process, and it should ONLY be able to see files listed in “inputs”

  • Allow caching to be fully disabled (run in-place with no dir proxy) for CI/CD

  • Cache should have allow a timeout parameter

decorator used to implement caching:
  • decorator args:
    • inputs: list or glob

    • outputs (do we need this, can we just detect this??)

      Default to “.” (everything)

    • timeout=0 Seconds before cache should be considered stale

    • name=None If not given, default to the short name of the function.

      (Cache “slot”), must be filesystem safe]

class ksconf.builder.core.BuildManager

Bases: object

Supports an application building process by managing individual build steps

New in version v0.8.0.

cache(inputs: List[str], outputs: int, timeout: int | None = None, name: str | None = None, cache_invalidation: dict | list | str | None = None) None

function decorator for caching build steps Wrapped function must accept BuildStep instance as first parameters

XXX: Clearly document what things are good cache candidates and which are not.


  • No extra argument to the function (at least currently)

  • Changes to inputs files are not supported

  • Deleting files aren’t supported

  • Can only operate in a single directory given a limited set of inputs

  • Cannot read from the source directory, and agrees not to write to dist (In other words, limit all activities to build_path for deterministic behavior)

get_build_step(output=None) BuildStep
get_cache_info(name: str) CachedRun
set_folders(source_path: Path, build_path: Path, dist_path: Path | None = None)

ksconf.builder.steps module

ksconf.builder.steps: Collection of reusable build steps for reuse in your build script.

ksconf.builder.steps.clean_build(step: BuildStep) None

Ensure a clean build folder for consistent build results.

ksconf.builder.steps.copy_files(step: BuildStep, patterns: List[str], target: str | None = None) None

Copy source files into the build folder that match given glob patterns

ksconf.builder.steps.pip_install(step: BuildStep, requirements_file: str = 'requirements.txt', dest: str = 'lib', *, python_path: str | None = None, isolated: bool = True, dependencies: bool = True, handle_dist_info: str = 'remove', remove_console_scripts: bool = True) None

Module contents

exception ksconf.builder.BuildCacheException

Bases: Exception

exception ksconf.builder.BuildExternalException

Bases: Exception

class ksconf.builder.BuildStep(build: ~pathlib.Path, source: ~pathlib.Path | None = None, dist: ~pathlib.Path | None = None, output: ~typing.TextIO = <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Bases: object

alternate_path(path) BuildStep

Construct a new BuildStep instance with only ‘build_path’ altered.

get_logger(prefix: str | None = None) Callable
property is_quiet
run(executable, *args, cwd=None)

Execute an OS-level command regarding the build process. The process will run withing the working directory of the build folder.

  • executable (str) – Executable to launch for a build step.

  • args (str) – Additional argument(s) for the new process.

  • cwd (str) – Optional kw arg to change the working directory. This defaults to the build folder.

run_ksconf(*args, cwd=None)

Execute ‘ksconf’ command in the build folder. Currently this runs as a separate process, but in the future is may be optimized to run from within the same python process. This is an implementation detail the caller shouldn’t care about.

  • args (str) – Additional argument(s) for the ksconf command.

  • cwd (str) – Optional kw arg to change the working directory. This defaults to the build folder.

ksconf.builder.default_cli(build_manager: BuildManager, build_funct: Callable, argparse_parents: List[ArgumentParser] = ())

This is the function you stick in the: if __name__ == '__main__' section of your code :-)

Pass in a BuildManager instance, and a callback function. The callback function must accept (steps, args). If you have need for custom arguments, you can add them to your own ArgumentParser instance and pass them to the argparse_parents keyword argument, and then handle additional ‘args’ passed into the callback function.