bigconfig — workspace manual

Overview #

This directory is a multi-project workspace, not a single application. It groups three layered libraries that build on one another, with parallel implementations in three languages. Concepts are shared across implementations; syntax and idioms are not. This page is the union of the three project manuals, organized by project and tabbed by language where the code diverges.

Read each project's section from top to bottom for an end-to-end manual, or jump directly to a topic via the sidebar. Code examples inside each project section are presented in Clojure / Python / TypeScript tabs.

Workspace layout #

bigconfig/
├── Selmer/                  # Template engine
│   ├── master/              # Clojure (canonical, .git)
│   ├── python/
│   └── typescript/
├── big-config/              # Workflow + render engine; depends on Selmer
│   ├── main/                # Clojure (canonical, .git, full feature set)
│   ├── clojure/             # JVM-only Clojure subset
│   ├── python/
│   └── typescript/
└── once/                    # Cloud provisioning CLI; depends on big-config
    ├── main/                # Clojure (canonical, .git)
    ├── clojure/             # Clojure variant (`bb once package …` CLI shape)
    ├── python/
    └── typescript/

Each language directory is a self-contained build target with its own deps.edn, pyproject.toml, or package.json. Commands should be run from inside the relevant subproject directory.

Dependency graph #

Same-language siblings depend down the stack:

Git roots & ports #

Only three directories are real Git repositories: Selmer/master, big-config/main, and once/main. Every other language directory (python/, typescript/, clojure/) is a snapshot/port maintained alongside the canonical Clojure source. Commit inside the relevant canonical directory; do not initialize git at the workspace root.

Two Clojure directories exist for both big-config and once:

Overview #

Selmer is a fast, Django-inspired template system. The original is a pure-Clojure library by Dmitri Sotnikov (Yogthos). This workspace also contains dependency-free ports for Python 3.12+ and TypeScript/Node.js. All three implementations keep the same template language: variables ({{name}}), filters ({{name|upper}}), tags ({% if … %}), template inheritance, includes, validation, caching, missing-value handling, and custom tags/filters.

Selmer is the rendering engine inside big-config (via big-config.render), which is itself the rendering layer used by once to materialize Terraform/Tofu and Ansible files from per-profile parameters.

Install & usage #

;; deps.edn
{selmer/selmer {:mvn/version "1.13.1"}}

(require '[selmer.parser :refer [render render-file]])

(render "Hello {{name}}!" {:name "Yogthos"})
;;=> "Hello Yogthos!"

(render-file "home.html" {:name "Yogthos"})

(let [a 1, b "hello"]
  (<< "{{b|upper}}, {{a}} + {{a}} = 2"))
;;=> "HELLO, 1 + 1 = 2"

uv add selmer
# or for local dev:
uv sync

from selmer import render, render_file

render("Hello {{name}}!", {"name": "Yogthos"})
# "Hello Yogthos!"

render_file("templates/home.html", {"name": "Yogthos"})

npm install selmer

import { render, renderFile } from "selmer";

render("Hello {{name}}!", { name: "Yogthos" });
// => "Hello Yogthos!"

renderFile("templates/home.html", { name: "Yogthos" });

Templates & resource path #

A template is plain text containing {{variable}} substitutions and {% tag %} directives. Templates are parsed once and cached; a recompile is triggered when the file's mtime changes. Note that changes to included or extended templates do not automatically invalidate the cache of the outer template — touch the outer file to force a recompile.

(require '[selmer.parser :as p])

;; Templates resolved relative to classpath by default
(p/set-resource-path! "/var/html/templates/")
(p/set-resource-path! (clojure.java.io/resource "META-INF/foo/templates"))
(p/set-resource-path! nil)   ; reset

;; Cache control
(p/cache-on!)
(p/cache-off!)

from selmer import set_resource_path, cache_on, cache_off

set_resource_path("/var/html/templates/")
set_resource_path(None)

cache_on()
cache_off()

import { setResourcePath, cacheOn, cacheOff, clearCache } from "selmer";

setResourcePath("/var/html/templates/");
setResourcePath(null);

cacheOn();
cacheOff();
clearCache();

Variables #

Dot-paths walk nested maps and vectors. Numeric segments index into sequences. A doubled dot escapes a literal dot in a key name (useful for namespaced keys).

(render "{{person.name}}"      {:person {:name "John"}})
(render "{{foo.bar.0.baz}}"    {:foo {:bar [{:baz "hi"}]}})
(render "{{foo..bar/baz}}"     {:foo.bar/baz "hello"})

render("{{person.name}}",   {"person": {"name": "John"}})
render("{{foo.bar.0.baz}}", {"foo": {"bar": [{"baz": "hi"}]}})
render("{{foo..bar/baz}}",  {"foo.bar/baz": "hello"})

render("{{person.name}}",   { person: { name: "John" } });
render("{{foo.bar.0.baz}}", { foo: { bar: [{ baz: "hi" }] } });
render("{{foo..bar/baz}}",  { "foo.bar/baz": "hello" });

Filters #

Filters transform a value with a | pipe. Arguments are colon-separated.

{{ name|upper }}
{{ items|count }}
{{ value|default:"fallback" }}
{{ html|safe }}
{{ amount|currency-format:"USD" }}

Built-in filters

The Clojure original exposes the largest set; the Python and TypeScript ports cover the same core. Available across all three implementations (subject to per-port omissions):

abbreviate, add, addslashes, between?, capitalize, center, count, count-is, currency-format, date, default, default-if-empty, divide, double-format, drop, drop-last, email, empty?, first, get, get-digit, hash, join, json, last, length, length-is, linebreaks, linebreaks-br, linenumbers, lower, multiply, name, not-empty, phone, pluralize, rand-nth, range, remove, remove-tags, replace, round, safe, sort, sort-by, sort-by-reversed, sort-reversed, str, subs, sum, take, title, upper, urlescape.

The TypeScript port additionally exposes abbr-left, abbr-middle, abbr-right, abbr-ellipsis, and number-format. The block.super reference is available inside {% block %} bodies.

Custom filters

(require '[selmer.filters :refer [add-filter! remove-filter!]])

(add-filter! :embiginate #(.toUpperCase (str %)))
(render "{{shout|embiginate}}" {:shout "hello"})
;;=> "HELLO"

(remove-filter! :embiginate)

from selmer import add_filter, remove_filter, render

add_filter("embiginate", lambda value: str(value).upper())
render("{{shout|embiginate}}", {"shout": "hello"})
# "HELLO"

remove_filter("embiginate")

import { addFilter, removeFilter, render } from "selmer";

addFilter("embiginate", (value) => String(value).toUpperCase());
render("{{shout|embiginate}}", { shout: "hello" });
// => "HELLO"

removeFilter("embiginate");

Tags #

Tags are block- or inline-level directives wrapped in {% %}.

Built-in tags: block, comment, cycle, debug, if (with elif/else), ifequal, ifunequal, include, extends, firstof, for (with empty), now, safe, script, style, verbatim, with.

{% if user %}Hello {{user}}{% else %}Hello guest{% endif %}

{% for x in xs %}{{x}}{% empty %}none{% endfor %}

Custom tags

(require '[selmer.parser :refer [add-tag!]])

(add-tag! :join (fn [args context] (clojure.string/join "," args)))
(render "{% join a b c %}" {})
;;=> "a,b,c"

;; Block tag: takes a closing tag keyword
(add-tag! :uppercase
  (fn [_args context content]
    (clojure.string/upper-case (get-in content [:uppercase :content])))
  :enduppercase)

(render "{% uppercase %}hello {{name}}{% enduppercase %}" {:name "selmer"})
;;=> "HELLO SELMER"

from selmer import add_tag, remove_tag, render

add_tag("join", lambda args, context: ",".join(args))
render("{% join a b c %}", {})
# "a,b,c"

# Block tag — receives rendered block content
add_tag(
    "uppercase",
    lambda args, context, content: content["uppercase"]["content"].upper(),
    "enduppercase",
)
render("{% uppercase %}hello {{name}}{% enduppercase %}", {"name": "selmer"})
# "HELLO SELMER"

import { addTag, removeTag, render } from "selmer";

addTag("join", (args) => args.join(","));
render("{% join a b c %}", {});
// => "a,b,c"

addTag(
  "uppercase",
  (_args, _context, content) => content?.uppercase?.content.toUpperCase(),
  "enduppercase"
);

render("{% uppercase %}hello {{name}}{% enduppercase %}", { name: "selmer" });
// => "HELLO SELMER"

Inheritance & includes #

A child template declares {% extends "base.html" %} and overrides the blocks defined in the parent. {% include %} inlines another template; with supplies defaults.

<!-- base.html -->
<html>
<body>
{% block content %}{% endblock %}
</body>
</html>

<!-- child.html -->
{% extends "base.html" %}
{% block content %}Hello {{name}}{% endblock %}

{% include "partials/header.html" %}
{% include "card.html" with title="Untitled" %}

Inside a child {% block %}, {{ block.super }} renders the parent's block content for composition.

Escaping #

Variables are HTML-escaped by default. The safe filter and tag opt out per value; global toggles exist for opting out wholesale.

(require '[selmer.util :refer [turn-on-escaping! turn-off-escaping! without-escaping]])

(render "{{x}}" {:x "<tag>"})       ;; => "&lt;tag&gt;"
(render "{{x|safe}}" {:x "<tag>"})  ;; => "<tag>"

(without-escaping
  (render "{{x}}" {:x "<tag>"}))    ;; => "<tag>"

from selmer import turn_off_escaping, turn_on_escaping, without_escaping

render("{{x}}", {"x": "<tag>"})        # "&lt;tag&gt;"
render("{{x|safe}}", {"x": "<tag>"})   # "<tag>"

without_escaping(lambda: render("{{x}}", {"x": "<tag>"}))

import { turnOffEscaping, turnOnEscaping, withoutEscaping } from "selmer";

render("{{x}}", { x: "<tag>" });        // "&lt;tag&gt;"
render("{{x|safe}}", { x: "<tag>" });   // "<tag>"

withoutEscaping(() => render("{{x}}", { x: "<tag>" }));

Missing values #

By default, a missing variable renders as the empty string. A custom missing-value formatter can render an inline marker — useful for debugging incomplete contexts.

(require '[selmer.util :refer [set-missing-value-formatter!]])

(set-missing-value-formatter!
  (fn [tag context-map]
    (str "<missing " (or (:tag-value tag) (:tag-name tag)) ">"))
  :filter-missing-values false)

from selmer import set_missing_value_formatter

set_missing_value_formatter(
    lambda tag, context: f"<missing {tag.get('tag_value') or tag.get('tag_name')}>",
    filter_missing_values=False,
)

import { setMissingValueFormatter } from "selmer";

setMissingValueFormatter(
  (tag) => `<missing ${tag.tagValue ?? tag.tagName}>`,
  { filterMissingValues: false }
);

Validation #

By default, parsing reports unclosed tags or malformed syntax. Validation can be turned off if your templates use a foreign DSL that resembles Selmer syntax.

(require '[selmer.validator :refer [validate-on! validate-off!]])
(validate-off!)
(validate-on!)

from selmer import validate_on, validate_off
validate_off()
validate_on()

import { validateOn, validateOff } from "selmer";
validateOff();
validateOn();

Introspection #

Selmer can list the declared variable references in a template. Useful for generating a context schema or validating a render call upstream.

(require '[selmer.parser :refer [known-variables known-variable-paths]])

(known-variables "{{person.name}}")
;;=> #{:person}

(known-variable-paths "{{person.name}}")
;;=> [[:person :name]]

from selmer import known_variables, known_variable_paths

known_variables("{{person.name}}")
# {"person"}

known_variable_paths("{{person.name}}")
# [["person", "name"]]

import { knownVariables, knownVariablePaths } from "selmer";

knownVariables("{{person.name}}");
// Set { "person" }

knownVariablePaths("{{person.name}}");
// [["person", "name"]]

Per-language API surface #

License: EPL — same as the original Selmer project. Copyright © 2015 Dmitri Sotnikov and contributors.

Overview #

big-config is a workflow and template engine for infrastructure-as-code. It bridges general-purpose programming and specialized CLI tools (Tofu, Ansible, Kubectl, …) by threading a single opts map through a sequence of pluggable steps. Selmer renders configuration; shell steps execute it; Git tags provide pessimistic locking; a Redis-backed store and a system-lifecycle workflow round out the Clojure feature set.

Install & commands #

# Install BigConfig as a Clojure tool
clojure -Ttools install-latest :lib io.github.bigconfig-ai/big-config :as big-config

# Print help for all available templates
clojure -A:deps -Tbig-config help/doc

# Tests
just test
clojure -M:test
clj -X:test :dirs '["test/clj"]'
bb test-bb-task

# REPL
clojure -M:dev

# From inside big-config/python
uv sync
uv run pytest -q
uv run big-config --help

# Frozen install
uv sync --frozen

npm install
npm run check        # tsc --noEmit
npm test             # vitest run
npm run build        # tsc -> dist/

Modules #

All three implementations share the same module-level decomposition. The Clojure variants additionally provide store, system, tools, and build; the JVM-only big-config/clojure subset drops these.

The opts map #

Every workflow function receives and returns an opts map (Clojure map, Python dict, TypeScript Record<string, any>). The engine threads it through each step. Reserved top-level keys live in the big-config namespace (Clojure aliases as bc; Python and TypeScript use string keys verbatim).

Minimal step

(ns my.app
  (:require [big-config.core :as core]))

(defn my-step [opts]
  ;; do work…
  (core/ok opts))            ; sets :big-config/exit 0

(defn failing-step [opts]
  (merge opts {:big-config/exit 1
               :big-config/err  "Something went wrong"}))

from big_config import core

def my_step(opts):
    # do work…
    return core.ok(opts)              # sets "big-config/exit" -> 0

def failing_step(opts):
    return {**opts,
            "big-config/exit": 1,
            "big-config/err":  "Something went wrong"}

import { ok } from "big-config/core";

export function myStep(opts: Record<string, any>) {
  // do work…
  return ok(opts);              // sets "big-config/exit" -> 0
}

export function failingStep(opts: Record<string, any>) {
  return { ...opts,
           "big-config/exit": 1,
           "big-config/err":  "Something went wrong" };
}

The engine catches exceptions thrown by step functions, merges any structured error data into opts, and sets exit to 1 with the message and stack trace. If a step returns nil/None/undefined or sets a non-natural exit, the engine throws — contracts are enforced.

CLI DSL #

big-config exposes a concise DSL for running workflows from a shell:

bb render lock tofu:init tofu:plan -- tofu apply -auto-approve
#  │      │    │                      └── raw shell command
#  step   step colon-syntax (tofu init)

• Steps (e.g. render, lock) are predefined workflow names.
• Colon syntax: tool:subcommand rewrites to tool subcommand.
• Separator --: everything after it is forwarded as one raw shell command to exec.

Per-language invocation

bb render lock tofu:init -- tofu apply -auto-approve

# Aliased: wrap any tool in the safety-first pipeline
alias tofu="bb render git-check lock exec git-push unlock-any -- tofu"

(workflow/parse-args ["render" "lock" "tofu:init" "tofu:plan"
                      "--" "tofu" "apply" "-auto-approve"])
;; => {:big-config.workflow/steps [:render :lock :exec]
;;     :big-config.run/cmds       ["tofu init" "tofu plan" "tofu apply -auto-approve"]}

;; Recognized step keywords live in workflow/*parse-args-steps*. Defaults:
;; #{:lock :git-check :render :create :delete :validate :describe
;;   :exec :git-push :unlock-any}

uv run big-config render lock tofu:init -- tofu apply -auto-approve

from big_config import workflow

workflow.parse_args(["render", "lock", "tofu:init", "tofu:plan",
                     "--", "tofu", "apply", "-auto-approve"])
# => {"big-config.workflow/steps": ["render", "lock", "exec"],
#     "big-config.run/cmds":       ["tofu init", "tofu plan",
#                                   "tofu apply -auto-approve"]}

# Known step names live in big_config.workflow.PARSE_ARGS_STEPS

npx big-config render lock tofu:init -- tofu apply -auto-approve

import { parseArgs } from "big-config/workflow";

parseArgs(["render", "lock", "tofu:init", "tofu:plan",
           "--", "tofu", "apply", "-auto-approve"]);
// => { "big-config.workflow/steps": ["render", "lock", "exec"],
//      "big-config.run/cmds":       ["tofu init", "tofu plan",
//                                    "tofu apply -auto-approve"] }

Workflow engine #

The fundamental constructor is ->workflow (Clojure) / workflow (Python) / toWorkflow (TypeScript). It produces a function that, given step functions (middleware) and an opts map, loops through steps, threads opts, and decides the next step via a wire-fn.

(ns my.app
  (:require [big-config.core :as core]))

(core/->workflow
  {:first-step ::start                  ; required — qualified keyword
   :last-step  ::end                    ; optional, defaults to `::end` in same ns
   :wire-fn    (fn [step step-fns]
                 (case step
                   ::start [my-fn ::end]
                   ::end   [identity]))
   :next-fn    nil})                    ; optional — for branching

(def lock
  (core/->workflow
    {:first-step ::generate-lock-id
     :wire-fn (fn [step _]
                (case step
                  ::generate-lock-id [generate-lock-id ::delete-tag]
                  ::push-tag         [push-tag ::get-remote-tag]
                  ::end              [identity]))
     :next-fn (fn [step next-step opts]
                (case step
                  ::push-tag (core/choice {:on-success ::end
                                           :on-failure next-step
                                           :opts opts})
                  (core/choice {:on-success next-step
                                :on-failure ::end
                                :opts opts})))}))

from big_config import core

START = "my-app/start"
END   = "my-app/end"

def wire(step, _step_fns):
    if step == START:
        return core.ok, END
    return lambda opts: opts, None

wf = core.workflow({"first_step": START, "wire_fn": wire})
result = wf([], {})       # (step_fns, opts)

import { ok, toWorkflow, choice } from "big-config/core";

const START = "my-app/start";
const END   = "my-app/end";

const wf = toWorkflow({
  firstStep: START,
  wireFn: (step, _stepFns) => {
    if (step === START) return [ok, END];
    return [(opts: any) => opts, undefined];
  },
});

const result = wf([], {});

Step functions (middleware) #

A step function is middleware around an individual step. It receives the step implementation, the step keyword/name, and opts — and must return an updated opts. ->step-fn / step_fn / toStepFn is a convenience constructor that wraps before/after callbacks.

Built-in step functions

The Python (big_config.step_fns) and TypeScript (step-fns.ts) ports expose exit_step_fn / exitStepFn and print_error_step_fn / printErrorStepFn equivalents; the bling-coloring step-fn is Clojure-only.

Pluggable steps #

The pluggable module provides ->workflow* (Clojure) / workflow_star (Python) / toWorkflowStar (TypeScript) — a drop-in that looks up each step in a per-step registry. If a custom handler is registered for a step, it replaces the implementation from wire-fn; otherwise the original runs.

(require '[big-config.pluggable :as pluggable]
         '[big-config.workflow  :as workflow])

(defmethod pluggable/handle-step ::my-step
  [f step step-fns opts]
  (println "Custom step!" step (count step-fns))
  (f opts))

;; Register the step keyword so the DSL parser recognizes it
(binding [workflow/*parse-args-steps*
          (conj workflow/*parse-args-steps* :my-step)]
  (workflow/parse-args ["my-step" "render"]))

;; Unhook for testing
(remove-method pluggable/handle-step ::my-step)

from big_config import pluggable, workflow

@pluggable.handle_step("my-app/my-step")
def my_step(f, step, step_fns, opts):
    print("Custom step!", step, len(step_fns))
    return f(opts)

# Make the DSL parser recognize it
workflow.PARSE_ARGS_STEPS.add("my-step")

import { handleStep } from "big-config/pluggable";
import { PARSE_ARGS_STEPS } from "big-config/workflow";

handleStep("my-app/my-step", (f, step, stepFns, opts) => {
  console.log("Custom step!", step, stepFns.length);
  return f(opts);
});

PARSE_ARGS_STEPS.add("my-step");

Workflow types #

Tool workflow

(ns wf
  (:require [big-config.render   :as render]
            [big-config.workflow :as workflow]))

(defn tofu [step-fns opts]
  (let [opts (workflow/prepare
               {::workflow/name ::tofu
                ::render/templates [{:template "tofu"
                                     :overwrite true
                                     :transform [["tofu" :raw]]}]}
               opts)]
    (workflow/run-steps step-fns opts)))

(defn tofu* [args & [opts]]
  (let [opts (merge (workflow/parse-args args) opts)]
    (tofu [] opts)))

{:deps {group/artifact {:local/root "."}}
 :tasks
 {:requires ([wf :as wf])
  tofu     {:doc "bb tofu render tofu:init tofu:apply:-auto-approve"
            :task (wf/tofu* *command-line-args*)}}}

from big_config import workflow, render

def tofu(step_fns, opts):
    opts = workflow.prepare({
        "big-config.workflow/name":      "wf/tofu",
        "big-config.render/templates":   [{"template": "tofu",
                                           "overwrite": True,
                                           "transform": [("tofu", "raw")]}],
    }, opts)
    return workflow.run_steps(step_fns, opts)

def tofu_star(args, opts=None):
    opts = {**(opts or {}), **workflow.parse_args(args)}
    return tofu([], opts)

import { prepare, runSteps, parseArgs } from "big-config/workflow";

export function tofu(stepFns: any[], opts: Record<string, any>) {
  opts = prepare({
    "big-config.workflow/name": "wf/tofu",
    "big-config.render/templates": [{ template: "tofu",
                                      overwrite: true,
                                      transform: [["tofu", "raw"]] }],
  }, opts);
  return runSteps(stepFns, opts);
}

export function tofuStar(args: string[], opts: Record<string, any> = {}) {
  return tofu([], { ...opts, ...parseArgs(args) });
}

Composite workflow

Composite workflows orchestrate multiple tool workflows through a :pipeline of [tool-keyword [args opts-fn]] pairs. The opts-fn is the glue: it inspects outputs of previous steps (e.g. by parsing tofu show --json) and maps them into ::workflow/params for the next tool — keeping downstream tools (Ansible) decoupled from upstream providers.

(defn opts-fn [opts]
  (let [ip (-> (p/shell {:dir (workflow/path opts ::tool/tofu)
                         :out :string}
                        "tofu show --json")
               :out
               (json/parse-string keyword)
               (->> (s/select-one
                     [:values :root_module :resources
                      s/FIRST :values :ipv4_address])))]
    (merge-with merge opts {::workflow/params {:ip ip}})))

(def resource-create
  (workflow/->workflow*
   {:first-step ::start-create-or-delete
    :last-step  ::end-create-or-delete
    :pipeline   [::tool/tofu          ["render tofu:init tofu:apply:-auto-approve"]
                 ::tool/ansible       ["render ansible-playbook:main.yml" opts-fn]
                 ::tool/ansible-local ["render ansible-playbook:main.yml" opts-fn]]}))

Subworkflow isolation #

run-steps and ->workflow* are a composition layer — a "workflow of workflows." The pure-step contract (a single opts threaded through the steps) holds within one workflow. Across the composition layer the contract is subworkflow isolation:

• Isolated input. Every subworkflow — ::create / ::delete, or each :pipeline step — runs on a purpose-built opts (create-opts / delete-opts, or (merge step-args globals-opts <step>-opts)) seeded from the shared globals — never the parent's running opts. One subworkflow cannot leak transient state into the next.
• Accumulated output. Each subworkflow's terminal opts is collected under its step key (a vector in run-steps, so repeated create/delete runs are kept side-by-side as history). Only ::bc/exit and ::bc/err propagate upward to drive the parent's branching and short-circuit.

Built-in steps #

Tool-workflow steps

Composite-workflow steps

run-steps dispatch

The dynamic workflow built by run-steps iterates the user-provided ::workflow/steps vector and dispatches each step to its implementation. Steps without a namespace are auto-qualified to big-config.workflow. validate and describe are regular step functions of shape [step-fns opts] -> opts.

(workflow/run-steps
  step-fns
  {::workflow/steps       [:validate :create :describe]
   ::workflow/create-fn   my-create-fn
   ::workflow/delete-fn   my-delete-fn
   ::workflow/validate-fn my-validate-fn
   ::workflow/describe-fn my-describe-fn})

Render (template engine) #

big-config.render is a Selmer-based engine inspired by seancorfield/deps-new. A minimal template configuration is a resource path, a target directory, and one or more transforms:

{:big-config.render/templates
 [{:template   "resource-path"
   :target-dir "target"
   :transform  [["."]]}]}

By default, render copies the source folder to :target-dir and replaces any {{key-a}} in filenames and contents with the corresponding :key-a from the data map.

Map glossary

edn keys

Transform shapes

;; src can be a folder or symbol/function
{:transform
 [['ansible/render-files
   {:inventory "inventory.json"
    :config    "default.config.yml"}
   :raw]]}

;; target, files, custom delimiters
{:transform
 [["src" "target"
   {"config.json" "config.json"}
   {:tag-open    \<
    :tag-close   \>
    :filter-open \<
    :filter-close \>}]]}

;; :only — copy just the listed files
{:transform
 [["src" "src/{{data-key-a}}"
   {"main.clj" "{{data-key-b|selmer-filter:param-a:param-b}}.clj"}]
  ["test" "test/{{data-key-b}}"
   {"main_test.clj" "{{data-key-c}}_test.clj"}
   :only]]}

;; :raw — disable Selmer rendering for a directory
{:transform
 [["resources" "resources/{{data-key-a}}"]
  ["templates" "resources/{{data-key-b}}/templates" :raw]]}

Whitespace control workaround

Selmer does not natively support whitespace-trimming delimiters, so big-config implements {{- / -}} via a pre-processing pass (see selmer-filters/whitespace-control). Use it like this:

{{- some-var -}}    # trims surrounding whitespace

Binary files

The dynamic var render/*non-replaced-exts* (Clojure) — and the equivalent constant in the Python and TypeScript ports — lists extensions that bypass Selmer rendering: jpg jpeg png gif bmp bin. Rebind / override to extend.

Lock / Unlock #

big-config's lock is Atlantis-like — but entirely client-side, using Git tags as the backing store. This restores interactive operation by removing the mandatory pull-request flow, and works equally well with Terraform, Ansible, and any other tool.

Required opts keys

Behavior

• The lock id is the hex hash of the selected lock-keys values.
• Re-acquiring a lock with the same owner succeeds (idempotent).
• Acquiring a lock held by a different owner fails with exit 1 and the message "Different owner".

Steps

;; big-config.lock/lock pipeline
::generate-lock-id → ::delete-tag → ::create-tag → ::push-tag
       → on push success: done
       → on push failure: ::get-remote-tag → ::read-tag → ::check-tag

;; big-config.unlock/unlock-any pipeline
::generate-lock-id → ::delete-tag → ::delete-remote-tag → ::check-remote-tag

Usage

(require '[big-config.lock :as lock])

(lock/lock {::lock/owner     "alberto"
            ::lock/lock-keys [::env ::region]
            ::env            "prod"
            ::region         "eu-west-1"})

;; force release
(require '[big-config.unlock :as unlock])
(unlock/unlock-any {::lock/owner     "alberto"
                    ::lock/lock-keys [::env ::region]
                    ::env            "prod"
                    ::region         "eu-west-1"})

Run (shell execution) #

big-config.run wraps the host's process API:

• generic-cmd — runs a single command, captures stdout/stderr, merges results into opts.
• run-cmd — runs one command from ::cmds using ::shell-opts; output handling depends on ::bc/env.
• run-cmds — workflow that runs all of ::cmds sequentially, short-circuiting on failure.
• handle-cmd — strips ANSI escape sequences, accumulates a :big-config/procs trace.
• mktemp-create-dir / mktemp-remove-dir — create and clean throw-away dirs.

Environment behavior:

(require '[big-config.run :as run])

(run/run-cmds
  [(fn [f step opts] (println step) (f step opts))]
  {::bc/env :repl
   ::run/shell-opts {:continue true
                     :dir "big-infra"
                     :extra-env {"FOO" "BAR"}}
   ::run/cmds ["bash -c 'echo Error >&2 && exit 1'"]})

The runner is exposed behind a seam — Clojure: big-config.run/runner; Python and TS expose injectable equivalents — so tests can avoid spawning real processes.

Git helpers #

big-config.git/check is a workflow that asserts the local HEAD matches the upstream:

::git-diff → ::fetch-origin → ::upstream-name → ::pre-revision
          → ::current-revision → ::origin-revision → ::compare-revisions

If the working directory is dirty, the upstream is missing, or the local revisions diverge from origin, the workflow exits with code 1 and "The local revisions don't match the remote revision".

big-config.git/git-push is a thin wrapper around git push, surfaced as the git-push step.

Store (Redis event-sourcing) clj #

big-config.store is a Redis-backed event-sourcing store — a fork of prevayler-clj ideas. It journals each event in a Redis sorted set, replays them on restart, and provides ACID-like guarantees. It exists only in the full Clojure implementation (big-config/main).

• Atomicity — successful WATCH/MULTI/EXEC writes the event and bumps state atomically; failures retry.
• Consistency — guaranteed by the business function; throw to abort.
• Isolation — locking this serializes handle! calls.
• Durability — events live in Redis with a per-event hash for replay verification.

(require '[big-config.store :as store])

(def s
  (store/store!
    {:initial-state {:cnt 0}
     :business-fn   (fn [state {:keys [op]} _ts]
                      (case op :inc (update state :cnt inc)))
     :store-key     "store-state"
     :snapshot-every 10
     :wcar-opts     {:pool (car/connection-pool {})
                     :spec {:uri "redis://localhost:6379/"}}}))

(store/handle! s {:op :inc}) ; => {:cnt 1}
(store/snapshot! s)
(.close ^java.io.Closeable s)

System (lifecycle) clj #

big-config.system is a workflow-based alternative to Integrant. Components are steps, and each step that starts a resource calls add-stop-fn to register cleanup. The terminal stop step iterates the accumulated ::stop-fns. Clojure-only.

(defn background-process [opts]
  (let [re-opts {:cmd     "clj -X my.app/main :shutdown-timeout 100"
                 :regex   #"token"
                 :timeout 3000
                 :key     ::proc}
        opts    (re-process re-opts opts)]
    (add-stop-fn opts
      (fn [{:keys [::proc] :as opts}]
        (when proc (destroy! proc 1000))))))

(def ->system
  (core/->workflow
    {:first-step ::start
     :wire-fn (fn [step _]
                (case step
                  ::start [background-process ::end]
                  ::end   [stop]))}))

;; synchronous (default): blocks until stop
(->system [log-step-fn] {::bc/env :repl})

;; asynchronous — stop later
(def system (->system [log-step-fn] {::bc/env :repl ::async true}))
(stop! system)

BigTofu (OpenTofu / Terraform constructs) #

big-tofu.core defines a To protocol (Clojure) / To interface (TS) / To Protocol (Python) and a Construct record for assembling HCL via plain data. A Construct can be referenced by property (Terraform-style interpolation) or converted into an ARN string.

(require '[big-tofu.core :as t]
         '[big-tofu.create :as tc])

(def kms-blocks  (tc/kms :alpha/big-kms))
(def sqs-blocks  (tc/sqs :alpha/big-sqs))
(def buckets     (tc/bucket :alpha/big-bucket "foo" "bar"))

(->> kms-blocks
     (map t/construct)
     (apply utils/deep-merge)
     utils/sort-nested-map
     pp/pprint)

The stdlib (big-tofu.create) provides convenience builders:

• bucket — variadic, accepts a base FQN and optional suffix tokens.
• sqs — single SQS queue.
• kms — KMS key with an IAM policy permitting the AWS account root.
• provider — AWS + S3 backend stanza, optional assume_role.

The To protocol surfaces:

BC_PAR_ overrides #

Override any project parameter from CI by prefixing the env var with BC_PAR_. The prefix is stripped, the rest is lower-cased, underscores and dots become dashes, and the result is converted to a key inside ::workflow/params.

export BC_PAR_PROVIDER_BACKEND="local"
# becomes {:provider-backend "local"} in ::workflow/params

This is applied by workflow/read-bc-pars (Clojure) / workflow.read_bc_pars (Python) / readBcPars (TypeScript), which merges the env-derived params with the existing ones.

Templates clj #

big-config ships a template scaffolding CLI (big-config.tools) that exposes generators as Clojure tools. This is Clojure-only — the Python and TypeScript ports deliberately omit scaffolding.

Active templates

Deprecated deprecated

Example

clojure -Tbig-config package \
  :owner amiorin :repository joe \
  :target-dir ../../joe

Utilities & macros #

The debug macro (Clojure)

(require '[big-config.utils :refer [debug]])

(debug tap-values
  (defn fn-wip []
    (tap> [:fn-wip {::b {::d 4 ::c 3} ::a 1}])
    {::b {::d 4 ::c 3} ::a 1})
  (fn-wip))
;; tap-values is now def'd in the current ns as a sorted vector
;; of every tap> emitted inside the body.

Even when the body throws, tap-values is still populated for inspection. LSP indentation for the macro is configured in .lsp/config.edn.

Selmer extras #

big-config.selmer-filters registers two filters and exposes whitespace-control:

• :lookup-env — reads an environment variable.
• :->file — replaces . with / and - with _, suitable for converting a namespace into a file path.
• whitespace-control — pre-processor that implements {{- / -}} and {%- / -%} (and the matching closers) by trimming surrounding whitespace before Selmer runs.

{{ "PATH"|lookup-env }}            # reads $PATH
{{ "big-config.core"|->file }}     # "big_config/core"

Dependencies #

Overview #

once is a big-config package that wraps a six-stage Tofu + Ansible pipeline behind a single CLI. It is named for Basecamp's ONCE deployment model — the audience is the vibe coder who wants a one-click experience for getting a vibe-coded app online on Hetzner, Oracle Cloud, DigitalOcean, or an already-provisioned server.

All three implementations share the same six-stage shape and the same template tree under src/resources/io/github/amiorin/once/tools/ (Tofu and Ansible manifests). What differs is the CLI entry point and the language of options / params / tools / validation / describe.

Install & commands #

All three implementations require external tools at runtime: tofu, ansible-playbook, ssh, curl, skopeo, and per-provider CLIs. The remote deploy ForceCommand script and the once Ansible module are Babashka scripts that run on the provisioned host — they ship as resources, not host code.

bb -tidy                          # clean-ns + format
clojure -M:test                   # runs cognitect test-runner against test/clj
bb validate                       # shortcut for `bb once [package] validate`
bb once [package] describe        # post-provisioning report
bb once [package] build           # render all 6 stages — `once/clojure` only
bb once [package] create          # full 6-stage create pipeline
bb once [package] delete          # reverse the 4 Tofu stages (4→3→2→1)
bb once [package] validate create # validate, then create only if validation passes
bb once [package] delete  create  # clean-slate redeploy

uv sync --dev
uv run pytest
uv run once --help
uv run once package validate
uv run once package describe
uv run once package create
uv run once package delete

npm install
npm test                 # vitest run
npm run typecheck        # tsc --noEmit
npm run build            # tsc -> dist/

once package validate    # pre-flight checks
once package describe    # providers + SSH reachability + apps
once package build       # render without applying
once package create      # full 6-stage pipeline
once package delete      # reverse the 4 Tofu stages
once validate            # shortcut for `once package validate`

# During development, prefix with: npm run once -- ...
npm run once -- package validate

The six-stage create pipeline #

1. tofu — provision compute (DigitalOcean / hcloud / OCI / no-infra).
2. tofu-smtp — set up SMTP (Resend).
3. tofu-dns — configure DNS (Cloudflare), injecting SMTP records.
4. tofu-smtp-post — finalize SMTP after DNS verification.
5. ansible-local — local config: update ~/.ssh/config so the remote host is reachable as Host once for the next stage.
6. ansible — remote host config: install Docker, ONCE, s-nail; configure .mailrc; provision the restricted deploy user; deploy applications listed under :once {:applications [...]}.

delete reverses the Tofu stages (4→3→2→1 destroy order). Compute resources are rendered with lifecycle { prevent_destroy = true } by default; override with BC_PAR_COMPUTE_PREVENT_DESTROY=false before delete.

validate and describe are opt-in workflow/run-steps steps. They do not run automatically before create.

Profiles #

Two layers compose into a profile: private compute base maps (oci, hcloud, digitalocean, no-infra-compute) and public application profiles (online, space, website, no-infra; the TS port calls them profileAlpha, profileBeta, profileGamma, profileNoInfra) that pin a domain, package name, and the deployed applications list. All four application profiles also merge a private deploy sub-profile carrying two SSH public keys:

• :compute-pubkey — its private half must be loaded in ssh-agent so Ansible can reach the freshly provisioned VM. The validate step enforces this for cloud compute profiles.
• :deploy-pubkey — authorized on the remote deploy user with ForceCommand.

(def space
  (merge-with merge resend cloudflare r2 oci deploy
              {::render/profile "space"
               ::workflow/params
               {:domain  "bigconfig.space"
                :package "space"
                :once {:applications
                       [{:host  "marketplace-api.bigconfig.space"
                         :image "ghcr.io/amiorin/once-pocketbase"
                         :env   ["SUPERUSER_PASSWORD=<{ superuser-password }>"]}]}}}))

(def bb website)  ; change this to switch profiles

space = deep_merge(
    resend, cloudflare, r2, oci, deploy,
    {
        "big-config.render/profile": "space",
        "big-config.workflow/params": {
            "domain":  "bigconfig.space",
            "package": "space",
            "once": {
                "applications": [
                    {"host":  "marketplace-api.bigconfig.space",
                     "image": "ghcr.io/amiorin/once-pocketbase",
                     "env":   ["SUPERUSER_PASSWORD=<{ superuser-password }>"]},
                ],
            },
        },
    },
)

bb = website  # change this to switch profiles

export const profileBeta = compose(resend, cloudflare, r2, oci, deploy, {
  "big-config.render/profile": "space",
  "big-config.workflow/params": {
    "domain":  "bigconfig.space",
    "package": "space",
    "once": {
      "applications": [{
        "host":  "marketplace-api.bigconfig.space",
        "image": "ghcr.io/amiorin/once-pocketbase",
        "env":   ["SUPERUSER_PASSWORD=<{ superuser-password }>"],
      }],
    },
  },
});

export const bb = profileAlpha;  // change this to switch profiles

Cloud providers #

All cloud profiles combine with resend (SMTP) and cloudflare (DNS) sub-profiles. The Cloudflare DNS template (provider ~> 5.0) creates apex (@) and wildcard (*) A records proxied through Cloudflare and applies a fixed bundle of zone settings (TLS 1.3, strict SSL, always-use-HTTPS, brotli, etc.). Outgoing mail is sent from info@notifications.<domain>.

Parameter flow #

1. Profiles in options define base params under ::workflow/params.
2. params/opts-fn composes three transformations: workflow/read-bc-pars (reads BC_PAR_*) → tofu-smtp-params (extracts SMTP records from Tofu output) → tofu-params (extracts IP from Tofu output).
3. Each later stage inherits the outputs of earlier stages.

(def opts-fn (comp tofu-params tofu-smtp-params workflow/read-bc-pars))

BC_PAR_* overrides

export BC_PAR_DOMAIN="example.com"
export BC_PAR_PROVIDER_BACKEND="s3"   # or "r2" / "local"
export BC_PAR_S3_BUCKET="my-tf-state-bucket"
export BC_PAR_HCLOUD_TOKEN="xxx"

Variable names are uppercased; hyphens / dots become underscores. Sensitive credentials go in .envrc.private (gitignored).

Plugin system (remote-state backend) #

tools uses big-config's pluggable/handle-step for the remote-state backend plugin (::render-tofu-backend). After each render step, the plugin injects the backend configuration (S3, R2, or local) based on :provider-backend — done via run-steps-with-plugin in Clojure and equivalent dispatch in Python/TS.

(defmethod pluggable/handle-step ::render-tofu-backend
  [f step step-fns opts]
  (let [backend (get-in opts [::workflow/params :provider-backend])]
    (case backend
      "s3"    (-> opts (assoc-in [::render/templates 0 :transform]
                                 [["tofu-backend/s3"]]) f)
      "r2"    (-> opts (assoc-in [::render/templates 0 :transform]
                                 [["tofu-backend/r2"]]) f)
      "local" (-> opts (assoc-in [::render/templates 0 :transform]
                                 [["tofu-backend/local"]]) f)
      (f opts))))

Validation #

validation/validate is a workflow step. validate-report / validateReport is the pure report builder — it accepts injected collaborators for testability. Checks include:

• Profile schema — validated against a Malli schema (Clojure) or equivalent shape check (Python/TS).
• Required CLIs — verifies tofu, ansible-playbook, ssh, skopeo, and per-provider CLIs are on PATH.
• Credentials — checks per-provider env vars (e.g. DIGITALOCEAN_TOKEN, HCLOUD_TOKEN, CLOUDFLARE_API_TOKEN, RESEND_API_KEY, OCI config).
• Image references — uses skopeo inspect to verify each configured app's image exists.
• ssh-agent — confirms :compute-pubkey is loaded in ssh-agent (cloud providers only).

bb validate is a strict shortcut that exits non-zero if any extra command-line arguments are passed. bb once [package] validate is the general form usable in chains, e.g. bb once [package] validate create.

Describe (post-provisioning report) #

describe/describe is a workflow step; describe-report / describeReport is the pure report builder. The report lists configured providers, SSH reachability, and per-application status:

The deploy ForceCommand #

Stage 6 (Ansible) provisions a restricted deploy user on the host:

• NOPASSWD sudo limited to /usr/local/bin/once *.
• SSH ForceCommand bound to a Babashka script at /usr/local/bin/deploy. The script accepts only sudo once update <host> for hosts present in once list.
• Authorized by :deploy-pubkey; CI-friendly redeploys without root SSH.

Tests of the deploy script live in test/clj/.../deploy_test.clj (Clojure) and in the corresponding pytest / Vitest files. They require babashka (bb) on PATH and are skipped automatically when it is missing.

Individual tools #

Each stage can be driven directly via its tool entry point. Every tool must run render first.

bb -tofu           render tofu:init tofu:apply:-auto-approve
bb -tofu-smtp      render tofu:init tofu:apply:-auto-approve
bb -tofu-dns       render tofu:init tofu:apply:-auto-approve
bb -tofu-smtp-post render tofu:init tofu:apply:-auto-approve
bb -ansible        render -- ansible-playbook main.yml
bb -ansible-local  render -- ansible-playbook main.yml

uv run once tofu           render tofu:init tofu:apply:-auto-approve
uv run once tofu-smtp      render tofu:init tofu:apply:-auto-approve
uv run once tofu-dns       render tofu:init tofu:apply:-auto-approve
uv run once tofu-smtp-post render tofu:init tofu:apply:-auto-approve
uv run once ansible        render -- ansible-playbook main.yml
uv run once ansible-local  render -- ansible-playbook main.yml

once tofu           render tofu:init tofu:apply:-auto-approve
once tofu-smtp      render tofu:init tofu:apply:-auto-approve
once tofu-dns       render tofu:init tofu:apply:-auto-approve
once tofu-smtp-post render tofu:init tofu:apply:-auto-approve
once ansible        render -- ansible-playbook main.yml
once ansible-local  render -- ansible-playbook main.yml

# During development:
npm run once -- tofu render tofu:init tofu:apply:-auto-approve

Per-language layout #

Template directories

• tofu/ — multi-cloud .tf templates (DigitalOcean, hcloud, OCI, no-infra)
• tofu-backend/ — remote state backend templates (s3, r2, local)
• tofu-smtp/ — SMTP (Resend) setup templates
• tofu-dns/ — DNS (Cloudflare) templates
• tofu-smtp-post/ — SMTP post-verification templates
• ansible/ — remote host playbooks (incl. files/deploy bb script)
• ansible-local/ — local machine playbooks

Conventions #

Naming

Error handling

• Throw structured errors (ex-info / raise MyError(...) / throw new Error(...)).
• The workflow engine catches exceptions and merges error data + err + stack-trace into opts.
• Validate at system boundaries; trust internal function contracts.

REPL / interactive development

The Clojure source uses (comment …) blocks for documentation-as-tests:

(comment
  (debug tap-values
    (once* "create" options/oci))
  (-> tap-values))

Use CIDER with :dev alias. Python/TS use pytest fixtures and Vitest watch mode for the equivalent feedback loop.

Git

• The canonical Clojure repos (Selmer/master, big-config/main, once/main) follow trunk-based workflows — commit directly to main; do not create feature branches.
• Conventional Commits are used: feat:, refactor:, deps:, docs:.
• Commit only when explicitly asked.

What to avoid #

• Do not pass step-fns inside the ->workflow map — pass them as the first positional argument when invoking the workflow.
• Do not use build as a synonym for render — render is the current name (the old deps-new integration was renamed).
• Do not modify .dist/ — it is generated output, not source.
• Do not reintroduce Clojure source / Clojure tests / Babashka tasks into the Python or TypeScript trees.
• Do not vendor Selmer into a consumer — keep it as the configured local-path dependency.
• Do not "de-atomize" run-steps / ->workflow* — the closed-over mutable state is the subworkflow-isolation barrier.
• Do not add error handling for cases that cannot happen — big-config handles step failure via ::bc/exit / ::bc/err.
• Do not create new namespaces in once unless a genuine new concern arises; the six existing modules (options, package, params, tools, validation, describe) map cleanly to their responsibilities.
• Do not add a store, system lifecycle, or build-helper module to the JVM-only subset (big-config/clojure) or to the Python/TypeScript ports unless explicitly requested.
• Do not commit credentials. Use .envrc.private (gitignored).
• TypeScript only: keep imports using explicit .js extensions (required by NodeNext module resolution).

Glossary #

opts map

The big-config map threaded through every workflow step.

step

A qualified keyword / namespaced string naming a node in a workflow's state graph.

step function

Middleware wrapping a step. Built via ->step-fn / step_fn / toStepFn or written by hand [f step opts] -> opts.

tool workflow

The fundamental unit: render + execute one CLI tool.

comp workflow

Sequences tool workflows for create / delete lifecycles and can expose opt-in validate / describe hooks.

system workflow

Lifecycle orchestrator for background components (Clojure only; see big-config.system).

profile

Convention for environment identifiers (dev, prod, …) and, in once, the active application profile. Available to templates as :profile.

module

Convention for monorepo sub-projects (borrowed from Integrant). Available to templates as :module.

prefix / object-prefix

Base directory and object naming. Defaults: .dist and tofu. Stamped with a unique hash by workflow/new-prefix so concurrent invocations don't collide.

lock-id

Hex hash of the values selected by :big-config.lock/lock-keys — uniquely identifies a locked resource.

BC_PAR_*

Environment-variable prefix that maps to ::workflow/params overrides.

ForceCommand

The OpenSSH authorized_keys directive used by once to restrict the deploy user to a single Babashka script.