tmc.dev

How Should a Command-Line Tool Let Other People Extend It?

This is the fundamental decision in any tool that expects an ecosystem. The answer determines whether third parties can write extensions at all, what languages they can write them in, how the tool behaves when an extension is broken, and how much of the tool's own complexity leaks into everyone else's code.

Git answered it in 2006 with two lines:

strvec_pushf(&cmd.args, "git-%s", argv[0]);
strvec_pushv(&cmd.args, argv + 1);

Prepend git- to the subcommand name, append the arguments unchanged, and exec it. There is no registry, no manifest, no plugin API, no ABI, and no daemon. The operating system's PATH lookup is the discovery mechanism. Twenty years later, thousands of third-party subcommands exist and none of them had to be registered anywhere.

I used to read this as a historical accident — the kind of thing you get when a tool starts as shell scripts and never grows up. I now think it is the most carefully considered part of git's user interface, and that the care went somewhere other than where I expected.

Everything below is from git 2.52.0, commit 9a0c4701. I checked each behavioral claim against a running git instead of inferring it from the code, and twice that changed my answer.

The mechanism

The dispatch lives in git.c, in cmd_main and run_argv. It tries four things in order:

  1. a builtin, from a static array
  2. a git-foo executable, via PATH
  3. an alias, from config
  4. a spelling correction, once

The order matters more than the list. Builtins come first, so no executable on your PATH can shadow git log; I put a git-log script on PATH and ran git log -1, and the builtin ran. Aliases come last, after externals, which is surprising until you read why:

/*
 * It could be an alias -- this works around the insanity
 * of overriding "git log" with "git show" by having
 * alias.log = show
 */

Someone wanted alias.log = show. Rather than forbid it, git ordered the lookup so the alias never gets the chance to shadow the real command, and wrote down what it thought of the idea.

The registry is a static array:

static struct cmd_struct commands[] = {
	{ "add", cmd_add, RUN_SETUP | NEED_WORK_TREE },
	{ "apply", cmd_apply, RUN_SETUP_GENTLY },
	{ "branch", cmd_branch, RUN_SETUP | DELAY_PAGER_CONFIG },
	{ "diff", cmd_diff, NO_PARSEOPT },
	...
};

The flags are the interesting column. RUN_SETUP means the command must be in a repository. RUN_SETUP_GENTLY means it will use one if present. NEED_WORK_TREE means a bare repository is not enough. Each command declares its preconditions, and run_builtin establishes them before the command runs. cmd_add does not check whether it is in a repository. It cannot run outside one.

Most CLI frameworks do the opposite. They hand each subcommand a context object and let it decide what to do about it, which means the answer to "does this command need a work tree" lives in a hundred places instead of one column of one table.

Failure is the interface

The part of execv_dashed_external that does the real work is not the exec. It is what happens when the exec fails:

if (status >= 0)
	exit(status);
else if (errno != ENOENT)
	exit(128);

ENOENT means the file was not found, so fall through and try an alias. Any other error is fatal. Without that distinction there is no layered lookup: either every failure ends the search, and aliases never run, or no failure does, and a plugin that crashes gets silently retried as something else.

This is the whole trust model, and it is a conditional on errno.

Git also modifies PATH before any of this happens. setup_path prepends git's own libexec/git-core, which has two consequences people rarely expect. Git's 176 shipped executables take precedence over anything in your PATH, and your subcommand inherits the modified PATH and can invoke git-sh-setup directly. The second is why it does this; the first is the price.

What a subcommand actually gets

Here I was wrong, and I only found out by running it.

I wrote a git-probe script that prints its environment, put it on PATH, and ran it from a subdirectory of a repository:

argv: --flag arg1
GIT_PREFIX=<unset>
GIT_DIR=<unset>
GIT_WORK_TREE=<unset>
cwd=/repo/content/posts

An external subcommand gets no repository context at all. It runs in whatever directory the user was in, and it must find the repository itself.

Aliases are different. The same script, invoked through alias.aprobe = !git-probe:

GIT_PREFIX=content/posts/
cwd=/repo

The working directory is now the repository root and GIT_PREFIX records where the user actually was. handle_alias arranges this on purpose:

if (alias_string[0] == '!') {
	/* Aliases expect GIT_PREFIX, GIT_DIR etc to be set */
	setup_git_directory_gently(the_repository, &nongit_ok);

So git has two extension mechanisms with opposite conventions. Shell aliases run from the root with the prefix in the environment. External subcommands run in place with nothing.

The likely reason is history rather than design — I am inferring this from the code and the shapes of the two paths, not from a list thread. Aliases are expanded by git, in git's own process, where the repository has already been located, so passing it along costs nothing. External subcommands were originally shell scripts that sourced git-sh-setup and called git rev-parse themselves, so there was nothing to pass along.

Of the two, the external convention is the better one, which took me a second look to see. Moving to the repository root breaks every relative path the user typed. From a subdirectory containing x.go:

$ git argvprobe ./x.go        # external
cwd=/repo/sub
./x.go EXISTS

$ git ap ./x.go               # alias to the same script
cwd=/repo
./x.go MISSING

GIT_PREFIX is not a convenience the external path lacks. It is the repair for damage the alias path does to argv, and the external path does not need it.

So: if you write a git-foo today, resolve the repository yourself and do not read GIT_PREFIX.

Where the effort went

The mechanism is two lines. The care is somewhere else, and it took me a while to see it.

When an alias expands to a cycle, git does not report a depth limit. It prints the cycle:

alias loop detected: expansion of 'a' does not terminate:
  a <==
  b
  c ==>

A counter would have been three lines of code and would have told the user that something was wrong. This is twenty lines and tells them which config entry to edit.

When an alias contains a flag that would change the environment, git refuses, and the refusal contains the fix:

if (envchanged)
	die(_("alias '%s' changes environment variables.\n"
	      "You can use '!git' in the alias to do this"),
	    alias_command);

And when exec fails with ENOENT on a command that git can see exists, it does not offer a spelling correction for the word you typed correctly:

/*
 * An exact match means we have the command, but
 * for some reason exec'ing it gave us ENOENT; probably
 * it's a bad interpreter in the #! line.
 */
if (!strcmp(candidate, cmd))
	die(_(bad_interpreter_advice), cmd, cmd);

That is the most confusing error in the whole system — the file is right there, and the kernel says it does not exist — turned into a specific diagnosis. It happens because the shebang line names an interpreter that is missing, and ENOENT refers to the interpreter, not the script.

None of this makes git faster or more capable. All of it is spent on the moment when something has already gone wrong.

There is also an honest admission in the alias path. After expanding an alias, git re-executes itself as a subprocess rather than calling the builtin directly, because the environment may have changed:

/*
 * NEEDSWORK: if we can figure out cases
 * where it is safe to do, we can avoid spawning a new
 * process.
 */

The comment has been there for most of git's life. Shipping the slower path and writing down why is better than shipping a faster one that is subtly wrong, and better than pretending the question is settled.

Delegation, not inheritance

Extensions need to know things about the environment they run in. The obvious way to tell them is an environment variable, and git mostly does not do that. It exposes queries instead.

Color is the clearest case. Users configure color.ui and per-command color.<cmd> keys, and a subcommand is expected to honor them. Rather than exporting the resolved answer, git ships two plumbing commands:

c_branch=$(git config --get-color color.wt.branch "green")

That returns the ANSI escape for the configured slot, falling back to the spec you supply, in git's own color grammar — bold blue, brightyellow ul, #ff0000 for truecolor. The slot does not have to be one git knows about.

Whether to colorize at all is a separate query, because --get-color always emits escapes:

if git config --get-colorbool color.wt; then
	use_color=1
fi

The interesting part is that --get-colorbool knows more than you do:

if (*is_tty_p || (fd == 1 && pager_in_use() && pager_use_color)) {
	if (!is_terminal_dumb())
		return true;
}

It checks whether stdout is a terminal, but also says yes when output is going to a color-capable pager, and no when TERM=dumb. Hand-rolled color detection usually gets the first case and misses both others.

There is a trap here, and I walked into it. The optional second argument does not just supply a TTY hint. It switches the command from returning an exit status to printing a word:

git config --get-colorbool color.wt        # exit 1, prints nothing
git config --get-colorbool color.wt true   # exit 0, prints "true"
git config --get-colorbool color.wt false  # exit 0, prints "false"

So if git config --get-colorbool color.wt true is always true, no matter how color is configured. With two arguments you have to read stdout. Use the one-argument form.

The pattern generalizes past color. The host owns a policy, the policy has more cases than any extension author will think of, and the host exposes it as a command instead of a variable. Extensions in any language get the same answer, and when the policy grows a case — a new pager, a new environment variable — it grows in one place.

What Go should copy, and what it should not

The mechanism ports directly. Discovery and dispatch are about fifteen lines:

// findExternal returns the path to the tool-<name> executable, or "".
func findExternal(name string) string {
	path, err := exec.LookPath("tool-" + name)
	if err != nil {
		return ""
	}
	return path
}

func runExternal(ctx context.Context, path string, args []string, root, prefix string) error {
	cmd := exec.CommandContext(ctx, path, args...)
	cmd.Stdin, cmd.Stdout, cmd.Stderr = os.Stdin, os.Stdout, os.Stderr
	cmd.Env = append(os.Environ(), "TOOL_ROOT="+root, "TOOL_PREFIX="+prefix)
	return cmd.Run()
}

Keep the precedence — builtin, external, alias — because a command name should mean the same thing on every machine, and only builtins-first guarantees that. But say so when it bites: if a name resolves to a builtin and something on PATH, warn. Git added stash, switch, and restore as builtins over the years, and each conversion silently disabled anyone's plugin of that name. Git is big enough to absorb that. A young ecosystem is not, and one extra LookPath buys warning: tool-foo on PATH is shadowed by the builtin.

Then there is the part I got wrong, and it is worth walking through because the language does not line up with the C the way I assumed.

Git's distinction is between ENOENT and every other exec failure. The obvious Go translation is exec.ErrNotFound from LookPath against *exec.ExitError from a command that ran and failed. That translation is incorrect. exec.ErrNotFound is wider than ENOENT: a file that exists on PATH but is not executable also reports ErrNotFound, and errors.Is(err, fs.ErrPermission) is false.

tool-noexec      LookPath err=executable file not found in $PATH
                   ErrNotFound=true ErrPermission=false

So the plugin someone forgot to chmod +x does not produce an error. It falls through to alias lookup, then to spelling correction, and the user is told their command does not exist while they are looking at the file. That is the confusing fallback this post keeps warning about, reintroduced by a one-line translation error. To recover git's behavior you have to stat the candidate yourself.

The same measurement turns up something better, and it is the reverse case. A script with a broken shebang passes LookPath and fails at Run:

tool-badshebang  LookPath err=<nil>
                   Run err=fork/exec ...: no such file or directory
                   ExitError=false ErrNotExist=true

That is the diagnosis git spends twenty lines and a Levenshtein table to reach — ENOENT on a file that is right there means the interpreter is missing. In Go you get it for free, because LookPath and Run are separate calls and success of the first tells you the file exists:

if err := cmd.Run(); errors.Is(err, fs.ErrNotExist) {
	return fmt.Errorf("%s: bad interpreter in #! line", path)
}

Three lines, and it fires on exactly the case that produces git's most confusing error. Two of the three moments where a user of your tool is already confused — missing +x, broken shebang — are cheaper to diagnose in Go than in C. The third, a name shadowed by a new builtin, costs one LookPath.

Propagate the child's exit code rather than inventing your own.

Two more things are worth doing differently.

Pass the repository context — but not the way aliases do. Export TOOL_ROOT and TOOL_PREFIX, and leave the working directory alone. Git's external convention is the correct one here: it keeps argv honest. Run git foo ./x.go from a subdirectory and the external subcommand can open ./x.go; the alias cannot, because it has already been moved to the repository root, and GIT_PREFIX exists to repair the damage. Give extensions the information without moving them.

Leave PATH alone. Git prepends its own directory because it has sibling executables to find — 176 names in git --exec-path on my machine, though only 26 are distinct files and the rest are symlinks, so treat that as a packaging detail rather than a number. If you ship one binary you do not need any of it, and a silently modified PATH inherited by every descendant process is a strange thing to do to someone.

I originally had a third item here, about killing the process group so a shell-wrapper plugin does not leak its grandchildren. It is not that simple. CommandContext sets cmd.Cancel to cmd.Process.Kill(), which signals the direct pid, so Setpgid alone redirects nothing; you also have to override Cancel to signal -pid and set WaitDelay, or Wait blocks on a pipe the survivors still hold. And a new process group is outside the terminal's foreground group, so Ctrl-C no longer reaches the plugin and a plugin reading stdin takes SIGTTIN and stops. Doing it correctly means tcsetpgrp and handling SIGTTOU. Git uses atexit cleanup rather than process groups, and now I understand that as a choice rather than an omission.

The environment, and a boundary that is not one

Git passes the parent environment to subcommands unchanged. In 2006 an environment held HOME, PATH, and a proxy setting. Today it holds cloud credentials, registry tokens, and API keys, so the obvious move is an allowlist: copy through HOME, PATH, TERM, and a handful of others, and drop the rest.

I had a dozen lines of that here, recommended without qualification. It does not survive the test this post applies to everyone else's ideas.

The allowlist keeps HOME, and every credential that matters lives under HOME: ~/.aws/credentials, ~/.netrc, ~/.config/gh/hosts.yml, ~/.docker/config.json, ~/.npmrc. A plugin that wants them opens the files. Two paragraphs earlier the same code hands that process os.Stdin, os.Stdout, and os.Stderr — your terminal — because streaming and interactivity are worth more than the theoretical boundary. The threat model was already conceded there. What the allowlist stops is the subset of secrets that happen to live in environment variables, from a process that can read the filesystem.

What it costs is longer than what it keeps. Drop HTTPS_PROXY and a plugin fails on a corporate network. Drop SSL_CERT_FILE and custom CA bundles stop working. Drop SSH_AUTH_SOCK and anything doing git-over-ssh breaks. Drop GOFLAGS and GOMODCACHE and a Go plugin gets a different cache or dies in sandboxed CI. On Windows it is fatal rather than annoying: without SYSTEMROOT winsock will not initialize, and without PATHEXT LookPath finds nothing.

The test I would apply to somebody else's plugin manifest is whether it enforces anything or merely claims something. An environment allowlist does not pass it, and I was about to sell it as though it did. It is worth doing as tidiness, the way you would avoid logging a token; it is not worth doing as a security boundary, and it breaks real things in exchange.

If you want a boundary, take a real one — sandbox-exec, Landlock, a container — and accept the cost. If you do not, inherit the environment like git does and say plainly that a plugin on PATH runs as you. The allowlist is the middle that breaks working setups without buying protection.

The rest of the hygiene still holds, because none of it pretends to be a boundary: stdout carries data and stderr carries diagnostics, never merged; bound what you feed a plugin rather than piping a whole repository; and remember that finding a file on PATH tells you nothing about whether to trust it.

When the process model is not enough

Two cases genuinely exceed it.

The obvious one is typed discovery — you want tool plugin list to print descriptions, not bare names. This does not require leaving the process model. Ask the plugin:

tool-foo --tool-plugin-metadata

Have it answer with JSON, and cache the answer by path, size, and modification time.

Notice what that verb is doing, though. PATH has no listing primitive, so list means scanning every directory on PATH for tool-* — which is what git help -a does — and the metadata probe then executes all of them. On a cold cache, a read-only-sounding subcommand runs every matching binary on the machine. That is defensible, but it should be deliberate: cache aggressively, set a timeout, give the probe no stdin, and bound its stdout.

For supply chain concerns, the useful move is to keep compatibility and trust separable rather than to build a marketplace. Plain tool-foo on PATH is the compatibility layer. Above it, optionally: a metadata probe, then a manifest declaring version, source commit, checksum, and network and write intent, then verification with signatures and govulncheck -mode binary, then local policy about what may run.

A manifest is not a sandbox. It is a claim that can be checked and shown to a user, which is worth something exactly as long as nobody mistakes it for enforcement — the same test the environment allowlist above fails. govulncheck -mode binary answers one question, whether a built Go binary contains known reachable vulnerabilities, and not who built it or whether the shell wrapper around it is safe.

Git has the compatibility layer and nothing above it. That has been enough for twenty years, which is evidence about what the layers above are worth, though not proof.

The other case people reach for is an agent-facing tool surface, which is what MCP is for. Ship it as a second binary and keep the executable that already works: CLI first for people and scripts, MCP second for agents and rich hosts.

The mistake

The failure mode is to solve trust and orchestration inside the host: a daemon, a registry, a sandbox, a policy engine, an installer, a verifier, a transport, and three kinds of cache. Each is defensible alone. Together they are a second product, and they arrive before anyone has agreed on what the first one's interface is.

Before adding any of it, it is worth asking whether the plain executable model is actually insufficient, whether users are asking for this or only implementers find it interesting, and whether it could ship as a separate tool. When the answer is unclear, the cost of waiting is low and the cost of an interface you cannot withdraw is high.

Git's extension mechanism is a string concatenation, an exec, and a careful reading of errno. The sophistication went into the alias cycle that names its own members, the shebang diagnosis, the refusal that tells you to write !git — into the moments when someone is already confused.

That is the part worth copying. Not the two lines, which anyone can write, but the decision about where to spend the next two hundred.

goclipluginsgitmcpsecuritysupply-chain