Add system(command; args) operator (disabled by default)

[Copilot]

Adds a system operator that executes an external command and returns its stdout as a string value. Disabled by default for security — requires explicit opt-in via --enable-system-operator.

Behaviour

• Disabled (default): logs a warning, returns null for each matched node
• Enabled: runs the command, pipes the current node's YAML representation to stdin, returns trimmed stdout as a !!str

# Without flag: warns and returns null
yq '.country = system("/usr/bin/echo"; "test")' data.yml
# => country: null

# With flag: executes the command
yq --enable-system-operator '.country = system("/usr/bin/echo"; "test")' data.yml
# => country: test

# Array arguments
yq --enable-system-operator '.value = system("/bin/echo"; ["hello", "world"])' data.yml
# => value: hello world

# No arguments (stdin-only)
yq --enable-system-operator '.result = system("/usr/local/bin/my-filter")' data.yml

Changes

• security_prefs.go — adds EnableSystemOps bool (default false)
• operator_system.go — operator implementation; supports system("cmd"), system("cmd"; "arg"), and system("cmd"; ["arg1", "arg2"]); captures stderr in error messages
• operation.go — registers systemOpType
• lexer_participle.go — registers simpleOp("system", systemOpType)
• cmd/root.go — adds --enable-system-operator persistent flag
• operator_system_test.go — covers disabled/enabled states, array args, and command failure
• doc/operators/headers/system-operators.md — documentation header

---

📱 Kick off Copilot coding agent tasks wherever you are with GitHub Mobile, available on iOS and Android.

[Copilot]

Pull request overview

Adds a new system(...) operator to yqlib that can execute external commands (disabled by default) and wires it through parsing, CLI configuration, tests, and operator documentation.

Changes:

• Introduces SecurityPreferences.EnableSystemOps (default false) and a --enable-system-operator CLI flag to opt in.
• Implements system operator parsing/registration and execution logic (stdin = current node YAML, stdout -> string).
• Adds operator scenarios + generated documentation pages/headers.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
pkg/yqlib/security_prefs.go	Adds security preference toggle for system operator.
pkg/yqlib/operator_system.go	Implements system operator execution and result mapping.
pkg/yqlib/operation.go	Registers SYSTEM operation type.
pkg/yqlib/lexer_participle.go	Adds system to the lexer/operator table.
cmd/root.go	Adds --enable-system-operator persistent flag.
pkg/yqlib/operator_system_test.go	Adds scenarios for disabled/enabled behavior and failures.
pkg/yqlib/doc/operators/system-operators.md	Adds generated operator docs with examples/output.
pkg/yqlib/doc/operators/headers/system-operators.md	Adds doc header snippet included by generator.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

command/argsExpression are resolved once using context.ReadOnlyClone() before iterating over context.MatchingNodes, and only the first match is used. This means system(<expr>) and system(<expr>; <expr>) cannot vary per matched node (e.g. system(.cmd; .args)), and may pick an unrelated first result when multiple nodes are matched. Consider evaluating the command/args expressions inside the per-candidate loop using context.SingleReadonlyChildContext(candidate) (and failing if they don't yield a single scalar/sequence) so parameter expressions behave consistently with other operators like load.

[mikefarah]

@copilot apply changes based on this feedback

[Copilot]

Done in da611f7. Command and args expressions are now evaluated inside the per-candidate loop using context.SingleReadonlyChildContext(candidate), matching the pattern used by loadOperator. This allows expressions like system(.cmd; .arg) to resolve .cmd and .arg against each matched node individually. Also added a test case for per-node evaluation, used exec.LookPath for portability, and fixed CRLF-aware newline trimming.

[Copilot]

These tests depend on hard-coded absolute paths like /usr/bin/echo and /bin/echo, which can fail on some Linux distros/containers and all non-POSIX platforms. Consider resolving the executable at runtime (e.g. via exec.LookPath) or skipping these scenarios when the required commands aren't available, to keep the test suite portable and reliable.

[Copilot]

This failure test relies on /bin/false, which may not exist in minimal environments. Consider a more portable failure mechanism (e.g. executing a guaranteed-nonexistent command and asserting the error, or skipping when false isn't found).

[Copilot]

The enabled example command omits the required --enable-system-operator flag, so the snippet as written would return null rather than test. Please update the documented command (or adjust the doc generation for this operator) so the example is runnable and matches the shown output.

[mikefarah]

@copilot apply changes based on this feedback

[Copilot]

Fixed in 884c2d8. Added a yqFlags field to expressionScenario and updated documentInput to include those flags in generated command snippets. The enabled system operator scenarios now set yqFlags: "--enable-system-operator", so the doc shows the correct runnable commands (e.g. yq --enable-system-operator '.country = system(...)' sample.yml).

[Copilot]

SecurityPreferences currently uses a Disable* naming pattern (DisableEnvOps, DisableFileOps), but this introduces EnableSystemOps. For consistency and to reduce confusion when adding future prefs, consider renaming so all fields use the same polarity/pattern (e.g. DisableSystemOps with an inverted default, or rename the existing fields).

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

command is taken from the first matched node’s .Value without validating type/tag. If the command expression yields !!null or a non-scalar node, exec.Command will be invoked with an empty/invalid command (leading to confusing errors like “exec: no command”). Consider validating that the command node is a non-null scalar (and possibly erroring if multiple results are returned) and returning a clearer system operator: command must be a string scalar error.

[mikefarah]

@copilot apply changes based on this feedback If there are multiple results, just use the first (and log a debug, not a warning, of all the matches).

[Copilot]

Done in 53abbba. Extracted a resolveCommandNode helper that validates the command is a non-null scalar (returning "system operator: command must be a string scalar" if not) and logs a Debugf when multiple results are returned before using the first.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The PR description says stdout is "trimmed", but the implementation only strips a single trailing \n/\r\n. If the intent is to trim all trailing whitespace/newlines, use strings.TrimRight/TrimSpace; otherwise update the description/docs to match the current behavior ("strip one trailing newline").

[Copilot]

The doc-generating enabled scenarios embed echoPath from exec.LookPath, which can vary across environments (e.g. /usr/bin/echo vs /bin/echo) and cause non-deterministic generated docs. Consider hardcoding a stable command path/name for scenarios that are included in docs (or marking these scenarios skipDoc: true and adding a separate deterministic doc scenario).

[Copilot]

resolveSystemArgs silently converts non-scalar sequence elements to child.Value (which will be empty for maps/seqs) and also treats any non-null non-sequence node as a single arg. This can produce unintended empty/garbled argv entries (e.g. system("cmd"; {a:1}) yields an empty string arg) instead of failing fast. Consider validating that the args value is either a scalar (non-null) or a sequence of scalar (non-null) nodes, and return a clear error when the type is not supported.

Suggested change

	if argsNode.Kind == SequenceNode {
	args := make([]string, 0, len(argsNode.Content))
	for _, child := range argsNode.Content {
	args = append(args, child.Value)
	}
	return args
	}
	if argsNode.Tag != "!!null" {
	return []string{argsNode.Value}
	}
	return nil
	if argsNode == nil {
	return nil
	}
	if argsNode.Kind == SequenceNode {
	args := make([]string, 0, len(argsNode.Content))
	for _, child := range argsNode.Content {
	// Only non-null scalar children are valid arguments.
	if child == nil {
	continue
	}
	if child.Kind != ScalarNode || child.Tag == "!!null" {
	log.Warningf("system operator: argument must be a non-null scalar; got kind=%v tag=%v - ignoring", child.Kind, child.Tag)
	continue
	}
	args = append(args, child.Value)
	}
	if len(args) == 0 {
	return nil
	}
	return args
	}
	// Single-argument case: only accept a non-null scalar node.
	if argsNode.Tag == "!!null" {
	return nil
	}
	if argsNode.Kind != ScalarNode {
	log.Warningf("system operator: args must be a non-null scalar or sequence of non-null scalars; got kind=%v tag=%v - ignoring", argsNode.Kind, argsNode.Tag)
	return nil
	}
	return []string{argsNode.Value}