Skip to content

Add string slicing support#2639

Open
Copilot wants to merge 5 commits intomasterfrom
copilot/add-string-slice-support
Open

Add string slicing support#2639
Copilot wants to merge 5 commits intomasterfrom
copilot/add-string-slice-support

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 27, 2026

yq supported array slicing (.[1:3]) but not string slicing — .field[4:] on a string value silently returned the wrong result or panicked.

Changes

  • operator_slice.go — Added sliceStringNode (rune-based for Unicode correctness); sliceArrayOperator now routes !!str scalar nodes to it instead of the array path.
  • operator_traverse_path.go — Fixed traverseArrayOperator to evaluate the LHS before calling sliceArrayOperator. Previously it passed the raw document context, so .country[4:] would operate on the whole map rather than the country value.
  • lexer.go — Extended the implicit-0 insertion (for omitted start index in [:N]) to fire on openCollect ([) in addition to traverseArrayCollect (.[), enabling .field[:N] syntax.
  • operator_slice_test.go — Added string slicing scenarios: start-only, end-only, both, negative indices, out-of-bounds clamping, full [:], and Unicode.
  • doc/operators/headers/slice-array.md — Updated header to reflect string support.

Example

# sample.yml
country: Australia
yq '.country[4:]'   sample.yml  # ralia
yq '.country[:5]'   sample.yml  # Austr
yq '.country[-5:]'  sample.yml  # ralia
yq '.country[1:-1]' sample.yml  # ustrali

📱 Kick off Copilot coding agent tasks wherever you are with GitHub Mobile, available on iOS and Android.

Agent-Logs-Url: https://github.com/mikefarah/yq/sessions/a8525fbb-77a7-4bb0-a3a7-b24f99ae8710

Co-authored-by: mikefarah <1151925+mikefarah@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for string slicing in yq Add string slicing support Mar 27, 2026
Copilot AI requested a review from mikefarah March 27, 2026 10:15
```
then
```bash
yq '.country[0:5]' sample.yml
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test says without the second number, but both numbers are present

description: "Slicing strings - without the second number",
subdescription: "Finishes at the end of the string",
document: `country: Australia`,
expression: `.country[0:5]`,
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep this has the second number...should this be .country[5:] ?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot fix this ^;

also the build is failing with:

Error: pkg/yqlib/operator_slice.go:19:98: sliceStringNode - result 1 (error) is always nil (unparam)
func sliceStringNode(lhsNode *CandidateNode, firstNumber int, secondNumber int) (*CandidateNode, error) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 9a9399a:

  • Changed "Slicing strings - without the second number" expression from .country[0:5] to .country[5:] (now actually omits the second number, giving alia)
  • Changed the first "Slicing strings" test to use both numbers: .country[0:5]Austr
  • Removed the always-nil error return from sliceStringNode to fix the build warning

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds correct string-slicing behavior to yq’s slice operator and fixes traversal so slice operations apply to the evaluated LHS value (e.g. .country[4:] slices the country string, not the root document).

Changes:

  • Add rune-based string slicing support and route !!str scalar nodes through it.
  • Fix traverseArrayOperator to evaluate/pipeline the LHS before applying slice logic.
  • Extend lexer post-processing to support omitted-start string slicing syntax like .field[:N].

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
project-words.txt Adds new words used by tests/docs to the project dictionary.
pkg/yqlib/operator_traverse_path.go Ensures slice operations receive the evaluated LHS context.
pkg/yqlib/operator_slice.go Introduces sliceStringNode and routes string scalars through it.
pkg/yqlib/lexer.go Enables implicit 0 insertion for [:N] after [ (not just after .[).
pkg/yqlib/operator_slice_test.go Adds coverage for string slicing (including Unicode and bounds cases).
pkg/yqlib/doc/operators/slice-array.md Updates generated operator docs to include string slicing examples.
pkg/yqlib/doc/operators/headers/slice-array.md Updates the slice operator header to reflect string support.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +77 to +80
relativeFirstNumber := firstNumber
if relativeFirstNumber < 0 {
relativeFirstNumber = len(lhsNode.Content) + firstNumber
}
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Array slicing can still panic when the start index is more negative than the array length (e.g. .[-100:]): relativeFirstNumber is adjusted by len(lhsNode.Content)+firstNumber but is not clamped back to 0, so the loop can index lhsNode.Content with a negative i. Clamp relativeFirstNumber (and similarly relativeSecondNumber) into [0,len] before iterating to avoid runtime panics and match the existing out-of-bounds clamping behavior for other cases.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot apply changes based on this feedback

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 341e252. Array slice now clamps both relativeFirstNumber and relativeSecondNumber into [0, len(Content)] after adjusting for negative indices, preventing a panic on expressions like .[-100:]. Also added test cases for both directions of out-of-bounds negative indices.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@mikefarah mikefarah marked this pull request as ready for review March 28, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants