From b4023b32f4dfd0b626db6c9e6a963b0b41d84add Mon Sep 17 00:00:00 2001 From: Ethan Chiu <17chiue@gmail.com> Date: Fri, 24 Jul 2020 13:08:03 -0400 Subject: [PATCH 1/7] Hotfix: adding support for script execution see: https://github.com/actions/runner/pull/615 --- docs/adrs/0549-composite-run-steps.md | 34 +++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/docs/adrs/0549-composite-run-steps.md b/docs/adrs/0549-composite-run-steps.md index 09e8ed763ae..25785874c3f 100644 --- a/docs/adrs/0549-composite-run-steps.md +++ b/docs/adrs/0549-composite-run-steps.md @@ -63,6 +63,40 @@ echo hello world 4 We add a token called "composite" which allows our Runner code to process composite actions. By invoking "using: composite", our Runner code then processes the "steps" attribute, converts this template code to a list of steps, and finally runs each run step sequentially. If any step fails and there are no `if` conditions defined, the whole composite action job fails. +### Running Local Scripts + +Example 'workflow.yml': +```yaml +jobs: + build: + runs-on: self-hosted + steps: + - uses: user/composite@v1 +``` + +Example `user/composite/action.yml`: + +```yaml +runs: + using: "composite" + steps: + - run: chmod +x ${{env.GITHUB_ACTION_PATH}}/script.sh + - run: chmod +x ${{env.GITHUB_ACTION_PATH}}/test/script2.sh + - run: ${{env.GITHUB_ACTION_PATH}}/script.sh + - run: ${{env.GITHUB_ACTION_PATH}}/test/script2.sh +``` +Where `user/composite` has the file structure: +``` +. ++-- action.yml ++-- script.sh ++-- test +| +-- script2.sh +``` + + +Users will be able to run scripts located in their action folder by first prepending the relative path and script name with `GITHUB_ACTION_PATH` which contains the path in which the composite action is downloaded to and where those "files" live. + ### Inputs Example `workflow.yml`: From c09275850a325bbcb4cd54122196172589a14a42 Mon Sep 17 00:00:00 2001 From: Ethan Chiu <17chiue@gmail.com> Date: Mon, 27 Jul 2020 10:48:17 -0400 Subject: [PATCH 2/7] fix expected behavior --- docs/adrs/0549-composite-run-steps.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/adrs/0549-composite-run-steps.md b/docs/adrs/0549-composite-run-steps.md index 25785874c3f..a174ab43e0c 100644 --- a/docs/adrs/0549-composite-run-steps.md +++ b/docs/adrs/0549-composite-run-steps.md @@ -80,10 +80,10 @@ Example `user/composite/action.yml`: runs: using: "composite" steps: - - run: chmod +x ${{env.GITHUB_ACTION_PATH}}/script.sh - - run: chmod +x ${{env.GITHUB_ACTION_PATH}}/test/script2.sh - - run: ${{env.GITHUB_ACTION_PATH}}/script.sh - - run: ${{env.GITHUB_ACTION_PATH}}/test/script2.sh + - run: chmod +x ${{ github.action_path }}/test/script2.sh + - run: chmod +x $GITHUB_ACTION_PATH/script.sh + - run: ${{ github.action_path }}/test/script2.sh + - run: $GITHUB_ACTION_PATH/script.sh ``` Where `user/composite` has the file structure: ``` @@ -95,7 +95,7 @@ Where `user/composite` has the file structure: ``` -Users will be able to run scripts located in their action folder by first prepending the relative path and script name with `GITHUB_ACTION_PATH` which contains the path in which the composite action is downloaded to and where those "files" live. +Users will be able to run scripts located in their action folder by first prepending the relative path and script name with `$GITHUB_ACTION_PATH` or `github.action_path` which contains the path in which the composite action is downloaded to and where those "files" live. Note, you'll have to use `chmod` before running each script if you do not git check in your script files into your github repo with the executable bit turned on. ### Inputs From d1508fa6867226d63ae33f3db1a2f41db7f24a30 Mon Sep 17 00:00:00 2001 From: Ethan Chiu <17chiue@gmail.com> Date: Mon, 27 Jul 2020 11:53:52 -0400 Subject: [PATCH 3/7] bash shell for each step in example --- docs/adrs/0549-composite-run-steps.md | 70 ++++++++++++++++++--------- 1 file changed, 47 insertions(+), 23 deletions(-) diff --git a/docs/adrs/0549-composite-run-steps.md b/docs/adrs/0549-composite-run-steps.md index a174ab43e0c..c98de223bad 100644 --- a/docs/adrs/0549-composite-run-steps.md +++ b/docs/adrs/0549-composite-run-steps.md @@ -49,7 +49,9 @@ runs: using: "composite" steps: - run: pip install -r requirements.txt + shell: bash - run: npm install + shell: bash ``` Example Output @@ -63,6 +65,31 @@ echo hello world 4 We add a token called "composite" which allows our Runner code to process composite actions. By invoking "using: composite", our Runner code then processes the "steps" attribute, converts this template code to a list of steps, and finally runs each run step sequentially. If any step fails and there are no `if` conditions defined, the whole composite action job fails. +### Defaults + +We will not support "defaults" in a composite action. + +### Shell and Working-directory + +For each run step in a composite action, the action author can set the `shell` and `working-directory` attributes for that step. The shell attribute is **required** for each run step because the action author does not know what the workflow author is using for the operating system so we need to explicitly prevent unknown behavior by making sure that each run step has an explicit shell **set by the action author.** On the other hand, `working-directory` is optional. Moreover, the composite action author can map in values from the `inputs` for it's `shell` and `working-directory` attributes at the step level for an action. + +For example, + +`action.yml` + + +```yaml +inputs: + shell_1: + description: 'Your name' + default: 'pwsh' +steps: + - run: echo 1 + shell: ${{ inputs.shell_1 }} +``` + +Note, the workflow file and action file are treated as separate entities. **So, the workflow `defaults` will never change the `shell` and `working-directory` value in the run steps in a composite action.** Note, `defaults` in a workflow only apply to run steps not "uses" steps (steps that use an action). + ### Running Local Scripts Example 'workflow.yml': @@ -81,9 +108,13 @@ runs: using: "composite" steps: - run: chmod +x ${{ github.action_path }}/test/script2.sh + shell: bash - run: chmod +x $GITHUB_ACTION_PATH/script.sh + shell: bash - run: ${{ github.action_path }}/test/script2.sh + shell: bash - run: $GITHUB_ACTION_PATH/script.sh + shell: bash ``` Where `user/composite` has the file structure: ``` @@ -120,6 +151,7 @@ runs: using: "composite" steps: - run: echo hello ${{ inputs.your_name }} + shell: bash ``` Example Output: @@ -140,6 +172,7 @@ steps: - id: foo uses: user/composite@v1 - run: echo random-number ${{ steps.foo.outputs.random-number }} + shell: bash ``` Example `user/composite/action.yml`: @@ -154,6 +187,7 @@ runs: steps: - id: random-number-generator run: echo "::set-output name=random-number::$(echo $RANDOM)" + shell: bash ``` Example Output: @@ -184,6 +218,10 @@ We'll pass the secrets from the composite action's parents (ex: the workflow fil ### If Condition +** If and needs conditions will not be supported in the composite run steps feature. It will be supported later on in a new feature. ** + +Old reasoning: + Example `workflow.yml`: ```yaml @@ -200,10 +238,14 @@ runs: using: "composite" steps: - run: echo "just succeeding" + shell: bash - run: echo "I will run, as my current scope is succeeding" + shell: bash if: success() - run: exit 1 + shell: bash - run: echo "I will not run, as my current scope is now failing" + shell: bash ``` See the paragraph below for a rudimentary approach (thank you to @cybojenix for the idea, example, and explanation for this approach): @@ -237,11 +279,14 @@ runs: - id: foo1 run: echo test 1 timeout-minutes: 10 + shell: bash - id: foo2 run: echo test 2 + shell: bash - id: foo3 run: echo test 3 timeout-minutes: 10 + shell: bash ``` A composite action in its entirety is a job. You can set both timeout-minutes for the whole composite action or its steps as long as the the sum of the `timeout-minutes` for each composite action step that has the attribute `timeout-minutes` is less than or equals to `timeout-minutes` for the composite action. There is no default timeout-minutes for each composite action step. @@ -277,36 +322,15 @@ runs: steps: - run: exit 1 continue-on-error: true + shell: bash - run: echo "Hello World 2" <----- This step will run + shell: bash ``` If any of the steps fail in the composite action and the `continue-on-error` is set to `false` for the whole composite action step in the workflow file, then the steps below it will run. On the flip side, if `continue-on-error` is set to `true` for the whole composite action step in the workflow file, the next job step will run. For the composite action steps, it follows the same logic as above. In this example, `"Hello World 2"` will be outputted because the previous step has `continue-on-error` set to `true` although that previous step errored. -### Defaults -We will not support "defaults" in a composite action. - -### Shell and Working-directory -For each run step in a composite action, the action author can set the `shell` and `working-directory` attributes for that step. These attributes are optional for each run step - by default, the `shell` is set to whatever default value is associated with the runner os (ex: bash =\> Mac). Moreover, the composite action author can map in values from the `inputs` for it's `shell` and `working-directory` attributes at the step level for an action. - -For example, - -`action.yml` - - -```yaml -inputs: - shell_1: - description: 'Your name' - default: 'pwsh' -steps: - - run: echo 1 - shell: ${{ inputs.shell_1 }} -``` - -Note, the workflow file and action file are treated as separate entities. **So, the workflow `defaults` will never change the `shell` and `working-directory` value in the run steps in a composite action.** Note, `defaults` in a workflow only apply to run steps not "uses" steps (steps that use an action). - ### Visualizing Composite Action in the GitHub Actions UI We want all the composite action's steps to be condensed into the original composite action node. From 5d1ba41993d60dfe4c3f0b9a89b22d1a0aa6c7f1 Mon Sep 17 00:00:00 2001 From: Ethan Chiu <17chiue@gmail.com> Date: Mon, 27 Jul 2020 11:57:10 -0400 Subject: [PATCH 4/7] add what we are not supporting --- docs/adrs/0549-composite-run-steps.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/adrs/0549-composite-run-steps.md b/docs/adrs/0549-composite-run-steps.md index c98de223bad..0dd69351c80 100644 --- a/docs/adrs/0549-composite-run-steps.md +++ b/docs/adrs/0549-composite-run-steps.md @@ -16,7 +16,6 @@ We don't want the workflow author to need to know how the internal workings of t A composite action is treated as **one** individual job step (this is known as encapsulation). - ## Decision **In this ADR, we only support running multiple run steps in an Action.** In doing so, we build in support for mapping and flowing the inputs, outputs, and env variables (ex: All nested steps should have access to its parents' input variables and nested steps can overwrite the input variables). @@ -211,7 +210,7 @@ In the Composite Action, you'll only be able to use `::set-env::` to set environ ### Secrets -**Note** : This feature will be focused on in a future ADR. +**We will not support "Secrets" in a composite action for now. This functionality will be focused on in a future ADR.** We'll pass the secrets from the composite action's parents (ex: the workflow file) to the composite action. Secrets can be created in the composite action with the secrets context. In the actions yaml, we'll automatically mask the secret. @@ -248,6 +247,8 @@ runs: shell: bash ``` +**We will not support "if Condition" in a composite action for now. This functionality will be focused on in a future ADR.** + See the paragraph below for a rudimentary approach (thank you to @cybojenix for the idea, example, and explanation for this approach): The `if` statement in the parent (in the example above, this is the `workflow.yml`) shows whether or not we should run the composite action. So, our composite action will run since the `if` condition for running the composite action is `always()`. @@ -289,6 +290,8 @@ runs: shell: bash ``` +**We will not support "timeout-minutes" in a composite action for now. This functionality will be focused on in a future ADR.** + A composite action in its entirety is a job. You can set both timeout-minutes for the whole composite action or its steps as long as the the sum of the `timeout-minutes` for each composite action step that has the attribute `timeout-minutes` is less than or equals to `timeout-minutes` for the composite action. There is no default timeout-minutes for each composite action step. If the time taken for any of the steps in combination or individually exceed the whole composite action `timeout-minutes` attribute, the whole job will fail (1). If an individual step exceeds its own `timeout-minutes` attribute but the total time that has been used including this step is below the overall composite action `timeout-minutes`, the individual step will fail but the rest of the steps will run based on their own `timeout-minutes` attribute (they will still abide by condition (1) though). @@ -327,6 +330,8 @@ runs: shell: bash ``` +**We will not support "continue-on-error" in a composite action for now. This functionality will be focused on in a future ADR.** + If any of the steps fail in the composite action and the `continue-on-error` is set to `false` for the whole composite action step in the workflow file, then the steps below it will run. On the flip side, if `continue-on-error` is set to `true` for the whole composite action step in the workflow file, the next job step will run. For the composite action steps, it follows the same logic as above. In this example, `"Hello World 2"` will be outputted because the previous step has `continue-on-error` set to `true` although that previous step errored. From a63585dc5925782272391303c9a2267ae261ea92 Mon Sep 17 00:00:00 2001 From: Ethan Chiu <17chiue@gmail.com> Date: Mon, 27 Jul 2020 12:02:43 -0400 Subject: [PATCH 5/7] explicitly define what we are allowing --- docs/adrs/0549-composite-run-steps.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/docs/adrs/0549-composite-run-steps.md b/docs/adrs/0549-composite-run-steps.md index 0dd69351c80..bcbb5967c0b 100644 --- a/docs/adrs/0549-composite-run-steps.md +++ b/docs/adrs/0549-composite-run-steps.md @@ -20,6 +20,28 @@ A composite action is treated as **one** individual job step (this is known as e **In this ADR, we only support running multiple run steps in an Action.** In doing so, we build in support for mapping and flowing the inputs, outputs, and env variables (ex: All nested steps should have access to its parents' input variables and nested steps can overwrite the input variables). +## Composite Run Steps Features +This feature supports at the top action leve: +- name +- description +- inputs +- runs +- outputs + +This feature supports at the run step level: +- name +- id +- run +- env +- shell +- working-directory + +This feature **does not support** at the run step level: +- timeout-minutes +- secrets +- conditionals (needs, if, etc.) +- continue-on-error + ### Steps Example `workflow.yml` From b2e5bbdcfe62f1e55d690275dc1c3dc55aae2237 Mon Sep 17 00:00:00 2001 From: Ethan Chiu <17chiue@gmail.com> Date: Tue, 28 Jul 2020 10:18:09 -0400 Subject: [PATCH 6/7] fix headers + spelling --- docs/adrs/0549-composite-run-steps.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/adrs/0549-composite-run-steps.md b/docs/adrs/0549-composite-run-steps.md index bcbb5967c0b..370e7142529 100644 --- a/docs/adrs/0549-composite-run-steps.md +++ b/docs/adrs/0549-composite-run-steps.md @@ -4,7 +4,7 @@ **Status**: Accepted -## Context +### Context Customers want to be able to compose actions from actions (ex: https://github.com/actions/runner/issues/438) @@ -16,12 +16,12 @@ We don't want the workflow author to need to know how the internal workings of t A composite action is treated as **one** individual job step (this is known as encapsulation). -## Decision +### Decision **In this ADR, we only support running multiple run steps in an Action.** In doing so, we build in support for mapping and flowing the inputs, outputs, and env variables (ex: All nested steps should have access to its parents' input variables and nested steps can overwrite the input variables). -## Composite Run Steps Features -This feature supports at the top action leve: +### Composite Run Steps Features +This feature supports at the top action level: - name - description - inputs @@ -373,6 +373,6 @@ Here is a visual represenation of the [first example](#Steps) ``` -## Consequences +### Consequences This ADR lays the framework for eventually supporting nested Composite Actions within Composite Actions. This ADR allows for users to run multiple run steps within a GitHub Composite Action with the support of inputs, outputs, environment, and context for use in any steps as well as the if, timeout-minutes, and the continue-on-error attributes for each Composite Action step. From 6e3c46c6b2d3142a321a30399cfd581fa7c35493 Mon Sep 17 00:00:00 2001 From: Ethan Chiu <17chiue@gmail.com> Date: Mon, 10 Aug 2020 11:28:06 -0400 Subject: [PATCH 7/7] h2 --- docs/adrs/0549-composite-run-steps.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/adrs/0549-composite-run-steps.md b/docs/adrs/0549-composite-run-steps.md index 370e7142529..023ba7d457f 100644 --- a/docs/adrs/0549-composite-run-steps.md +++ b/docs/adrs/0549-composite-run-steps.md @@ -4,7 +4,7 @@ **Status**: Accepted -### Context +## Context Customers want to be able to compose actions from actions (ex: https://github.com/actions/runner/issues/438) @@ -16,7 +16,7 @@ We don't want the workflow author to need to know how the internal workings of t A composite action is treated as **one** individual job step (this is known as encapsulation). -### Decision +## Decision **In this ADR, we only support running multiple run steps in an Action.** In doing so, we build in support for mapping and flowing the inputs, outputs, and env variables (ex: All nested steps should have access to its parents' input variables and nested steps can overwrite the input variables). @@ -373,6 +373,6 @@ Here is a visual represenation of the [first example](#Steps) ``` -### Consequences +## Consequences This ADR lays the framework for eventually supporting nested Composite Actions within Composite Actions. This ADR allows for users to run multiple run steps within a GitHub Composite Action with the support of inputs, outputs, environment, and context for use in any steps as well as the if, timeout-minutes, and the continue-on-error attributes for each Composite Action step.