Add Impersonate Service Account argument by wintermi · Pull Request #2015 · dataform-co/dataform

wintermi · 2025-09-11T06:43:03Z

This PR adds an --impersonate-service-account argument to the run and test commands, along with the required changes to allow for the impersonation of service accounts without the need to change ADC or call gcloud

This would resolve issue #2000 and would be an alternative to solution than PR #2001

Impersonation could then be achieved by executing:

dataform run --impersonate-service-account=<sSERVICE_ACCT_EMAIL>

google-cla · 2025-09-11T06:43:08Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

camilleAmaury · 2025-10-30T10:44:42Z

+1, this would enable to use impersonation in CI rather than giving the rights directly to the CI service account.
There is no way to workaround that currently.

kolina · 2025-11-11T11:28:51Z

/gcbrun

kolina · 2025-11-13T23:33:49Z

Sorry for the late review. A couple of things:

Integration tests are failing, can you take a look at fixing them? Now we have a guide of running them locally
Let's resolve conflicts

kolina · 2025-11-13T23:17:49Z

+        clientConfig.authClient = new Impersonated({
+          sourceClient: authClient,
+          targetPrincipal: this.bigQueryCredentials.impersonateServiceAccount,
+          targetScopes: ['https://www.googleapis.com/auth/cloud-platform']


EXTRA_GOOGLE_SCOPES?

Not sure what you would like done here?

I mean using EXTRA_GOOGLE_SCOPES here instead of hard-coding

kolina · 2026-01-02T21:13:06Z

@wintermi, in the current version tests are failing due to linter checks: output

You can check errors using this lint script.

wintermi · 2026-01-05T13:31:51Z

Resynced the PR with the latest commit

wintermi · 2026-01-05T14:36:31Z

@wintermi, in the current version tests are failing due to linter checks: output

You can check errors using this lint script.

Fixed the linter issues

wintermi · 2026-01-05T14:37:06Z

@kolina ready for retesting, thanks

kolina · 2026-01-07T12:34:01Z

/gcbrun

kolina · 2026-01-07T12:45:04Z

+        clientConfig.authClient = new Impersonated({
+          sourceClient: authClient,
+          targetPrincipal: this.bigQueryCredentials.impersonateServiceAccount,
+          targetScopes: ['https://www.googleapis.com/auth/cloud-platform']


I mean using EXTRA_GOOGLE_SCOPES here instead of hard-coding

…st' commands and the required changes to allow for the impersonation of service accounts without the need to change ADC

wintermi · 2026-04-10T00:59:44Z

I removed the parse-duration dependency to avoid CVE-2025-25283
Moving to the parse-duration@2.1.3 seemed a far bigger change than necessary.

wintermi · 2026-04-10T01:09:29Z

I added a IMPERSONATION_GOOGLE_SCOPES as the EXTRA_GOOGLE_SCOPES only mention drive.

kolina · 2026-04-14T20:08:26Z

    projectId = projectId || this.bigQueryCredentials.projectId;
    if (!this.clients.has(projectId)) {
-      this.clients.set(
+      const clientConfig: any = {


BigQueryOptions instead of any?

kolina · 2026-04-14T20:52:24Z

        projectId,
-        new BigQuery({
+        scopes: EXTRA_GOOGLE_SCOPES,


Are projectId and scopes here used by the auth library if you ovewrite authClient? If they're not used in this case, I'd only set them in else branch below

kolina · 2026-04-14T20:53:56Z

          projectId,
-          scopes: EXTRA_GOOGLE_SCOPES,
-          location: this.bigQueryCredentials.location,
+          scopes: IMPERSONATION_GOOGLE_SCOPES,


Let's add an explaining comment why we're passing different set of scopes with impersonation and without

kolina · 2026-04-14T20:57:28Z

            getCredentialsPath(argv[projectDirOption.name], argv[credentialsOption.name])
          );
+          if (argv[impersonateServiceAccountOption.name]) {
+            (readCredentials as any).impersonateServiceAccount =


Can we extend dataform.IBigQuery with your new option to avoid dynamic casts breaking static typing?

kolina · 2026-04-14T21:04:15Z

  return `${value} ${units[i]}`;
 }

+const DURATION_UNITS_IN_MILLIS: { [unit: string]: number } = {


Can you please elaborate about the effort to avoid it and upgrade parse-duration dependency?

kolina · 2026-04-14T21:09:38Z

 import Long from "long";
 import { PromisePoolExecutor } from "promise-pool-executor";
-
+import { BigQuery, GetTablesResponse, TableField, TableMetadata } from "@google-cloud/bigquery";


It seems to break lint checks

Step #1: ERROR: /workspace/cli/api/dbadapters/bigquery.ts:4:1 - Import sources within a group must be alphabetized. Step #1: ERROR: /workspace/cli/api/dbadapters/bigquery.ts:5:1 - Imports from this module are not allowed in this group. The expected groups (in order) are: external, internal.

kolina · 2026-04-14T21:10:07Z

+    throw new Error("Duration cannot be empty.");
+  }
+
+  if (/^[+-]?\d+(\.\d+)?$/.test(normalizedDuration)) {


Lint checks are failing

Step #1: ERROR: /workspace/cli/util.ts:90:7 - Unsafe Regular Expression Step #1: ERROR: /workspace/cli/util.ts:97:27 - Unsafe Regular Expression

wintermi requested a review from a team as a code owner September 11, 2025 06:43

wintermi requested review from Ceridan and removed request for a team September 11, 2025 06:43

Ceridan requested review from kolina and removed request for Ceridan November 13, 2025 09:15

kolina requested changes Nov 13, 2025

View reviewed changes

ashish10alex mentioned this pull request Dec 23, 2025

[Dataform CLI] Error when using service account impersonation through ADC #2056

Closed

wintermi requested a review from kolina January 7, 2026 05:35

kolina reviewed Jan 7, 2026

View reviewed changes

wintermi added 7 commits April 10, 2026 10:30

Add the '--impersonate-service-account' argument to the 'run' and 'te…

a9071c7

…st' commands and the required changes to allow for the impersonation of service accounts without the need to change ADC

Make changes as requested

469df21

Make changes as requested

7c0ff44

Make changes as requested

233e7d1

Add the '--impersonate-service-account' argument to the 'run' and 'te…

38d7130

…st' commands and the required changes to allow for the impersonation of service accounts without the need to change ADC

Make changes as requested

69a9401

Make changes as requested

21fb05f

wintermi force-pushed the main branch from 763681d to 21fb05f Compare April 10, 2026 00:42

fix(cli): remove parse-duration dependency

3a61218

refactor(cli): dedupe bigquery impersonation scopes

eb2b141

wintermi requested a review from kolina April 10, 2026 01:09

kolina requested changes Apr 14, 2026

View reviewed changes

Conversation

wintermi commented Sep 11, 2025

Uh oh!

google-cla bot commented Sep 11, 2025

Uh oh!

camilleAmaury commented Oct 30, 2025

Uh oh!

kolina commented Nov 11, 2025

Uh oh!

kolina commented Nov 13, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kolina commented Jan 2, 2026

Uh oh!

wintermi commented Jan 5, 2026

Uh oh!

wintermi commented Jan 5, 2026

Uh oh!

wintermi commented Jan 5, 2026

Uh oh!

kolina commented Jan 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wintermi commented Apr 10, 2026

Uh oh!

wintermi commented Apr 10, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants