Using start_from_trigger with context and/or template_fields

[gilt-cl]

Hey all,

We are running Airflow on AWS EKS running CeleryKubernetesExecutor and have ~100K daily tasks and ~1500 daily dag_runs.

In Airflow 2.10.0 the start_from_trigger feature was introduced for deferrable tasks and it's a feature that could greatly improve our performance and scaling of the cluster.

We have a lot of tasks that are deferrable (mainly Redshift and BigQuery queries).
Looking and the docs for this feature, trying to implement it and looking at the feature code, it seems that

1. template_fields are not being rendered before they are passed to the trigger
2. There is no way to pass context to the trigger in order to render the fields that we need to render.

Is there any way to achieve this? is there any future plans to implement fields rendering before the trigger is called?

[jroachgolf84]

Might it make sense to bring this into an issue, rather than a discussion?

[jroachgolf84]

I have a user reporting the same behavior. I'm going to take a peek into the docs and see if this is expected.

[jroachgolf84]

cc: @karenbraganz, are you able to convert this to an issue?

[karenbraganz]

I have created #51245

[AndresElizondo]

FYI, this seems to have been resolved in #55068