Azure Pipeline Parameters

teambuild
690

In a previous post, I did a deep dive into Azure Pipeline variables. That post turned out to be longer than I anticipated, so I left off the topic of parameters until this post.

Type: Any

If we look at the YML schema for variables and parameters, we’ll see this definition:

variables: { string: string }

parameters: { string: any }

Parameters are essentially the same as variables, with the following important differences:

  • Parameters are dereferenced using “${{}}” notation
  • Parameters can be complex objects
  • Parameters are expanded at queue time, not at run time
  • Parameters can only be used in templates (you cannot pass parameters to a pipeline, only variables)

Parameters allow us to do interesting things that we cannot do with variables, like if statements and loops. Before we dive in to some examples, let’s consider variable dereferencing.

Variable Dereferencing

The official documentation specifies three methods of dereferencing variables: macros, template expressions and runtime expressions:

  • Macros: this is the “$(var)” style of dereferencing
  • Template parameters use the syntax “${{ parameter.name }}”
  • Runtime expressions, which have the format “$[variables.var]”

In practice, the main thing to bear in mind is when the value is injected. “$()” variables are expanded at runtime, while “${{}}” parameters are expanded at compile time. Knowing this rule can save you some headaches.

The other notable difference is left vs right side: variables can only expand on the right side, while parameters can expand on left or right side. For example:

# valid syntax
key: $(value)
key: $[variables.value]
${{ parameters.key }} : ${{ parameters.value }}

# invalid syntax
$(key): value
$[variables.key]: value

Here's a real-life example from a TailWind Traders I created. In this case, the repo contains several microservices that are deployed as Kubernetes services using Helm charts. Even though the code for each microservice is different, the deployment for each is identical, except for the path to the Helm chart and the image repository.

Thinking about this scenario, I wanted a template for deployment steps that I could parameterize. Rather than copy the entire template, I used a “for” expression to iterate over a map of complex properties. For each service deployment, I wanted:

  • serviceName: The path to the service Helm chart
  • serviceShortName: Required because the deployment requires two steps: “bake” the manifest, and then “deploy” the baked manifest. The “deploy” task references the output of the “bake” step, so I needed a name that wouldn't collide as I expanded it multiple times in the “for” loop

Here's a snippet of the template steps:

# templates/step-deploy-container-service.yml
parameters:
  serviceName: ''  # product-api
  serviceShortName: '' # productapi
  environment: dev
  imageRepo: ''  # product.api
  ...
  services: []

steps:
- ${{ each s in parameters.services }}:
  - ${{ if eq(s.skip, 'false') }}:
    - task: KubernetesManifest@0
      displayName: Bake ${{ s.serviceName }} manifest
      name: bake_${{ s.serviceShortName }}
      inputs:
        action: bake
        renderType: helm2
        releaseName: ${{ s.serviceName }}-${{ parameters.environment }}
        ...
    - task: KubernetesManifest@0
      displayName: Deploy ${{ s.serviceName }} to k8s
      inputs:
        manifests: $(bake_${{ s.serviceShortName }}.manifestsBundle)
        imagePullSecrets: $(imagePullSecret)

Here's a snippet of the pipeline that references the template:

...
  - template: templates/step-deploy-container-service.yml
    parameters:
      acrName: $(acrName)
      environment: dev
      ingressHost: $(IngressHost)
      tag: $(tag)
      autoscale: $(autoscale)
      services:
      - serviceName: 'products-api'
        serviceShortName: productsapi
        imageRepo: 'product.api'
        skip: false
      - serviceName: 'coupons-api'
        serviceShortName: couponsapi
        imageRepo: 'coupon.api'
        skip: false
      ...
      - serviceName: 'rewards-registration-api'
        serviceShortName: rewardsregistrationapi
        imageRepo: 'rewards.registration.api'
        skip: true

In this case, “services” could not have been a variable since variables can only have “string” values. Hence I had to make it a parameter.

Parameters and Expressions

There are a number of expressions that allow us to create more complex scenarios, especially in conjunction with parameters. The example above uses both the “each” and the “if” expressions, along with the boolean function “eq”. Expressions can be used to loop over steps or ignore steps (as an equivalent of setting the “condition” property to “false”). Let's look at an example in a bit more detail. Imagine you have this template:

# templates/steps.yml
parameters:
  services: []

steps:
- ${{ each s in parameters.services }}:
  - ${{ if eq(s.skip, 'false') }}:
    - script: echo 'Deploying ${{ s.name }}'

Then if you specify the following pipeline:

jobs:
- job: deploy
  - steps: templates/steps.yml
    parameters:
      services:
      - name: foo
        skip: false
      - name: bar
        skip: true
      - name: baz
        skip: false


you should get the following output from the steps:

Deploying foo
Deploying baz

Parameters can also be used to inject steps. Imagine you have a set of steps that you want to repeat with different parameters - except that in some cases, a slightly different middle step needs to be executed. You can create a template that has a parameter called “middleSteps” where you can pass in the middle step(s) as a parameter!

# templates/steps.yml
parameters:
  environment: ''
  middleSteps: []

steps:
- script: echo 'Prestep'
- ${{ parameters.middleSteps }}
- script: echo 'Post-step'

# pipelineA
jobs:
- job: A
  - steps: templates/steps.yml
    parameters:
      middleSteps:
      - script: echo 'middle A step 1'
      - script: echo 'middle A step 2'

# pipelineB
jobs:
- job: B
  - steps: templates/steps.yml
    parameters:
      middleSteps:
      - script: echo 'This is job B middle step 1'
      - task: ...  # some other task
      - task: ...  # some other task

For a real world example of this, see this template file. This is a demo where I have two scenarios for machine learning: a manual training process and an AutoML training process. The pre-training and post-training steps are the same, but the training steps are different: the template reflects this scenario by allowing me to pass in different “TrainingSteps” for each scenario.

Extends Templates

Passing steps as parameters allows us to create what Azure DevOps calls “extends templates”. These provide rails around what portions of a pipeline can be customized, allowing template authors to inject (or remove) steps. The following example from the documentation demonstrates this:

# template.yml
parameters:
- name: usersteps
  type: stepList
  default: []
steps:
- ${{ each step in parameters.usersteps }}:
  - ${{ each pair in step }}:
    ${{ if ne(pair.key, 'script') }}:
      ${{ pair.key }}: ${{ pair.value }}

# azure-pipelines.yml
extends:
  template: template.yml
  parameters:
    usersteps:
    - task: MyTask@1
    - script: echo This step will be stripped out and not run!
    - task: MyOtherTask@2

Conclusion

Parameters allow us to pass and manipulate complex objects, which we are unable to do using variables. They can be combined with expressions to create complex control flow. Finally, parameters allow us to control how a template is customized using extends templates.

Happy parameterizing!