Filtering tools

[alexgagnon]

Hi, we're currently in a bit of an internal team debate as to how we should go about migrating from using service-hosted agents to self-hosted agents. We set up a container with the tooling we need for an Azure Pipelines run, however one of the pros of using service-hosted is that the images are continuously patched, saving us from having to constantly manage it. One of the cons is the long build and bloated images because of the large amounts of tools that come installed, many of which we don't need. Is there the possibility of updating the packer script to allow us to filter out which tooling we need? Something along the lines of below:

Current:

build {
  ...
  provisioner "shell" {
    environment_vars = ["HELPER_SCRIPTS=${var.helper_script_folder}", "INSTALLER_SCRIPT_FOLDER=${var.installer_script_folder}", "DEBIAN_FRONTEND=noninteractive"]
    execute_command  = "sudo sh -c '{{ .Vars }} {{ .Path }}'"
    scripts          = [
                        "${path.root}/scripts/installers/azcopy.sh",
                        "${path.root}/scripts/installers/azure-cli.sh",
                        "${path.root}/scripts/installers/azure-devops-cli.sh",
                        ...
                        ]
  }
}

Could this be replaced with the following?

var "tools" {
  type = list(string)
  default = ["azcopy", "azure-cli", "azure-devops-cli", ...]
}

build {
  ...
  provisioner "shell" {
    environment_vars = ["HELPER_SCRIPTS=${var.helper_script_folder}", "INSTALLER_SCRIPT_FOLDER=${var.installer_script_folder}", "DEBIAN_FRONTEND=noninteractive"]
    execute_command  = "sudo sh -c '{{ .Vars }} {{ .Path }}'"
    scripts          = [for tool in var.tools : "${path.root}/scripts/installers/${tool}.sh"]
  }
}

Thanks for all the hard work on this project!

---

EDIT: sorry for the spam, just had an addition to the original idea, you could auto read all the files in the installers directory as the default

var "tools" {
  type = list(string)
  default = null
}

locals {
  installers_path = "${path.root}/script/installers"
  tools = var.tools != null ? var.tools : [ for filepath in fileset(local.installers_path, "*.sh") : split(".", filepath)[0] ]
}

build {
  ...
  provisioner "shell" {
    environment_vars = ["HELPER_SCRIPTS=${var.helper_script_folder}", "INSTALLER_SCRIPT_FOLDER=${var.installer_script_folder}", "DEBIAN_FRONTEND=noninteractive"]
    execute_command  = "sudo sh -c '{{ .Vars }} {{ .Path }}'"
    scripts          = [for tool in locals.tools : "${local.installers_path}/${tool}.sh"]
  }
}

[miketimofeev]

Hi @alexgagnon!
The idea sounds interesting and we had some thoughts regarding tools separation back in the day. We will discuss these changes for sure, but I can't provide any ETA now.

[alexgagnon]

Awesome! If you require any help or a PR, let me know!