Using a terraform template to produce a cloud-init yaml: template variables do not respect yaml context/whitespace

[jules]

I am using Terraform. Terraform has a notion https://www.terraform.io/language/functions/templatefile which can be used to take variables that Terraform has access to (locally supplied or retrieved externally) and to apply them to a Jinja2 template. https://learn.hashicorp.com/tutorials/terraform/cloud-init .

People use these templates to generate cloud-init files with Jinja2 templates. Here is a snipped of the template,

ssh_keys:
    ecdsa_private: "${ssh_ecdsa_key_private}"

Then these variables get set in Terraform, like this,

data "template_file" "user_data" {
  template = templatefile(
    "cloud/cloud.cfg",
    {
      ssh_ecdsa_key_private  = file("./ssh/acme-bitbucket-id_ecdsa")
    }
  )

This is however problematic for me presumably because cloud-init is yaml.

  in "", line 22, column 1:
    "
    ^
could not find expected ':'
  in "", line 24, column 8:
          "-----BEGIN OPENSSH PRIVATE KEY-----
           ^"
2022-03-18 00:20:32,611 - util.py[WARNING]: Failed at merging in cloud config part from part-001
2022-03-18 00:20:32,611 - util.py[DEBUG]: Failed at merging in cloud config part from part-001
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/cloudinit/handlers/cloud_config.py", line 140, in handle_part
    self._merge_part(payload, headers)
  File "/usr/lib/python3/dist-packages/cloudinit/handlers/cloud_config.py", line 116, in _merge_part
    (payload_yaml, my_mergers) = self._extract_mergers(payload, headers)
  File "/usr/lib/python3/dist-packages/cloudinit/handlers/cloud_config.py", line 95, in _extract_mergers
    mergers_yaml = mergers.dict_extract_mergers(payload_yaml)
  File "/usr/lib/python3/dist-packages/cloudinit/mergers/__init__.py", line 79, in dict_extract_mergers
    raw_mergers = config.pop('merge_how', None)
AttributeError: 'NoneType' object has no attribute 'pop'
2022-03-18 00:20:32,616 - __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (__end__, None, 3) with frequency once-per-instance

I believe what's happening here is the file isn't being indented properly. Let's say ./ssh/acme-bitbucket-id_ecdsa looked like this,

-----BEGIN OPENSSH PRIVATE KEY-----
FOO
BAR
BAZ
-----END OPENSSH PRIVATE KEY-----

I think that's getting processed into the template like this,

ssh_keys:
    ecdsa_private: -----BEGIN OPENSSH PRIVATE KEY-----
FOO
BAR
BAZ
-----END OPENSSH PRIVATE KEY-----

What is the way to stop properly produce a cloud-init file with Terraform?

[kalena]

Here is how I got around it, I declared these as HDL object,

data "template_file" "user_data" {
  template = templatefile(
    "cloud/cloud.cfg",
    {
      ssh_keys = {
        ssh_ecdsa_key_private  = file("./ssh/acme-bitbucket-id_ecdsa")
        ssh_ecdsa_key_public   = file("./ssh/acme-bitbucket-id_ecdsa.pub")
      }
    }
  )
}

And then I pulled it into the yaml file using https://www.terraform.io/language/functions/yamlencode .

There is a bit of documentation that covers this, over at https://www.terraform.io/language/functions/templatefile#generating-json-or-yaml-from-a-template