-- KevinCasteels - 2015-06-02

CloudInit yaml merging

The following describes how to merge multiple yaml configuration files together when submitting as user-data to CloudInit

Prerequisites

Mime Multi Part Archives In User Data

Sometimes it is required to provide multiple .yaml as user data to cloudinit during a VM install. If the different .yaml files include the same keywords, for example 'write_files', then by default previous declearations of 'write_files' will be overwritten by the final occurance. To illustrate, say we have two .yaml files which both use the keyword 'run_cmd' as follows:

#cloud-config (1)
run_cmd:
  - bash1
  - bash2

#cloud-config (2)
run_cmd:
  - bash3
  - bash4

This is the default way cloudinit merges these keywords and lists:

#cloud-config (merged)
run_cmd:
  - bash3
  - bash4

By default, the second incidence of the keyword 'run_cmd' has over written the first. In this situation we would prefer the lists be appended, as follows:

#cloud-config (merged)
run_cmd:
  - bash1
  - bash2
  - bash3
  - bash4

To specify the way merging keyword lists is done, cloudinit allows a new keywords to be included in the .yaml files. These keywords are 'merge_how' and 'merge_type', and are searched for by cloudinit in that order. If these keywords are used multiple times in different .yaml files, the first occurance is used and the following ones ignored.

The format for using the mergering keywords is:

classname1(option1,option2)+classname2(option3,option4)....

There appear to be three 'classnames' available, and these are

list()
dict()
str()

These classnames can accpet different options. I have scoured the cloudinit source code and have found the following options:

append
prepend
recurse_array
recurse_list
recurse_str
allow_delete
replace
no_replace

The function of some of these seem obvious, others do not. Unfortunately I have been unable to find documentation describing these anywhere. This page will be updated as we use these classes and options and better understand how they work.

The following are some examples of how 'merge_how' and 'merge_type' can be used:

merge_how: 'dict(recurse_array,no_replace)+list(append)'
merge_how: 'dict(recurse_list)+list(append)'
merge_how: 'dict(recurse_array,no_replace)+list(prepend)'
merge_how: 'dict(allow_delete,no_replace)+list()'
merge_how: 'dict(replace)+list(append)'
merge_how: 'dict(recurse_str)+str(append)'
merge_how: 'dict(recurse_array)+list(recurse_list,recurse_str)+str()'
merge_how: "dict(recurse_array)+list(append)"
merge_how: "dict(allow_delete,recurse_array)+list(recurse_array,append)"
merge_type: 'list(append)+dict(recurse_array)+str()'

For more information see:

http://cloudinit.readthedocs.org/en/latest/topics/merging.html#overview