I have previously written an article about file manipulation with with Ansible, specifically how to convert a text file to json. I took a few minutes to throw together an example of how to convert a relatively simple file, a list of strings broken up by new lines, and output it as YAML.

Doing file conversions or data munging is an important skill to learn when growing in a DevOps position. While this is a fairly basic example, utilize this as a stepping stone to understanding more complex data structures. Starting with a relatively simple scenario like writing a script to convert a txt file to yaml is a great way to build experience with different data activities. Doing this conversion with Ansible is also a great way to get some baseline experience with a specific DevOps tool's syntax.

Related Articles

Simple Text to YAML Conversion with Ansible

Below is the Ansible code to read in a file name input.txt delimited by newline characters and then do a simple conversion to YAML writing to output.yaml.

- name: Simple Read file and output YAML
  hosts: localhost
  connection: local 
  become: false
  tasks:
  - name: Read Data File
    set_fact:
      data: "{{lookup('ansible.builtin.file', './input.txt').split('\n')}}"

  - name: Show debug data
    ansible.builtin.debug: var=data

  - name: Write output yaml file
    copy:
      dest: output.yaml
      content: "{{ data | to_yaml }}"

Task By Task Breakdown

The first task reads the input.txt file into a data variable that Ansible can use. It also splits the file into a list via the \n (newline) character.

- name: Read Data File
  set_fact:
    data: "{{lookup('ansible.builtin.file', './input.txt').split('\n')}}"

set_fact can be one of your best friends in Ansible. It allows you to take either data from what you have available in your playbook run, or lookup data and set it to a useable var for future tasks. Here I am calling lookup to read an input.txt file and saving it to a data var.

- name: Show debug data
  ansible.builtin.debug: var=data

ansible.builtin.debug lets you output the values of variables to the terminal during a playbook run. This code should not be utilized anywhere except for testing.

- name: Write output yaml file
  copy:
    dest: output.yaml
    content: "{{ data | to_yaml }}"

Finally, utilizing the built-in copy module, you can feed it a variable, like data, and then pipe that to a to_yaml jinja2 function which will then write your file out to dest.

The result of running this playbook will look something like this:

["/site/foo/", "/site/bar/"]

Complex Text to YAML Conversion with Ansible

Below is the Ansible code to read in a file name input.txt delimited by newline characters. After reading the file, a new variable is initialized name data_elements. A loop is then run to create new dictionary objects with a key of ‘name' and a value of each line of input.txt. Finally, a copy is run to write the data to disk under output.yaml.

- name: Complex Read file and output YAML
  hosts: localhost
  connection: local 
  become: false
  tasks:
  - name: Read Data File
    set_fact:
      data: "{{lookup('ansible.builtin.file', './input.txt').split('\n')}}"

  - name: Show debug data
    ansible.builtin.debug: var=data

  - name: Create a new array
    set_fact: 
      data_elements: []

  - name: Loop through lines and add data elements to array
    set_fact:
      data_elements: "{{ data_elements + [{'name': item}] }}"
    loop: "{{ data }}"

  - name: Show debug data
    ansible.builtin.debug: var=data_elements

  - name: Write output yaml file
    copy:
      dest: output.yaml
      content: "{{ {'records': data_elements} | to_yaml }}"

Task By Task Breakdown

Since most of the code here is the same, I will only focus on breaking out the differences.

- name: Create a new array
  set_fact: 
    data_elements: []

The block above sets up the empty data_elements array that we will utilize to map our file lines onto.

- name: Loop through lines and add data elements to array
  set_fact:
    data_elements: "{{ data_elements + [{'name': item}] }}"
  loop: "{{ data }}"

This is setting up a loop to loop over our initial data variable which was read from input.txt. It is appending new dictionaries into a larger array with a key of ‘name' and a value of the item in a loop.

- name: Write output yaml file
  copy:
    dest: output.yaml
    content: "{{ {'records': data_elements} | to_yaml }}"

Finally, we are doing the same write operation as the simple example, but this time we are creating a dictionary with a key of ‘records' and a value of the data_elements array. Basically, you are building a Python dictionary with a key of records and a value of data_elements and passing that to the ansible to_yaml function via a pipe"

This will have an output that looks like this:

records:
- {name: /site/foo/}
- {name: /site/bar/}

Conclusion

As I stated at the beginning of the article, data manipulation in DevOps is a key skill to learn. Hopefully, this helps kickstart someone on their DevOps engineer journey!