I was looking through Reddit today and saw an interesting question about how to manipulate files with Ansible. I took a few minutes to throw together an example of how to convert a relatively simple file, a list of strings broken up by new lines, and output it as JSON. A similar method can be utilized to read any file into a variable in ansible by skipping the JSON parsing steps later in this tutorial.
Doing file conversions or data munging is an important skill to learn when growing in a DevOps position. While this is a fairly basic example, utilize this as a stepping stone to understanding more complex data structures. Starting with a relatively simple scenario like writing a script to convert a txt file to json is a great way to build experience with different data activities. Doing this conversion with Ansible is also a great way to get some baseline experience with a specific DevOps tool's syntax.
Related Articles
- Convert Text File To YAML With Ansible
- How ChatGPT Can Speed Up DevOps By Writing Ansible
- Unlock The Secrets To Becoming A Successful DevOps Engineer
Simple Text to JSON Conversion with Ansible
Below is the Ansible code to read in a file name input.txt delimited by newline characters and then do a simple conversion to JSON writing to output.json.
- name: Simple Read file and output JSON
hosts: localhost
connection: local
become: false
tasks:
- name: Read Data File
set_fact:
data: "{{lookup('ansible.builtin.file', './input.txt').split('\n')}}"
- name: Show debug data
ansible.builtin.debug: var=data
- name: Write output json file
copy:
dest: output.json
content: "{{ data | to_json }}"
Task By Task Breakdown
The first task reads the input.txt file into a data variable that Ansible can use. It also splits the file into a list via the \n (newline) character.
- name: Read Data File
set_fact:
data: "{{lookup('ansible.builtin.file', './input.txt').split('\n')}}"
set_fact
can be one of your best friends in Ansible. It allows you to take either data from what you have available in your playbook run, or lookup data and set it to a useable var for future tasks. Here I am calling lookup to read an input.txt file and saving it to a data var.
- name: Show debug data
ansible.builtin.debug: var=data
ansible.builtin.debug
lets you output the values of variables to the terminal during a playbook run. This code should not be utilized anywhere except for testing.
- name: Write output json file
copy:
dest: output.json
content: "{{ data | to_json }}"
Finally, utilizing the built-in copy module, you can feed it a variable, like data, and then pipe that to a to_json jinja2 function which will then write your file out to dest.
The result of running this playbook will look something like this:
["/site/foo/", "/site/bar/"]
Complex Text to JSON Conversion with Ansible
Below is the Ansible code to read in a file name input.txt delimited by newline characters. After reading the file, a new variable is initialized name data_elements. A loop is then run to create new dictionary objects with a key of ‘name' and a value of each line of input.txt. Finally, a copy is run to write the data to disk under output.json.
- name: Complex Read file and output JSON
hosts: localhost
connection: local
become: false
tasks:
- name: Read Data File
set_fact:
data: "{{lookup('ansible.builtin.file', './input.txt').split('\n')}}"
- name: Show debug data
ansible.builtin.debug: var=data
- name: Create a new array
set_fact:
data_elements: []
- name: Loop through lines and add data elements to array
set_fact:
data_elements: "{{ data_elements + [{'name': item}] }}"
loop: "{{ data }}"
- name: Show debug data
ansible.builtin.debug: var=data_elements
- name: Write output json file
copy:
dest: output.json
content: "{{ {'records': data_elements} | to_json }}"
Task By Task Breakdown
Since most of the code here is the same, I will only focus on breaking out the differences.
- name: Create a new array
set_fact:
data_elements: []
The block above sets up the empty data_elements array that we will utilize to map our file lines onto.
- name: Loop through lines and add data elements to array
set_fact:
data_elements: "{{ data_elements + [{'name': item}] }}"
loop: "{{ data }}"
This is setting up a loop to loop over our initial data variable which was read from input.txt. It is appending new dictionaries into a larger array with a key of ‘name' and a value of the item in a loop.
- name: Write output json file
copy:
dest: output.json
content: "{{ {'records': data_elements} | to_json }}"
Finally, we are doing the same write operation as the simple example, but this time we are creating a dictionary with a key of ‘records' and a value of the data_elements array. Basically, you are building a Python dictionary with a key of records and a value of data_elements and passing that to the ansible to_json function via a pipe"
This will have an output that looks like this:
{
"records":[
{
"name":"/site/foo/"
},
{
"name":"/site/bar/"
}
]
}
Convert an Ansible List to JSON
Converting any in memory data structure to JSON is pretty simple in Ansible. Below is a code snippet that will help:
- name: Write output json file
copy:
dest: output.json
content: "{{ your_variable | to_json }}"
Assume that your_variable
is a list that contains the data that you would like to convert to JSON. You should be able to do this with most any variable in ansible, though your output results may vary depending on how structured your data is up front. If you have a large text blob in a variable, the to_json
function will not automagically parse the data into something useable. If you have a complex dict or list that you would like to convert to JSON and potentially deserialize into a list again, this method works wonders.
Conclusion
As I stated at the beginning of the article, data manipulation in DevOps is a key skill to learn. Hopefully, this helps kickstart someone on their journey into becoming a DevOps engineer!