This post will help anyone who is attempting to use Ansible to create new VMs with automation when Practicing DevOps. I use this in my home lab to help stay current with new trends in DevOps. Not everyone is going to be fortunate enough to have cloud playgrounds to test out new things, and home labs are a great way to make a one-time investment in your future.

The code for the project can be found here!

Related Articles

Prerequisites

This article is going to assume that you already have VMWare running in a home lab somewhere. This also assumes that you have some familiarity with VMware and networking.

This post will also assume that you have already installed a baseline OS. Packer is a good tool to use to automate your baseline operating system installation to create a reusable disk image.

The Playbook

Below is the full code to deploy a virtual machine from a disk image.

The general idea is to run ansible-playbook create-vmware-esxi-vm.yaml -i inventory.yaml from your command line. The playbook will then check to see if a machine exists. If set to destructive, it will then delete any machine with the same name, copy an existing disk image, set some machine specifics like IP address, and then power on your machine.

VMWare vSphere makes this a fairly novel activity via the use of templates. A standalone ESXi node does not have the benefit of templates which is why things get a bit more complex in this space.

A majority of this playbook is also setup with delegate_to: localhost. This means that the computer running the ansible script will also execute anything with the defined delegate_to: localhost. The remote machine from inventory does not yet exist, which means; if delegate_to: localhost was not set, you would immediately receive an ansible connection error.


---
- name: Create VMWare ESXi VM
  hosts: all
  gather_facts: false
  vars:
    destructive: false
  tasks:
    - include_vars: config.yaml
      delegate_to: localhost

    - include_vars: creds.yaml # Remember, this should be in a vault
      delegate_to: localhost   # These creds are for example use only

    - name: Check if virtual machine exists
      community.vmware.vmware_guest_info:
        validate_certs: false
        hostname: "{{ vmware_host }}"
        username: "{{ vmware_user }}"
        password: "{{ vmware_password }}"
        datacenter: "ha-datacenter"
        name: "{{ vmware.name }}"
      delegate_to: localhost
      ignore_errors: true
      throttle: 1
      register: vmware_validate_guest
      when: destructive is not defined or not destructive

    - set_fact:
        destructive: yes
      when: |
        vmware_validate_guest is defined and (
          (vmware_validate_guest.skipped is defined and not vmware_validate_guest.skipped) 
          or (vmware_validate_guest.failed is defined and vmware_validate_guest.failed == True)
        )

    - name: Add vmware as a host
      add_host:
        hostname: "{{ vmware_host }}"
        ansible_ssh_host: "{{ vmware_host }}"
        ansible_user: "{{ vmware_user }}"
        ansible_ssh_pass: "{{ vmware_password }}"
      when: destructive

    - set_fact:
        num_cpu_cores_per_socket: "{{ vmware.num_cpus | int / 2 }}"

    - name: Remove virtual machine from inventory
      community.vmware.vmware_guest:
        validate_certs: false
        hostname: "{{ vmware_host }}"
        username: "{{ vmware_user }}"
        password: "{{ vmware_password }}"
        name: "{{ vmware.name }}"
        state: absent
        force: True
      delegate_to: localhost
      throttle: 1
      when: destructive

    - name: Delete VM Folders
      ansible.builtin.file:
        path: "{{storage_path}}/{{ vmware.name }}/"
        state: absent
      delegate_to: "{{ vmware_host }}"
      when: destructive
      throttle: 5

    - name: Create a virtual machine on vmware
      community.vmware.vmware_guest:
        validate_certs: false
        hostname: "{{ vmware_host }}"
        username: "{{ vmware_user }}"
        password: "{{ vmware_password }}"
        folder: /vm/
        name: "{{ vmware.name }}"
        state: poweredoff
        guest_id: "{{ vmware.guest_id }}"    
        disk: "{{ vmware.disks }}"
        datastore: "{{ datastore_name }}"
        hardware:
          boot_firmware: efi
          memory_mb: "{{ vmware.memory_mb }}"
          num_cpus: "{{ vmware.num_cpus }}"
          num_cpu_cores_per_socket: "{{ num_cpu_cores_per_socket | int }}"
          scsi: paravirtual
        networks:
        - name: "{{ vmware.port_group }}"
          connect: yes
          mac: "{{ vmware.mac_address  }}"
          ip: "{{ vmware.ip_address }}"
          netmask: "{{ vmware.netmask }}"
          device_type: vmxnet3
          type: static
      delegate_to: localhost
      register: deploy_vm
      throttle: 5
      when: destructive

    - name: Copy boot disk VMDK Files
      ansible.builtin.command: "{{ item }}"
      delegate_to: "{{ vmware_host }}"
      loop:
        - vmkfstools -i "{{storage_path}}/{{ image_name }}/{{ image_name }}.vmdk" "{{storage_path}}/{{ vmware.name }}/{{ image_name }}.vmdk" -d thin
      when: destructive
      throttle: 3

    - name: Add boot disks to virtual machine
      community.vmware.vmware_guest_disk:
        validate_certs: false
        hostname: "{{ vmware_host }}"
        username: "{{ vmware_user }}"
        password: "{{ vmware_password }}"
        name: "{{ vmware.name }}"
        datacenter: ha-datacenter
        disk:
          - filename: "[{{datastore_name}}] {{ vmware.name }}/{{ image_name }}.vmdk"
            scsi_type: 'paravirtual'
            scsi_controller: 0
            unit_number: 1
      delegate_to: localhost
      register: disk_facts
      when: destructive
      throttle: 5

    - name: Power on virtual machine
      community.vmware.vmware_guest:
        validate_certs: false
        hostname: "{{ vmware_host }}"
        username: "{{ vmware_user }}"
        password: "{{ vmware_password }}"
        name: "{{ vmware.name }}"
        state: poweredon
        wait_for_ip_address: yes
      delegate_to: localhost
      register: deploy_vm
      when: destructive

A Task By Task Breakdown

First, we should look at the inventory.

This example has only one example host defined in it under a group called example. You are able to define as many hosts in inventory that you would like.

The structure of the vars defined under example-01.valewood.lab are there to provide everything the playbook needs in order to provision a server in ESXi.


---
all:
  hosts:
  children:
    example:
      hosts:
        example-01.valewood.lab:
          ip: 192.168.1.10 
          ansible_host: 192.168.1.10
          vmware:
            ip_address: 192.168.1.10
            gateway_address: 192.168.1.1
            dns_address: 192.168.1.1
            netmask: 255.255.255.0
            mac_address: "00:16:3E:A0:06:43"
            memory_mb: 8192
            name: example-01
            num_cpus: 4
            port_group: Example-App
            guest_id: centos64Guest
            disks: []

Below is the code that reads in configurations and credentials. These are broken up into 2 different files because credentials should always be encrypted and never stored in plaintext.


- include_vars: config.yaml
  delegate_to: localhost

- include_vars: creds.yaml # Remember, this should be in a vault
  delegate_to: localhost   # These creds are for example use only

The next step is to check to see if a VM exists in VMWare or not with the same name. This is a crucial step to ensure that duplicate names are not used in VMWare. Here are some key notes about additional configurations:

  • validate_certs is set to false because most of us are not going to be using trusted 3rd party certs in our VMware environment. If you are doing this in a professional environment, ensure you have valid certs installed and leave validate_certs set to the default of true.
  • validate_certs should also be set to false if you are connecting to your VMWare server via IP address since your cert will not match an IP address.
  • vmware_user and vmware_password should be set to a user that has SSH access to your VMWare server. I would not recommend that you do this in any sort of production environment, but it is perfectly fine for a home lab.
  • datacenter is set to ha-datacenter. This is the default name of a "datacenter" in ESXi. There is no reason to change this.
  • throttle is set to 1 to provide time to ctrl+c IF you happen to run your playbook by mistake against many machines at once.
  • destructive is a variable that can be set which tells this playbook to not destroy and rebuild machines if they already exist. By default, the destructive variable it is not defined which means a check will be performed.

- name: Check if virtual machine exists
  community.vmware.vmware_guest_info:
    validate_certs: false 
    hostname: "{{ vmware_host }}"
    username: "{{ vmware_user }}"
    password: "{{ vmware_password }}"
    datacenter: "ha-datacenter"
    name: "{{ vmware.name }}"
  delegate_to: localhost
  ignore_errors: true
  throttle: 1
  register: vmware_validate_guest
  when: destructive is not defined or not destructive

The next block of code will set the destructive variable to true IF we did the validation check from the previous block. The validation check in the previous block will not run if destructive is set to true. This acts as a one-way gate.


- set_fact:
    destructive: yes
  when: |
    vmware_validate_guest is defined and (
      (vmware_validate_guest.skipped is defined and not vmware_validate_guest.skipped) 
      or (vmware_validate_guest.failed is defined and vmware_validate_guest.failed == True)
    )

Things get a little bit tricky here, but I will explain the rationale behind this next block. Your inventory file only has virtual machines defined in it, but for ansible remote connections to work properly, a host must be in inventory. To get around this, in the middle of this playbook run, we call add_host which has an explicit delegate_to statement so that it runs on localhost.

The reason this is tricky is because of the way that ansible does variable expansion. One would think that a single host is being added to the inventory, but what is really happening in a playbook? Ansible is expanding variable scope out to each individual machine in the inventory. This means that add_host is actually being added in the context of each host in the inventory. If you have 10 hosts defined in inventory, add_host adds variables for your VMWare host for each inventory host.

TLDR; each host in inventory now has the ability to utilize a delegate_to: {{ vmware_host }} statement as part of their tasks.


- name: Add vmware as a host
  add_host:
    hostname: "{{ vmware_host }}"
    ansible_ssh_host: "{{ vmware_host }}"
    ansible_user: "{{ vmware_user }}"
    ansible_ssh_pass: "{{ vmware_password }}"
  when: destructive

Next, we need to do a little bit of math. This could be done live through future executions, but I like to split things like this out into separate set_fact tasks to ensure I can easily run any debug tasks in the future.

Basically, this accomplishes telling VMWare that I want my CPU cores evenly distributed across sockets. This is not perfect because it does not take care of edge cases of 1 CPU or 3 CPUs being defined in inventory. For my use cases, I never define anything that way, but this could easily be updated to accommodate.


- set_fact:
    num_cpu_cores_per_socket: "{{ vmware.num_cpus | int / 2 }}"

Next, if we are destructive, we need to delete an existing VM. I am not going to rehash any of the special variables from above. The only relevant thing to notice here is the state being set to absent which will delete the machine, and force is set to true which will bypass any safety checks VMWare might do.


- name: Remove virtual machine from inventory
  community.vmware.vmware_guest:
    validate_certs: false
    hostname: "{{ vmware_host }}"
    username: "{{ vmware_user }}"
    password: "{{ vmware_password }}"
    name: "{{ vmware.name }}"
    state: absent
    force: True
  delegate_to: localhost
  throttle: 1
  when: destructive

Next is a little bit of cleanliness. The delete statement above will not actually cleanup the VMWare datastore filesystem. We are performing a delegate_to: {{ vmware_host }} which means we will be running a direct delete over SSH to your server running vmware.


- name: Delete VM Folders
  ansible.builtin.file:
    path: "{{storage_path}}/{{ vmware.name }}/"
    state: absent
  delegate_to: "{{ vmware_host }}"
  when: destructive
  throttle: 5

Finally, the meat of the playbook. This task creates the VM which is defined in the inventory. I will break down the relevant variables below.

  • folder is neccisary for vShpere. When only running ESXi, this only requires a default value of /vm/. Most home labs will not need to change this.
  • state is set to poweredoff. You could set this to poweredon but leaving it poweredoff ensures you can do post-processing after this create task.

Please note, Create a virtual machine on vmware is not referencing any kind of existing disk image. This will create a new completely blank VM. Subsequent tasks will utilize your disk image to ensure it is properly attached to this machine.


- name: Create a virtual machine on vmware
  community.vmware.vmware_guest:
    validate_certs: false
    hostname: "{{ vmware_host }}"
    username: "{{ vmware_user }}"
    password: "{{ vmware_password }}"
    folder: /vm/
    name: "{{ vmware.name }}"
    state: poweredoff
    guest_id: "{{ vmware.guest_id }}"    
    disk: "{{ vmware.disks }}"
    datastore: "{{ datastore_name }}"
    hardware:
      boot_firmware: efi
      memory_mb: "{{ vmware.memory_mb }}"
      num_cpus: "{{ vmware.num_cpus }}"
      num_cpu_cores_per_socket: "{{ num_cpu_cores_per_socket | int }}"
      scsi: paravirtual
    networks:
    - name: "{{ vmware.port_group }}"
      connect: yes
      mac: "{{ vmware.mac_address  }}"
      ip: "{{ vmware.ip_address }}"
      netmask: "{{ vmware.netmask }}"
      device_type: vmxnet3
      type: static
  delegate_to: localhost
  register: deploy_vm
  throttle: 5
  when: destructive

Next, perform a copy of an existing disk image into the folder that was just created for this new virtual machine. This will essentially perform the same action that vSphere performs when cloning templates.

The key here is to utilize vmkfstools over ssh to ensure that you are able to get a thin provisioned copied disk. If this is performed directly via ansible modules, you are not able to get a thin disk without vSphere.


- name: Copy boot disk VMDK Files
  ansible.builtin.command: "{{ item }}"
  delegate_to: "{{ vmware_host }}"
  loop:
    - vmkfstools -i "{{storage_path}}/{{ image_name }}/{{ image_name }}.vmdk" "{{storage_path}}/{{ vmware.name }}/{{ image_name }}.vmdk" -d thin
  when: destructive
  throttle: 3

Next, we need to tell VMWare to mount our cloned disk to the recently created VM. The new VM will have a scsi_controller so we just need to give it the right controller number of 0, and unit_number 1 to ensure we do not overwrite the new disk created via Create a virtual machine on vmware.


- name: Add boot disks to virtual machine
  community.vmware.vmware_guest_disk:
    validate_certs: false
    hostname: "{{ vmware_host }}"
    username: "{{ vmware_user }}"
    password: "{{ vmware_password }}"
    name: "{{ vmware.name }}"
    datacenter: ha-datacenter
    disk:
      - filename: "[{{datastore_name}}] {{ vmware.name }}/{{ image_name }}.vmdk"
        scsi_type: 'paravirtual'
        scsi_controller: 0
        unit_number: 1
  delegate_to: localhost
  register: disk_facts
  when: destructive
  throttle: 5

And finally we tell ESXi to power on the virtual machine.


- name: Power on virtual machine
  community.vmware.vmware_guest:
    validate_certs: false
    hostname: "{{ vmware_host }}"
    username: "{{ vmware_user }}"
    password: "{{ vmware_password }}"
    name: "{{ vmware.name }}"
    state: poweredon
    wait_for_ip_address: yes
  delegate_to: localhost
  register: deploy_vm
  when: destructive

Conclusion

As you can see, with a fairly concise set of steps, you can level up your DevOps automation game by rapidly creating and tearing down virtual machines in your home lab.