ansible

Complex data structures and the Ansible json_query filter

While Ansible is busy fighting its own internal battle not to become a fully fledged programming language, instead remaining as simple and purely declarative as possible, it is still often necessary to work with more complex data structures. This is where the Jinja2 and Ansible filters can really shine.

Recently I stumbled across the Ansible json_query filter as a very neat solution to a problem that would have been otherwise messy to solve in Ansible. The filter is already well-documented, but I thought I would share a few examples of how it came in handy for me.

Example 1 – Finding a specific value in a list of objects

In the first example, I needed to iterate over a list of objects returned from an API query, find an object based on a supplied name, and return the ID of the object. To illustrate what I mean, the API query result looks similar to this:

{
  "json": [
    {
      "id": "91af4658-ed14-4058-bf34-dae98435fbca",
      "name": "example-group-01",
      "path": "/example-group-01",
      "subGroups": []
    },
    ...
  ]
}

Using the Ansible json_query filter, this requirement is easily achieved:

- set_fact:
    group_id: "{{ groups | json_query(query) | first }}"
  vars:
    query: "json[?name=='{{ group_name }}'].id"

Firstly, the query string has been split out into a task variable for both readability and to avoid quote-escaping issues. As for what is actually going on, let’s break it down:

  1. The query is being run against our API results variable, groups, which contains the data structure illustrated above
  2. The query is iterating over all list items under the json attribute
  3. The query filters on objects in the list that have an attribute name that equals the variable group_name. In other words, does result.json[N].name match my requirement
  4. It returns the ID attribute, result.json[N].id, for any matching objects, and appends it to another list
  5. As the group name is unique in this situation, we are passing the whole result through another filter to retrieve the first and only value

Example 2 – Using a complex object in an Ansible loop

In the next example, I had yet another API query result that I wanted to utilise in a loop. While this could normally be achieved using a standard loop, I had added complexity due to the fact that the object was actually the collated result of many API queries run in a loop:

- name: Get user information
  uri:
    url: "{{ api_url }}/users?username={{ item }}"
  register: users
  with_items: "{{ username_list }}"

The returned object structure is somewhat similar to the group example above, but of course since the data is returned in a loop, Ansible appends each result to a results list.

So, to cap off this example, here is the Ansible loop I used to add these users to a group:

- name: Add users to group
  uri:
    url: "{{ api_url }}/users/{{ item }}/groups/{{ group_id }}"
    method: PUT
  with_items: "{{ users | json_query('results[*].json[*].id') }}"

Again, breaking this down:

  1. Taking the users variable above, we are iterating on all items in the results list, using a wildcard
  2. Similarly for each results item, we are iterating on all json items
  3. Finally, we are using the id attribute as our Ansible loop item

Example 3 – Filtering on a list item

Lastly, I ran into a situation where the conditional filter was based on an item existing in a list, rather than an attribute equaling a specific value. In real terms, I wanted to find a VM name based on a known IP address, where each VM may have one or many IP addresses. Here is the rough data structure:

{
  "vms": {
    "json": {
      "resources": [
        {
          "ips": [
            "192.168.1.10",
            "10.0.0.10"
          ],
          "name": "vm-001"
        },
        {
          "ips": [
            "192.168.1.11",
            "10.0.0.11"
          ],
          "name": "vm-002"
        },
        ...
      ]
    }
  }
}

Since the json_query filter is built on JMESPath, there are a large amount of built-in functions available. We are interested in the contains function, which can be used for both strings and lists. Putting this into action we get:

- set_fact:
    my_vm: "{{ vms | json_query(query) }}"
  vars:
    query: "json.resources[?ips.contains(@, '192.168.1.11')].name"

Given our data above, the my_vm variable will have the value [“vm-002”]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.