Ansible – How to Merge Two Files into a Single File on a Remote Server

ansibleansible-playbookcopyjinja2python

I need to merge two files without duplicate entries in it. is there any way i can achieve it through ansible modules.
Ex i have two files /etc/hosts1 and /etc/hosts2. I need to have one /etc/hosts file with all entries present in both /etc/hosts1 and /etc/hosts2 without duplicate entries.
How can i achieve this.
An example would be appreciated

- name: Merge two files
  assemble:
    src: /etc/hosts1
    dest: /etc/hosts2

The above assemble module fails

Best Answer

This works. It reads the contents of all files and reduces the resulting array of lines to unique values. Then a new file with those lines is created.

- hosts: localhost
  gather_facts: no
  vars:
    hostsfiles:
      - /tmp/hosts1
      - /tmp/hosts2
  tasks:
  - name: read files
    command: awk 1 {{ hostsfiles | join(' ') }}
    register: hosts_contents
  - name: create hosts file
    copy:
      dest: /tmp/hosts
      content: "{{ hosts_contents.stdout_lines | unique |join('\n') }}"

I'm using awk 1 instead of cat to add potentially missing line breaks to the end of the source files.

Related Solutions

Use file from host as input for lineinfile module

As Mxx pointed out in the comments, lookup is the answer. Furthermore, I would not recommend using the lineinfile for adding authorized_keys. There is already an Ansible module specifically for that, and it's a lot less likely to get you into trouble. Here's an example play. It will grab from the local public key, and properly make sure it exists for the target user.

tasks: - name: Install SSH authorized key authorized_key: > user=root key="{{ lookup('file', '/root/.ssh/id_rsa.pub') }}" state=present

How to clean up stuff from roles that aren’t used any more on a server

With Ansible, you wouldn't really do this any differently from how you would do it with Puppet.

In your example where you would set

class { 'nginx':
  ensure => absent,
}

you are relying on the author of that puppet module having written the necessary code to deal with removing everything. Not every puppet module has this.

Similarly, with Ansible, you might have roles that have both the necessary steps to install it as well as to remove it. The difference is only in how to invoke the two.

One approach could be one where the role in question exposes a variable to toggle the behaviour. For example, that nginx role might take a variable nginx_state which takes the values installed and absent.

In the role's tasks/main.yml, the role author might have something along the lines of..

- include: install.yml
  when: nginx_state|default('present') == "present"

- include: uninstall.yml
  when: nginx_state|default('present') == "absent"

..with the respective install/uninstall logic being split up between those two conditionally-included files.

Ansible roles can also be nested. As another way to do the same, a role author might for example provide a role nginx with another role inside of it, called uninstalled. You could then do:

- name: Uninstall nginx
  hosts: some_group
  roles:
    - nginx/uninstalled

Ansible, when compared to Puppet, arguably has fewer rules and guidelines about how things should be done so practices vary a bit more out in the wild, but the same concepts apply.

Best Answer

Related Solutions

Use file from host as input for lineinfile module

How to clean up stuff from roles that aren’t used any more on a server

Related Topic