Ansible S3 to EC2 – How to Fix Download File Error

amazon ec2amazon s3amazon-web-servicesansible

I'm trying to learn Ansible. I'm working on creating an instance and uploading a file into it, the file i want to put into the ec2 instance is in stored in S3, but it keeps saying that the destination inside the c2 doesn't exist, it does exist though.

This is which is failing, everything else before that including the creation of instance is working fine : 

- name: Deploy war file
        aws_s3:
            bucket: "{{ war_bucket }}"
            object: "{{ war_file }}"
            dest: "{{ war_deploy_path }}/{{ war_file }}"
            mode: get
            overwrite: no
        register: war_downloaded

And this is how i declared my variables : 

war_file: file.war
war_bucket: ansible-bucket
war_deploy_path: /opt/folder/file.war

And this is the error I get :

[Errno 2] No such file or directory: '/opt/folder/file.war.1f1ccA91'

Why is it adding this weird code "1f1cA91" ? Is it causing the problem?

Update : I tried changing the destination from "{{ war_deploy_path }}/{{ war_file }}" to "{{ war_deploy_path }}" but the same problem persists, only the error is [Errno 2] No such file or directory: '/opt/folder.Ac2926c3' now.

Important Update 2 : Ok, so for the sake of testing, i decided to create the same path in my local machine, and to my surprise, this script is actually running it on my local machine instead of the ec2 instance lol, so now, how do i make it run on the ec2 instance xD.

Best Answer

To create a host in ansible and then work on it in the same playbook is possible but requires some on the fly changes to the inventory file and a re-read of the inventory in your playbook.

First, add a place holder to your inventory file like:

[local]
    localhost ansible_connection=local ansible_python_interpreter=python

[new_ones]

Second, in your playbook you're going to need two sections, one to run the local job and the second to run against the host(s) you created from the first section. In the first part, you'll create the hosts and then add the host IP to the inventory you created above. Then you'll tell ansible to re-read the inventory with the meta command and then wait for the host(s) to come up with a pause command. Here's an example:

---

- name: Testing Part One
  hosts: local
  become: yes
  tasks:
    - name: create an ec2 instance
      local_action: 
          module: ec2
          aws_secret_key: <redacted>
          aws_access_key: <redacted>
          group_id: sg-1234567
          key_name: my_key
          instance_type: t2.micro
          image: ami-0123456789abcde
          wait: yes
          count: 1
          vpc_subnet_id: subnet-987654321
          assign_public_ip: no
          region: us-east-1
      register: ec2

# This part adds the IP address of the host that was created above to the
#   inventory file

    - name: Add instance to inventory
      local_action:
          module: lineinfile
          path: inv/hosts_default
          regexp: "{{ item.private_ip }}"
          insertafter: "new_ones"
          line: "{{ item.private_ip }}"
      with_items: '{{ ec2.instances }}'

# Have the playbook reread the inventory file
    - meta: refresh_inventory

# Wait for a bit to ensure SSH is enabled
    - pause:
        minutes: 5

Then you create another entry in the same playbook to copy your file. I don't have pip installed on my hosts by default so added that just in case you're in the same boat:

- name: Testing Part Two
  hosts: new_ones
  become: yes
  tasks:

# Install pip, boto, boto3, and botocore.  You may not need this
    - name: install pip
      easy_install:
        name: pip
        state: latest

    - name: install boto, boto3 and botocore
      pip:
         name: "{{ item }}"
      loop:
        - boto
        - boto3
        - botocore

# Finally we get to what you were trying to do to begin with...

    - name: Deploy war file
      aws_s3:
          aws_secret_key: <redacted>
          aws_access_key: <redacted>
          bucket: "mybucketname"
          object: "blah.txt"
          dest: "/tmp/blah.txt"
          mode: get
          overwrite: no
      register: war_downloaded

Here's the complete playbook in case you're still confused:

---

- name: Testing Part 1
  hosts: local
  become: yes
  tasks:
    - name: create an ec2 instance
      local_action: 
          module: ec2
          aws_secret_key: <redacted>
          aws_access_key: <redacted>
          group_id: sg-1234567
          key_name: my_key
          instance_type: t2.micro
          image: ami-0123456789abcde
          wait: yes
          count: 1
          vpc_subnet_id: subnet-987654321
          assign_public_ip: no
          region: us-east-1
      register: ec2

    - name: Add instance to inventory
      local_action:
          module: lineinfile
          path: inv/hosts_default
          regexp: "{{ item.private_ip }}"
          insertafter: "new_ones"
          line: "{{ item.private_ip }}"
      with_items: '{{ ec2.instances }}'

    - meta: refresh_inventory

    - pause:
        minutes: 5

- name: Testing Part Two
  hosts: new_ones
  become: yes
  tasks:
    - name: install pip
      easy_install:
        name: pip
        state: latest

    - name: install boto, boto3 and botocore
      pip:
         name: "{{ item }}"
      loop:
        - boto
        - boto3
        - botocore

    - name: Deploy war file
      aws_s3:
          aws_secret_key: <redacted>
          aws_access_key: <redacted>
          bucket: "mybucketname"
          object: "blah.txt"
          dest: "/tmp/blah.txt"
          mode: get
          overwrite: no
      register: war_downloaded

References

Ansible Meta Module

P.S. I only tested this building one host, if you build multiples your mileage may vary.

Related Topic