I was struggling with this, too, but I found an answer over here https://stackoverflow.com/a/17162973/1750869 that helped resolve this issue for me. Reposting answer below.
You don't have to open permissions to everyone. Use the below Bucket policies on source and destination for copying from a bucket in one account to another using an IAM user
Bucket to Copy from – SourceBucket
Bucket to Copy to – DestinationBucket
Source AWS Account ID - XXXX–XXXX-XXXX
Source IAM User - src–iam-user
The below policy means – the IAM user - XXXX–XXXX-XXXX:src–iam-user has s3:ListBucket and s3:GetObject privileges on SourceBucket/* and s3:ListBucket and s3:PutObject privileges on DestinationBucket/*
On the SourceBucket the policy should be like:
{
"Id": "Policy1357935677554",
"Statement": [
{
"Sid": "Stmt1357935647218",
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::SourceBucket",
"Principal": {"AWS": "arn:aws:iam::XXXXXXXXXXXX:user/src–iam-user"}
},
{
"Sid": "Stmt1357935676138",
"Action": ["s3:GetObject"],
"Effect": "Allow",
"Resource": "arn:aws:s3::: SourceBucket/*",
"Principal": {"AWS": "arn:aws:iam::XXXXXXXXXXXX:user/src–iam-user"}
}
]
}
On the DestinationBucket the policy should be:
{
"Id": "Policy1357935677554",
"Statement": [
{
"Sid": "Stmt1357935647218",
"Action": [
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": "arn:aws:s3::: DestinationBucket",
"Principal": {"AWS": "arn:aws:iam::XXXXXXXXXXXX:user/src–iam-user"}
},
{
"Sid": "Stmt1357935676138",
"Action": ["s3:PutObject"],
"Effect": "Allow",
"Resource": "arn:aws:s3::: DestinationBucket/*",
"Principal": {"AWS": "arn:aws:iam::XXXXXXXXXXXX:user/src–iam-user"}
}
]
}
command to be run is s3cmd cp s3://SourceBucket/File1 s3://DestinationBucket/File1
Yes, via the --query
option.
This option allows you to filter down and return specific elements from the structure. There is a special wildcard [*]
notation that allows you to iterate over lists of objects; we'll use this to iterate over the list of EC2 instances and return your desired values.
Retrieving InstanceID, Instance Tag Name as JSON:
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId, Tags[?Key==`Name`].Value | [0]]'
[
[
[
"i-abcd1234",
"MyFirstInstance"
]
],
[
[
"i-efgh5678",
"MySecondInstance"
]
]
]
Retrieving InstanceID, Instance Tag Name as Text:
If you don't want to work with json, you can also add the --output
option to output text:
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[InstanceId, Tags[?Key==`Name`].Value | [0]]' --output text
i-abcd1234 MyFirstInstance
i-efgh5678 MySecondInstance
Further reading:
Best Answer
Try by double quoting the commands, e.g.:
Here is the example with the loop:
Or the opposite, e.g.