SNS is a distributed publish-subscribe system. Messages are pushed to subscribers as and when they are sent by publishers to SNS.
SQS is distributed queuing system. Messages are not pushed to receivers. Receivers have to poll or pull messages from SQS. Messages can't be received by multiple receivers at the same time. Any one receiver can receive a message, process and delete it. Other receivers do not receive the same message later. Polling inherently introduces some latency in message delivery in SQS unlike SNS where messages are immediately pushed to subscribers. SNS supports several end points such as email, SMS, HTTP end point and SQS. If you want unknown number and type of subscribers to receive messages, you need SNS.
You don't have to couple SNS and SQS always. You can have SNS send messages to email, SMS or HTTP end point apart from SQS. There are advantages to coupling SNS with SQS. You may not want an external service to make connections to your hosts (a firewall may block all incoming connections to your host from outside).
Your end point may just die because of heavy volume of messages. Email and SMS maybe not your choice of processing messages quickly. By coupling SNS with SQS, you can receive messages at your pace. It allows clients to be offline, tolerant to network and host failures. You also achieve guaranteed delivery. If you configure SNS to send messages to an HTTP end point or email or SMS, several failures to send message may result in messages being dropped.
SQS is mainly used to decouple applications or integrate applications. Messages can be stored in SQS for a short duration of time (maximum 14 days). SNS distributes several copies of messages to several subscribers. For example, let’s say you want to replicate data generated by an application to several storage systems. You could use SNS and send this data to multiple subscribers, each replicating the messages it receives to different storage systems (S3, hard disk on your host, database, etc.).
From aws documentation:
To use message attributes with Amazon SQS endpoints, you must set the subscription attribute, Raw Message Delivery, to True. For more information about raw message delivery, see Appendix: Large Payload and Raw Message Delivery.
https://docs.aws.amazon.com/sns/latest/dg/SNSMessageAttributes.html
https://docs.aws.amazon.com/sns/latest/dg/large-payload-raw-message.html
Added an example from real-life project. Hopefully it helps to clarify things.
The message published to the sns topic is following:
aws sns publish --topic-arn arn:aws:sns:us-west-2:xxx:pollution-event --message '{"operatorId":3375001,"eventTypeId":1,"eventLevelId":1,"validFrom":"2018-03-10T09:00:00Z","validTo":"2018-03-11T09:00:00Z"}' --message-attributes '{"Type" : { "DataType":"String", "StringValue":"Orchestration.Services.Model.Pollution.PollutionMessage"}}'
Enable Raw Delivery is false (default). The message received by sqs contains only content, there are no attributes
{
"Type": "Notification",
"MessageId": "78d5bc6f-142c-5060-a75c-ef29b774ec66",
"TopicArn": "arn:aws:sns:eu-west-2:xxx:pollution-event",
"Message": "{\"validFrom\": \"2018-03-10T09:00:00Z\",\"validTo\": \"2018-03-11T09:00:00Z\",\"eventLevelId\": 1,\"eventTypeId\": 1,\"operatorId\": 3375001}",
"Timestamp": "2018-04-17T11:33:44.770Z",
"SignatureVersion": "1",
"Signature": "xxx==",
"SigningCertURL": "https://sns.eu-west-2.amazonaws.com/SimpleNotificationService-xxx.pem",
"UnsubscribeURL": "https://sns.eu-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:eu-west-2:xxx",
"MessageAttributes": {
"Type": {
"Type": "String",
"Value": "Orchestration.Services.Model.Pollution.PollutionMessage"
},
"AWS.SNS.MOBILE.MPNS.Type": {
"Type": "String",
"Value": "token"
},
"AWS.SNS.MOBILE.MPNS.NotificationClass": {
"Type": "String",
"Value": "realtime"
},
"AWS.SNS.MOBILE.WNS.Type": {
"Type": "String",
"Value": "wns/badge"
}
}
}
Enable Raw delivery is true. The message contains the message attributes and proper content
Best Answer
Did you check the Queue Policy? When you create it, by default it has no policy and only the owner is able to use it.
Check the IAM user you are using with the API and make sure you have the policy on the SQS granting rights to him.
I have used SNS in the past to deliver around 1.5MM notifications each month and I had no problems. But I have no proven track record or experience at the rate you need. But I suggest you to take a look at the following article http://www.quora.com/Push-Notifications/Which-is-best-to-use-Amazon-SNS-Google-Cloud-Messaging-or-Parse-Why which talks a little bit about SNS x GCM.
Hope this helps.