AWS EC2 – Expected Performance Characteristics of NVMe Instance Storage

amazon ec2amazon-web-services

What performance characteristics should I expect from an EC2 instance with NVMe instance storage?
I've switched to c5d.large.
I've created the filesystem. Mounted the device etc.

I'm testing it with hdparm and dd:

ubuntu@ip-172-31-9-169:/mnt$ lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
nvme0n1     259:1    0  200G  0 disk 
└─nvme0n1p1 259:2    0  200G  0 part /
nvme1n1     259:0    0 46.6G  0 disk /mnt/nvme
ubuntu@ip-172-31-9-169:/mnt$ sudo hdparm -Tt /dev/nvme1n1 

/dev/nvme1n1:
 Timing cached reads:   16624 MB in  2.00 seconds = 8331.98 MB/sec
 Timing buffered disk reads: 346 MB in  3.05 seconds = 113.55 MB/sec
ubuntu@ip-172-31-9-169:/mnt$ sudo dd if=/dev/zero of=/mnt/nvme/test bs=1G count=1 oflag=direct
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 25.6933 s, 41.8 MB/s
ubuntu@ip-172-31-9-169:/mnt$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.4 LTS
Release:    16.04
Codename:   xenial
ubuntu@ip-172-31-9-169:/mnt$ uname -a
Linux ip-172-31-9-169 4.4.0-1063-aws #72-Ubuntu SMP Fri Jul 13 07:23:34 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Aren't the values a bit too low for NVMe?
I've found that the kernel might be not optimized so I've installed the linux-aws deb package and restarted.

That didn't improve much, the benchmark results are after the package installation.

Best Answer

I've asked the same question on AWS forum and got a meaningful response.

Turns out that with the smaller instances we don't get the full bandwidth to the NVMe drive.

The bigger the instance the more bandwidth will be at disposal up to the point where multiple NVMe are available above c5d.9xlarge).

The benefit is that we're getting a reduced latency in all cases.

More details here: https://forums.aws.amazon.com/message.jspa?messageID=865656

Related Topic