Nfs – Autoscaling and NFS Server

amazon ec2amazon-ebsamazon-web-servicesautoscalingnfs

I have a WebServer say WS-1 and a NFS server say NFS-1 setup on AWS. WS-1 is being managed by an elastic load balancer and also autoscaled. It also has an EBS mounted on /var/www which contains all application code.

During autoscaling if another WS-X is launched will the /var/www mounted EBS also cloned and attached to that as well? If not, what are my options besides hosting code on root EBS volume?
Access inside NFS is defined on IP basis like 10.0.0.1/32(rw,…). During autoscaling more instances will be launched, how can i allow them to connect to NFS server and mounted the shared directory? I don't want to give access to private IP subnet using NFS, while on the Security Group level i have given access to NFS server to 0.0.0.0/0. NFS server uses fixed ports like 111, 2049, 4000-4002.

Best Answer

On scaling up, the EBS volume and its data will not be "cloned". To have this behavior you'd want to automate it at boot.

Grab the latest snapshot of WS-1 EBS volume
Create and attach the volume

Another method, depending on how much data is on the EBS, is to pull it down from S3.

With the security group, you can allow any server in the app_security_group to have access to any server in the nfs_server_group. This will allow you to dynamically update the security groups.

Hope that makes sense.

Related Solutions

EC2 Autoscaling AMI and auto attaching new EBD from snapshot

This is possible to a degree with some basic API calls. How to make the API calls is up to you, they can be done from the command line using the tools already in the path (or at least they should already be in there) or by using the various libraries Amazon has build SDK's for, including Java, Ruby, Python, PHP and .NET. The instructions below are targeted at a linux machine using bash scripts, although there are comparable Windows commands for them, I do not know the quirks related to their use.

I use AMI to refer to the Amazon Machine Image, and API call to refer to the command line tools, both those referred to as AMI tools, and API tools, since their naming is unique.

To generate a snapshot from an EBS volume, the volume must be detached from the instance. To implement this in a production environment you will likely want to have a write only slave with some kind of operations log, so you can unmount it, snapshot it, remount the drive, and apply any missing operations to catch it up.
- Unmount the instance from your machine, taking care to deal with any consequences in your application (like not having a database to write to). Normally I use umount -v /dev/sdf /$MOUNT_DIR. Use the -v switch to get some output for error handling.
- Pass the volume-id for the volume to create the snapshot using ec2-create-snapshot. The drive does not need to be unmounted to do this, however it is safer if you do. Certain filesystems, like xfs, can freeze the filesystem, to allow for safe mounted snapshots. Be sure to save the snapshot ID for use later.
- Remount the drive if necessary using mount -v -t [filesystem type] /dev/sdf /$MOUNT_DIR
Bundling a new instance from an already deployed AMI is fairly straightforward. Ideally, your AMI is set up so that all mostly static information (configurations, libraries, installed programs etc) are on the ephemeral storage of the device, and all data (dbs, logs, etc) are stored on the attached drives. This is ideal, because the AMI bundling commands need to lock the FS, but do allow you to exclude certain directories. NOTE: For each of these you will need to provide your Amazon credentials, which spans several things from your account page including your Access Key ID, Secret Access Key, Cert, Private Key, and AWS Account ID.
- Bundle your root file system using ec2-bundle-vol
- Upload the bundle to Amazon using ec2-upload-bundle
- Register the bundle with Amazon so you can launch an instance with it using ec2-register. Keep track of the new AMI ID for use later to relaunch.
- To launch an image, either one you just created or one you would like to reuse, call ec2-run-instances, specify details as needed.
Creating an EBS from a snapshot is easy with an API call.
- Once you have the snapshot ID, just create your volume and specify the ID with ec2-create-volume. Be careful however, the volume must be the same size as the snapshot, if you need to increase the size, mount the larger drive at the same time and copy the data over in your favorite fashion.
To attach and mount the drive, you need to do a combination of API calls on operating system commands.
- Use ec2-create-volume and parse the return of the command for the Volume ID, so that you can call ec2-attach-volume. For the volume information, mount to the later device locations, as earlier ones are often in use for ephemeral storage (the instance itself). On linux, start with /dev/sdf or later. I have use /dev/sdf in this post, but as long as you are consistent, any lettering after f and up to p should work.
- Mount the device in the file system. In linux, I use mount -v -t [File System Type] /dev/sdf /$MOUNT_DIR. The -v switch gives you some output you can parse for error handling.
Again, another API call. I don't have a ton of experience with using this via automated scripts, but it should be as simple as calling elb-register-instances-with-lb.

Be sure to do error handling with the responses from these calls. A few months back I had particular trouble with the ec2-upload-bundle calls timing out and failing to finish uploading the entire image. If you decide to use an SDK, the steps should be mostly the same as all the Amazon API commands are included in the SDKs, albeit with different syntax.

Nfs – Understanding NFS4 (Linux server)

Great questions, highlights a bigger point with the documentation IMO. Here is an attempt at a complete answer:

What does "subsequent exports on that line only" mean?

An example's probably easiest here:

/export/stuff -rw 10.0.0.54 10.0.0.55

is equivalent to:

/export/stuff 10.0.0.54(rw) 10.0.0.55(rw)

Is `fsid=0` not required anymore?

This depends on your use case. It looks like you are exporting regular disk based filesystems from the rest of your query, in that case you are best to drop fsid=0 (which in nfsv4 changes the behavior to reference the root filesystem of the export).

To change this behaviour remove the no_subtree_check option

`rmtab`-related stuff

Is the handling of rmtab a security risk?
I guess it depends on your use case to answer that, on my network it doesn't present a credible information leak but i can see cases where it could potentially.
Won't rmtab be full of stale entries?
Potentially yes, again depending on your deployment scenario / use case.
Is this why some clients running mount -v erroneously see "nothing was mounted"?
I've not come across this yet