Nfs – iSCSI or NFS for Oracle VM shared storage

iscsinfsoraclevmxen

I'm configuring a 4 node Oracle VM environment. One management server, two vm servers and a storage server with direct attached storage.

The storage server is running Linux and I thus have the option of providing the shared storage for the VM servers on iSCSI (with OCFS2) or NFS.

Is it likely that one would perform better or more reliably than the other?
Any other things one should take into consideration when deciding between the two?

Note that Oracle VM is the Xen based virtualization product from Oracle, not a generic term for virtualized Oracle databases.

Best Answer

All other things being equal, I would expect a block-level protocol (iSCSI) to beat out a file-level protocol (NFS) performance-wise.

An NFS storage area (file system) can be shared simultaneously by more than one host, the storage server's OS would be responsible for arbitrating access to files. You could more easily share a pool of storage for both servers this way. For hosting Oracle VM data, this may be a fairly unimpressive benefit.

iSCSI has the advantage of making the storage available as a block device. This is likely to be more similar to your production environment. The drawback is that only one host can attach to each iSCSI target at a time.

I do not know that one would be significantly more reliable than another, but I have a preference for iSCSI.

Your backup strategy may vary somewhat depending on which solution you go with.

Related Solutions

Nfs – What are “good” speeds for iSCSI and NFS over 1GB ethernet

We've pushed a Sun X4100 writing over bonded GigE and iSCSI to an Sun X4500 to 280MB/s.

There's a lot that can be done to tune the TCP stack on Solaris to help things out, this my my stock tuning config. (take from some collection of Sun whitepapers):

$ cat /etc/rc3.d/S99ndd
#!/bin/bash

NDD=/usr/sbin/ndd

$NDD -set /dev/tcp tcp_xmit_hiwat 1048576
$NDD -set /dev/tcp tcp_recv_hiwat 8388608
$NDD -set /dev/tcp tcp_max_buf 8388608
$NDD -set /dev/udp udp_xmit_hiwat 1048576
$NDD -set /dev/udp udp_recv_hiwat 8388608
$NDD -set /dev/udp udp_max_buf 8388608
$NDD -set /dev/tcp tcp_conn_req_max_q 65536
$NDD -set /dev/tcp tcp_conn_req_max_q0 65536
$NDD -set /dev/tcp tcp_fin_wait_2_flush_interval 67500

Also, worth looking into on your OpenSolaris machine is changing the fsflush interval, the interrupt adjustment "magic" and disabling soft rings. Append the following to /etc/system (reboot required):

* "fsflush" tuning
set tune_t_fsflushr = 5
set autoup = 300
* Disable the Automatic Interrupt Adjustment
set dld:dld_opt = 2
* Disable "soft rings"
set ip:ip_squeue_fanout = 0
set ip:ip_soft_rings_cnt = 0

Worth mentioning, I do this on Solaris 10 -- not OpenSolaris -- but I think the tunables should work for you just the same.

I'm a big fan of Filebench for playing around with tuning options and doing throughput tests.

The (recently renamed) OpenSolaris 2009.06 release looks to be very exciting in the world of iSCSI and ZFS.

Hope this helps some!

How to manage Xen virtual machines on shared SAN storage

Xen has a built-in locking mechanism that should be used, especially if using hot or live migration. We have a NFS share mounted on all of our Dom0 hosts, and Xen checks there first to see if someone else is running an instance of a DomU before attempting to start it.

Look in your xend-config.sxp file -- the config settings your looking for start at about line 261 on my system, and are called xend-domain-lock, xend-domain-lock-path, and if you want to set up an external utility to manage it in another way, you can specify an external command to run with xend-domain-lock-utility. I'm using Xen 3.3.1 on SLES11; your mileage may vary.

We manage xend, xendomains, and nfs with pacemaker/openais, and store the lock files on an NFS mount that is made highly available by means of pacemaker and a partition on the san.

Best Answer

Related Solutions

Nfs – What are “good” speeds for iSCSI and NFS over 1GB ethernet

How to manage Xen virtual machines on shared SAN storage

Related Topic