How to mount rbd devices with the fuse driver

cephfuse

This is a follow-up question from: How do I mount an rbd device from fstab Which I asked some time back.

How do I mount rbd devices with the fuse driver? (I can't use the kernel driver from docker without opening up lots of privileges)

As far as I can tell I create an entry into the fstab file with something like:

id=client.admin /mnt fuse.ceph 0 0

However I can't see how to specify the rbd block device. If I run this mount, I see a file in /mnt/mariadb1 (mariadb1 is the name of my rbd device created with rbd create –size 250000 mariadb1)

How do I actually mount it as a filesystem?

Note: Yes I'd really like to use the kernel rbd driver and mount it from the host and then expose it to my docker container, but…. I'm using CoreOS.

If it is possible to mount filesystem in the host mount namespace I'd settle for that, but don't see a way forward yet. The best I've managed is to mount it from a container that has lots of privileges. But I can't share that container with –volumes-from. I don't really fancy having this particular container with full access to the host.

Best Answer

I have to assume that you are referring to RBD-fuse when talking about this because ceph-fuse is a cephfs utility and nothing to do with RBD.

However the premise of your question is further broken because your understanding of what rbd-fuse actually does is incomplete.

librbd provides you with an image file that is stored in Ceph. Most people (through rbd map) use that image to provide a pseudo block device in the form of /dev/rbd*.

The only difference with rbd-fuse is that it exposes its own pseudo file-system and provides the images as plain POSIX compliant files as opposed to Block devices.

Hence rbd-fuse has no more knowledge about the actual contents of the image file than rbd map does.

When invoking rbd-fuse mountpoint any RBD images inside the pool will be visible within mountpoint. It takes command options, notably -p if you use a different pool in ceph other than rbd and -c if you use a different configuration file other than /etc/ceph/ceph. You can also expose individual images using -r.

What you do with those files is up to you. They can have a file-system written to them (beware Linux will grumble as they aren't specialist block devices). You can mount them as loop devices if a file-system is present. 

$ rbd create test
# mkdir -p /rbd_images
# rbd-fuse /rbd_images
# mkfs /rbd_images/test
# mkdir -p /mnt/rbd_test
# mount /rbd_images/test /mnt/rbd_test

Should present something along the lines as follows:

$ mount
rbd-fuse on /rbd_images type fuse.rbd-fuse (rw,nosuid,nodev)
/rbd_images/rbd_test on /mnt/rbd_test type ext4 (rw)

Whether this is the correct way (for your usage) to do this though is a different matter. However it can allow you access to RBD images that would be unsupported otherwise by your systems krbd version.

Related Topic