Skip to content

Manual CephFS Config

CephFS Manual Client Configuration

For PCs which are not managed by the CERN-IT Puppet Infrastructure, you may manually configure a CephFS client to mount an OpenStack Manila share.

Requirements:

To follow this tutorial you will need the following information.

  • Which CephFS cluster are you mounting?
    • The examples below use Meyrin CephFS, aka cephflax.cern.ch
  • What is the remote path you are mounting?
    • This will look like /volumes/_nogroup/<uuid>
    • $CEPHFS_PREFIX in the recipes below.
  • What are the cephx id and secret to access the mount?
    • $CEPHX_ID and $CEPHX_SECRET in the recipes below.

Installing Required Packages:

We first add the ceph.repo and install ceph-fuse and ceph-common RPMs.

> sudo -i

# rpm --import http://linuxsoft.cern.ch/mirror/download.ceph.com/release.asc

# cat << EOF > /etc/yum.repos.d/ceph.repo
[ceph]
name=Ceph nautilus packages for x86_64
baseurl=http://linuxsoft.cern.ch/mirror/download.ceph.com/rpm-nautilus/el7/x86_64
enabled=1
gpgcheck=1
priority=1
type=rpm-md
gpgkey=http://linuxsoft.cern.ch/mirror/download.ceph.com/release.asc

[ceph-noarch]
name=Ceph nautilus noarch packages
baseurl=http://linuxsoft.cern.ch/mirror/download.ceph.com/rpm-nautilus/el7/noarch
enabled=1
gpgcheck=1
priority=1
type=rpm-md
gpgkey=http://linuxsoft.cern.ch/mirror/download.ceph.com/release.asc
EOF

# yum install ceph-fuse ceph-common

Manual Client Configuration:

You can choose to use ceph-fuse or the native Linux ceph driver to mount your CephFS share.

Ceph FUSE Client

If you want to mount CephFS using ceph-fuse, you must first create the needed config files. The example given is for Meyrin CephFS, aka cephflax.cern.ch. The parameter mon host should be changed if using a different cluster (see Share types).

First, the ceph.conf file:

# cat << EOF > /etc/ceph/ceph.conf
[global]
admin socket = /var/run/ceph/\$cluster-\$name-\$pid.asok
client reconnect stale = true
debug client = 0/2
fuse big writes = true
mon host = cephflax.cern.ch:6789
EOF

Next, create a cephx keyring file:

# CEPHX_ID=<create and get from openstack>
# CEPHX_SECRET=<create and get from openstack>
# cat << EOF > /etc/ceph/ceph.client.${CEPHX_ID}.keyring
[client.${CEPHX_ID}]
    key = ${CEPHX_SECRET}
EOF

Then, create an entry in /etc/fstab to mount the share at boot, for example:

# CEPHFS_PREFIX=<get from openstack>
# cat << EOF >> /etc/fstab
none    /cephfs fuse.ceph   ceph.id=${CEPHX_ID},ceph.conf=/etc/ceph/ceph.conf,ceph.client_mountpoint=${CEPHFS_PREFIX},x-systemd.device-timeout=30,x-systemd.mount-timeout=30,noatime,_netdev,rw 0   2
EOF

Now you can mount /cephfs to access your share.

Ceph Kernel Client

Alternatively, you can mount cephfs with a recent linux kernel.

First, create a cephx secret file:

# CEPHX_ID=<create and get from openstack>
# CEPHX_SECRET=<create and get from openstack>
# cat << EOF > /etc/ceph/ceph.${CEPHX_ID}.secret
${CEPHX_SECRET}
EOF

Then, create an entry in /etc/fstab to mount the share at boot. The example given is for Meyrin CephFS, aka cephflax.cern.ch. The hostname to connect to should be changed if using a different cluster (see Share types).

# CEPHFS_PREFIX=<get from openstack>
# cat << EOF >> /etc/fstab
cephflax.cern.ch:6789:{$CEPHFS_PREFIX}  /cephfs ceph    name=${CEPHX_ID},secretfile=/etc/ceph/ceph.${CEPHX_ID}.secret,x-systemd.device-timeout=30,x-systemd.mount-timeout=30,noatime,_netdev,rw 0   2
EOF

Now you can mount /cephfs to access your share.

Last update: June 10, 2021