Manual CephFS Config
CephFS Manual Client Configuration
For PCs which are not managed by the CERN-IT Puppet Infrastructure, you may manually configure a CephFS client to mount an OpenStack Manila share.
Requirements:
To follow this tutorial you will need the following information.
- Which CephFS cluster are you mounting?
- The examples below use
Meyrin CephFS
, akacephflax.cern.ch
- The examples below use
- What is the remote path you are mounting?
- This will look like
/volumes/_nogroup/<uuid>
$CEPHFS_PREFIX
in the recipes below.
- This will look like
- What are the cephx id and secret to access the mount?
$CEPHX_ID
and$CEPHX_SECRET
in the recipes below.
Installing Required Packages:
We first add the ceph.repo
and install ceph-fuse
and ceph-common
RPMs.
Begin by adding the ceph.repo
file in the /etc/yum.repos.d
directory. In the example below, replace {ceph-release}
with the current stable major release of Ceph (e.g., octopus
or pacific
as of Jan 2022) and {distro}
with your Linux distribution (e.g., el7
or el8
). You may browse the rpm-{ceph-release}
dirs at https://download.ceph.com/ to see which distributions Ceph supports. Some Ceph packages (e.g., EPEL) must take priority over standard packages, so you must ensure that you set priority=1. (If running CentOS 7, also ensure that yum-priorities
is installed -- yum install yum-priorities
).
> sudo -i
# rpm --import http://linuxsoft.cern.ch/mirror/download.ceph.com/release.asc
# cat << EOF > /etc/yum.repos.d/ceph.repo
[ceph]
name=Ceph {ceph-release} packages for $basearch
baseurl=http://linuxsoft.cern.ch/mirror/download.ceph.com/rpm-{ceph-release}/{distro}/$basearch
enabled=1
gpgcheck=1
priority=1
type=rpm-md
gpgkey=http://linuxsoft.cern.ch/mirror/download.ceph.com/release.asc
[ceph-noarch]
name=Ceph {ceph-release} noarch packages
baseurl=http://linuxsoft.cern.ch/mirror/download.ceph.com/rpm-{ceph-release}/{distro}/noarch
enabled=1
gpgcheck=1
priority=1
type=rpm-md
gpgkey=http://linuxsoft.cern.ch/mirror/download.ceph.com/release.asc
EOF
# yum install ceph-fuse ceph-common
Manual Client Configuration:
You can choose to use ceph-fuse
or the native Linux ceph
kernel driver to mount your CephFS share. As of Jan 2022, the latter is recommended.
Ceph Kernel Client
Ensure that ceph-common
is already installed and that you are already running the latest kernel for your distro. (yum update && reboot
).
Then, create a cephx secret file:
# CEPHX_ID=<create and get from openstack>
# CEPHX_SECRET=<create and get from openstack>
# cat << EOF > /etc/ceph/ceph.${CEPHX_ID}.secret
${CEPHX_SECRET}
EOF
Next, create an entry in /etc/fstab
to mount the share at boot.
The example given is for Meyrin CephFS
, aka cephflax.cern.ch
. The hostname to connect to should be changed if using a different cluster (see Share types).
# CEPHFS_PREFIX=<get from openstack>
# cat << EOF >> /etc/fstab
cephflax.cern.ch:6789:{$CEPHFS_PREFIX} /cephfs ceph name=${CEPHX_ID},secretfile=/etc/ceph/ceph.${CEPHX_ID}.secret,x-systemd.device-timeout=30,x-systemd.mount-timeout=30,noatime,_netdev,rw 0 2
EOF
Tip: If you would like to enable automounting for /cephfs, you can append ,noauto,x-systemd.automount,x-systemd.idle-timeout=30
to the fstab mount options in the example above, then run as root systemctl daemon-reload
and systemctl restart remote-fs.target
for this to take effect.
Now you can mkdir -p /cephfs; mount /cephfs
to access your share.
Ceph FUSE Client
Ensure that both ceph-fuse
and ceph-common
are installed as above.
If you want to mount CephFS using ceph-fuse
, you must first create the needed config files.
The example given is for Meyrin CephFS
, aka cephflax.cern.ch
. The parameter mon host
should be changed if using a different cluster (see Share types).
First, the ceph.conf
file:
# cat << EOF > /etc/ceph/ceph.conf
[global]
admin socket = /var/run/ceph/\$cluster-\$name-\$pid.asok
client reconnect stale = true
debug client = 0/2
fuse big writes = true
mon host = cephflax.cern.ch:6789
EOF
Next, create a cephx keyring file:
# CEPHX_ID=<create and get from openstack>
# CEPHX_SECRET=<create and get from openstack>
# cat << EOF > /etc/ceph/ceph.client.${CEPHX_ID}.keyring
[client.${CEPHX_ID}]
key = ${CEPHX_SECRET}
EOF
Then, create an entry in /etc/fstab
to mount the share at boot, for example:
# CEPHFS_PREFIX=<get from openstack>
# cat << EOF >> /etc/fstab
none /cephfs fuse.ceph ceph.id=${CEPHX_ID},ceph.conf=/etc/ceph/ceph.conf,ceph.client_mountpoint=${CEPHFS_PREFIX},x-systemd.device-timeout=30,x-systemd.mount-timeout=30,noatime,_netdev,rw 0 2
EOF
Tip: If you would like to enable automounting for /cephfs, you can append ,noauto,x-systemd.automount,x-systemd.idle-timeout=30
to the fstab mount options in the example above, then run as root systemctl daemon-reload
and systemctl restart remote-fs.target
for this to take effect.
Now you can mkdir -p /cephfs; mount /cephfs
to access your share.