Creating block volumes
OpenStack provides two classes of block storage, ephemeral storage and persistent volumes. Ephemeral storage exists only for the life of an instance, it will persist across reboots of the guest operating system but when the instance is deleted so is the associated storage. The size of the Ephemeral storage is defined in the flavor of the virtual machine and is fixed for all virtual machines of that particular flavor. The service level for ephemeral storage is dependent on the underlying hardware.
Volumes are persistent virtualized block devices independent of any particular instance. Volumes may be attached to a single instance at a time (i.e. not like a distributed filesystem such as AFS or DFS), but they may be detached or reattached to a different instance while retaining all data, much like a USB drive. Volumes are stored on redundant storage and are therefore more resilient against hardware failures.
You can define additional volumes to be attached to a virtual machine in addition to the system disk. This is useful if you need more space than is provided on the initial flavor and can also allow some interesting flexiblity for snapshotting or moving volumes around between virtual machines.
The steps are
- Define a disk volume of the size required (10GB in example below) with the volume name defined (testvol in the example below)
- Check that the volume is available for mounting using
openstack volume list - Attach that volume to the instance that you would like it to be available on
- Log in to the instance and format/mount as required
For example, to define a volume of 10GB named timvol143, attach it to the VM test143 and fornat/mount it as ext4, you do the following on a machine which has the unified openstack command line tool installed.
The volume type describes the type of storage that is being requested.
The following storage types are currently available. Note: the standard type is allocated for projects by default. To request volumes of the other types, please fill out the project quota request form.
| Storage Type | Description | Datacentre | Location | Maximum Performance |
|---|---|---|---|---|
| standard | The 'standard' volume type provides a reliable store which is tolerant to at least one disk failure without user impact or data loss. This is the default. | Meyrin | Main room | 80MB/s and 100 IO operations (both, read and write) |
| io1 | The 'io1' volume type provides the same reliability level as 'standard' volumes, but is meant for applications with higher IO demands. | Meyrin | Main room | 120MB/s and 500 IO operations (both, read and write) |
| cp1 | The 'cp1' volume type provides the same performance level as 'standard' volumes, but is meant for applications with higher reliability demands: 'cp1' volumes are hosted on critical power which guarantees availability even in the event of a power cut. 'cp1' volumes should hence be used for projects mapped to the critical area. | Meyrin | Critical Area | 80MB/s and 100 IO operations (both, read and write) |
| cpio1 | The 'cpio1' volume type combines the characteristics of 'cp1' and 'io1' volumes. | Meyrin | Critical Area | 120MB/s and 500 IO operations (both, read and write) |
| vault-100 | The 'vault-100' provides the same reliability level as 'standard' volumes | Meyrin | Vault | 80MB/s and 100 IO operations (both, read and write) |
| vault-500 | The 'vault-500' volume type provides the same reliability level as 'vault-100' volumes, but is meant for applications with higher IO demands. | Meyrin | Vault | 120MB/s and 500 IO operations (both, read and write) |
Volume creation
$ openstack volume create --description "Volume for documentation" --size 10 timvol143
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| created_at | 2014-12-18T13:27:27.754856 |
| display_description | Volume for Documentation |
| display_name | timvol143 |
| encrypted | False |
| id | d8d8be88-f1a4-4120-b1c2-50621a981c74 |
| properties | |
| size | 10 |
| snapshot_id | None |
| source_volid | None |
| status | creating |
| type | standard |
+---------------------+--------------------------------------+
The size is measured in GB.
The status reported will become available when the volume has been created. This status can be checked using openstack volume show.
Attaching a volume
Once the volume has been created, it can be attached to the virtual machine.
$ openstack server add volume tim-centos7-143 timvol143
The arguments are
- tim-centos7-143 is the virtual machine where you would like to attach the image. This is as previously created using
openstack server create. - timvol143 is the volume id from
openstack volume listas created above
Note: The optional device argument to specify the device name should not be used. It does not work properly, and will be removed in the near future.
The operation can be checked by seeing if this is now shown in the openstack server list command.
Log into the VM to check and format the disk:
$ ssh root@tim-centos7-143
# grep vdb /proc/partitions
252 48 12582912 vdb
Create a file system on it and mount it. Using a volume label is useful in the future to avoid needing to find the exact device name as you attach different volumes. Using a label, such as testvol in the example below, which is the same as the name in OpenStack also makes tracking easier when volumes are not mounted. The filesystem type can be one supported by the Linux distribution but xfs and ext4 are the most widely used.
# mkfs -t ext4 -L testvol /dev/vdb
# mount -L testvol /mnt
# df
If you'd like non-root users to be able to access the mounted volume similar to the way /tmp is accessible , you'll need to set the sticky bit on the mount point.
# chmod +t /mnt
The filesystem should be added to /etc/fstab if the desired behaviour is to mount automatically on restart. Using the LABEL parameter here will ensure the correct volume is chosen if multiple volumes are attached.
LABEL=testvol /mnt ext4 noatime,nodiratime,user_xattr,nofail 0 0
Note: The use of nofail is recommended to skip (and not block on) mounting the volume if it is unavailable, e.g. in case of network issues. Remove this option from the fstab line if you want your VM to block the boot process if the volume is unavailable.
Note: The use of nobarrier for the mount options is not recommended as volumes are accessed via a cache, so ignoring the correct ordering of journal commits may leave you with a corrupted file system in case of a hardware problem.
Removing disk volumes
The steps to properly remove disk volumes are:
- Umount the volume on the virtual machine
- Detach it from the virtual machine
- Delete the volume
- Logged into the virtual machine that has the volume mounted, unmount it first
# umount /mnt
Note: detaching a volume while it is being mounted is possible, but the next access on the VM to that volume will return I/O errors.
Then use the openstack command line interface to detach the volume from the virtual machine:
$ openstack server remove volume tim-centos7-143 timvol143
where tim-centos7-143 is the virtual machine and the second parameter, timvol143, is the volume as shown in openstack volume list.
Check that the volume is in state 'available' again.
If that's the case, the volume is now ready to either be attached to another virtual machine or, if it is not needed any longer, to be completely removed (note that this step cannot be reverted!):
$ openstack volume delete timvol143
Your volume will now go into state 'deleting' and completely disappear from the openstack volume list output
Volume snapshots
A user can also create snapshots of a volume, that then later can be used to create other volumes or to rollback to a precedent point in time. Volume snapshots are pointers in the RW history of a volume. The creation of a snapshot takes a few seconds and it can be done while the volume is in-use.
Once you have the snapshot, you can use it to create other volumes based on this snapshot. Creation time for these volumes may depend on the type of the volume you are creating as it may entitle some data transfer.
$ openstack volume snapshot create --volume timvol143 timvolsnap143
+-------------+--------------------------------------+
| Field | Value |
+-------------+--------------------------------------+
| created_at | 2020-12-10T09:16:45.749407 |
| description | None |
| id | 7c63c503-8cb5-40bd-b434-9b638ecb4fc3 |
| name | timvolsnap143 |
| properties | |
| size | 1 |
| status | creating |
| updated_at | None |
| volume_id | f3709518-b67b-49a8-ad03-0ba5f115ab67 |
+-------------+--------------------------------------+
Note: if the volume is in-use, you may need to specify --force
You can list the volume snapshots with the following command.
$ openstack volume snapshot list
+--------------------------------------+---------------+-------------+-----------+------+
| ID | Name | Description | Status | Size |
+--------------------------------------+---------------+-------------+-----------+------+
| 7c63c503-8cb5-40bd-b434-9b638ecb4fc3 | timvolsnap143 | None | available | 1 |
+--------------------------------------+---------------+-------------+-----------+------+
Once the volume snapshot is in available state, then you can create other volumes based on that snapshot. You don't need to specify the size of the volume, it will use the size of the snapshot.
$ openstack volume create --description --source timvolsnap143 "Volume from an snapshot" timvol143
+---------------------+--------------------------------------+
| Field | Value |
+---------------------+--------------------------------------+
| attachments | [] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2020-12-10T09:19:45.000000 |
| description | None |
| encrypted | False |
| id | 68fb4e47-39e6-40b5-a12d-371b34989b83 |
| multiattach | False |
| name | timvol144 |
| properties | |
| replication_status | None |
| size | 1 |
| snapshot_id | 7c63c503-8cb5-40bd-b434-9b638ecb4fc3 |
| source_volid | None |
| status | creating |
| type | standard |
| updated_at | None |
+---------------------+--------------------------------------+
You can delete the snapshots just by issuing the following command
$ openstack volume snapshot delete timvolsnap143
Common Error Messages
From the openstack volume command, the following errors can be received
| Error | Cause |
|---|---|
| VolumeSizeExceedsAvailableQuota: Requested volume or snapshot exceeds allowed Gigabytes quota ERROR: openstack Request Entity Too Large (HTTP 413) |
Your total space for the volume service exceeds the allowance for the project. If you require further quota, please contact the helpdesk. Please note that additional quota is only granted for shared projects. Additional personal project quota requests will not be accepted. |
| VolumeLimitExceeded: Maximum number of volumes allowed (N) exceeded | You have exceeded the number of volumes. Please contact the helpdesk to request to raise this limit for a project. Additional personal project quota requests will not be accepted. |
| BadRequest: Failed to delete volume with name or ID: Invalid volume: Volume status must be available or error or error_restoring or error_extending or error_managing and must not be migrating, attached, belong to a group, have snapshots or be disassociated from snapshots after volume transfer. (HTTP 400) | If you are deleting a volume and has |
| associated snapshots, you need to delete them first, before deleting the volume. |