Lustre

The Lustre® file system is an open-source, parallel file system that supports many requirements of leadership class HPC simulation environments. The main advantage of Lustre, a global parallel file system, over others such as NFS and SAN is that it provides; wide scalability, both in performance and storage capacity; a global name space, and the ability to distribute very large files across many nodes. Because large files are shared across many nodes in the typical cluster environment, a parallel file system, such as Lustre, is ideal for high end HPC cluster I/O systems.

Currently /home directory is of type Lustre filesystem. Currently it is provided from only two OSTs. This effectively means that one big file may be served from two storage servers, if this file is created in appropriate way. To convert a directory to “stripe” files in this way one can do the following:

lfs setstripe <filename|dirname> --count 2

to set file or directory to use striping with 2 OSTs in parallel. Currently there only setting the count to two may be advisable if high I/O rates are expected for files in a given directory. The Lustre filesystem is optimized for processing of large files. This means that creating large amount of small files may be relatively slow process.

GNU General Public License 2.0