Lightweight File Systems on Polynesia

Lightweight File Systems

Overview

The LWFS wiki has a mostly up to date project status.

What is Where?

The LWFS tools are currently installed in my home directory (/net/hc283/lofstead) under the lwfs directory. Hereafter, the path will be refered to as HOME.

Building LWFS

The system is currently built for use on a Linux machine, in particular, the polynesia cluster. If you wish to rebuild it elsewhere for another purpose, follow the steps below:

Go to HOME/lwfs/software and add all of the appropriate bin directories to the path and add some important environment vars. A shortcut for performing this is provided in HOME/lwfs/software/go.
Build each of the following:
1. db - The Berkeley Sleepycat DB (used for namespace item storage)
2. mpich2 - Used for the MPI examples
3. portals - (required) The Portals library (used for client-server communication). When building, specify using a user-level protocol unless kernel support is available. Be sure to also specify the env var PTL_IFACE to set the interface method. For user-level operation, use 'lo' (no quotes).
  1. (when downloading new code) Add these lines to your .bashrc file on software.sandia.gov so you can use mercurial.
```
export PYTHONPATH=/home/raoldfi/software/lib/python
export PATH=${PATH}/home/raoldfi/software/bin
```
  2. Clone the portals directory.
```
% hg clone
ssh://gflofst@software.sandia.gov//home/raoldfi/repos/portals
```
    Of cource you need to replace the gflofst with your username.
  3. Configure
```
% cd portals
% regen
% cd build/i386
% sh init-configure.sh --prefix=<PORTALS_DIST_DIR>
```
  4. Make
```
% make install
```
4. sysio - (required) A library that provides POSIX-like file and name space support for remote file systems from application program address space.
5. ebofs - Extent-Based Object File System (alternative for storage instead of normal filesystem files.
6. pablo - Pablo Trace Library for debugging.
7. gengetopt - (required) Command-line option processor used for server startup.
Execute configure providing the path to all of the include directory and library directories for the support software in use. An example call script is in HOME/lwfs/trunk/go. Once this is complete, make will build everything. It will take a couple of minutes.

Important Notes:

The gcc version used to compile lwfs must be at least a 4.x version. The go script mentioned above sets the env vars used by configure to force the use of gcc 4.x on the polynesia machines. An alternative setting may be necessary for your environment.
The build for the documentation is not setup by default. Proper build instructions have not been formalized (by me) at this point. Ron has the information and I believe he has sent them to me, but I haven't had a chance to try them out.

Running Programs

All of the example code should be built as part of the main build procedure. Each example has different requirements to operate. In general, the following startup steps are required for using the LWFS tools:

Authorization server startup (authentication is currently ignored)
All storage and naming servers can be started in any order since there is no direct relationship between these services.
Client code is started.

When starting up each service, it defaults to the services being on standard ports on the same machine. If the ports selected or machines are different, then use of the nid (node id) and pid (port id) paramters are required. These values can be printed by a server on startup given a verbose startup level of at least 3 (or higher).

Client applications need to be notified of all of the server locations and ports as part of startup. Since it is assumed there are LOTS of storage servers, the parameters only support specifying the authorization, authentication, and naming service parameters directly. The storage server information is expected to be in a text file with each line containing a node id, a space character, a port. An example is provided in HOME/lwfs/src/testing/xchange-tests/storage_servers.in.

NOTE: The default storage location for all of the databases and objects (files) is in the under the working directory in which the service was started.

Questions?

Please email me if there are any questions on how things work or the philosophy behind the design.

Back to Home