Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
software:start [2019/02/25 21:24] frey |
software:start [2019/02/25 22:07] (current) frey [Putting it all together] |
||
|---|---|---|---|
| Line 61: | Line 61: | ||
| </code> | </code> | ||
| + | ==== Finding libraries: the LD_LIBRARY_PATH variable ==== | ||
| + | |||
| + | Compiled subroutine/function libraries that are dynamically linked into a program at runtime exist as files, the same as executables. Just as executables are typically found in a ''bin'' directory, these libraries are often found in a ''lib64'' or ''lib'' directory. And equally similar is the fact that the user may not always be allowed to copy new libraries into ''/usr/local/lib64'' or ''/usr/lib64'' where they'll be found by default. The ''LD_LIBRARY_PATH'' variable is used by the runtime linker the same way ''PATH'' is used by the shell: the colon-separated list of directories is searched in sequence for the library that an executable is requesting. So when installing into ''/usr/local/lib64'' is not an option, adding to ''LD_LIBRARY_PATH'' is: | ||
| + | |||
| + | <code bash> | ||
| + | $ export LD_LIBRARY_PATH="/home/1001/programs/lib64:$LD_LIBRARY_PATH" | ||
| + | </code> | ||
| + | |||
| + | ===== Organizing software ===== | ||
| + | |||
| + | Given the different directories used by Unix/Linux to hold the various components of software and the ''PATH'' and ''LD_LIBRARY_PATH'' variables' ability to alter where the shell and runtime linker look to find executables and libraries, you have all the tools necessary to organize the software you maintain: any directory can have the ''bin'', ''lib64'', ''include'', et al. directories created within it as in ''/usr/local''. Adopting the same filesystem layout makes clear the purpose of each file therein. Adding ''<prefix>/bin'' to the ''PATH'' makes any executables present within that directory available to you. | ||
| + | |||
| + | ==== Source code ==== | ||
| + | |||
| + | Many Linux distributions have a ''/usr/local/src'' directory present. This directory is meant to hold source code packages that are built and installed under ''/usr/local''. For any software directory you manage, adopt the same strategy: | ||
| + | |||
| + | * If the software directory holds components of a single source package, unpack that source code as the ''<prefix>/src'' directory | ||
| + | * For a software directory containing components from multiple source packages (like ''/usr/local'') unpack each constituent source package as a directory under the ''<prefix>/src'' directory | ||
| + | |||
| + | ==== Versioning ==== | ||
| + | |||
| + | It is often the case when maintaining software that more than one //version// of the software must be present. Each distinct //version// implies a different rendition of the software source code. //Variants// of a version of the software may be present when, for example, different compilers are used to build the same version of the software. | ||
| + | |||
| + | In these cases, installation into a common software directory like ''/usr/local'' is inadvisable, since a single version or variant of the software is present and will likely change as time goes by. Using a unique software directory for each version or variant — and adopting the Unix/Linux filesystem layout therein — keeps each distinct copy of the software isolated from the others, and is easily added to the shell by altering the ''PATH'' and ''LD_LIBRARY_PATH'' variables, for example. | ||
| + | |||
| + | ==== Putting it all together ==== | ||
| + | |||
| + | I need to manage multiple versions and variants of the Open MPI tools. To start, I create a directory that will hold all of the versions and variants I build: | ||
| + | |||
| + | <code bash> | ||
| + | $ mkdir -p ~/programs/open-mpi | ||
| + | $ cd ~/programs/open-mpi | ||
| + | </code> | ||
| + | |||
| + | I will download the official source code and store each version I download in a directory — if at any time I need to build another variant of that version, I will have that version's source code available: | ||
| + | |||
| + | <code bash> | ||
| + | $ mkdir attic | ||
| + | $ cd attic | ||
| + | $ wget 'https://download.open-mpi.org/release/open-mpi/v3.1/openmpi-3.1.3.tar.bz2' | ||
| + | $ ls -l | ||
| + | total 1 | ||
| + | -rw-r--r-- 1 frey everyone 9442937 Oct 29 17:51 openmpi-3.1.3.tar.bz2 | ||
| + | </code> | ||
| + | |||
| + | To build a variant of the 3.1.3 version with standard system compilers, I'll create a directory and unpack the source code inside it: | ||
| + | |||
| + | <code bash> | ||
| + | $ cd .. | ||
| + | $ mkdir 3.1.3 | ||
| + | $ cd 3.1.3 | ||
| + | $ tar -xf ../attic/openmpi-3.1.3.tar.bz2 | ||
| + | $ mv openmpi-3.1.3 src | ||
| + | $ cd src | ||
| + | </code> | ||
| + | |||
| + | Open MPI uses the GNU Autoconf tools, so configuring the build to install the software into this variant's software directory is very straightforward: | ||
| + | |||
| + | <code bash> | ||
| + | $ ./configure --prefix="$(realpath ~/programs/open-mpi/3.1.3)" | ||
| + | : | ||
| + | $ make | ||
| + | $ make install | ||
| + | $ cd .. | ||
| + | $ pwd | ||
| + | /home/1001/programs/open-mpi/3.1.3 | ||
| + | $ ls -l bin | ||
| + | total 474 | ||
| + | -rwxr-xr-x 1 frey sysadmin 1740 Oct 8 11:04 aggregate_profile.pl | ||
| + | lrwxrwxrwx 1 frey sysadmin 12 Oct 8 11:05 mpic++ -> opal_wrapper | ||
| + | lrwxrwxrwx 1 frey sysadmin 12 Oct 8 11:05 mpicc -> opal_wrapper | ||
| + | lrwxrwxrwx 1 frey sysadmin 12 Oct 8 11:05 mpiCC -> opal_wrapper | ||
| + | : | ||
| + | lrwxrwxrwx 1 frey sysadmin 12 Oct 8 11:05 shmemcxx -> opal_wrapper | ||
| + | lrwxrwxrwx 1 frey sysadmin 12 Oct 8 11:05 shmemfort -> opal_wrapper | ||
| + | lrwxrwxrwx 1 frey sysadmin 6 Oct 8 11:05 shmemrun -> mpirun | ||
| + | </code> | ||