Linux – How to install packages on Linux or Solaris on non-default paths

installationlinuxpackage-managementsolaris

By "default path", I mean "/usr/local", or other paths managed by root ("system paths").

I have to install many application packages (by that, I mean: svn, httpd, git, perl, python, …) on a few Linux (RedHat) or Solaris (10, in local zones) servers.

But:

those servers manage many different applications (sometimes using different versions of svn or perl or …)
I am not administrator on those servers (no sudo root for me)

I tried using, for instance on Solaris, pkgadd -R to try an install pre-compiled packages in a custom path (namely within the homedir of a specific user, rather than in the normal default path of /usr/local/...), but said pre-compiled packages all comes with references to other resources in /usr/...:

A ldd /path/to/local/installed/packages will show many dependencies to system paths:

ldd /home/myuser/usr/local/git
  libz.so => /usr/local/lib/libz.so
  libiconv.so.2 => /usr/local/lib/libiconv.so.2
  libsocket.so.1 => /usr/local/libsocket.so.1
  ...

That will not do, because:

In Solaris, I have no way to write anything on /usr which is only writable from the global zone, not from a local zone.
In Solaris or Linux, I am not root anyway, so I cannot write anything in system path.
I don't manage the upgrades on those servers, so if any library changes, it can potentially breaks many of my installed services.

What would you recommend to do in order to install in an isolated way different "services" on a same (Linux or Solaris) server, each one potentially requiring their own version of (perl, python, …)?

I propose a solution below, but if you have other alternatives, I am interested.

Best Answer

The only solution I have found so far:

compatible with installing (as non-root) multiple applications,
allowing each application to have with their own set of dependencies (with potential different versions used from one set of dependencies to another)

is:

recompile everything

(and by everything, I mean even gcc itself if needed, since the /usr/swf/bin/gcc installed by default on our Solaris servers is even older than the pre-requisite gcc 3.4.6)

All the versions used in this global recompilation are comming from sunfreeware, which will details all the necessary dependencies, and will provide a link to the sources for each packages.
That works both on Linux and Solaris.

Each package source is downloaded, compiled, and installed in $HOME/usr/local (i.e. not in a system path).
The key is to have a .bashrc (for instance) which will change the $PATH in order to not have any /usr/bin or /usr/local/bin in it, but only $HOME/usr/local/bin.

I found over time several advantages:

The libraries in /usr can change, that will have 0 impact on the several services currently running (because they all have been compiled on their own set of dependencies installed in $HOME/usr/local)
The non-root user who runs a compiled application is pretty much root in his own environment, and that user can launch/kill/update/recompile any element within $HOME/usr/local
It is easy to make a new directory in the $HOME and compile again all dependencies for testing an upgrade of a given application. You can end up with several versions of a same package, and test/switch from one version to another.
You control the compilation options, and it is very easy to compile an Apache Httpd with all modules activated in it if you want (as opposed to a pre-compiled package where you take what you get).

The main disadvantages are:

Any full compilation can takes time (up to 1 hour on Linux, 3 to 4 hours on Solaris).
But you don't always need to recompile everything.
The compilation options are different for every packages, and can be quite complex to set properly
The environment variables (LDFLAGS, CFLAGS, CPPFLAGS, LD_LIBRARY_PATH) can be tricky to setup with the right values
If you don't have a script able to extract for you the right dependencies and to launch the compilations, that means: manual process, which is a drag.
(I am in the process of making that script, and will publish it on GitHub)

Related Solutions

Debian – Running Stable and Installing Packages from Testing

Many people seem to be afraid of mixing stable with testing, but frankly, testing is fairly stable in its own right, and with proper preferences and solution checking, you can avoid the "stability drift" that puts your core packages on the unstable path.

"Testing is fairly stable??", you ask. Yes. In order for a package to migrate from unstable to testing, it has to have zero open bugs for 10 consecutive days. Chances are that, especially for the more popular packages, somebody is going to submit a bug report for an unstable version if something is wrong.

Even if you don't want to mix the environments, it's still nice to have the option there in case you run into something that requires a newer version than what is in stable.

Here's what I recommend for setting this up:

First, create the following files in /etc/apt/preferences.d:

stable.pref:

# 500 <= P < 990: causes a version to be installed unless there is a
# version available belonging to the target release or the installed
# version is more recent

Package: *
Pin: release a=stable
Pin-Priority: 900

testing.pref:

# 100 <= P < 500: causes a version to be installed unless there is a
# version available belonging to some other distribution or the installed
# version is more recent

Package: *
Pin: release a=testing
Pin-Priority: 400

unstable.pref:

# 0 < P < 100: causes a version to be installed only if there is no
# installed version of the package

Package: *
Pin: release a=unstable
Pin-Priority: 50

experimental.pref:

# 0 < P < 100: causes a version to be installed only if there is no
# installed version of the package

Package: *
Pin: release a=experimental
Pin-Priority: 1

(Don't be afraid of the unstable/experimental stuff here. The priorities are low enough that it's never going to automatically install any of that stuff. Even the testing branch will behave, as it's only going to install the packages you want to be in testing.)

Now, creating a matching set for /etc/apt/sources.list.d:

stable.list: Copy from your original /etc/apt/sources.list. Rename the old file to something like sources.list.orig.

testing.list: Same as stable.list, except with testing.

unstable.list: Same as stable.list, except with unstable, and remove the security lists.

experimental.list: Same as unstable.list, except with experimental.

You can also add a oldstable in sources.lists.d and preferences.d (use a priority of 1), though this moniker will tend to expire and disappear before the next stable cycle. In cases like that, you can use http://archive.debian.org/debian/ and "hardcode" the Debian version (etch, lenny, etc.).

To install the testing version of a package, simply use aptitude install lib-foobar-package/testing, or just jump into aptitude's GUI and select the version inside of the package details (hit enter on the package you're looking at).

If you get complaints of package conflicts, look at the solutions first. In most cases, the first one is going to be "don't install this version". Learn to use the per-package accept/reject resolver choices. For example, if you're installing foobar-package/testing, and the first solution is "don't install foobar-package/testing", then mark that choice as rejected, and the other solutions will never veer to that path again. In cases like these, you'll probably have to install a few other testing packages.

If it's getting too hairy (like it's trying to upgrade libc or the kernel or some other huge core system), then you can either reject those upgrade paths or just back out of the initial upgrade altogether. Remember that it's only going to upgrade stuff to testing/unstable if you allow it to.

EDIT: Fixed some priority pins, and updated the list.

Linux – How to Set Up Permissions for the WWW Folder

After more research it seems like another (possibly better way) to answer this would be to setup the www folder like so.

sudo usermod -a -G developer user1 (add each user to developer group)
sudo chgrp -R developer /var/www/site.com/ so that developers can work in there
sudo chmod -R 2774 /var/www/site.com/ so that only developers can create/edit files (other/world can read)
sudo chgrp -R www-data /var/www/site.com/uploads so that www-data (apache/nginx) can create uploads.

Since git runs as whatever user is calling it, then as long as the user is in the "developer" group they should be able to create folders, edit PHP files, and manage the git repository.

Note: In step (3): '2' in 2774 means to 'set Group ID' for the directory. This causes new files and sub directories created within it to inherit the group ID of the parent directory (instead of the primary group of the user) Reference: http://en.wikipedia.org/wiki/Setuid#setuid_and_setgid_on_directories

Best Answer

Related Solutions

Debian – Running Stable and Installing Packages from Testing

Linux – How to Set Up Permissions for the WWW Folder

Related Topic