LINUX GAZETTE
[ Prev ][ Table of Contents ][ Front Page ][ Talkback ][ FAQ ][ Next ]

"Linux Gazette...making Linux just a little more fun!"


Installing Software from Source
-or-
What Do I Do with this file.tar.gz Thing?

By Ben Okopnik


The other day, I decided to download "cuyo" (see Mike Orr's review in this issue), a new game that had been mentioned on the Answer Gang admin list. When I went to the website, however, I found only a source tarball instead of a package - even though the e-mail had mentioned an available Debian archive. No big deal, I thought - I've done this before...

[The cuyo .deb is in the Debian Unstable distribution. But this article applies to any program you want to install that's either not in your distribution, or where the distribution's version is old or inadequate. -Iron.]
For those who don't know, a tarball is a "tar"red and usually "gzip"ped list of source files that can be compiled to produce a program; the filename of a tarball is usually either "progfile-1.23.tgz" or "progfile-1.23.tar.gz", with "progfile" being the name of the program and "1.23" (obviously, the numbers can be almost anything) standing for the version. When you install a package - whether RPM, DEB, or whatever your distro uses - you're simply placing the libraries, documentation, and the precompiled binary or binaries in the directories where they belong. Compiling the source is the part that normally gets done for you by the package maintainer.

After downloading the tarball to my "/home/ben/TGZs" subdirectory, one I'd created specifically for the purpose of storing downloaded tarballs, I put a copy of it in "/tmp", where I would be compiling the sources. Note that some folks prefer to do it in "~/tmp", a directory under their home; the reasoning there is that "/tmp" usually gets wiped on bootup, and a compile that goes REALLY wrong could lock your machine... which would require a reboot (oops!) I can't fault their thinking, but continue to be the dangerous daredevil that I am - I trust my Linux. :)

The file was called "cuyo-1.03.tar.gz" - so, the appropriate bit of magic which turns it back into useful files is

tar xvzf cuyo-1.03.tar.gz

This created a directory called "cuyo-1.03" right there in "/tmp".

(OK, so that's not exactly how I did it; I looked inside the tarball with Midnight Commander, opened "/tmp" in my second pane, and hit "F5" to copy out the compressed directory. I'm spelling it out here for those folks who want to or have to do it manually.)

Note that some program authors are not that "polite" about making up their tarballs: sometimes, untarring one dumps the entire list of files in the current directory. What a mess, especially if it's in your home directory! Several dozen files intermixed with yours; a bunch of directories, too - and it gets much worse if some of these have the same name as your files (not that yours will be overwritten, but it's still a mess) or your directories (stuff will get dumped into them which you would have to then fish out.) How rude! This is why I like to peek into tarballs and copy, instead of just wholesale untarring. For those who don't use Midnight Commander or another file manager that's capable of looking inside a tarball, just do

tar tvf <filename>

This will show you the contents of it - and if everything isn't prefixed with a directory name, beware! Well, not really: all you have to do is create a directory (if you make it the same as the tarball "progname", you won't lose track of what it is, later) and untar the file inside it.

mkdir rudeprogram-6.66
tar xvf rudeprogram-6.66.tgz -C rudeprogram-6.66

Now, all of the files from the "rudeprogram" tarball will be extracted to the specified subdirectory.

Fortunately, the author of "cuyo" is a polite fellow (as most authors are), and "cuyo" was tarred up in a subdir of its own. Inside it, there was the list of files, including the ones that you should always check out prior to starting operations: "README" and "INSTALL". The first is usually the author's instructions, recommendations, etc. The second is fairly standard - it's a file that explains the operation of "configure", an extremely cool program usually created by "autoconf" that will check out your system and correctly (well, usually) set up the Makefile, which is what we need to compile the program. The huge advantage of this is that, if the author was careful in writing his program, "configure" will create the correct Makefile on any version of Unix - and perhaps even other OSs.

Allow me to digress for a moment: some programs are so simple that they do not require "configure", and simply come with a Makefile (these may be capitalized or all lower-case). Others are simpler yet - all you see is a single "progfile.c", or "progfile.cc". With these, compilation consists of simply running "make" in the first case, or "cc progfile.c -o progfile" in the second.

Anyway - I ran "configure" in the "cuyo" subdir; it chewed on my system for a while, which is its job, and built me a Makefile. Wasn't that nice of it? :) There was a bit of a problem, though: "configure" prints out messages as it runs, and warns you in case of failures (usually by stopping and printing an error.) The message that it gave me - without stopping, however - was

checking the Qt meta object compiler... (cached) failure 
configure: warning: Your Qt installation seems to be broken!

Hmm. Well, it built the makefile, anyway. Usually, the non-fatal errors just mean that you won't get some of the features of the program, but it will still compile. Well, let's try it.

I then ran "make" - just by typing "make" on the command line, which by default reads the "Makefile" (or "makefile"), and follows the commands specified in target "all" and, ...

Ooops. It failed.

It was at this point that I decided to write this article. I hadn't been thinking of doing that; I actually had lot s of work to do this month - but I believe that installing from tarballs is a skill that is necessary for any Linux user, and my thought here was to document the process, including troubleshooting installations that go wrong. It's something I struggled with in my early Linux days, and I'd like to save others at least a bit of that pain. :)

So. We go bravely on. When I say that it failed, exactly what did I see? Well, a "make" should run without errors. Sometimes - often - you'll get warnings, which are not the same thing; your libraries may be slightly different, or perhaps your compiler is a bit more strict about declarations - but these are usually not fatal. The errors that drop you out of a compile without finishing it - those are the ones that we have to fix. So, here's what it all looked like:

Baldur:/tmp/cuyo-1.03$ make
make all-recursive 
make[1]: Entering directory `/tmp/cuyo-1.03'
Making all in src 
make[2]: Entering directory `/tmp/cuyo-1.03/src' 
c++ -DHAVE_CONFIG_H -I. -I. -I.. -DPKGDATADIR=\"/usr/local/share/cuyo\" 
	-Wall -ansi -pedantic -c bildchenptr.cpp
In file included from bildchenptr.h:21, 
	from bildchenptr.cpp:18: 
inkompatibel.h:13: qglobal.h: No such file or directory
make[2]: ** [bildchenptr.o] Error 1 
make[2]: Leaving directory `/tmp/cuyo-1.03/src'
make[1]: [all-recursive] Error 1 
make[1]: Leaving directory `/tmp/cuyo-1.03'
make:  * [all-recursive-am] Error 2 
Baldur:/tmp/cuyo-1.03$

The error begins at the line that starts with "In file included...", and ends with (at least the part we want) "...qglobal.h: No such file or directory". OK - we're missing a header file. I took a quick look through the source tree of "cuyo", just to make sure that the author didn't forget to include one of his own files (yeah, it happens) - nope. Must be one of mine - that is, his program must be looking for a file that comes with a library which I need to have on my system for his program to compile. Hmm. Which one? Whichever one contains "qglobal.h", of course.

On my system, I have set up several scripts to help me with standard installation tasks. One of these is "pkgf" - it finds whatever file I'm looking for in the entire Debian distro, and tells me in what package that file exists (this is not the same as "dpkg -S <file>", which does that for installed packages only.) If you use Debian, you can get the same functionality by downloading the current "Packages.gz" from <ftp://ftp.debian.org> and "zgrep"ping through it for the name of the file - or, just go to <http://www.debian.org/Packages> and use their search utility. The point is to find which package contains "qglobal.h" and install it.

pkgf "qglobal.h" 
usr/include/qt/qglobal.h 	devel/libqt-dev 
				devel/libqt3-emb-dev 
				devel/libqt3-dev 
				devel/libqt-emb-dev

Well, well - it looks like I have a choice of packages. OK, "libqt3-dev" looks like the latest thing:

apt-get install libqt3-dev

The installation went fairly quickly, and... I got the same error when I re-ran "make". And so would you. So, don't do that. The thing to remember here (and I knew that I would get the error - I did this to make a point) is that you already ran "./ configure": the old (broken) values are still in the Makefile, as well as in several other files, so, rather than wasting time and trying to find out where they may be:

ben@Baldur:/tmp/cuyo-1.03$ cd ..
ben@Baldur:/tmp$ rm -rf cuyo-1.03 
ben@Baldur:/tmp$ tar xvzf ~/TGZs/cuyo-1.03.tar.gz -C . 
ben@Baldur:/tmp$ cd cuyo-1.03

In other words, I just blew away the entire "cuyo" directory and replaced it with a fresh copy of the source. This is a good rule of thumb in general: when in doubt, go back to the original sources. Believe it or not, I learned that trick from a boat mechanic who did extraordinarily good work. The way Kenny phrased it was "whack it back to the stuff that you know is good, then build it up from there." I've never seen his advice go wrong; admittedly, clients tend to scream when you tell them that you have to throw away the piece of garbage software that they have right now and replace from the ground up... but after a while, the word spreads: "Hey, this guy's work is good." You might lose some jobs that way - I know I do - but, like Kenny, I'm not willing to have my name on a piece of garbage.

I know, I know - I'm talking about things that are more generalized than just a plain old tarball install. The thing is, the philosophy of how you do things has to come from somewhere - and it's best if you figure out how you're going to do things before you actually do them, overall methodology as well as job specifics. OK, so, back to the main question - did it work or not???

ben@Baldur:/tmp/cuyo-1.03$ ./configure
<No errors>
ben@Baldur:/tmp/cuyo-1.03$ make
<lots of output elided> 
make[2]: Leaving directory `/tmp/cuyo-1.03/src'
Making all in data 
make[2]: Entering directory `/tmp/cuyo-1.03/data'
make[2]: Nothing to be done for `all'. 
make[2]: Leaving directory `/tmp/cuyo-1.03/data'
Making all in docs 
make[2]: Entering directory `/tmp/cuyo-1.03/docs'
make[2]: Nothing to be done for `all'. 
make[2]: Leaving directory `/tmp/cuyo-1.03/docs' 
make[2]: Entering directory `/tmp/cuyo-1.03' 
make[2]: Nothing to be done for `all-am'. 
make[2]: Leaving directory `/tmp/cuyo-1.03' 
make[1]: Leaving directory `/tmp/cuyo-1.03' 
ben@Baldur:/tmp/cuyo-1.03$

Ta-daaa!!! No errors - and when I enter the "cuyo-1.03/src" directory, there's a very nice-looking executable called "cuyo" sitting in there. At this point, if I wanted to continue the installation (rather than just testing the game to see if I like it), I would type

make install

This would read the Makefile and execute all the commands under the "install" target which would most likely install the executable[s], the man pages, and the documentation. However, I tend to play with the program first, and see if I like it - most tarball makefiles do not include an "uninstall" target (which I think is a shame; that would make tarball packages almost as easy to install and remove as it is, say, RPMs or DEBs.)

To recap the entire tarball install:

1) Check if it contains a directory or just (how rude!) scattered files
2) Untar into a directory under "/tmp" or "~/tmp"
3) Run "configure" if it exists.
4) Run "make", or "cc" if it's just a plain single "file.c" or "file.cc"
5) Run "make install" if the result is what you wanted.

That's pretty much it. Note that I did not discuss security anywhere in here (do you really trust the author of this tarball or package? You're not logged in as root while playing with that binary, right?), nor many of the other issues that pertain to system administration; these issues are very important and highly pertinent, but outside the scope of this short article. The wise system administrator - and that, my dear home Linux user, is you; there isn't anyone else for your machine! - will read much, think deeply, and consider wisely.

Good luck, and may all your dependencies end up being resolved ones. :)

Ben Okopnik

Ben Okopnik

A cyberjack-of-all-trades, Ben wanders the world in his 38' sailboat, building networks and hacking on hardware and software whenever he runs out of cruising money. He's been playing and working with computers since the Elder Days (anybody remember the Elf II?), and isn't about to stop any time soon.


Copyright © 2002, Ben Okopnik.
Copying license http://www.linuxgazette.net/copying.html
Published in Issue 74 of Linux Gazette, January 2002

[ Prev ][ Table of Contents ][ Front Page ][ Talkback ][ FAQ ][ Next ]