...making Linux just a little more fun!
By Anonymous
It all started with SAK, the Secure Attention Key, that was supposed to get me out of trouble in ubuntu. The help I got was a kernel panic.
Investigations followed, doubts followed the investigations. I report starting with a recap on SysRq, mainly from
http://www.kernel.org/doc/documentation/sysrq.txt
This doc lacks important information but it is included here.
(1) What is the 'magic' SysRq key?
It is a key combo the kernel will immediately respond to, whatever it is doing - assuming it was compiled with the CONFIG_MAGIC_SYSRQ option. This is the case with the major GNU/Linux distros.
(2) Need the SysRq key be enabled?
No, when running a kernel with SysRq compiled in, the key is enabled. But you can disable or restrict it courtesy of
/proc/sys/kernel/sysrq
By default, this file contains 1 and SysRq is fully enabled. To disable SysRq write 0 to the file.
To restrict functionality selectively, look at the table
value option ----- --------
2 enable control of console logging level 4 enable control of keyboard (e.g. SAK) 8 enable debugging dumps of processes etc. 16 enable sync command 32 enable remount read-only 64 enable signalling of processes (term, kill, oom-kill) 128 allow reboot/poweroff 256 allow rescheduling
Pick your options, sum their values, write the sum to /proc/sys/kernel/sysrq. After which, only your options will be allowed.
The value in /proc/sys/kernel/sysrq determines the shortcuts available to all users and not subject to permissions. However, writing to this file requires root privileges. And since the file is not really a file and dies when shutting down or rebooting, the writing has to be repeated after each boot.
The option 'sysrq_always_enabled' in the kernel line of the boot loader will let the kernel ignore /proc/sys/kernel/sysrq.
(3) How do I use the SysRq key on x86 PCs?
Press the key combo
Alt-SysRq-command_key
where SysRq is same as PrintScreen and command_key is a case insensitive letter or a digit from
b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,z,0-9
Needless to say, it is awkward or even unfeasible to hold down three keys at once when two of them are a stretched hand apart. Just do:
hold down Alt (either on the right or on the left) press and release SysRq speedily press command_key speedily release everything
Speedily means: don't dither on the desktop or ubuntu will pop up a Screen Shot dialog.
The position of keys is as on the US keyboard! Keep this in mind when using a non-US keyboard. For instance, 'q' is the key just right of Tab even if that same key carries the label 'a' on a French keyboard.
Indeed, you are not sending an 'a' or a 'q', you are sending a well defined scancode to the kernel. Keyboards needn't apply if they produce scancodes different from the common PC keyboards'.
Only details for a couple of the commands are of interest in the current context:
i Kill all processes, except init.
k Secure Attention Key (SAK) kills all applications on the current virtual console.
0-9 Sets the console log level, controlling which kernel messages will show on your console: 0 would let only emergency messages reach the console while 9 is fastidious.
(4) Kernel Panic
So why do I endure very reliably a kernel panic when I press SAK on ubuntu 9.10 both in the text console and on the desktop? The difference between the two is that the kernel panic is immediate in the text console. On the desktop it may look like everything is ok, then you press e.g. alt-ctrl-f1 to switch to a text console and it happens.
Well, this is a bug already reported in Feb. 2009 for a previous version:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/329576
An addendum to the bug report says the bug manifests on AMD boards but not necessarily on Intel boards. I can confirm that I do get my kernel panic on AMD hardware but I have also seen it on an Intel motherboard with a VIA graphic adapter.
A related bug was reported for Debian in Aug. 2009:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=543324
but here the console-kit-daemon was pointed out as the culprit. I disabled it, the bug is still active on my ubuntu.
Changing the log level with
Alt-SysRq-9
allows for a flood of low level details for the initiated. What is clear reading and watching is that an attempt to kill the 'init' process occurs - certainly not the kernel's intention. Who is trying to kill init? Something specific to ubuntu, shall we say, since openSuSe doesn't do it.
Now, when I try
Alt-SysRq-i
to kill all processes except init, I get a kernel panic exactly as with SAK, exactly with the same message about init's attempted murder.
That ubuntu bug of Feb. 2009 got no follow-up, not even an assignment to a developer, not even a priority rank. What does it take for serious bugs to get fixed in ubuntu?
(5) Security by Good Luck
At the end of the day, what we know is disturbing:
- Everybody wants the magic of SysRq, otherwise the major distros would not compile it in.
- The administrator can reduce SysRq's functionality but not just for the unwashed masses. Any reduction in SysRq's functionality will equally affect the administrator. Functionality cannot be changed on the fly, because when a critical situation arises you already need it.
- Even disregarding bugs like ubuntu SAK's bug, SysRq has options to bring down and damage the entire system. If you start disabling them, SysRq looses its magic in the difficult situations it is called for. So if you as an administrator want SysRq, you are offering vandals and experimenters a comfortable way to wreak havoc.
Is a GNU/Linux system the most unsecure system in the history of computers? Why is the SysRq magic not reserved for root?
Talkback: Discuss this article with The Answer Gang
A. N. Onymous has been writing for LG since the early days - generally by
sneaking in at night and leaving a variety of articles on the Editor's
desk. A man (woman?) of mystery, claiming no credit and hiding in
darkness... probably something to do with large amounts of treasure in an
ancient Mayan temple and a beautiful dark-eyed woman with a snake tattoo
winding down from her left hip. Or maybe he just treasures his privacy. In
any case, we're grateful for his contributions.
-- Editor, Linux Gazette
By Silas Brown
Chinese characters are used not only in Mainland China but also in Taiwan, Hong Kong, Japan, and sometimes in Korea and other places. There are differences in the way these countries write certain characters, and ideally this should just be a matter of selecting a different font. However, for various reasons some (but not all) of the "Simplified Chinese" characters used on the mainland have been given their own Unicode numbers that are different from the Unicode numbers of their nearest equivalent characters in Japan etc. This leads to a problem.
Imagine a typical text in Simplified Chinese. Some of the characters have their own special code numbers that occur only in Simplified texts, whereas other characters have code numbers that can also occur in Japanese and other texts. Now imagine that a rendering system such as Pango is going to render this text. Pango takes the text character by character, and tries to find each character in the available fonts. Suppose it first of all finds a character that occurs in a Japanese font. There is a Japanese font on the system, so it takes that character from the Japanese font and renders it. But then the next character is not in the Japanese font because it is a special Simplified character that occurs only in the Simplified Chinese font. There is a Simplified Chinese font on the system too, so off it goes and renders that second character from the Simplified Chinese font.
The problem is that some of the characters (the ones that have special Simplified-only codepoints) will be rendered from the Simplified Chinese font, but others (the ones that share codepoints with Japanese etc) will be rendered from another font. If the fonts happen to have the exact same style and weight etc, then nobody will notice, but usually there are noticeable differences in style between the fonts, so this mixing of fonts, with some characters taken from one font and others taken from another, can make the display of Simplified Chinese on Linux look very un-professional.
Many users can tolerate this situation on the screen, but printing is a different matter. For example, the music typesetting system GNU Lilypond uses Pango to render the text of lyrics etc, and if you want to produce beautiful-looking copy with Simplified Chinese text, this character-by-character mixing of fonts could be a showstopper.
The Pango renderer does have facilities for application programmers to specify the language and therefore influence the choice of fonts, such as by calling pango_context_set_language() or by using Pango markup. However, this is small consolation for those using applications that do not expose this functionality to the user.
One thing you can do is edit your ~/.fonts.conf file to tell the system which fonts you prefer. This will affect all applications, so you may need to change it back when you want to see Traditional Chinese or Japanese etc. Below is a ~/.fonts.conf file which specifies the Arphic Simplified Chinese font as a preference (Debian/Ubuntu package ttf-arphic-gbsn00lp). It also specifies DejaVu for the Latin fonts, along with Gandhari, a font which is good for Pinyin markup.
<fontconfig>
<alias>
<family>serif</family>
<prefer>
<family>gandhari unicode</family>
<family>dejavu serif</family>
<family>ar pl sungtil gb</family>
</prefer>
</alias>
<alias>
<family>sans-serif</family>
<prefer>
<family>gandhari unicode</family>
<family>dejavu sans</family>
<family>ar pl sungtil gb</family>
</prefer>
</alias>
<alias>
<family>monospace</family>
<prefer>
<family>dejavu sans mono</family>
<family>ar pl sungtil gb</family>
</prefer>
</alias>
</fontconfig>
On a Debian or Ubuntu system, most of the families can be seen by looking at /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType/fonts.dir. To install Gandhari, download it from Andrew Glass's site, visit fonts:/ in Konqueror (package konqueror if you don't already have KDE) and drag gur.ttf into that folder.
This approach does not solve everything; for example, the on-screen display of Chinese in Tk applications might still be inconsistent (it does not seem possible to set a preferred order of fonts in Tk/X11; you can set one preferred font but there's no obvious way to control what it falls back on when displaying characters that are not available in that font). However, the use of ~/.fonts.conf should at least help with applications that you are likely to use for printing.
Talkback: Discuss this article with The Answer Gang
![[BIO]](../gx/authors/brownss.jpg)
Silas Brown is a legally blind computer scientist based in Cambridge UK. He has been using heavily-customised versions of Debian Linux since 1999.
Keywords: PXE Kickstart Linux
I had a friend, Adam, who passed away far too young. Adam seemed to be competent at an amazing number of activities, mostly self-taught. He built himself a house almost single-handedly.
He once shared with me the secret of building. Adam would say that it was not possible to build a house without making mistakes. The secret, he said, was hiding your mistakes. For example, builders can never get the floor and walls to fit together exactly. So they invented skirting boards; for walls and ceiling: mouldings and cornices.
Subsequently, somebody must have decided to extend the concept and they became decorative. Now that's really hiding your mistakes - pretend that they are a feature!
Perhaps because I know (or think I know) more than the average bear about Linux, I get "adventurous", particularly on my projects at home. When I extend my activities beyond my areas of competence I frequently make mistakes. That's not intrinsically a bad thing: you have to break eggs to make an omelette. But it's important to be able to recover from mistakes.
As a result, I've adapted Adam's advice to fields which interest me.
My project for last week was to build a server. Sounds simple. What could possibly go wrong? Watch and learn.
I run (the fairly old) Fedora Core 5 on my desktop, so I thought I'd install the latest Fedora on my server. I say "server" but it's actually a Pentium II, one of a pair a friend was throwing out. The other one replaced my aging (12-year-old Pentium 1 100) firewall.
But my project is more about concept than commercial-strength mail and web-serving, so grunt is not important to me.
Installing from CDs holds no fascination for me: I've done it so many times, the novelty has worn off. I decided I'd do a "Look, Ma! No hands!" install.
The Pentium II was a Compaq which must have been quite leading edge when it was purchased. Of particular interest was the fact that it had on-board hardware PXE capability. I conceived a plan which would see the machine perform a network boot, then perform a network install driven by a kickstart file. Everything would be automatic: partitioning, timezone, system language, configuration of 2 NICs, root password - the lot.
As an exercise in building a single server, this approach is not very efficient. However, as an exercise in building multiple servers, it is excellent. One of the dangers of having a human (me) perform the same task repeatedly is that boredom sets in and the human (me) loses concentration, leading to mistakes. It is worth investing effort into automating the process. Although I currently have neither the hardware nor the need to build a farm of servers, I would like to learn the technique for the future. So for me this is a learning exercise.
The following sections are written from the perspective of the machine to be built, which I'll call the target machine.
To provide some context, here is an overview of a typical install. Bear in mind that my old Compaq only has a CD drive (not DVD).
1. Turn on the target machine. 2. Insert the first install CD. Machine boots from the removable disk, loading the install environment. 3. Proceed through many screens answering questions about partitioning, timezone, system language, etc. The machine begins to install from the first CD. After a while, the machine asks for the second CD. 4. Change CDs. 5. Repeat until all 6 install CDs have been processed.
Note that the numbered steps involve human intervention. The indented sentences describe the target machine's behaviour.
Here is my rough plan for how things were supposed to go.
1. Turn on the target machine. Machine starts the boot process, asks the network for an IP address, then downloads the install environment. The install environment uses the kickstart file to guide the rest of the install. It obtains the install files from the network. No further intervention is required.
And here's what happened the first time.
1. I turned on the target machine. The machine booted, asked the network for an IP address, then downloaded the install environment. The install environment used the kickstart file to guide the rest of the install.
However, it was here that I discovered the first of several mistakes.
I had not defined the partitioning information completely, so the machine stopped to ask for clarification.
I guess I could have gone back and fixed the problem and restarted, but it occurred to me that I might have made other mistakes. I decided to answer the partitioning questions and proceed.
Remarkably, that was the only problem. Or so it seemed. The install proceeded. It seemed to take a very long time, but I was not too surprised by this: the target machine is after all a Pentium II.
It is at this stage that a kickstart install starts to pay back. The install may take a long time, but I don't have to stay around to change CDs. There are always other tasks to be done, and in this case I left the room for several hours.
On my return, the machine was again asking for clarification about partitioning!
What had happened?!
There is a temptation when things go wrong to start doing. This is almost invariably bad. It often leads to regretting! It takes some discipline and experience to stop and think.
"Doing" quite often leads to activities which modify the "crime scene". One then gets to a point where one wants to know what something looked like before the "doing" began. Too late.
"Thinking" doesn't prevent all mistakes, only some.
What had gone wrong?
How does one even begin to think in such a case? Well, every activity typically has a beginning, a middle and an end.
The install had started ok. I'd then had the partitioning problem. After that, the install seemed to proceed as expected. So beginning and middle seemed to be off the hook. I concluded that I should look at the end.
What should have happened after the install completed?
I remembered that one of the kickstart parameters related to this. I had requested that the machine reboot after install.
In an install from CD, once the install environment has processed the last CD, it asks the user to remove any install media from the drive. By default, when the machine subsequently reboots, it takes its data from the recently installed hard drive.
However, in my case, on reboot, the machine performed its PXE boot - which restarted the install process. Thank heavens for my partitioning mistake. Without it, the install would have proceeded as before, come to the end - and started all over again. It might still be at it now, repeatedly booting and installing!
I guess after a few days I might have grown suspicious.
It's an ill wind that blows no good; and sometimes, as in this case, a mistake can be your friend.
My first mistake was unexceptional. To install FC10 I needed to get a number of steps right. As we shall soon see, I've glossed over some of the steps to tell this part of the story simply. It is unsurprising that I did not get everything right the first time. (I would have been very surprised had everything just worked!)
The second mistake was much more interesting. It relates to the case where an action is repeated but the conditions change between the first and a later occurrence of that action. It's the sort of problem that is common in loops.
In this case, the first time I booted the machine, I wanted it to use PXE; the second time, I didn't. I hadn't realised that I had wanted different behaviours.
Even stated like this, it was not clear to me what to do next. At first sight, the simple solution is to request that the install environment not reboot at the end of the install. But, as we shall see in the next part, that solution is somewhat short-sighted.
When something goes wrong, don't rush to act. Stop, think and plan before acting.
Talkback: Discuss this article with The Answer Gang
![[BIO]](../gx/authors/grebler.jpg)
Henry was born in Germany in 1946, migrating to Australia in 1950. In his childhood, he taught himself to take apart the family radio and put it back together again - with very few parts left over.
After ignominiously flunking out of Medicine (best result: a sup in Biochemistry - which he flunked), he switched to Computation, the name given to the nascent field which would become Computer Science. His early computer experience includes relics such as punch cards, paper tape and mag tape.
He has spent his days working with computers, mostly for computer manufacturers or software developers. It is his darkest secret that he has been paid to do the sorts of things he would have paid money to be allowed to do. Just don't tell any of his employers.
He has used Linux as his personal home desktop since the family got its first PC in 1996. Back then, when the family shared the one PC, it was a dual-boot Windows/Slackware setup. Now that each member has his/her own computer, Henry somehow survives in a purely Linux world.
He lives in a suburb of Melbourne, Australia.
Dogs are the leaders of the planet. If you see two life forms, one of them's making a poop, the other one's carrying it for him, who would you assume is in charge? - Jerry Seinfeld
I'm not in the habit of bagging Microsoft. No, really. Oh, all right, I am.
Have a look at this picture. I swear that it's for real. It popped up
on more than one occasion on my company laptop. I don't know
whether it came from XP, some other piece of Microsoft software, or
some other software altogether. But it just beggars belief!
I just looked on my Linux desktop: uptime - 126 days. I don't even think that's a particularly impressive number.
All right, my desktop is not a laptop. So?!
And what is it with turning it off for at least 15 minutes? Nap time?! Imagine if they used this technology in, oh, fridges! Ships! Rockets to the moon! Military equipment! I'm sorry. Your war is important to us. To maintain you military lethality and extend its life, please shut down your war for at least 15 minutes.
Ok, I've arrived at the office. 7:45 am. Nice and early. Good. I'm a morning person. It's great to come in before most of the others get in. Gives me time to do some work that requires concentration before the distraction level ramps up.
Wonder why they put us into these open-plan areas. Do they really think they are saving money? Or is it to do with pecking order? They're paying me a shirt-load of money and yet I can hear the guy way over there on the phone to one of his customers. Try to concentrate. Crumbs, he's loud! They used to call me loud, but now I'm in the presence of a real master.
Get out the laptop. Turn it on. Get some real work done. Come on, boot up already! First password. Second password. Come on, XP! Finally: login password. What's it doing?
What's this? Virus scanner. Updating signatures. No response for me. Come on!
Why did I turn on this #@$#&^ing computer? Who's the human? WTF is going on here?
What now? Some of my software is out of date. I don't even use that software. Why would I care if it is out of date? I just wanna do some work, for crisake! Who's in charge here?
Start up Adobe. Wait for it to update something. This is crazy!
There seems to be this view that I'm the laptop's lackey. I'm here to service its requirements. It gives the orders; I follow. Its needs are more important; mine are secondary.
Are we for real? The lunatics are in charge of the asylum!
Are we saying that we have reached the pinnacle of technical engineering? That this is the best we can do?
Well, we know that there are various forms of improvement. On my Linux systems, I run no regular antivirus software. Or antispyware. Or any other sort of anti-malware software. (The only reason I have any antivirus software at all is so that I can scan my teenage son's Microsoft computer.)
The argument is that if Linux were as ubiquitous as Microsoft, then the malware generators would attack Linux. As it is, they're just after the low-hanging fruit.
All right. Let's accept that. For the sake of argument, let's assume that it is necessary to update things on a Microsoft PC. Fine. Can't we do that in a less obnoxious way?
Let's start with some low-hanging fruit of our own.
Sometimes, I reboot my computer repeatedly in a short space of time. (Never mind why; it can happen.) Is it really asking too much of the cretinous antivirus software to keep track of when it was last invoked (or when I last rebooted), and to not even run if less than X hours have elapsed?! And, ok, if some user actually wants every one of the three reboots in the last hour to be followed by a scan, let him/her configure the software to do that. Hell, I don't even mind if it's the default. Just give me the option to do what I want.
That's what's really missing from a lot of software which runs on Microsoft platforms: user choice. The technology which should be the most liberating has been engineered to be antidemocratic. And down-right irritating.
Ok. There are times during the day when I get up from my seat and walk away from my desk. Corporation rules say that I'm supposed to screenlock the laptop when I leave. Even if I don't, after a few minutes, the laptop does it anyway. Hey, you now know that I'm not at the computer. You know I'm not pressing keys. Wouldn't you think that now would be a good time to use up CPU cycles? I don't need them. I've gone off to make myself a coffee and discuss the weekend's footy with Alex.
I come back to the computer, enter my password. Don't abort what you were doing; pause it. You are a machine of infinite patience. (Or software of infinite patience.) Make a little note somewhere of where you were up to - and take a break. When I wander off next time, continue from where you left off.
Eventually, I head off for lunch. Hey, knock yourself out. Scan all the hard drives. As far as I care, make the whole machine walk across the desk from the vibrations. Do whatever you need to do. Just make sure you pause when I come back.
I'm at work for about 8 hours. Despite the little pop-up I showed before, I often turn the laptop on on Monday morning and don't turn it off again until Friday evening. In 24 hours, I only use 8-10 hours. You can have a minimum of 14 hours to do whatever you want. Just not on my watch.
What am I missing?
Is this so hard?
Is it beyond the capabilities of the mind of homo sapiens?
Perhaps it would not be possible 100% of the time. Sometimes, perhaps, something might need to be updated urgently. Personally, I think this most unlikely; but I entertain the possibility. But even if we can only arrange things so that I am undisturbed 90% of the time, that's still a huge improvement over what I see currently.
What is it with Microsoft and reboots?! Look at this:
Ok, it's a joke, a reductio ad absurdum of the state of play at Microsoft. Compare that to BeOS which was designed to never need rebooting. Perhaps it's asking a bit too much of Microsoft to never need rebooting.
But why does it need to reboot if I change my IP address? Or a host of other fairly innocuous changes I make to the system?
All day long, I am bombarded with stupid questions: Are you sure? Did you really mean that? Is it ok to reboot now? Do you want fries with that? No, wait, that last one was from some other company.
Now here's an idea: how about, when I ask to shutdown or reboot, I get a prompt? Wouldn't that be a perfect time to say something like this?
X Hey! All day long I've been nagging you to upgrade your software, all the while discouraging you with threats that such upgrades will almost certainly necessitate a reboot (which I call a "restart" because God forbid we should spook the bunnies who use our, um, software with technical terms). Well, I notice that you're about to shutdown or restart. Is now a good time to do those upgrades? I mean, do you have the time? (If you're heading home, you could take a pit-stop.) If now is not convenient, that's ok. I'll remind you next time. OK Cancel
First of all, isn't that a good idea? And second, just maybe, isn't that a more appropriate tone for a pile of silicon to adopt when addressing a superior life-form?
Perhaps I shouldn't be surprised. I've always thought that Microsoft treats its users with contempt. Surely the tone it adopts in its dialogues and messages supports that view. The smug little utterances like "Windows must shutdown your computer now". The higher power has spoken!
Of course, once it has completed the upgrade, when it's time for the, ahem, restart, it will take cognizance of my original selection. In particular, if I chose to shutdown, it will, at this time, shutdown.
When next I fire up my machine, it will proceed with any subsequent actions needed to complete the install. The idea is to maximise utility and convenience for me. Because I'm the human.
It's not about you; it's about me.
Turn on wife's machine. Windows 98. What's this? Disk scan? Scan disk? What is it doing? Just another way to slow me down.
In fairness, I should say that I haven't seen this on XP. But on my wife's Windows 98, it is very frequent.
I speculate that one day, quite some time ago, Bill was walking past a machine when it was booting into Linux. It was just at the point where it was doing an fsck.
"What's happening?" he asked.
The Linux user explained it to him.
Later, Bill summoned his Council of Technical Acolytes.
"Linux does something called fsck when it starts up. I want something like that for our next release of Windows."
"But, sire," offered one of his Council. "We don't need to do that. Our software works differently."
"Nonsense!" snapped Bill. "If Linux has it, we need a competitive story. We can't be open to criticism that we lack technical rigour.
"Further, we need it for another reason. It's important that our users get into the habit of obeying, preferably without thinking. If they ever get to thinking about our offering, well, ..." He trailed off, shaking his head to discard the unthinkable.
"Ok, gentlemen. Take notes. Here's how it's going to work. We will insist that they shutdown their machines in a certain way. It's important that they learn to follow orders. We'll detect if they obeyed. If they did, fine. The plan is working.
"If not, we'll use this - what did he call it? fsck? Stupid name - we'll use our version of fsck to punish the user."
Sensing resistance, he glared at the members of his Council. One or two, whose hands had been up, lowered them.
"Excellent! Now let's get down to the most important part of this exercise. fsck is such a stupid name. And anyway it's used by the competition. So what will we call it?"
Bill looked up encouragingly. It looked like he was really asking for suggestions. Several names were bandied around. Bill's countenance darkened.
"You don't seem to be getting it. None of your suggestions reach out to me. I guess I have to do everything myself. Now let me think ..."
The buzz of conversation died down as the The Great One cogitated, the silence broken only by an almost imperceptible sound, like gears grinding.
"I've got it!" he cried out in satisfaction. "SCANDISK."
He looked triumphantly around the Council Chamber.
"But, Sire, don't we already have something called DISKSCAN?"
"That's not the same thing at all!" snapped Bill. "Hands up anyone who thinks SCANDISK is like, um, DISKSCAN."
It doesn't actually do anything - unless you call moving the progress bar doing something. It's just there to punish recalcitrance.
And the ultimate irony? Soon afterwards, Linux adopted a journaled file system, eliminating the need to check the file system even after an unclean shutdown.
You're probably thinking to yourself that my wife and I get what we deserve. I'd agree with you about me; but my wife is what is so coyly referred to as collateral damage.
Do you want to know why we have to run SCANDISK all the time? Because Windows lies to us.
My wife finishes her work. She shuts down as prescribed - by clicking on Start (!) and then following the bouncing ball. Eventually a splash screen appears.
Windows is shutting down
She waits. She does other things. It's time for bed, so she performs her nightly ablutions. She likes to read before going to sleep. You're not really interested in my wife's nightly routine.
Hours later, the stupid machine still hasn't turned off. She calls me to investigate. I look at her screen.
Windows is shutting down
Bill's message looks back at me mockingly. How much can a koala bear? Enraged, I turn off the power at the wall.
Next time her machine is powered up, Bill gets his revenge. D'oh!
Talkback: Discuss this article with The Answer Gang
Henry was born in Germany in 1946, migrating to Australia in 1950. In his childhood, he taught himself to take apart the family radio and put it back together again - with very few parts left over.
After ignominiously flunking out of Medicine (best result: a sup in Biochemistry - which he flunked), he switched to Computation, the name given to the nascent field which would become Computer Science. His early computer experience includes relics such as punch cards, paper tape and mag tape.
He has spent his days working with computers, mostly for computer manufacturers or software developers. It is his darkest secret that he has been paid to do the sorts of things he would have paid money to be allowed to do. Just don't tell any of his employers.
He has used Linux as his personal home desktop since the family got its first PC in 1996. Back then, when the family shared the one PC, it was a dual-boot Windows/Slackware setup. Now that each member has his/her own computer, Henry somehow survives in a purely Linux world.
He lives in a suburb of Melbourne, Australia.
Disclaimer
Your author is writing this article when he has finished his latest task -
conducting numerous experiments processing a lot of data, and finally writing
the paper in just under five days. He wonders how he would do it without some
of the tools he will be talking about in this article, leaving aside the
pleasure of working on Linux systems. His views are biased and are meant to
be taken cum grano salis, but I hope you will give him a fair
chance.
My meeting with Linux happened accidentally 10 years ago, with the fun of wiping out and putting in a Linux distribution numerous times on my first computer without having any clue about how to work on it. Finally, after a lot of such iterations I finally settled in with it, working on it full time for the last 5 years. The accident that happened 10 years stood me in good stead both when I was working in a software company and now when I'm working in a research lab. In a research lab, we need to plot figures, run batch jobs remotely, perform scientific and numerical computing, and last but not least, publish results by writing papers. Just a few days before I wrote to the Editor-in-Chief about this article idea, I helped a Windows friend draw some plots (fixing them for publication) for her paper, and wondering all the while "Jeez, how do Windows users do it?" Most tools that I used were primarily for Linux. On Linux, things are so much easier. You can focus on the real job instead of bothering about setting up your tools. This article is therefore an unabashed claim about the coolness of working with these tools while you focus on your research. Some of these tools can be set up on a non-Linux system too; however, I shall address their usage on a Linux system only and make no attempt to cover any non-Linux issues. The primary focus here is to introduce my readers to these tools, and not to convert them to Linux. It would, in fact, be a good exercise to get all these tools that I talk about here on Windows!
In this article, I shall take a look at (in no particular order) getting up and on with Linux, using LaTex, gnuplot, GNU octave, Python scientific libraries, Beamer, and some other tools and libraries.
Before I start talking about using the scientific tools, I shall write a few lines about how can you set up a Linux system for yourself - if you don’t already have one, that is. While bare-metal Linux installations are the best case scenario, it may not always be possible because your primary work may be on Windows. In such cases, the best solution is to go for a virtual Linux installation. In my humble opinion, I have found using VirtualBox to be a breeze when you need to set up a virtual machine. Just download a Linux distribution like Ubuntu and install it using VirtualBox. Once you have the basic installation, take a little time to gain some hands on familiarity with the terminal, using the package manager to install packages, and using a text editor.
Here are some resources which can help you getting started:
LaTex is a document preparation system for high-quality typesetting and is widely used for writing scientific papers, technical manuals, and even creating presentations (more on this later). TeX Live is an easy way to get up and running with LaTex on your Linux system. A file written in LaTex is usually saved as a .tex file. Once you have created your .tex file, you will compile the file using the latex command, which gives you a dvi file, which you can convert to a PDF file using a tool like divpdf. You can also directly get a PDF output using the tool pdflatex.
You can edit LaTex in your text editor of choice. The widely used VIM and Emacs both support LaTex editing with syntax-highlighting, code-completion, etc.
The best way to get started with LaTex is by getting started. Start your text editor, type in some Latex and see the results. Here are some great resources to help you master it:
If you are doing experimental work, there is a fair chance that you are going to present your results in a graphical plot. Please welcome gnuplot. It is a command-line driven graphing utility for creating 2-dimensional as well as 3-dimensional graphical plots. It supports output to many file formats with eps and fig being of special interest to the researcher. The easiest way to install gnuplot is using the package manager of you Linux distribution.
Here are a couple of links to help you get started with gnuplot:
A cool way to use gnuplot is to use it to display continuously changing data that your program may be generating. For this example, let’s assume that you have a program written in the C programming language and you want to send the data its generating to gnuplot. The simplest way to do this is to use the popen system call. Please refer to this 2-cent tip published here sometime back.
There is also a ANSI C interface to gnuplot available here, but I haven’t personaly used.
GNU Octave (referred to as Octave from now on), is a software tool in- tended for numerical computations. Octave has extensive tools for solving common numerical linear algebra problems, finding the roots of nonlinear equations, integrating ordinary functions, manipulating polynomials, and in- tegrating ordinary differential and differential-algebraic equations. Octave is available for use on Unix systems - Linux, Solaris, Mac OSX and Windows. If your study or play involves numerical computations, Octave is for you. Octave has been designed with MATLAB compatibility in mind, so with care- ful design you could write scripts which run on both MATLAB and Octave. It is easily extensible and customizable via user-defined functions written in Octave’s own language, or using dynamically loaded modules written in C++, C, Fortran, or other languages.
Octave is most definitely available in the package repository of your Linux distribution and that is the easiest way to get it.
GNU Octave comes with an extensive documentation, which is the best place to get started.
In case you happen to like this author’s writings, he is currently writing an article series on Octave elswhere. Please refer the author’s blog post for the PDFs of the articles.
I shall not use this section of an article for Python language advocacy. Rather, I shall point you to some ways you may consider using Python in your research, if you are already using it, or think of using it in the future as just another of your many experiments. Python is already installed on most of the Linux distributions. Let us now see some of the ways you can use Python:
Using its math module, Python can be used as a very useful command-line scientific calculator. For example:
$ python Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import math >>> math.factorial(10) 3628800 >>>
As you can guess, you will have to import the math module, everytime you want to use it. You can avoid doing this by automatically importing it at startup:
import math
export PYTHONSTARTUP= $HOME/.pythonrc
Now, everytime you start Python interactively, you should have the math module already imported.
$ python Python 2.6.4rc1 (r264rc1:75270, Oct 10 2009, 02:40:56) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> math.pi 3.1415926535897931
The decimal module is another interesting module which provides support for decimal fixed point and floating point arithmetic. fractions is another interesting module, which can be used for rational number arithmetic.
The beauty is that, you do not need to know the Python programming language for using these modules. Just fire up the Python interpreter and import the module(s) you want to use, and call the appropriate function.
If you are already conversant with Python, then you would definitely like to know about some scientific libraries which you may find useful in your research:
You have done your experiments, sent in your findings and your findings have been accepted and now you are going to present your findings in a conference. You want to prepare some slides to give you company as you talk about your work. OpenOffice.org Impress, KOffice? How about taking a look at something different? You will surely like the results, I promise. Beamer is a LaTex class for making presentation slides. In LaTex parlance, it is just another Document Class. You shouldn’t even make an attempt to use Beamer, if you have no idea what LaTex is. However, if you already use LaTex, chances are you swear by it, then Beamer is for you.
The easiest way to install is to use the package manager on your Linux distribution. You will see that some more packages providing other extra LaTex classes need to be installed. Prominent among these are ’latex-xcolor’ and ’pgf’.
There is ofcourse the way to install the LaTex class manually. Its easy to do it, but not easier than the previous method. The LaTex Beamer user guide shows how to do it.
With the danger of being pulled up by the chief editor for self-publicity, the author would like to his article Typesetting Presentations with Beamer, published elsewhere and the PDF copy is available from here.
In this section, I shall talk about some of the other tools, which are useful for scientific computing that I know of:
One category of tools that I haven't talked about yet is some of the Linux utilities, the ones that ship along with the Linux system- Shell scripting, awk and sed. My knowledge of each is very limited, but I have a working knowledge of them all and I can get things done with them, which is what matters a lot of times.
Its often the case that we do not know of a great tool which would simplify our work as we go about our research work everyday. My hope as a novice researcher with this article is to acquaint the kind of software tools available in the Open Source world which are very useful, capable and fun to work with and we have looked at some of these today. I have made no attempt to be exhaustive here. There are a lot of other tools out there, which I haven’t yet used, so I have refrained from writing about them. I hope you have enjoyed reading this article as much as I have writing this one!
Credits are due to the folks on the mailing lists and the documentation of all the projects that I have talked today, because they are the source of my learning.
This document was translated from LATEX by HEVEA.