Suramya's Blog

Visit suramya.com Who am I?

December 14, 2014

Cleaning your Linux computer of cruft and duplicate data

When you use a computer and keep copying data forward everytime you upgrade or work with multiple systems it is easy to end up with multiple copies of the same file. I am very OCD about organizing my data and still I ended up with multiple copies of the same file in various locations. This could have happened because I was recovering data from a drive and needed a temp location to save the copy or forgot that I had saved the same file under another directory (because I changed my mind about how to classify the file). So this weekend I decided to clean up my system.

This was precipitated because after my last system reorg I didn’t have a working backup strategy and needed to get my backups working again. Basically I had moved 3 drives to another server and installed a new drive on my primary system to serve as the Backup drive. Unfortunately this required me to format all these drives because they were originally part of a RAID array and I was breaking it. Once I got the drives setup I didn’t get the chance to copy the backup data to the new drive and re-enable the cron job that took the daily backup snapshots. (Mostly because I was busy with other stuff). Today when I started copying data to the new Backup drive I remembered reading about software that allowed you to search for duplicate data so thought I should try it out before copying data around. It is a good thing I did because I found a lot of duplicates and ended up freeing more than 2 GB of space. (Most of it was due to duplicate copies of ISO images and photos).

I used the following software to clean my system:

Both of them delete files but are designed for different use cases. So let’s look at them in a bit more detail.

FSlint

FSlint is designed to remove lint from your system and that lint can be duplicate files, broken links, empty directories and other cruft that accumulates when a system is in constant use. Installing it is quite easy, on Debian you just need to run the following command as root

apt-get install fslint

Once the software is installed, you can either use the GUI interface or run it from the command line. I used the GUI version because it was easier to visualize the data when seen in a graphical form (Yes I did say that. I am not anti-GUI, I just like CLI more for most tasks). Using the software was as easy as selecting the path to search and then clicking on Find. After the scan completes you get a list of all duplicates along with the path and you can choose to ignore, delete all copies or delete all except one. You need to be a bit careful when you delete because some files might need to be in more than one location. One example for this situation is DLL files installed under Wine, I found multiple copies of the same DLL under different directories and I would have really messed up my install if I had blindly deleted all duplicates.

Flossmanuals.net has a nice FSlint manual that explains all the other options you can use. Check it out if you want to use some of the advanced features. Just ensure that you have a good backup before you start deleting files and don’t blame me when you mess up your system without a working backup.

BleachBit

BleachBit is designed for the privacy conscious user and allows you to get rid of Cache, cookies, Internet history, temporary files, logs etc in a quick and easy way. You also have the option to ensure that the data deleted is really gone by overwriting the file with random data. Obviously this takes time but if you need to ensure data deletion then it is very useful. Bleachbit works on both Windows and Linux and is quite easy to install and use (at least on Linux, I didn’t try it on Windows). The command to install it on Debian is:

apt-get install bleachbit

The usage also is very simple, you just run the software and tick the boxes relevant to the clutter that you want gone and BleachBit will delete it. It does give you a preview of the files it found so that you can decide if you actually want to delete the stuff it identifies before you delete it.

Well this is all for now. Will write more later.

Thanks to How to Sort and Remove Duplicate Photos in Linux for pointing me towards FSlint and Ten Linux freeware apps to feed your penguin for pointing me towards BleachBit.

– Suramya

March 8, 2013

Citrix on Raspberry Pi: Updated instructions and working download image

Filed under: Knowledgebase,Linux/Unix Related,Techie Stuff,Tutorials — Suramya @ 2:36 PM

A couple of folks have reached out to me via email/messages to tell me that the instructions I posted at the Raspberry Pi forums don’t work with the latest version of Rhaspbian. Basically the problem is that the latest version of the Citrix client is not compiled for the armhf architecture (Which is what the latest version of Rhaspbian OS is compiled for), so you need to download and install the armel version of the OS (‘Soft-float Debian “wheezy”’) from http://www.raspberrypi.org/downloads.

To make life simpler for people I have created a snapshot of my Pi install with Citrix installed and configured. You can download it from here. The image is 4GB so you will need to use a card of atleast that size when using this image. Follow these steps to install the image to an SD card in Linux:

  • Download the image file from the mirror (Approx 1GB compressed)
  • Unzip the file using the command
  • unzip Raspberry_Citrix.img.zip
  • Find out what the partition the SD card you are using has been assigned running the following command as root
  • fdisk -l

    Once you run the command, you will get an output that will show you all the disks attached to your system, look for the entry that corresponds to your card. In my case it looked like this:

     Disk /dev/sde: 3965 MB, 3965190144 bytes
    122 heads, 62 sectors/track, 1023 cylinders, total 7744512 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00016187
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sde1            8192      122879       57344    c  W95 FAT32 (LBA)
    /dev/sde2          122880     7744511     3810816   83  Linux
    
  • So now we know that the card is at /dev/sde. All we have to do is write the image to the card and that is done using the following command. Make sure you replace the /dev/sde with the correct path otherwise you will end up destroying all data on the wrong drive.
  • dd if=Raspberry_Citrix.img of=/dev/sde bs=4096

    You will not see any output on the screen so don’t worry about it, just let it run and wait for the process to complete as it will take some time because of the amount of data being written. Once the process completes you can eject the card and if all went well you should be able to boot the Raspberry Pi from the card.

The login password for this image is root/password, please do change the password if you use the image. Let me know if you have any questions or have an issue using this image.

Update (3/28/2013): Adding instructions on how to write the image when using windows. (Please note that I haven’t tested the windows instructions as I don’t have a windows machine. Use at your own risk)

Once you download the zip file from the mirror, right-click on it and select extract (I think that’s what it says, but I don’t have a windows machine so can’t confirm). After the image is extracted you will have a file called Raspberry_citrix.img on your computer. Now follow these steps to write the image to an SD card (Instructions taken from eLinux)

  • Insert the SD card into your SD card reader and check what drive letter it was assigned. You can easily see the drive letter (for example G:) by looking in the left column of Windows Explorer. If the card is not new, you should format it and make sure there is only one partition (FAT32 is a good choice); otherwise Win32DiskImager can make corrupt your SD card!
  • Download the Win32DiskImager utility. The download links are on the right hand side of the page, you want the binary zip.
  • Extract the executable from the zip file and run the Win32DiskImager utility. You should run the utility as Administrator!
  • Select the Raspberry_citrix.img image file you extracted earlier
  • Select the drive letter of the SD card in the device box. Be careful to select the correct drive; if you get the wrong one you can destroy your data on the computer’s hard disk!
  • Click Write and wait for the write to complete.
  • Exit the imager and eject the SD card.

You should also go through the Basic setup guide for Raspberry Pi. Hope this helps.

– Suramya

March 3, 2012

Configuring Dual monitors in Debian

Filed under: Knowledgebase,Linux/Unix Related,Techie Stuff,Tutorials — Suramya @ 12:01 AM

[Update 8th Aug 2012: This is an older method of setting up the monitor kept for historic reasons. In the newer version of KDE the process is a lot simpler, please refer to this post for the updated steps – Suramya.

Recently I went ahead and bought two new Dell 20″ monitors for my home system as I had gotten used to working with two monitors at work and wanted the same experience at home as well. The problem started because initially I tried installing another graphics card and hooking up the second monitor to that card using VGA. For some reason maybe because I was to tired and wasn’t thinking clearly, I couldn’t get both the cards to work at the same time. I would get one or the other but not both. To make things even more fun, the monitors are 16:9 aspect ratio and when I used the Opensource driver the only resolution with that aspect ratio I would get was 1600×900 which was too small and the fonts looked kind of jagged at that resolution.

Since I was going to be out of town and was planning on switching to DVI cables anyways I left the system like that (after spending a bit of time experimenting) and left. Once I got back I ordered DVI cables and finally managed to get the dual monitor setup working after spending about an hour one the issue. Below is the sequence I followed to get stuff to work (documenting this so that if I ever have to do this again I have a record of what I did):

  • Removed the second video card to reduce complexity. Might add it back later if required, or if I want to hook my old monitor as a third display.
  • Connected both monitors to the onboard ATI Radeon HD 4250 card, one over DVI and the second using VGA
  • Removed the Proprietary ATI and nVidia drivers (both installed in my previous attempts to get this working). Instructions here
  • Restarted X
  • Installed Catalyst (a.k.a fglrx) a proprietary “blob” (closed source binary) driver, using the following command:
  • apt-get install fglrx-atieventsd fglrx-control  fglrx-driver fglrx-glx fglrx-modules-dkms glx-alternative-fglrx libfglrx libgl1-fglrx-glx libxvbaw

Once the driver was installed I restarted X once again and got both monitors working, but the second monitor’s display was a clone of the first one which is not what I wanted so I had to do some more digging and finally managed to fix that using the following steps:

  • Open a terminal/Command Prompt
  • Disable access control so that clients can connect from any host by issuing the following command as a regular user
  • xhost +

    This is required so that we can start a GUI command from a root shell. If we don’t do this you will get an error similar to the following in the next step:

    No protocol specified
    No protocol specified
    amdcccle: cannot connect to X server :0
  • Run ‘Ati Catalyst Control Center’ as root
  • sudo amdcccle
  • Click on ‘Display Manager’ and configure your monitors (Resolution, location etc)
  • Click on ‘Display Options’ -> ‘Xinerama’ and enable ‘Xinerama’
  • There is a bug in the display manager that prevents it from saving any changes if the xorg.conf file exists, to fix:

  • Run the following command as root:
  • mv /etc/X11/xorg.conf /etc/X11/xorg.conf_original
  • Click ‘Apply’ in the Catalyst Control Center
  • Restart X

That’s it. Once I did all that, my dual monitor setup started working without issues. Well… mostly. For some reason my desktop effects have stopped working (Transparent/Translucent windows etc) but I am not going to worry about it for now. That’s a battle for another day, maybe over the weekend.

Please note, that setting up Dual monitors usually is not this complicated in Linux. When I hooked up my TV to this same system I didn’t have to make any changes to get it to work. In this case since I was fiddling around I had to fist fix the mess I made before I was able to get this to work properly.

For those of you who are interested, the final xorg.conf that the above steps created is listed below:

Section "ServerLayout"
        Identifier     "amdcccle Layout"
        Screen      0  "amdcccle-Screen[1]-0" 0 0
        Screen         "amdcccle-Screen[1]-1" 1440 0
EndSection

Section "ServerFlags"
        Option      "Xinerama" "on"
EndSection

Section "Monitor"
        Identifier   "0-CRT1"
        Option      "VendorName" "ATI Proprietary Driver"
        Option      "ModelName" "Generic Autodetecting Monitor"
        Option      "DPMS" "true"
        Option      "PreferredMode" "1440x900"
        Option      "TargetRefresh" "60"
        Option      "Position" "0 0"
        Option      "Rotate" "normal"
        Option      "Disable" "false"
EndSection

Section "Monitor"
        Identifier   "0-DFP1"
        Option      "VendorName" "ATI Proprietary Driver"
        Option      "ModelName" "Generic Autodetecting Monitor"
        Option      "DPMS" "true"
        Option      "PreferredMode" "1440x900"
        Option      "TargetRefresh" "60"
        Option      "Position" "0 0"
        Option      "Rotate" "normal"
        Option      "Disable" "false"
EndSection

Section "Device"
        Identifier  "amdcccle-Device[1]-0"
        Driver      "fglrx"
        Option      "Monitor-DFP1" "0-DFP1"
        BusID       "PCI:1:5:0"
EndSection

Section "Device"
        Identifier  "amdcccle-Device[1]-1"
        Driver      "fglrx"
        Option      "Monitor-CRT1" "0-CRT1"
        BusID       "PCI:1:5:0"
        Screen      1
EndSection

Section "Screen"
        Identifier "amdcccle-Screen[1]-0"
        Device     "amdcccle-Device[1]-0"
        DefaultDepth     24
        SubSection "Display"
                Viewport   0 0
                Depth     24
        EndSubSection
EndSection

Section "Screen"
        Identifier "amdcccle-Screen[1]-1"
        Device     "amdcccle-Device[1]-1"
        DefaultDepth     24
        SubSection "Display"
                Viewport   0 0
                Depth     24
        EndSubSection
EndSection

Hope all this made sense and helps someone. If not feel free to ask questions.

– Suramya

October 26, 2011

Connecting a WordPress blog to Facebook

Filed under: Computer Software,Linux/Unix Related,Techie Stuff,Tutorials — Suramya @ 5:01 PM

Over the past few months I have been trying to connect my blog to my Facebook account so that whenever a post is made on the blog it automatically gets posted on Facebook to varying degree’s of success. Most of the attempts would work for a while and then stop. I even tried using some of the existing plugins for WordPress but since they required a developer account (which needs a valid phone no or CC#) and for some reason I never get the validation code on my cell I was never able to get them to work.

Then I found an article on Linux Magazine on a Command Line interface for Facebook and decided to build on top of that to get the linkage working. Now this is a very hackey way and is not at all elegant or anything but it gets the work done which is what I wanted, so I am good. :) All the work was done in about 2 hours including testing so that should tell you something on its own.

I had to install this on my local system since my webhost didn’t have all the per-requisites to get this to work. That and the fact that I can’t connect to my MySQL db’s from a machine outside of my hosting provider is why this convoluted method was created. The steps I followed to get this to work are as follows.

Install Facebook Commandline

To install Facebook Commandline, follow the instructions on their site.

Authenticate the Application to be able to talk to Facebook

For some reason there was a difference when I run the application from the commandline and when I run it from the web, in as to where the preferences file and the session details were saved, so all the steps have to be done either from the command line or via the web, you can’t interchange the two.

Creating a Web interface for the FBCMD

Since I wanted to be able to get data from WordPress and pass it on to FBCMD I created a new PHP page called run.php that basically pulls the data from WordPress and then passes it to FBCMD as command line parameters. I know that using passthru is probably not very secure and I should have modified the FBCMD file to accept parameters as a URL but didn’t want to spend that much time trying to get this to work. (Hey! I told you it was a quick and dirty ‘fix’).

The contents of this file are very simple:

error_reporting(E_ALL);
$handle = fopen('http://www.suramya.com/blog/LatestPost.php', 'r');
$current = fopen('/var/www/fbcmd/latest.dat', 'r');
$current_id = fgets($current, 4096);
fclose ($current);

if ($handle) 
{
 $ID = fgets($handle, 4096);
 $link = fgets($handle, 4096);
 $title = fgets($handle, 4096);
 $content = fgets($handle, 596);
 $content = chunk_split(htmlspecialchars(strip_tags($content)), 500) . "...";

 if($ID != $current_id)
 {
  // If we have a new post then call FBCMD to make a post
  $command = '/usr/bin/php /var/www/fbcmd/lib/fbcmd/fbcmd.php POST " " "' . chop($title) . '" "' . 
              chop($link) . '" "' . $content . '"';
  passthru ($command);
  // Write the new PostID to a file
  $current = fopen('/var/www/fbcmd/latest.dat', 'w');
  fputs($current, $ID);
  fclose($current);
 }
}

The file basically calls ‘LatestPost.php’ and gets the latest post details on the blog(see below for details), then it checks if the post made is newer than the last post processed and if so it proceeds to post to Facebook using FBCMD.

‘LatestPost.php’ file looks like this:

< ?php

define('WP_USE_THEMES', true);
require_once( dirname(__FILE__) . '/wp-load.php' );

$month = $_GET['month'];
$year = $_GET['year'];

$args = array( 'numberposts' => 1);
$myposts = get_posts( $args );

//print_r($myposts);

foreach( $myposts as $post ) : setup_postdata($post); 
echo $post->ID . "\n";
the_permalink();
echo "\n";
the_title();
echo "\n";
the_content();
endforeach; ?>

This file need to be put on the server in the WordPress Root directory and when called returns an output in the following format:

Post ID
Post Link
Post Title
Post Content

Once all this is done and the FBCMD has access to post to Facebook all we need is a cron job to run on a frequent basis to run the code. So I created a shell script that contains the following line and have it run every 15 mins.

/usr/bin/curl http://localhost/fbcmd/run.php > /tmp/FBPost.out

That’s it. So far it looks like its working great and if this post shows up on my FB wall then all is well. If not, then its back to the code to see what went wrong this time.

– Suramya

September 5, 2011

Getting RTL8111/8168B PCI Express Gigabit Ethernet controller to work in Debian 6

Filed under: Knowledgebase,Linux/Unix Related,Techie Stuff,Tutorials — Suramya @ 11:28 PM

Once I got Debian 6 installed on my server I needed to connect it to the internet to download updates etc, however my network card wasn’t being detected correctly so I had to perform the following steps to get it to work correctly:

  • Download the latest Linux drivers for the RTL8111 Chipset from the Realtek site on a computer that can connect to the Internet.
  • Copy the file over to your new system via USB or smoke signals
  • Login as root to the server
  • Identify the kernel version that you are running, using the following command:
  • uname -a

    It will give you a result like the following:

    Linux StarKnight 2.6.30-2-686 #1 SMP Sat Aug 27 16:41:03 UTC 2011 i686 GNU/Linux

    Now you need to install the kernel source code for this version on the server. First we need to find the package name of the kernel source code, we do that by running the following command:

    apt-cache search linux |grep header |grep 2.6 

    If you have a 2.4.x kernel, replace grep 2.6 with grep 2.4. Once you have the package name install it using the following command as root:

    apt-get install linux-headers-2.6.30-2-686

    Make sure you replace linux-headers-2.6.30-2-686 with the package name you got.

    Once we have the kernel source installed we can go ahead and install the driver using the following commands:

    tar -jxvf r8168-8.025.00.tar.bz
    cd r8168-8.025.00
    ./autorun.sh 
    

    This will compile the drive and install it. I didn’t get any errors when I ran it, but if you do get errors try searching for the error message on Google, it usually provides a solution.

    After I installed the driver I tried initializing my network but kept getting the following error message:

    StarKnight:~# ifdown eth0
    ifdown: interface eth0 not configured
    StarKnight:~# ifup eth0
    Ignoring unknown interface eth0=eth0.
    

    Fixing it was fairly simple though, all I had to do was edit the /etc/network/interfaces file and add the following lines to it (This assumes you are using DHCP):

    auto eth0
    iface eth0 inet dhcp
    

    Once you add the lines, you can try starting the network again using the command:

    ifup eth0

    If all went well, you will be assigned an IP address and will now be able to successfully browse the net.

    Hope this helped.

    – Suramya

    February 5, 2010

    Learn to use search effectively in Linux using grep

    Filed under: Linux/Unix Related,Techie Stuff,Tutorials — Suramya @ 11:59 PM

    grep is a really powerful tool that allows you to search for specific string/data within a given text. This text can be a list of files or the content of a given file, even a list of programs running. Basically it allows you to filter the required text from background noise.

    As you can imagine it is a very powerful tool but most people don’t really learn to use it well. Zahid Irfan wrote a very nice blog post on ‘Why grep almost never yields something productive‘ for new Linux users and has some great examples that explain grep usage quite well for all users both new and experienced

    Check it out.

    – Suramya

    July 6, 2006

    IP subnetting made easy

    Filed under: Knowledgebase,Tutorials — Suramya @ 10:06 AM

    Good tutorial on IP Subnetting. Now it makes a bit more sense to me than it used to…

    Check it out: IP subnetting made easy

    – Suramya

    March 8, 2006

    An Introduction to awk

    Filed under: Linux/Unix Related,Tutorials — Suramya @ 4:08 PM

    A good introductory tutorial on awk. I have heard a lot about it, even used it a couple of times in shell scripts but never programmed using it. Check this article to learn more about it.

    Synopsis:

    The awk programming language often gets overlooked for Perl, which is a more capable language. Out in the real world, however awk is found even more ubiquitously than Perl. It also has a smaller learning curve than Perl does, and awk can be used almost everywhere in system monitoring scripts, where efficiency is key. This brief tutorial is designed to help you get started in awk programming.

    The awk language is a small, C-style language designed for the processing of regularly formatted text. This usually includes database dumps and system log files. It’s built around regular expressions and pattern handling, much like Perl is. In fact, Perl is considered to be a grandchild of the awk language.

    Complete Article: An Introduction to awk

    – Suramya

    January 24, 2006

    Mastering Ajax: Make asynchronous requests with JavaScript and Ajax

    Filed under: Tutorials — Suramya @ 3:49 PM

    Good tutorial by IBM that explains how to make asynchronous requests with JavaScript and Ajax.

    Abstract:

    Most Web applications use a request/response model that gets an entire HTML page from the server. The result is a back-and-forth that usually involves clicking a button, waiting for the server, clicking another button, and then waiting some more. With Ajax and the XMLHttpRequest object, you can use a request/response model that never leaves users waiting for a server to respond. In this article, Brett McLaughlin shows you how to create XMLHttpRequest instances in a cross-browser way, construct and send requests, and respond to the server.

    Complete Article: Make asynchronous requests with JavaScript and Ajax

    - Suramya

    January 22, 2006

    Extract a tar archive relative to the current directory

    Filed under: Linux/Unix Related,Tutorials — Suramya @ 9:02 PM

    How to extract a tar that looks like relative to the current directory:

    $ tar -tvf MyArchive.tar
    -rw-r–r– 100/100 58239140 Jan 20 05:17 2006 /home/suramya/bin/File1.dat
    -rw-r–r– 100/100 56660587 Jan 19 16:07 2006 /home/suramya/bin/File2.dat

    I don’t want to extract these files to /home/suramya/bin I want them relative to my current directory. The GNU tar program does this automatically but the Solaris version doesn’t. To fix this problem use the following command:

    pax -r -s ',^/,,' -f MyArchive.tar

    This will extract MyArchive.tar in the current directory (By creating a home/suramya/bin directory in the current directory)

    Tip courtesy: Solaris IAOQ

    - Suramya

    Older Posts »

    Powered by WordPress