Suramya's Blog : Welcome to my crazy life…

September 3, 2018

Software hack to keep my speaker powered on

Filed under: Computer Hardware,Linux/Unix Related,Techie Stuff,Tutorials — Suramya @ 6:37 PM

A little while ago I bought a new klipsch speaker as my previous one was starting to die and I love it except for a minor irritation. The speaker has builtin power saving tech that powers it off if its not used for a certain period of time and that means that I have to physically power it on every time I wanted to listen to music which was annoying. As I would invariably be comfortably seated and start the music before remembering that I needed to power it on. Also, I could not start the music from my phone whenever I felt like as the speaker was powered off and I would have to walk to the room to power it on.

After living with the irritation for a while I finally decided to do something about it and whipped up a small script that checks if any music/audio is already playing on the system and if not it plays a 1 second mp3 of an ultrasonic beep. This forces the system to keep the speaker on and I love it as now I can start the music first thing in the morning while lazing in bed. 🙂

The script requires the mpg123 to be installed and you can install it on a Debian system by issuing the following command:

apt-get install mpg123

The Script itself is only 4 lines long:

#!/bin/bash

if ! grep RUNNING /proc/asound/card*/pcm*/sub*/status &> /dev/null ; then
    /usr/bin/mpg123 -q /home/suramya/bin/KeepSpeakerOn.mp3 &> /dev/null
fi

What it does is to check if any of the PCM soundcards have a status of RUNNING and if not it plays the mp3. I have a cron job scheduled to run the script every one min:

XDG_RUNTIME_DIR=/run/user/1000

* * * * * /home/suramya/bin/KeepSpeakerOn.sh 

One interesting issue I hit during the initial testing was that the mpg123 application kept segfaulting whenever I initiated it from the Cron but it would work fine if I ran the same command from the command prompt. The error I got in the logs was:

High Performance MPEG 1.0/2.0/2.5 Audio Player for Layers 1, 2 and 3
        version 1.25.10; written and copyright by Michael Hipp and others
        free software (LGPL) without any warranty but with best wishes
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
/home/suramya/bin/KeepSpeakerOn.sh: line 5: 10993 Segmentation fault      /usr/bin/mpg123 /home/suramya/bin/KeepSpeakerOn.mp3 -v

Spent a while trying to debug and finally figured out that the fix for this issue was to add XDG_RUNTIME_DIR=/run/user/<userid> to the cron where you can get the value of <userid> by running the following command and taking the value of uid:

id <username_the_cronjob_is_running_under> 

e.g.

suramya@StarKnight:~/bin$ id suramya
uid=1000(suramya) gid=1000(suramya) groups=1000(suramya),24(cdrom)....

Putting that line in the cron entry resolved the issue. Not sure why but it works so…

Well this is all for now. Will write more later.

– Suramya

August 24, 2018

Fixing the appstreamcli error when running apt-get update

Filed under: Computer Software,Knowledgebase,Linux/Unix Related,Techie Stuff — Suramya @ 12:05 AM

Over the past few days everytime I tried to update my Debian system using apt-get it would fail with the following error message:

(appstreamcli:5574): GLib-CRITICAL **: 20:49:46.436: g_variant_builder_end: assertion '!GVSB(builder)->uniform_item_types || 
GVSB(builder)->prev_item_type != NULL || g_variant_type_is_definite (GVSB(builder)->type)' failed

(appstreamcli:5574): GLib-CRITICAL **: 20:49:46.436: g_variant_new_variant: assertion 'value != NULL' failed

(appstreamcli:5574): GLib-ERROR **: 20:49:46.436: g_variant_new_parsed: 11-13:invalid GVariant format string
Trace/breakpoint trap
Reading package lists... Done
E: Problem executing scripts APT::Update::Post-Invoke-Success 'if /usr/bin/test -w /var/cache/app-info -a -e /usr/bin/appstreamcli; then appstreamcli refresh-cache > 
/dev/null; fi'
E: Sub-process returned an error code

Spent a couple of hours trying to figure out what was causing it and was able to identify that it was caused because of a bug in appstream as tunning the command manually also failed with the same error. When I tried to remove the package as recommended by a few sites it would have removed the entire KDE desktop from my machine which I didn’t want so I was at a loss as to how to fix the problem. So I put the update on hold till I had a bit more time to research the issue and identify the solution.

Today I got some free time and decided to try again and after a little bit of searching stumbled upon the following Bug Report (#906544) where David explained that the error was caused due to a bug in the upstream version of appstream and a little while later Matthias commented that the issue is fixed in the latest version of the software and it would flow down to the Debian repositories in a little bit. Normally I would have just done an apt-get update and then install to get the latest package but since the whole issue was that I couldn’t get the system to finish the update command I had to manually install the package.

To do that I went to the Debian site and opened the software package list for Debian Unstable (as that is what I am using) and searched for appstream. This gave me a link to the updated package (0.12.2-2) that fixed the bug (I had 0.12.2-1 installed). Once I downloaded the package (Make sure you download the correct package based on your system architecture) I manually installed it using the following command as root:

dpkg -i appstream_0.12.2-2_amd64.deb

This installed the package and I was then able to do an apt-get update successfully. I still get the GLib-CRITICAL warnings but that apparently can be ignored without issues.

Hope this helps people who hit the same issue (or reminds me of the solution if/when I hit the issue again).

– Suramya

February 7, 2018

Hacking the Brainwaves Cyber Security CTF Hackathon 2018

Earlier this year I took part in the Brainwaves Cyber Security Hackathon 2018 with Disha Agarwala and it was a great experience. We both learnt a lot from the hackathon and in this post I will talk about how we approached the problems and some of our learning’s from the session.

Questions we had to answer/solve in the Hackathon:

  • Find the Webserver’s version and the Operating system on the box
  • Find what processes are running on the server?
  • What fuzzy port is the SSH server running on?
  • Discover the site architecture and layout.
  • Describe the major vulnerability in the home page of the given website based on OWASP TOP 1. Portal Url: https://socgen-ctf.0x10.info
  • Gain access to member area and admin area through blind sql, or session management.
  • Dump all user account from member area. [SQLi]
  • [Broken Validation] Demonstrate how you can modify the limit in order management.
  • [Open Redirect] Redirect site/page to hackerearth.com
  • List any other common bug came across while on the site
    • After logging into the member area, perform the following functions:
    • Find the master hash & crack it
    • Dump all user’s
    • Find the email ID and password of saved users

Information Gathering:

In order to find the services running on the server, the first thing we had to do was find the IP/hostname of the actual server hosting the site which was a bit tricky because the URL provided is protected by CloudFlare. So, any scans of socgen-ctf.0x10.info took us to the CloudFlare proxy server instead of the actual server which was a problem.

We figured this out by trying to access the IP address that socgen-ctf.0x10.info translated to in the browser.

suramya@gallifrey:~$ host socgen-ctf.0x10.info 
socgen-ctf.0x10.info has address 104.28.15.64 

Since the site homepage didn’t do anything except display text that refreshed every 15 seconds we needed to find other pages in the site to give us an a attack surface. We checked to see if the site had a robots.txt (It tells web crawlers not to index certain directories). These directories are usually ones that have sensitive data and in this case the file existed with the following contents:

# robots.txt
Sitemap: http://socgen-ctf.0x10.info/sitemap.xml
User-agent: *
Disallow: images
Disallow: /common/
Disallow: /cgi-bin/

The images directory didn’t have any interesting files in it but the /common/ directory on the other hand had a file named embed.php in it which basically ran a PHP Info dump. This dump has a lot of information that can be used to attack the site but the main item we found here was the IP address of the actual server where the services were running (38.109.218.93).

Using this information we were able to initiate a nmap scan to get the services running on the site. The nmap command that gave us all the information we needed was:

nmap -sV -O -sS -T4 -p 1-65535 -v 38.109.218.93

This gave us the following result set after a really really long run time:

PORT     STATE    SERVICE       VERSION
23/tcp   filtered telnet
25/tcp   open     smtp?
80/tcp   open     http          This is not* a web server, look for ssh banner
81/tcp   open     http          nginx 1.4.6 (Ubuntu)
82/tcp   open     http          nginx 1.4.6 (Ubuntu)
137/tcp  filtered netbios-ns
138/tcp  filtered netbios-dgm
139/tcp  filtered netbios-ssn
445/tcp  filtered microsoft-ds
497/tcp  filtered retrospect
1024/tcp open     kdm?
1720/tcp open     h323q931?
2220/tcp open     ssh           OpenSSH 6.6.1p1 Ubuntu 2ubuntu2.8 (Ubuntu Linux; protocol 2.0)
2376/tcp open     ssl/docker?
3380/tcp open     sns-channels?
3389/tcp open     ms-wbt-server xrdp
5060/tcp filtered sip
5554/tcp filtered sgi-esphttp
8000/tcp open     http          nginx 1.4.6 (Ubuntu)
8080/tcp open     http          Jetty 9.4.z-SNAPSHOT
8086/tcp open     http          nginx 1.10.3 (Ubuntu)
9090/tcp open     http          Transmission BitTorrent management httpd (unauthorized)
9996/tcp filtered palace-5
19733/tcp filtered unknown
25222/tcp filtered unknown
30316/tcp filtered unknown
33389/tcp open     ms-wbt-server xrdp
33465/tcp filtered unknown
34532/tcp filtered unknown
35761/tcp filtered unknown
35812/tcp filtered unknown
35951/tcp filtered unknown
37679/tcp filtered unknown
38289/tcp filtered unknown
38405/tcp filtered unknown
38995/tcp filtered unknown
40314/tcp filtered unknown
44194/tcp filtered unknown
47808/tcp filtered bacnet

For some reason the results from the nmap scan varied so we had to run the scan multiple times to get all the services on the host. This was possibility because the server was setup to make automated scanning more difficult.

Once we identified the port where the SSH server was running on (2220) we were able to connect to the port and that gave us the exact OS Details of the server. We did already know that the server was running Ubuntu along with the kernel version from the PHP Info dump but this gave us the exact version.

Discovering Site architecture:

Since we had to discover the URL to the members & admin area before we could attack it, we used dirb which is a Web Content Scanner to get the list ofall the public directories/files on the site. This gave us the URL’s to several interesting files and directories. One of the files identified by dirb was https://socgen-ctf.0x10.info/sitemap.xml. When we visited the link it gave us a list of other URL’s on the site of interest (we had to replace the hostname to socgen-ctf.0x10.info) including the members area (http://socgen-ctf.0x10.info/members.php?p=login) and siteadmin (http://socgen-ctf.0x10.info/siteadmin).

After a long and fruitless effort to use SQL Injection on the siteadmin area we started to explore the other files/URL’s identified by dirb. This gave us a whole bunch of files/data that seem to be left over from other hackathons so we ignored them.

SQL Injection

The main site https://socgen-ctf.0x10.info/index.php?p=. appeared to be vulnerable to SQL at the first glance because when we visit https://socgen-ctf.0x10.info/index.php?p=.’ (note the trailing single quote) it reloads the page. This meant that we could write queries to it however since it didn’t display a true or false on the page a SQL injection wasn’t easily possible. (We could have tried a blind injection but that would require a lot of effort for a non-guaranteed result.

As we explored the remaining URL’s in sitemap.xml one of the links (https://socgen-ctf.0x10.info/embedframe.php) was interesting as it appeared to give a dump of data being read from the site DB. Opening the site while watching the Developer Toolbar for network traffic identified a URL that appeared to be vulnerable to SQL injection (https://socgen-ctf.0x10.info/ajax.php?cid=&p=view_channel&id=28) and once we tested the url we found that the variable id was indeed vulnerable to injection.

We used blind sql to gain access by executing true and false statements and see that it returns different results for true(displays ‘1’ on the webpage) and false (displays 0) . We checked whether a UNION query runs on the site which it did and using other queries we identified the DB backend to be a mysql database (5.xx.xxx version). Then we found out the table name (members) which was an easy guess since the website had an add customer field. After identifying the number of columns in the table we got stuck because any statements to list the available tables or extract data were failing with an error about inconsistent column numbers.

Finally, we ran sqlmap which is an open source tool for automating SQL injection. It took us a few tries to get the software running because initially any attempt to scan the site was rejected with a 403 error message. Turns out that the connections were being rejected because the site didn’t like the useragent the software was sending by default and adding a flag to randomize the useragent resolved the permission denied issue.

Once the scan ran successfully we tried to get access to the MySQL usertable but that failed because the user we were authenticating as to the MySQL server didn’t have access to the table required.

sqlmap -u 'https://socgen-ctf.0x10.info/ajax.php?cid=&p=view_channel&id=28' --random-agent -p id --passwords

So, then we tried getting an interactive shell and an OOB shell both of which failed. We finally ran the command to do a full dump of everything that the system allowed us to export using SQL injection via SQLMap. This included the DB schema, table schema’s and a dump of every table on the database server which the mysql user had access to. The command we used is the following:

sqlmap -u 'https://socgen-ctf.0x10.info/ajax.php?cid=&p=view_channel&id=28' --random-agent -p id  --all --threads 3

This gave us a full dump of all the tables and the software was helpful enough to identify password hashes when they existed in the table and offered to attempt decryption as well. In this case the password was encrypted with a basic unsalted MD5 hash which was cracked quite easily. Giving us the password for the first two accounts in the database (admin & demo).

Looking at the rest of the entries in the users table we noticed that they all had funny values in the email address field, instead of a regular email address we had entries that looked like the following:

,,,"0000-00-00 00:00:[email protected]509a6f75849b",1
,1,RU,

As we had no clue what this was about the first thing we attempted was to access the
https://socgen-ctf.0x10.info/cdn-cgi/l/email-protection URL. This URL gave us a message that told us that the email addresses in the DB were obfuscated by CloudFlare to protect them from Bots. A quick Google search gave us a 21 line python script which we tweaked to convert all the hash to email address and passwords. (The code is listed below for reference)

#! /usr/bin/env python 
# -*- coding: utf-8 -*- 
# vim:fenc=utf-8 
# 
# Copyright © 2016 xl7dev  
# Distributed under terms of the MIT license. 

""" 

""" 
import sys 
import re 
fp = sys.argv[1] 
def deCFEmail(): 
   r = int(fp[:2],16) 
   email = ''.join([chr(int(fp[i:i+2], 16) ^ r) for i in range(2, len(fp), 2)]) 
   print email 
if __name__ == "__main__":                                                                                                                                                                       
   deCFEmail() 

This gave us the email addresses and passwords for all the users on the site. Since the accounts appeared to be created by SQL injection a bunch of them didn’t have any passwords but the remaining were valid accounts for the most part and we verified a couple by logging in manually with the credentials.

OWASP TOP 10 Vulnerability

To find the vulnerabilities in the home page we tried various manual techniques at first but drew a blank so we decided to use the owasp-zap. This tool allows you to automatically scan for vulnerabilities in a given URL along with a whole other stuff.

At first the scan failed because of the same issue as earlier with the user-agent. This time we took a different approach to resolve the issue by configuring owasp-zap as a proxy server and configuring Firefox traffic to use this proxy server for all traffic. This gave us the site in the software and we were then able to trigger both an active scan and spider scan of the site.

This gave us detailed reports that highlighted various issues in the site which we submitted.

Redirecting HomePage

The redirection of the home page was quite simple. We tried inserting a customer name with javascript tags in it and were able to do so successfully. So we inserted the following into the DB and the system automatically redirected the page when the Customer list section was accessed.

Other Interesting Finds

The nmap scan told us that in addition to port 80 a web server was listening on ports 81, 82, 8000, 8080 and 8086.

Ports 82, 8000 and 8086 were running standard installs of nginx and we didn’t find much of interest at these ports even after we ran dirb on all of them. Port 8080 appeared to be running a proxy or a Jenkins instance.

Port 81 was the most interesting because it was running a nginx server that responded to any queries with a 403 error. When we tried accessing the site via the browser we got an error about corrupted content.

We were unable to identify what the purpose of this site was but it was interesting.

SSH Banner / PHP Shell

The webserver instance running on port 80 had the version set to the following text “This is not* a web server, look for ssh banner Server at private-tunel.wehostservers.ru Port 80” so we went back and investigated the SSH Banner from the ssh server on port 2220. The banner was encrypted and to decrypt the SSH banner, we continuously converted the cipherText from its hex value to ASCII value . It gave us the following results on each conversion

3333333733333333333333373333333333333336333333383333333233333330333333363333333233333336333333313333333633363335333333363336333533333336333333353
3333337333333323333333233333330333333363333333633333336333633363333333733333332333333373333333733333336333333313333333733333332333333363333333433333332333333303333
3337333333333333333633363333333333363333333133333337333333333333333633333338333333323333333033333336333333333333333633363336333333373333333533333336333633333333333
63333333433333332333333303333333633363333333333363333333533333336333333313333333633333334333733393336363633373335373436663230363132307368336c6c2e706870

3337333333373333333633383332333033363332333633313336363533363635333633353337333233323330333633363336363633373332333733373336333133373332333633343332333033373333333
636333336333133373333333633383332333033363333333636363337333533363633333633343332333033363633333633353336333133363334373936663735746f206120sh3ll.php
 37333733363832303632363136653665363537323230363636663732373736313732363432303733366336313733363832303633366637353663363432303663363536313634796f75to a #

ssh banner forward slash could lead you to a #sh3ll.php

Once we got the full decrypted text we knew that there was a potential webshell on the server but it wasn’t apparent where the shell was located. After hit and try failed we turned back to our old faithful dirb to see if it could find the shell.

dirb allows us to specify a custom word list which is used to iterate through the paths and we can also append an extension to each of the words to search for, so we created a file called test with the following content:

suramya@gallifrey:~$ cat test 
shell
sh3ll
sh311

and then ran the following command:

suramya@gallifrey:~$ dirb https://socgen-ctf.0x10.info/ test  -X '.php'

This gave us the location of the shell.


Accessing the link gave us a page with a message “you found a shell, try pinging google via sh3ll.php?exec=ping 8.8.8.8”

Accessing the URL with the additional parameter gave us a page with the following output:

February 20, 2016

How to encrypt your Hard-drive in Linux

We have heard multiple stories where someone looses a pendrive or a laptop containing sensitive/private data which is then published by the person who found the drive embarrassing the owner of the data. The best way to prevent something like that from happening to you if you loose a disk is to make sure all your data is encrypted. Historically this used to be quite painful to setup and required a lost of technical know-how. Thankfully this is no longer the case. After trying a bunch of different options I found Linux Unified Key Setup-on-disk-format (LUKS) to be the most user-friendly and easy to setup option for me.

Setting it up is quite easy by following the instructions over at www.cyberciti.biz. However since things on the internet have a tendency of disappearing on a fairly frequent basis, I am using this post to save a paraphrased version of the installation instructions (along with my notes/comments) just in case the original site goes down and I need to reinstall. All credit goes to original author. So without further ado here we go:

Install cryptsetup

First we need to install cryptsetup utility which contains all the utilities we need to encrypt our drive. To install it in Debian/Ubuntu you just issue the following command as root:

apt-get install cryptsetup

Configure LUKS partition

Warning: This will remove all data on the partition that you are encrypting. So make sure you have a working backup before proceeding amd don’t blame me if you manage to destroy your data/device.

Run the following command as root to start the encryption process:

cryptsetup -y -v luksFormat <device>

where <device> is the partition we want to encrypt (e.g. /dev/sda1). The command will ask you for confirmation and a passphrase. This passphrase is not recoverable so make sure you don’t forget it.

Create drive mapping

Once the previous command completes you need to create a mapping of the encrypted drive by issuing the following command:

cryptsetup luksOpen <device> backup2

You can also map a partition to using its UUID (which is what I do) by issuing the following command instead (This works great if you want to script automated backups to an external drive):

cryptsetup luksOpen UUID=88848060-fab7-4e9e-bac2-f9a2323c7c29 backup2

Replace the UUID in the example with the UUID of your drive. (Instructions on how to find the UUID are available here).

Use the following command to see the status for the mapping and to check if the command succeeded:

cryptsetup -v status backup2

Format LUKS partition

Now that we have created the mapping we need to write zeroes to the encrypted device, to ensure that the outside world sees this as random data and protects the system against disclosure of usage by issuing the following command:

dd if=/dev/zero of=/dev/mapper/backup2

Since this command can take a long time to complete depending on the drive size and dd by default doesn’t give any feedback on the percentage completed/remaining I recommend that you use the pv command to monitor the progress by issuing the following command instead:

pv -tpreb /dev/zero | dd of=/dev/mapper/backup2 bs=128M

This will take a while to run so you can go for a walk or read a book while it runs. Once the command completes you can create a filesystem on the device (I prefer to use ext4 but you can use any filesystem you like) by formatting the device:

mkfs.ext4 /dev/mapper/backup2

After the filesystem is created you can mount and use the partition as usual by issuing the following command:

mount /dev/mapper/backup2 /mnt/backup

That’s it. You now have an encrypted partition that shows up as a regular partition in Linux which you can use as a regular drive without having to worry about anything. No special changes are needed to use this partition which means any software can use it without requiring changes.

How to unmount and secure the data

After you are done transferring data to/from the drive you can unmount and secure the partition by issuing the following commands as root:

umount /mnt/backup

followed by

cryptsetup luksClose backup2

Creating a backup of the LUKS headers

Before you start anything else, you should create a backup copy of the LUKS header because if this header gets corrupted somehow then all data in the encrypted partition is lost forever with no way to recover it. From the cryptsetup man page:

“LUKS header: If the header of a LUKS volume gets damaged, all data is permanently lost unless you have a header-backup. If a key-slot is damaged, it can only be restored from a header-backup or if another active key-slot with known passphrase is undamaged. Damaging the LUKS header is something people manage to do with surprising frequency. This risk is the result of a trade-off between security and safety, as LUKS is designed for fast and secure wiping by just overwriting header and key-slot area.”

Create a backup by issuing the following command:

cryptsetup luksHeaderBackup <device> --header-backup-file <file>

Important note: a LUKS header backup can grant access to most or all data, therefore you need to make sure that nobody has access to it.

In case of disaster where our LUKS header gets broken, we can restore it by issuing the following command:

cryptsetup luksHeaderRestore <device> --header-backup-file <file>

How to remount the encrypted partition?

Issue the following commands in sequence to mount the partition:

cryptsetup luksOpen <device> backup2
mount /dev/mapper/backup2 /mnt/backup

Please note that data encrypted by LUKS is quite obvious with most Linux systems identifying it as an encrypted partition automatically. So if someone examines your system they will know you have encrypted data and can force you to divulge the password by various means (including the use of Rubber-hose Cryptanalysis. )

If you want the encrypted partition to be hidden then you can use Deniable encryption/Hidden Partition or use steganography. I haven’t really used either so can’t comment on how to set it up correctly but maybe I can talk about it in a future post after I explore them a bit more.

Well this is all for now, hope you find this useful. Will write more later.

– Suramya

October 11, 2015

Finally managed to upgrade to latest Debian Testing without breaking my install completely

Filed under: Linux/Unix Related,My Life — Suramya @ 10:19 PM

Some of you might have wondered (for a very brief amount of time) based on my last post and then lack of activity if I had managed to get myself sent to Mars but unfortunately that wasn’t the case. The problem was more mundane, basically I had somehow managed to get my desktop in a state WHERE it thought that it needed to uninstall KDE whenever I tried to upgrade to the latest Debian packages using ‘apt-get upgrade’ or for that matter when I tried to install any new package as well. After ignoring the issue for a while I decided to take the plunge and went ahead with the upgrade thinking that I would just reinstall KDE after the upgrade completed. Unfortunately that didn’t work out as planned and I had to do a full reinstall from scratch. It was something I thought about doing so that I could resize the partition allocation but didn’t have the time/incentive to do it. So this was the perfect time to take the plunge.

I re-partitioned the drive and started the install. The first few attempts failed quite spectacularly because apparently some of the packages in the ‘Unstable’ branch are broken (which is not unexpected because after all it *is* called the unstable branch.) After I switched to the Testing branch which is more stable than unstable I got a bit further along but hit another snag while installing KDE as during the upgrade systemd kept complaining about not being able to talk to policykit and died after giving the following error message a few hundred times:

Error getting authority: Error initializing authority: Error calling StartServiceByName for org.freedesktop.PolicyKit1: Timeout was reached (g-io-error-quark, 24)
Failed to execute operation: Connection timed out

This stumped me for a while since I didn’t have the energy at the time to research the issue in too much detail. I would have resolved it faster but as I had my laptop working I was able to get work done and access the net, (even if it wasn’t the most comfortable way to work.) which reduced the urgency to fix the problem. After a few weeks I finally got time to sit and work on the issue. Turns out the problem was caused because systemd was expecting a later version of policykit than what was installed and apt-get wasn’t upgrading policykit before it started installing KDE. The solution was quite simple after I figured out what the problem was, which was to upgrade policykit by issuing the following command as root before installing KDE:

apt-get install policykit-1

After running that command I was able to upgrade to the latest Debian Testing and with that I also got a newer version of KDE (5:90 is what apt-get calls it) which is nice and has a lot of eye candy. However as with all KDE releases/upgrades to a new system (Plasma) it still has issues/missing features. Annoying stuff that I have found so far is listed below:

  • There is no quick icon widget. I put all the software I regularly use in the Quick icon bar and not having it is really annoying. There is a temp workaround where I can right click on a window and select ‘Show as a Launcher when not running’ but it is not the same.
  • There is a bug that prevents you from disabling the beep for system events for all events. So everytime I try to delete sometime I get a loud and annoying beep. I can’t switch off all sounds because then I can’t hear my music either. Lots of folks have been complaining about this so hopefully there will be a fix out soon.
  • Not all software minimizes to the system tray. Specifically, Tomboy which I use to take notes exits completely when I close it instead of minimizing to the system tray which is what it used to do. I am sure there is a setting that I am missing but I haven’t found the fix yet.

There are other minor annoyances but they are mostly caused because the new version does things slightly differently so I guess I just need to get used to the new way. To top things off my UPS battery has gone for a toss and I get an amazing 0.2 mins of backup with a full charge. 🙁 I have ordered new batteries but till they arrive I need to remember to power off the system when I am not around.

Well this is all for now. Will post more later (hopefully more regularly now that I have a working desktop again).

– Suramya

March 29, 2015

Rosetta Stone for Unix/Linux

Filed under: Knowledgebase,Linux/Unix Related,Techie Stuff — Suramya @ 9:53 PM

If you have been in the industry for a while then you have been in a situation where you need to do something on the server but have no idea what the appropriate command is because you always worked on a different variant/version of the Operating System. Think having to work on Solaris or Linux when all you have worked on is the Mac OS. To make things easier for the poor admins that have to keep switching OS’s, Bruce Hamilton has created a site he calls the ‘Rosetta Stone: A Sysadmin’s Universal Translator‘. This site has a list of tasks and the corresponding command that you would have to run for each of the OS’s. The Stone supports the following OS’s:

  • AIX
  • A/UX
  • DG/UX
  • FreeBSD
  • HP-UX
  • IRIX
  • Linux
  • Mac OS X
  • NCR Unix
  • NetBSD
  • OpenBSD
  • Reliant
  • SCO OpenServer
  • Solaris
  • SunOS 4
  • Tru64
  • Ultrix
  • UNICOS

and covers tasks in the following categories:

  • hardware
  • firmware
  • devices
  • disks
  • kernel
  • boot
  • files
  • networking
  • security
  • software
  • patching, tracing, logging

Check it out, bookmark it. It will save you some grief down the line the next time you are in this situation.

– Suramya

December 14, 2014

Cleaning your Linux computer of cruft and duplicate data

When you use a computer and keep copying data forward everytime you upgrade or work with multiple systems it is easy to end up with multiple copies of the same file. I am very OCD about organizing my data and still I ended up with multiple copies of the same file in various locations. This could have happened because I was recovering data from a drive and needed a temp location to save the copy or forgot that I had saved the same file under another directory (because I changed my mind about how to classify the file). So this weekend I decided to clean up my system.

This was precipitated because after my last system reorg I didn’t have a working backup strategy and needed to get my backups working again. Basically I had moved 3 drives to another server and installed a new drive on my primary system to serve as the Backup drive. Unfortunately this required me to format all these drives because they were originally part of a RAID array and I was breaking it. Once I got the drives setup I didn’t get the chance to copy the backup data to the new drive and re-enable the cron job that took the daily backup snapshots. (Mostly because I was busy with other stuff). Today when I started copying data to the new Backup drive I remembered reading about software that allowed you to search for duplicate data so thought I should try it out before copying data around. It is a good thing I did because I found a lot of duplicates and ended up freeing more than 2 GB of space. (Most of it was due to duplicate copies of ISO images and photos).

I used the following software to clean my system:

Both of them delete files but are designed for different use cases. So let’s look at them in a bit more detail.

FSlint

FSlint is designed to remove lint from your system and that lint can be duplicate files, broken links, empty directories and other cruft that accumulates when a system is in constant use. Installing it is quite easy, on Debian you just need to run the following command as root

apt-get install fslint

Once the software is installed, you can either use the GUI interface or run it from the command line. I used the GUI version because it was easier to visualize the data when seen in a graphical form (Yes I did say that. I am not anti-GUI, I just like CLI more for most tasks). Using the software was as easy as selecting the path to search and then clicking on Find. After the scan completes you get a list of all duplicates along with the path and you can choose to ignore, delete all copies or delete all except one. You need to be a bit careful when you delete because some files might need to be in more than one location. One example for this situation is DLL files installed under Wine, I found multiple copies of the same DLL under different directories and I would have really messed up my install if I had blindly deleted all duplicates.

Flossmanuals.net has a nice FSlint manual that explains all the other options you can use. Check it out if you want to use some of the advanced features. Just ensure that you have a good backup before you start deleting files and don’t blame me when you mess up your system without a working backup.

BleachBit

BleachBit is designed for the privacy conscious user and allows you to get rid of Cache, cookies, Internet history, temporary files, logs etc in a quick and easy way. You also have the option to ensure that the data deleted is really gone by overwriting the file with random data. Obviously this takes time but if you need to ensure data deletion then it is very useful. Bleachbit works on both Windows and Linux and is quite easy to install and use (at least on Linux, I didn’t try it on Windows). The command to install it on Debian is:

apt-get install bleachbit

The usage also is very simple, you just run the software and tick the boxes relevant to the clutter that you want gone and BleachBit will delete it. It does give you a preview of the files it found so that you can decide if you actually want to delete the stuff it identifies before you delete it.

Well this is all for now. Will write more later.

Thanks to How to Sort and Remove Duplicate Photos in Linux for pointing me towards FSlint and Ten Linux freeware apps to feed your penguin for pointing me towards BleachBit.

– Suramya

April 20, 2014

Facebook Stat generation followup

Filed under: Computer Tips,Linux/Unix Related,My Thoughts,Techie Stuff — Suramya @ 2:16 AM

In my previous post I had talked about some of the stats I pulled from Facebook about it’s usage by my friends. This was an ad-hoc number crunching done just because I was bored and got curious. After the post went live a friend of mine, Ankush asked for more details on how I generated the numbers so in this post I am going to go over my process and how I got the numbers I shared.

Before we start, keep in mind that this is all data that is publicly available on FB, or at least shared with me. If you don’t want others to generate data about your activity on FB, you should change your privacy settings on FB and restrict access. Please don’t try to use this information to try to get access to data you are not supposed to. You will get in trouble and I will not take responsibility for it. Now that all that is out of the way, lets get to the details of the process.

The first thing you need is to have the Facebook Command Line client installed. Instructions on how to install are here so I am not going to repost them here. Make sure you authenticate the install and follow the steps in ‘Obtain Additional Authorization’ section of the installation guide otherwise the rest of the guide won’t be of much use to you.

Once you have FBCMD installed and configured, you can start playing with the options. Check out fbcmd Commands for the list of available options. You can also run the script with –help for the same.

/usr/bin/php /var/www/fbcmd/lib/fbcmd/fbcmd.php --help

Since I was interested in the photos uploaded the first command I ran was:

/usr/bin/php /var/www/fbcmd/lib/fbcmd/fbcmd.php OPICS =all FB_Pics

This command gets all the photos uploaded by folks in my friend list and downloaded them to the FB_Pics folder. As I mentioned in the previous post, this downloaded over 58k photos to my system. So be careful when you run it. You can also restrict it to a particular user by passing their name as a parameter.

To get the wall post count’s of all my friends, I ran the following command:

/usr/bin/php /var/www/fbcmd/lib/fbcmd/fbcmd.php FINFO wall_count =all

This gave me a output similar to the following:


NAME WALL_COUNT
Suramya Tomar 247
ABC 1405

I took this output, put it in an Excel file and did some analysis on it to get the max post count, least post count, Total count and top 10 user post counts. I could have done this using shell commands as well, but since this was a one time task I didn’t see the point. Maybe in the future I could set up a job that would do this periodically and do trending on the data but lets see. I don’t see much use for this data except for the coolness factor and to satisfy my curiosity.

Getting the birthday count was as easy as running the following command:

/usr/bin/php /var/www/fbcmd/lib/fbcmd/fbcmd.php FINFO birthday_date =all |wc -l

This returned the number of folks who had shared their birthday’s on FB and then I got the current location count using the following command:

/usr/bin/php /var/www/fbcmd/lib/fbcmd/fbcmd.php FINFO current_location =all |wc -l

So there you have it. This is how I generated the numbers I had posted earlier. As you can see there is nothing too complicated about it, so if you want you can generate similar stats for your friends as well.

Let me know if you have any questions and I will do my best to answer.

Well this is all for now. I should go and get some sleep now.

– Suramya

July 24, 2013

My RaspberryPi Camera module finally arrived and it works great!

Filed under: Computer Hardware,Linux/Unix Related,My Thoughts,Techie Stuff — Suramya @ 2:06 AM

After months of waiting (about 2 to be exact) my RaspberryPi Camera module finally arrived today and I took it for a brief spin. Setting it up was really easy, I just installed the Rasbian image I have, updated the install to the latest version and then enabled the camera. Once that was done and I rebooted the Pi, the camera started working without any issues. The instructions I followed are at: RaspberryPi Camera.

As part of the test I got the pi to transmit the images to my desktop and the clarity was pretty good, there was a slight lag in the display of about 1/2 a second but at 1080p I don’t think I can complain. Below are some pics I took of the setup and of the image being streamed to my desktop.


The Camera module next to my keyboard for a size comparison


Photo of my hand being streamed live to my desktop from the Pi.

I do plan to take pics using the camera module itself, but there is nothing interesting on my desk that I wanted to take photos of so you will have to wait for a day or two and I will take pics of the view from my window and post.

Well this is all for now, I should go get some sleep now considering I have an early start tomorrow.

– Suramya

March 8, 2013

Citrix on Raspberry Pi: Updated instructions and working download image

Filed under: Knowledgebase,Linux/Unix Related,Techie Stuff,Tutorials — Suramya @ 2:36 PM

A couple of folks have reached out to me via email/messages to tell me that the instructions I posted at the Raspberry Pi forums don’t work with the latest version of Rhaspbian. Basically the problem is that the latest version of the Citrix client is not compiled for the armhf architecture (Which is what the latest version of Rhaspbian OS is compiled for), so you need to download and install the armel version of the OS (‘Soft-float Debian “wheezy”’) from http://www.raspberrypi.org/downloads.

To make life simpler for people I have created a snapshot of my Pi install with Citrix installed and configured. You can download it from here. The image is 4GB so you will need to use a card of atleast that size when using this image. Follow these steps to install the image to an SD card in Linux:

  • Download the image file from the mirror (Approx 1GB compressed)
  • Unzip the file using the command
  • unzip Raspberry_Citrix.img.zip
  • Find out what the partition the SD card you are using has been assigned running the following command as root
  • fdisk -l

    Once you run the command, you will get an output that will show you all the disks attached to your system, look for the entry that corresponds to your card. In my case it looked like this:

     Disk /dev/sde: 3965 MB, 3965190144 bytes
    122 heads, 62 sectors/track, 1023 cylinders, total 7744512 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x00016187
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sde1            8192      122879       57344    c  W95 FAT32 (LBA)
    /dev/sde2          122880     7744511     3810816   83  Linux
    
  • So now we know that the card is at /dev/sde. All we have to do is write the image to the card and that is done using the following command. Make sure you replace the /dev/sde with the correct path otherwise you will end up destroying all data on the wrong drive.
  • dd if=Raspberry_Citrix.img of=/dev/sde bs=4096

    You will not see any output on the screen so don’t worry about it, just let it run and wait for the process to complete as it will take some time because of the amount of data being written. Once the process completes you can eject the card and if all went well you should be able to boot the Raspberry Pi from the card.

The login password for this image is root/password, please do change the password if you use the image. Let me know if you have any questions or have an issue using this image.

Update (3/28/2013): Adding instructions on how to write the image when using windows. (Please note that I haven’t tested the windows instructions as I don’t have a windows machine. Use at your own risk)

Once you download the zip file from the mirror, right-click on it and select extract (I think that’s what it says, but I don’t have a windows machine so can’t confirm). After the image is extracted you will have a file called Raspberry_citrix.img on your computer. Now follow these steps to write the image to an SD card (Instructions taken from eLinux)

  • Insert the SD card into your SD card reader and check what drive letter it was assigned. You can easily see the drive letter (for example G:) by looking in the left column of Windows Explorer. If the card is not new, you should format it and make sure there is only one partition (FAT32 is a good choice); otherwise Win32DiskImager can make corrupt your SD card!
  • Download the Win32DiskImager utility. The download links are on the right hand side of the page, you want the binary zip.
  • Extract the executable from the zip file and run the Win32DiskImager utility. You should run the utility as Administrator!
  • Select the Raspberry_citrix.img image file you extracted earlier
  • Select the drive letter of the SD card in the device box. Be careful to select the correct drive; if you get the wrong one you can destroy your data on the computer’s hard disk!
  • Click Write and wait for the write to complete.
  • Exit the imager and eject the SD card.

You should also go through the Basic setup guide for Raspberry Pi. Hope this helps.

– Suramya

Older Posts »

Powered by WordPress