Suramya's Blog : Welcome to my crazy life…

August 24, 2018

Fixing the appstreamcli error when running apt-get update

Filed under: Computer Software,Knowledgebase,Linux/Unix Related,Techie Stuff — Suramya @ 12:05 AM

Over the past few days everytime I tried to update my Debian system using apt-get it would fail with the following error message:

(appstreamcli:5574): GLib-CRITICAL **: 20:49:46.436: g_variant_builder_end: assertion '!GVSB(builder)->uniform_item_types || 
GVSB(builder)->prev_item_type != NULL || g_variant_type_is_definite (GVSB(builder)->type)' failed

(appstreamcli:5574): GLib-CRITICAL **: 20:49:46.436: g_variant_new_variant: assertion 'value != NULL' failed

(appstreamcli:5574): GLib-ERROR **: 20:49:46.436: g_variant_new_parsed: 11-13:invalid GVariant format string
Trace/breakpoint trap
Reading package lists... Done
E: Problem executing scripts APT::Update::Post-Invoke-Success 'if /usr/bin/test -w /var/cache/app-info -a -e /usr/bin/appstreamcli; then appstreamcli refresh-cache > 
/dev/null; fi'
E: Sub-process returned an error code

Spent a couple of hours trying to figure out what was causing it and was able to identify that it was caused because of a bug in appstream as tunning the command manually also failed with the same error. When I tried to remove the package as recommended by a few sites it would have removed the entire KDE desktop from my machine which I didn’t want so I was at a loss as to how to fix the problem. So I put the update on hold till I had a bit more time to research the issue and identify the solution.

Today I got some free time and decided to try again and after a little bit of searching stumbled upon the following Bug Report (#906544) where David explained that the error was caused due to a bug in the upstream version of appstream and a little while later Matthias commented that the issue is fixed in the latest version of the software and it would flow down to the Debian repositories in a little bit. Normally I would have just done an apt-get update and then install to get the latest package but since the whole issue was that I couldn’t get the system to finish the update command I had to manually install the package.

To do that I went to the Debian site and opened the software package list for Debian Unstable (as that is what I am using) and searched for appstream. This gave me a link to the updated package (0.12.2-2) that fixed the bug (I had 0.12.2-1 installed). Once I downloaded the package (Make sure you download the correct package based on your system architecture) I manually installed it using the following command as root:

dpkg -i appstream_0.12.2-2_amd64.deb

This installed the package and I was then able to do an apt-get update successfully. I still get the GLib-CRITICAL warnings but that apparently can be ignored without issues.

Hope this helps people who hit the same issue (or reminds me of the solution if/when I hit the issue again).

– Suramya

August 23, 2018

Identifying Programmers by their Coding Style

Filed under: Computer Security,Computer Software,Techie Stuff — Suramya @ 8:42 PM

There is an interesting development in the field of identifying people by what they write. As some of you may already know researchers have been able to identify who wrote a particular text based on the analysis of things like word choice, sentence structure, syntax and punctuation using a technique called stylometry for a while now but it was limited to natural languages and not artificial ones like programming languages.

Now there is new research by Rachel Greenstadt & Aylin Caliskan who are professors of computer science at Drexel University & at George Washington University respectively that proves that code, like other forms of writing is not anonymous. They used Machine Learning algorithms to de-anonymize coders and the really cool part is that they can do this even with reverse compiled code from Binaries with a reasonable level of confidence. So you don’t need access to the original source code to be able to identify who coded it. (Assuming that we have code samples from them in the training DB)

Here’s a simple explanation of how the researchers used machine learning to uncover who authored a piece of code. First, the algorithm they designed identifies all the features found in a selection of code samples. That’s a lot of different characteristics. Think of every aspect that exists in natural language: There’s the words you choose, which way you put them together, sentence length, and so on. Greenstadt and Caliskan then narrowed the features to only include the ones that actually distinguish developers from each other, trimming the list from hundreds of thousands to around 50 or so.

The researchers don’t rely on low-level features, like how code was formatted. Instead, they create “abstract syntax trees,” which reflect code’s underlying structure, rather than its arbitrary components. Their technique is akin to prioritizing someone’s sentence structure, instead of whether they indent each line in a paragraph.

This is both really cool and a bit scary because suddenly we have the ability to identify who wrote a particular piece of code. This removes or atleast reduces the ability of people to release code/software anonymously. This is a good thing when we look at a piece of Malware or virus because now we can find out who wrote it making it easier to prosecute cyber criminals.

However the flip side is that we can now also identify people who write code to secure networks, bypass restrictive regime firewalls, create privacy applications etc. There are a lot of people who contribute to opensource software but don’t want to be identified for various reasons. For example if a programmer in China created a software that allows a user to bypass the Great Firewall of China they would definitely not want the Chinese government to be able to identify them for obvious reasons. Similarly there are folks who wrote some software that they do not want to be associated with their real name for some reason and this would make it more difficult for them to do so.

But this is not the end of the world, there are ways around this by using software to scramble the code. I don’t think many such systems exist right now or if they do they are at a nacent stage. If this research is broadly applied to start identifying coders then the effort to write such scramblers would take high priority and lots of very smart people would start focusing their efforts to invalidate the detectors.

Well this is all for now. Will write more later.

– Suramya

Original source: Schneier’s Blog

August 12, 2018

Critique of a sextortion scam email that I received

Filed under: My Thoughts,Techie Stuff — Suramya @ 11:27 PM

Earlier this month I got an email that claimed to have photos/videos of me viewing adult sites and threatened that they would mail the photos to all my contacts if I don’t send them $7000. To make the email look authentic and scare me, they also included an old password of mind that they got from one of the many leaks over the past few years. I think this one was from a BBS that I used for a bit around 2000-2005.

The reason I am publishing this email and my critique is to show how full of crap such emails are. Basically if you ever get such emails you should never give them money because then they know that they can frighten you to pay and they will keep putting the pressure on to squeeze more and more money out of you.

On the other hand if you know that someone has managed to get their hands on some incriminating photos (they gave proof or you had sent it to them) and are blackmailing you then you should never give in to the blackmail. Instead reach out to the authorities and file a formal complaint. If you are a kid then talk to your parent and have them raise a complaint. Never ever give more photos/videos to the sick person blackmailing you because that just gives them more ammo to blackmail you.

Here are some links to sites that can help guide you:

UK National Crime Agency
Interpol Sextortion
FBI Sextortion

So lets get started, I am going to take apart the email I got to show you how useless and full of it the email is..

I know ***** is your password. Lets get directly to purpose. You do not know me and you are probably thinking why you are getting this email? None has compensated me to check you.

Umm ok… That’s an old password that I haven’t used in over a decade and even then it was used for throwaway logins that I didn’t really care about. It did catch my eye, good job adding it to the subject to catch my attention. Yes, no one compensated you initially but you sure want to get compensated now.

Well, I installed a malware on the adult video clips (adult porn) web site and guess what, you visited this web site to experience fun (you know what I mean). When you were watching video clips, your web browser started out operating as a RDP that has a keylogger which provided me accessibility to your display screen and also web camera. after that, my software collected your complete contacts from your Messenger, FB, as well as email. After that I created a double-screen video. 1st part shows the video you were viewing (you have a fine taste hahah), and second part displays the view of your webcam, and its you.

Wow! You must teach me how you did this. How did you manage to get a browser to act as an RDP, especially on a Linux machine that doesn’t even support the protocol natively? Please sensei, teach me 🙂

Actually the even more amazing trick is how you managed to activate a webcam on my computer as I don’t have any camera’s connected to it. 🙂 Did you hack the display to turn it into a camera? Or did you send nanobots via the wire to reprogram/repurpose one of the parts on my desktop to convert it into a camera?

You got two different choices. Let us understand each of these options in aspects:

1st choice is to disregard this email. In this case, I am going to send your actual video clip to almost all of your contacts and just consider about the humiliation you feel. And consequently in case you are in an important relationship, how it will affect?

Now comes the threat… how are you going to send a video that I just proved can’t exist?

Latter solution is to give me $7000. We are going to think of it as a donation. As a result, I will without delay delete your video footage. You will go forward daily life like this never happened and you would never hear back again from me.

You will make the payment via Bitcoin (if you don’t know this, search “how to buy bitcoin” in Google search engine).

BTC Address to send to: 1FwvWtFdGBRvoiCa8BQdzqpu5QoiCSRFMa
[CASE SENSITIVE, copy & paste it]

Holy S**T! You really expect people to pay you $7000 for an email that offers no proof of this supposed video that you managed to magically capture? Lets check if anyone was stupid enough to fall for this nonsense. We can use bitref.com to check the balance of any bit coin address and here’s what the current balance is for this address: $0.0. Yup you have received a big fat 0 for your trouble. In fact I would suggest you sell your software/tech to the NSA/MI5 or other spy agencies around the world and you will get a much better payday.


The money this idiot made from this scam so far.

If you may be thinking of going to the cop, good, this email message cannot be traced back to me. I have covered my moves. I am just not trying to charge you so much, I just like to be paid for. I have a unique pixel in this email, and right now I know that you have read through this mail. You now have one day to pay. If I do not receive the BitCoins, I will certainly send your video recording to all of your contacts including friends and family, colleagues, and so forth. However, if I do get paid, I will erase the video right away. It’s a non-negotiable offer and thus please do not waste my personal time & yours by responding to this mail. If you really want evidence, reply with Yeah! then I will send out your video recording to your 6 contacts.

I am really quaking in my boots. Its been over 3 weeks since you sent out the email, and I don’t know how many of my contacts have received this magical email. Though if I had to guess I would place the number at 0. Since the entire email is a scam to steal money from unsuspecting fools. I think if the person sending out the email hadn’t been so greedy and asked for $7000 but rather asked for something in the range of a few hundred they might have made some money.

Well this is all for now. Will write more later.

– Suramya

August 11, 2018

Free Digital Collection of 6,000 19th-Century Children’s Books

Filed under: Interesting Sites — Suramya @ 10:02 PM

Do you like fairy tales? Did you love reading books like Aesop’s Fables, The Adventures of Robinson Crusoe and the Grimm’s Fairy Tales? If so then you should check out this amazing collection of over 6000 children’s books from the 19th and early 20th century. It is available via the University of Florida’s Baldwin Library archive and is pretty comprehensive.

Unfortunately it looks like you can’t download the books for offline reading (at least not that I could find) and the pages of the books have not been converted to text but are rather images. Most are very good scans but still text versions would allow you to search for keywords. I wonder if they would be ok with me downloading the collection and running OCR on the books and then sharing the text versions. Something to think about for my next project

Source: LifeHacker.com

– Suramya

August 9, 2018

Road trip to Belum caves and Lepakshi Temple

Filed under: My Life,Trips — Suramya @ 1:58 AM

Last week while I was getting bored while commuting back from office I stumbled upon an article on LLB about Belum Caves which is the largest and longest cave system open to the public on the Indian subcontinent and it sounded fascinating so I reached out to friends and after a little back and forth 5 of us decided to drive down to the cave for a day trip on Saturday. Since we were planning to start from Bangalore at 5am Anirudh, Jani and Shahrukh came over to my place Friday night while Shakshi came over at 4:30am… Hats off to her for waking up so early and making it to my place on time. After a quick breakfast (and a Red bull for me) we left home at 5:15am. The drive was quite nice and since we left so early in the morning the traffic was minimal (which is a minor miracle in BLR).

To pass the time we talked about all sorts of random topics from astronomy to the percentage of water in various items like cucumbers, milk and human blood etc. By 7am we all started feeling a bit hungry so we stopped for breakfast and had a road-side picnic and I really mean roadside. We stopped next to some newly planted fields about 2 feet away from the road and had a lovely breakfast of sandwiches, paratha’s and boiled eggs. All we were missing to make this a proper picnic was a picnic basket.


Early morning Road-side picnic

After food we were back on the road and made good time to the caves and were there at about 10:40am. We were one of the first groups into the cave and so were able to explore the caves without having to deal with a lot of crowds. The caves were amazing and I am surprised that not a lot of people know about it.


Entrance to Belum Cave


Stalactites in the cave

We spent about two hours in the cave and visited all the important/noteworthy parts even though it was very hot and humid in there. Jani was the only one who was comfortable and enjoyed the temperature as her body’s thermostat is broken (She likes hot and humid weather). There were some pretty cool natural carvings/structures in the cave that looked man-made and some man made structures to host camera’s and ventilation ducts that were mostly hidden so we spend a good amount of time trying to identify which of the structures were man made and which were natural.


The Saint Bed where its rumored that Buddhist monks used to meditate/rest

At one point we were ~150 feet underground at Pathala Ganga which is the deepest part of the cave. Here there is an underground water source that looked quite deep and even though we considered pushing one of the group in the water to see how deep it was common-sense prevailed and we decided not to try. The caves are supposed to have a section that makes musical sounds when struck but we couldn’t find that section. In part it was because we didn’t want to walk around hitting random formations and because we were fascinated by the structures and forgot to search for it.


Group photo in the cave


Trying to ensure we don’t get crushed by the low ceiling

Looking at the structures I was reminded about the Thai cave rescue and it made me think how hard and scary it would have been for them to be stuck in a cave for so long without light. I do want to try cave exploring (spelunking) and have started looking for options in India.

After we came out we fooled around on the playground which was quite fun and then had a picnic lunch. This gave us the opportunity to relax, stretch and enjoy the fresh air. There is a restaurant at the site but has limited options in food. Basically they make about a kg of rice in the morning and if you are early enough you get your food quickly else you have to wait for them to cook the rice. If you are visiting as a big group and are planning to eat there it is advisable to place your order before you head down so that the food is ready by the time you come back from the cave.

After lunch we started back but didn’t head directly for Bangalore, instead we went to Lepakshi Temple which was about midway between the caves and Bangalore. The drive was again quite nice even though everyone in the car (except me obviously) had a post lunch nap during which I entertained myself by playing loud music and singing along. I have a feeling that Jani and Shahrukh woke up after a while just to stop me from singing 😉

The temple is beautiful and we spent a good amount of time walking around the premises and enjoying the carvings. Describing the wonders of the temple would require a whole another post so I am going to be a bit lazy and just link to this post over at the RevolvingCompass.com that describes the 7 wonders of the temple. About 1/2 Km from the temple there is a huge statue of Jatayu but we were unable to visit it because of time (it was getting dark and I wanted to minimize night driving on the highway).


Group photo in front of the Kalyan Mandapa


Us practicing the tree pose for prayer

It was a humbled group that headed back but that didn’t stop our stomachs from rumbling so we stopped for another road-side picnic. This time we found a ready made stone bench for us to use as a table and we made full use of it for a snack break. It was fun to make sandwiches and eat cucumber & tomato with salt and chili. Honestly speaking I could have sat there for another hour but we had to cut the break short because of the time constraints and start back for Bangalore. We made good time to Bangalore and then hit the Bangalore traffic spending a bit over 2 hours to reach home after we entered the city. We finally made it home at ~10:15pm at which point I was ready to crash since I had driven for over 12 hours in the day. But still it was worth the effort and drive.


Road-side Picnic for evening snacks

We ended the day with Ice-cream after which everyone went home and I crashed for the night. We will be doing similar day long road trips in the future as it was quite cheap and a lot of fun. The only limitation is the no of people we can take on the trip since I don’t want to have more than 2 cars. More than 10-12 people makes the group unwieldy and encourages the creation of sub-groups.

Well this is all for now. Will post more later.

– Suramya

August 8, 2018

Work-life balance, Is it something to strive for?

Filed under: My Life,My Thoughts — Suramya @ 11:42 PM

A couple of days ago I read this article by a lady who was the founder of a start-up and she had a whole different take on the work life balance question. She felt that it’s not something that you should focus on and that if your work is a major part of your life then having artificial boundaries about allowed topics of discussion / things is not correct.

The article made me think about the pro’s and cons of having a work life balance.I have in the past worked in companies where we have worked 14-18 hours a day and I have worked in companies where I was out of the office at 6pm everyday.

I think that having a work life balance is good, actually I think it’s essential. You can sustain the insane hours over a short period of time but in the long term it’s not sustainable. I am not saying that once you leave the office don’t have any conversations related to work, that is not realistic. But make an effort to disconnect frequently.It will help recharge your mental energies and let you come back refreshed and eager to work.

I am one of the last people to tell folks not to work too much because I have a tendency of spending too much time working if what I am working is something interesting. But I have seen from personal experience when I take a break from work and do something unrelated it helps me focus and get things done.

In the course of the normal day I read, watch some shows to decompress and once a month I try to go for a trek/trip and over the past two years I have seen what a difference it makes in my sanity and ability to deliver projects. When I go for these trips I don’t check office emails. I have spent some time talking about work with folks but for the most part I disconnect from work. The idea is to stop worrying about work and focus on other things for a while. If conversation or idea related to work does come up then don’t stress about it either, spend a few mins on the topic and then go back to whatever you were doing. Trust me it will help. 

I have seen that some of the best ideas I have had have come to me when I was doing something other than work/actively thinking about the problem. 

At my previous job I used to go for evening snacks with the team and one of the semi-enforced rules was that for the duration of the snacking conversations related to work were discouraged. We would talk about other stuff like hobbies, movies, travel etc. It helped us know each other better and become a more tightly integrated team. If a work related topic came up we would all discuss it for a bit and then someone or other would say something to the effect of ‘no work related talks’ and we would stop. But if the issue was interesting enough we have spent significant time discussing it as well.

So having a hard and fast rule is not a good idea. You should be flexible and take it as it happens.

What do you think? Is work life balance something to stride for?

April 12, 2018

Adventures eating a sauce made from the the worlds Hottest Pepper

Filed under: My Life — Suramya @ 1:30 PM

Regular readers of the blog and folks who know me well know that I love spicy food and am always on the lookout for new spicy sauces/pickles to try. When I visited Bhutan a few years ago I picked up a bag of the spiciest chilly available there to be made into a pickle and I have done the same for all places I visit. Mom made a pickle from Bhoot Jolokia (the spiciest chilly in the world till a few years ago) for me and that is my favorite pickle ever.

Recently Vinit went to the US and got a Hot Sauce made from Carolina Reaper (current record holder for spiciest chilly in the world) for dad and I stole borrowed some of it for my use here in Bangalore. I knew the sauce was spicy but didn’t quite realize how spicy it was till I tried it over the weekend. I was having boiled eggs and they tasted a bit bland so I thought I would use my new sauce to spice things up. I added one drop on each half of the two eggs I had and took ate the whole egg one shot. By the time I finished one egg I had tears coming out my eyes and had to run to eat some butter, cheese and water to take away the heat as my mouth was on fire. After multiple slices of cheese, butter and bread I was ready to face the eggs again and though I did manage to finish the remaining egg I realized that even one drop of the sauce was too much for me.

I am thankful that I decided to play it safe and didn’t slather the egg with the sauce like I usually tend to do, else I could have ended up in the hospital like this poor chap who ate a whole Carolina Reaper for a shot at a record in the Guinness book of World Records. He started with the regular burning sensation and graduated to thunderclap headaches. When the docs did a scan of his brain they found that his arteries had become constricted due to the high levels of capsaicin. To be fair the docs think that this is not a normal reaction and that he must have heightened sensitivity to capsaicin. But still…

Interestingly there is a new contender for the throne of the Spiciest Chilly in the world called Dragon’s Breath which is 1.5 times spicier than even the Carolina Reaper. Which makes me want to try it out even more. I wonder if I can get some of it shipped to India. 🙂

Well this is all for now. Will post more later.

– Suramya

April 7, 2018

Went trekking to Pyramid Hills, Shivanasamudra

Filed under: My Life,Trips — Suramya @ 10:36 PM

This last weekend I went for my first trek of 2018 to Pyramid Hills which is situated in the Kolegal village near Shivanasamudra. The Trek was organized by Plan the Unplanned and was my first experience with the group.

The trek started at 10:30am from Bangalore instead of Friday nights like most of the Treks I have been to in the past which was a bit surprising. Since there were 7 of us friends going for it together and the pickup point was walking distance from my place we decided to have everyone over to my place for breakfast at 9am so that we start the trip on a full stomach. Everyone made to my place on time (yeah I was surprised too…) and we had a filling breakfast of Apple Pie, Eggs, Mattar Paneer and bread before boarding the bus and started the journey into the unknown (or rather to Sivansamudra).

Interestingly this group was a lot less diverse than some of the other Treks I have been to… in the sense that most of us were in the software industry and the average age was in the mid to late 20’s. (If you exclude me and Jani). Usually the groups are a lot more diverse but that could be because our group accounted for 7 out of the 14 people. That being said it was a fun trip which I enjoyed quite a bit and I made some new friends 🙂

Since a lot of us already knew each other, I was expecting the introductions to be a bit boring and repetitive. However I am happy to say that was not the case. The group leaders (Tarun and Aditi) asked us to act out our names or the name of our parent (if we had already introduced ourselves) using dumb charades. It was hilarious to watch folks trying to act out their names and it served as a good ice breaker for the team. The intro followed by an actual game of dumb charades helping us pass the time during the 4 hour drive to reach the base camp. We reached the camp just in time for Lunch and Ratna, the owner of Calver camps had lunch ready for us under an open air setup. As all of us were quite hungry at this time we attacked the food and finished in record time. On a side note, the food on this trip was among the best I have had on treks, usually it is just lemon rice and some snacks but this time it was really good and filling.


The Open Air Dinning hall and welcome area (PC: Durga)

Post lunch we rested for a little while to allow everyone to recover from the post lunch coma and then started on the first trek of the trip. Jani immediately inaugurated the trek by walking into thorny bushes and getting scratched all over. The first part of the trek was through fields that had recently been plowed but there was a clear path through so it was easy walking. The second part of the trek involved some steep climbing, with parts of it at almost 60 degrees. It was steep enough that none of us had breath to sing like we normally do during the walk. Most of us took it easy except for JD who had a bet with Ratna to reach the top within 23 mins. To all our surprise he actually made it to the top in 23 minutes flat to win the bet. The rest of us took quite a lot longer with frequent breaks but were there in time to watch the sunset.


Group Pic a the beginning of the Trek


The Trail to the top of Pyramid Hill (This is how steep it was) [PC: Tarun]


Taking a breather on the way [PC: Tarun]


We Made it! Group Pic at the top of Pyramid Hill [PC: Ayush]

After reaching the top we all relaxed on the rocks for a while, took a whole lot of pics before starting back down. Personally I felt that coming down was easier but not everyone agreed with me as the way down was slippery. At least this time we didn’t have folks having a contest about who would fall most often while coming down. 🙂 By the time we reached the bottom it was dark and we really didn’t have an idea of which way the campsite was supposed to be. So we followed the dog who had accompanied us for the climb, and it took us through a roundabout way but we finally made it back in one piece without falling into a ditch.


Resting after conquering Pyramid Hill

Once back we all freshened up and then relaxed for a bit playing with the kids. The original plan was to do a bit of star gazing before dinner and a movie but as it was cloudy we couldn’t really see anything except for the moon and that was also mostly covered, so we just fooled around for a bit while Tarun and Aditi tried to get the projector and sound system working for an outdoor movie night. Tarun had brought a good collection of movies to choose from and the majority (or at least the loudest) wanted to watch a horror movie, because that would have been a lot of fun. Unfortunately Tarun’s hard-disk’s connector cable died so we only had a choice between two movies and we choose ‘Three Idiots’. This was the first time I have attempted to watch a movie during a trek and to be honest I wasn’t in big favor of it initially but it was quite enjoyable.


Our Movie Hall [PC: Tarun]

We watched half the movie and then stopped to do a bit of star Gazing as Nikhitha & Mahesh noticed the sky was finally clear enough. Unfortunately we were only able to see the moon clearly as Jupiter was not cooperating in getting focused. We spent quite a lot of time just chatting and watching the stars which they both worked on getting the telescope aligned.


Trying to find the moon at 1am [PC: Nimisha]

After the star gazing session we all decided to sleep outside in the open itself as the cabins felt a little hot as compared to outside and since there were no mosquitoes choosing between overheating or getting eaten alive wasn’t a concern either. Most of the night was quite comfortable but it got chilly in the morning to the point where 3 of us (Me, Shahrukh and Shashank) ended up sharing a blanket (all of us were too lazy to walk up to the cottage to get another blanket).


Sharing a blanket as it was cold!

After a heavy breakfast me and Jani spent the morning lazing around the campsite while the rest of the folks attempted the second trek (I had a twisted back). They all survived the trek and came back in one piece in time for lunch which was again quite scrumptious. Post lunch we went ‘fishing’ in a nearby river. Our expectations were that the water was deep enough to swim in so folks who wanted to fish would do so and the rest of us were planning to swim. Unfortunately due to the heat and the fact that the dam upstream hadn’t let out much water that day, the water was only knee deep at the deepest end. 🙁 We still spent some time soaking our feet in the water and tried to get a fish pedicure. In the end we decided to call it a bust and walked back to the bus.

While walking back we saw a mango orchard next to the path and decided to pick a few mangoes to snack on. Me, Aditi and Nimisha were picking the mangoes when all three got an electric shock from the fence as it was an electrified fence which we didn’t know when we started picking the mangoes. Thankfully it was a mild shock but still stung. We collected our bounty of 3-4 mangoes and gave up on collecting more and boarded the bus to go for a coracle ride. Since that was on the same river as the fishing spot the water was not inviting enough to tempt us so we had a whole lot of coconut water and then started back to camp to collect our stuff before leaving for Bangalore. Just as we started back to camp it started raining heavily and we enjoyed the cool rain during the drive. Thankfully it stopped raining by the time we got back to camp and we avoided getting drenched while loading our luggage into the bus.

The drive back was fairly uneventful and while a lot of the folks slept some of us talked the entire time and I am surprised that none of the folks sleeping threw things at us to shut us up. We reached Bangalore by 9:30pm and as we hadn’t had dinner yet and didn’t want the trip to end some of us decided to have dinner together at my place. We picked up food from a restaurant near my place and spent the next few hours chatting away. We would have stayed up longer but everyone had office the next day so we had to call it a night with a heavy heart.

I would recommend the trek to anyone who doesn’t mind staying in a place that doesn’t allow smoking and drinking but serves the most amazing and simple food.

Well this is all for now. Will post more later.

– Suramya

February 13, 2018

Explaining HTTPS using carrier pigeons

Filed under: Interesting Sites,Security Tutorials,Techie Stuff — Suramya @ 7:07 PM

HTTPS is something that a lot of people find hard to explain without going into a lot of technical jargon which frankly just confuses most people and causes them to zone out. However it is an essential service/protocol so understanding it is a good idea. To address this issue Andrea Zanin who is a student created the following primer that explains how HTTPS works using carrier pigeons as the messengers.

Below is an explanation on how HTTP would work with carrier pigeons:

If Alice wants to send a message to Bob, she attaches the message on the carrier pigeon’s leg and sends it to Bob. Bob receives the message, reads it and it’s all is good.

But what if Mallory intercepted Alice’s pigeon in flight and changed the message? Bob would have no way of knowing that the message that was sent by Alice was modified in transit.

This is how HTTP works. Pretty scary right? I wouldn’t send my bank credentials over HTTP and neither should you.

Check out the link for the full writeup.

Well, this is all for now. Will write more later.

– Suramya

February 7, 2018

Hacking the Brainwaves Cyber Security CTF Hackathon 2018

Earlier this year I took part in the Brainwaves Cyber Security Hackathon 2018 with Disha Agarwala and it was a great experience. We both learnt a lot from the hackathon and in this post I will talk about how we approached the problems and some of our learning’s from the session.

Questions we had to answer/solve in the Hackathon:

  • Find the Webserver’s version and the Operating system on the box
  • Find what processes are running on the server?
  • What fuzzy port is the SSH server running on?
  • Discover the site architecture and layout.
  • Describe the major vulnerability in the home page of the given website based on OWASP TOP 1. Portal Url: https://socgen-ctf.0x10.info
  • Gain access to member area and admin area through blind sql, or session management.
  • Dump all user account from member area. [SQLi]
  • [Broken Validation] Demonstrate how you can modify the limit in order management.
  • [Open Redirect] Redirect site/page to hackerearth.com
  • List any other common bug came across while on the site
    • After logging into the member area, perform the following functions:
    • Find the master hash & crack it
    • Dump all user’s
    • Find the email ID and password of saved users

Information Gathering:

In order to find the services running on the server, the first thing we had to do was find the IP/hostname of the actual server hosting the site which was a bit tricky because the URL provided is protected by CloudFlare. So, any scans of socgen-ctf.0x10.info took us to the CloudFlare proxy server instead of the actual server which was a problem.

We figured this out by trying to access the IP address that socgen-ctf.0x10.info translated to in the browser.

suramya@gallifrey:~$ host socgen-ctf.0x10.info 
socgen-ctf.0x10.info has address 104.28.15.64 

Since the site homepage didn’t do anything except display text that refreshed every 15 seconds we needed to find other pages in the site to give us an a attack surface. We checked to see if the site had a robots.txt (It tells web crawlers not to index certain directories). These directories are usually ones that have sensitive data and in this case the file existed with the following contents:

# robots.txt
Sitemap: http://socgen-ctf.0x10.info/sitemap.xml
User-agent: *
Disallow: images
Disallow: /common/
Disallow: /cgi-bin/

The images directory didn’t have any interesting files in it but the /common/ directory on the other hand had a file named embed.php in it which basically ran a PHP Info dump. This dump has a lot of information that can be used to attack the site but the main item we found here was the IP address of the actual server where the services were running (38.109.218.93).

Using this information we were able to initiate a nmap scan to get the services running on the site. The nmap command that gave us all the information we needed was:

nmap -sV -O -sS -T4 -p 1-65535 -v 38.109.218.93

This gave us the following result set after a really really long run time:

PORT     STATE    SERVICE       VERSION
23/tcp   filtered telnet
25/tcp   open     smtp?
80/tcp   open     http          This is not* a web server, look for ssh banner
81/tcp   open     http          nginx 1.4.6 (Ubuntu)
82/tcp   open     http          nginx 1.4.6 (Ubuntu)
137/tcp  filtered netbios-ns
138/tcp  filtered netbios-dgm
139/tcp  filtered netbios-ssn
445/tcp  filtered microsoft-ds
497/tcp  filtered retrospect
1024/tcp open     kdm?
1720/tcp open     h323q931?
2220/tcp open     ssh           OpenSSH 6.6.1p1 Ubuntu 2ubuntu2.8 (Ubuntu Linux; protocol 2.0)
2376/tcp open     ssl/docker?
3380/tcp open     sns-channels?
3389/tcp open     ms-wbt-server xrdp
5060/tcp filtered sip
5554/tcp filtered sgi-esphttp
8000/tcp open     http          nginx 1.4.6 (Ubuntu)
8080/tcp open     http          Jetty 9.4.z-SNAPSHOT
8086/tcp open     http          nginx 1.10.3 (Ubuntu)
9090/tcp open     http          Transmission BitTorrent management httpd (unauthorized)
9996/tcp filtered palace-5
19733/tcp filtered unknown
25222/tcp filtered unknown
30316/tcp filtered unknown
33389/tcp open     ms-wbt-server xrdp
33465/tcp filtered unknown
34532/tcp filtered unknown
35761/tcp filtered unknown
35812/tcp filtered unknown
35951/tcp filtered unknown
37679/tcp filtered unknown
38289/tcp filtered unknown
38405/tcp filtered unknown
38995/tcp filtered unknown
40314/tcp filtered unknown
44194/tcp filtered unknown
47808/tcp filtered bacnet

For some reason the results from the nmap scan varied so we had to run the scan multiple times to get all the services on the host. This was possibility because the server was setup to make automated scanning more difficult.

Once we identified the port where the SSH server was running on (2220) we were able to connect to the port and that gave us the exact OS Details of the server. We did already know that the server was running Ubuntu along with the kernel version from the PHP Info dump but this gave us the exact version.

Discovering Site architecture:

Since we had to discover the URL to the members & admin area before we could attack it, we used dirb which is a Web Content Scanner to get the list ofall the public directories/files on the site. This gave us the URL’s to several interesting files and directories. One of the files identified by dirb was https://socgen-ctf.0x10.info/sitemap.xml. When we visited the link it gave us a list of other URL’s on the site of interest (we had to replace the hostname to socgen-ctf.0x10.info) including the members area (http://socgen-ctf.0x10.info/members.php?p=login) and siteadmin (http://socgen-ctf.0x10.info/siteadmin).

After a long and fruitless effort to use SQL Injection on the siteadmin area we started to explore the other files/URL’s identified by dirb. This gave us a whole bunch of files/data that seem to be left over from other hackathons so we ignored them.

SQL Injection

The main site https://socgen-ctf.0x10.info/index.php?p=. appeared to be vulnerable to SQL at the first glance because when we visit https://socgen-ctf.0x10.info/index.php?p=.’ (note the trailing single quote) it reloads the page. This meant that we could write queries to it however since it didn’t display a true or false on the page a SQL injection wasn’t easily possible. (We could have tried a blind injection but that would require a lot of effort for a non-guaranteed result.

As we explored the remaining URL’s in sitemap.xml one of the links (https://socgen-ctf.0x10.info/embedframe.php) was interesting as it appeared to give a dump of data being read from the site DB. Opening the site while watching the Developer Toolbar for network traffic identified a URL that appeared to be vulnerable to SQL injection (https://socgen-ctf.0x10.info/ajax.php?cid=&p=view_channel&id=28) and once we tested the url we found that the variable id was indeed vulnerable to injection.

We used blind sql to gain access by executing true and false statements and see that it returns different results for true(displays ‘1’ on the webpage) and false (displays 0) . We checked whether a UNION query runs on the site which it did and using other queries we identified the DB backend to be a mysql database (5.xx.xxx version). Then we found out the table name (members) which was an easy guess since the website had an add customer field. After identifying the number of columns in the table we got stuck because any statements to list the available tables or extract data were failing with an error about inconsistent column numbers.

Finally, we ran sqlmap which is an open source tool for automating SQL injection. It took us a few tries to get the software running because initially any attempt to scan the site was rejected with a 403 error message. Turns out that the connections were being rejected because the site didn’t like the useragent the software was sending by default and adding a flag to randomize the useragent resolved the permission denied issue.

Once the scan ran successfully we tried to get access to the MySQL usertable but that failed because the user we were authenticating as to the MySQL server didn’t have access to the table required.

sqlmap -u 'https://socgen-ctf.0x10.info/ajax.php?cid=&p=view_channel&id=28' --random-agent -p id --passwords

So, then we tried getting an interactive shell and an OOB shell both of which failed. We finally ran the command to do a full dump of everything that the system allowed us to export using SQL injection via SQLMap. This included the DB schema, table schema’s and a dump of every table on the database server which the mysql user had access to. The command we used is the following:

sqlmap -u 'https://socgen-ctf.0x10.info/ajax.php?cid=&p=view_channel&id=28' --random-agent -p id  --all --threads 3

This gave us a full dump of all the tables and the software was helpful enough to identify password hashes when they existed in the table and offered to attempt decryption as well. In this case the password was encrypted with a basic unsalted MD5 hash which was cracked quite easily. Giving us the password for the first two accounts in the database (admin & demo).

Looking at the rest of the entries in the users table we noticed that they all had funny values in the email address field, instead of a regular email address we had entries that looked like the following:

,,,"0000-00-00 00:00:[email protected]509a6f75849b",1
,1,RU,

As we had no clue what this was about the first thing we attempted was to access the
https://socgen-ctf.0x10.info/cdn-cgi/l/email-protection URL. This URL gave us a message that told us that the email addresses in the DB were obfuscated by CloudFlare to protect them from Bots. A quick Google search gave us a 21 line python script which we tweaked to convert all the hash to email address and passwords. (The code is listed below for reference)

#! /usr/bin/env python 
# -*- coding: utf-8 -*- 
# vim:fenc=utf-8 
# 
# Copyright © 2016 xl7dev  
# Distributed under terms of the MIT license. 

""" 

""" 
import sys 
import re 
fp = sys.argv[1] 
def deCFEmail(): 
   r = int(fp[:2],16) 
   email = ''.join([chr(int(fp[i:i+2], 16) ^ r) for i in range(2, len(fp), 2)]) 
   print email 
if __name__ == "__main__":                                                                                                                                                                       
   deCFEmail() 

This gave us the email addresses and passwords for all the users on the site. Since the accounts appeared to be created by SQL injection a bunch of them didn’t have any passwords but the remaining were valid accounts for the most part and we verified a couple by logging in manually with the credentials.

OWASP TOP 10 Vulnerability

To find the vulnerabilities in the home page we tried various manual techniques at first but drew a blank so we decided to use the owasp-zap. This tool allows you to automatically scan for vulnerabilities in a given URL along with a whole other stuff.

At first the scan failed because of the same issue as earlier with the user-agent. This time we took a different approach to resolve the issue by configuring owasp-zap as a proxy server and configuring Firefox traffic to use this proxy server for all traffic. This gave us the site in the software and we were then able to trigger both an active scan and spider scan of the site.

This gave us detailed reports that highlighted various issues in the site which we submitted.

Redirecting HomePage

The redirection of the home page was quite simple. We tried inserting a customer name with javascript tags in it and were able to do so successfully. So we inserted the following into the DB and the system automatically redirected the page when the Customer list section was accessed.

Other Interesting Finds

The nmap scan told us that in addition to port 80 a web server was listening on ports 81, 82, 8000, 8080 and 8086.

Ports 82, 8000 and 8086 were running standard installs of nginx and we didn’t find much of interest at these ports even after we ran dirb on all of them. Port 8080 appeared to be running a proxy or a Jenkins instance.

Port 81 was the most interesting because it was running a nginx server that responded to any queries with a 403 error. When we tried accessing the site via the browser we got an error about corrupted content.

We were unable to identify what the purpose of this site was but it was interesting.

SSH Banner / PHP Shell

The webserver instance running on port 80 had the version set to the following text “This is not* a web server, look for ssh banner Server at private-tunel.wehostservers.ru Port 80” so we went back and investigated the SSH Banner from the ssh server on port 2220. The banner was encrypted and to decrypt the SSH banner, we continuously converted the cipherText from its hex value to ASCII value . It gave us the following results on each conversion

3333333733333333333333373333333333333336333333383333333233333330333333363333333233333336333333313333333633363335333333363336333533333336333333353
3333337333333323333333233333330333333363333333633333336333633363333333733333332333333373333333733333336333333313333333733333332333333363333333433333332333333303333
3337333333333333333633363333333333363333333133333337333333333333333633333338333333323333333033333336333333333333333633363336333333373333333533333336333633333333333
63333333433333332333333303333333633363333333333363333333533333336333333313333333633333334333733393336363633373335373436663230363132307368336c6c2e706870

3337333333373333333633383332333033363332333633313336363533363635333633353337333233323330333633363336363633373332333733373336333133373332333633343332333033373333333
636333336333133373333333633383332333033363333333636363337333533363633333633343332333033363633333633353336333133363334373936663735746f206120sh3ll.php
 37333733363832303632363136653665363537323230363636663732373736313732363432303733366336313733363832303633366637353663363432303663363536313634796f75to a #

ssh banner forward slash could lead you to a #sh3ll.php

Once we got the full decrypted text we knew that there was a potential webshell on the server but it wasn’t apparent where the shell was located. After hit and try failed we turned back to our old faithful dirb to see if it could find the shell.

dirb allows us to specify a custom word list which is used to iterate through the paths and we can also append an extension to each of the words to search for, so we created a file called test with the following content:

suramya@gallifrey:~$ cat test 
shell
sh3ll
sh311

and then ran the following command:

suramya@gallifrey:~$ dirb https://socgen-ctf.0x10.info/ test  -X '.php'

This gave us the location of the shell.


Accessing the link gave us a page with a message “you found a shell, try pinging google via sh3ll.php?exec=ping 8.8.8.8”

Accessing the URL with the additional parameter gave us a page with the following output:

« Newer PostsOlder Posts »

Powered by WordPress