Suramya's Blog : Welcome to my crazy life…

May 17, 2021

IBM’s Project CodeNet: Teaching AI to code

Filed under: Computer Software,Emerging Tech,My Thoughts — Suramya @ 11:58 PM

IBM recently launched a new program called Project CodeNet that is an opensource dataset that will be used to train AI to better understand code. The idea is to automate more of the engineering process by applying Artificial Intelligence to the problem. This is not the first project to do this and it won’t be the last. For some reason AI has become the cure all for all ‘ills’ in any part of life. It doesn’t matter if it is required or not but if there is a problem someone out there is trying to apply AI and Machine Learning to the problem.

This is not to say that Artificial Intelligence is not something that needs to be explored and developed. It has its uses but it doesn’t need to be applied everywhere. In one of my previous companies we interacted with a lot of companies who would pitch their products to us. In our last outing to a conference over 90% of the idea’s pitched had AI and/or Machine Learning involved. It got to the point where we started telling the companies that we knew what AI/ML was and ask them to just explain how they were using it in their product.

Coming back to Project CodeNet, it consists of over 14M code samples and over 500M lines of code in 55 different programming languages. The data set is high quality and curated. It contains samples from Open programming competitions with not just the code, it also contains the problem statements, sample input and output files along with details like code size, memory footprint and CPU run time. Having this curated dataset will allow developers to benchmark their software against a standard dataset and improve it over a period of time.

Potential use cases to come from the project include code search and cloud detection, automatic code correction, regression studies and prediction.

Press release: Kickstarting AI for Code: Introducing IBM’s Project CodeNet

– Suramya

May 15, 2021

Providing Oxygen through the intestines in Mammals is now possible as per research

Filed under: My Thoughts,News/Articles,Science Related — Suramya @ 11:53 PM

It takes a certain kind of mind to decide that today I am going to experiment if mammals can absorb oxygen through their intestines. Apparently a some of the aquatic animals like sea cucumbers and catfish, breathe through their intestines and since humans can absorb medicines through their intestines Takanori Takebe, a gastroenterologist from Cincinnati Children’s Hospital decided to do a study to see if they can absorb oxygen as well. So to test this out they basically injected pure pressurized oxygen into the rectums of the scrubbed mice (the mucus layer was thinned) and four of the seven unscrubbed ones. There was an immediate improvement in the O2 levels of the mice, with 75% the scrubbed mice surviving the procedure.

Obviously that is not a great survival rate and the scrubbing procedure is dangerous/involved but it did prove that mammals can absorb o2 with their intestines. So they looked at using perfluorocarbons which have a high O2 level and giving the rats & pigs an enema of the fluid. They saw an almost 15% improvement in the blood oxygen saturation allowing the subjects to recover from hypoxia.

These two tests prove that mammals can breath through their intestine but there is still a lot of study that needs to be done to check for the safety of this procedure. But if things go smoothly we can be looking at a new way to provide oxygen to patients when O2 canisters are in limited supply like the case currently in India due to the Covid crises.

But this doesn’t mean that mouth to mouth CPR will be replaced with mouth to ass CPR. (I can hear the sigh of relief from medical professional/emergency care folks).

More details on the study: ScienceMag: Mammals can breathe through their intestines
Full Paper: Mammalian enteral ventilation ameliorates respiratory failure

– Suramya

May 14, 2021

NTFS has a massive performance hit on Linux compared to ext4

Filed under: Computer Software,Linux/Unix Related,My Thoughts,Techie Stuff — Suramya @ 12:47 PM

NTFS has long been a nemesis of Linux. I remember in the 2000’s getting NTFS working on linux required so much effort and config changes that I stopped using it on my systems as FAT32 was more than sufficient for my needs at that time. Initially the driver was very unstable and it was recommended that you only use it for Read operations rather than Read/Write as there was a high probability of data corruption. That has changed over the years and the driver is stable. However, there is a massive performance hit when using NTFS vs ext4 on a Linux machine and I saw this when I tried using a NTFS partition on my laptop instead of ext4.

I have a 1 TB drive on my laptop along with a SSD. I dual boot the laptop (need it for my classes) between Windows & Debian and wanted to have all my files available on both OS’s. When I last tried this, ext support on Windows was not that great (and I didn’t feel like searching for options) so I decided to format the drive to NTFS so that I would have access to the files on both OS. The formatting took ages and once the drive was ready I was able to copy my files from the desktop to the laptop. While the files were being copied I noticed very high CPU usage on the laptop and the UI was lagging randomly. Since I was busy with other stuff I let it be and ignored it.

Yesterday I was trying to move files around on the laptop so that the root partition had enough space to do an upgrade and I again noticed that file copy and most of the disk operations were taking way longer than I expected. For example there would be a second of delay when I tried listing the directory when it had a lot of files. So, I decided to test it out. My data on the Laptop is an exact copy of the files on the Desktop. I timed the commands on the desktop with the same command on the laptop and there was a significant difference.

My desktop is obviously a lot more powerful than the laptop so I decided to try an experiment where I would run a command on the NTFS drive, then format the drive to ext4 and run the same command. (after copying all the files back). When I did this I saw that there was a massive difference in the time it took to run the command. On ext4 the command took less than 1 second (0.107s) whereas it took almost 34 seconds (33.997s) on NTFS parition. The screenshot for both commands are below:

du -hs command on a ext4 partition

du -hs command on a NTFS partition

That’s a ridiculous amount of difference between the two. So I obviously have to switch back to ext4 which brought us back in a full circle – I still needed to be able to access my files from Windows as well as from Linux. Decided to go a search on the Internet for options and found out that Windows 10 now lets you mount Linux ext4 filesystems in WSL 2. I haven’t tried it yet but I will test over the next few days once I am done with some of my assignments. If there is something interesting I will blog about it in the near future.

As of now, I am back to using ext4 on the laptop and the OS performance is a lot better.

Well this all for now. Will post more later.

– Suramya

May 13, 2021

Some thoughts on the last episode of ‘The Flash’

Filed under: My Thoughts — Suramya @ 12:36 PM

One of the shows I’ve been watching consistently for the past few years is ‘The Flash’ and the last episode of the show was a bit weird and I really need to vent about it. If you have not seen the last episode (Season 7 Episode 9: Timeless) yet stop reading because this post has major spoilers for the episode.

Ok since you are still here, I assume you seen the episode. So lets get started. In the past few episode Barry/Flash has been fighting the ‘forces’ and in the last episode Nora/Speed Force killed the Strength Force. This was a big deal. So now Team Flash is trying to figure out how to stop the other forces who are basically Gods from being killed and removing their powers at the same time. Barry decides that the best way to do that is to travel back in time to prevent the forces from being formed in the first place. It’s like they didn’t learn anything thing from the whole Flashpoint fiasco. The entire team opposes the idea but Barry just handwaves their concerns away to say that he is going to do it even if they disagree. He then pulls in the Timeless Wells who would create a ‘bubble’ around the timeline to prevent it from changing after Barry prevents the forces from being born. This would be a huge paradox and the show just ignores it.

Finally Barry & Wells are back in the past and well into the process of stopping the forces from being born when suddenly Barry does an about face for no good reason (Other than to keep the show going) and lets the forces be born. I don’t mind when the characters change their decision because of some discussion or new input but this guy just makes arbitrary decisions and then changes them for no reason. He is like a Yo-Yo with these decisions and it is really annoying.

Then they both come back to the present and because Iris & Barry have a ‘connection’ to the speed force, they are able to bring Fuerza/Alexa back to life. First of all why have they put her on a medical bed if she is dead? Are they planning on doing experiments on her? Or was it just so that she could be brought back to life because the character was needed in the show.

This episode was just lazy writing and this could have been done a lot better. Let’s see what the next episode brings, hopefully it will make more sense.

– Suramya

May 12, 2021

Using a centrifuge to improve a compressor

Filed under: My Thoughts,Science Related — Suramya @ 11:19 PM

A compressor is such a standard piece of equipment that we no longer think about it and it is used everywhere from pumps to air conditioners. If we can improve the design then it can lead to a huge power savings. A California company (Carnot) has come up with a new design that reduces the noise output, lasts longer and reduces the cost of ownership by 20 percent while using no oil. The new design uses a Trompe and combines it with a centrifuge to get it to work more efficiently.

A Trompe is an ancient technology which uses falling water to compress air. It was used extensively till fossil fuels made it less desirable by producing more power. Historically, miners have used large-scale trompes to power their mining equipment and provided ventilation. In fact a 350-feet-deep trompe was setup in Michigan which was able to produce 5,000 horsepower.

A trompe, often placed in a river, has a simple design. Flowing water falls into an intake pipe which has an air cone (or some other aerating device) on top. The water falling around the cone creates suction, pulling the air down with it. Air bubbles travel down the pipe with the water until reaching an air chamber.

At the air chamber (also called a plenum or reservoir), the bubbles escape from the water. In the process, the air has been compressed, dehumidified, and cooled to the same temperature as the water. The pressurized air can now be put to use.

Meanwhile, the water leaves through the outtake pipe. Air pressure from the reservoir pushes the water upwards, nearly to the same height it originally fell from, and the water returns to the river.

The issue with Trompe is that it needs a large setup if we want to generate enough compression and that is obviously not possible in all places. So the Carnot team explored options to accelerate the water movement and arrived at using a centrifuge. In their setup the compressor sucks in air through a filter at the top and mixes it with water on top of a fast moving drum, this outward force compresses the air and the mixture of air & water is forced out at that bottom where they are separated into air and water as they pass through the exit channel. The image below gives a good overview of how the technology would work:

Model of Carnot compressor

The resultant compressor is very quiet since the only moving parts are the spinning drum powered by an electric motor and an exhaust fan. In lab tests the system was found to operate indoors at below 70 dB. Obviously the technology is still quite new and needs to mature a lot before bulk deployment, but it is very interesting and I see a great future for it if it works as advertised.

Source: Carnot puts a centrifugal spin on a 500-year-old air compressor design
The Trompe: A Basic Overview

– Suramya

May 11, 2021

Stop hating on people because they don’t use the same tools as you

Filed under: Linux/Unix Related,My Thoughts — Suramya @ 6:31 PM

Recently there was an idiot on twitter who kept harassing a lady (@insiderPhd) to tell her that she was not a real hacker because she didn’t use vi. The screen shot of the original post is at the bottom of the post. The quote I really found interesting is “It’s not that you ‘can’t use’ vim’, it is that you haven’t taken ANY time to learn it. If you can learn API hacking tools, you can learn vim. Don’t be lazy!”. How is not learning vim/vi being lazy. I didn’t learn it for a long time because I didn’t need to. Then Matt convinced me that I need to know it and his reasoning was that if a system goes down and you have to boot into recovery mode or are working on an embedded system the only editor you can be sure will be available is vi and that made sense to me. It did take me a while to get comfortable with it but I still prefer a GUI based editor like EditPlus on Windows or kWrite on linux. Does that mean that I am not a ‘hacker’? If so then so be it. I am comfortable with what/who I am and I don’t need you to validate me. Unfortunately, for newcomers into the field this can be discouraging so please stop doing that. Plus, we need to call out these idiots who think they are the sole authority on how is a hacker or a techie or a geek or whatever.

I really don’t understand these guys and it is always a guy who is trying to gatekeep and tell others that they are the sole authority to decide who is a hacker or who is not. Who is in and who is not. I have much better things to worry about than trying to get approval from some jackass who thinks their way is the only way. Remember the famous “infinite diversity in infinite combinations” quote? It means that there is more than one way to do it.

We have a lot of old-timers who are scared of the new generation and their new-fangled ideas. This is because they stopped learning and are now scared that they will be replaced. In one of my previous companies, I was responsible for creating a system that replaced a legacy system with a user friendly web based system. It worked great (if I say so myself) but one of the senior engineers I was working with did everything they could to sabotage the project because they were comfortable with the old system and didn’t want to change. Plus it was job security because hardly anyone else understood the system so they were always needed. My feeling is that if you need to prevent upgrades because you think you would loose your job because only you know the system then understand that the company is already looking for ways to replace you. Single points of failures are a big issue for a company.

Now, coming back to the original point about tools. I really don’t get why people get so worked up over a simple tool. This whole religious war over vi vs emacs or Windows vs Linux or Android vs iPhone and so on is just silly. It is ok to have a preference, I for example prefer using vi, Linux and Android phones. But that is because those tools work for me and I am comfortable with them. There are aspects of iPhone that I don’t like because of the design philosophy behind it. I have spent hours debating which phone is better and I will continue to do so. That being said, it doesn’t mean that people who don’t use an Android phones are non-technical or uneducated or whatever. End of the day a phone is a tool, it needs to do what I want it to do and respect my freedom to do what I want with it.

It doesn’t matter if I use vi to code or I use an IDE that helps me code. It matters what I do with it, what program I am writing. I can never remember syntax’s for functions, even for languages I have been using for decades. Does that mean I can’t program? of-course not. That’s what google and the reference books are there for. In my 10th Board exam, during my computer science viva I was asked to give the syntax for the locate command and I mixed up the rows & columns parameters and inverted them. My teacher scolded me and told me how can I believe you coded this program if you can’t even tell me the syntax, so I told her that I can always look up the syntax when I am coding and the important part is that I need to know when to use the command, not what the syntax is. There was pin-drop silence after I told her that. I did get full marks so I guess I made sense. But the point is that tools are there to help you. You need to figure out how to use them effectively.

I use Linux on all my machines but Jani and my parents use Windows because that is what works for them. It doesn’t mean that they are scum or unintelligent or whatever, (these are actual terms I have hear people use about folks who use Windows) it is just that for them it doesn’t matter. It is not what is important to them.

Too many people try to use the tools you use as a criteria to like you or hate you and to me that is a sad way to live your life.

I learn a lot from others who are different from me because they have a different view point. If everyone in the world was the same imagine how boring it would be? I am not a singer, but others are and they create great music so does that make them better or worse than me? It doesn’t. They are just different and that is good. We need that diversity.

We all need to stop focusing on the differences and start focusing on how we all love the same stuff.

Original post on Twitter, ranting on how someone is not a hacker because they don’t use vim

– Suramya

May 9, 2021

Teaching Cyber Security basics to kids

Filed under: Computer Security,My Thoughts — Suramya @ 8:04 PM

There is an ongoing effort over at Australia to teach cyber-security to five-year-old kids. I am sure that it will be no surprise to anyone who knows me that I think that this is a brilliant idea. Security is a mindset and the earlier we can teach kids about the pitfalls and dangers online, the safer they will be online.

Our generation grew up with the internet and still I see that most people are not that serious about security. I had a long argument/discussion with Jani on why she had to have a passcode for her phone and why she couldn’t use the same password for everything. Now she understands what I was talking about and uses a password manager with unique password for each account. But that is not the same with my parents, I still have not managed to convince them to use a password manager. 🙁

A little while ago I was talking to mom and she commented that my nephew Vir doesn’t share his account passwords with anyone and when my mom is typing her password he looks away. I credit Vinit for teaching him this and am really happy about it. This is what you get when a kid is taught about security from the get go. Instead of learning it later as an add on. Another year or so and I will have him start using a password manager as well.

Habits learnt as a kid are really hard to unlearn and that is why I think it is really important that we get to kids as early as possible and teach them about cyber security. I mean we already teach them regular security and safety so why not cyber security and safety? Remember, they are spending a lot more time on the computer and the internet than we ever did and they need to be taught how to be careful online.

Well this is all for now. Will post more later.

– Suramya

May 2, 2021

Infinite Nature: Creating Perpetual Views of Natural Scenes from a Single Image

Filed under: Emerging Tech,Interesting Sites,My Thoughts — Suramya @ 11:28 PM

Found this over at Hacker News , where researchers have created technologies that use existing video’s and images and extrapolate them into an infinite scrolling natural view that is very relaxing to watch and at times looks very tripy. The changes are slow so you don’t see how the image is changing but if you wait for a 20 seconds and compare that image with the first one you will see how it differs.

We introduce the problem of perpetual view generation—long-range generation of novel views corresponding to an arbitrarily long camera trajectory given a single image. This is a challenging problem that goes far beyond the capabilities of current view synthesis methods, which work for a limited range of viewpoints and quickly degenerate when presented with a large camera motion. Methods designed for video generation also have limited ability to produce long video sequences and are often agnostic to scene geometry. We take a hybrid approach that integrates both geometry and image synthesis in an iterative render, refine, and repeat framework, allowing for long-range generation that cover large distances after hundreds of frames. Our approach can be trained from a set of monocular video sequences without any manual annotation. We propose a dataset of aerial footage of natural coastal scenes, and compare our method with recent view synthesis and conditional video generation baselines, showing that it can generate plausible scenes for much longer time horizons over large camera trajectories compared to existing methods.

The full paper is available here Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image with a few sample generated videos. One of the examples is below:

This is a very impressive technology. I can see a lot of uses for it in video games to generate real estate for flight simulators to fly over or fight over. It can be used for VR world developments or just to relax people. It might also be possible to take footage from TV shows and extrapolate them to allow folks to explore it in VR. (After a lot more research is done on this as the tech is still experimental). We could also simulate alien worlds using pics taken by our probes to train astronauts and settlers realistically instead of relying on fake windows and isolated area’s.

Check the site out for more such videos. Looking forward to future technologies built up over this.

– Suramya

May 1, 2021

Do you need to display your books in a way to look more sophisticated?

Filed under: My Thoughts — Suramya @ 10:32 PM

Anyone who knows me, knows that I love books and own a lot of them, both digital and physical. I prefer physical books as they are more tactile and because I grew up reading physical books but I love my kindle just for the fact that I can add 20 books to it and still have to carry only a slim device. The first time I traveled to the US my carry on luggage consisted of 6-7 novels a jacket and my documents/other important papers etc and I finished all the books before I landed. I still carry a lot of books when I travel. So, a good book shelf is something I am really interested in. I tried some of the fancy book shelves but I have too many books for them to work for me. At my parents place we have a whole room with shelves up to the ceiling just for books and at my place in Bangalore I have a whole wall of shelves going up to the roof with a provision to extend the shelves if required. I have slowed down a bit on the book purchases in the past 6 months because I am supposed to be studying 🙂 but I plan to get back to it with a vengeance once that is done.

There is an article on Lifehacker on How to Display Your Books Like a More Sophisticated Adult. I looked at it because I thought I could learn some interesting/new shelving technique, but that is not the case (pun not intended) here. It is yet another effort for people to show off / pretend that they read books and try to impress others with their collection & ‘wisdom’. Basically this whole thing is to look down on others, pretend to have sophistication and judge others. As the article says

Nice-looking bookcases look professional. Elegant, even. They show off a sense of sophistication that others will notice.

Take the Twitter account Bookcase Credibility, for example, as a great place for learning (and judging) bookcase arrangement.

Don’t believe me? Let’s look at some of the suggestions from the post:

Consider arranging your books by color : I really don’t see the point in this. I don’t think anyone who likes reading would want to shelve their books by color, it just makes it harder to find other books by the same author, or of the same topic. Making it look colorful is not the objective for me, it is to organize it in a way where I can find books quickly and easily. Now we might disagree on how to order the books (personally I like by author) but I can see the sense of putting them by genre or categories as well. Color is just such a random way of putting the books.

Display your books in wall boxes : This is pure show-off for books. If I tried it I would only have boxes all over the wall and the extra space used by the wood would be unusable to store books. Which is the point of the shelves.

Use a suspension book rack to make a statement Another just for show book shelf. If I tried this for my books the weight would be too much for the anchor. Plus there is a lot of space wasted that can be used to store books.

How does this make any sense? Books are not laundry to be hung around

Make coffee tables and nightstands with larger books : I thought about this, I don’t want to put my books as a night stand I have spilled enough fluids on the stand and stored random crap on them that it would absolutely spoil the books. Plus if I want to read a book at the bottom there is no way to take it out without having to dismantle the entire thing and then I can’t put it back together till I finish the book and can put it back. Which means that I would have a pile of books on my floor, plus no nightstand. How is this a recommendation for someone who actually reads the books they purchase? Within a year the books will have stains and marks all over them from spills etc.

Someone please explain to me how I would take out the Unix Installation book or Art at Auction to read without having to dismantle everything?

Now, since I have been talking about bookshelves, here are mine:

My collection in Delhi

My collection in Bangalore

The Delhi pic is a few years old, but the Bangalore one is current (I took it while writing this post). Yes, I am read most of the books in both places. The only ones I haven’t read are the Law books and the few Biographies that Jani and mom have bought. My dream house would have a few rooms just for books. so that is a goal that I am working towards… 🙂

So, to conclude don’t bother with showing off your books in some fancy way. If you are passionate about something it will come through without you having to showcase it. Also, people can make out if the books are there just for show so keep that in mind as well.

Well this is all for now. Will write more later.

– Suramya

April 30, 2021

Review and test of Fawkes: Software to protect your pictures from AI/Reverse searches.

Filed under: Computer Software,My Thoughts — Suramya @ 11:28 PM

Yesterday, I wrote about Fawkes & Photo Ninja which can be used to protect your photos from facial recognition models and reverse image searches. This is a very interesting field and I had mentioned about creating a service that does it for free instead of charging like what Photo Ninja is doing.

The first step to that is to check if the program (Fawkes) actually works the way it is supposed to, so I downloaded a pic from the internet (my profile pic on Twitter) and ran it through Fawkes. The program takes a while to run (~20 seconds per image) depending on the no of people in the photo. It detected the faces very reliably and modified the image. When using the default settings the output is saved as a PNG file but you can override it using a command line parameter. It requires you to provide the directory you want to run it against but if you don’t pass it the directory, it doesn’t give any errors. It took me a few mins to figure out what the issue was (yes, I know… My brain is tired). The command to run it in the current directory with debug (because I like seeing what the software is doing) is:

./protection --debug --directory .

I then took the resultant, file and searched for it via Google Images, Yandex and TinEye. None of them were able to find any results with the new image. So that part of the software works great. 🙂 Now coming to how the software modifies the image, I saw that it adds 2 rows of pixelisation to the image. First is near the hairline and cuts across the hair and forehead, and the second is near the chin and is about 5-10 pixels wide. It is clearly visible in larger photos, but when zoomed out it doesn’t look too jarring. Frankly it looks like the image got damaged and is kind of obvious when you look at it.

In my very basic tests it made the same change everytime so I have a feeling that we can train image recognition software to look for this modification and ignore it. It might be more powerful to put the modifications at random locations in the image (over the faces) that way it is harder to train the software to counter it. Plus if the visual noise section can be reduced it would be great. Maybe instead of a long blur that is noticeable we can try to do multiple small changes that change the pic without making it obvious that the image was modified.

Below are the two images, the original on the left and the modified version on the right.

Sample output of the Fawkes

I then looked at running this on my webserver, but due to the restrictions there I wasn’t able to get it to run. Although, to be honest I only tried for about 20-30 mins because I was tired. If I can’t get it to run on the server then the other option is that I run it on my home computer but I will need to look at that in more detail before I commit to making this site. I have a rough draft of the requirements and feature list but still looking at the options before I start working on it. It will be a good way to take my mind of what is going on in the world so that is good.

Well this is all for now. Will keep you posted on how this project goes.

– Suramya

Older Posts »

Powered by WordPress