Suramya's Blog : Welcome to my crazy life…

March 13, 2024

Computers/Technology is not the cure to the worlds problems and its time we stop pretending otherwise

Filed under: Artificial Intelligence,My Thoughts,Tech Related — Suramya @ 11:56 PM

As a software developer we tend to be pretty confident that software or algorithms can solve all the problems in the world because we are using ‘technology’/AI/LLM/Blockchain or whatever the buzzword of the day is to solve a problem. This is an issue because when we look at a problem from an outsider’s perspective it looks fairly simple because we don’t know enough to realize the complexity. Or put another way we don’t know enough to know what we don’t know (the unknown unknowns). As always XKCD has a comic that talks about this:


Megan: Our field has been struggling with this problem for years.
Cueball: Struggle no more! I’m here to solve it with algorithms!
Six months later:
Cueball: Wow, this problem is really hard.
Megan: You don’t say.

To be fair, computers have solved a lot of problems in the world and have had a tremendous impact on it, but that doesn’t mean that they are the key solving for every problem. There is a saying that I love quoting “When all you have is a hammer, everything looks like a nail” and as a developer/techie a lot of us tend to forget this. We look at a problem and think that its an easily solved problem and in most cases that is true during the testing in controlled situations. Once you try the same in the real world things turn out a lot more differently. For example, in a 2020 study, a deep learning model was shown to be more accurate in predicting whether bladder cancer has spread in a patient and other models also showed similar results. Unfortunately, when the model was implemented in the real world the results where a lot more ambiguous and not as rosy as we thought.

The major problem we have right now is that AI can give us information at sounds authoritative and accurate especially if it is about a topic you know nothing about because you don’t quite know well enough to identify the nonsense it sprouts. This is similar to how movies and TV shows portray technology or medical science, they will bombard us with buzz words and if you know nothing about the topic it sounds impressive otherwise you are either completely confused or rolling on the floor laughing.

We need to actually look at the problem, understand it and then start implementing a solution. Move fast and break things is not a feasible working model unless you just want to create a buzz so that your technology/company gets acquired and then it is not your problem to get it to work.

– Suramya

March 11, 2024

Fans who froze themselves to watch a match rewarded by having to get their frostbitten limbs amputated

Filed under: My Thoughts — Suramya @ 4:14 PM

People are crazy about sports. In India its cricket, US it’s football, UK is rugby and Brazil has soccer. I am not a huge fan but even if I was I don’t think I would sit outside in minus-27 degrees just to watch a game. However, there are people in the US who actually did this to watch the Kansas city & Chiefs game earlier this year and ended up getting hypothermia & frostbite bad enough that 70% of them need amputations.

Yes you read that right. 70% of the fools who sat outside in the freezing cold are now looking at having their body parts amputated just because they didn’t want to leave a football match. I mean come on…

The other 30% are not out completely either, they’ll have sensitivity and pain for the rest of their lives and always will be more susceptible to frostbite in the future. The weather is not a joke, whether its the heat or the cold. You can pretend you don’t believe in it and that it is not becoming more extreme but you will still be suffering if you don’t take precautions and not take chances by avoiding exposure whenever possible.

– Suramya

March 8, 2024

alphaXiv – Forum to discuss any papers posted on ArXiv

Filed under: Interesting Sites,Science Related — Suramya @ 4:40 PM

Research papers can be hard to understand and sometimes you don’t have people or a community around where you can discuss a given paper and collaborate. In Bangalore we have a community called ‘Papers we love’ which did something similar but for a single paper every week as an in-person event.

There is a site called alphaxiv.org/ that I found out about recently. It is an interactive forum for anyone to comment line-by-line on arXiv papers. This allows you to collaborate and discuss the paper with others who find it interesting as well. One thing to keep in mind is that ArXiv papers can be pre-published versions which are not fully peer reviewed yet.

To start using the site, you need to create an account with them. If you don’t have an educational institute email address then you will have to verify your phone no as well in the near future. As of now you don’t need to do that. I guess they are doing this to prevent automated spam. Once you have an account and are logged in using the site is quite easy. You can browse for currently trending topics or search for a specific paper. If you are reading a paper on arXiv, you can change “arxiv.org” to “alphaxiv.org” in your URL search bar, and you will automatically be redirected to the forum for that paper.

Check it out when you get a chance as it is quite interesting.

– Suramya

March 7, 2024

Cloudflare announces Firewall for LLMs to protect them

Filed under: Artificial Intelligence,Computer Security,My Thoughts — Suramya @ 10:52 PM

As is always the case when the attackers invent technology / systems to attack a system the defenders will immediately come up with a technology to protect (might not always be great protection at the beginning). Yesterday I posted about Researchers demo the first worm that spreads through LLM prompt injection and today while going through my feeds I saw the news that earlier this week cloudflare announced a Firewall for AI . Initially when I read the headline I thought it was yet another group of people who are claiming to have created a ‘perfect firewall’ using AI. Thankfully that was not the case and in this instance it looks like an interesting application that will probably become as common as the regular firewall.

What this system does is quite simple, it is setup in front of a LLM so that all interactions with the LLM goes through the firewall and every request with an LLM prompt is scanned for patterns and signatures of possible attacks. As per their blog post attacks like Prompt Injection, Model Denial of Service, and Sensitive Information Disclosure can be mitigated by adopting a proxy security solution like Cloudflare Firewall for AI.

Firewall for AI is an advanced Web Application Firewall (WAF) specifically tailored for applications using LLMs. It will comprise a set of tools that can be deployed in front of applications to detect vulnerabilities and provide visibility to model owners. The tool kit will include products that are already part of WAF, such as Rate Limiting and Sensitive Data Detection, and a new protection layer which is currently under development. This new validation analyzes the prompt submitted by the end user to identify attempts to exploit the model to extract data and other abuse attempts. Leveraging the size of Cloudflare network, Firewall for AI runs as close to the user as possible, allowing us to identify attacks early and protect both end user and models from abuses and attacks.

OWASP has published their Top 10 for Large Language Model Applications, which is a fantastic read and a good overview of the security risks targeting LLM’s. As per cloudfare this firewall mitigates some of the risks highlighted in OWASP for LLM’s. I would suggest taking the announcement with a grain of salt till we have independent validation of the claims. That being said it is def a step in the correct direction though.

– Suramya

Source: Hacker News: Cloudflare Announces Firewall for AI

March 6, 2024

Researchers demo the first worm that spreads through LLM prompt injection

Filed under: Artificial Intelligence,Computer Security,Computer Software — Suramya @ 10:17 PM

In the past year we have seen an uptick in the tech industry looking towards embedding LLM (Large Language Models) or AI as they are being pitched to the world in all possible places. Windows 11 now has built in Copilot that is extremely hard to disable. Email systems are using LLM’s to get additional details/information using the data from the email to add context etc. This creates new attack surfaces that attackers can target and we have seen instances where attackers have used prompt injection to gain access to data or systems that were restricted.

Building on top of that researchers have now created (and demo’d) the first worm that spreads through prompt injection. This is breakthrough work similar to how the Morris Worm was in the late 80’s. Basically, researchers created an email which has an adversarial prompt embedded in it. This prompt is then ingested by an LLM (using Retrieval-Augmented Generation which allows it to enhance the reliability of the LLM by fetching data from external sources when the email is processed by the LLM) where it jailbreaks the GenAI service and can steal data from the emails (or do whatever else the attacker wants such as changing email text, removing data etc). In addition the prompt also has the ability to make the email assistant forward the email with the malicious prompt to other email addresses allowing it to spread. The researchers have christened their worm as Morris II giving homage to the first email worm.

Abstract: In the past year, numerous companies have incorporated Generative AI (GenAI) capabilities into new and existing applications, forming interconnected Generative AI (GenAI) ecosystems consisting of semi/fully autonomous agents powered by GenAI services. While ongoing research highlighted risks associated with the GenAI layer of agents (e.g., dialog poisoning, membership inference, prompt leaking, jailbreaking), a critical question emerges: Can attackers develop malware to exploit the GenAI component of an agent and launch cyber-attacks on the entire GenAI ecosystem?

This paper introduces Morris II, the first worm designed to target GenAI ecosystems through the use of adversarial self-replicating prompts. The study demonstrates that attackers can insert such prompts into inputs that, when processed by GenAI models, prompt the model to replicate the input as output (replication), engaging in malicious activities (payload). Additionally, these inputs compel the agent to deliver them (propagate) to new agents by exploiting the connectivity within the GenAI ecosystem. We demonstrate the application of Morris II against GenAI-powered email assistants in two use cases (spamming and exfiltrating personal data), under two settings (black-box and white-box accesses), using two types of input data (text and images). The worm is tested against three different GenAI models (Gemini Pro, ChatGPT 4.0, and LLaVA), and various factors (e.g., propagation rate, replication, malicious activity) influencing the performance of the worm are evaluated.

This is pretty fascinating work and I think that this kind of attack will start becoming more common as the LLM usage goes up. The research paper is available at: ComPromptMized: Unleashing Zero-click Worms that Target GenAI-Powered Applications.

– Suramya

March 5, 2024

Yet another example on why we need controls and audit logs around sensitive data

Filed under: Computer Security,My Thoughts — Suramya @ 11:40 AM

People like the one in the example below are why cyber security and privacy policies insist on having access control rules and oversight on who has access to data and audit logs for why they are accessing that data.

My favourite thing about working in HR is being able to look up anyone's age or salary. Its like having a version of IMDB, but for real people.
My favorite thing about working in HR is being able to look up anyone’s age or salary. Its like having a version of IMDB, but for real people.

If you are building of maintaining a system that has sensitive data, or PII (Personal Identifiable Information) you need to ensure that you not just have access controls around the data but also have a way to audit who is accessing the data and for what reason. If the reason is not work related then action should be taken and their access revoked. Law enforcement has access to various monitoring systems and there have been multiple examples in the past where law enforcement officers looked up their ex’s, stalked people etc. Again, that is something that can be prevented in part by having strictly enforced policies on who can access the data and for what purpose.

In one of my previous companies, everytime you accessed any production or critical systems you had to give a reason and link to either a support or incident ticket. Then a manager and the system owner would review the access log along with the keylogs from the sessions and sign off on it. With that they were personally confirming that all activity was required and was justified. If that wasn’t the case then action would be taken against the person who signed off on the logs along with the person who did the accessing.

We need more of that in all systems.

– Suramya

March 4, 2024

Watch Article 370, the movie if you haven’t already

Filed under: My Thoughts — Suramya @ 1:41 PM

Watched the movie ‘Article 370‘, the movie yesterday and I loved it. Each and every Indian should watch this movie as it walks you through a slightly dramatized version of what it took to get Section 370 revoked and which the revocation was required. A lot of people don’t realize how bad the situation in Kashmir was before 370 was revoked and I am glad that the stories of the atrocities over there are now coming out with movies like Article 370 and Kashmir Files.

The acting in the movie is phenomenal and when they show the Article passing in the Rajya Sabha the entire theater burst out in cheers and clapping. Same happened during a lot of the other scenes as well. Its been a while since I have watched a movie that made you think so much and made you proud of the people who work in the shadows, on the frontlines and in the halls of power.

After the movie, me and Ronak were talking and just wondering how much more India could have been if we had Sardar Patel as our PM instead of Nehru. With him in power Pakistan would still be part of India, Kashmir would be a non-issue. We would not have lost land to China and would be taking back our rightful place as one of the leaders of the world. Instead what we are doing now is working on undoing the generational harm done to the spirit and confidence of India.

Imagine if we actually taught people about how advanced and civilized India was in the past. It wasn’t for nothing that India was known as sone ki chidya (Bird of gold/Land of the riches). Folks are taught about how the burning of the Library of Alexandria was so bad but hardly anyone talks about the burning of the Taxila and Nalanda universities. Centuries of records were burnt and all teachers and students killed so that they couldn’t share their knowledge with others.

Our books hardly talk about the achievements of Ancient India but mostly focus on Mugal and British rule. Even though the Chola dynasty ruled for more than 1,500 years, making them one of the longest-ruling families in human history, if not the longest we hardly learnt about them; instead everyone keeps talking about how long the Romans ruled even though their empire lasted only 844 years which is little more than half the time the Chola FAMILY ruled their empire. All this is because we were taught to be ashamed of being Indians by keeping us ignorant of our own history.

The history books keep teaching Nicolaus Copernicus was the first person to come up with the theory that the Earth, along with the other planets, rotated around the Sun in 1543, completely ignoring the fact that the heliocentrism had first been theorized by Yajnavalkya, an Indian philosopher sometime between 9th-7th Century BCE. There are so many examples of how advanced India used to be and how far we fell because people wanted to take advantage and the best way to rule a population is to convince them that they were nothing before the benevolent rulers came and took over for our own benefit.

The interesting trend is that earlier people who knew the actual ancient Indian history tried teaching it via books, articles and seminars etc but most people found them to be very dry and ignored it. Now, we have movie makers taking the stories of the glory of ancient India along with their trials and achievements and creating an engrossing tale that everyone loves. RRR showcased to the world some of the atrocities that were done by the British. Ponniyin Selvan: I (PS: I) showcased the Chola empire to the world and similarly many other movies are now showing our rich heritage to the world. This is bringing the actual history to the masses (with some masala and a lot of special effects added to make them fun to watch)

Well this is all for now. Looking forward to more such movies and India actually becoming sare jahan se achha (Best in the whole world).

– Suramya

March 2, 2024

EU adopts euro instant payments rules, which India has had for over a decade

Filed under: My Thoughts — Suramya @ 3:42 AM

Most people don’t realize how much advanced the Indian Banking system is as compared to other countries. Last week EU adopted euro instant payments rules that would allow customers to transfer money within 10 seconds to any account both within the same country and to another EU member state. What is interesting is that this is being portrayed as a major breakthrough advance in Banking technology.

In India we have a service called Immediate Payment Service (IMPS), which is enables instant inter-bank electronic funds transfer in India. Funds transferred using IMPS are credited to the receiver within seconds and the service is available 24×7 throughout the year including bank holidays. This service was launched in 2010, i.e. over 13 years ago.

The UPI (Unified Payments Interface) is built on top of IMPS and that can be used to make payments/transfers to select countries outside India. List of countries where UPI can be used is available here.

I just find it amusing that something we have been using for over a decade is being portrayed as a breakthrough innovation outside India. Although, to be fair for good amount of time in 80’s & early 90’s things/systems which were common outside India were launched in India as an innovation. So I guess life is a cycle.

Source: Reuters: EU adopts euro instant payments rules to take on Visa, Mastercard

March 1, 2024

If buying isn’t owning, then piracy isn’t stealing

Filed under: Computer Software,My Thoughts — Suramya @ 12:27 PM

Nowadays it is quite common for folks to move from having physical copies of books, movies, TV series etc to having digital copies of the same. I like having digital versions of things because they are easier to store but that brings a whole lot of problems. Basically having an electronic item specially something that is hosted on a service or needs an external system to approve your right to open that file/game/book then it means that you don’t really own that item. The service can arbitrarily decide to remove it from their system, alter it without telling you or decide to reduce functionality and make you pay more for something that you already paid for.

We have so many examples of this happening, such as services removing movies that you bought from your systems because they figured it was more cost effective to not renew rights to the movie. Amazon Music has removed music that was paid for from their system or have changed it. Books were removed from Kindle by amazon when they felt it needed to be inspite of the users having purchased it.

Earlier this week in example no 400035 that shows that we don’t really own the digital content we ‘bought’, Sony deleted content that they had promised would be there forever with little to no recourse for the users to get their content back when though they had paid for it.

Funimation, a Sony-owned streaming service for anime, recently announced that subscribers’ digital libraries on the platform will be unavailable after April 2. For years, Funimation had been telling subscribers that they could keep streaming these digital copies of purchased movies and shows, but qualifying it: “forever, but there are some restrictions.”

But soon, people who may have discarded or lost their physical media or lack a way to play DVDs and Blu-rays won’t have a way to access the digital copies that they were entitled to through their physical copy purchase.

A little while ago Philippe Tremblay, director of subscriptions at Ubisoft made a comment that gamers need to get used to the idea they don’t own their games anymore and embrace digital downloads. This is absolutely ridiculous and should not be normalized. If I own something I should be able to do whatever I want with it. Unfortunately that is not the case because the content is protected by DRM (Digital Rights Management) which is supposed to be a tool to prevent piracy but instead is a tax or punishment for doing the right thing and buying content legally.

If I pirate a movie or a book I can do whatever I want with it and watch it wherever I want or convert it to another format that is easier for me to consume (mobi->epub for books as an example) But when I legally buy something the DRM on it stops me from doing the same thing as it is a felony for me to remove the DRM so that I can access stuff I paid for in a way that is convenient for me.

Before streaming services and digital stores became popular, at times the only way to get content was to pirate it. To give an example, back in 2007/2008 books by most of the authors I like were not available in India so if I wanted to read a book I would have to buy it from Amazon and have it shipped to India. Amazon used to charge $10 PER book to ship it to India at that time even if you ordered multiple books and paying that for a book that costed $7 made absolutely no sense. Same was the case with movies and tv series. With streaming and digital media taking off, I can now buy a digital book when it is released or watch a new TV series when it is launched legally. Now with this nonsense of deleting stuff that people have bought, we need to start keeping copies of all the stuff we buy offline so that I still have access to what I paid for even when a corporation decides that it is more cost effective to delete/remove access to it.

Source: Techdirt.com: Here We Go Again: Sony Disappears Digital Content That Was Pitched To Customers As ‘Forever’

– Suramya

February 29, 2024

Scientists are using wood of all things to build a satellite and launch it to orbit

Filed under: Astronomy / Space,My Thoughts,Science Related — Suramya @ 9:08 PM

When people talk about strong materials that can be used to make durable satellites wood is nowhere even close to being considered. I mean come on, the image people (including me) have in their mind of wood is that it is something not too strong and for the most part is not durable. However, NASA and the Japan Aerospace Exploration Agency (JAXA) are set to prove that is not the case and are planning to launch the world’s first wooden satellite into space. Yes, you did read that correctly; they are planning to use wood to make a satellite instead of Stainless Steel or titanium which will make the process a lot more sustainable. LignoSat is a coffee mug-size satellite made from magnolia wood and will be launched into orbit sometime during the Summer of 2024. Wood has an advantage over other materials in that it will burn up into ash during reentry into the atmosphere without releasing harmful chemicals into the atmosphere (which is what happens when Steel or Titanium reenters the atmosphere and burns up).

To verify the feasibility of using the material Kyoto University researchers sent samples of magnolia, cherry and birch up to the International Space Station and exposed it to space for 10 months, after which tests were run against the samples and when they confirmed that there was no decomposition or deformation in the samples it gave them confidence to move ahead with the project. LignoSat will be made using the wood from magnolia trees primarily because of the relative ease of working with it, dimensional stability, and overall strength as compared to the other candidates.

Another advantage of a wooden satellite is that it is less reflective than other materials which means that it would reduce the amount of ambient light pollution that astronomers have been complaining about over the past few years. Especially since the Starlink low orbit satellites were launched. Wood is also easier to grow so the overall cost might be lower.

One potential problem with this approach would be that it will require a lot of wood which would mean that forests would be logged offsetting the environmental gains made by using wood. Assuming the test is successful, we can look forward to using wood in more places instead of non eco friendly materials. But I doubt that it would replace the traditional materials completely.

– Suramya

Source: Wood Working Network: Japan set to launch first wood satellite

Older Posts »

Powered by WordPress