Suramya's Blog : Welcome to my crazy life…

May 24, 2024

OpenSSF launches Siren to provide real-time security warning for Open Source Software

Filed under: Computer Security,Interesting Sites,Security Tools,Tech Related — Suramya @ 6:41 PM

Securing OpenSource software (OSS) can be a bit of a challenge at times and a lot of the Infosec feeds that give information on Security issues in software are commercial paid entities. There are software that scan for OSS vulnerabilities but we can always use more threat intelligence networks.

Open Source Security Foundation (OpenSSF) has launched a new threat intelligence sharing group called ‘OpenSSF Siren‘ that aims to provide real-time security warning bulletins and deliver a community-driven knowledge base to fill the gap between the open-source and enterprise communities.

The OpenSSF Siren is a collaborative effort to aggregate and disseminate threat intelligence specific to open source projects. Hosted by the OpenSSF, this platform provides a secure and transparent environment for sharing Tactics, Techniques, and Procedures (TTPs) and Indicators of Compromise (IOCs) associated with recent cyber attacks. Siren is intended to be a post-disclosure means of keeping the community informed of threats and activities after the initial sharing and coordination.

The Key features of the OpenSSF Siren include:

  • Open Source Threat Intelligence: shared with the community about actively exploited public vulnerabilities and threats.
  • Real-Time Updates: List members receive notifications via email about emerging threats which may be relevant to their projects, enabling swift action to mitigate risks.
  • TLP:CLEAR: To facilitate effective unrestricted transparent communication, the list follows the Traffic Light Protocol (TLP), Clear guidelines for the sharing and handling of intelligence.
  • Community-driven: Contributors from diverse backgrounds collaborate to enrich the intelligence database, fostering a culture of shared responsibility and collective defense.

You can sign up for it here: Siren Sign-Up
Source: OpenSSF sings a Siren song to steer developers away from buggy FOSS

– Suramya

May 23, 2024

Windows 11 will feature builtin Spyware in the near future or Recall AI as Microsoft Calls it

Filed under: Artificial Intelligence,Computer Security,Computer Software,My Thoughts,Security Tools,Tech Related — Suramya @ 7:24 PM

Till recently if you wanted to spy on someone and see what they have been doing on the computer, you had to infect their computer by making them visit a dodgy site or get physical access and download a RAT (Remote Access Trojan) & install it on the target’s computer, configure the Antivirus to ignore it and put in a backdoor so that you can access the data remotely. Obviously this was a lot of work so looks like some cyber criminals reached out to Microsoft (MS) and asked for help. MS being a super helpful company, has added a functionality called ‘Windows Recall’ to it’s windows 11 Preview build to solve this. Recall takes a snapshot (literally) of the screen every few seconds and stores it in a searchable database ‘stored locally’. Basically it does exactly what spyware does without having to install anything new on your system. As per the company below is how the Recall works:

Recall uses Copilot+ PC advanced processing capabilities to take images of your active screen every few seconds. The snapshots are encrypted and saved on your PC’s hard drive. You can use Recall to locate the content you have viewed on your PC using search or on a timeline bar that allows you to scroll through your snapshots. Once you find the snapshot that you were looking for in Recall, it will be analysed and offer you options to interact with the content. What actions you can take depend on the content and the chat provider capabilities in Copilot in Windows. For example, you may highlight a block of text and decide to summarise it, translate it, or open it with a text editor like Word or Notepad. If you highlight an image, you will be able to edit it or use your chat provider in Copilot in Windows to find or create a similar image.

Recall will also enable you to open the snapshot in the original application in which it was created, and, as Recall is refined over time, it will open the actual source document, website or email in a screenshot. This functionality will be improved during Recall’s preview phase.

The best part is that according to their own announcement the snapshots will not hide passwords/account numbers etc. However, it does block you from recording DRM’d video you might be watching because protecting that is important not simple things like personal information etc.

Note that Recall does not perform content moderation. It will not hide information such as passwords or financial account numbers. That data may be in snapshots that are stored on your device, especially when sites do not follow standard internet protocols like cloaking password entry.

This is a gold mine for data thieves, abusers, industrial espionage, identity thieves and other cyber criminals. Once they have access to a PC they don’t need to do anything else except copy the data from the Recall DB to their own system and happily browse through the users personal data at their leisure.

I don’t think MS has thought about folks who use public computers such as the ones in an Internet Cafe or Hotels or Libraries. With this feature enabled all someone has to do is wait a few days then come back and copy incredibly private information that they can then sell/use. Privacy and Domestic Abuse experts are raising questions about this as well because sure as night follows day, abusers will use this to track what their victims are doing on a computer and that can go bad very quickly.

Even if the data is supposedly only on the local machine we don’t know when MS is going to force it to be uploaded to their servers using OneDrive or other similar setups. All the coverage I have seen for this functionality 99% of them have raised similar concerns about the security, privacy and quite frankly the need for this kind of surveillance.

Imagine what would a regieme like Taliban, China or other conservative/restrictive governments do with information they get from this system. You are dreaming if you think that they will not force MS to make this information available to them at the risk of losing access to that market if they don’t. Once you have the capability to do this, feature creep will happen for sure and we will end up in a Surveillance state.

The only Windows 11 system at my place is my wife’s laptop and you can be sure that I am going to disable this ‘feature’ as soon as it launches.

Source: Bleepingcomputer: Windows 11 Recall AI feature will record everything you do on your PC

– Suramya

May 12, 2024

A High-Level Technical Overview of Fully Homomorphic Encryption

Filed under: Computer Security,Emerging Tech,Interesting Sites,My Thoughts,Tech Related — Suramya @ 10:13 AM

Homomorphic Encryption is an interesting application of data encryption in that it allows us to encrypt data in a way such that we can perform computations on it without first having to decrypt it. The more formal definition states “Homomorphic encryption is the conversion of data into ciphertext that can be analyzed and worked with as if it were still in its original form. Homomorphic encryption enables complex mathematical operations to be performed on encrypted data without compromising the encryption.”

I have been following the work on Homomorphic Encryption solutions since 2017 onwards, which was when I first became aware of it and have read tons of articles and papers on it. The overview by Jeremy Kun is probably the best one I have seen so far. His post with A High-Level Technical Overview of Fully Homomorphic Encryption goes into enough technical details that you understand it without going so deep that you are lost in the details.

Homomorphic encryption lets you encrypt data in such a way that you can run programs on it without ever decrypting it. This means that the computer running the program has no access to the underlying data while running the program—neither via intermediate computed values, nor even the result. In particular, if a nefarious human had access to the machine’s raw memory, they still could not learn any information about the underlying data (without breaking the cryptography). A user sends the program an encrypted input, and when the program is done, the encrypted result is sent back to the user to decrypt.

Running a program on encrypted data sounds magical. It works by choosing an encryption scheme that is “compatible” with addition and multiplication in the following sense:

Adding ciphertexts gives you an encryption of the sum of the underlying plaintexts.
Multiplying two ciphertexts give you an encryption of the product of the underlying plaintexts.

Given this power, you can encrypt your data bit by bit, express your program as a boolean circuit—an XOR gate is addition and an AND gate is multiplication—and simulate the circuit. Since XOR and AND form a universal basis for boolean logic, you can always decompose a circuit this way.

Check it out if you are curious about Homomorphic Encryption and want to learn more.

– Suramya

April 21, 2024

Crescendo Method enables Jailbreaking of LLMs Using ‘Benign’ Prompts

Filed under: Artificial Intelligence,Computer Security,My Thoughts,Tech Related — Suramya @ 12:11 PM

LLMs are becoming more and more popular across all industries and that creates a new attack surface for attackers to target to misuse for malicious purposes. To prevent this LLM models have multiple layers of defenses (with more being created every day), one of the layers attempts to limit the capability of the LLM to what the developer intended. For example, a LLM running a chat service for software support would be limited to answer questions about software identified by the developer. Attackers attempt to bypass these safeguards with the intent to achieve unauthorized actions or “jailbreak” the LLM. Depending on the LLM, this can be easy or complicated.

Earlier this month Microsoft published a paper showcasing the “Crescendo” LLM jailbreak method called “Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack“. Using this method a successful attack could usually be completed in a chain of fewer than 10 interaction turns.

Large Language Models (LLMs) have risen significantly in popularity and are increasingly being adopted across multiple applications. These LLMs are heavily aligned to resist engaging in illegal or unethical topics as a means to avoid contributing to responsible AI harms. However, a recent line of attacks, known as “jailbreaks”, seek to overcome this alignment. Intuitively, jailbreak attacks aim to narrow the gap between what the model can do and what it is willing to do. In this paper, we introduce a novel jailbreak attack called Crescendo. Unlike existing jailbreak methods, Crescendo is a multi-turn jailbreak that interacts with the model in a seemingly benign manner. It begins with a general prompt or question about the task at hand and then gradually escalates the dialogue by referencing the model’s replies, progressively leading to a successful jailbreak. We evaluate Crescendo on various public systems, including ChatGPT, Gemini Pro, Gemini-Ultra, LlaMA-2 70b Chat, and Anthropic Chat. Our results demonstrate the strong efficacy of Crescendo, with it achieving high attack success rates across all evaluated models and tasks. Furthermore, we introduce Crescendomation, a tool that automates the Crescendo attack, and our evaluation showcases its effectiveness against state-of-the-art models.

Microsoft has also published a Blog post that goes over this attack and potential mitigation steps that can be implemented along with details on new tools developed to counter this attack using their “AI Watchdog” and “AI Spotlight” features. The tools attempt to identify adversarial content in both input and outputs to prevent prompt injection attacks.

SCM Magazine has a good writeup on the attack and the defenses against it.

– Suramya

Source: Slashdot: ‘Crescendo’ Method Can Jailbreak LLMs Using Seemingly Benign Prompts

April 2, 2024

Soon it will be possible to update Apple Devices while still in the box

Filed under: Computer Security,My Thoughts — Suramya @ 11:43 PM

Apple has come up with an interesting new technology that allows stores to install the latest updates to an iPhone without removing it from the box. If the technology works (and it looks like it does) it will remove one of the major hassles of buying a new phone or device which is to install the latest updates and patches on the phone.

This device can wirelessly turn on the iPhone, update its software and then power it back down. We still don’t have a full explanation on how it works but based on at a guess, it leverages the fact that the NFC chip in the phone can work potentially work even when the phone is switched off (it already works with a low battery). Placing the phone in the device would potentially trigger the NFC chip which would then start the phone in a special mode that allows it to connect to the WiFi and download the updates. Post completion the system would shutdown the phone and it would be ready to use.

In theory this sounds like a great enhancement but I fear that unless the system has sufficient controls and checks around it it will open up a whole new attack vector. Previously, there have been attacks where Nation States or Criminal organizations would intercept hardware being delivered to a target open the package, make changes and then reseal and send it on to the target. This is a sure shot way of ensuring that a device is compromised before it reaches the target, however it requires a lot of resources and manual effort to implement and there is a risk of exposure since multiple folks are involved. With this new update option an attacker just has to have physical access to the device and can be done by simply taking the packaged device and putting it in the updater for a little while.

This assumes that the security checks and authentication built around the process can be bypassed. That being said, once the tech is live there are going to be a lot of very smart people trying to bypass the checks to be able to update the phone. Keep in mind that there is nothing stopping anyone from updating the phone using this method even after someone is actively using it.

Source: Mastodon.social: arstechnica

March 7, 2024

Cloudflare announces Firewall for LLMs to protect them

Filed under: Artificial Intelligence,Computer Security,My Thoughts — Suramya @ 10:52 PM

As is always the case when the attackers invent technology / systems to attack a system the defenders will immediately come up with a technology to protect (might not always be great protection at the beginning). Yesterday I posted about Researchers demo the first worm that spreads through LLM prompt injection and today while going through my feeds I saw the news that earlier this week cloudflare announced a Firewall for AI . Initially when I read the headline I thought it was yet another group of people who are claiming to have created a ‘perfect firewall’ using AI. Thankfully that was not the case and in this instance it looks like an interesting application that will probably become as common as the regular firewall.

What this system does is quite simple, it is setup in front of a LLM so that all interactions with the LLM goes through the firewall and every request with an LLM prompt is scanned for patterns and signatures of possible attacks. As per their blog post attacks like Prompt Injection, Model Denial of Service, and Sensitive Information Disclosure can be mitigated by adopting a proxy security solution like Cloudflare Firewall for AI.

Firewall for AI is an advanced Web Application Firewall (WAF) specifically tailored for applications using LLMs. It will comprise a set of tools that can be deployed in front of applications to detect vulnerabilities and provide visibility to model owners. The tool kit will include products that are already part of WAF, such as Rate Limiting and Sensitive Data Detection, and a new protection layer which is currently under development. This new validation analyzes the prompt submitted by the end user to identify attempts to exploit the model to extract data and other abuse attempts. Leveraging the size of Cloudflare network, Firewall for AI runs as close to the user as possible, allowing us to identify attacks early and protect both end user and models from abuses and attacks.

OWASP has published their Top 10 for Large Language Model Applications, which is a fantastic read and a good overview of the security risks targeting LLM’s. As per cloudfare this firewall mitigates some of the risks highlighted in OWASP for LLM’s. I would suggest taking the announcement with a grain of salt till we have independent validation of the claims. That being said it is def a step in the correct direction though.

– Suramya

Source: Hacker News: Cloudflare Announces Firewall for AI

March 6, 2024

Researchers demo the first worm that spreads through LLM prompt injection

Filed under: Artificial Intelligence,Computer Security,Computer Software — Suramya @ 10:17 PM

In the past year we have seen an uptick in the tech industry looking towards embedding LLM (Large Language Models) or AI as they are being pitched to the world in all possible places. Windows 11 now has built in Copilot that is extremely hard to disable. Email systems are using LLM’s to get additional details/information using the data from the email to add context etc. This creates new attack surfaces that attackers can target and we have seen instances where attackers have used prompt injection to gain access to data or systems that were restricted.

Building on top of that researchers have now created (and demo’d) the first worm that spreads through prompt injection. This is breakthrough work similar to how the Morris Worm was in the late 80’s. Basically, researchers created an email which has an adversarial prompt embedded in it. This prompt is then ingested by an LLM (using Retrieval-Augmented Generation which allows it to enhance the reliability of the LLM by fetching data from external sources when the email is processed by the LLM) where it jailbreaks the GenAI service and can steal data from the emails (or do whatever else the attacker wants such as changing email text, removing data etc). In addition the prompt also has the ability to make the email assistant forward the email with the malicious prompt to other email addresses allowing it to spread. The researchers have christened their worm as Morris II giving homage to the first email worm.

Abstract: In the past year, numerous companies have incorporated Generative AI (GenAI) capabilities into new and existing applications, forming interconnected Generative AI (GenAI) ecosystems consisting of semi/fully autonomous agents powered by GenAI services. While ongoing research highlighted risks associated with the GenAI layer of agents (e.g., dialog poisoning, membership inference, prompt leaking, jailbreaking), a critical question emerges: Can attackers develop malware to exploit the GenAI component of an agent and launch cyber-attacks on the entire GenAI ecosystem?

This paper introduces Morris II, the first worm designed to target GenAI ecosystems through the use of adversarial self-replicating prompts. The study demonstrates that attackers can insert such prompts into inputs that, when processed by GenAI models, prompt the model to replicate the input as output (replication), engaging in malicious activities (payload). Additionally, these inputs compel the agent to deliver them (propagate) to new agents by exploiting the connectivity within the GenAI ecosystem. We demonstrate the application of Morris II against GenAI-powered email assistants in two use cases (spamming and exfiltrating personal data), under two settings (black-box and white-box accesses), using two types of input data (text and images). The worm is tested against three different GenAI models (Gemini Pro, ChatGPT 4.0, and LLaVA), and various factors (e.g., propagation rate, replication, malicious activity) influencing the performance of the worm are evaluated.

This is pretty fascinating work and I think that this kind of attack will start becoming more common as the LLM usage goes up. The research paper is available at: ComPromptMized: Unleashing Zero-click Worms that Target GenAI-Powered Applications.

– Suramya

March 5, 2024

Yet another example on why we need controls and audit logs around sensitive data

Filed under: Computer Security,My Thoughts — Suramya @ 11:40 AM

People like the one in the example below are why cyber security and privacy policies insist on having access control rules and oversight on who has access to data and audit logs for why they are accessing that data.

My favourite thing about working in HR is being able to look up anyone's age or salary. Its like having a version of IMDB, but for real people.
My favorite thing about working in HR is being able to look up anyone’s age or salary. Its like having a version of IMDB, but for real people.

If you are building of maintaining a system that has sensitive data, or PII (Personal Identifiable Information) you need to ensure that you not just have access controls around the data but also have a way to audit who is accessing the data and for what reason. If the reason is not work related then action should be taken and their access revoked. Law enforcement has access to various monitoring systems and there have been multiple examples in the past where law enforcement officers looked up their ex’s, stalked people etc. Again, that is something that can be prevented in part by having strictly enforced policies on who can access the data and for what purpose.

In one of my previous companies, everytime you accessed any production or critical systems you had to give a reason and link to either a support or incident ticket. Then a manager and the system owner would review the access log along with the keylogs from the sessions and sign off on it. With that they were personally confirming that all activity was required and was justified. If that wasn’t the case then action would be taken against the person who signed off on the logs along with the person who did the accessing.

We need more of that in all systems.

– Suramya

September 4, 2023

Mashing Enter can allow you bypass full disk encryption in certain scenarios

Filed under: Computer Security,Linux/Unix Related,My Thoughts — Suramya @ 12:30 PM

When folks think about hacking and people bypassing secure systems they have this mental image of folks writing complex code or physically reading the data byte by byte but that is not always true. Sometimes, it is as simple as just keeping the enter key pressed while the system is booting up. Yes, you read that right. A few days ago a vulnerability was found in a TPM-protected system that is configured to implement unattended unlocking for LUKS full disk encryption using RedHat’s Clevis and dracut software along with systemd.

Generally, a Linux computer using TPM-protected unattended disk encryption will still allow a user to view the output of the boot process and optionally manually enter a decryption password with the keyboard. This allows for situations where the computer fails to boot and needs someone to troubleshoot the startup process. While the unattended TPM unlocking is taking place, the user is still presented with the password prompt and an opportunity to enter input.

There’s a limited window of time before the TPM will unlock the disk and the boot process will proceed automatically to the login prompt, so how can we effectively fuzz this input opportunity? What if we could type faster than a human being? Using an Atmel ATMEGA32U4 microcontroller (such as you’d find in an Arduino Leonardo development board) we can emulate a keyboard that sends virtual keypresses at essentially the maximum rate that the computer will accept. The following short Arduino program sets up a Leonardo as a keyboard emulator:

#include "Keyboard.h"
void setup() {
delay(1000);
Keyboard.begin();
}
void loop() {
Keyboard.press(KEY_RETURN);
delay(10);
Keyboard.releaseAll();
delay(10);
}

One second after being plugged in this program begins to simulate pressing the Enter key on a virtual keyboard every 10 milliseconds. This is about 10x faster than the usual keyboard repeat rate you’d get simply holding down a key, and Linux seems to recognise around 70 characters per second using this method, or one keypress approximately every 15 milliseconds.

Sending keypresses this fast quickly hits the maximum number of password entry retries, while keeping the system from unlocking the disk automatically due to password guess rate limiting, and systemd eventually gives up trying to unlock the disk. It takes a minute or two but the recovery action in this failure scenario is to give us a root shell in the early boot environment

The simplest way to address the most immediate problem: Add rd.shell=0 and rd.emergency=reboot to the kernel command line. This ensures that if anything fails during the early boot process the computer will reboot immediately rather than dropping into a root shell.

However, this goes to show us that the old statement about security is still absolutely valid: “Physical access is root access. You can’t spend thousands on protecting the cyber threat landscape and ignore physical security such that people can just walk up to your computer and stick things inside. That being said, having a physical security program doesn’t necessarily protect your from an insider threat so that is also something to keep in mind.

Source: Pulsesecurity: Mashing Enter to bypass full disk encryption with TPM, Clevis, dracut and systemd

– Suramya

August 29, 2023

Excel holding up the Global Financial System, now with Python support

Filed under: Computer Security,Computer Software,My Thoughts,Tech Related — Suramya @ 1:12 PM

It is both impressive and scary how much of the world’s financial systems is being run using Microsoft Excel. Folks have created formulars/macros/scripts/functions etc in Excel that allows them to generate data that is used to take major financial decisions with real world impact.

In one of my previous companies we actually had a full discussion on how to get an inventory of all the Excel code in use at the company and how to archive it so that we have backups and version control on them. Unfortunately, I left before much headway was made but I did learn enough about excel use to scare me. (Especially since I am not the biggest fan of Microsoft software 😉 )

Now you might ask why so many people are using excel when there are better tools available in the market and these companies have inhouse teams to create custom software for the analyst and I asked the exact same questions when I started. I think it is probably because the tool makes it easy for folks to come up with formulas and scripts that get their work done instead of having to wait for an external team to make the changes etc that they need.

Now, a few days ago Microsoft made a surprise announcement that going forward they are going to support running Python inside an Excel file. Yikes!! In order to use this functionality you will need to be part of the Microsoft 365 Insider program and then you can type Python code directly into cells using the new =PY() function, which then gets executed in the cloud. From what I have read, this will be enabled by default and needs to be disabled via a registry key.

Since its inception, Microsoft Excel has changed how people organize, analyze, and visualize their data, providing a basis for decision-making for the millions of people who use it each day. Today we’re announcing a significant evolution in the analytical capabilities available within Excel by releasing a Public Preview of Python in Excel. Python in Excel makes it possible to natively combine Python and Excel analytics within the same workbook – with no setup required. With Python in Excel, you can type Python directly into a cell, the Python calculations run in the Microsoft Cloud, and your results are returned to the worksheet, including plots and visualizations.

We already have issues with Excel Macros being used as vectors for malware & viruses, this just opens a whole new front in that war. Now, admins will have to worry about attackers using Python in Excel to infiltrate the organization or to send data outside the org. I can see how it is useful for people working with datasets and MS is adding this functionality to keep up with other tools such as Tableau etc which are more powerful but still I feel that this is a bad move.

Another problem that folks are going to face is that now your Excel sheets have Python programs inside them, how are we supposed to version the code, how is code review done? Basically this code should be going through the standard SDLC (Software Development Life Cycle) process but wouldn’t. We also need to ensure that all changes are reviewed and monitored to protect against insider attacks but the way the system is setup this is going to be extremely difficult (We have already seen that with Macros and Formulas etc).

Lets see how folks address this risk profile.

– Suramya

Older Posts »

Powered by WordPress