Suramya's Blog

February 28, 2023

You shouldn’t spend money teaching folks how to survive in your toxic work culture, instead fix the Toxic culture

Filed under: My Thoughts — Suramya @ 5:39 PM

The following post showed up on my LinkedIn feed earlier today, it wants companies to put in time, effort & money to train employees on office politics so that they learn to survive and perform in a toxic work environment…

Personally, I would rather that the company spent all the effort to remove the toxicity from their work culture so that people don’t have to deal with it. This is like saying that we know that the water you drink is toxic, but we are not going to fix the root cause but will give you free chemo-therapy to mitigate your cancer.

If you run a company, and are hearing reports about toxic culture in the company then you need to address that immediately and not by giving people training on how to survive it, but rather by removing the toxic elements ruthlessly. Short term you might have to take a hit and might miss milestones or deliverables but long term your other employees who were just doing the bare minimum to get by would start engaging more, new talented folks would be happy to join your team/company because they wouldn’t get emails from friends telling them that it is a bad idea to join company X because of the toxic work environment.

What do you think?

– Suramya

Comments (0)

February 27, 2023

It is now possible to put undetectable Backdoors in Machine Learning Models

Filed under: Computer Software,Emerging Tech,My Thoughts,Tech Related — Suramya @ 10:18 PM

Machine Learning (ML) has become the new go to buzzword in the Tech world in the last few years and everyone seems to be focusing on how they can include ML/AI in their products, regardless of whether it makes sense to include or not. One of the bigest dangers of this trend is that we are moving towards a future where an algorithm would have the power to make decisions that have real world impacts but due to the complexity it would be impossible to audit/check the system for errors/bugs, non-obvious biases or signs of manipulation etc. For example, we have had cases where the wrong person was identified as a fugitive and arrested because an AI/ML system claimed that they matched the suspect. Others have used ML to try to predict crimes with really low accuracy but people take it as gospel because the computer said so…

With ML models becoming more and more popular there is also more research on how these models are vulnerable to attacks. In December 2022 researchers (Shafi Goldwasser, Michael P. Kim, Vinod Vaikuntanathan and Or Zamir) from UC Berkely, MIT and Princeton published a paper titled “Planting Undetectable Backdoors in Machine Learning Models” in the IEEE 63rd Annual Symposium on Foundations of Computer Science (FOCS) where they discuss how it would be possible to train a model in a way that it allowed an attacker to manipulate the results without being detected by any computationally-bounded observer.

Abstract: Given the computational cost and technical expertise required to train machine learning models, users may delegate the task of learning to a service provider. Delegation of learning has clear benefits, and at the same time raises serious concerns of trust. This work studies possible abuses of power by untrusted learners.We show how a malicious learner can plant an undetectable backdoor into a classifier. On the surface, such a backdoored classifier behaves normally, but in reality, the learner maintains a mechanism for changing the classification of any input, with only a slight perturbation. Importantly, without the appropriate “backdoor key,” the mechanism is hidden and cannot be detected by any computationally-bounded observer. We demonstrate two frameworks for planting undetectable backdoors, with incomparable guarantees.

First, we show how to plant a backdoor in any model, using digital signature schemes. The construction guarantees that given query access to the original model and the backdoored version, it is computationally infeasible to find even a single input where they differ. This property implies that the backdoored model has generalization error comparable with the original model. Moreover, even if the distinguisher can request backdoored inputs of its choice, they cannot backdoor a new inputa property we call non-replicability.

Second, we demonstrate how to insert undetectable backdoors in models trained using the Random Fourier Features (RFF) learning paradigm (Rahimi, Recht; NeurIPS 2007). In this construction, undetectability holds against powerful white-box distinguishers: given a complete description of the network and the training data, no efficient distinguisher can guess whether the model is “clean” or contains a backdoor. The backdooring algorithm executes the RFF algorithm faithfully on the given training data, tampering only with its random coins. We prove this strong guarantee under the hardness of the Continuous Learning With Errors problem (Bruna, Regev, Song, Tang; STOC 2021). We show a similar white-box undetectable backdoor for random ReLU networks based on the hardness of Sparse PCA (Berthet, Rigollet; COLT 2013).

Our construction of undetectable backdoors also sheds light on the related issue of robustness to adversarial examples. In particular, by constructing undetectable backdoor for an “adversarially-robust” learning algorithm, we can produce a classifier that is indistinguishable from a robust classifier, but where every input has an adversarial example! In this way, the existence of undetectable backdoors represent a significant theoretical roadblock to certifying adversarial robustness.

Basically they are talking about having a ML model that works correctly most of the time but allows the attacker to manipulate the results if they want. One example use case would be something like the following: A bank uses a ML model to decide if they should give out a loan to an applicant and because they don’t want to be accused of being discriminatory they give it to folks to test and validate and the model comes back clean. However, unknown to the testers the model has been backdoored using the techniques in the paper above so the bank can modify the output in certain cases to deny the loan application even though they would have qualified. Since the model was tested and ‘proven’ to be without bias they are in the clear as the backdoor is pretty much undetectable.

Another possible attack vector is that a nation state funds a company that trains ML models and has them insert a covert backdoor in the model, then they have the ability to manipulate the output from the model without any trace. Imagine if this model was used to predict if the nation state was going to attack or not. Even if they were going to attack they could use the backdoor to fool the target into thinking that all was well.

Having a black box making such decisions is what I would call a “Bad Idea”. At least with the old (non-ML) algorithms we could audit the code to see if there were issues with ML that is not really possible and thus this becomes a bigger threat. There are a million other such scenarios that could be played and if we put blind trust in an AI/ML system then we are setting ourselves up for a disaster that we would never see coming.

Source: Schneier on Security: Putting Undetectable Backdoors in Machine Learning Models

– Suramya

Comments (0)

February 21, 2023

Fixing problems with nvidia-driver on Debian Unstable after latest upgrade

Filed under: Computer Software,Linux/Unix Related,Tech Related — Suramya @ 10:54 PM

Earlier today I ran my periodic update of my main desktop that is running Debian Unstable. The upgrade finished successfully and since a new kernel was released with this update I restarted the system to ensure that all files/services etc are running the same version. After the reboot the GUI refused to start and I thought the problem could be because of a NVIDIA kernel module issue so I tried to reboot to an older kernel but that didn’t work either. Then I tried running apt-get dist-upgrade again which gave me the following error:

root@StarKnight:~# apt-get dist-upgrade 
Reading package lists...
Building dependency tree...
Reading state information...
You might want to run 'apt --fix-broken install' to correct these.
The following packages have unmet dependencies:
 nvidia-driver : Depends: nvidia-kernel-dkms (= 525.85.12-1) but 515.86.01-1 is installed or
                          nvidia-kernel-525.85.12 or
                          nvidia-open-kernel-525.85.12 or
                          nvidia-open-kernel-525.85.12
E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or specify a solution).

So I ran the apt –fix-broken install command as recommended and that failed as well with another set of errors:

root@StarKnight:/var/log# apt --fix-broken install
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Correcting dependencies... Done
0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
dpkg: dependency problems prevent configuration of nvidia-driver:
 nvidia-driver depends on nvidia-kernel-dkms (= 525.85.12-1) | nvidia-kernel-525.85.12 | nvidia-open-kernel-525.85.12 | nvidia-open-kernel-525.85.12; however:
  Version of nvidia-kernel-dkms on system is 515.86.01-1.
  Package nvidia-kernel-525.85.12 is not installed.
  Package nvidia-open-kernel-525.85.12 is not installed.
  Package nvidia-open-kernel-525.85.12 is not installed.

dpkg: error processing package nvidia-driver (--configure):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 nvidia-driver
E: Sub-process /usr/bin/dpkg returned an error code (1)

Looking at the logs, I didn’t see any major errors but I did see the following message:

2023-02-21T19:48:27.668268+05:30 StarKnight kernel: [    3.379006] NVRM: loading NVIDIA UNIX x86_64 Kernel Module  515.86.01  Wed Oct 26 09:12:38 UTC 2022
2023-02-21T19:48:27.668286+05:30 StarKnight kernel: [    4.821755] NVRM: API mismatch: the client has the version 525.85.12, but
2023-02-21T19:48:27.668287+05:30 StarKnight kernel: [    4.821755] NVRM: this kernel module has the version 515.86.01.  Please
2023-02-21T19:48:27.668287+05:30 StarKnight kernel: [    4.821755] NVRM: make sure that this kernel module and all NVIDIA driver
2023-02-21T19:48:27.668288+05:30 StarKnight kernel: [    4.821755] NVRM: components have the same version.

Searching on the web didn’t give me a solution but since I am running the Debian Unstable branch it is expected that once in a while things might break and sometimes they break quite spectacularly… So I started experimenting and tried removing and reinstalling the nvidia-driver but that was failing as well because the package was expecting nvidia-kernel-dkms version 525.85.12 but we had 515.86.01-1 installed.

root@StarKnight:~# apt-get install nvidia-driver
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  nvidia-driver
0 upgraded, 1 newly installed, 0 to remove and 14 not upgraded.
Need to get 0 B/494 kB of archives.
After this operation, 1,398 kB of additional disk space will be used.
Selecting previously unselected package nvidia-driver.
(Reading database ... 439287 files and directories currently installed.)
Preparing to unpack .../nvidia-driver_525.85.12-1_amd64.deb ...
Unpacking nvidia-driver (525.85.12-1) ...
dpkg: dependency problems prevent configuration of nvidia-driver:
 nvidia-driver depends on nvidia-kernel-dkms (= 525.85.12-1) | nvidia-kernel-525.85.12 | nvidia-open-kernel-525.85.12 | nvidia-open-kernel-525.85.12; however:
  Version of nvidia-kernel-dkms on system is 515.86.01-1.
  Package nvidia-kernel-525.85.12 is not installed.
  Package nvidia-open-kernel-525.85.12 is not installed.
  Package nvidia-open-kernel-525.85.12 is not installed.

Now I had a couple of options, first was to wait for a couple of days (if I am lucky) for someone to upload the correct versions of the packages to the channel. The second option was to remove the package and installed the Open Source version of the Nvidia driver. I didn’t want to do that because that package is a memory hog and doesn’t work that well either. The last option was to try to manually install the older version (525.85.12) of the nvidia-kernel-dkms package and this is what I decided to go with, a search on the Debian Packages site gave me the .deb file for nvidia-kernel-dkms and firmware-nvidia-gsp (a dependency for the dkms package). I downloaded both the packages and installed them using the following command:

root@StarKnight:/home/suramya/Media/Downloads# dpkg -i firmware-nvidia-gsp_525.85.12-1_amd64.deb 
root@StarKnight:/home/suramya/Media/Downloads# dpkg -i nvidia-kernel-dkms_525.85.12-1_amd64.deb

Once the packages were successfully downgraded I rebooted the system and the GUI came up without issues post the reboot.

Moral of the story is that you need to be prepared to have to troubleshoot your setup if you are running Debian Unstable or Debian Testing on your system. If you don’t want to do that then you should stick to Debian Stable which is rock solid or one of the other distributions such as Ubuntu or Linux Mint etc.

– Suramya

Comments (2)

February 20, 2023

February 10, 2023

Massive 5.9 million tonnes of Lithium deposits found in Jammu Kashmir

Filed under: Tech Related — Suramya @ 11:42 PM

Lithium is a critical metal for the production of batteries and the worldwide demand for it is only increasing with the push for more Electric Vehicles. Till date India has been import dependent for Lithium along with other critical metals that created a risk considering India’s ongoing push towards EV and slow movement away from traditional fossil fuels which made us reliant on China and other countries to meet our needs. On Thursday India’s mining ministry announced that they have found a massive 5.9 million tonnes of Lithium deposits in Jammu and Kashmir. To give you an idea of the scale this puts India in the top 6 “mine reserves” of the metal overtaking China which has ~4.6 million tonnes of the deposit for the metal.

Bolivia – 21 million tonnes
Argentina – 17 million tonnes
Chile – 9 million tonnes
United States – 6.8 million tonnes
Australia – 6.3 million tonnes
China – 4.5 million tonnes

Even though other countries have larger deposits of the metal, China controls 80% of the world’s raw material refining, 77% of the world’s cell capacity and 60% of the world’s component manufacturing. With a larger deposit being found in India and the government undertaking a concentrated effort to mine the metal safely and quickly, India has the opportunity to disrupt China’s control over the metal trade. The possibilities are endless as the demand for Lithium is only going to go up (till a safer and more sustainable option is found).

Till date, this deposit was not found because of the internal issues going on in Kashmir, after the article 370 was revoked more and more industries are looking at Kashmir and who know what other hidden treasures will be found in the state. Already we see that tourism to Kashmir has shot up massively with over 1.62 crore tourists visiting Jammu and Kashmir in 2022, which is the highest number since independence in 1947.

Looking forward to more such news and am happy to see our nation taking another step towards becoming atmanirbhar (self-reliant).

Source: Wion News: India discovers huge deposits of Lithium critical for electric mobility

– Suramya

Comments (0)

February 2, 2023

KeePass vulnerability that allows anyone with write access to the config file export all stored passwords

Filed under: Computer Security,Tech Related — Suramya @ 5:17 PM

Keepass is an Open source password manager that I have been using for a few years. For me the main attraction was that the password database is stored locally instead of using a cloud based solution. Although, you can configure it to save the password file on a cloud store if you want.

Recently a security ‘vulnerability’ has been found in KeePass that allows someone with access to the system where keypass is installed to export the entire password database to clear-text without any user intervention. Basically if an attacker has write access to the XML configuration file they can add an export trigger to get cleartext passwords as and when they are used. There is an ongoing dispute where it is being debated that this is technically not a vulnerability in KeePass as anyone with write access to the configuration file already has enough access to perform more powerful attacks. The issue has been assigned a CVE-2023-24055.

“In both cases, having write access to the KeePass configuration file typically implies that an attacker can actually perform much more powerful attacks than modifying the configuration file (and these attacks in the end can also affect KeePass, independent of a configuration file protection),” the KeePass developers explain. “These attacks can only be prevented by keeping the environment secure (by using an anti-virus software, a firewall, not opening unknown e-mail attachments, etc.). KeePass cannot magically run securely in an insecure environment.”

Personally, I think that while it does follow that physical access equals root access, there should be some notification / mechanism that gets triggered when the XML configuration file is modified outside the KeePass interface. Something like an alert that tells you that “The XML configuration file was directly modified. Please ensure that no extra triggers are installed” (or words to that effect). This way even if an attacker gets access to the system, they can’t easily get access to stored passwords would hopefully be protected with another password.

Source: Naked Security: Password-stealing “vulnerability” reported in KeePass – bug or feature?

Comments (0)

February 1, 2023

Product Idea: Use ChatGPT to write movie/TV Tech Jargon dialogs

Filed under: Emerging Tech,My Thoughts,Tech Related — Suramya @ 3:00 AM

ChatGPT has taken the world by storm and everyday we hear more and more stories about how good it is and how it is changing the world. Personally I think the best description of the software is how Christine Lemmer-Webber put it:

ChatGPT is Mansplaining As A Service. A service that instantly generates vaguely plausible sounding yet totally fabricated and baseless lectures in an instant with unflagging confidence in its own correctness on any topic, without concern, regard or even awareness of the level of expertise of its audience.

People are using it for anything and everything including writing exams, writing articles, job posts, etc etc. While the result we get looks quite plausible there have been multiple cases where the output is subtly wrong and using the result as is can cause problems. However, there is one place that I can think of that ChatGPT would be extremely useful without any major drawbacks. If you have seen any TV show recently there are a lot of times where the characters will use technobabble to explain something and a lot of times the explanations are so ridiculous that they end up distracting you from the actual plot (such as it is).

My idea is that we encourage the writers to start using ChatGPT to write the jargon, even if it gets it wrong it wouldn’t be worse than what we have already so it is a win win. Plus since we all know that media tech works differently we don’t have to worry about the results being 100% accurate or correct.

One example is as below:

Forensic Scientist 1: Hey, have you seen the results from the trace DNA sample we collected from the crime scene?
Forensic Scientist 2: Not yet, I was just about to head over to the lab and check. Why, did you already receive them?
Forensic Scientist 1: Yeah, I just got an email from the lab. They’re pretty clear cut.
Forensic Scientist 2: Great, let’s take a look. What do the results show?
Forensic Scientist 1: Well, the sample is a match with the reference sample from our suspect. The lab confirmed a positive identification.
Forensic Scientist 2: That’s fantastic. Did they provide any details on the genetic markers?
Forensic Scientist 1: Yes, they identified several short tandem repeat markers that match between the sample and the reference. They also ran a mitochondrial DNA analysis and got a match there too.
Forensic Scientist 2: This is really strong evidence. But we need to be sure to double check all the details, especially since this is a high-profile case.
Forensic Scientist 1: Absolutely. I was thinking we should do a full review of the lab’s methodology and verify that the samples were handled and stored properly.
Forensic Scientist 2: That’s a great idea. Let’s also review the lab’s accreditation and see if they meet all the necessary standards.
Forensic Scientist 1: Right, we want to make sure our evidence is solid and can stand up in court.
Forensic Scientist 2: Definitely. Well, let’s head over to the lab and start our review.
Forensic Scientist 1: Sounds good. Let’s do it.

Another example:

Cybersecurity Professional 1: Hey, have you seen the alert about the recent cyberattack?
Cybersecurity Professional 2: Yes, I just received the notification. What do we know so far?
Cybersecurity Professional 1: The attack appears to be a phishing campaign. The attackers are using fake emails to steal login credentials from unsuspecting victims.
Cybersecurity Professional 2: That’s a common tactic. Did they manage to compromise any systems?
Cybersecurity Professional 1: Yes, we have reports of several systems being breached. Our incident response team is currently investigating the extent of the damage.
Cybersecurity Professional 2: Okay, let’s see if we can help with the investigation. What’s the first step?
Cybersecurity Professional 1: First, we need to isolate the infected systems and contain the spread of the attack. Then we can start looking into how the attackers gained access and what they did once they were in.
Cybersecurity Professional 2: Right. I’ll start working on the containment and isolation. Have you seen any indications of what the attackers are after?
Cybersecurity Professional 1: It’s not clear yet, but it looks like they are targeting sensitive information, such as financial data and personal information. We need to be proactive and make sure all our systems and data are secure.
Cybersecurity Professional 2: Agreed. We need to inform the relevant stakeholders about the attack and what measures we’re taking to prevent further damage.
Cybersecurity Professional 1: Absolutely. We also need to start preparing for the worst-case scenario, in case the attackers managed to exfiltrate any data.
Cybersecurity Professional 2: That’s a good point. We need to be prepared for the aftermath and make sure we have a plan in place to respond effectively.
Cybersecurity Professional 1: Right. Let’s get to work and make sure we minimize the impact of this attack.

What do you think?

– Suramya

Comments (0)

January 25, 2023

Fewer than 50% of smart appliances are connected to the internet by users and that percentage is too high

Filed under: My Thoughts,Tech Related — Suramya @ 3:04 AM

Smart Appliances or Smart Devices are a pain. Yes, sometimes they make your life easier by allowing you to control your lighting from the bed or switch on your devices remotely but they cause major security problems. There is a term for these ‘smart’ devices: “Internet of Shit”. This refers to all the devices that are connected to the internet that have no business being connected to it. One example is an IoT enabled toilet, Smart Toaster, Smart Mirror etc etc.

There are a lot of potential issues in IoT devices including but not limited to:

If the manufacturer decides to shut down the service the device becomes an expensive paperweight
- Older model Lexus cars lost all remote capabilities because the 3G towers it depended on shut down
- Older Sonos speakers lost functionality when the company decided to decommission them
- Tesla car bricked itself because it lost connectivity after owner parked it in a basement.
Need a 24×7 active connection
- People got locked out of their house because a server upgrade was happening
- During an AWS outage folks were unable to change the temperature using their NEST systems
Security Issues: IoT devices are the amongst the most insecure devices out there
Lack of Interoperability

and much more. When I bought my AC a couple of years ago I had to argue for a good 15 mins to get a non-smart/internet connected AC. The salesperson couldn’t digest why I wasn’t interested in having a smart AC. A few years later I got a washing machine and there was no option to get a dumb machine, but thankfully it has the option to create a local hotspot that I can connect to from my phone to control the machine. Yeah, it would be more convenient to do it from a distance but guess what I am already next to the machine when I am putting clothes for a wash and it is not that hard to connect to the local AP, start the cycle and then switch back.

A lot of appliance manufacturers bet big on the Smart revolution and according to recent studies are quite surprised to find out that fewer than 50% of their consumers actually connect their smart devices to the internet. The main reason behind it, in my opinion is that there is no major value add for a toaster or a fridge or a toilet to be connected to the internet.

This whole conversation reminded me of the following quote:

Tech enthusiasts: My entire house is smart. Tech workers: The only piece of technology in my house is a printer and I keep a gun next to it so I can shoot it if it makes a noise I don’t recognize.

I am not as bad but nothing at my house is connected to the internet if it doesn’t need to be and then also it is on a separate network isolated from my home machine.

Source: Slashdot: Appliance Makers Sad That 50% of Customers Won’t Connect Smart Appliances

– Suramya

Comments (0)

January 22, 2023

Working remotely doesn’t leads to lonely people no matter what folks on Twitter say

Filed under: My Thoughts — Suramya @ 1:11 AM

“100% remote work combined with a no meeting culture is going to create a large group of lonely, isolated workers” – Alex Cohen posted this on Twitter a little while ago. Apparently the only way to ward off loneliness is to work in an office and attend meetings since having friends and spending time with them is obviously so last millennium.

I am someone who tends to be a little on the workoholic side if I am not careful. I have spent all weekend in office multiple times and one of my managers frequently used to message me asking me why I was still online at 3am and logged into office systems. That being said, there is more to life than work. In one of our Townhalls in a previous organization, someone asked the CEO a question “What are you doing to ensure employees have a proper work-life balance” and he responded (and I am paraphrasing since this was a while ago) “It is not my or the company’s responsibility to ensure you have a proper work-life balance. Only you know what balance would work for you and you need to tell us when you have enough work and need to stop for a break. If you allow/want we can give you work 24 hours a day 7 days a week because we have enough things to do. You need to be the one to decide what work-life balance means for you and then let us know. Then we will ensure that you get to follow that.” This stuck with me since I heard it and I quote it often with my reportees when we talk about Work-life balance.

This is not to say that you shouldn’t make friends at work or people objecting to the statement don’t enjoy working with others on a project. However, if you make your work the defining factor in your life, you are going to burn out. (Telling from experience) It is good to step away from work to do something else as that will rejuvenate you and might even give you a new perspective. For me I read books, go hiking and travel to get a break from work. You will not believe how many of my project ideas and fixes for issues I was facing were conceptualized while I was slogging away on a hilltop or driving to a new location for a trip.

Someone once told me that it is ok to love your job but don’t expect that love to be always returned. One recent example is the Microsoft Engineer who spent 21 years with MS as a great performer but was still fired earlier this week with little notice. There are multiple such stories and I could fill books with various similar examples.

I have been working remotely since 2020 and it does get tiring as I can’t meet folks face to face for discussions and I do miss the coffee breaks and the random discussions. However, once I log off from work I spend time with family, I go out with friends, I watch movies and do other things and guess what I am happy. Work is one aspect of my life and while it is an important part it doesn’t necessarily define who I am as a person.

When I was a kid I complained to my dad that I was bored and he responded that the fact that you are bored is your problem, you need to figure out how to keep yourself entertained because it is not the job of others around you to keep you entertained. So after that me and my sister would come up with things to keep us busy and entertained. The same concept applies to feeling lonely, it is not the job of others in your life to keep you from feeling lonely (they will obviously be there for you) but rather you need to reach out to them and make connections and do things that would keep you entertained & connected.

What do you think?

– Suramya

Comments (0)

January 21, 2023

Fixing AssertionError: Font Arial,Bold can not represent ‘E’ when using Borb to modify PDF Files

Filed under: Computer Software,Knowledgebase,Tech Related — Suramya @ 12:47 AM

I have a bunch of PDF files that I need to modify to remove text from them. Initially I was using LibreDraw but that was a manual task so I thought that I should script it/Automate it. Little did I know that programmatically editing PDF’s is not that simple. I tried a bunch of libraries such as PyPDF4, pikepdf etc but the only one which worked was borb which is a library by Joris Schellekens. They have a great collection of examples and using that I got my first script that searched and replaced text in the PDF working.

However, when I tried to run the script against my pdf file the script fails with the following error:

Traceback (most recent call last):
  File "/home/suramya/Temp/BorbReplace.py", line 26, in 
    main()
  File "/home/suramya/Temp/BorbReplace.py", line 18, in main
    doc = SimpleFindReplace.sub("Manual", "", doc)
  File "/usr/local/lib/python3.10/dist-packages/borb/toolkit/text/simple_find_replace.py", line 80, in sub
    page.apply_redact_annotations()
  File "/usr/local/lib/python3.10/dist-packages/borb/pdf/page/page.py", line 271, in apply_redact_annotations
    .read(io.BytesIO(self["Contents"]["DecodedBytes"]), [])
  File "/usr/local/lib/python3.10/dist-packages/borb/pdf/canvas/canvas_stream_processor.py", line 290, in read
    raise e
  File "/usr/local/lib/python3.10/dist-packages/borb/pdf/canvas/canvas_stream_processor.py", line 284, in read
    operator.invoke(self, operands, event_listeners)
  File "/usr/local/lib/python3.10/dist-packages/borb/pdf/canvas/redacted_canvas_stream_processor.py", line 271, in invoke
    self._write_chunk_of_text(
  File "/usr/local/lib/python3.10/dist-packages/borb/pdf/canvas/redacted_canvas_stream_processor.py", line 203, in _write_chunk_of_text
    )._write_text_bytes()
  File "/usr/local/lib/python3.10/dist-packages/borb/pdf/canvas/layout/text/chunk_of_text.py", line 145, in _write_text_bytes
    return self._write_text_bytes_in_hex()
  File "/usr/local/lib/python3.10/dist-packages/borb/pdf/canvas/layout/text/chunk_of_text.py", line 160, in _write_text_bytes_in_hex
    assert cid is not None, "Font %s can not represent '%s'" % (
AssertionError: Font Arial,Bold can not represent 'E'

Process finished with exit code 1

I tried a couple of different files and the font name changes but the error remains

The script I was using is:

from borb.pdf import Document
from borb.pdf import PDF
from borb.toolkit import SimpleFindReplace

import typing

def main():

    # attempt to read a PDF
    doc: typing.Optional[Document] = None
    with open("/home/suramya/Downloads/t/MAA1.pdf", "rb") as pdf_file_handle:
        doc = PDF.loads(pdf_file_handle)

    # check whether we actually read a PDF
    assert doc is not None

    # find/replace
    doc = SimpleFindReplace.sub("PRIVATE", "XXXX", doc)

    # store
    with open("/home/suramya/Downloads/t/MAABLR_out.pdf", "wb") as pdf_file_handle:
        PDF.dumps(pdf_file_handle, doc)


if __name__ == "__main__":
    main()

I searched on the web and didn’t find any solutions so I reached out to the project owner and they responded with the following message “Not every font can represent every possible character in every language. you are trying to insert a piece of text that contains a character that Arial can not represent. Maybe some weird kind of “E” (since uppercase E should not be a problem).”. The problem was that I wasn’t trying to replace any strange characters, just a normal uppercase E.

To help trouble shoot, they asked me for a copy of the file. So I was masking the data in the PDF file to share it and the script suddenly started working. Turns out that there was an extra space after the word PRIVATE in the file and when I removed it things started working (even on the unmasked file). So it looks like the issue is caused when there is an encoding issue with the PDF file. Opening it in Libre Draw and exporting as a new PDF file seems to resolve the issue.

Now we are a step closer to the solution, I just need to figure out how to convert the file from the command line and I will be home free. Something to work on when I have had some sleep.

– Suramya

Comments (0)

« Newer Posts — Older Posts »

James on Identify what files are being used by a given process/TCP connection: “nice” Mar 18, 06:29

Suramya on Fixing problems with nvidia-driver on Debian Unstable after latest upgrade: “I already have that. The issue is that the versions for firmware-nvidia-gsp, nvidia-kernel-dkms and nvidia-kernel in the repository were not…” Feb 28, 11:22

X-Tech on Fixing problems with nvidia-driver on Debian Unstable after latest upgrade: “Add “non-free-firmware” to /etc/apt/sources.list.” Feb 28, 11:05

Jason on Identify what files are being used by a given process/TCP connection: “Thanks for that my dear friends!” Feb 23, 18:07

Suramya's Blog : Welcome to my crazy life…

February 28, 2023

February 27, 2023

February 21, 2023

February 20, 2023

February 10, 2023

February 2, 2023

February 1, 2023

January 25, 2023

January 22, 2023

January 21, 2023