Suramya's Blog : Welcome to my crazy life…

November 18, 2022

Twitter Extract: Downloading data not exposed in the Official Data export

Filed under: Computer Software,Software Releases — Suramya @ 2:46 PM

It looks like Twitter is imploding and even though I don’t think that it will go down permanently it seemed like a good time to export data so that I have a local backup available. Usually I just ask Twitter to export my data but this time I needed additional data as I was preparing a backup that would work even when Twitter was down completely. The Twitter Export didn’t give me all the data I wanted, specifically I needed an export of the following which wasn’t there in the official export:

  • Owned Lists (Including Followers and Members of the list)
  • Subscribed Lists with Followers and Members of the List
  • List of all Followers (ScreenName, Fullname and ID)
  • List of all Following (ScreenName, Funnname and ID)

So I created a script that exports the above. It is available for download at: Github: TwitterExtract. Instructions for installation and running are there in the ReadMe file.

This was created as a quick and dirty solution so it is not productionalized (i.e. it doesn’t have a lot of error checking, hardening etc) but it does what it is supposed to do. Check it out and let me know what you think. Bug Fixes and additional features are welcome…

– Suramya

July 30, 2022

Identifying the least used packages on Debian

My main system was running low on disk space in the root partition and I wanted to clean out some of the unused software from the system. In order to do that I thought that I should find out what the least used applications on my system were and then remove them. Unfortunately I couldn’t find any existing way of doing this so it was a dead end. However, the problem remained stuck in my head and I came up with a quick and dirty way of identifying the packages and when they were last used.

The way it works is:

  • Get a list of all files on the system (using locate, since its already there so why duplicate effort)
  • For each file figure out what package it belongs to using dpkg-query -S
  • If the file belongs to a package, get the last access time (using stat) and log it
  • Once we do this for all files, sort the results.

This gives us a list of packages and the latest access date for each package (based on the latest access date for any of the files in it). Since this is a quick and dirty implementation, it is slow as molasses, doesn’t have any error checking or anything but still gets the job done. Would love to get some feedback. The code is available at: https://github.com/suramyatomar/leastUsedPackage.

The output of the script looks like:

...
...
xz-utils | 2022-07-18
yelp-xsl | 2022-04-05
yelp-xsl | 2022-04-05
youtube-dl | 2022-07-17
zim | 2022-07-17
zip | 2022-07-17
zlib1g-dev | 2022-07-17
zlib1g-dev | 2022-07-17
zlib1g-dev | 2022-07-17
zstd | 2022-07-18

Feel free to try it out if you have a similar usecase. Let me know if you have any suggestions on improving the script or if you found it useful.

– Suramya

September 20, 2011

New Script Web2Email Released

Filed under: Linux/Unix Related,Software Releases,Tech Related — Suramya @ 2:47 AM

Just released Ver 0.1 of Web2Email. Web2Email is a Perl Script that allows the user to send a message with a list of URL’s to a special mail account and get back a snapshot of the websites as PDF files.

You can download it from Scripts.Suramya.com: Web2Email.

If you want a demo, send an email to: web2email@suramya.com with website URL’s in the email body (one per line) and after about 5 mins you will get an email with the snapshots of the sites listed as PDF files.

Update: The cron job running this on my machine is having issues, will fix it once I get back home and this should be back up again tonight.

Update (22nd Sep 2011): The project wasn’t the cron job, there is an issue with the version of wkhtmltopdf that is available in the Debian repo. Basically if you have both http & https images in a site it goes for a toss. The static built version should fix this issue but haven’t got the time to install and test. Till that is done the demo version will be on hold.

As always I would appreciated any feedback/suggestions you might have.

– Suramya

August 15, 2009

Posting to WordPress Blogs from Lifeblog on Nokia Phones

Filed under: Computer Software,Software Releases,Tech Related — Suramya @ 4:45 AM

I have been wanting to have the option to make blog posts directly from lifeblog on my N95 for a while now but so far was unsuccessful because no plugin existed that worked with the latest version of WordPress.

So I downloaded the latest available copy of the plugin from this blog post and modified it to work with the latest version of WordPress.

You can download the latest version of the plugin from the Scripts Directory. This version works with WordPress 2.8.x (I tested it with 2.8.2 and 2.8.4).

Changelog :
Ver 3.5: (15th Aug 2009):

* Plugin is now compatible with WordPress 2.8.x (Tested upto 2.8.4)
* Replaced the SQL queries for categories creation/check with WordPress Functions
* Replaced the SQL queries for user checks with WordPress functions
* Added ability to Enable/Disable Debug Logging by changing one variable

Let me know if you have any questions/issues.

– Suramya

February 18, 2006

CollageGen Ver 0.5 released

Filed under: Software Releases,Tech Related,Website Updates — Suramya @ 5:13 PM

Released Version 0.5 of the Collage Generator. This time the source code for the script is also available for download at the site.

I do have a working demonstration page but it don’t works if one person uses it at a time, because it queues up each request it recieves and process it in a FIFO (First In First Out) format, so if too many people are using it at the same time the page time’s out. Will probably switch to a threaded model to allow parallel processing of requests.

ChangeLog:

  • Rewrote the Backend Script to speed up the collage generator faster
  • Added XML-RPC support to allow better communication between the frontend and the backend

Your Feedback/Comments would be appreciated.

Thanks,
Suramya

December 11, 2005

Ver 0.3 of getHype released

Filed under: Software Releases,Website Updates — Suramya @ 1:49 AM

After a long delay Ver 0.3 of getHype has been finally released.

Change Log

Ver 0.3: (11th Dec 2005):

* getHype now gets the list of mp3’s to download from the RSS feed instead of getting them from the GetHype frontpage.
* The files are now downloaded using the LWP perl module instead of using wget. So its a bit more secure/safer.
* Added an option to let the user set the network timeout in seconds.

Check it out and let me know what you think.

Thanks,
Suramya

October 25, 2005

Added WebForum to gWiFi.net

Filed under: gWiFi,Software Releases,Tech Related — Suramya @ 1:20 AM

Added a new webforum to the gWiFi.net site to allow visitors to post their feedback and suggest new Hot Spots for inclusion in the Database.

Check it out and let me know what you think of it.

Thanks,
Suramya

September 27, 2005

CollageGen is down

Filed under: Software Releases,Website Updates — Suramya @ 11:02 AM

Had to take down the CollageGen as the backend process kept dying for some wierd reason. Till I figure out whats going on there’s no point in keeping the frontend up as you won’t see anything if the backend is down.

I hope to figure this out soon, lets see. Will keep you posted.

– Suramya

September 17, 2005

Ver 0.3 of the CollageGen script released

Filed under: Software Releases,Tech Related,Website Updates — Suramya @ 3:14 AM

I have just Released CollageGen, Version 0.3. Incase you are wondering, CollegeGen is an application that uses the Yahoo search API to create a collage for any search term entered by the user.

Changes made in this release:

  • The backend script was rewritten from scratch to make the collage generation faster and more efficient.
  • Redesigned the CollageGen access website
  • Threading support was removed from the backend

Check it out and let me know what you think.

Thanks,
Suramya

September 7, 2005

Moderation capability added to the Feedback section

Filed under: Software Releases,Website Updates — Suramya @ 6:53 AM

Just finised writing Version 3 of my Guestbook script. This version has a moderation capability and all comments posted in the feedback section will now need to be approved before they show up.

This was done to stop the idiots who post links to gambling sites/porn sites etc from posting messages. Earlier I had to manually edit the DB and delete the entries, now I have a new interface that allows me to edit/delete/approve comments all from one screen.

Next on the things to do is to move the rest of the admin interfaces to this new system. This will consolidate the admin function to one GUI making it easier for me to maintain and upgrade. Plus this GUI looks a lot nicer too. 🙂

You shouldn’t notice any differences but Let me know if you have any questions.

Thanks,
Suramya

Older Posts »

Powered by WordPress