Posts Tagged ‘Python’

TWiT Early Show Downloader

For me, TWiT network is one of the most valuable resources on the internet: their programming is always informative, slick and balanced, and the guests are always top-teir tech talkers.

As with most pod/netcasts broadcast from the States, the scheduling of live shows is always at pretty peculiar hours of the day thanks to the time difference, which means waiting for the edited version to be uploaded and published to the TWiT website.

Handily, the editors seem to work much quicker than the web folks over in Petaluma, CA, and the shows are normally uploaded quite a bit earlier than they are listed on the website. I’ve been using a little Python program which I wrote to download the shows before they are linked on the site, which you can grab using the instructions below. It uses cURL to download the files so it’ll work on Macintosh and Linux systems.

(more…)

Super Simple Linux RAM Disk Manager – rdmanage

I’m currently developing a piece of server software which needs to be able to create, modify, serve up and dispose of many temporary files many times every second. Because a great number of I/O operations will be occurring, the program requires very rapid disk access – ideally, it would run on a RAID array or a solid-state disk. Because the server I’m using doesn’t have either, I’m using RAM disks as a substitute.

RAM disks take a portion of your system RAM memory and make it available for storage by mounting it in your file system. Using a RAM disk, you gain a working folder with speed way far in excess of any hard disk or even solid state drive. The usual rules of RAM apply – this is volatile memory, so anything stored on a RAM disk will be erased when the power is cut – but it’s perfect for temporary file storage.

I’ve written a very simple program for Linux called ‘rdmanage’ to help create and manage RAM disks easily at the command line.

Operation:

There are three modes of operation for this program – create a new RAM disk, create a RAM disk from an existing folder, and remove existing RAM disk. As promised, the program is very simple to operate:

To create a RAM Disk:

$ ./rdmanage create [size in mb]

For example, to create a 100MB RAM disk:

$ ./rdmanage create 100

To create RAM Disk from an existing folder:

$ ./rdmanage from /path/to/folder

For example, to create a RAM disk from the folder ‘/home/randymarsh/sim’:

$ ./rdmanage from /home/randymarsh/sim

To remove an existing RAM Disk:

$ ./rdmanage remove

Notes:

– When using the ‘from folder’ mode, the maximum size of the folder is 512MB

– You will need root permissions to mount the RAM drive

– The ‘remove’ mode will commit the state of the RAM drive to disk in a folder called ‘ramdisk.contents’

– All operations will occur within the current working directory (i.e. the folder you’re presently in). To check which folder you’re in, type ‘pwd’ at the command prompt.

– Depending on how you download the program, you might have to set the file as executable:

$ chmod +x rdmanage

– If you want to check the status of your drive, it will be listed when you use the ‘df’ command

– For program help, type:

$ ./rdmanage help

Download

You can download the program by right-clicking here and selecting ‘Save As’/’Save Target As’, or if you would prefer to download straight to your Linux machine from the command line, use wget:

$ wget http://zebpedersen.co.uk/python/rdmanage

Hopefully this program can save you a bit of time creating and managing your RAM disks. Enjoy!

Super Simple WordPress Backup Program

Whatever you do with your computer, it’s absolutely vital to back up your data. Never is this more true than when you’re administering a website, where creating backups should be a well organised and regular procedure.

This site is no exception to that rule, so to help me out with my daily backups I’ve written a small program in the Python programming language which automates the process of archiving daily snapshots of my site. For peace of mind, I still prefer to launch and organise backups myself rather than use completely automatic solutions; this program helps make that process much quicker.

Download

You can download the program by right-clicking here and pressing ‘Save As’/’Save Target As’.

If you want to download straight to your Linux machine from the command line, use wget:

$ wget http://zebpedersen.co.uk/python/wordpressbackup

The program is designed to run under Linux, but theoretically it will work on any Unix system (Mac OS X, Solaris etc…) with Python installed.

Operation

All you need to feed the program is the path to your WordPress installation (the directory containing your wp-****.php files). The database login credentials are borrowed from WordPress, and the backup is placed in a date-stamped folder for easy archiving.

Each backup contains a dump of your MySQL database and a tarball of your installation directory – everything you need to restore your site in case of a disaster. The backup will be created in the folder from which you start the program.

To launch the program:

$ ./wordpressbackup /path/to/wordpress/installation

For example, if your site is located in the directory ‘/home/randymarsh/web’, the command would be:

$ ./wordpressbackup /home/randymarsh/web

If you need help or more information, use the help & about functions:

$ ./wordpressbackup help

$ ./wordpressbackup about

Hopefully this little script can help out anyone else like me who only trusts a backup they’ve done by hand. Enjoy!

 

Linux Command Line Dropbox Tips & Tricks [dropbox.py]

The fantastic, wonderful, life enhancing file sync platform Dropbox is easily the best way to share your files between your (sometimes many) devices and across (sometimes many) platforms. Every computer you add to your account increases the redundancy of the data you have stored, and if you value your work it’s a real weight off to know your files are stored safely in multiple geographic locations.

Dropbox is super easy to set up for Mac OS X and Windows (although harder for some Linux distros), but if you’ve got a Linux server that you want to hook up, you’ll have to jump through some command-line hoops before you’re ready to go. Luckily, there’s a tutorial page on the Dropbox wiki which is really helpful and, provided there are no hitches, will get you up and running pretty quickly.

There are a couple of points I’d like to make in addition to the previously mentioned document to address a couple of issues I had when I first tried to install the Dropbox CLI Linux client:

  • Make sure you’re logged in as the user you’ll be accessing the files as when you install. Sounds obvious, but some people like to install with an outright ‘su -' rather than a regular ‘sudo' which can lead to syncing the files as root.
  • When you get to the stage where you’re ready to launch the daemon for the first time, make sure you return the command line. Instead of running…
    # ~/.dropbox-dist/dropboxd
    …do yourself a favour and add an & to the end:
    # ~/.dropbox-dist/dropboxd &
    Especially if you have one of the premium accounts (>2GB), the first sync is going to take a long time, so make sure you use the ampersand so you can carry on working in the mean time.
  • Once you’ve installed, grab the dropbox.py control script. You can get it from the Dropbox website by following this link – it’s mentioned on the wiki linked earlier, but is buried right down at the bottom of the page. It should be the first thing you download!
  • If you’re having trouble running the script then try installing ‘python2.6’ and running using:
    # python2.6 dropbox.py
    Again, this is hidden at the bottom of the wiki and is dead important.
  • The three most useful commands for the dropbox.py script are:
    # python2.6 dropbox.py start
    # python2.6 dropbox.py stop
    and
    # python2.6 dropbox.py statusThese three commands give you all the control you should need to proceed with your enjoyment of Dropbox on your headless Linux server.

So there’s a few tips and tricks I want to add to the excellent official Dropbox wiki page. As with everything I post on my site, these are collections of solutions I’ve had to either research from multiple sources, have worked out for myself or have had trouble finding answers for elsewhere, so hopefully I can save you some trouble and help you get up and Dropboxing in less time than it took me the first go around!