build-couchdb on Ubuntu 12.04 LTS (Precise)

I'm not sure if anyone else has had this issue with running rake for the awesome build-couchdb script on Ubuntu 12.04, but I kept getting an error:

odd number of arguments to Hash

I don't really know Ruby, but I think this might just be because the default ruby install in Ubuntu is ruby 1.8, and the Hash initialization that build-couchdb uses in the rake task is a ruby 1.9 thing only.  However, after I hacked the distros.rb file to use a saner (1.8-like, I guess) initialization, it seems to work just fine.  So, if you do this, I guess you don't have to worry about installing a later ruby 1.9 build or anything.  You can keep using the default Ubuntu one.  Here's the gist:
https://gist.github.com/4392033

Just copy that over to build-couchdb/tasks/lib/distros.rb, and all the rake commands for build-couchdb should work.

Google Music with Scrobbling to Last.fm

There's a user javascript script for this written by this dude, Daniel Slaughter:
http://www.danielslaughter.com/projects/google-music-with-lastfm/

Now you can scrobble and <3 the music from your ridiculous Google Music collection into last.fm.

You'll need Gresemonkey if you use Firefox (https://addons.mozilla.org/en-US/firefox/addon/greasemonkey/)

You should be able to just install and run the script if you use Chrome (no extensions required).

I don't know about Safari, and Opera has that user.js thing which I forgot how to use.  But, it's probably possible.

Oh yeah, and if you want to check out my last.fm:  http://www.last.fm/user/omiethenull

Android on HP Touchpad

I put the latest cyanogenmod on my Touchpad between traveling around Christmas.   I was pretty happy with it.  So much so that I decided to leave my beloved Kindle DX at home and give the Android Kindle app a shot while I was away.

I had the kindle app installed, but hadn't actually tried it out yet.  While waiting at the airport terminal for my flight, I thought I would fire up the kindle app so that I could download a couple of books off the airport wifi so they'd be there for me to read during the flight.  No such luck, apparently.  The app would sync my library and even show me the books available for download, but every time I tried to download a book I would see, "0% Download Interrupted," or something like that anyway.  I stubbornly tried to download several different titles and got the exact same results every time.  I tried force-stopping the app, clearing the cache, and rebooting the tablet.  Nothing seemed to work, and as the clock at the bottom of the tablet marched steadily to my departure time, I thanked fate that the android installer set the tablet up with a dual boot of WebOS and Android.  I had to boot into WebOS and use its Kindle App to download my books and to read during the flight.

I had resigned to the idea of just booting to WebOS to read my kindle books, but then I started noticing other problems that seemed related.  You might see this or a similar issue manifested in a few different ways, but they all present as if there's an issue with reading or writing to your SD (e.g., it can't access it, it's missing, or it's out of space).  You'll see symptoms that allude to this assessment, anyway.  You may not see any actual error messages.  Many apps seem to just behave much like the Kindle app:  refusing to download or install data, modules, or other apps.  In fact, the reason I got a hard head about trying to fix the issue with the Kindle app is because none of my Amazon apps were working correctly.  Amazon app store would loop for about 5 iterations trying to download apps before giving up silently and offering the "Install" button again.  Amazon mp3 wouldn't stream music, but it would try forever -- a spinning loading icon forever looping in vanity where the play/pause button should have been.

And then the real clincher:  Shortyz crosswords would install, and then appear to download puzzles., but none would be there.  Upon restart, it did actually give me an error message about the SD card.  That's when I decided all of the issued s were likely related and would really hinder enjoyment of android on my TP.
I know the "SD card" is really just the Touchpad's internal memory.  I dug around some forums and saw that it was mounted on "/mnt/sdcard."  I don't know why, but apparently it gets unmounted or remounted read-only sometimes.  Here's how I remount it read-write:

1. Open "Terminal Emulator" from the app laucnher.
2. Type:  su <enter>
3. Type: mount -o remount rw /mnt/sdcard <enter>
4. That's it.

There were many other solutions and vague problem descriptions that could have been this same issue all over the Android forums, including rebooting the device (which never worked for me), and booting into ClockWorkRecovery and mounting /mnt/sdcard from a menu item (which may just do the exact same thing as what I listed above).  Anyway, this seems to be the simplest and quickest fix.

After that, I had to "Uninstall" some of the apps that were causing issues.  The first one I uninstalled and re-installed was Shortyz Crosswords, and, after re-installing, it downloaded all of the crossword puzzles without a hitch.  Emboldened, that's when I thought that this could have fixed the other issues, too.  Sure enough, my Kindle books downloaded (you can't re-install the Kindle app, though -- apparently I didn't need to).  The Kindle App store also began to work, and so did the Amazon MP3 app.

Unfortunately, I don't know of a permanent solution.  It seems as though I have to pull up terminal and run this every so often.  Some of the posts on the forums indicated that a full re-install of the Android ROM on their Touchpad fixed the issues, and others swore that a "Factory Reset" of Android fixed it for them.  I haven't tried either yet, and if you're not ready to nuke all of your settings after typing in all of your passwords and account info into all of the apps, then this is a good temporary fix.

Google 2-factor Authentication (aka OTP) and Application Specific Passwords

This should be more widely publicized as it is now available for every one.

When I search for "Google 2-factor" on Google.com, the best I get is news on 2-factor and not a direct link.  Hopefully, this quick write-up will help some people find this.

What is it?
2-factor or OTP (One-Time-Passcode) (or 2-step verification) is a 2nd level of security that can be added to a Google account.  This technology is not exclusive to Google, but we're lucky that Google offers it.  The way it works is that you use a mobile phone to generate a passcode that only lasts for a very limited period of time.  You then enter this password along with your normal password when you sign into your Google account on the web.

Here's what it looks like after you enter your user name and password and you have 2-factor enabled:

The codes that you'll be generating will look something like this:  581775


You'll need a cell phone that you carry around often.  Once you enable and setup 2-step verification, any computer or new browser you use will show you a screen like the one pictured above.  You'll need your phone and an application installed on your phone to generate a code.  You'll enter this code into the box to sign-in.  Google supports Android, Blackberry, and iPhone through native apps for each platform.  But, if you have a dumb phone, you can select an option to use SMS text messaging.  If all you have is a landline you can even have Google send an automated voice message to deliver your passcode!  This means that anyone who usually has a phone nearby should be able to use this feature.

Using 2-step verification will also enable the use of Application Specific Passwords.  This might, at first, turn some people off of the idea.  However, I highly recommend using these, too.

What are Application Specific Passwords?
If you never access any Google accounts or apps through anything but a web browser, then you don't need to worry about Application Specific Passwords.  If you use native chat clients, an Android or iPhone to access your Google account (through the apps, not the web browser), POP or IMAP email clients like Outlook or Apple Mail, or you use automated scripts to access your Google account, then you'll need to understand and use Application Specific Passwords.

Application Specific Passwords are set up through your Google account and are used to grant access only to the applications you specify.  One "Specific Password" for each "Application."  If you use Adium to sign into Google Talk, then you'd go to your account page, generate an Application Specific Password for Adium, and then enter that into the "Password" field in Adium.  You use the Application Specific Password you generated from your Google account instead of using your normal Google password.

The generated password will be long and difficult, but you can just copy and paste it from your Google accounts page.  You should never have to memorize this Application Specific Password.  And, don't worry -- if you lose it you can pull it up again later.

You can probably see that if Google turned on 2-step but allowed external programs to access your account without 2-step, then that would leave a hole in the 2-step security mechanism.  There are actually some great advantages to Application Specific Passwords beyond just patching a security hole:

• If you suspect your password for your email client has been stolen, you don't have to reset your Google password at all.  You just have to delete the old one, revoke access, and then generate a new one just for that one application.
• You always know exactly which apps are accessing your Google account because you gave them access from your accounts page.  And if you suspect one is compromised, again -- you can just revoke access.

So, let's say you are a Google user.  You use Gmail from the web, you use Adium to log into Google Talk, you use Apple Mail to check your Gmail, and you have an Android phone.  Here's what you'll be doing:

1. Set up 2-step verification on your Google Account.
2. Download the Google Authenticator app for Android.
3. Install Google Authenticator on your Android.
4. Start using 2-step.
5. Set up an Application Specific Password for your Android phone's login through Google Accounts.
6. Generate an Application Specific Password for Apple Mail.  Start using it to check your mail.
7. Generate an Application Specific Password for Adium, and start using it right away.

Then you and your account will be that much more secure.  Any time you (or, more importantly, someone else) tries to access your Google Account from another web browser, a second password step must be completed (thus, 2-step).  Your 2-step verification code will have to be entered.  If it's not you on the other end of the screen, and it's instead a bad guy trying to get in, his hacking attempt will be stymied!  He won't have your phone to generate the verification code.  If any other application besides the ones you've authorized try to access your account (even if they have your main Google password), they will also fail.

More astute readers can probably come up with several implications.  For example, let's say you have a prying mother, ex-girlfriend, landlord, thief, hacker, cat, or dog who has been routinely and furtively spying on you because she's had your Google Account password.  Well, the next time this amateur casually types in your password, she will be greeted with this extra verification screen.  Someone could be regularly using your account right now without you even knowing it, and by applying these optional security measures you'd be locking them out of ever accessing your account again.

However, when you log in to your account, you'll just whip out your trusty cell phone, punch in your code, and be using your Google Account with ease.

Also, note that there is an option to have your browser save your verification on the particular browser you're using for 30 days.  This way, if it's your home computer, you don't have to punch in a verification every time.  Use this feature sparingly -- I would only use it on my home computer.  Now you see another implication -- you can use those scary public computers and kiosks with a lot more confidence!  Even if there are key-loggers installed, they won't have your verification code for the next log-in because it constantly changes.  It doesn't matter if any one gets your temporary code.  It's not possible to predict or derive future codes from the current code.  Once you use the current code it won't be reusable.

Doing the Two-Step
To start setting up 2-step verification, go here:  https://www.google.com/accounts/SmsAuthConfig while logged into your Google account.

You'll see a screen like this:

Now it will guide you through the process to setup 2-step verification.  Since I started writing the draft for this article, Google has published some good documentation about it.  Here's a link to the documentation to get you started and help you understand exactly what you're getting into:

2-step:
http://www.google.com/support/accounts/bin/static.py?page=guide.cs&guide=1056283&topic=1056284

Application Specific Passwords:
http://www.google.com/support/accounts/bin/static.py?page=guide.cs&guide=1056283&topic=1056286

At first, you may bemoan the very idea of having yet another security loop to jump through.  But, I hope that I've helped you understand the implications and shed some light on the very real and cogent benefits to using these options.  Two-step verification can be disabled if you decide you don't like it.  I haven't regretted switching over yet.  At first, the extra step was a minor inconvenience, and now it's an unobtrusive good habit.

suds and appengine

suds is an incredibly useful and easy to use client library for SOAP.  Many times better than using SOAPy or ZSI (for fun client stuff, anyway).

However, suds has a couple of problems working with appengine.  This is from versions 0.3.9 and 0.4.

(complete file for what follows: suds_memcache.py

1. Its default client constructor has a hard initialization of self.options.cache = ObjectCache(days=1), and ObjectCache by default is a FileCache, and FileCache tries to use a temporary file which app engine doesn't like.  Maybe we can use memcache instead?

2. For some reason the suds.transport.http.HttpTransport.u2open() function breaks here:

        if self.u2ver() < 2.6:

            socket.setdefaulttimeout(tm)

            return url.open(u2request)

        else:

            return url.open(u2request, timeout=tm)

at least in app engine, that is.

I wanted to make it work with app engine without actually changing any of the suds code itself.  Here's how I did it.  Now I realize I could have overridden ObjectCache(days=1) with a similar effect, but then it would have had to ignore days which seems dirty.

First, I overrode the u2open() function in transport.http.HttpTransport:


def new_u2open(self, u2request):
    tm = self.options.timeout
    url = self.u2opener()
    if self.u2ver() < 2.6:
        return url.open(u2request)
    else:
        return url.open(u2request, timeout=tm)


transport.http.HttpTransport.u2open = new_u2open


Then, I had to override the entire Client constructor because of the way suds initializes options.cache in the original.  And, I'm not sure if it's possible to just override pieces of a constructor in Python without executing it (I imagine it isn't).



class Client(client.Client):
    def __init__(self, url, **kwargs):
        options = Options()
        options.transport = HttpAuthenticated()
        self.options = options
        options.cache = MemCache()
        self.set_options(**kwargs)
        reader = DefinitionsReader(options, Definitions)
        self.wsdl = reader.open(url)
        plugins = PluginContainer(options.plugins)
        plugins.init.initialized(wsdl=self.wsdl)
        self.factory = Factory(self.wsdl)
        self.service = ServiceSelector(self, self.wsdl.services)
        self.sd = []
        for s in self.wsdl.services:
            sd = ServiceDefinition(self.wsdl, s)
            self.sd.append(sd)
        self.messages = dict(tx=None, rx=None)


And then I threw together a memcache implementation of suds.cache.Cache.  I copied the definitions from suds's suds.cache.Cache abstract class, so it has extra "getf" and "putf" methods, and I'm not sure what they do, so I just overrode them too:



class MemCache(cache.Cache):
    '''
    attempt to implement a memcache cache in suds
    '''
    def __init__(self, duration=3600):
        self.duration = duration
        self.client = memcache.Client()
    
    def get(self, id):
        thing = self.client.get(id)
        return thing
    
    def getf(self, id):
        return self.get(id)
    
    def put(self, id, object):
        self.client.set(id, object, self.duration)
    
    def putf(self, id, fp):
        self.put(id, fp)
    
    def purge(self, id):
        self.client.delete(id)
    
    def clear(self):
        pass

I'm not sure how well the MemCache() Cache works, but I know it doesn't crash appengine on my local instance, and my app runs fine with an WSDL URL loaded.  Here's how you want to call it in your appengine client.  Imagine the file you put all this junk into is named, suds_memcache:

from suds_memcache import Client

    url = ('http://www.webservicex.net/CurrencyConvertor.asmx?      WSDL')

    c = Client(url)

    print c

Well, hopefully you get the idea, anyway.  Here's the complete file:

suds_memcache.py

Note:  If MemCache() doesn't work for you, or you just don't want to use it, you can use suds.cache.NoCache() instead (which comes with suds).  But you still need to put it in the constructor override.

Addendum:
I just noticed that I had a few problems using this with 0.3.9 (need to remove the Plugin stuff because it's only in 0.4).  Also, when loading a wsdl off the disk, I noticed that the MemCache() implementation fails when reading or writing to app engine's memcache.  This is because suds passes an ObjectId object to the caching functions (at least in 0.3.9), but app engine expects a string and throws a TypeError otherwise.  I fixed this by overriding suds.reader.ObjectId with a __str__ method (it didn't have one), and then wrapping the id variables in the MemCache() implementation with str().  I'll post the new version later tonight.  Maybe I'll put it on my github or something, too.  UPDATE:  Uploaded the newer file.

UPDATE:
I found out you don't actually have to override the Client constructor.  Chalk it up to not fully understanding the suds code.  You can just pass in cache=None or cache=suds_memcache.MemCache() (or you should be able to, anyway.  I've had success so far).  e.g.
client = suds.client.Client(url, cache=None)
I'm still having to use the u2open() override, though.