Build a Talking Calendar with GoogleCL and Android Scripting Environment

At the end of my post on how to Setup GoogleCL on WinXP I joked about running GoogleCL, Google's new command line tool for accessing Google service, on the Android OS. Trying to make that joke a reality, I found the Android Scripting Environment project which allows running Python scripts on Android - step one: accomplished.

Warning: this is not for the faint of heart. The instructions that follow will require you to not only install software on your Android device, but also to download and modify Python source code. I will assume that you are capable of using SVN to get source code from code.google.com and applying a .patch file. And you'll probably end up having to type on a really tiny keyboard at some point. Don't say I didn't warn you.

Install Android Scripting Environment (ASE)

The easiest way to install ASE is to open up the ASE home page on your PC and scan the barcode.

Hack GoogleCL

Note: this change is now in trunk and won't be necessary in the 0.9.8 release.

Now grab the source code for GoogleCL (see here).

I needed to apply a small patch to get my setup to work with the 0.9.7 release.

The Patch

Index: src/googlecl/__init__.py
===================================================================
--- src/googlecl/__init__.py (revision 299)
+++ src/googlecl/__init__.py (working copy)
@@ -156,7 +156,7 @@
   import pickle
   token_path = os.path.join(GOOGLE_CL_DIR, TOKENS_FILENAME_FORMAT % user)
   if os.path.exists(token_path):
-    with open(token_path, 'r') as token_file:
+    with open(token_path, 'rb') as token_file:
       token_dict = pickle.load(token_file)
     try:
       token = token_dict[service.lower()]
@@ -184,7 +184,7 @@
   token_path = os.path.join(GOOGLE_CL_DIR, TOKENS_FILENAME_FORMAT % user)
   success = False
   if os.path.exists(token_path):
-    with open(token_path, 'r+') as token_file:
+    with open(token_path, 'rb+') as token_file:
       token_dict = pickle.load(token_file)
       try:
         del token_dict[service.lower()]
@@ -236,12 +236,12 @@
   import stat
   token_path = os.path.join(GOOGLE_CL_DIR, TOKENS_FILENAME_FORMAT % user)
   if os.path.exists(token_path):
-    with open(token_path, 'r') as token_file:
+    with open(token_path, 'rb') as token_file:
       token_dict = pickle.load(token_file)
   else:
     token_dict = {}
   token_dict[service] = token 
-  with open(token_path, 'w') as token_file:
+  with open(token_path, 'wb') as token_file:
     # Ensure only the owner of the file has read/write permission
     os.chmod(token_path, stat.S_IRUSR | stat.S_IWUSR)
     pickle.dump(token_dict, token_file)

I made it small on purpose because it's hard to read anyway. Hopefully, this way you'll be able to select it all and copy it easily. Apply the patch to your GoogleCL code.

The patch makes GoogleCL open the auth token file in binary mode. I needed this because later I'll be copying a token file from my desktop to my phone. Without the patch, the phone rejected token the file.

Install GoogleCL

Now you need to copy your GoogleCL source onto the phone. Your Android phone may vary. On mine, I just connect to my desktop computer via USB and select "Disk Drive" as the USB connection type. This mounts the phone's storage as two removable disks. You can also remove the SD card from your phone and put it in a reader connected to your computer, since it's the card we're putting files on.

Find ase/extras/python/ on your SD card. This directory should contain a number of Python modules already, such as gdata and xml. Copy your googlecl/src/googlecl in next to them. Also copy googlecl/src/google to ase/scripts/google.py. You'll use this later when you write your talking calendar scripts.

Write a Script to Call GoogleCL

This script changes your HOME dir to /sdcard so that GoogleCL can write your .googlecl configuration file. I wasn't able to figure out how to let GoogleCL write to the default HOME location. The other settings are what ASE uses when it runs Python scripts. I don't know how many of them are required to actually make this work, and I haven't felt like fiddling with it too much since it's working.

Replace [USERNAME] with your Google account username and save it as ase/scripts/calendar_today.sh.

#!/bin/bash 
export HOME=/sdcard 
export TEMP="/sdcard/ase/extras/pythontmp" 
export PYTHONHOME="/data/data/com.google.ase/python" 
export PYTHONPATH="/sdcard/ase/extras/python:/sdcard/ase/scripts/"
/data/data/com.google.ase/python/bin/python /sdcard/ase/scripts/google.py 
    --user [USERNAME] calendar list --date `date +%Y-%m-%d`

Make It Talk

I love this part because it's so easy with Android. This script calls the one above and passes the output to the phone's text-to-speech service. Save it as ase/scripts/say_calendar.py.

import android 
import subprocess  
p = subprocess.Popen(
        ['/system/bin/sh',  '/sdcard/ase/scripts/calendar_today.sh'],
        stdout=subprocess.PIPE, 
        stderr=subprocess.PIPE)
output, errors = p.communicate()
print errors 
print output 
droid = android.Android() 
droid.ttsSpeak(output)

Important Note for Windows Users: be sure to convert any files written on Windows to UNIX line endings before copying them to your phone. If you need to, you can also open up the files on your phone and remove the extra line end characters there. ASE has a file editor built in and I use ASTRO for editing other files.

Preauthentication

I couldn't get the GoogleCL OAuth process to work on my phone. When I returned to ASE from completing the auth process in the browser the script had already stopped running, interrupting the authentication. I got around that by preparing a auth token file on my desktop and copying it to the phone.

Using your modified copy of GoogleCL on the desktop, authenticate to each of the services you want to use from your phone. Then take the access_tok_[username] file from the .googlecl directory under your home directory and put that file in /sdcard/.googlecl/ on your phone.

Dismount your phone's drives from your desktop, or reinstall the SD card in your phone.

Make It So

Now, assuming you've followed my instructions exactly and I haven't left out anything important, you should be able to:

  • Open up ASE on your phone
  • Select Menu->View->Interpreters
  • Select Menu->Add->Python
  • Press back to return to the list of scripts

You should see a number of scripts listed, including the ones you just wrote. Select say_calendar.py. Your calendar entries for today should be displayed and the phone should speak them soon after. Now, it may not sound the way you expect - the TTS isn't perfect. But I think that's pretty good for a few minutes setting up some free scripting tools.

I would like to extend a very big thank you to Jason Holt and Tom Miller. Their encouragement and advice made this possible. Thanks, guys.

Fixing a bad habit with GoogleCL

The Problem

I have a bad habit to confess. When I'm on the telephone and I need to write something down, I don't reach for paper and pencil. Instead, I open Notepad. I take my notes and save the file on my desktop where I can easily find it later. Information preserved! But that file is just sitting on my hard drive on my home PC. For a modern, cloud-connected individual such as myself, this is bad. I can't read that note from my Android phone, from my laptop, or even from the PC in the other room. Because it's on that hard drive.

This isn't long-term storage. I usually don't need the notes for long and I trash them within a few hours or days. If they have information of greater value, I'll copy the text into Google Docs where I can get it later. But what if I need that file before it gets deleted and I'm not sitting at my computer?

The Solution

Sure, I could open up Chrome, navigate to Google Docs, and create a new document to take my notes. But that takes a lot longer than win+r > notepad. Enter GoogleCL:

google docs edit --format "txt" --editor "notepad" --title "notes"

On Friday I wrote about setting up GoogleCL on Windows. After that setup, I can use the above command to edit a text file named "notes" that resides in Google Docs. The file is automatically downloaded and displayed in Notepad or any other editor. After saving my changes and closing Notepad, the file is uploaded to Google Docs again. Pretty slick.

Now I just need to turn that command into something more manageable.

@ECHO OFF
SET filename=%*

IF "%filename%" EQU "" (
ECHO File name:
SET /p filename=
)

google docs edit --format "txt" --editor "notepad" --title "%filename%"

I called it "gd.cmd" -- "gd" for "Google Docs" should be quick to type and easy to remember. That script will take an argument for the file name and even prompt me if I don't give a file name initially. You can change the --editor option and even the --format if you don't like plain text.

There's also a --folder option which I expected would let me keep my notes in a folder in Google Docs. Well, it does look in that folder but if it doesn't find the file, it will create the file outside of the folder, so that next time it still won't find it! I opened issue #133 to see about fixing that.

Setup GoogleCL on WinXP

UPDATE: These instructions were originally written for version 0.9.5. I have started updating them with differences found in the newer 0.9.7 version of GoogleCL. If you find more differences I should mention, please leave a comment. Thanks!

Google just announced their new command line utility for Google services. The GoogleCL project is an Open Source project hosted on Google Code. Although the project page is obviously slanted toward Linux, I figured it's Python so it shouldn't be too hard to run on Windows too. Here's what I had to do to get things setup on Windows XP Professional SP3.

First some downloads:

Note: GoogleCL requires Python 2.5 or 2.6.

Install Python

Run the .msi file to install Python. I installed to C:\dev\Python\Python25 and I added C:\dev\Python\Python25 to my PATH for easier command line access to python.exe.

Install Google Data API

I unzipped gdata-2.0.10.zip to C:\dev\Python\gdata-2.0.10 and ran setup.py install from C:\dev\Python\gdata-2.0.10. No problems encountered there.

C:\dev\Python\gdata-2.0.10>setup.py install

Install GoogleCL

I unzipped googlecl-0.9.5.tar.gz to C:\dev\Python\googlecl-0.9.5 and ran setup.py install from C:\dev\Python\googlecl-0.9.5. Again, no problems.

C:\dev\Python\googlecl-0.9.5>setup.py install

Write a "google" script

Next I wanted a "google" command that would be as easy to run from the Windows command line as the Linux examples were. I wrote this google.cmd file and put it on my PATH:

@echo off
SET googlecl_home=C:\dev\Python\googlecl-0.9.5\src

python %googlecl_home%\google %*

Now, running google docs list got me a prompt for my user name. But instead of prompting for a password, GoogleCL uses OAuth. This is the message I got:

Note: this was true as of GoogleCL v0.9.5 - see below for v0.9.7

(Hint: You can automatically launch your browser by adding "auth_browser = " to your config file under the GENERAL section, or define the BROWSER environment variable.)

Please log in and/or grant access via your browser at
https://www.google.com/accounts/OAuthAuthorizeToken?oauth_token=[token removed]
then hit enter.

Well, obviously that wouldn't do. It would have been nice if it had told me where to find that config file, but it wasn't hard to locate. As alert reader Brian B. pointed out, the first time you run GoogleCL it will try to locate the config file and if it doesn't find one, it will tell you the path and create the file for you. Mine was in C:\Documents and Settings\itruett\.googlecl.

The path for my browser, Google Chrome, is a little less intuitive. I found it, and added this to my GoogleCL config file:

auth_browser = C:\Documents and Settings\itruett\Local Settings\Application Data\Google\Chrome\Application\chrome.exe

The next time I ran GoogleCL, Chrome launched and showed me the authorization page. A couple of clicks and that was done. I had to authorize once for each Google service.

Update: v0.9.7 Browser Authorization

The first time you try to access each service, GoogleCL will display this message:

Please log in and/or grant access via your browser at
https://www.google.com/accounts/OAuthAuthorizeToken?oauth_token=[token removed]
then hit enter.

GoogleCL will launch your default browser to the authorization page where you can grant your approval. You only need to do this once for each service.

Version 0.9.5 required auth_browser to be set in your GoogleCL config file in order to launch the browser automatically. Reader Joe Ledvina reported that auth_browser causes problems with version 0.9.7. Removing auth_browser will allow your default browser to launch. I believe this to be a bug and reported it as such in a comment to issue #51.

One complaint I have about the authorizations is that they just show up as "anonymous" in My Account. If I use GoogleCL for the same account from two different computers, I don't know how I'd be able to tell the difference if I wanted to revoke access for one of them.

No, unfortunately I did not write this post offline and then submit it with GoogleCL. Maybe next time. Now I wonder if I can get it to run on Android.