Sending Mail with Python and Googleapi for Emacs

So I love Emacs. Why I love it would be the subject of another blog post, but as I love it I want to do everything from it, including sending and reading mail as it is very convenient for me. Emacs can automate a lot of tasks for me.

Anyway so mu4e does a really good job of indexing and reading mails in emacs. It has a good compose feature also and it leaves receiving and sending mail up to the mail client of user’s choice.

For receiving mail I’ve been using offlineimap which also does a fairly good job. It’s not fussy and works as expected. The problem I faced is with sending mail. offlineimap can work with xoauth2 (xoauth2 is a variant of Oauth with some custom extensions by Google) and I have indeed retrieved and stored tokens for it which it can use to fetch my mail. And earlier I was using Emacs’s smtpmail-send-it to send mail. The problem occurs with the auth-source, the backend Emacs uses to authenticate with any server.

Now I did find this auth-source-xoauth2 and it works fine actually, but…and it’s a very annoying but, sometimes smtpmail-send-it does not authenticate and gives me a 334 error (authentication required). And it happens often enough for me to get irked.

So the solution? One solution would be to use postfix to send mail and actually I did install postfix to see if I could configure Emacs for it, but it proved to be overkill for my use. It’s supposed to be a general purpose mail server and there are too many configurations and plugins for my simple use case.

So after a bunch of research (at which I’m fairly good) and hacking, I discovered that google has fairly good python library https://github.com/googleapis/google-api-python-client with a fairly extensive set of features. And sending mail from it is not that difficult and decided to write a simple server which can send mail instead of smtpmail-send-it.

What it does basically is given a mail message in a proper format with all correct fields and boundaries, it reads the credentials from a password store, stored as a JSON formatted string and initiates a service instance. From that we can send and receive mail messages, and we only need the send part of that.

I’ve made it a flask server so it only needs to read the credentials once from the store at startup, so I don’t need to feed them again after the timeout expires to emacs in case I need to send the mail later. You can take a look at it here.

OK, fine you ask, but who’ll format the message and attachments and insert proper fields? Why, Emacs of course! Instead of launching a network process, we simply send to gmailer instead. I’ve posted that function as a gist here.

Basically, mu4e and smtpmail do all the hard work and we steal the final product and dump it to the google client! Problem solved (for now, until the bugs come :-D). I’m sure there’ll be bugs, but I’ll keep maintaining the gists and if required gather them into a repo. Happy Emacs’ing!

Update

So after all this, one issue still niggled me. Currently I use offlineimap to retrieve mail from the IMAP servers and it authenticates via some parameters in .offlineimaprc. One can fetch passwords from pass or through GPG encrypted files directly but it can get annoying as you’ll have to fill in passphrase each time.

So what the gmailer does is ask for credentials once and then they’re with the app with no way to retrieve them. Of course you have to trust your python interpreter and the OS that the memory cannot be snooped, but we’ll assume that that’s the case.

Now offlineimap is run from the command line and while the docs say that it can be used as a library, I found several problems with that.

    oi = OfflineImap()
              oi.run()

Now, the problem that arises is that you can only call this once. They’ve used some weird ConstProxy class to make sure that all the variables are set only once, so that the class can be instantiated only once also and the parameters are set only from either the command line or the config file. That’s insane! Though I think the restrictions are there so that multiple processes don’t try to write to same maildir.

Also did I mention that the entire thing is in python2? While there’s a python3 project underway, it’s still far from being production ready. So I can either hack the class or let it go for now, and I think I’ll just let things be. Maybe one of these days I’ll get annoyed and hack it.