Our ‘Adventures in Tech’ series has seen me take my first steps in using a Linux distro, Fiona making weaving robots and George try to code a bot that will render copywriters obsolete. It’s been a mix of programming, problem solving and tinkering with hardware.
This time I’ll be sharing with you what happened when I tried to pull together my own tribute to the Horse_ebooks Twitterbot account using a Raspberry Pi.
My Raspberry Pi
The Raspberry Pi is a stripped down minicomputer. It’s tiny, comes without a case, and has peripheral ports and GPIO pins. You can install any Linux operating system on the Raspberry Pi that can cope with its resource limitations. But the Pi does have its own OS: Raspbian.
Picking a project
Following the first few chapters of the second edition of Raspberry Pi For Dummies, by Sean McManus and Mike Cook, I started to think about what project I wanted to do with my Pi. I went and talked to Dave Griffiths of FoAM about the kinds of things I might be able to get up to..
My first idea was to use the Pi to control a bubble machine. Bubbles being blown every time certain Twitter accounts tweeted
But I was a little nervous about working with the GPIO (general-purpose input/output) pins and controlling hardware. I decided to remain purely code based for the moment.
In between getting a handle on the Pi, I had also been learning some of the coding language for Python. At least the legacy version from Python 2.
I still wanted to do something involving Twitter and I wondered if I could use Python to help me achieve this.
Then I remembered the Twitterbot Horse_ebooks.
What’s a Twitterbot?
A Twitterbot is a program that runs a Twitter account, automating all or some aspects of the account’s activities. From composing and sending Tweets to following other Twitter accounts – much of what you do on Twitter can be automated with the right lines of code.
Like Horse_ebooks was in its early days.
And Horse_ebooks is…?
Horse_ebooks is now a defunct Twitterbot, but its oddball Tweets are still available for perusal. Its origins are the stuff of internet legend.
The bot would post tweets like:
Do not attempt to carry crockery or glassware upon a
— Horse ebooks (@Horse_ebooks) December 17, 2010
I enjoyed the unusual Tweets that Horse_ebooks use to dole out, so I wanted to see if I could create a Twitterbot that was just as random.
Not coding from scratch
I decided early on that I was not going to create the bot from scratch for my first Twitterbot project. Much like a novice mechanic or car engineer, learning about a combustion engine for the first time – it tends to help seeing a working engine first before trying to design your own.
Looking for ways and means to build a “Horse_ebooks Twitterbot in Python” via Google, I found this guide by Lauren Orsini: Five Steps To Build Your Own Random Non-Sequitur Twitter Bot. The guide was for how to configure and set up a bot using Tom Meagher’s Heroku_ebooks GitHub repository.
Later, after configuring my bot, I discovered that someone had taken Heroku_ebooks, tweaked it for their own ends and got it running on a Pi. But they hadn’t written detailed instructions on how to do it, missing out some crucial steps. Plus it didn’t do what I wanted it to do, which was for it to be directly based on my personal Twitter account.
Heroku_ebooks – how its code works
This bot works by pulling in Tweets from at least one existing Twitter account, chewing them up – sometimes adding random extra words – then posting them. There’s a “sensical” rating you can adjust, depending on how off-the-wall you want it to sound.
You can also adjust whether it always posts when it’s run and it won’t send a Tweet if the one it’s created is too similar to the original.
Simple configuration
The bot had one file that you were meant to edit in order to make it your bot. It was local_settings.py. In this file, I:
- Added the bot’s Twitter account handle
- Put in all the information (lots of tokens and secret keys) it needed be allowed by Twitter to read Tweets and post to it
- Added Twitter accounts I wanted to pull Tweets from
- Changed the sensical rating to 3 (so somewhat coherent)
- Adjusted its chances of Tweeting so that it would always try to post, so long as the Tweet wasn’t the same as the original
Going off the beaten track
The thing about the Heroku_ebooks code is that the bot is originally meant to run from Heroku, a cloud based platform for building and hosting apps. There are a lot of things you don’t have to configure or setup if you’re running the code in Heroku, because the platform does it for you.
I didn’t realise how much Heroku provided until I started trying to run the bot’s code on my Pi.
Problem #1: Why isn’t it running?
I didn’t originally realise that Heroku was responsible for running various dependencies that the bot needed. Having downloaded the code files for the bot and altered them according to Lauren Orsini’s guide, I tried running the bot from Terminal for the first time.
This is the message I got back in Terminal when I tried to run the bot:
Traceback (most recent call last):
File "ebooks.py", line 4 in <module>
Import twitter
ImportError: No module named twitter
After talking to Dave Griffiths, I understood that I needed a Twitter module, like Tweepy, to get things going.
A module, in Python, is a file that contains lines of code that can be imported from a file to work as part of a larger script. They are key to getting longer programs to work. In the case of Heroku_ebooks, the script that actively runs the bot (ebooks.py) asks for the following modules:
import random
import re
import sys
import twitter
import markov
Solution: install PIP and python-twitter
Throughout much of getting this bot working, I was talking to Dave, several friends and just generally on Twitter about what I was doing. Sourcing ideas and solutions from actual developers and fellow hobbyists.
Tunnocks Teasnake made the following suggestions:
@ek6891 hmmm – are you using pip to install the twitter package?
— Tunnocks Teasnake (@badbonobo) June 2, 2016
@ek6891 yeah, maybe. I always use the python-twitter package, it’s nice.
— Tunnocks Teasnake (@badbonobo) June 2, 2016
PIP lets you install Python packages, modules and so on, that aren’t found in Raspbian’s default software archives. Modules like python-twitter aren’t in the default archives.
My bot’s first Tweet
Modules sorted, I ran ebookek for the first time and this was its first Tweet:
So twohours The sevennipples,
— Emily in ebooks (@ebookek) June 2, 2016
Nipples aside, the bot worked, but: why was there no spacing between the last two words?
And why was I getting a long message about SSL errors in Terminal?
Problem #2: Why is the spacing weird?
The spacing issue seemed the simplest to investigate, so I researched that first. I wasn’t the only one to have this issue, and found information around it on Heroku_ebooks’ GitHub pages. The problem was with a line of code in markov.py.
However, the change to this line of code wasn’t in the downloadable files for the bot, so I had to manually make the change in my own code. It was just a tiny addition to line 71:
sentence += res[-2] + res[-1]
Became:
sentence += res[-2] + " " + res[-1]
We’ll get Cas Cue discomforting trail
— Emily in ebooks (@ebookek) June 2, 2016
But I was still getting error messages about SSL in Terminal.
Problem #3: The disconcerting SSL error
While the bot did function despite this error, it wasn’t something I wanted to leave hanging open, as reporting the error was taking longer than posting the Tweet itself.
Solution: Switch from Python 2.7.3 to Python 2.7.9
Reading up on the error, I found out that if I was using Python 2.7.9 or newer, then I would not have this problem.
Simple, I thought: I’ll just update Python 2.7.3 to 2.7.9.
After reading up on how to compile an installation of Python 2.7.9, because it wasn’t available in the Raspbian archives, I installed Python 2.7.9. (I wasn’t switching to Python 3 and above as the bot’s code would need to be rewritten in order to work with Python 3.)
It killed the bot, fudged the Pi’s ability to work with Python and borked the Pi.
Problem #4: I killed my Pi
Happening upon this thread on reddit about updating Python on the Pi, a redditor had remarked:
I heard that installing 2.7.9 over 2.7.3 causes the OS to explode as some OS dependencies require 2.7.3?
I can confirm this is the case.
Solution: Upgrade Raspbian OS
That same reddit thread went on to describe how it was the Raspbian Wheezy OS that was the problem. Raspbian has had numerous iterations and the version I was working with was Wheezy, which came with as default: Python 2.7.3.
I backed up my bot’s files off the Pi and then looked into how to fix it. The simplest solution was upgrading to Raspbian Jessie, which comes with Python 2.7.9 as default.
However upgrading like that is no simple matter when there’s already an OS present. And for nostalgic reasons, I wanted to keep Wheezy around. So I got myself a new mini SD card, which is what the Pi uses to run its OS from and for most of its file storage.
I reinstalled Wheezy on my existing card. Then, using my PC, I made a fresh Raspbian Jessie image on the new mini SD card, following instructions from Raspberry Pi For Dummies.
(An image, or system image, is a completely clean version of an operating system in its default state.)
The bot was back and error free
Getting the new OS up and running with everything needed to run the bot, and the bot itself, took less than two hours. This was thanks to me keeping developer notes about what I had been doing.
Soon, the bot was able to Tweet again and no longer had the SSL error. There was just one more hurdle to surmount.
Problem #5: How could I automate the bot?
It was great that ebookek was Tweeting about weird and wonderful things like:
Again with the office .
— Emily in ebooks (@ebookek) June 3, 2016
But I had to manually tell the bot to run from Terminal each time.
There had to be a way to make the bot Tweet automatically, at set intervals.
Solution: Set up a shell script and run it from crontab
Dave Griffiths had mentioned to me at one point that I would be able to use crontab to automate the bot.
In Linux distributions, crontab is a means of scheduling the system to run specific tasks at specific times. You control it through a text editor like nano from Terminal.
It was difficult to find examples of using crontab to get a program to run a script. Discussions around it tended to be by people who already knew how to do these things and skirted around the basics.
Talking with Robin Duckett of Diodes Ltd, I was able to finally understand how I could run a shell script through crontab.
A shell script is like a series of commands you would normally type into the command line and execute from there. Only the commands are in a text file that’s been made to run in a specific way, and you use something like crontab to activate the file at set intervals.
So this shell script, written in a text editor and using an absolute path:
#!/bin/bash
python ~/awesome_bot/ebooks.py
Could be given a file name like run_bot_run.sh and then configured to run past every half hour in crontab by telling crontab something like:
30 */1 * * * ~/awesome_bot/run_bot_run.sh
When I finally configured what I needed to automate my Twitterbot, it worked. Now, when I leave my Raspberry Pi on, it will run the bot’s code in the background and try to Tweet half-past every hour.
Moving on from ebookek v1
I am slowly reviewing sections of code from the bot and tinkering with it as I go in order to make it run more efficiently or have the quirks I want it to have. Becoming more and more familiar with the bot is making me want to work more from scratch and code my own one.
And after all this I definitely want to revisit my bubble machine idea.
Want to make your own ebookek for the Raspberry Pi?
I’ve put together a step-by-step guide for a random non-sequitur Twitterbot for the Raspberry Pi and you can find it here.
