Building the @brokenlifts Twitter bot

The BVG public transport network in Berlin is reasonably accessible for anyone with mobility issues - an increasing number of the trains have level access and wide through-aisles, and about 75% of the stations are accessible via street-level lifts.

That assumes that the lifts are working, of course. A lift is a complex mechanism with lots of moving parts, and BVG lifts break on a depressingly-regular basis. Knowing if you can get on and/or off the network is fairly fundamental if you have accessibility issues, and while this information is available if you can find it on the BVG website, having to check a website feels a bit 2010.

Wouldn't it be more convenient if the information could be pushed to you over more immediate channels?

That gave me the idea of creating a Twitter bot to do the pushing bit.

Raw information about lift outages as a usable data feed is available, albeit on the unofficial brokenlifts.org site. That seems to have been built at a hackday about 10 years ago, and not really maintained since. It works, but it's not secure, and it's web-only. On the other hand, the visual interface is quite nice, and it does provide an RSS feed, so that's where my setup starts.

At a high level, I grab the raw data on a regular basis, figure out which lifts are newly-broken and which are fixed, and then post that information as a series of tweets. Here's how things work on a more technical level.

There's basically four moving parts:

• a web service built on the Next.js framework and hosted on Vercel
• a MongoDb database, hosted on their Atlas service
• a cron job which lives at easycron.com
• the brokenlifts.org RSS feed

The whole process is a small web service that is triggered by a cron job every 15 minutes between 0600 and 2400 each day. The cron calls an API endpoint over HTTPS - this triggers a function to grab the RSS feed from the brokenlifts.org site, and then parse it into plain 'ole Typescript objects.

When each retrieval process runs, the previous list of broken lifts has already been stored as entries in a MongoDB database - each lift has a unique identifying number, so it's possible to track them independently of their station location.

The starting point of the tweet process is doing a diff between the lift IDs in the database and those contained in the feed. If there's a database entry for the lift, but it's not contained in the feed, then the assumption is that the list is back in service.

If a lift doesn't exist in the database, but has now shown up in the feed - that's one that's newly broken. And if there's both a database entry AND a RSS feed item, then we can assume that BVG hasn't got around to fixing it yet.

Having figured out what's fixed and what's newly broken, that information can be grabbed from the database and used to populate the tweets. The RSS feed provides the station name and a description of the lift's location, so creating the tweet content is just a case of concatenating some strings.There's also a timestamp saved in the database to record the time that the process runs, so this can be used for the "since <time>" part of the tweet.

Once the array of tweet content is set up, it's iterated across and sent to the Twitter API.

The whole service is built using the Next.js framework, and deployed directly from Github to Vercel using their built-in CD functionality. All the necessary config details - access tokens, Twitter API keys and so on - are stored as Vercel environment variables.

Vercel and Next.js play together extremely well, to the point where the ease of deploying the service into "live" makes the overhead of an entire framework worthwhile. Strictly speaking half of the Next stack isn't needed because there's no frontend component - but the marginal increase in size of the deployment is more than worth it when deployment is literally as simple as pushing changes to a Git branch.

This is an MVP, and there's some future improvements that I'll get around to at some point>

• consuming the data directly from BVG, although this is going to require figuring out the process to get them to issue the API keys required
• adding a Telegram and WhatsApp bot to the mix, so that the information can be pushed across arguably more extensively-adopted channels
• figuring out a way of creating "commute-friendly" messages so you can get timely updates about the stations (or even specific lifts) that are relevant to your journey
• replacing the MongoDb side of things with something a bit more lightweight - not that Mongo is complicated, but an entire hosted cluster does feel a bit over the top for this use case
• stripping down the server side of things and reimplementing it in a minimalist framework like AdonisJS. That's not going to really affect the functionality, but would be an interesting technical exercise, and could open up the possibility of hosting somewhere other than Vercel.

The Twitter account is @berlinlifts, and the source is at Github: https://github.com/timd/brokenlifts