another year, another bot

My 2015 has been filled with long and arduous international and domestic travel, plenty of family business, a cross-country move, and a deferred to-do list a mile high. I haven't been keeping regular hours since the beginning of the year, so I'm glad that the dust has settled and I have a little free time to learn and work on new things again.

I finally made a version of the bang bot that is police-oriented (and less pervy-sounding). The Washington Post's fatal police shootings database is reliably updated, and it's on GitHub so it was much easier for me to get raw data from (otherwise I would have to do some manual scraping-- not hard, just time-consuming).

The bot's checking mechanism was the simplest one I could come up with:

  1. Pull the latest .csv file on WaPo's GitHub
  2. Check if the number of lines is greater than that of the last stored version, and
  3. Tweet that number if it is.

The implementation's still got a few kinks (apparently you can't use Twitter's API to tweet the same thing twice, which may be a problem soon), but my previous experience with Node made building it fairly easy.

It was the deployment that gave me some trouble. I used Heroku to run my app this time, and it took me a while to realize that the reason my files weren't properly saving was because Heroku workers don't do data persistence (derp). So I had to pick a data storage addon. I would have liked to use MongoDB to make the app JS end-to-end, but it wasn't free. And I'm pretty familiar with Postgres after having used it for a class earlier this year, but I ultimately went with Redis, another NoSQL choice that's fast and has an easy-to-use Node.js client. I hear that relational databases are the way to go if you have a data structure that's even remotely complicated, but I only had to store two things.

A screenshot of callback hell. Can you tell I'm using Node?

Besides storage, I still haven't quite gotten the hang of JavaScript's asynchronous processes yet (please refer to the callback hell in the screenshot above). I tried to keep things readable, but I think I'm gonna just bite the bullet next time and use async.

The reason I started this project was to get an update on Twitter whenever WaPo confirms another death by police shooting, and I'm glad I reached that goal. But seeing the bot tweet for the first time on production was the first time I had ever felt sad about seeing something I built actually work.

Now that I'm getting into full-stack development, I'm exploring what else I can do with open data. My final project for the back-end class that I took this spring used data made available by San Francisco's Human Services Agency. Similar municipal open data initiatives are happening in New York City, which is where I'm based now. I hope to build bigger and more useful applications with some of this information in the future.

time flies

Hi, my job has (surprise!) swallowed up most of the last few months. But now that the most intense part of this development sprint is over for me, I need to take stock of the things I'd been working on before I stopped having a life.

dev projects

  • bang bot - Twitter bot that tweets whenever someone is killed by a gun. I only barely got this Node app started over the summer.
  • sfpd and property assessment map - One of many projects for the Anti-Eviction Mapping Project. It's been in and out of development hell since the summer.
  • bunny - I wrote this absurdist text adventure while learning about Python last year. I've been meaning to finish it and turn it into a tiny but full-fledged Node+jQuery web app so people can play it outside of the command line.
  • [catmap - Essentially abandoned after I found out that someone took this basic idea and made something better, maybe even creepier.]

art projects

  • electronic music - After the past couple of years, I am finally comfortable assembling minimal stuff in Ableton Live. But the whole point was to make the music I hear in my head, and I can't do that yet, so I have to keep going.
  • novel - This is a long-running inside joke that I'd almost gotten done in a blast a few years ago. Never managed to write the last five or so chapters, and might as well. I have a surprising number of friends who want to see it published (at least as an e-book), but the manuscript is typewritten, so they'll have to OCR it first.
  • another collaboration with joel - When we first started dating, we made a gif site together. We've been meaning to do more of that, especially now that we both know more about computers and now that he's finally ventured into the SF/Oakland art scene. We're currently busy enough trying to get hitched, though.

A lot of traveling and personal events are happening over the next year, so I hope I manage to finish at least some of the things on this list.

armies of robots of lines of code

Lately I've been building a bot that tweets whenever a goal is scored in the World Cup. I've always wanted to make a twitter bot, and was pleasantly surprised at how easy it was.

The twit library for Node.js got me tweeting within minutes. I hooked it up to this super useful World Cup JSON API, wrote a hilariously complicated function to determine how many Os to tweet in the word "GOL" for each team, pushed it up to a tiny Heroku worker, and BAM! Now a silly idea I tweeted last month has become reality, and I learned a few things out of it.

Working on the gol bot has been fun, but as someone who tries to use technology to do good, I'd like to tackle more useful and thought-provoking work with this newfound knowledge. For example, I was thinking of making a bot that tweets the word "Bang" as often as someone in the United States is killed by a gun, which is more than once an hour. I could also make bots that do the same for other countries and have a site that live-updates all of them side-by-side. Other ideas are welcome.

21st century innocence


Anamanaguchi - Endless Fantasy

Most people to whom I show the above video think it is ridiculous. Of course it's ridiculous. It's some weirdly Americanized kawaii iOS 7 vision of the 90s as reimagined by someone who grew up in the 90s but is now technically an adult. It's cheesy and dumb, like drunk hipsters who jump in front of an exploding firework just to take a badly-focused Instagram photo that will get them 50 likes. And what the hell kind of entitled jerks spent a bunch of money to send a piece of pizza to space?

In spite of myself, I dig this song. As for the video, I obviously don't think anyone should make that their life all the time. But it reminds me of the one kind of fun I can no longer have: the carefree joy I felt when I still believed I could do anything, when I was still blissfully ignorant of how the world worked.

I'm glad I know better now. But sometimes, on the days so tiring that I need to use my last shred of faith in humanity just to go outside, I'd give anything to get that innocence back.

method to the madness

"As a project drags on, my git commit messages get less and less informative."

Much of my programming experience has been gained through trial and error. For a teenager doing raw frontend development, this is not a totally terrible strategy. In fact, if you have the patience to nudge a box a few pixels over and over then eyeball it in different platforms and sizes to see what looks best, being a frontend dev might be for you.

However, for things that can't be eyeballed, and with a large framework like Ember, that kind of manual plugging takes forever. For a while, I was honestly just Googling my problems and trying everything that Stack Overflow was suggesting one by one. But if it's 3 a.m. and you barely have enough functioning brain cells to maintain your sanity, much less keep track of which things worked, you can't do this.

So what I've started to do is keep a dev diary.

Writing entries for a dev diary is like an extended version of writing explanations for a problem set answer, where you also include everything that went wrong. I write down:

  1. The desired goal, or the problem I want to solve
  2. A description of the trouble I'm running into (this might be the same as step 1)
  3. My theory on why I'm having trouble
  4. A course of action I'm going to try based on what I think is going wrong
  5. Whether that course of action worked
  6. (Repeat steps 3-5, with adjustments as necessary, until problem is solved or goal is achieved)
  7. A summary of what I did right in TL;DR form that I usually put at the beginning of the entry

Much like a real diary, this helps me organize my thoughts so I can sleep at night without trying to keep anything in my head, then return to the task in the morning without having forgotten what I've already done.

There's also the added bonus of having a beautiful written record of the knowledge I've gained, instead of a bunch of uninformative git commits called "Still broken", "Less broken", "Fixed this, now trying to fix this other thing", and "OMG WTF". I highly recommend it.

silicon valley blues

I try not to read articles about the tech industry. I live in San Francisco and have heard it all: techies are insensitive aspie jerks, techies are displacing communities, techies are sexist, techies are racist, techies are ageist, techies are gleefully blind to the problems of real life.

To some degree, all of these epithets are true. These are some of the many reasons why I stayed away from the industry for so long. I don't like being constantly reminded of how terrible it is because it gets me down, and I can't make anything useful when I'm deeply depressed with the world around me. I can't preserve my wide-eyed wonder at what computers can do.

I hate the tech industry. I am also knee-deep in it because I can see technology's potential to help people. I share the dream of a bunch of acid-addled Bay Area brains in the 1960s who envisioned how computers could change the course of history. I think that the term "hacker" should not confer elite status, but imply a humble willingness to keep learning, to keep "hacking" away at a problem until you've developed the tools to solve it.

I believe that the problems technology can solve are not just problems of the privileged. The most impactful technologies do not include hookup apps geared towards well-heeled 23-year-olds with iPhones. But that's what gets funded, and it's frustrating. This is not what we should be doing with computers.