The Javascript30 Day 1: adding instruments to the Drum Machine

I’ve long enjoyed his blog posts, so I was really excited for Wes Bos’s ‘Javascript 30’: 30 short tutorials on building cool stuff in vanilla JS. Since I’m not a real frontend developer, the lessons have included all kinds off stuff that was news to me, and beyond that it’s just fun to have fun coding.

I’m going to be posting extension ideas for the first few lessons, with an eye to adding each one to a portfolio. Day 1 is a drum machine that kicks off sound effects from keyboard input, and restarts the sound if you trigger it more than once.

screen shot of Wes's original drum machine page.

All I’ll do for this first project as extension is:
* change the background image to a local file
* add an instrument on a new keyboard button

New background!

To start with, I didn’t like a web request going out for a background image, and since the image was something grabbed from imgur, I didn’t really know its copyright status. No good for a portfolio, so I looked for a good online art archive that was releasing open-source images: turns out the Getty Museum has a great search tool for their archives

Okay, now I’ve downloaded the image I want to use, how do I place it as the background? I looked around the sample files and found this line in the CSS:

background:url(http://i.imgur.com/b9r5sEL.jpg) bottom center;

it took some digging to figure out the right syntax to link to a local image, turned out it was:

background: url("./images/dance-party-lo.jpg") bottom center;

note the double quotes and a ./ before the path. There are so few files in this thing it would be fine not to use an /images folder but that just felt too sloppy.

Okay, now that the image is replaced, time to add an instrument

adding a button

It’s always better to make changes incrementally. In this case, I’ll do two steps:
1. add a new keyboard button that plays the same instrument as another key
2. add a new sound effect for that button

Wes calls out in the tutorial you can use (keycode.info)[http://keycode.info/] to find every keyboard button’s code, and it seemed logical to add semicolon since all the other home-row buttons are there. I think the reason Wes didn’t do this originally is internationally issues: not every keyboard has ; to the right of L.

once I had the new keyboard code, there was a bunch of copy-paste as I added the button in the html. It worked first try!

new sound effect

Its kind of daunting trying to figure out what audio formats are supported, but all the other sound effects were .wav so I figured that would work. The Freesound Project makes its files available as .wavs, so I grabbed an appropriate ‘airhorn’ noise.

It wasn’t as confusing adding a new local sound, since the syntax was the same as the ones Wes already had. The final result looks nice:

image of new drum machine

Ideas for more stuff

  1. I tried making the sound play back at different pitches at each time. There’s even a call for ‘playback speed’ but it didn’t work the way I thought it was. Ended up dropping this
  2. some different arrangements would be cool
  3. change the look of a button the whole time the sound is playing

The shopify exporter!

When a friend asked “is there a way I can automatically backup the important info from my Shopify store?” I figured there had to be an easy way, and sure enough there were a ton of available plugins/services to do just that. Since they’d asked me to do it myself I’d much rather write code than buy a thing and try and configure it. So I wrote a node app to grab it all and send it to S3. Behold the Shopify exporter!

Lessons learned:

  1. Heroku automatically lets you schedule node jobs! I thought I had to write all that code myself! Look out Twitter bots! 😛
  2. There really wasn’t a simple enough walkthrough of s3 uploads for me. They were all covering how to upload user-submitted data, which adds a whole front-end piece I wasn’t worried about.
  3. From the s3 dashboard there’s a button to upload files directly. A parenthetical warns you “Under 5 TB”. lol

Upgrade your security in 10 minutes

The text of a 10-minute talk first given to donut-js in November 2016. Here’s a .pdf of the presentation

Where do we start?

Normally, when we talk about security, we have to start with a discussion of a ‘threat model’

‘Threat Model’?

A threat model helps us decide which security measures to use, taking into account:

  • likelihood of attack
  • cost of a successful attack
  • reduction of likelihood given a countermeasure
  • cost of a countermeasure

We won’t be doing that in 10 minutes!

That’s okay, because the countermeasures here are all cheap!

  • Fast setup
  • no $$$ cost
  • no added hassle to doing ‘stuff’ online

Top 5 things to do

  1. Secure your email
  2. Effective Ad-Blocking
  3. (Windows) Automatic updates!
  4. Signal for secure conversations
  5. Secure your other services

1. Use ‘2-factor Authentication’ for email

Why: if someone accesses your email, they can probably get access to everything else

How: for Gmail, you’ll enable 2-factor Authentication, and install the ‘authenticator’ app on your phone.

2. Effective Ad-Blocking

Why: Beyond sites becoming readable, ads are the primary vector for privacy and security problems

How: Switch to Chrome or Firefox and install uBlock Origin

3. (Windows) Update and use Microsoft Antivirus

Why: Most attacks on computer secure target Windows PC’s that are not fully up-to-date

How: A great guide is at decentsecurity.com, but in general:
* upgrade to windows 7 or higher
* use included “Windows Defender”
* enable automatic updates

4. Signal for secure conversations

Why: In a conversation via email or social media, all your security is worthless if the other person has been hacked

How: Search your app store for Signal and use it when you want your conversation to be private

5. 2-factor authorization for other services

Why: Once your email is secure, time to make sure no one can access your Facebook, Twitter, LinkedIn, etc without your permission!

How: A few services only have 2-factor login, including Snapchat and Squarecash, for others Google ‘enable 2 factor authentication on [social network]’

What’s next?

  • check out the EFF guides for different security needs with much better explanation of the how’s and why’s of each of these steps
  • Toby offers security consults to do some personalized threat modeling. These consultations are free if you’re a political activist or fear being targeted by hate groups.

Science Hack Day pt. 2: The stepper motor

My first post about Science Hack Day covered the fits and starts of working on the hackathon day and covers why the raspberry Pi turned out to be way less than ideal. This post is drawn from the last part of the hackathon, when we were driving the stepper, and what I learned about accuracy, and timing, in that experimentation

caveat: I am 100% not a mechanical engineer

Dozens of concerns were mentioned to me about the design of this tool, and I found a few others with a little googling, that implied we might have difficulty while trying to ‘drive’ the tool at the right speed with sufficient accuracy. None of these turned out to be actual problems, but I’m so ignorant on the subject I might not even have the right terms as I try to describe them. Sorry!

Okay what does this device even look like?

The process we’re talking about is stereotactic surgery, where bones are used to lock in a coordinate system for finding an exact point in soft tissue. The term could refer to any kind of surgical procedure, but usually we’re talking about the brain, since the skull is such a convenient ‘hard point’ to start your coordinate system.

To perform surgeries like this, they use a tool like this one:

The instrument. Image taken from lab supply company http://www.stoeltingco.com/
The instrument. Image taken from lab supply company http://www.stoeltingco.com/

The part of this that really matters are the two ‘knobs’ you see at the top left. The scientist twists these to move the probe. If you’ve ever worked with a 3d printer, this probably looks familiar. You’re looking at a hand-cranked 3d control system.

The upgraded version of this tool looks like this:

There are a lot of ways to 'computerize' this tool, but I chose this one since the conversion is more obvious. ibid
There are a lot of ways to ‘computerize’ this tool, but I chose this one since the conversion is more obvious. ibid

This version should look very familiar to someone who has worked with either a 3d printer or a CNC: the hand cranks are now motors, and a controller for those motors is hooked up to PC control.

The paper that Daniela found was about building a full 3d tool with open source hardware at a significant cost saving. She would find such a tool useful someday, but for the moment she just wanted one that moved on the Z-axis automatically, with manual controls for the X and Y.

So this whole goal is to build something that uses a stepper motor to turn a single knob.

Again, I had the advantage of a machinists’ work before the hack day to build the mechanical connection between motor and tool.

My concerns about the motor

  • Accuracy
    The probe needs to move ‘in’ by a total of 2mm over the course of a minute. Since we’re doing this in a surgery, it’s prooooooobably not good to end up moving 2.1mm. The ‘stepper’ part of stepper motor means it moves forward in discrete steps. My big fear was that the right distance would be between steps. If the system moved 3mm in 17 steps, there wouldn’t be a right number of steps to go exactly 2mm. Thankfully, since we were using a ‘geared’ stepper motor that steps down the speed of turning with a gearbox, this wasn’t a problem. A full rotation of the motor took a few thousand steps. Plenty of resolution.

  • Smoothness
    Again taking the example where the stepper moves the whole length over a dozen or so steps, this motion might look smooth to the naked eye, but microscopically they’d look like separate thrusts, which might not look good. Again, the resolution of steps resolved this.

  • Calibration
    I knew there were ways to look at the machines involved and calculate the expected distance of movement but 1) I didn’t know how to do that and 2) it sounded hard so I came up with the clever idea of just making the tool move, measuring the distance, and ‘calibrating’ to find the right number of steps to go 2 mm. At first I thought it would be tough to measure these tiny distances, but the version of the tool that Daniela brought had little dial readout of the distance of movement. In the end this was super easy!

  • Speed and sync
    The stepper driver in use probably had a spec available somewhere of the maximum rate it could accept instructions. The controller (raspberry Pi and/or Arduino) also have a maximum accuracy with which they can send signals out. One request from Daniela was that the probe move faster coming back out than it did going in, and I was worried that this higher speed might end up with signals ‘lost’. If you send ‘step’ signals faster than the motor can actually get them, you could miss steps and end up not moving the full distance.

I tested this extensively by throwing a ton of steps at the system at speeds much higher than required. Even with the Pi, which should have much more difficulty sending signals at high frequency, had no trouble. This wasn’t that surprising, after all we were going super slowly.

  • heat
    Stepper motors eat up a lot of current. Both the motors themselves and the stepper driver get really hot
See that big metal bit toward the type? It's a rather large heatsink to dissipate the heat. Image from tronixstuff.com
See that big metal bit toward the top? It’s a rather large heatsink to keep this part cool. Image from tronixstuff.com

Where a lot of the other problems I came up with were obviated by the slow speed and high number of steps, energy usage and concomitant heat buildup are actually worse for a more accurate tool.

Stepper motors can accept a big range of voltage. The more voltage, the stronger the grip of their magnets and the more accurate each step would be. That meant we ran our stepper at its max allowed voltage. We actually killed two of the tiny ‘cute’ stepper motor drivers in our testing.

We ended up using a version of the driver above with a huge heatsink. The motor itself also got pretty hot hot. Similar rigs to do 3d printing have cooling fans mounted on the frame to cool the motor itself. But I calculated that on the hack day we moved the probe more distance, at a higher speed, than it would move over a year of use. The system didn’t melt, we were okay.

stuff other people mentioned

  • free-wheeling
    If you turn off all current to the coils of a stepper motor, it should allow the wheel to turn freely. This would be super duper bad if the surgical probe just started moving around during a procedure. Normally this is handled by the hardware/software driving the stepper motor: giving it no instructions actually leaves one coil on holding the motor. I started to do a bunch of checks on this before I tried just turning off the power to my motor entirely. It turned out that the gearbox and the probe hardware presented way too much resistance for the thing to move on its own

  • reversing
    I’m not going to go to deep on this, but when you try to move back the way you came, there’s some ‘slack’ in the gears that needs to be taken up, losing you some steps. This difference proved too small to measure, and the probe gets ‘zeroed’ before every procedure anyway, so a problem can’t build up over a bunch of procedures.

  • timing from the Pi
    See my previous post: the Raspberry Pi was totally capable of running the driver.

what we learned

Optimization of a system, whether that system is electical, mechanical, or software, is only really meaningful after you have a working prototype. Problems like heat on the stepper motor driver are easily predicted and are worth handling early. Other problems should come up in trial runs, and if not, any time spent preventing them was largely wasted.

So build, and test! When you test, test a lot of stuff. Try your best to simulate uses more extreme than the real world should throw at them, and document what you find and notice.

the next and final post about science hack day will cover the control software I wrote for the arduino, and why an open-source stereotactic surgical might not be that great an idea.

Science Hack Day!!! -or- The Raspberry Pi, Stepper Motors, and Man’s Folly

Science hack day Portland was, without a doubt, my favorite hackathon event I’ve ever attended. So many things to love about it:

  • Slack channel invite a few days before to get on the same page/find teams
  • lots of real scientists attending, some bringing real problems
  • generous spread of hardware
  • not organized 100% by dudes
  • food was amazing

The Plan

I signed up thinking I might make some simple electronic whatnot or maybe some kind of ‘science-related’ complete project. Like a… weather station? Those are still a thing, right? Thanks to the slack channel set up beforehand, I found a scientist who was trying to get a real piece of lab equipment made, and even had an open-source project that they were basing their project on. So I’d be making a tool to carefully manipulate a probe. As it pushes into… an animal’s brain.

Science hack day had officially gotten real.

The version that Daniela Saderi wanted for her lab wasn’t going to grind bone like the open-source original, it was just going to handle a probe, moving only on one axis. The good news was that she already had an Electrical Engineer and Machinist on the project, otherwise the coupling between the probe and the stepper motor would have taken me the whole weekend.

So she was bringing all the necessary parts, and a stepper motor connected to a probe manipulator, all I needed to do was design some software to control the thing.

The Huge Mistake

I decided to try and control the system with a Raspberry Pi. This was a huge mistake for reasons I’ll get into, but I picked it for what seemed like good reasons:

  1. Mozilla had generously donated 35 of the buggers to the event.
  2. the user (Daniela) wanted a lot of customization on the tool. Adjustable speed, depth, pausing and restarting, all needed to be controllable.

The question I got asked throughout was ‘why not use the Arduino?’ And I did my usual scoff at this kind of question: there’s always a better tool out there, just let me use what I have dammit!

But really, motivation was the complexity of the interface Daniela wanted. I knew that displays and button inputs existed for the Arduino, but I didn’t really know much about them, I didn’t have the hardware necessarily, and I felt like the programming would be more complex.

Prompting the user to enter a speed and depth on a command line interface is easy for me, but making some little LCD display data and handling button input both sounded… hard.

Arduino and Raspberry Pi side by side
The Raspberry Pi is a complete PC (below) next to an Arduino (above) which, while programmable, isn’t much smarter than the electronics in your microwave’s keypad.

Oh and also: while Johnny-Five lets you write in Javascript for the Arduino it doesn’t actually… work the way you think it does: it requires the Arduino be hooked to the PC. To write code that just runs on the Arduino, you gotta write C.

I’m not that terrified of swapping programming languages? But it just seemed fun to be able to get the thing working in Javascript! And Johnny-Five can run directly on the Raspberry Pi (of course it can, it’s a PC!). I even plugged Johnny-five on the Pi to a few people in Slack in the days leading up to the event.

Why that was a mistake

a PC is a PC

In the end, I never got Johnny-Five to work on the Pi, and ended up having to write the Raspberry Pi controls in Python. I’d never written any Python, so that wasn’t super.

Also the thing about using a super-compact computer to be your computer is it just took so much time getting that fukkin’ PC to work. I was all excited about a tutorial that would let me get the Pi on WiFi and let me SSH over to it, but one of the steps was to update the OS. Of course, the OS update took half an hour, and with other little problems with hardware and video it was noon before I was even really working on my Pi.

Compared to setting up a full-blown new PC 3 hours isn’t terrible but it really shows why people prefer making simple projects on the Arduino.

just one letter separates ‘networking’ from ‘not working’

One thing that REALLY bit me: I followed a lovely tutorial on getting the Pi set up for SSH and Johnny-Five development, and the last few steps did not work right. First, I couldn’t get Mongo to serve the Pi at a hostname, and wrestled with that for a while.

After that, I couldn’t get the fileshare set up with the method they described, and ended up having to use… something else (I’m sorry, I’m sorry, I have no idea what it was and all the results I found were for Samba and smb and that did not work for me)

Once I had that done, I was able to remote into the Pi and write code from there. With the fileshare I could use Atom on my laptop and not be stuck with whatever weird IDE the Pi had. Great right? Not so much.

Since I didn’t get Samba set up right, I was accessing the Pi by its IP address, and this got reassigned by the network once or twice during the event. When this happened my attempts to load new code failed silently, and the sequence I was triggering would fail because it was running code from 10 minutes ago.

When I write code and it doesn’t work, network issues preventing the system from getting the latest version of the code are not my first guess about the cause. As a result this cost me an hour of debugging before I figured out the real problem.

Oh Johnny why you gotta do me like you do

After all that, possibly related: I couldn’t get Johnny-Five to work. I would get a few steps forward and then hit new roadblocks. The final version would run the code and throw me into a REPL. Was that it working? It sent the signals. But by throwing me into the REPL I couldn’t do console logging like I expected. It sent me some error messages that I… couldn’t understand.

johnny-five

Johnny-Five is an awesome project and it has a ton of rad uses. There’s also an “Espruino” hardware platform that takes Javascript direction. There was actually a presentation on this at the hack day that I missed on the Flasher.js sdk to do just this.

But don’t knock the Raspberry Pi

One thing I heard a few times, even read from some people who were writing hardware control software for the Pi, is that the Pi can’t control a stepper motor. This concern turned out to be quite overblown, I’ll explain:

A stepper motor is a wheel of magnets surrounded on four sides by electro magnets. As the magnets around the edge switch off and on, the wheel inside moves forward.

Stepper Motor animation, described in the paragraph above

for precision, they’re the dominant design, since at low speed it’s a bit like rolling a square wheel: you’re never unsure exactly how many times that wheel has turned.

If you’re curious: steppers aren’t the best at everything since they’re very power inefficient. They also require a ton of power to stay still (this wasn’t a problem for me since our motor had a gearbox that required force to move, so our probe wasn’t just wobbling around when the motor was off.)

All of this to say: controlling a stepper motor directly requires you to turn off and on these magnets around the edge. Obviously, the switching has to be really precise to ‘drop’ the wheel from one magnet and ‘catch’ it with the next.

the Raspberry Pi is a cut down PC, and it lacks a bunch of hardware involved with timing.

Here a sidenote: the chips on a normal PC that precisely control timing and allow for multiple tasks to work on in parallel while remaining in sync is called a ‘clock chip’ and is unrelated to telling time. My Father has worked for several decades making this exact part for a large semiconductor company.

Since the Pi can’t sync up its signals perfectly, it can’t handle controlling these magnets.

Okay, but these are really strong magnets, surely we’re not going to run them with power directly from the board, right? At the very least, you’d want a transistor to let a little bit of current from your controller direct a bunch more to the motor.

closeup photo of the pins on an Arduino
The General-Purpose Input/Output or GPIO pins can do a lot, but they can’t power your toaster oven

And here’s where we come to a basic thing with hardware hacking: powering LED’s or getting button input are all things that these controllers can do directly, but for more complex stuff you have a choice!
* design your own fancy circuit
* send simpler signals from your microcontroller to specialized hardware.

These are the famous “shields” for the Arduino. A lot of them aren’t necessarily required to do whatever, but they make it a ton easier to write the software to control whatever.

In this case, there exist stepper motor drivers that take very simple signals instead of requiring you to control each magnetic coil. Set up right, these ‘drivers’ will just take signals on 2 pins: a ‘step’ signal and a ‘direction’ signal, and handle all the fiddly “turn on this magnet, then turn on the next one right away, no, NOW”

By the by, we borrowed a Stepper Motor Driver from… someone… on the day of the event and I haven’t actually found another one that works just like this. I’ll figure it out eventually, but anyway!

Lessons learned

doing too much at once makes debugging a nightmare

At my job, where I’m using a library I’m very familiar with, I still rarely write more than a few lines of code without checking some debug output. Trim a string of a few characters, escape it for HTML, and then send it in a JSON object? I’ll often console.log() both the input and the escaped string before I move on to step 3.

By contrast, at 1pm on the hack day, I was SSH’ed into a Raspberry Pi running a Linux build I’d never used before, with an unfamiliar Javascript Library, to make the Pi control a stepper motor driver, which I’d first heard of the day before.

The first attempts to turn the motor was 27 lines of code. Not even counting the other tools, this was extreme.

After an hour of frustration, I disconnected the Pi from the motor, and went back to basics: trying to make the Pi blink some LED’s

It was doing the easiest possible task that revealed that Johnny-Five wasn’t working correctly, and that Python was.

In the end

The hackathon was a single day. By the end of the day I had the actual probe hardware moving the exact right distance in, pausing, and then moving back out. This wasn’t really a minimum viable product, since it needed to wait on input before going to the next stage. Still pretty cool to know the hardware setup and the Pi could manage such precise control, to move a probe just 2mm in 100 seconds!

If I get to it, I’ll write up how I went from that to the Arduino version in both hardware and software. For now you can check out my code for in Javascript, Python, and C

The Ever-Elusive Full-text search -or- Postgres magic

Search is, by it’s very nature, a complex problem. Just when you feel proud of yourself for getting everything pertinent stored in a DB, getting the right columns indexed, making find operations fast and efficient, someone’s going to ask ‘now I want to search the complete text of every post, and-
Okay, so that’s difficult, right?

And you’ll see some pretty kludgey solutions out there: using google indexing for your site, or just restricting searches to titles rather than full text. What are the key points in getting a really complete site search feature working?
1) Do we really need full-text search?

A lot of the wizardry we’re going to talk about feels like just that: wizardry. It’s easy to get caught up in algorithmic speed testing before thinking about what the business needs of this search really are.
If we’re letting a user search our e-commerce site, searching the text of reviews might be super unhelpful.

If we’re going to end up wanting to manipulate these results later, it would be a lot easier to just have ‘search’ go through manually selected tags than full text. E.g: when the user searches for ‘Ray Ban Sunglasses’ we want to offer him the cheap (and higher margin) substitutes. Rather than putting in some weird exception to our full-text search system, it would be a lot easier to just have search look at tags first, then we could tag whatever we really wanted without having to put our ‘thumb on the scale.’
2) do we really really need full text search?

trying to search, and even worse live-search, to work with a complex data set is a complex task, and we need to have user stories that justify the time and effort.
One question most ‘search’ user stories need to answer: would an extract or frequency table work equally well?

To be more specific, what about exporting a frequency table of words from the full text, removing the particles, and just suggesting the ‘matches’ from the top 10,000 words? What about an object containing frequently searched terms, with an array of post ID’s? All of these could be indexed ahead of time, with hourly re-indexing, for super fast performance, they all look a lot like full-text search, and they’re easier to manipulate.

You’re still here?
Okay, so let’s talk about how to make Postgres really sing through full-text search.
1) Get your indexes tuned

I came to programming from professional card playing, not a CS program, so I don’t always have the theoretical knowledge I’d like on every subject.
I do know this though: the speed with which you can search a set depends a lot on what you know about the set before hand
A simple add index for any column you’ll repeatedly query against is a really smart first step, but you should go back and mark every index for columns that you know are unique.
CREATE UNIQUE INDEX name ON table (column [, …]);
The reason why this helps is pretty easy to imagine: even with an index that shows you where you can find records of type ‘x’ is great, but it saves a ton of time if you know this listing is the only listing of its type in the index.
2) make what structure you can

I call this the ‘title subtitle optimization:’ if you’re concerned about search performance later, as much as possible give your stored information structure. That means if each post has a title and a subtitle, store them in separate columns rather than one text blob with a few tags.
The advantage of this approach mainly comes up in page ranking: it’s kind of a gimme that every match to a page title should be ranked above pages with the search term deep in the description.
Also: the shorter the blobs you’re searching in Postgres the faster your results will be.
3) your data has structure whether you know it or not

tl:dr version: use JSONB to store arbitrary structured data
It’s not actually possible to index (or for that matter compress) fully random data but good news: you’re a thinking machine not a bag o’ dice, you really can’t produce random output even if you wanted to.
JSONB lets you store structured data that can be indexed without knowing anything about its structure. and vastly improves performance
4) GIN indexing is your friend

I’d go so far as to call it your best friend. While it can index arbitrary structured data as JSONB, it’ll also let you index arbitrary text
CREATE INDEX idxgin ON api USING GIN (jdoc);

gin indexing was significantly improved in 9.4
Remember that you’ll need to format the query to tsvector to get to actually use this index. e.g.
NEW.tsv = to_tsvector(array_to_string(array( select json_array_elements(NEW.items)->>’name’ ),’ ‘));
I could not have written this article, nor had my career in programming, without the writing of David Bryant Copeland on Postgres

Watch a Rails app wither

My favorite parts of visualizations that show a commit history are the moments of destruction: that moment when whole trees are zapped into oblivion.

I don’t have anything so dramatic for my latest Rails toy but it WAS fun to make a full-fledged rails app and slowly winnow it down to just an API serving stuff to Angular.

I’ve talked about Angular a few times and we all know the total teardown coming with Angular 2 has made its future look grim. For the moment it was kinda cool to see how well it handled half a dozen PUTs all at once while keeping the UI clean, and ui-router module makes pretty neat multi-part forms.

Re-factoring

Always wanted to travel back in time to try fighting a younger version of yourself? Software development is the career for you!
-@Loh

It’s so weird to come back to something you were happy with, so happy with you posted it to your site, and say ‘gah, this is terrible.’ There were so many things wrong with that last calculator.

It’s super important to say here that there is no absolute way to determine whether code is good or bad. Beyond ‘working’ vs. ‘not working’ there are a few other standards: code that is easy for most other developers to understand, that handles error conditions in a predictable way, code that uses fewer server resources to run. Every thing except ‘doesn’t run’ is just a matter of degree, and often you gain performance by sacrificing readability etc. etc. My gripes about my own code are more a list ‘red flags’ than any absolute problem

Let’s talk about some ways to evaluate code that is in general, not so good:

  • tons of comments – comments are good and helpful when they are good and helpful. If every third line of your code needs explanation, there’s a chance your code path is pretty convoluted

  • multiple safeties – my first version had two different ‘checker’ methods for a few dozen code lines. Two methods to check one line of user input isn’t great :/ At a basic level: methods have error conditions already, or ways to handle weird input. Double-triple-checking that you’re not calling pop() on an empty array isn’t actually necessary, just make your other functions handle undefined.

  • long – I’ve talked about this before, hell everyone has talked about it, but there’s usually a good chance that going from 60 to 12 lines for the same functionality will be easier to read and understand.

so here’s the ‘better Reverse Polish Calculator’:

var stack = []
function handleInput(expressions){
  if (expressions == "q"){
    console.log("recieved 'q,' quitting")
    process.exit(0)
  }
  var inputArray = expressions.split(' ')
  inputArray.forEach(function (expression){
    if (/\d/.test(expression)){
      stack.push(parseFloat(expression))
    }
    switch (expression) {
      case '+':
        stack.push(stack.pop() + stack.pop())
        break
      case '-':
        stack.push(stack.pop() - stack.pop())
        break
      case '*':
        stack.push(stack.pop() * stack.pop())
        break
      case '/':
        stack.push(stack.pop() / stack.pop())
        break
    }
    console.log(stack)
  })
}


process.stdin.resume()
process.stdin.setEncoding('utf8')
process.stdout.write('\033[32m> \033[0m') //input prompt
process.stdin.on('data', function(datum) {
  handleInput(datum.trim())
  process.stdout.write('\033[32m> \033[0m')
});


One thing I love about this refactoring is it doesn’t actually do anything new. I did some research into whether it would be possible to just use the "+" as an operator without having to recognize the string, but there was no easy way to get it done.

My own Postfix Calculator

Job hunt time and that means writing some code to do random stuff to things!

Given the requirements that the code to do this be ‘readable, easy to extend, and understand’ I wrote a Reverse Polish Notation calculator.

//reminder: 'operand' == some kinda number, 'operator' == a math symbol
var operandCount = 0 //for checking if we have > 2 operands to do stuff to
var stack = [] //list of everything entered, with operands as floats and operators as strings
var validCharacters = ['1', '2', '3', '4', '3', '4', '5', '6', '7', '8', '9', '10',
' ', '*', '+', '/', '-' ]
function handleInput (datum){
  if (datum == 'q'){
    console.log("recieved 'q,' quitting")
    process.exit(0);
  }
  var response = ''
  if (!checkInvalid(datum)){
    console.log ('Sorry, you can only enter '+validCharacters)
  } else { //the input wasn't invalid characters
    datum = datum.replace(/^\s*|\s*$/, '') //Pattern's just removing initial or terminal whitespace
    //if more thn one value was given at once, grab each
    //TODO: if you enter more than one value the 'operandCount' checker will
    //become inaccurate. Basically the entering of multiple values isn't fully
    //supported in this version.
    datum.split(' ').forEach(function (value){stack.unshift(value)})
    var lastVal = stack.shift()
    if (isNumeric(lastVal)){
      //console.log('oh my, that's a number')
      operandCount++
      stack.unshift (parseFloat(lastVal))
    } else{
      if (operandCount < 2){
        console.log ('please enter '+(2-operandCount)+' more value!')
      } else {
        operandCount -- //one operand is destroyed in any function
        switch (lastVal){
          case '+' : performAddition() ; break
          case '/' : performDivision() ; break
          case '-' : performSubtraction() ; break
          case '*' : performMultiplication() ; break
        }

      }
    }


  }
  response = stack[0]
  console.log (response)
}

//this function could be written on one line with functional JS but it would be harder to read
function checkInvalid(datum){
  var reply = true
  datum.trim().split('').forEach(function(character){
    if (validCharacters.indexOf(character) < 0){
      reply = false
    }
  })
  return reply
}

function isNumeric(n){
  n = parseFloat(n) //everything's coming in as a string from stdin.on('data') :shrug emoji:
  return (typeof n == 'number' && !isNaN(n))
}
//are you at all worried I'm shift()ing the operator here? nope! I shifted it already
function performAddition (){
  stack.unshift(stack.shift() + stack.shift())
}

function performSubtraction (){
  stack.unshift(stack.shift() - stack.shift())
}

function performMultiplication (){
  stack.unshift(stack.shift() * stack.shift())
}

function performDivision (){
  stack.unshift(stack.shift() / stack.shift())
}





process.stdin.resume();
process.stdin.setEncoding('utf8');
process.stdout.write('\033[32m> \033[0m'); //input prompt
process.stdin.on('data', function(datum) {
  handleInput(datum.trim())
  process.stdout.write('\033[32m> \033[0m');
});

Install

  1. Make Node work on your system
  2. download this script to TFPostfixCalc.js
  3. run node TFPostFixCalc.js
  4. enter numbers or operators into the prompt q or EOF should both quit

TODO

In order to seem like a real human being I left one of the issues as a TODO in the code, because if you enter 10 42 the code will sorta work, but to improve it I’d also have to handle 10 * where I’m not totally clear on for the standard… I think it’s just an error?

Other stuff I don’t love:

  • all those separate functions for the operators could at least be written more densely
  • several checks could be rewritten with chaining and Functional patterns to be way more dense, but this is the way I write it first and I think this ‘fluffy’ version is easier to read
  • I feel it’s correct on bad input to just ignore the last entered line and maintain the state otherwise. Wasn’t in the spec, just said ‘won’t accept junk’ so :shrug emoji:. The other option would be to quit at that point.
  • Might want to color the output? IDK hardly major…

13 Lessons for every game jam

I’ve been doing game jams since before I could program anything, and there are a few things I’ve learned in that time. Here are all of those things:

1. Define your goals

Go around the table and ask people what their goals are. “I want to finish a game” and “I want to make something original” are different things! It’s not that you can’t work together, it’s just good to know when you’re both exhausted Sunday morning what everyone went into this wanting. Here are some great hard questions to ask some near-strangers on Friday night:

1) if you had to choose between something you thought would make people laugh, and something people would buy if we put it on sale, which would you pick?
2) If we have nothing finished on Sunday, when would you like to finish the game? Can you take Monday off? Or would you rather just chalk it up to experience and move on?
3) Is there a dream game you’ve always wanted to make?

2. Fuck that fucking theme

So you’re brainstorming with new people. You’re throwing out ideas. Suddenly you feel a little stuck.

What makes an idea good? What makes it fun? What if when you say ‘blue’ you mean what I mean but you really see a totally different color?

These are hard questions! If you ask your brain to answer them it’ll probably try to find easier questions to answer, like the one question that will 100% guarantee that you’re going to make a terrible game:

“But how well does that really fit with the theme?”

YIKES!

I discussed above defining your team’s goals. No one in that discussion is going to say ‘I want to follow all the rules.’ But in that difficult brainstorming moment when no one is sure if they should move forward or keep workshopping, the one thing that will be pretty easy to agree on is how well something fits with the theme. Suddenly all discussion becomes how to fit your great idea into the box you think someone has laid out for you.

Now, when people play your game, judge your game, decide to buy your game, none of them give a rat’s ass that you were assigned a theme of ‘harvest’ and ‘your body is a weapon.’

Back in my visual art days, we called this ‘assignment work.’ It’s when someone’s portfolio was obviously stuff they’d made for a class assignment, often with pages describing how their teacher had forced them to use graphite, and draw ‘something that grows in our backyard.’

The idea with game jam themes is to 1) prevent you from re-using old work 2) to make sure there’s some consistent thread between all games. Those are worthy goals, and you shouldn’t cheat at the game jam. But that’s as far as it goes: you need to pursue your own goals, and while a theme can help if you have no other preference for what to do, a theme should never stop you from making the game you want to make.

Another way to put this is that abandoning good ideas for ‘not fitting’ is putting the responsibility of making a good game on someone else. If the game is bad, it’s not your fault, you were assigned the theme of ‘lost at sea’ and so you made the game they wanted. Why, if your game is really a failure, surely you can sue the judges!

The worst thing about producing garbage that conforms to someone else’s requirements is that it feels so hollow. If you make a game where you eat hair to teach Andrew Jackson a lesson, and everyone hates it, you still made a game that one person wants to play.

that which does not kill me only makes me weirder

So use the theme for inspiration, don’t come up with an idea before you hear it, and them make the game you want. If your game was about farming but the fun part turns out to be the way crows follow you, pursue that.

3. start with a screenshot

People are going to judge your game based on its screenshots. Whether it’s on a jam site or an app store, your screenshots promise how fun your game will be. Once you’ve got your idea hammered out, start working on a sketch of a screenshot right away. Here’s our first one:

first game rough
first rough screenshot. The bars at the side are for the gameplay buttons

This has multiple benefits:
* you clearly communicate with your team what the most important elements are. Anything not in the screenshot should wait until everything in there is working.
* you get scale clear early. If working in 2d, this screenshot should actually be a PSD that artists can slice up and fit buttons and characters to. That way you’re not stuck re-importing and re-scaling way-too-big sprites on Sunday
* you get something to share on Twitter
* gameplay problems are clear earlier. Take a look at our first game ‘screenshot’ above:

As the artists and designers looked at it, it became clear that the background action was totally concealed, and the ‘counter’ in the foreground was also crowded, leaving little space to display the sandwich you were making (later we realized we wanted to show all the ingredients you’d selected on the counter, so even more space was required)

second screemshot
small but important layout changes.

Version two had a lot more space to show the background and the foreground, but the ‘hero’ was pretty tiny. I thought this was fine: he’s the player character, no one says “I want to see more of Gordon Freeman’s arms.” But the artists pushed back saying this cool character was now in the margins. We settled on adding some narrative frames to the beginning and end of the game to feature his story more.

4. Only program things you know how to program

The essential difference between a game jam and any full-time development is that you actually can tell your team “That feature is impossible.”

With a hard stop Sunday night, any feature that you can’t get working by Saturday at noon is impossible.

But isn’t it hard to predict, when coming up with ideas and making plans, what you can get done by Saturday? It really shouldn’t be. You should really only agree to make a game that you understand perfectly how to create.

If you’ve never made a game before, Unity has several great tutorials on creating several types of games, and if you use a bit of imagination, I think you can see how about 90% of the games you enjoyed last years are just different versions of the games they list there. Try to complete 1 or 2 of these tutorials before the jam and you’ll have a full quiver of stuff you do know how to do.
I have approximate knowledge of many things

I address elsewhere some good reasons to leave the beaten track and figure stuff out yourself. I assure that even if you are certain that the game you’re working on is just a re-skin of the Space Shooter you made before, you will get many chances to work on new things. That little power-up idea everyone liked? That homing missile that you thought would take you a second to add? New challenges come up all the time.

5. Watch the stupid video

The biggest new feature for Game Jams in 2015 wasn’t the Unity GUI or Github’s improved binary support. It was YouTube’s ability to play a video at 1.5x speed

No one can code around the clock, and when you’re having a snack or just waking up, that’s the perfect moment to start up a tutorial on whatever you’re working on. The direct Unity ones are great, and YouTube is brimming with amateur tutorials. Just make sure you’re watching from 2014 or later.

6. stay on pace

My schedule works for me, and by sticking to it I’ve never failed to deliver a game:

Friday

6pm: meet people,
6:30PM: fidget through a keynote that feels interminable but maybe lasts 45 minutes
7PM: brainstorm an idea.
8pm: Look up some programming stuff for how to do it.
9PM: Read but only in a desultory fashion, open a new Unity project, feel overwhelmed. Go home and get some sleep

Saturday

8am: start coding up your features using grey, black, and white boxes
12am: call over your whole team and say ‘the flamethrower that makes you jump higher is working!’ and show them a grey cube with some other white cubes that move around. And a debug log message. Assure them this means it’s working.
2pm: realize you, the artists, and the designer are out of sync and either roll back their expectations or start furiously working on an additional feature or two
4pm: by this time Saturday, have all features programmed. Give the designer the bad news about any stuff not yet started.
5pm: start throwing in final assets
6pm-10pm: You’ve been programming for 4 hours and haven’t done… anything important at all. you keep moving around sprites that aren’t final and adding safeties that aren’t important. Realize this and go home and get some rest

Sunday

10 am: finally start working after being oddly slow all morning
11am: let the artists start testing builds. In the first 30 seconds of them playing, notice five broken things you meant to fix over a day ago
2pm: This game better be finished bud
3pm: toss in game opening, and game over, splash screens
4pm: add those sound effects and some music
5pm: fix the music volume
6pm: someone will tell you the ‘end movie’ is done. Give them a puzzled look.
7pm: UPLOAD

7. Use a bug log

Even when I was hitting play on a game to run it for less than 30 seconds, I’m in the habit of using a notepad to jot down bugs. That way I can keep running the game and maybe notice other broken stuff, instead of stopping right there to fix the bug.

The time savings seems really minor: I rarely have ‘clusters’ of bugs to work on, so I’m just saving the time it would take me to start the game again and play to bug #2. But I find it enormously helpful.

I think the other advantage is that it clears space in my mind: I wrote down the bug, I don’t have to try and hold it in memory for even a few minutes. Maybe I wanna go use the can right after my test run, I won’t feel so tense if I know all the stuff I noted is jotted down.

One other good habit: cross stuff off your bug log, either when you fix it, or when you realize it’s low priority. Writing down a bug doesn’t mean it’s top priority, just that you don’t have to re-discover it later.

8. This is a game jam. Not a film Jam

Film Jams exist apparently! But this isn’t one. And that film you’re making to show before or after your game, it’s not going to work out. Couple things:

1) Game engines aren’t super good at showing videos. I wish I had more to say about this but I really don’t. Unity doesn’t have a way to show a video outside of the pro version, other game engines don’t do it well
2) your artists, your writers, whoever else, are going to pour time into this narrative section. They’re going to make something that includes all the ideas they came up with, the theme prompts, all of it. None of that work will be reflected in the actual game.
3) if you’re making a game where the player controls the outcome in any way, players have a less than 100% chance of seeing game-ending material. The player might win. They also might lose, or get bored and stop playing. That third option is the one you’re trying to prevent, and anything the player sees on winning the game won’t help avoid them giving up before the end.
4) people want to play the game you made. Anything that forces them into watching a prologue is actually a barrier between your audience and your game.

8. Make a great title card

image of the Nostromo from Alien booting up
I think so much of Alien’s aesthetic is described in this early shot

Most people won’t play your game.

While I do have serious concerns about time spent creating intro cinematics, I cannot stress enough how key a good title card is.

People looking at your portfolio, randos browsing a game jam site, people seeing the site preview card on Twitter, all will see your game’s title card. If it’s some random ‘programmer art’ with buttons tossed over a random image, then to some portion of your audience, that’s what they’ll know about your game.

Like the elaborate paintings on old arcade cabinets, a good title card lets people project an aesthetic into a game where it might not really be during gameplay.

Also a title card painting is a great Sunday activity for an artist, because otherwise:

9. Any resources made on Sunday will never be in the game.

When things are going really well this won’t happen. But I’ve never had things go really well. My first 3 or 4 jams I had artists bring me beautiful models, animations, sound effects, etc while I was desperately trying to get the game to build successfully. Needless to say these pieces never saw the light of day.

very impressive but no one is watching

So what can an artist do on Sunday? 1 of 2 things:
1) Improve the resources that are in the game
If you already got a placeholder version of a sprite/model/texture, it should be easy to swap it for an improved version. This way you shouldn’t be caught fiddling with import settings late on Sunday, and you know if some final versions don’t get done you’ll have something in their place
2) test!
This is my ideal flow: on Sunday morning hand the art team a rough build, and let them play it to death, writing down every bug. They might notice assets that need to be re-done at the same time, but this QA is invaluable.

the less code, the better

My first game jam game had over 900 lines of code. The game did not work at all. My last had less than 75. In Unity, the less code you’re writing yourself, the more time you’re spending using GUI tools that are extensively tested and designed to make games. Here are some things you should probably be using instead of coding up your own behavior:

  • physics/physics2d and colliders
  • GUI
  • Animations, especially those with callouts to other methods
  • An animatorcontroller with several parameters

Here are two good reasons not to use an inbuilt feature and write the code yourself:

1) you came here to have fun and learn stuff, not drag icons around
2) you don’t know how animations work

Both are fair points! I’d say everything above except maybe colliders takes longer to learn than you have for the game jam. If you know how to write a state machine for animations already, or you’ll be happier knowing you tried it, have at it.

10. The more talk, the better

Whatever you can do to foster communication is beneficial. At least once in the weekend, your overtaxed brain will notice that you can’t work on the thing you really should be because you don’t have the asset/design doc/answer/test build you need to move forward. In that moment I hope you think to tell someone, but it’s super helpful to have scheduled times to check in and make sure you’re moving in the same direction.

Last weekend was my first jam where I used Slack and it was a godsend. The ability to preview images inline, share files, and save a discussion ‘history’ that we could show to others later was all key.

It also let me tune out chatter for hours knowing the important stuff would be in Slack when I wanted to get caught up.

11. There are no 10x devs, but there are 10x artists.

Lots and lots of people are good artists. They make art that looks really good, and can add a lot to your game. Sadly there’s a big range in how well an artist can help during a jam. In general, no matter how great someone’s portfolio pieces look, a production artist is going to be able to help you make a professional looking game.

If you want three characters each with a walk cycle and attack animations, then it’s best to work with someone who has experience with producing a lot of stuff fast and in the right format.

If someone has no experience or training in producing game assets, they might be great, but you’ll need to keep a close eye on their pace through Saturday morning. If they’ve just got a few roughs done by that time, consider using open game assets for your characters and having them draw backgrounds, or character portraits to use in dialogue.

This problem tends to get magnified by Sunday, when it turns out that a bunch of sprites are at the wrong scale, or their colors don’t blend well, or whatever else.

People who are good at doing visual stuff fast will also get you out of countless self-imposed scrapes. That moment when you realize the cool GUI you wrote for the start of the game won’t be ready in time, you’ll be so glad that you have someone on the team who can make an awesome splash screen:

Game Title Splash Screen
So awesome
in like minutes

13. Instruct your users

My last game was very simple. There were only onscreen controls, no advanced mode, no special goals. I still tried 4 distinct strategies to show users how to play my game.

Every time you get someone to come over and try your game, remain totally silent once they start. Anything you have to tell them must be told to the user.

Are panels like this one hacky?

TutorialScreen

you bet! But you just don’t have time to design the slow-build ‘tutorial’ level that show cleverly shows you how to play the game.