Introduction

Statistics are a huge part of any spectator sport. For those who haven’t heard, I’ve been working on a project to gather statistics directly out of Melee. The result of the work was showcased for the first time at HTC Throwdown where live stats were shown on stream. This is part 1 in a series of posts that discuss multiple aspects of the project.

The goal of part 1 is to introduce the technology that allows it all to work. It is written in the form of a narrative detailing the process I went through on the road to HTC Throwdown.

The Journey

This section briefly describes the different components that make it all work. The components are introduced in chronological order of where I focused my attention, this is done to provide insight into my decision making.

When reading through this section, keep in mind the following:

1. This section by no means describes what the final version will look like.
2. The primary goal of the proof of concept (which is what is described below) was to make something that works as quickly as possible.
3. Optimization and cost efficiency were not a major consideration.

The Memory Card Slot

An image defining the values of the memory card slot pins – all credit to original author, link to source post in references section

The first thing considered in the early phases of the project was how data could be transferred in and out of the game. The memory card slot was an obvious first place to look to accomplish this – data has to be transferred in both directions for reads and writes to be possible. The question became: “how does it work?”.

Luckily a little research was able to at least partially answer that question. The GC (Gamecube) has what it calls an EXI (EXternal Interface) bus on which generic data can be passed to and from the console. It turns out that EXI is actually an SPI (Serial Peripheral Interface) bus that operates in mode (0,0). No need to explain exactly what that means but know that SPI is a very common communication protocol used by microprocessors and other devices to communicate data. It was looking like the memory card slot was exactly what I was looking for.

The Arduino Due

Now that I knew how the GC communicates, I needed something for it to communicate with. I figured the communication speeds for memory card operations were fairly quick and I wanted something easy to work that has loads of support available on the internet. The Arduino Due is a development board built on a rather fast processor with three individual SPI interfaces. Figuring it was likely a good choice, I bought one.

The Arduino Due

The breakout board

I still needed a way to connect the GC memory slot pins to the Arduino somehow. In order to accomplish this I designed a very simple breakout board to replace a memory card. All this board does is plug into the console but instead of containing memory hardware, like a memory card would, it just exposes all the pins for easy connection to something else. The board was designed to fit into an existing memory card case with the back broken off.

Memory card breakout boards

The first attempt

At this point I had enough hardware to find out if I could read data coming out of the GC. In order to attempt this I plugged the output of the breakout board to a standard GC memory card and hooked up the Arduino to try to sniff out the data.

I encountered a few stumbling blocks attempting this. One of which was that although Arduino provides an SPI library, it apparently only supports SPI master mode. Unfortunately in this configuration, the GC is the master and the Arduino must act as the slave. That said, the hardware does support it and it is possible to set it up.

Assuming I set slave mode up properly, the major problem I then encountered was that I couldn’t seem to read out the data as fast as it was being written. In other words, the Arduino seemed to be too slow. I do think it may be possible to implement the communication using solely the Arduino Due, but at this point in the project I decided to go with another approach.

The Cyclone II

Stuck behind a wall, I needed some inspiration. I ended up looking at the implementation of the USB gecko, a rather similar device to what I was trying to create. The USB Gecko uses an FPGA (field-programmable gate array) to implement an EXI to USB bridge. FPGAs are effectively programmable hardware. This means that they can be made to execute tasks very very fast. I knew that by using an FPGA I certainly would not have any issues with missing data and so I purchased a Cyclone II development board.

I programmed the FPGA into two major components. The task of the first component was to communicate with the GC at high speeds, writing the data to the FPGA’s internal memory. The second component then read the data from memory at a slower speed and passed it to the Arduino. Using this configuration, I could now receive data I was interested in reliably at the Arduino – so long as I didn’t fill up the internal memory of the FPGA.

The EXI registers

With the hardware in a functional state, there was one big problem left to tackle. How do I send interesting data from the game over the EXI bus? I knew I had to mod the game but I didn’t know the best way to approach it. Originally I wanted to hijack the existing memory card write operation after a game ends. Unfortunately the data passed to the memory card is encrypted and/or compressed. I was stuck again and feeling rather down about the project as a whole, I took some time off. Eventually I knew I needed to get back at it but I still didn’t know what to do… So I made a Smashboards post requesting help.

Two users took interest in this post: Achilles1515 (of 20xx fame) and SinsOfApathy. This spawned a private discussion between the three of us on Smashboards and eventually led to SinsOfApathy pointing me in exactly the direction I needed.

There are memory mapped registers in the GC that can be used to control the EXI bus directly. The link SinsOfApathy gave me was this one: http://hitmen.c02.at/files/yagcd/yagcd/chap5.html#sec5.9. Immediately upon viewing this page I knew it meant that I could write out whatever I wanted, at whatever speed I  wanted, whenever I wanted just by writing a little bit of assembly code.

The demo

Hardware configuration one revision earlier than the one used for the demo

So I wrote some assembly code, learned how to create a DOL mod, and created a Melee ISO that wrote out some rudimentary data. I finally tasted success and damn was it satisfying. Here is the video I made that very day to introduce my project to the community: 

HTC Throwdown

Given the overly positive responses I received from the demo, I was hyped up to do more. I wanted to give the device a trial run that would truly showcase its power. HTC Throwdown was coming up and I was attending due to a conveniently timed business trip. I reached out to smash.gg and Showdown and they were willing to support trying it. I knew the timeline would be difficult with little over 2 weeks left before the tournament but there wasn’t much to lose – so began the late nights.

An image from HTC Throwdown – taken just before Plup vs MacD

Leading up to HTC Throwdown I made a lot of improvements. These are the major ones:

• Modified the hardware structure such that much more data could be transferred out than was possible in the demo.
• With a lot of help from Achilles1515, I wrote the assembly code that transferred all sorts of interesting character information out every frame.
• Wrote the algorithms used by the Arduino to compute interesting, complex statistics (these algorithms will be discussed in detail in the next part). I must thank Kadano here because his in depth knowledge of the game was extremely helpful in writing the algorithms.
• USB communication wasn’t really the communication method I wanted to use going forward so I bought the Arduino Ethernet shield and implemented ethernet communication logic.
• JSON (JavaScript Object Notation) is a very popular way to transfer data via the internet. I packaged up all of the data in JSON to be sent over ethernet.
• Wrote a simple TCP server in Go that would receive the JSON from the Arduino and write it to files that could be used to display information on stream as well as log everything for analysis afterwards.

Spenser from Showdown put it a ton of work to get the overlay working at the last minute. Huge thanks to him because without his effort nothing could have been showed on stream. It was amazing to see him still willing to work on the overlay during the tournament even though it seemed like he had been working non-stop for many days in a row.

Closing

All in all from the time I had the original idea to HTC Throwdown was a total of 3 months and 4 days. I can’t thank those that helped me enough – there’s no way I could have accomplished such a task without some major help. I’ve greatly enjoyed working on this project up to now and am extremely excited about my plans for the future – thanks for all the support I’ve received thus far. If you are interested in project updates, please follow me @fizzi36 on Twitter.

The following section will be less storytelling and will more closely resemble technical documentation. Look forward to a description of the raw data that can be extracted out of the game and the algorithms that made the existing stats possible in part 2 of the series! Thanks for reading.

Special thanks to:

SinsOfApathy – For being integral in unlocking the EXI mystery
Achilles1515 – For your support in getting me acclimated with the internal workings of the game and assembly programming.
Kadano – For your amazing knowledge of the game and your impressive response time to my silly questions.
Spenser – I don’t know where you found the motivation to finish it up man. Towards the end of the day I was personally feeling spent. Amazing stuff pushing through to get something working.
Tafokints – For discussing stats ideas with me and being very supportive.
Shan – For helping connect me with Showdown and vouching for me.
Ashen – For the informative image detailing the memory card pin values.

References:

http://forums.modretro.com/viewtopic.php?f=36&t=7994
http://www.gc-linux.org/wiki/EXI:USBHostAdapter
https://www.arduino.cc/en/Main/ArduinoBoardDue
http://hitmen.c02.at/files/yagcd/yagcd/chap5.html#sec5.9