In the 35 days the Halo 3 Public Beta existed, players spent roughly 816 years in-game with discarded Hot Pockets cardboard container thingys and empty soda can mountains slowly growing around them. During those 816 years, a snapshot of the hive’s file share would show at least 2.7 terabytes of sharable data. But we didn’t conduct the Halo 3 Beta to suffocate you with a pillow of statistics; instead we were actually culling data, watching player behavior and listening to squawking, raucous feedback from the community. Now that it’s over and the multiplayer guys are getting their heads above water (only to put them drown again in a sea of polish), Multiplayer Lead Tyson Green, Designer Lars Bakken and Producer Joe Tung explain some of what they learned from the Halo 3 Public Beta (yes, we know the Carbine is really effective, Interwebz).
What sort of information were you guys focusing on in the Halo 3 Public Beta?
Tyson Green: First and foremost, to see what happens when some hundreds of thousands of people leap from on high and land on our server infrastructure. We found that out pretty quickly (it goes “crunch”).
After that, we wanted to see how people responded to the entire package. When you’re obsessing over a particular game, or even a particular weapon or piece of equipment, it’s really hard to take a step back and see what the big, overall issues are. The Beta was good because it let a few thousand people help us do that. When you think the biggest issue in the world is “man, I think the Spikers do at least 10% too much damage”, and someone says “no stupid, Slayer comes up too much in Team Skirmish!” that helps you refocus.
And after that? We check to see if the Spiker is, in fact, doing 10% too much damage.
Lars Bakken: First off, like Tyson said, we wanted to see what would happen when hundreds of thousands of players jumped online to play. There were a lot of technical back-end things that we learned from the Beta, so from that standpoint it was really useful. Matchmaking really got stressed, which is great to get all this information well before the game has to be done. We also were interested in public reaction after playing the game. Of course there is the “gee whiz” I get to play the game early reaction, but we were really interested in people’s thoughts after playing for a week or three.
Joe Tung: The primary goal of the beta was to test systems in the wild that are difficult for us to test internally, that is to say, systems that require massive scale to be adequately stressed before bugs can be found. In particular, Networking, Matchmaking, our ranking system and file sharing are all systems that benefitted greatly from the beta.
Did the players alert you guys to anything you weren’t aware of?
TG: Certainly a ton of networking related things. We also had some good feedback on how playlists ought to be set up (which was something we were able to react to on the fly.) There was good stuff too, hearing generally positive things about particular weapons or equipment that we were maybe a little unsure of.
LB: Of course, our fans are ridiculously good at finding things in our games; both good and bad. Instead of having the standard test team on a dev cycle, we were actually lucky enough to have a few hundred thousand people actively help us work out the kinks. There were some animation glitches, and of course some ways to get out of the maps were the biggest things I can think of right now.
JT: The biggest thing I was alerted to by a particular user was that Team Slayer has a -blam!- orientation.
When tuning weapons from the beta onto one of the numerous revisions the game has seen, what sort of changes are being made?
TG: At this point, tweaks. The major functionality of the weapons in the Beta is not going to change much, but how much damage done will vary a bit, how much ammo a weapon has will be adjusted, error and rate of fire… all the sort of thing we do constantly, but now with some clear basis for those changes.
LB: It’s mostly minor things at this point. Believe it or not, most inconsistencies in the weapon balancing can be fixed with very minor changes. At this point, there’s no reason to make drastic changes, because things work together very well. A little tweak really goes a long way.
Do you feel like the playerbase got a good grasp on equipment from the Halo 3 Beta?
TG: Yes, the pieces we showed were adopted very quickly, and it’s hard to find a write-up about the Beta that doesn’t contain a story or anecdote about a timely bubble shield or trip mine. Pre-Beta fears about them being unbalanced game wreckers calmed down pretty quickly — our players are pretty sharp.
LB: I think people “got” equipment. I know there was a backlash when the bubble shield was first revealed, and the community outcry was almost deafening. “Everyone will spawn with shield grenades! ZOMG Halo 3 is teh ruined!” Once the public actually played the game though, they got it right away. All it takes is deploying the bubble shield successfully that first time to save your ass… and you get it. It just clicks.
JT: Don’t you watch YouTube?
Once maps and weapons are unleashed into the wild they can take on a life of their own, something different than how a weapon was used, did any things from the beta perform differently than expected once the masses got their hands on them?
TG: Not too many things strayed widely from what we were expecting. The Carbine was more powerful than we were anticipating, and the Warthog was somewhat harder to kill, but generally things were pretty stable. Spawning on some of the maps showed some unhappy patterns, but in that case, we were sort of expecting the unexpected — dialing in spawning in is half personal preference, half black magic.
LB: I wouldn’t say things were very different than we were expecting. We do a lot of internal testing, so for the most part it was like watching a lab experiment you’ve seen before. Sure there are anomalies here and there, but for the most part it’s what you expect. The tree jumping in High Ground wasn’t something I was even thinking about until I saw that trick video, but those are the kinds of things that really make the Halo community special.
There were also some issues we knew going in, like spawning, that we knew weren’t going to perform up to what we expect, but it we were able to learn a lot from it.
We all saw the video of a Banshee going into space; will there be a way to adjust collision to keep people from escaping a map in the final game?
TG: Yes, we have some more robust defenses to protect our players from The Terrible Secret of Space. But having said that, players are crafty. The saying here is “if a player wants to get somewhere, he will.” We just do our best to make sure he can’t use it to wreck the game for other people.
LB: Yep, like Tyson said, we have some more tricks up our collective sleeves to keep people in the maps. I’ll never say that people won’t get out of the maps, because if there’s one thing Halo fans have shown us is that they love a challenge and they’ll always find a way to do something they want to do.
Did the playerbase play any of the gametypes differently than anticipated?
TG: I’m just starting to dig into that data, so I can’t say for 100% sure. I’m curious about capture/detonation rates on CTF and Assault in particular. But I think, in general, no. Not many surprises, just affirmation of some things we were hoping (like Territories being fun) and things we sort of already knew (like VIP being kind of campy on Valhalla).
LB: Nothing too out of the ordinary. Like I said previously we play a lot internally, and ran an Alpha and an internal Beta before the public one; so in a lot of ways we were expecting some of the things we saw.
JT: Frank and Joe beat me and Luke at Double Team yesterday. Badly. I admire their skill greatly [Ed. note - paraphrased (Editor's editor's note: Frankie is erecting palaces of lies for you, dear reader)].