[1:22] Q: Do bugs ever turn cyclical?
A: Yes, all the time, fixing one issue will cause another issue to reappear and fixing that will cause the first bug to reappear. You just need to go back and fix it. QA is helpful. This happened in some cases with 2.4.
[3:56] Q: There was a story about spaceship movement being bugged which no one could figure it out until finally someone said something about CryEngine simulating everything as if underwater. Are are any similar stories nightmare bugs that were actually easy to fix if you just knew a particular oddity of the engine?
A: Roughly two years ago, John Pritchett had flown out from Kansas and everyone was working to get Arena Commander out but the ships just started acting weird. John spent two or three days trying to figure out what was going on. Finally he walked up to the other engineers and said “Our ships are underwater”. Turns out that by default CryEngine has water under every level so anything that goes below 0 on the z-axis is considered to be underwater even when you’re in space! After that it was a simple fix.
[7:24] Q: What are some of the pros and cons between closed development and Bugsmasher’s open development models?
A: Closed development allows more detailed feature work without being interrupted by the debugging process, but lacks the numbers to test thoroughly. Open development often has more numbers to test, but can often distract development from the tasks at hand.
[9:58] Q: Do you get assigned to fix your own bugs? Does CIG have a system in place to fix design ‘bugs’? How do fixing code bugs and design errors compare?
A: Every team works closely with all the other departments. If there’s an issue with one thing you might have to go speak to someone in another team to get it fixed. Every department has their own processes for handling a bug. When implementing a new system, they’ll often consult with the departments that’ll use that system to get their opinion on workflow. Everyone has their own process and needs.
[12:51] Q: Do you ever have it where you find bugs and fix them, but then later find the source and have to fix it again?
A: Yes! There’s many times where a bug is found and fixed, but the actual source is discovered later on, sometimes months down the road when it finally appears after adding a feature or changing some other pieces of code.
[16:30] Q: How much of the company would you say is devoted to getting the release out? How much of the company isn’t impacted by the release cycle and simply continues to make content?
A: It depends for each release. Everyone is working on a different thing towards a common goal for any given release. Sometimes you’ll get pulled off new features to help with bug fixing a release. It’s different for every release – different resources are required.
[20:13] Q: Tools you have developed internally and are in usage?
A: Goal based tuning which helps designers input information into a database designed by John Pritchett. DataForge, will start cleaning up from having to do hand XML or hand scripting. Item port editor, move/remove stuff in game.
[25:22] Q: How was the transition from developer to producer?
A: It’s still ongoing.
[31:43] Q: Do patches with fundamental changes make the Live team do anything special in preparation or is the hand-off from the PTU and Q&A the same?
A: There isn’t really a live team. Resources are pulled from the entire company where required for a release – typically the people leading the features that are in the goals for that release. There is a handoff everyday between UK and the US – the UK handles Frankfurt stuff too. Sometimes they need to push releases to PTU they know are broken because they need more debug information. Sometimes certain features are disabled to focus testing on a specific area. Nerf guns hurt.
[35:27] Q: With PTU patches and live, do our bug reports really make a difference?
A: Yes, the team can’t thank the community enough for their hard work and dedication on testing and playing the game. Without the community’s support, they couldn’t come close to the amount of QA they need to track down and hunt bugs. Even if they don’t reply to your submissions, chances are they’ve used that information and passed it along, but don’t have enough time to reply to each individual person that has helped them.
Mark Abent (MA): Oh.
Randy Vasquez (RV): Hello.
RV: How are you doing?
MA: Not bad.
RV: Not bad, not bad.
MA: Who are you?
RV: I’m Randy Vasquez, I’m Associate Producer here at CIG. Who are you?
MA: Oh you’re a producer?
RV: I’m a producer.
MA: Do you know who I am?
RV: I don’t know who you are.
MA: I’m the BugSmasher.
RV: No one watches that show.
[Randy Nods his heads with a smirk]
MA: I’m the BugSmasher, Mark Abent, Gameplay Programmer and everyone watches me.
RV: It’s true.
RV: Not even creepy..
MA: Oh a little bit creepy.
RV: Well we’re here to answer some questions. First we’d like to thank the subscribers for everything they’ve done to make this possible for us to have this show, 10 for the Developers and I guess that one show you just mentioned what was it again?
MA: I don’t remember, I don’t have coffee anymore. Something about smashing bugs, Bug smasher thing?
RV: Sounds right. So let’s get started.
MA: Let’s start off with our very own questions from…
BaconofWar asks: Have you ever had a bug that fixing it turned cyclical? Ex: You fixed a bug in this piece of code but that caused this piece of code not to work so you fixed that bit of code only to find it broke the original piece of code again.
RV: So bugs turned cynical, or cyclical? MA: Cylindrical? RV: I don’t.. MA: Yes, oh yes, bugs have turned cyclical and cynical. This actually happened recently for 2.4, we’re having this persistence shenanigans where you can walk up to your ship and if I have a cup on the ship or a weapon I can select it, remove it and add maybe a pencil or a gun or something and in order to see that interaction we had to use our interaction system which was all the developed for Item System 2.0 – not for the original Item System. So we had to basically start making the original Item System work with this interaction system which meant we had to install a new network component, a new physics component and finally the interaction component and the interaction component and network component were like “Alright, I’ll work. I got this. Cool.”, physics component was like “Na na na, I got bugs for you.”, so when I finally got it installed, got it in, QA started finding some bugs… RV: Yeah, we had a flood of bugs coming. All of QA was hitting me with like “all the stuff is now falling through things, the turrets were not working- MA: You put a poster on the wall, walk up to it, slap it and it’d fall down to the ground. RV: It was so funny. MA: There was like a weapon thing to restock ammo and you could push it into the wall it was like “Oh no, what did I do?”. So yes, you fix one and then a bunch more come up because QA – especially Andrew- RV: Yeah. MA: He’ll call you up- RV: Yeah. MA: -this cynical laugh, and go “I found something.”, and you’re like [motions facepalm and shakes head]. RV: It’s even worse when he re-opens it and lets you know, “By the way, this is for you.”. MA: Not even the way he re-opens it, he skypes you directly and to give you that laugh – it’s great. RV: [laughs] MA: [sighs] So yeah, fix some, break ‘em, happens all the time, then you just go back and fix it. Luckily, that’s why we have QA – QA is awesome. RV: Yes.
Piet11111 asks: I hear a neat story that when the first physics stuff was being built for Star Citizen the spaceship movement was bugged and the guy in charge of this was trying to figure out what the heck was going on, double checking, triple checking everything to find out what was going on. He talked with Chris Roberts at some point and he just went, ‘oh yeah CryEngine simulates everything as if underwater’. Now I probably got something wrong in the story above but what I want to know is if there are any similar stories to share about someone losing lots of sleep trying to figure out a problem that turned out to be incredibly easy to fix … if you just knew that particular oddity of the engine.
RV: So I didn’t … I didn’t even know about this story. What is this? MA: Oh this is a great story. So this happened roughly two years ago before we even launched Arena Commander. And we had our physics programmer John Pritchett, he flew out from Kansas, and we’re working with him trying to get Arena Commander 1 out. Everything was going … sort of working well, and then all of a sudden our ships just started acting weird: it would fly but it would go the wrong way and then you would just stop. RV: So like our missiles sometimes? MA: Worse. Actually our missiles were underwater but that’s another story. But yeah … and we’re trying to figure out what’s going on. And John Pritchett, he’s debugging, debugging this, and he spent like two or three days because we’re all still new at the engine, especially him, trying to figure out what’s going on with physics: is it the game code, is it the thrusters, is it the … whole big system? RV: Yeah. MA: And he finally figures out, he walks up to us and he goes, I still remember his face, “Our ships are underwater!” Everyone around just looks and goes “Can you repeat that?” [slightly hysterical laugh] “Our ships are under water!” And what happened is CryEngine by default, like if … RV: Has water under everything. Under every single level in CryEngine has water. MA: If you think of an xyz plane you have [indicating each axis] x and then maybe y and then z up and down. So if this table was the xy and z up and down, anything below 0 would be considered “water”. So any entity that went past 0,0,0 instantly got the water flag. So if you were flying up in here [above the table] but as soon as you go down here [below the table] you’re in water. RV: Even though the plane was invisible because you can’t really … MA: Even though we got rid of terrain … RV: Yeah. MA: … and got rid of water for space you were still going under water. RV: Nice. MA: And it was a simple fix. It was just like really? Oh my gosh. And there’s been many bugs like that but … ugh … RV: That’s kind of awesome. I can see … I can just see John’s face too. Like “By the way …” MA: “By the way” RV: “This thing water. Everything’s in it.” MA: Yes. RV: Cool. MA: I don’t even know how you knew that. But that was a good … that was a good story. Alright, let’s go on to our next question. So Mr Mortis Rigger. Oh that’s good! We have … I have to say before I go on to this question we have … I’m going to get … we have a guy named John and his last name is Riggs. So we call him Riggs the Rigger. RV: Yeah. MA: He doesn’t like it. Hopefully he doesn’t see this but [whispers] Riggs the Rigger!
Ripking asks: Bugsmashing open development vs closed development pros and cons.
RV: Favorite pro/con… most hated con, go. So, we should talk about this for a little bit in that, there’s a big difference in the team set up when you’re live and basically open development compared to when you’re behind closed doors and the players don’t get a chance to see what you’re actually working on. Cause like, I remember when I was a designer working for the studios and when we… before we released any betas or alphas or anything like we were able to just work on our feature and really nail that feature down. We also had… you know QA didn’t really bug it up, because they knew it was being worked on. Where now in open development we’re always live and people get to see everything we do every single push… MA: Oh yeah. [nods] RV: And the features we push forward. The QAs … again QA is awesome, because they’re basically our last line of defense as well as they’re on all the bugs on live, they’re on the bugs on the features, so they’re bugging everything. MA: And they even talk with the community to find what they have … RV: Basically. MA: And going back and forward… RV: Yeah. MA: Gather all the information and drop it on our desk. Okay. [laughs] RV: Yeah, so it’s like it’s,I think open development to me takes a little bit longer. I think that’s the con, but again we get really valuable feedback really early on. I know port modification and the 2.4 stuff we’ve been working with the private group. We’ve been getting a lot of feedback on some of the features that we’re trying to push out and stuff we didn’t even think about we’re like, “Oh yeah.” MA: Yeah. RV: Okay, cool. We didn’t catch that. Cause, I mean QA we have like maybe 30 people if that. I don’t even know the actual number. MA: I think there is more, cause UK and Austin have a lot. RV: Yeah, but it’s like I don’t know. Like, I think it might be about like 30 something people and that’s not enough people like all the stuff we need from there. We need more. We need more people test. MA: Need big development access. RV: So that’s good. That’s definitely a good pro for having open development in that where in closed development we don’t have those same numbers of people being able to jump in and play test. Sometimes getting everyone involved is kind of crazy, especially in trying to get devs to take time out of the day to playtest. Which we do, from… MA: Well, that’s why you go around going, [pokes Randy in his shoulder repetitively] “Ready to playtest?” RV: Yeah. MA: “Ready to playtest? Ready to playtest?” RV: Yeah, little bit. MA: “5 minutes. 5 minutes” [holds up 5 fingers repetitively] [Both Laugh] RV: So yeah, that’s kind of problem I have for like open development versus closed development. I don’t have a preference. Sometimes I have, like if I’m working on a feature, I have a preference, but if I’m not working on a feature and I’m just like helping support stuff then I definitely prefer one to the other. MA: Yeah. RV: Anything on you? MA: Nah, I’m good.
Amontillado asks: I understand that Mark is both a feature programmer and a Bugsmasher at CIG. Do you typically get assigned to squash your own bugs? Also, Bugsmashing seems to be largely (entirely?) directed at coding errors. Does CIG have a similar system in place to smash design ‘bugs’? How do smashing code bugs and fixing design errors compare?
RV: So, this all kind of lumps into one big thing – I know I’m always talking to the engineers and I’m always talking to the designers and stuff. Design is kind of a weird thing in that designers are the glue for a lot of different departments – and all the departments here at CIG, they work closely, like hand-in-hand, like how many times do you work with designers- MA: Oh, all the time. RV: …at least, like every day, and then also, how many times do you work with artists? So every single department is, to use the word cyclical, they all work together, they all- because you and Mark McColl will always work on stuff together. MA: Mhm. RV: So the tech content team, the tech artists, the regular artists when they are making geometry – if the geometry isn’t working right with the tech then you have to go back. I know that Ariel was working on the port editor stuff- MA: Yeah. RV: So he’s working with the artists big time over the past two-three months, he’s been working- and then also working with the designers as well to make sure when they use the tools they can kind of go from there so it’s like- I don’t really think that it is a specific system to the whole entire company but every department has their own processes in which they handle the bug, like some of the stuff with the ships, I know you and Kirk talk a bunch because Kirk’s like “hey, what’s going on with this? what’s going with this?” and you and Sherman were talking about some of the weapon stuff, what last week? MA: Probably. RV: Was it the week- or- I don’t even remember.. MA: Well, it’s like, with the new Item 2.0 stuff, since we’re changing a lot of the systems I’ll go to design and go “hey, this is how I’m thinking of implementing it” because those are the guys that are actually going to do the brute force work of “here’s the xml, here’s the records, stick it all together” and I’ll go “here’s the flow I’m thinking, what do you have an opinion on? What do you think?” And they’ll give me feedback, then I’ll go to evil Mark or Mark McColl and- all other Marks are evil- RV: Even this one. MA: Even this one, [laughs] RV: [laughs] MA: I’ll go to also- evil Mark because he’ll do the asset and animations to link those together with the items so design has specific area, tech design has a specific area, so I’ll go to each one to make sure that when I’m making this new stuff that they both work together. RV: Yeah. So it’s a big mix, there’s a- everybody has their own process and what they need, and it really depends what the bug is and what they need for it – who to talk to. So, cool. Let’s go to the next one. MA: Alright, sounds good.
Steve-2001 asks: When fixing ‘bugs’, have you ever had an occasion where it was ‘fixed’ in one place, only to discover later that the actual cause was in some other place? So you have to do the ‘proper’ fix and ‘undo’ the original fix.
[Both] Yes MA: I actually did a BugSmashers video, yeah you know that show. RV: What? MA: Come on. RV: Never seen it. MA: I have to edumucate you after this. RV: Actually I think I’m on one or two of them. MA: No that’s your twin or something. RV: That’s my voice, maybe. MA: So yes all the time! An example would be going back with the new item system, we have these entities or items that they have – I love having this stuff, maybe it has this cup as what it looks like and its physics. So if I hits the cup, it reacts and it is just one thing that does need a slot where this geometry and physics go. We can have a number of slots, but for most part, we have one slot and our old items had one slot, but to get this interaction system up and running, we needed to have physics in different locations, maybe this opens a door, this one turns on power. So we had to have another physics thing, so now we have two slots in this entity and we have to physicalize both and it should have been as simple as saying negative one and what that would do is if it’s negative one, it would go through each slot and physicalize it. That was something we did about two years ago for the vehicle system, however it was working when we attached these items to ships, but then as soon as you detach it and put it on the ground, it had no physics and it was because of that change; everyone was like. “What’s going on? This used to work, this worked two years ago.” RV: It’s always kind of funny when bugs like that pop up because you guys are fighting old code and then me like, “Whoa, whoa, whoa, so we want this to happen, why is this happening?” and then like dig in. Sometimes I know you guys are like… one time he came over to my desk and was like, you had 12 different files open and you’re just like, “I just don’t know where it is”. MA: I’m just tracing back to the history- RV: Yeah MA: But ultimately I did find what was causing that and it was when it hit negative one it early aborted while the code back then was just missing and I literally was tracing back the history to find what happened because this code was there and I find it way back, I think a year ago it was an integration from a new engine version to some new tech for 64-bit and during that it removed some code and we never noticed this because we never had to do this type of thing. The vehicles ended up getting completely different code, so we never noticed it until we put this item in and then so yeah it was like, ”Alright he’s the old code that used to work”, and now when we plop this thing on the ground, it has physics and it’s just like [raises his arms in exasperation], the history is haunting me. It’s like I fixed this years ago! RV: And I don’t envy that at all. MA: But luckily, oh I love proforce, you can just scroll through the history and go where did it go? Where did it go? There it is! It’s called the blame history RV: So next one, actually for me is from…
SloanWarrior asks: When an impending release rears its head, roughly how much of the company would you say is devoted to getting the release out (Bugsmashing, testing, and so on)? Conversely, how much of the company isn’t impacted by the release cycle and simply continues to make content (SQ42 levels and so on)?
RV: This one right here is kind of a mixed bag and I can’t really answer it because this is a case-by-case, for instance, this release – we’re working on persistence, we’re working on shopping, we’re working on Item 2.0 here specifically and then we’re working on refactoring little things here and there and then the Art side, they’re actually working on the Caterpillar – and then Jeremiah and Omar are on more character stuff, Shane was working on a cool jetpack that’s supposed to be going in this release too. So every single person is working on tons of different things, tons of different aspects and they’re not necessarily all working on the same thing but they’re all working towards the goal of the release and every single release we have a goal. Like, “these are the main high points we want to put into this release” and then we go from there and it goes back to even bugfixing, it’s same thing with task fixing and management of going through and making sure “alright, what support does this task need?” and then going from there. So I know you’ve actually been pulled off, sometimes, from doing feature stuff cause I know you were working on seats, controller manager with some of our new engineers- MA: Yeah.. RV: And the links and stuff, and piping, and making sure all that stuff is situated but then Abent here gets pulled off and is like “Oh, well, 2.4 needs help with this. So I’m going to go help out with that.” so he will take a day or so here or there and Paul has been a madman, he’s been crazy. MA: He’s been breaking out those.. RV: Yeah dude- MA: It’s been crazy. RV: He’s been really helping out the 2.4 stuff and he’s been focused on that so that way his team can continue just focusing on the feature and- MA: Mhm. RV: But he’s a beast anyways like- MA: He is. RV: I mean, like, he- the amount of stuff he knows about the engine and the amount of stuff he knows globally is just insane. Like, my brain can only fit so much- MA: You could talk to him and say something and he’ll like “Give me a second, brain starts ticking and then spits out the answer” and you’re like “OK”. RV: Yeah. it’s- basically, everything, and that’s every release, it’s completely different. So the stuff we’ve got planned for 2.5 and going forward it’s going to need different resources and this is going to be spread across all the different studios that we’re working with, whether it’s Frankfurt, UK, Austin or here in LA. Even the guys in Turbulent and stuff, we’re all just working on different aspects of it and different things even for each release. So every release is it’s own little snowflake. MA: And to add on to that, I can be working on this subset of stuff from the new items things and then I can’t work on the next thing because I’m winning some from Patrick or even Chad so I take my code and put it on Preview and next thing I go “I’m blocked with this, is there anything I can help with?” and then I’ll just help with some 2.4 stuff or some other things and then once I get unblocked I’ll swap back. RV: And right now, I think I’m actually between you- between Paul, you and Chad Z now are the old people here, for engineers. MA: [nods] RV: You guys are basically the old crew and now we have a- MA: a new one.. RV: a new engineers are coming and then hopefully when we can Humphreys he’ll be kind of an old hat too because he knows the engine very well – he’ll be another senior here. MA: He’s not a Paul. RV: Yeah, It’s going to be crazy having you three together. Alright, hope we answered your question. Let’s go onto the next. MA: But I like rambling – I just got to give the details, man.. Last time… RV: I did. MA: I got this.
OverTurn asks: Can you tell us more about the tools you have developed internally and are in usage? For instance, a tool to balance the ships and weapons. If it doesn’t exist yet, are you planning to do it and with what approach?
RV: So, actually this is a good one like we talked about with John Pritchett before, the cool story. John’s actually been working on something, goal based tuning so that way it helps the designers input information into like a database, like a spreadsheet or something. All right, this is what we want the ship to do, then the system will extrapolate everything… all the information, kind of make it work with IFCS and everything like that and also kind of help the big ships… the Idris, the Javelin, all those also kind of work into it. MA: Yes, that is huge because his system is physically based so you needed to have actual proper physics, bunch of params, so designers are like, ‘what does this one do’… RV: Basically. MA: It’s like John has it all in his head… RV: I know, it’s scary. MA: He has all these valuables he knows of and it’s like, I just want to do this thing so he’s making a nice ingame tool to say, give me what you want and we’ll output those values. RV: I remember sitting down with John and I was like, ‘John, explain this to me’ and he was like, ‘Ok’. So, he explained it to me and I’m just like… jaw dropped, eyes glassed over and he was like, ‘too much?’, mhmm. I love talking to him, it’s funny, I’m like, ‘oh my god, all this stuff you’re putting into this is crazy’. So, he’s been in the UK for the past week or two, actually this is the second week he’s been in the UK, and he’s been working on the goal based tuning system before that… before he goes out there to train like Andrew Nicholson on how to basically tune it for balancing and everything like that. So, it’s been good for him to be out there so he can just sit down with the guys and be like, ‘all right, this is’… because tuning a ship used to take a week or two sometimes. Depending on how complicated things were and how thrusters were working, now they have it down to a few hours. So, it’s like a really big notable difference in balancing the ships now. MA: That too and it’s like, you hit different problems and because we’ve done it so many times, John knows which problems they’re going to be so he’s able to make these tools to help designers fix it and do it quickly. So, he’s got it down to science. RV: Yeah I know, he’s pretty good at just setting the designers up too and figuring out what they need. I remember on the like the first ship I was starting to work on, he was like, ‘ok, let’s get this thing flight ready’. I’m like, ‘ok’. Then he was talking about all the math and algorithms and trying to get the thrusters set up and I was trying to do the balance on the thrusters and I was like, ‘oh my god, so much stuff’. MA: He has a lot of hopeful things that just show the visual things. RV: Sometimes you want to just take a straw and suck the knowledge out. MA: His stuff is more, I guess, inengine debug screen stuff that outputs things but we do have other tools like DataForge or even the thing like Ariel’s working on… RV: Yeah, DataForge is a big help actually cause it’s going to start really cleaning up from doing like hand XML stuff and if anyone does like hand scripting, so that it’s like, if you have a tool it makes it so much easier to read and cleaner so that you can see where the issues are. MA: No more editing raw XMLs. RV: Going to be so awesome when it’s done. MA: We also have Ariel’s tool, the item port editor, which helps immensely cause you can see on the screen in game, your character and the ships and you can move/remove stuff and you can see it right then and there, and errors like, ‘hey this can’t fit there cause the size is wrong’. EV: He just seriously got the previous stuff set up too this past week or so. So it’s really good, like Sean Tracy, Mark McCall and the new guy, Linc… he’s actually going to be working with that tool and stuff so they’ve been giving him a lot of feedback. He’s been working side by side with them and all the tools are just… tools are awesome. People that can generate tools to be useful and stuff will just save time so we can develop more faster, better, a little better. MA: If we don’t have tools you can always just be Calix and write your own Python GUI system to do heat balance and yes he did do that. RV: He did. He did. MA: He was like, ‘hey look…’ RV: I’m prototyping this. MA: Ok. RV: Ok, at least he’s not flow graphing it. MA: He did. RV: I know but it’s like…flow graph. MA: He flow graphed the entire system and then once he figured out how he wanted it, then he made the GUI so he could explain it better to other people. RV: Yeah, cause the flow graph looked like spaghetti half the time. MA: Yeah. RV: I remember one time when he’d first done like the little thing for the… when you’re sitting in the cockpit and you can interact with things and then he like showed you guys the flow graph and you and Paul were like, ‘how does that work?’ and he was like, ‘I don’t know, it just works’. MA: He gets prototypes working great but they’re prototypes. RV: Remember trying to get his grabby hands stuff working in one of my levels and I was trying to mess with some of the cargo and some of the salvage and yeah… no, I was like, ‘Calix, I don’t understand this’. MA: It’s like, yeah that ain’t going to work, it’s a prototype.
RufusUltra asks: How was your transition from Dev to Producer, do you like your new job? Do you miss tinkering with the Caterpillar or do you enjoy running around with a stick and pock mark and the other devs, while demanding progress?
RV: Oo. My transition… my transmission is fine. Have you got yours checked out lately? MA: My what? I walk. RV: Oh forget it. MA: I power walk! I power walk every day. 30 miles! RV: 30 miles? That’s why you have calves like steel. MA: 30 miles up and 30 miles down. Why do you think I’m still up? RV: Nice. No the transition between dev to producer has … it’s still ongoing. I think I’ve been a producer now for three months, maybe … actually no, more like four months… MA: It feels like longer. RV: It feels like forever. MA: It feels like years! RV: Thanks. MA: I have that dread when you walk over, it’s like … RV: That’s right. “Get to work” MA: I’m like “No!” RV: “No keep …” Oh no, he had … MA: “Is there a Jira?” And then you’ll be like … [Randy holds up a piece of paper with “Jira” written on it] RV: What? There is a Jira sir! Yeah! Jira! MA: You had to do that didn’t you? He does this all the time. He’ll be like “Hey there’s this bug: have to do.” And I’m like “There’s no Jira: can’t do it.” And he’ll be like “Jira!” And I’ll be like “You… fine.” No, no. Then I’ll try to get out of it and go “Does my lead know?” And you’ll be like “Paul”… [Randy holds up a piece of paper with “He does know” written on it] RV: Oh, he knows! MA: And then Paul’s like [thumb’s up] and I’m like [sigh] “Fine, I’ll do it.” RV: Yeah, no, because, like… there always a problem in development, right? Where people will… since I come from a dev background and I’m transitioning over to Production it’s kind of like that whole thing. Dev’s will be like “I’m not doing anything a Producer says unless they have a Jira and okay finders document”. And I’m like “Yeah,I know these tricks. I’ve done them myself!” It’s like … kind of like … MA: You’ve turned against us. You’re a spy. RV: I’m a pretty open spy though. MA: Your tactics of spying needs a little work. RV: So, yeah, no, it’s like the transition’s still going. I really do enjoy it. Of course, there are some things, like I hate scheduling. Oh my god, I hate scheduling so much. Because it such… MA: Oh yeah. RV: In this type of environment, to schedule for such… it’s an asynchronous schedule because you guys are working on something completely different; the designers are working on something completely different. All the games that I’ve worked on before have all be regular MMOs or more standard MMOs, to where you have a zone and then you have all the quests that you are planning, you have everything else, so you have the progression. And you knew… we knew exactly what the progression was from level zero all the way to whatever level cap was. So it was easier for us to look at it. And I’ve always done production work in my other roles too because I’ve always been a Designer/Producer on some things. I’ve always made my own Jiras and stuff, I always had my own features that I was in charge of, so I’ve always run that. And when I was at 38 I think I had a team of nine people and I was the Scrum Master for the team, and I was running the features. I was in charge of the quest tool which, if anyone know about tools, like the quest tool itself touches everything in the entire game because everything is based on that. Then I had localisation stuff, I had some dialogue stuff, so I’ve always kind of done production stuff I’ve just never actually made it really official. MA: Yeah. RV: So it’s… kind of… it’s different. Different mentality: I’m still trying to… I still have that inner battle right now of like dev versus producer. But it’s also cool because I can go into meetings with you guys and… like one of the producers, when we were talking about the doors… the doors meeting, right? We’re talking all through the logic and everything like that and I understand it because I’ve done this before and I’ve been a dev for a while, but the other producer he’s like “I’m lost.” And I’m like “We’re just talking about doors.” He’s like “Oh”. MA: And then I think Sean walked up and he like, “Just made a door!” RV: Yeah, no, right? But even still I was able to talk logic and I’m able to give input and stuff and help facilitate that. So I have an idea of what the development team needs, granted I’m still working on it too because they always hit me up “Hey I want to do this, I want to do this!” “Okay.” Like the other day you… I saw you, I’m in a meeting and I see Abent just pulling a whiteboard and I’m like… I’m like “Hold on. What are you doing?” MA: I needed to schedule things. RV: So then we started working on that together and putting the task up there and then making sure that at least the people he need to work with, which right now you are working with Patrick a lot, so make sure that they’re always insync and they have one information and who’s doing what task. Since they don’t necessarily have same visions… visibility that I do. MA: We’ll have the high end Jiras and you’ll assign us like, “Hey make the seat” but there may be a bunch of things under that where I’ll have to work with Patrick and go “Alright, we have to plan this out of how are we going to do this. And to do we need any resources?” Like Kirk I needed some stuff him to do so we could get the new item stuff working. Alright so that’s another task, that’s a task, that’s a task. And it’s just like… RV: Actually to be clear. So a producer is never development’s boss. MA: Yes. RV: Their job is to facilitate and get… like to help them out to make sure they are not blocked so they can do their job. Because some people, they have the … that are outside the game industry they have notion of, “Oh, the producer’s the boss” and I’m like, “No, the Producer is never the boss.” Every time I make task it’s based on discussions with Paul and you and the rest of the team. And being like, “Okay, I’ll set up the meeting” and I’ll be like, “Alright guys, this is the goal that we want to do what are the tasks… what do we need to do with it?” And you, Paul and everyone else are like, “Okay, well to do that we need this, we need this, we need this.” Then I start taking the notes and I start jotting down all that stuff. Then I turn those notes into actionable items so that way you guys can do the work. But you guys already have a better idea of exactly what takes to do that work. I just try to make sure that work is visible to now a larger audience that way other people can understand what you guys are actually working on and doing. MA: And to track it. “What are they doing?” RV: Yeah. Mostly communication, tracking. MA: “I have no idea”. RV: Stuff! MA: Stuff. RV: They’re doing stuff. And things. MA: We’re making stuff work. With more stuff, well… RV: You keep breaking stuff. MA: You have to break it to make it. RV: Nice. I see the disapproving look by Hennessy over there. You saw that? He was like “Aw you people… “ MA: You mean QA buddy buddy? I… RV: You have him on speed dial. MA: Pretty much on my speed dial now. RV: Alright next question.
Far-Seeker asks: Question for Randy: For patches with the type of fundamental changes expected in 2.4, does the Live team do anything special in preparation or is the hand-off from the PTU and Q&A basically the same?
RV: So, this one right here – as mentioned before, every single- [laughs] MA: [raises hands] RV: …every single release- I know right- MA: If you don’t know, we’re being held by gunpoint with a Nerf gun. RV: We really are – and those things shoot fast too. MA: They do. RV: I’ve been put off on a tangent. [Both laugh] RV: So you’re like.. MA: That’s our stay off tangent machine. RV: So every single release there’s really no live team, there’s basically everyone and then based on the release they kind of pull from the resources and like I said before, basically Art, Design, Engineering – everything, they’re kind of like “this is what we want to do, this is the goal for this release” and then we pull the people who are in charge of those features and charge of that release and then they’ll do most of the work for each individual release and when we do handoffs, we do a handoff every day. And a handoff happens between the UK and here in the US and UK takes care of the stuff in Frankfurt as well, so they’ll have all those mix of bugs and then Alex Marschal over there will communicate with all of the producers over there and that way there’s main points of contact for each individual studio and then we’ll have meetings every day and now pushing out to live, or for pushing out to the PTU and stuff. Then what you have is more meetings because we’ll have our no-go- go/no-go meetings and we’ll talk with QA and we’ll talk with the main leads and be alright, “Is this good enough to go to PTU?” and sometimes it’s been completely crap but we needed to test some stuff, “hey, we just put this fix in, or we just put some debugging in” but we need to push this to PTU now” – so we’ll ask for a build and then QA will do a dirty-test of that build and then we’ll push it out to PTU anyways so just the people are on PTU they help us out and they go through and get everything situated so we can keep testing. MA: It’s like, “I know this is going to break this, but we really want to test that”. RV: So sometimes we’ve even disabled Arena Command so for like 2.4 stuff we’re testing on the PU right now, we don’t have Arena Commander working yet- MA: It’s only Crusader at the moment. RV: Because we want to really test the shopping, we really want to test that stuff and there’s enough crashes and breaks without introducing more things in there. So, yeah, no, we definitely do a handoff and it’s every day and it’s the communication between the different departments in QA and QA is really leading the charge in telling us what’s broken and what’s not. So I’m going to get shot. MA: I’m waiting, I’m just waiting. He’s waiting for us to get comfortable then he’ll shoot one and we’ll be like “What?!”. RV: I’ll do my girliest.. [gestures with arms flailing] MA: [Also gestures] MA: That’s not cool. Blocks the.. [Shot fires] RV: Oh geez. MA: Where’d he get you at? RV: I don’t want to say. [Laughs] RV: Right in my soul. MA: Oh no. RV: Alright, next question. MA: [Sways left and right] Ha! [Gets shot] Ugh.. RV: Ah, in ma squiggly spooch. MA: [Plays dead] He got me right where it hurts. Thomas Hennessey (TH): Continue.. RV: Alrighty… so, next question…
JimmyBeard asks: After each patch launch to the PTU, do our bug reports make a difference? I report bugs but have never received a message or email stating it was resolved or fixed does this happen?
RV: So again, yes. Every time you put in bug reports or using the issue council, anything you guys do using the forums, communities always going to that, QA is always pouring through that, design’s pouring through that, a bunch a people are always pouring through all the community stuff on the forums, council, like bug reports, everything! All the crash reports I know once the crashes get to a certain point, then all those crash reports are sent out to the producers and then we send out them out to our engineers so that way they can look at the crashes, get everything situated. So yes, everything you guys do, seriously, thank you. MA: Yes. RV: Because without you guys testing and helping us, actually find this stuff, we could not do our job and fix what is needed. MA: There can be bugs sometimes where we can’t even replicate, we need more information and when it goes out, maybe a ton of you guys get it and you put all the information about how you’re getting this thing and then all the information gets compiled up, sent into a gigantic Jira and it says, “This guy”, and it will link to what you guys have said and just like a big document of everything so we can try to scramble through what is actually happening, what people might think is happening and we’ve found and tracked down lots of bugs thanks to them. RV: QA is the one who gets those reports and then once QA gets those reports, then they go ahead and verify it, they fill out the information for like extra steps, alternate ways to get things, alternate ways because one of the ones recently – remember the spawning issue we had where like 50 something pirates were spawning and we could not repo that at all, but then people on PTU, they’re like “Hey this is happening”. So it took us a good three to five days to narrow that down and that’s between design and engineering, there in QA, those guys were really just hammering these different aspects. So yes, seriously thank you again for all your guys hard work, helping us make us Star Citizen better by testing, jumping in the PTU, jumping on Live, using the report system, using the forums, reaching out to us. It’s really, really appreciated on everything you guys do with this community. MA: And even if you don’t get a reply, we’re still more than likely using your information. It’s just our QA team is so small and they’re stretched wide, they don’t have really enough time to reply to everyone. So those that gather the information with what they can, get the report done so that the developer can get it and then if they have time, then they’ll reply – RV: Yeah, I don’t know MA: It’s really hard for them, but we do use a lot of information. I’ve seen information on Reddit, from the forums, anywhere. I’ve even seen posts on Facebook with a video. RV: Actually Youtube is really funny too because we’ve saw so many good things on Youtube.
RV: So seriously you guys, thank you guys for allowing us to do this-
RV: Subscribers are awesome. Normal studios don’t get the chance to do this – and this is actually really awesome, we get to be very open with you guys and answer your questions and take the time out of our day to again thank you guys because this is pretty awesome and we get the chance to do this.
MA: Without you guys we wouldn’t have Bugsmashers or whatever this show is.
RV: ..whatever this show is..
MA: 10 for the Randy
RV: That’s right, 10 for Randy.
RV: So yeah, thank you guys. See you in the ‘Verse.
MA: The what?
Enjoyed the transcript and want more content? Check out our latest preview on Hob from Pax 2016