Automated Testing for League of Legends. Hi, I’m Jim ‘Anodoin’ Merrill, and I work on test automation efforts for League of Legends, focused specifically on the in-game experience.
I currently serve as the tech captain to the Build Verification System Development (BVS-Dev) team. In large part, our team builds tools for automated testing and helps teams write better tests. For the past couple of years, we’ve been working on getting our test system and infrastructure up to snuff in order to increase developer efficiency and reduce the number of bugs we ship. We now run approximately 100,000 test cases a day, and automated testing at that volume helps get content to players sooner and with fewer bugs. I’d like to share a little bit of what we’ve done, and hopefully start a conversation about automated testing in the game space. Layers of Test Automation. Also referred to as the Test Automation Pyramid The intention of this post is to get across the idea that your testing strategy should include many layers of testing.
Layers of Test Automation. The Forgotten Layer of the Test Automation Pyramid. Even before the ascendancy of agile methodologies like Scrum, we knew we should automate our tests.
But we didn’t. Automated tests were considered expensive to write and were often written months, or in some cases years, after a feature had been programmed. One reason teams found it difficult to write tests sooner was because they were automating at the wrong level. An effective test automation strategy calls for automating tests at three different levels, as shown in the figure below, which depicts the test automation pyramid. At the base of the test automation pyramid is unit testing. Automated user interface testing is placed at the top of the test automation pyramid because we want to do as little of it as possible. All applications are made up of various services. Thinking Inside the Container. Containers have taken over the world, and I, for one, welcome our new containerized overlords.
They do, however, present interesting challenges for me and my fellow Rioters on the Pipeline Engineering team. My name is Maxfield Stewart, and I'm an engineer here at Riot. My team focuses on build pipelines—everything from code check-in to the artifacts we deploy, and sometimes beyond. Software Builds at EA: The 5000' View. A couple of days ago this tweet by John Carmack popped up.
In case the link ever goes away, he says, "Dealing with all the implicit state in the filesys and environment is probably going to be key to eventually making build systems not suck. " At EA we have spent a lot of time developing build systems that don't suck. They're not perfect but as we develop applications and games targeting platforms from Android to Xbox, plus everything in between, they work incredibly well for us. Framework. Spinroot. [Current Tool Version: 2.13 -- 26 October 2007] See also static.html for an overview of currently available static analyzers.
Uno is a simple tool for source code analysis, written in 2001. It has two main goals: George Dinwiddie's blog » How easy is it for your programmers to fix problems? A programmer, writing some new code, looks into some existing code that she needs to use.
Something doesn’t look quite right. DevOps Kata - Single Line of Code - devopsy. Code Kata is an attempt to bring this element of practice to software development.
A kata is an exercise in karate where you repeat a form many, many times, making little improvements in each. The intent behind code kata is similar. Since DevOps is a broad topic, it can be difficult to determine if a team has enough skills and is doing enough knowledge sharing to keep the Bus Factor low. Unit testing part 1 – Unit tests by the book. Www.rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf. What is Jidoka ? Otug mailing list archive for March 99: (OTUG) RE: Incredible XP. Meta Build Systems. This is a story about my search for a hassle-free cross-platform open source (meta-/meta-meta-) build system.
For our open source Bullet physics engine, I’ve been distributing the source code in a way that should make it as easy as possible to build out-of-the box. This means that for all supported platforms, the user (a developer who downloaded the Bullet SDK) should be able to download and unzip the zipfile or tarball on his machine, and get started as soon as possible. It should be hassle-free for the user but also for me, so I rather don’t manual updating too many files for each release. Aside from all the different platforms, we need to also support various compilers, compiler settings and integrated developer environments (IDEs).
Common platforms are Windows, Mac OSX, iPhone, Linux as well as PlayStation 3 and XBox 360. Here are a couple of build systems and modifications that I tried or considered to try: Visual Studio project files. What is continuous integration (CI)? - Definition from WhatIs.com. What is continuous integration (CI)?
Continuous integration (CI) is a software engineering practice in which isolated changes are immediately tested and reported on when they are added to a larger code base. The goal of CI is to provide rapid feedback so that if a defect is introduced into the code base, it can be identified and corrected as soon as possible. Continuous integration software tools can be used to automate the testing and build a document trail. Continuous integration has evolved since its conception. Originally, a daily build was the standard. According to Paul Duvall, co-author of Continuous Integration: Improving Software Quality and Reducing Risk, best practices of CI include:
Continuous Integration. Pitfalls In Automation. The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community. Chrome_Release_Cycle_12_16_2010.