The Push from Within: A Journey of Transformation

Abstract
Quality. We all realize it is important, yet so many companies are unwilling to invest the time to achieve it. Words like agile, startup mentality, and fail fast take precedence over quality, causing churn across organizations worldwide. Sound familiar? If you are a change agent for quality, yet find yourself frustrated by the lack of support from leadership, this talk is for you.

Claude Jones, Sr. Director of Engineering, provides an inside look of what it took to push from within and help one of the world’s largest companies adopt a culture of quality. This presentation will cover:

• Identifying the right project
• Building the right team structure
• Removing the fear of change
• Providing value with the right metrics
• Making testing fun
• Reinforcing the benefits

This case study will challenge you to re-think your approach or empower you to stay on your current path of adopting a culture of quality.

Walmart Labs is the tech arm of Walmart with the mission to provide a seamless omnichannel experience for customers whether shopping online or in-store. The technology solutions built by Walmart Labs serve over 2.6M customers weekly supported by 1.5M employees in the U.S. alone. At that scale, it is critical a focus on quality is part of their DNA.

Bio
Claude Jones is a seasoned technical leader with over 15 years of experience. He has been with WalmartLabs for 6 years and is responsible for managing a distributed team of 60+ product, user experience, project management, architects, and engineers focused on building custom tools to improve the way Walmart develops and delivers eCommerce experiences across the web, native app, and stores.

Claude has a passion for bringing out the best in others and believes that anything is possible with hard work and dedication.  In his spare time he runs the Elevate Foundation, that is an organization focused on helping others in need and giving back to the community.

Opening Keynote

Bio
With Forrester since July 2005, Diego primarily contributes to and advises on Forrester’s offerings for Application Development & Delivery Professionals. He partners with Forrester’s global application leaders and is a leading expert on SDLC processes and practices, covering topics such as Agile development, Agile and Lean transformations, Agile development sourcing strategies and services, Agile testing practices and tools, DevOps, and software testing and quality, with a key focus on systems of engagement. Diego also covers software delivery metrics, artificial intelligence, and open source governance.

His 28 years of industry experience, in addition to application development, allow him to give expert advice on change management programs for optimizing the overall modern application delivery process, execute technology management assessments, review technology management strategies, and make comparisons. He also has experience in complex mission-critical project and client engagement management.

Thursday AM Keynote

Bio
Christine McCarey is a recognized diversity, equity, and inclusion (DEI) leader, public speaker, and recovering lawyer. She founded ImpactDEI to effect positive change at scale. Her company helps organizations embed DEI initiatives into their processes and culture. Before founding ImpactDEI, she founded and led RetailMeNot’s D&I Program with the support of committed colleagues. Under her leadership, the Program was recognized by Glass Door as #1 and Built in Austin as #2. She also founded and led RMN’s D&I Council, Women@RMN (an employee resource group), and Legal Operations. Before RMN, Christine held business development and legal roles, including as General Counsel.

Continuous Deployment for ‘Mature’ Codebases

Abstract
When starting up a greenfield project, it’s easy to take advantage of the most modern development practices. But what about the rest of us, who are working on codebases greater than five minutes old? How do you take code that’s four years and hundreds of thousands of SLOC, and turn that into a lean, mean, continuous-deploying machine? In this talk, Melissa will walk through what continuous integration and deployment means for teams working on mature code bases, and what the roadmap looks like to get from a release cycle that may take weeks or months to one that deploys on-demand.

Attendees can expect to come away with:

• Roadmap template for transitioning to CI/CD
• Common pitfalls when migrating old code
• Understanding how and when to declare ‘victory’ for a migration

Bio
Melissa Benua has worked in nearly every software development role—dev, test, DevOps, and program management—at companies as big and small and somewhere in-between. She’s created and run high availability, high-quality services for PlayFab, Bing, Cortana, and Xbox One, and now for mParticle. Melissa discovered her love of massively-scaled systems while growing the Bing backend, where she honed the art of keeping highly-available complex systems up while undergoing massive code churn. Now an engineering manager with mParticle, she’s passionate not only about maximizing efficiency both in her product code and in her developer tools but also about sharing best practices among colleagues and the tech world at large.

Pyramids Are Ancient – Let’s Talk Automation Strategy

Abstract
‘Richard, what tests should I automate?’ ‘What should I be automating?’ ‘What layer should I be automating on?’ ‘Where do I start with automation Richard?’ These are questions I get asked regularly. My answer every single time? ‘Checkout The Test Automation Pyramid!’, ha, of course, it’s not, it’s ‘It depends…’ That’s because there isn’t a single correct answer to any of those questions, although most are looking for one. It certainly isn’t a picture of a triangle.

However, there are steps we can take, questions we should be asking, which will lead to answers. They won’t be the right answers, but they will be enough to progress, while we learn more and continuously adapt. These questions will allow you to map out your context, arguably the most important action to maximise your automation efforts. Knowing when to ask these questions, how to progress after the answers and who to involve are the most underrated skills in automation. You can learn all the programming languages, all the frameworks and tools, but if you don’t know how to audit your own context and frame your problems, they are irrelevant.

In this talk, I’m going to share the questions I ask in order to understand the context I’m in, allowing me to create an automation strategy. We’ll then compare those questions and answers to existing models highlighting the flaws in them, but also where they can provide some direction.

Takeaways:

• Understanding the importance of having an automation strategy
• A set of questions to help you understand any context and formulate an automation strategy
• How to evaluate and challenge existing automation models

Bio
Richard Bradshaw is an experienced tester, consultant, trainer and generally a friendly guy. He shares his passion for testing through consulting, training and giving presentations on a variety of topics relating to testing. With over 12 years of testing experience, he has a lot of insights in the world of testing and software development.

Richard is currently the BossBoss at the Ministry of Testing, co-creator of the Automation in Testing (AiT) namespace, blogs at https://automationintesting.com

Building Automation Engineers From Scratch

Abstract
Creating automation engineers from manual testers is hard. Even if testers are willing, they have a lot of hurdles to get over to feel like the same kind of subject matter experts in automation as they are in manual testing.

As a career-long manual tester making the leap to automation, Jenny Bramble has experience to explain frustrations and provide solutions. She will discuss managing the expectations of  testers and their managers (what’s the time frame? Why isn’t this working?), techniques for teaching (such as games! Pair/mob programming! Software fundamentals!), and how to know when testers have made it (what should manual testers be aiming for when they start?).

You’ll walk away from this talk with a powerful new set of tools in your toolbox:

• The basic framework your manual testers need to be successful—including how to determine where the gaps in knowledge are and filling them.
• Advice on managing the expectations of your testers and management from time constraints to what success looks like.
• Several methods for teaching framed around a case study of a team that built itself up from the inside out and is running a successful automation suite.
• Facing and overcoming other challenges such as ability and perceived ability, resources, time, tooling, and how to get your team excited for a new chapter in their professional development.

Bio
Jenny Bramble came up through support and DevOps, cutting her teeth on that interesting role that acts as the ‘translator’ between customer requests from support and the development team. Her love of support and the human side of problems lets her find a sweet spot between empathy for the user and empathy for the team. She’s done testing, support, or human interfacing for most of her career and is excited about the future of automation.

Trust Issues with UI Automation

Abstract
UI Test Automation is notorious for stability issues. UI automation is almost synonymous with “flaky test.” In this presentation, Bijoya Chatterjee discusses taking a proactive strategy towards test automation flakiness as opposed to a reactive approach of looking at failures and then deciding the course of action. How can the test automation community use and develop tools that helps eliminate flakiness and also identify flaky tests not only before they are run, but even before they are merged? How can we create these tools in a service model that plugs into our Continuous Integration pipelines?

Bio
Bijoya Chatterjee is a senior QA manager at Sony PlayStation and has been working in UI automation for 14 years. She has been leading quality and automation activities at Sony and over the years, has gathered a lot of experience in UI automation and solutions to make it more stable.

Utilizing Component Testing for Ultra Fast Builds

Abstract
A best practice of software architecture is to design your applications into independent modules or components, with a published contract for interaction between components. This is a principal part of the popular microservice style architecture, but also it applies to components created in a large monolith or with the out-of-the-box patterns that front end libraries promote.

Being able to test the functionality of the component independently and apply a level of trust that those components work then opens the door to rethinking an organization’s continuous integration and continuous delivery strategy. This in turn can potentially reduce the need for long test suites in many environments. It will also cause a rethinking of how to structure the unit testing strategy and the test pyramid.

Attendees of Tim’s talk will be equipped to:

• Understand the different kinds of component testing
• Know how/when to apply each type of component testing
• Understand the effect on your organization’s broader testing strategy

Bio
Tim Cochran works for ThoughtWorks NYC, and has been a full stack developer and architect for over 15 years, working on everything from large distributed enterprise projects to small NGO visualization apps. Most recently, Tim has been helping companies with digital transformation, moving towards continuous delivery and building a DevOps culture. He also is an automated testing zealot who practices TDD almost exclusively.

Automated Testing at the Speed of Headless

Abstract
Testing is shifting left. You are a developer who wants instant feedback on the quality of your code. Or you are a tester who wants to incorporate tests earlier in the pipeline without slowing down development efforts. Sauce Labs is great for cross browser functional testing, but using full browsers earlier in the pipeline can be clunky, and too expensive for the volume needed.

Enter Sauce Headless – a new solution from Sauce Labs that allows you to run your automated tests at speeds you used to only dream of! Using headless Chrome and Firefox on Linux containers, Sauce Labs is the first in the market to offer early pipeline testing infrastructure. The result is a fast and lightweight solution that is cost-effective for continuous testing at scale.

Join Sam Coffman, Product Manager at Sauce Labs, and Alissa Lydon, Product Marketing Manager at Sauce Labs, for a highly interactive session where they will discuss:

• How Sauce Labs is using container technology to create the industry’s first cloud-based headless testing solution, making it a fast and affordable option for early pipeline testing at scale
• The benefits of using  Sauce Headless in conjunction with our traditional VM offering to increase the speed you can deliver quality code
• How easily Sauce Labs customers can set up their own headless instance in Sauce, without having to build and maintain an infrastructure themselves
• Real stories of customers who are experiencing upwards of 50% faster test times while using Sauce Headless

Bio
Samantha Coffman is a Product Manager at Sauce Labs who manages the Virtual Desktop Cloud team, and is responsible for browser and OS updates as well as the introduction of the new Sauce Headless offering. Samantha joined Sauce a year ago and is energized about the direction in which desktop testing is heading! In the office, Sam loves to make charcuterie plates for her coworkers and play with all the office pups. Outside of work, you can find her traveling to a new city, exploring the food scene in the bay area, and spending time outside.

How I Learned to Stop Worrying and Love Record and Playback

Abstract
Record and playback tools like Selenium IDE have a stigma attached to them in our industry— they’re not worth your time, they produce tests that are impossible to maintain, and they’re only for non-technical people.

But with the new Selenium IDE, that’s all just a bad rap.

In this talk, Dave Haeffner will squash the stigma by showing you examples of how Selenium IDE can reliably augment and level-up your testing practice regardless of your team’s test automation maturity or your level of technical experience. He’ll also share how he went from a record-and-playback naysayer to a Selenium IDE maintainer.

Bio
Dave Haeffner is the writer of Elemental Selenium — a free, once weekly Selenium tip newsletter that’s read by thousands of testing professionals.

He’s also the creator and maintainer of the-internet (an open-source web app that’s perfect for writing automated tests against), author of The Selenium Guidebook, and an active member of the Selenium project.

As a Software Developer at Applitools, he works full-time as a maintainer of Selenium IDE.

He’s helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He’s also spoken at numerous conferences and meetups around the world about automated functional testing.

AI Testbot As A New Sauce For Continuous Testing Recipe

Abstract
Software QA has been burdened with laborious manual testing for decades. Test automation technologies significantly reduce the repetitive manual work associated with testing, but additional assistance is needed to achieve what’s truly required for Continuous Testing. With recent advances in AI and deep learning, we can build an AI surrogate—a testbot—to address the Continuous Testing pipeline from a new perspective…just like the chatbot is innovating the call center communications. Join this session to explore “the Good, the Bad, and the Ugly” faces of a few “testbot” implementations. Here are a few key takeaways that attendees can expect:

• A description of AI Testbot
• Discussion of why you need AI testbot in continuous testing
• The good, the bad, and the ugly of AI Testbot
• A practical use case: Add a new kick, AI Testbot, to an already existing Sauce Labs/Mobile testing recipe

Bio
Jae-Jun Hwang has more than 20 years of engineering experience in software development and research in large-scale cloud services and intelligent IT infrastructures. He is currently founder & CEO of AppTest.ai, which provides the industry leading self-testing service, a.k.a Testbot, using AI and deep learning to innovate continuous mobile testing domain.

Fashionable XCUITest for iOS Apps

Abstract
Apple is investing heavily in testing tools and techniques. They introduced the Swift programming language in 2014 and launched the Xcode UI Testing framework in 2015. Traditional tools such as Appium and Calabash don’t fit well in native app development with Swift. The patterns like Page Objects or Screenplay may work somehow, but they don’t fit in the protocol-oriented way of Swift. However, we can use a protocol-oriented approach to architect XCUITests that can be scaled easily within iOS CI/CD pipelines. In this talk, Shashikant Jagtap will cover the following topics:

• Introduction to XCUITest
• XCUITests Features with Xcode 10
• Protocol-oriented approach for the XCUITest
• Tips for writing CI-friendly XCUITests. e.g Stubs, Accessibility Identifiers, Real Device Tests, Xcode scheme strategy for UI Tests etc.

At the end of this talk, attendees will learn new features of XCUITests along with tips and tricks to use UI tests intelligently in iOS apps.

Bio
Shashikant Jagtap is a DevOps practitioner for mobile apps and Director of XCTEQ Limited. He has automated release pipelines, implemented CI/CD and enabled DevOps Test Automation practices for many apps. He blogs about mobile DevOps on “XCBlog” (Medium, iTunes), and also blogs at www.shashikantjagtap.net.

Using Headless In A CD Pipeline

Abstract
Learn how the RentPath team is using headless in their CD pipeline for rent.com. They run tests against each push to GitHub in headless, leveraging the lower cost of headless containers through Sauce Labs for greater concurrency. For multi-browser testing, the team relies on traditional Sauce Labs VMs for nightly regression runs and to check new builds more thoroughly prior to release.

In this presentation, attendees will understand:

• The speed advantage that headless testing provides
• The challenges the RentPath team has overcome and
• What tweaks they’ve had to make to the tests for headless

Bio
Abdur Jubair is a passionate programmer who enjoys learning new technologies. Abdur joined RentPath in 2015, and since joining, he has held multiple QA roles including manual testing and backend API testing. Currently, Abdur is working as an Automation Test Engineer using Nightwatch.js. In the past two years, Abdur has worked with the RentPath team on implementing CI/CD pipeline. Before joining RentPath, he worked as a manual tester for several years, working for companies like Sage Software and Allconnect. Outside of work, Abdur prefers spending time with his family and playing cricket.

Finding Your Community

Abstract
We all know that it’s dangerous to go alone.

Julie will give her experience and lessons learned in starting new meetups, building and scaling communities, and creating culture.

Coming from a non-traditional background of support/operations and being “thrown into” QA, Julie began her journey overwhelmed by information overload, not quite sure which organizations and resources were a good use of time.

Whether you are a new QA engineer trying to maximize your time out of a myriad of choices, want to put yourself out there but aren’t sure how to contribute, are a lone tester in your organization having to seek support outside your company, or are trying to start a community from scratch because your area is lacking, this session is for you.

Julie will present on lessons learned, tips and pain points in her first year of creating communities and meetups from scratc

Bio
Julie is one of the co-organizers of Austin Automation Professionals and a member of several local meetup communities. She has approximately five years QA experience and is currently at Keller Williams. She was previously at AT&T for eight years in various roles such as QA, NOC and management. Julie holds a Masters in Business Administration and Bachelors in Information Systems Technologies from Southern Illinois University.

Automated Testing at the Speed of Headless

Abstract
Testing is shifting left. You are a developer who wants instant feedback on the quality of your code. Or you are a tester who wants to incorporate tests earlier in the pipeline without slowing down development efforts. Sauce Labs is great for cross browser functional testing, but using full browsers earlier in the pipeline can be clunky, and too expensive for the volume needed.

Enter Sauce Headless – a new solution from Sauce Labs that allows you to run your automated tests at speeds you used to only dream of! Using headless Chrome and Firefox on Linux containers, Sauce Labs is the first in the market to offer early pipeline testing infrastructure. The result is a fast and lightweight solution that is cost-effective for continuous testing at scale.

Join Alissa Lydon, Product Marketing Manager at Sauce Labs, and Sam Coffman, Product Manager at Sauce Labs, for a highly interactive session where they will discuss:

• How Sauce Labs is using container technology to create the industry’s first cloud-based headless testing solution, making it a fast and affordable option for early pipeline testing at scale
• The benefits of using  Sauce Headless in conjunction with our traditional VM offering to increase the speed you can deliver quality code
• How easily Sauce Labs customers can set up their own headless instance in Sauce, without having to build and maintain an infrastructure themselves
• Real stories of customers who are experiencing upwards of 50% faster test times while using Sauce Headless

Bio
Alissa Lydon is a Product Marketing Manager at Sauce Labs. During her four years at Sauce, she has worked with companies of all sizes in their journey to automation and continuous testing. From educating potential customers about the benefits of test automation, to organizing educational events for QA and development teams to learn more about industry best practices, her goal is to leverage her experiences with Sauce Labs’ customers to spread the word about how others can succeed with testing. Outside of work, you can find her cuddling with her dog, traveling to far away lands with her family, or catching a baseball game (Go A’s!)

Merrill’s Journey to CI/CD and Continuous Testing

Abstract
Merrill was faced with the challenge of transforming product delivery to push new features or fixes to production every day without sacrificing on testing and validating quality before introducing changes. They achieved this by implementing an automated testing framework and automating regression testing suite of 600+ scenarios that could run automatically and in parallel across platforms/browsers using Sauce Labs, which helped them be a gating decision for go live into Production.

In this presentation, Ashish will discuss:

• How to achieve maximum parallelism of the automated scenarios.
• How to use API calls for the fastest test data setup and teardown, and how Merrill used those to keep tests atomic and autonomous
• How the team decided which manual test cases to automate, including examples of how they decided to rethink and automate 600+ scenarios

Bio
Ashish Mukherjee is a software engineering professional with more than 25 years of progressive experience in software engineering including Microservices development, full stack development, product engineering, architecture, QA, project and operational management, web and mobile applications, middleware and integration technologies.

Accelerating Innovation at Charles Schwab with Continuous Testing

Abstract
Charles Schwab transformed the quality assurance process for its Intelligent Portfolios robo-advisory platform product team by adopting a “shift left” continuous testing approach and validating quality through automation. In this presentation, Bala Mukund and Michael Schober will detail how they led this transformation of Schwab’s Wealth Management Technology team with the help of automated continuous testing via Sauce Labs. Delegates will learn how to streamline the software development process, practical processes for implementing continuous testing and how to reshape DevOps culture to better meet customer needs and expectations.

Bio
Bala Mukund is a Director of Technology for Charles Schwab with over 15 years of experience in managing, designing, and building enterprise level applications ranging from finance, sales, marketing, and banking systems. In the current role, Bala is the Technology Director and Solutions Architect for the Intelligent Portfolios platform – which provides automated investment advice. Bala worked closely with the IP technology teams to establish best development and quality automation practices leading to improved quality and delivery. Prior to joining Charles Schwab in 2015, Bala was a Technology Director at Morningstar and VP at Nationwide, in Chicago.

How Analytics Became Our Friend In Testing Web Responsiveness

Abstract
There is a huge combination of browsers and devices that require testing. We need to selectively categorize them into groups based on the statistics from analytics, and run them on Sauce Labs based on the categorized group. The statistics may be surprising. Teams may need to test the OS/browser combinations that they considered less important, but spend less time on the OS/browsers that they thought were important. Analytics is your friend!

Bio
Peter Nguyen is a QA and automation lead for Autodesk Virtual Agent (AVA), a new chatbot at Autodesk. He has 10 years of experience with automation and has held previous positions in software engineering. Peter is passionate about leveraging technologies to get things done instead of doing them manually.

BDD with Selenide

Abstract
What is Selenide and how does it hasten testing? Agile teams tend to struggle in getting development and testing in sync. Many teams run minified waterfalls, where testers get working code a few days before the end of the sprint. With Selenium these teams are ending up writing lot of boilerplate, buggy automation code, which will eventually become a nightmare when it comes to maintenance.

Selenide is an open source library that can make a huge impact and accelerate software delivery by introducing concise API, shorter expressions and many other capabilities.

Attendees of this session will leave with a good grasp of the following:

• Key features of Selenide they can use to make their automation code simpler, easier to read and more maintainable
• The most effective test automation design patterns with Selenide
• Selenide support for Angular based applications and other advanced Javascript libraries
• How to use Selenide with Cucumber for BDD
• Execution and generation of reports

Bio
Hima Bindu Peteti is a former developer who moved to test automation to focus on her passion. She is currently working as a Lead SDET in FannieMae. Bindu has worked on multiple open source automation frameworks that provides application modeling, unit/integration/functional/visual testing, and developing solutions. Those frameworks empower QA to shift closer to developers by moving beyond typical functional testing into the close quarters of unit testing.

Closer To The Metal: How and Why We Use XCTest and Espresso

Abstract
In this talk, Mario will describe the practices that ABN AMRO adopted in mobile teams when it comes to testing native applications on real devices. Since using Espresso and XCUITest is still relatively uncommon for large apps and there are various unique challenges due to being in an EU-regulated industry with various security restrictions, Mario will share the ABN AMRO team’s experiences, including:

• A brief architectural overview of the Mobile Banking app: why it is all native (Objective-C/Swift and Java/Kotlin), how it communicates with other apps and websites (what we call the Constellation approach)
• Why they choose to adopt Espresso and XCUITest: what are the advantages and the limitations of this choice
• How they run the test pipelines to spread them across time and devices and prevent teams from being blocked
• How they make tests reliable: faking network calls (pros and cons) or running an always up-to-date mock server on demand on AWS (why this is preferred when possible and what are the instances where it is not possible)
• Shortening integration test run time: skipping the device binding procedure (and any in-between long process in general) except where strictly needed
• Challenges: interactions across multiple apps, limitations of the test frameworks, dependencies on external hardware, problems testing with real data, automated penetration testing, web in app, web & app

Bio
Mario Negro Ponzi is part of the Centre of Expertise for Software Development at ABN AMRO. He started in the early 2000 developing on mobile phones (in Motorola, on the predecessor of the RAZR V3). In 2006, he moved to Amsterdam for TomTom where he worked first on Windows Mobile and then brought TomTom to the iPhone as soon as the SDK came out. Mario left TomTom in 2010, worked in a couple of startups until 2015 and then joined Philips on an app for medical monitoring of patients. Due to long commuting times, he left Philips at the end of 2016 to join ABN AMRO.

Welcome to SauceCon 2019

Bio
Charles Ramsey has been the Chief Executive Officer of Sauce Labs Inc since April 2015 and served as its Chief Revenue Officer from February 2015 to April 2015. Charles has 25 years of industry experience. He was a Venture Partner at JMI Equity. Prior to joining JMI Equity in 2005, Charles held a number of roles at Quest Software, Inc. including Vice President of Marketing and Sales. He served as Vice President of Sales at Computer Intelligence, and was employed in sales at IBM. Charles has served as a Director of ServiceNow, Inc. (Formerly, Service-now.com, Inc), and now serves as a Director of Configuresoft, Inc. Charles has a Bachelor of Arts from the University of California, San Diego and a Master of International Management from the American Graduate School of International Management.

AI Testbot As A New Sauce For Continuous Testing Recipe

Abstract
Software QA has been burdened with laborious manual testing for decades. Test automation technologies significantly reduce the repetitive manual work associated with testing, but additional assistance is needed to achieve what’s truly required for Continuous Testing. With recent advances in AI and deep learning, we can build an AI surrogate—a testbot—to address the Continuous Testing pipeline from a new perspective…just like the chatbot is innovating the call center communications. Join this session to explore “the Good, the Bad, and the Ugly” faces of a few “testbot” implementations. Here are a few key takeaways that attendees can expect:

• A description of AI Testbot
• Discussion of why you need AI testbot in continuous testing
• The good, the bad, and the ugly of AI Testbot
• A practical use case: Add a new kick, AI Testbot, to an already existing Sauce Labs/Mobile testing recipe

Bio
Asaf is Director of Product Management at Sauce Labs. Previously Chief Product Owner at Perfecto Mobile, Founder and CEO at TenKod and Area Product Owner of Software Engineering at SAP.

Accelerating Innovation at Charles Schwab with Continuous Testing

Abstract
Charles Schwab transformed the quality assurance process for its Intelligent Portfolios robo-advisory platform product team by adopting a “shift left” continuous testing approach and validating quality through automation. In this presentation, Michael Schober and Bala Mukund will detail how they led this transformation of Schwab’s Wealth Management Technology team with the help of automated continuous testing via Sauce Labs. Delegates will learn how to streamline the software development process, practical processes for implementing continuous testing and how to reshape DevOps culture to better meet customer needs and expectations.

Bio
Mike Schober is a Managing Director in Technology at Charles Schwab leading the software development and delivery of Schwab Intelligent Portfolios an innovative automated investing platform. Prior to Schwab he worked at large companies and startups with a personal driving force to deliver new and emerging technology solutions, including one of the first internet payment systems, an early one-click shopping experience, a first of its kind internet payment chip enabled credit card, and more. Mike has a passion for leading edge industry changing technology innovations and takes pride in delivering solutions that improve individual’s experiences and lives.

How We Got To A CI/CD World: A Vanguard Testing Journey

Abstract
CI/CD is all about speeding up your feedback loop, and one great way to do that when developing a microservice economy is through Contract Testing.

Vanguard has gone through many changes over the past 10+ years. Craig Schwarzwald will be briefly discussing each main phase their testing has undergone; from thousands of manual tests, to thousands of automated tests with Selenium, to Shifting Left, to now focusing on Contract Testing.

Craig will then discuss what Contract Testing is and talk about its importance in the CI/CD pipeline (giving the team virtually E2E coverage at the speed of Unit tests).  Each person should be able to relate to one, or many, of the Vanguard team’s main testing phases and see how to get to the next phase in their CI/CD journey by following some of the same steps that Vanguard did. The audience should also leave with:

• An understanding of what Contract Testing is
• How to implement Contract Testing
• How Contract Testing can help them get to a CI/CD world like the Vanguard team

Bio
Craig Schwarzwald has more than a decade of professional software development, scripting, and automation experience. Craig has worked on implementing Contract Testing directly within his organization’s CI/CD pipeline which is automatically utilized for all new microservices being developed within the company. Most recently he has taken on a role as an Automated Testing Coach where he helps teams with all of their most pressing and challenging aspects of unit, integration, and end-to-end testing.

Overcome The Challenges And Limitations of Android App Automation With Espresso

Abstract
One of the most popular automation tools for mobile apps—Appium—comes with challenges and limitations. Some of these challenges have included the following:

• Tests take a long time to run
• No support for testing activities outside the app
• Test flakiness
• False positives/negative results

Jagmit Singh found that Appium challenges/limitations were resolved on switching over to Espresso Framework. Espresso UI test automation framework is Google’s de-facto testing platform for Android app developers. There are a few key motivators to use Espresso. First of all, Espresso tests are within the application and you can mock certain layers of app like a white-box testing. Also,  Espresso has unique synchronization method in its core making the tests less flaky.

During this talk, Jagmit will demo how to:

• Set up Espresso
• Develop/execute tests and identify elements
• Execute the Espresso tests in a Continuous Integration environment using Jenkins and Firebase

Bio
Jagmit Singh has more than ten years of experience in QA, Release Engineering and Software Development. He has extensive experience in both mobile and web automation using Selenium, Appium, Espresso, Robot Framework, Java and Python. He is currently working as Senior Automation Engineer at CBC Canada, developing the automation framework for the iOS and Android native apps. He also advises other teams in implementing automation strategy and best practices. When he’s not working on automation, Jagmit enjoys playing racket sports, hiking and skiing.

Achieving Continuous Integration (CI) Excellence through Quality Engineering

Abstract

The session will feature an MC and a panel of customers and industry experts attending the conference.

They will address the following questions from the audience, pre-polling questions, and live polls:

• Journey to CI/CD with Docker / Kubernetes
• Continuous testing
• Best practices (test script creation, execution, optimizing code, and more)
• Organizational transformation: dedicated QA engineers or developers
• Thoughts on new open source testing tools, scriptless, and visual QA
• Leveraging analytics to measure success
• Tips, what to focus on automating the right things, not everything
• Characteristics of test script maintenance

Bio
Greg Sypolt is a Director of Quality Engineering at Gannett | USA Today Network. Greg has a blend of a developer, quality, and DevOps mindset allowing him to bridge the gaps between all the team members to achieve the desired outcomes. Greg helps shape the organization approach to testing, tools, processes, continuous integration, and support development teams. He’s an advocate for automating the right things and ensuring that tests are reusable and maintainable.

Using Headless in a CD Pipeline

Abstract
Learn how the RentPath team is using headless in their CD pipeline for rent.com. They run tests against each push to GitHub in headless, leveraging the lower cost of headless containers through Sauce Labs for greater concurrency. For multi-browser testing, the team relies on traditional Sauce Labs VMs for nightly regression runs and to check new builds more thoroughly prior to release.

In this presentation, attendees will understand:

• The speed advantage that headless testing provides
• The challenges the RentPath team has overcome and
• What tweaks they’ve had to make to the tests for headless

Bio
John Theodore has worked at RentPath as an Automation Test Engineer since 2015. In that time, he has worked with two different frameworks and languages – one based in Ruby and the current stack based in Nightwatch.js. Over the past two years, he has worked with Abdur and the rest of the Rentpath team to implement a CI/CD Pipeline. A self-taught programmer, he took an unconventional path to QA Automation, majoring theater at the University of Southern California with a minor in Video Game Design and Management. After graduation, he spent several years as a manual tester, working on major titles like Call of Duty and Guitar Hero as well as some lesser known and unreleased games. Outside of work, John has always been an avid NASCAR fan and hosts a sim racing stream on twitch – twitch.tv/john_a_theodore.

Test Armada: Evolution of a Value-based Test Automation Platform

Abstract
In this talk, Pranav Verma will discuss the ins and outs of Test Armada, an end-to-end non-opinionated test platform that the Walmart Labs team developed for web, native, and backend services with actionable reporting capabilities. Test Armada runs more than 100K tests every day at Walmart Labs, with significant external users contributing and benefiting from this open source platform.

This talk is ideal for anyone responsible for or curious about learning how to build a successful automation framework for a large organizations/open source communities. In this talk, Pranav will cover:

• The evolution of Test Armada
• Details of the Test Armada platform
• The core values of Test Armada, including case studies:
• Actionable Insights: Test reports should provide detailed insights into the failure to reduce, if not eliminate, debugging time.
• Non-Opinionated: Do not make choices for your users, compliment their current automation solutions.
• Reliability: Test results should always be reliable, and only fail due to real bugs in application code or setup.
• Speed: Test should always run fast, and scale well to fit the needs of a large organization.
• A demo of Test Armada
• Test Armada Benefits with Stats
• Test Armada roadmap

Attendees will the this talk with:

• An understanding of how and why Test Armada was developed
• Ideas for how to build an automation framework to fit a large organization
• Practical advice that will help with your current automation solution

Bio
Pranav is a Senior Engineering Manager at Walmart Labs.

How to Measure Success in Continuous Testing

Abstract
For many organizations, evaluating their progression in regards to continuous testing is mostly a subjective matter. Mainly because maturity is not well defined: is it the number of tests executing, the number of test cases per line of code, the frequency of execution, or the average time a developer spends fixing defects? As we all know, without a target and a way to measure up to that target, it’s very hard to optimize the path to achieve it.

Sauce Labs has a unique perspective on what makes an organization successful at leveraging automated testing. Using observations from running 2 billion tests for thousands of Sauce Labs customers, Fernando and Amir will outline the the key drivers of success in continuous testing and an provide an actionable way to see how your organization or team is doing relative to other successful testers. Learn why velocity, quality, efficiency, and coverage are the four pillars of continuous testing—and how to effectively measure these concepts.

In this talk, Fernando and Amir will discuss a new index, which can be used by any organization to evaluate their progression towards continuous testing. The audience will learn:

• The components of the new continuous testing index
• What makes those components such strong maturity indicators
• How can organizations improve on each of the metrics
• What happens organizationally when those metrics are improved

Bio
Fernando Vidal is a Senior Data Scientist at Sauce Labs who works on understanding how customers use Sauce Labs and what metrics best predict their testing success. Prior to Sauce Labs, he worked in quantitative investment research implementing machine learning models for trading systems.

Building The Blocks Of Trust In Automation

Abstract
Having followed best practices from previous Selenium Conference talks, the Zoll Lifevest team was able to shift from flaky tests to stable and reliable automated tests. During that time, Sneha Viswalingam learned the importance of building trust in the test suite to unite the team as a whole. Once trust is established in the automated tests, it became crucial to the overall software development lifecycle.

In this talk, Sneha will cover the following topics:

• Strategies that she used to make the tests reliable
• Trainings for the manual testers to contribute to the automation efforts
• Importance of having an Automation Test Reporting Dashboard
• Presenting the project status and test reports on a wallboard to reinforce trust

By implementing these steps in her organization, Sneha has built trust not only within the test suite but into the team as a whole.

Bio
Sneha Viswalingam is a staff web test automation engineer at Zoll Lifevest. Her passion for ‘All things Automation’ began when she started her career in Silicon Valley right after graduate school. Her experience revolves around Architecting and Developing end-to-end automation testing frameworks for web-based and Mobile applications. Currently, she resides in Pittsburgh, PA. When not coding, she is an artist who loves to create acrylic paintings.

Automating the New York Times Crossword

Abstract
The New York Times crossword grid is made up of hundreds of individual web elements. Automating game logic via the puzzle interface is a daunting technical (and logical) task. Find out how the New York Times Games team uses Webdriver.io, cheerio.js, event listeners, and Sauce Labs to deliver quality crosswords while continuously improving. Attendees in this session will leave with a few key takeaways:

• How to shave assertion-heavy tests’ running times from minutes down to seconds
• How to avoid race conditions when verifying animations on the page
• How the New York Times crossword gets made

Bio
Phil Wells is a software engineer, author, and dad from Fairview, NJ. He’s a software development engineer in test with the New York Times Games team, which pretty much means he’s been training a computer to do the crossword. Phil does not know Will Shortz.

HARgh Matey, Network Traffic Validation Made Easy

Abstract
Testing network calls manually is a tedious and time-consuming process; especially when multiple URLs or network calls need validation. Using the Sauce Labs Extended Debugging feature, USA Today Network can validate data at a much faster rate to ensure that our data is accurate. By generating a HAR (http archive) file, the team can search the network calls to find and validate needed data.

For USA Today Network, this is a valuable tool for validating that analytics data is correct. By searching the data in the HAR file, they are able to find the network calls for each of the three analytics providers and validate that the values in each call are present and exact. Incomplete or incorrect data can lead to business decisions based upon bad data. Therefore, regularly testing analytics is a business must in any industry.

Bio
Aaron Wolford is in his seventh year in Quality Engineering at Gannett | USA Today Network. He is currently a Software Development Engineer in Test supporting several products across the network. He also spent eight years at Sallie Mae as a Team Lead owning a variety of responsibilities including software testing, project management, and product ownership. His interests include test automation, process improvement, and analytics. He enjoys spending time with his son, puzzles, and sightseeing.