The Push from Within: A Journey of Transformation

Abstract
Quality. We all realize it is important, yet so many companies are unwilling to invest the time to achieve it. Words like agile, startup mentality, and fail fast take precedence over quality, causing churn across organizations worldwide. Sound familiar? If you are a change agent for quality, yet find yourself frustrated by the lack of support from leadership, this talk is for you.

Claude Jones, Sr. Director of Engineering, provides an inside look of what it took to push from within and help one of the world’s largest companies adopt a culture of quality. This presentation will cover:

• Identifying the right project
• Building the right team structure
• Removing the fear of change
• Providing value with the right metrics
• Making testing fun
• Reinforcing the benefits

This case study will challenge you to re-think your approach or empower you to stay on your current path of adopting a culture of quality.

Walmart Labs is the tech arm of Walmart with the mission to provide a seamless omnichannel experience for customers whether shopping online or in-store. The technology solutions built by Walmart Labs serve over 2.6M customers weekly supported by 1.5M employees in the U.S. alone. At that scale, it is critical a focus on quality is part of their DNA.

Bio
Claude Jones is a seasoned technical leader with over 15 years of experience. He has been with WalmartLabs for 6 years and is responsible for managing a distributed team of 60+ product, user experience, project management, architects, and engineers focused on building custom tools to improve the way Walmart develops and delivers eCommerce experiences across the web, native app, and stores.

Claude has a passion for bringing out the best in others and believes that anything is possible with hard work and dedication.  In his spare time he runs the Elevate Foundation, that is an organization focused on helping others in need and giving back to the community.

Accelerating Your Digital Agenda with Continuous Testing

As organizations accelerate their digital agendas and become obsessed about delivering a flawless customer experience, application development and delivery teams are under pressure to move faster than ever. This increasing emphasis on speed and quality has put testing in the eye of the Agile+DevOps hurricane, and made continuous testing critical to business success. What does continuous testing mean for testers, developers and business users? How are enterprise organizations adopting and adapting for quality at great speed? In this keynote, Forrester analyst Diego Lo Giudice will shed light on the trends and practices poised to shape the testing market in 2019 and beyond.

Bio
With Forrester since July 2005, Diego primarily contributes to and advises on Forrester’s offerings for Application Development & Delivery Professionals. He partners with Forrester’s global application leaders and is a leading expert on SDLC processes and practices, covering topics such as Agile development, Agile and Lean transformations, Agile development sourcing strategies and services, Agile testing practices and tools, DevOps, and software testing and quality, with a key focus on systems of engagement. Diego also covers software delivery metrics, artificial intelligence, and open source governance.

His 28 years of industry experience, in addition to application development, allow him to give expert advice on change management programs for optimizing the overall modern application delivery process, execute technology management assessments, review technology management strategies, and make comparisons. He also has experience in complex mission-critical project and client engagement management.

Thursday AM Keynote

Bio
Christine McCarey is a recognized diversity, equity, and inclusion (DEI) leader, public speaker, and recovering lawyer. She founded ImpactDEI to effect positive change at scale. Her company helps organizations embed DEI initiatives into their processes and culture. Before founding ImpactDEI, she founded and led RetailMeNot’s D&I Program with the support of committed colleagues. Under her leadership, the Program was recognized by Glass Door as #1 and Built in Austin as #2. She also founded and led RMN’s D&I Council, Women@RMN (an employee resource group), and Legal Operations. Before RMN, Christine held business development and legal roles, including as General Counsel.

What’s Next For Sauce Labs?

Abstract
In this keynote, Brad Adelberg, Vice President of Engineering at Sauce Labs, will dive into the 2019 Sauce Labs road map. Brad will discuss plans to expand existing functionality, and give attendees a sneak peek of some of the new features users can expect to see in 2019 and beyond.

Bio
Brad Adelberg is VP, Engineering at Sauce Labs. Prior to joining Sauce, he spent six years at Oracle as vice president of development focusing on data integration. Before then, he was vice president of engineering at startup C3. During his time there, Adelberg most notably led the development of what is now the C3 IoT Platform.

Brad holds a PhD and MS in Electrical Engineering from Stanford University where he also received his BS in Electrical Engineering and Computer Science.

Wizard of Oz Testing – Panelist

Abstract

Alan will be participating in a panel session moderated by Greg Sypolt, Director of Quality Engineering at Gannett | USA Today Networks.

The panel will address questions such as these from the audience, pre-polling questions, and live polls:

• Journey to CI/CD with Docker / Kubernetes
• Continuous testing
• Best practices (test script creation, execution, optimizing code, and more)
• Organizational transformation: dedicated QA engineers or developers
• Thoughts on new open source testing tools, scriptless, and visual QA
• Leveraging analytics to measure success
• Tips, what to focus on automating the right things, not everything
• Characteristics of test script maintenance

Bio
With over 20 years of experience in quality assurance, Alan Ark draws upon a great number of stories and lessons learned in the past into his daily life. Previously, he has presented at Pacific Northwest Software Quality Conference (PNSQC.org), Better Software, and Quality Week. Currently, he is a Sr. Quality Engineer with Cisco Systems working with the Duo Security brand (duo.com). He is working on multiple projects to streamline the delivery of products to production with confidence. In his spare time, Alan enjoys great beer, fine wine, and golf with his family.

Continuous Deployment for ‘Mature’ Codebases

Abstract
When starting up a greenfield project, it’s easy to take advantage of the most modern development practices. But what about the rest of us, who are working on codebases greater than five minutes old? How do you take code that’s four years and hundreds of thousands of SLOC, and turn that into a lean, mean, continuous-deploying machine? In this talk, Melissa will walk through what continuous integration and deployment means for teams working on mature code bases, and what the roadmap looks like to get from a release cycle that may take weeks or months to one that deploys on-demand.

Attendees can expect to come away with:

• Roadmap template for transitioning to CI/CD
• Common pitfalls when migrating old code
• Understanding how and when to declare ‘victory’ for a migration

Bio
Melissa Benua has worked in nearly every software development role—dev, test, DevOps, and program management—at companies as big and small and somewhere in-between. She’s created and run high availability, high-quality services for PlayFab, Bing, Cortana, and Xbox One, and now for mParticle. Melissa discovered her love of massively-scaled systems while growing the Bing backend, where she honed the art of keeping highly-available complex systems up while undergoing massive code churn. Now an engineering manager with mParticle, she’s passionate not only about maximizing efficiency both in her product code and in her developer tools but also about sharing best practices among colleagues and the tech world at large.

Pyramids Are Ancient – Let’s Talk Automation Strategy

Abstract
‘Richard, what tests should I automate?’ ‘What should I be automating?’ ‘What layer should I be automating on?’ ‘Where do I start with automation Richard?’ These are questions I get asked regularly. My answer every single time? ‘Checkout The Test Automation Pyramid!’, ha, of course, it’s not, it’s ‘It depends…’ That’s because there isn’t a single correct answer to any of those questions, although most are looking for one. It certainly isn’t a picture of a triangle.

However, there are steps we can take, questions we should be asking, which will lead to answers. They won’t be the right answers, but they will be enough to progress, while we learn more and continuously adapt. These questions will allow you to map out your context, arguably the most important action to maximise your automation efforts. Knowing when to ask these questions, how to progress after the answers and who to involve are the most underrated skills in automation. You can learn all the programming languages, all the frameworks and tools, but if you don’t know how to audit your own context and frame your problems, they are irrelevant.

In this talk, I’m going to share the questions I ask in order to understand the context I’m in, allowing me to create an automation strategy. We’ll then compare those questions and answers to existing models highlighting the flaws in them, but also where they can provide some direction.

Takeaways:

• Understanding the importance of having an automation strategy
• A set of questions to help you understand any context and formulate an automation strategy
• How to evaluate and challenge existing automation models

Bio
Richard Bradshaw is an experienced tester, consultant, trainer and generally a friendly guy. He shares his passion for testing through consulting, training and giving presentations on a variety of topics relating to testing. With over 12 years of testing experience, he has a lot of insights in the world of testing and software development.

Richard is currently the BossBoss at the Ministry of Testing, co-creator of the Automation in Testing (AiT) namespace, blogs at https://automationintesting.com

Performance Testing With Sauce Labs

Abstract
Every front-end engineer is concerned about the speed of their web application; in fact, a number of companies have SLAs that require apps be responsive after a certain period of time in order to not lose the attention of potential customers. Until now, though, most web applications are shipped either without any front-end performance testing, or with just a passive check.

Performance implications are difficult to understand and hard to predict. Application monitoring tools make it possible to check performance in production environments but by this time users may already be experiencing degraded performance. With Sauce Performance you can now ensure that your page runs fast and is performant as part of your CI/CD pipeline.

In this talk Christian Bromann will dive into performance testing using Sauce Performance. Learn how you can start running assertions against key performance metrics in your CI/CD pipeline.

Key Takeaways:

• Get to know everything around key performance metrics and how they affect the user experience on your page
• Learn about Sauce Performance
• Find out how to add performance tests within minutes to your existing CI/CD pipeline

Bio
Christian Bromann is the lead engineer at the DevTools team at Sauce Labs and is constantly trying to find ways that allow to automate beyond the WebDriver protocol using browser technologies that already exist today. His passion about open source made him the core maintainer of WebdriverIO, one of the first NPM packages that brought test automation to Node.js.

Trust Issues with UI Automation

Abstract
UI Test Automation is notorious for stability issues. UI automation is almost synonymous with “flaky test.” In this presentation, Bijoya Chatterjee discusses taking a proactive strategy towards test automation flakiness as opposed to a reactive approach of looking at failures and then deciding the course of action. How can the test automation community use and develop tools that helps eliminate flakiness and also identify flaky tests not only before they are run, but even before they are merged? How can we create these tools in a service model that plugs into our Continuous Integration pipelines?

Bio
Bijoya Chatterjee is a senior QA manager at Sony PlayStation and has been working in UI automation for 14 years. She has been leading quality and automation activities at Sony and over the years, has gathered a lot of experience in UI automation and solutions to make it more stable.

Utilizing Component Testing for Ultra Fast Builds

Abstract
A best practice of software architecture is to design your applications into independent modules or components, with a published contract for interaction between components. This is a principal part of the popular microservice style architecture, but also it applies to components created in a large monolith or with the out-of-the-box patterns that front end libraries promote.

Being able to test the functionality of the component independently and apply a level of trust that those components work then opens the door to rethinking an organization’s continuous integration and continuous delivery strategy. This in turn can potentially reduce the need for long test suites in many environments. It will also cause a rethinking of how to structure the unit testing strategy and the test pyramid.

Attendees of Tim’s talk will be equipped to:

• Understand the different kinds of component testing
• Know how/when to apply each type of component testing
• Understand the effect on your organization’s broader testing strategy

Bio
Tim Cochran works for ThoughtWorks NYC, and has been a full stack developer and architect for over 15 years, working on everything from large distributed enterprise projects to small NGO visualization apps. Most recently, Tim has been helping companies with digital transformation, moving towards continuous delivery and building a DevOps culture. He also is an automated testing zealot who practices TDD almost exclusively.

Page Objects: You’re Doing It Wrong

Abstract
Page Objects are the most commonly used abstraction pattern for functional UI Tests. They have the ability to enable users with little Selenium knowledge to write sophisticated tests against an application at scale, while reducing the maintenance costs as the application changes. Based on Sauce Labs Solution Architect code reviews, though, it is one of the most poorly understood and abused tools in a team’s framework. As an SDET at 5 companies before joining Sauce Labs, I’ve tried a number of different approaches and know first-hand what works well and what can cause problems. Experienced people will have disagreements with many of the points I will outline, and I will present both sides along with the reasons for my preferences.

The following principles (and more) will be discussed:

1. Imperative vs. Declarative
2. Page Object Alternatives
3. Deterministic vs. Non-Deterministic
4. Coupling vs. Decoupling
5. Inheritance vs. Composition

Bio
Titus Fortner is a core contributor to the Selenium project and the maintainer of the Ruby bindings. He spends a significant amount of time writing open source testing software built on top of Selenium. He is the project lead for Watir and is active in supporting these projects on Stack Overflow, message boards and in the Selenium Slack and irc. Titus has implemented automated tests at five different companies and currently works at Sauce Labs as a Solution Architect, working with the community to facilitate testing best practices.

How I Learned to Stop Worrying and Love Record and Playback

Abstract
Record and playback tools like Selenium IDE have a stigma attached to them in our industry— they’re not worth your time, they produce tests that are impossible to maintain, and they’re only for non-technical people.

But with the new Selenium IDE, that’s all just a bad rap.

In this talk, Dave Haeffner will squash the stigma by showing you examples of how Selenium IDE can reliably augment and level-up your testing practice regardless of your team’s test automation maturity or your level of technical experience. He’ll also share how he went from a record-and-playback naysayer to a Selenium IDE maintainer.

Bio
Dave Haeffner is the writer of Elemental Selenium — a free, once weekly Selenium tip newsletter that’s read by thousands of testing professionals.

He’s also the creator and maintainer of the-internet (an open-source web app that’s perfect for writing automated tests against), author of The Selenium Guidebook, and an active member of the Selenium project.

As a Software Developer at Applitools, he works full-time as a maintainer of Selenium IDE.

He’s helped numerous companies successfully implement automated acceptance testing; including The Motley Fool, ManTech International, Sittercity, and Animoto. He’s also spoken at numerous conferences and meetups around the world about automated functional testing.

AI Testbot As A New Sauce For Continuous Testing Recipe

Abstract
Software QA has been burdened with laborious manual testing for decades. Test automation technologies significantly reduce the repetitive manual work associated with testing, but additional assistance is needed to achieve what’s truly required for Continuous Testing. With recent advances in AI and deep learning, we can build an AI surrogate—a testbot—to address the Continuous Testing pipeline from a new perspective…just like the chatbot is innovating the call center communications. Join this session to explore “the Good, the Bad, and the Ugly” faces of a few “testbot” implementations. Here are a few key takeaways that attendees can expect:

• A description of AI Testbot
• Discussion of why you need AI testbot in continuous testing
• The good, the bad, and the ugly of AI Testbot
• A practical use case: Add a new kick, AI Testbot, to an already existing Sauce Labs/Mobile testing recipe

Bio
Jae-Jun Hwang has more than 20 years of engineering experience in software development and research in large-scale cloud services and intelligent IT infrastructures. He is currently founder & CEO of AppTest.ai, which provides the industry leading self-testing service, a.k.a Testbot, using AI and deep learning to innovate continuous mobile testing domain.

What’s that Smell? Tidying Up Our Test Code

Abstract
We are often reminded by those experienced in writing test automation that code is code. The sentiment being conveyed is that test code should be written with the same care and rigor with which production code is written.

However, many people who write test code may not have experience writing production code, so it’s not exactly clear what is meant by this sentiment. And even those who write production code find that there are unique design patterns and code smells that are specific to test code in which they are not aware.

Given a smelly test automation code base which is littered with several bad coding practices, we will walk through each of the smells and discuss why it is considered a violation and demonstrate a cleaner approach by refactoring the code live on stage.

Key takeaways include how to:

• Identify code smells within test code
• Understand the reasons why an approach is considered problematic
• Implement clean coding practices within test automation

Bio
Angie Jones is a Senior Developer Advocate who specializes in test automation strategies and techniques. She shares her wealth of knowledge by speaking at software conferences all over the world, teaching courses, and writing tutorials on angiejones.tech. Angie is known for her innovative and out-of-the-box thinking style which has resulted in more than 25 patented inventions in the US and China.

Using Headless In A CD Pipeline

Abstract
Learn how the RentPath team is using headless in their CD pipeline for rent.com. They run tests against each push to GitHub in headless, leveraging the lower cost of headless containers through Sauce Labs for greater concurrency. For multi-browser testing, the team relies on traditional Sauce Labs VMs for nightly regression runs and to check new builds more thoroughly prior to release.

In this presentation, attendees will understand:

• The speed advantage that headless testing provides
• The challenges the RentPath team has overcome and
• What tweaks they’ve had to make to the tests for headless

Bio
Abdur Jubair is a passionate programmer who enjoys learning new technologies. Abdur joined RentPath in 2015, and since joining, he has held multiple QA roles including manual testing and backend API testing. Currently, Abdur is working as an Automation Test Engineer using Nightwatch.js. In the past two years, Abdur has worked with the RentPath team on implementing CI/CD pipeline. Before joining RentPath, he worked as a manual tester for several years, working for companies like Sage Software and Allconnect. Outside of work, Abdur prefers spending time with his family and playing cricket.

Finding Your Community

Abstract
We all know that it’s dangerous to go alone.

Julie will give her experience and lessons learned in starting new meetups, building and scaling communities, and creating culture.

Coming from a non-traditional background of support/operations and being “thrown into” QA, Julie began her journey overwhelmed by information overload, not quite sure which organizations and resources were a good use of time.

Whether you are a new QA engineer trying to maximize your time out of a myriad of choices, want to put yourself out there but aren’t sure how to contribute, are a lone tester in your organization having to seek support outside your company, or are trying to start a community from scratch because your area is lacking, this session is for you.

Julie will present on lessons learned, tips and pain points in her first year of creating communities and meetups from scratc

Bio
Julie is one of the co-organizers of Austin Automation Professionals and a member of several local meetup communities. She has approximately five years QA experience and is currently at Keller Williams. She was previously at AT&T for eight years in various roles such as QA, NOC and management. Julie holds a Masters in Business Administration and Bachelors in Information Systems Technologies from Southern Illinois University.

Break Up the Monolith: Testing Microservices

Abstract
Microservices is more than a buzzword: it’s an industry-wide tidal wave. Companies are spending millions to break up monoliths and spin up microservices, but they usually only involve QA at the very end. This talk by Marcus Merrell centers around real-world experiences and will pose questions that attendees can ask their developers/product people, and offer solutions for teams to help make your service more discoverable, more testable, and easier to release. In this session, Marcus will cover:

• How micro is micro?
• Documentation & Contracts
• Versioning API Endpoints
• Cross-team communication/collaboration
• Definition of Done
• Feature Flagging
• Testing Pyramid
• When to Get Selenium Involved
• The Story of 13 Systems—The “Screenplay”

Attendees will leave this session with:

• A better understanding of testing microservices
• A host of questions to ask
• Ideas for how to get more involved in the development of microservices earlier in the project

Bio
Marcus Merrell has been working in quality engineering for 18 years. He has been using Selenium/WebDriver since 2007 and is a contributor to the Selenium project and the co-chair of the Selenium Conference Organizing Committee. He has released several open source projects for testing and IoT.

As a solution architect at Sauce Labs, Marcus is well qualified to help customers with his expertise in release management, CI/CD, cloud software and container management. He is also experienced in analytics and marketing automation.
Marcus frequently speaks at conferences worldwide.

Wizard of Oz Testing – Panelist

Abstract

Phil will be participating in a panel session moderated by Greg Sypolt, Director of Quality Engineering at Gannett | USA Today Networks.

The panel will address questions such as these from the audience, pre-polling questions, and live polls:

• Journey to CI/CD with Docker / Kubernetes
• Continuous testing
• Best practices (test script creation, execution, optimizing code, and more)
• Organizational transformation: dedicated QA engineers or developers
• Thoughts on new open source testing tools, scriptless, and visual QA
• Leveraging analytics to measure success
• Tips, what to focus on automating the right things, not everything
• Characteristics of test script maintenance

Bio
Phil Merwin is a SDET in automation at CBT Nuggets with 20 years experience in QA automation frameworks, systems, and architecture.

Merrill’s Journey to CI/CD and Continuous Testing

Abstract
Merrill was faced with the challenge of transforming product delivery to push new features or fixes to production every day without sacrificing on testing and validating quality before introducing changes. They achieved this by implementing an automated testing framework and automating regression testing suite of 600+ scenarios that could run automatically and in parallel across platforms/browsers using Sauce Labs, which helped them be a gating decision for go live into Production.

In this presentation, Ashish will discuss:

• How to achieve maximum parallelism of the automated scenarios.
• How to use API calls for the fastest test data setup and teardown, and how Merrill used those to keep tests atomic and autonomous
• How the team decided which manual test cases to automate, including examples of how they decided to rethink and automate 600+ scenarios

Bio
Ashish Mukherjee is a software engineering professional with more than 25 years of progressive experience in software engineering including Microservices development, full stack development, product engineering, architecture, QA, project and operational management, web and mobile applications, middleware and integration technologies.

Wizard of Oz Testing – Panelist

Abstract

Meena will be participating in a panel session moderated by Greg Sypolt, Director of Quality Engineering at Gannett | USA Today Networks.

The panel will address questions such as these from the audience, pre-polling questions, and live polls:

• Journey to CI/CD with Docker / Kubernetes
• Continuous testing
• Best practices (test script creation, execution, optimizing code, and more)
• Organizational transformation: dedicated QA engineers or developers
• Thoughts on new open source testing tools, scriptless, and visual QA
• Leveraging analytics to measure success
• Tips, what to focus on automating the right things, not everything
• Characteristics of test script maintenance

Bio
Meena Negi is an ISTQB certified Sr. Software Engineer in Test at Sony PlayStation. Throughout her 8 years of testing career, she has worked with different organizations and gained expertise in various domains — From enterprise mobile and web, to automotive and cloud gaming.

She is passionate about breaking things; therefore she has designed and developed test automation frameworks and tools from scratch and used numerous open source tools to continuously improve testing for various software components. Other team’s member calls her “True Detective” for making rigorous observations and hypothesis and bring the real outcome and make another hypothesis based on the outcome.

When she is not working, you will find her doing kickboxing and volunteering.

How Analytics Became Our Friend In Testing Web Responsiveness

Abstract
There is a huge combination of browsers and devices that require testing. We need to selectively categorize them into groups based on the statistics from analytics, and run them on Sauce Labs based on the categorized group. The statistics may be surprising. Teams may need to test the OS/browser combinations that they considered less important, but spend less time on the OS/browsers that they thought were important. Analytics is your friend!

Bio
Peter Nguyen is a QA and automation lead for Autodesk Virtual Agent (AVA), a new chatbot at Autodesk. He has 10 years of experience with automation and has held previous positions in software engineering. Peter is passionate about leveraging technologies to get things done instead of doing them manually.

But They Worked Locally … How To Fix Flaky Tests On The Real Device Cloud

Abstract
The most challenging aspect of running tests on a real device cloud is the multitude of possible failures. It could be that the test is inherently flaky, because writing reliable UI tests is hard. It could be that the app actually has a bug. It is possible that there is an unreliable device on the cloud or on very rare occasions, you could have discovered a bug in the real device platform. What you will see is that your tests always passed on your local device(s) but suddenly fail on the cloud.

Attendees of Daniel’s presentation will leave with:

• Effective ways of analyzing test failures using real-world examples
• An understanding of common pitfalls
• Some good advice to make your tests work again

Bio
Daniel Paulus is Senior Software Engineer at the Sauce Labs Real Device Cloud for continuous, automated testing. With his team he works on the low level internals of Android and iOS devices to help Sauce Labs deliver market-leading performance. Daniel previously held roles as developer, team leader, IT manager and project manager during his 13-year career as an officer in the German Military.

Closer To The Metal: How and Why We Use XCTest and Espresso

Abstract
In this talk, Mario will describe the practices that ABN AMRO adopted in mobile teams when it comes to testing native applications on real devices. Since using Espresso and XCUITest is still relatively uncommon for large apps and there are various unique challenges due to being in an EU-regulated industry with various security restrictions, Mario will share the ABN AMRO team’s experiences, including:

• A brief architectural overview of the Mobile Banking app: why it is all native (Objective-C/Swift and Java/Kotlin), how it communicates with other apps and websites (what we call the Constellation approach)
• Why they choose to adopt Espresso and XCUITest: what are the advantages and the limitations of this choice
• How they run the test pipelines to spread them across time and devices and prevent teams from being blocked
• How they make tests reliable: faking network calls (pros and cons) or running an always up-to-date mock server on demand on AWS (why this is preferred when possible and what are the instances where it is not possible)
• Shortening integration test run time: skipping the device binding procedure (and any in-between long process in general) except where strictly needed
• Challenges: interactions across multiple apps, limitations of the test frameworks, dependencies on external hardware, problems testing with real data, automated penetration testing, web in app, web & app

Bio
Mario Negro Ponzi is part of the Centre of Expertise for Software Development at ABN AMRO. He started in the early 2000 developing on mobile phones (in Motorola, on the predecessor of the RAZR V3). In 2006, he moved to Amsterdam for TomTom where he worked first on Windows Mobile and then brought TomTom to the iPhone as soon as the SDK came out. Mario left TomTom in 2010, worked in a couple of startups until 2015 and then joined Philips on an app for medical monitoring of patients. Due to long commuting times, he left Philips at the end of 2016 to join ABN AMRO.

Welcome to SauceCon 2019

Bio
Charles Ramsey has been the Chief Executive Officer of Sauce Labs Inc since April 2015 and served as its Chief Revenue Officer from February 2015 to April 2015. Charles has 25 years of industry experience. He was a Venture Partner at JMI Equity. Prior to joining JMI Equity in 2005, Charles held a number of roles at Quest Software, Inc. including Vice President of Marketing and Sales. He served as Vice President of Sales at Computer Intelligence, and was employed in sales at IBM. Charles has served as a Director of ServiceNow, Inc. (Formerly, Service-now.com, Inc), and now serves as a Director of Configuresoft, Inc. Charles has a Bachelor of Arts from the University of California, San Diego and a Master of International Management from the American Graduate School of International Management.

How to Measure Success in Continuous Testing

Abstract
For many organizations, evaluating their progression in regards to continuous testing is mostly a subjective matter. Mainly because maturity is not well defined: is it the number of tests executing, the number of test cases per line of code, the frequency of execution, or the average time a developer spends fixing defects? As we all know, without a target and a way to measure up to that target, it’s very hard to optimize the path to achieve it.

Sauce Labs has a unique perspective on what makes an organization successful at leveraging automated testing. Using observations from running 2 billion tests for thousands of Sauce Labs customers, Amir and Fernando will outline the the key drivers of success in continuous testing and an provide an actionable way to see how your organization or team is doing relative to other successful testers. Learn why velocity, quality, efficiency, and coverage are the four pillars of continuous testing—and how to effectively measure these concepts.

In this talk, Amir and Fernando will discuss a new index, which can be used by any organization to evaluate their progression towards continuous testing. The audience will learn:

• The components of the new continuous testing index
• What makes those components such strong maturity indicators
• How can organizations improve on each of the metrics
• What happens organizationally when those metrics are improved

Bio
Amir Rozenberg is a thought leader in the space of agile quality methodologies. Currently as Director of Product at Sauce Labs, he leads the quality in DevOps product vision. Prior to Sauce Labs, Amir worked at Perfecto Mobile, wher he drove the core product strategy and other initiatives in the areas of real user condition testing, accessibility, security, audio and conversational interfaces, AI/ML, IoT and medical devices etc. In addition, Amir led successful synergies with Microsoft, SmartBear, Blazemeter, Neotys, among others, to optimize value delivery to the market. Prior to Perfecto, Amir led the mobile production monitoring solution at Compuware.

Accelerating Innovation at Charles Schwab Through a Frictionless Software Development Practice

Abstract
Charles Schwab has created a scalable, frictionless, and repeatable software development practice that allows teams to develop applications and services with a focus on quality, throughput, risk mitigation, and reliability – thereby, allowing Charles Schwab, in the digital era, to reduce cost and deliver value to their customers, consistently.

Bio
Mike Schober is a Managing Director in Technology at Charles Schwab leading the software development and delivery of Schwab Intelligent Portfolios an innovative automated investing platform. Prior to Schwab he worked at large companies and startups with a personal driving force to deliver new and emerging technology solutions, including one of the first internet payment systems, an early one-click shopping experience, a first of its kind internet payment chip enabled credit card, and more. Mike has a passion for leading edge industry changing technology innovations and takes pride in delivering solutions that improve individual’s experiences and lives.

How We Got To A CI/CD World: A Vanguard Testing Journey

Abstract
CI/CD is all about speeding up your feedback loop, and one great way to do that when developing a microservice economy is through Contract Testing.

Vanguard has gone through many changes over the past 10+ years. Craig Schwarzwald will be briefly discussing each main phase their testing has undergone; from thousands of manual tests, to thousands of automated tests with Selenium, to Shifting Left, to now focusing on Contract Testing.

Craig will then discuss what Contract Testing is and talk about its importance in the CI/CD pipeline (giving the team virtually E2E coverage at the speed of Unit tests).  Each person should be able to relate to one, or many, of the Vanguard team’s main testing phases and see how to get to the next phase in their CI/CD journey by following some of the same steps that Vanguard did. The audience should also leave with:

• An understanding of what Contract Testing is
• How to implement Contract Testing
• How Contract Testing can help them get to a CI/CD world like the Vanguard team

Bio
Craig Schwarzwald has more than a decade of professional software development, scripting, and automation experience. Craig has worked on implementing Contract Testing directly within his organization’s CI/CD pipeline which is automatically utilized for all new microservices being developed within the company. Most recently he has taken on a role as an Automated Testing Coach where he helps teams with all of their most pressing and challenging aspects of unit, integration, and end-to-end testing.

Building Your Own Native App Visual Testing Module with Appium

Abstract
We all know that Appium can do (almost) everything you tell it to do, but in the end we always ask it to do the same thing: to automate our happy and/or our error flow. The result can give us the confirmation that the user can still buy a product, or verify if his bank account still has a positive balance.

But what if we need to know more than that: what if we also need to verify the layout of our app? The answer would be easy, as there are plenty of open-source or paid solutions that can help you with that. However, almost all of these solutions focus on web/hybrid apps, not on native iOS and or Android apps.

Wim faced this challenge when he was automating a React Native app for a customer. There was no image comparison tool that could do what he wanted it to do, so he rolled up his sleeves and started building one. In this talk, Wim will walk down the path he took and he will explain:

• Why he used ResembleJS as a core visual comparison solution
• The things he learned about Appium in the process (pro tips!)
• The differences between iOS and Android and how he managed to solve this in one cross-platform solution
• The pros and cons of the solution

Bio
Wim Selles is a Solution Architect for Sauce Labs based in the Netherlands. During the day, he assists customers with solving automation challenges in their organisation. By night, he practices his passion for front-end test automation with Javascript. He likes to create his own node.js modules to help and support automation engineers and is also a contributor to multiple open source projects that involve testing, such as WebdriverIO, Protractor, ng-Apimock and many more.

Wim also has extensive experience using Appium for automating Hybrid and React Native Apps. He enjoys sharing his automation experience as a speaker at conferences like AppiumConf in London and SeleniunConf India, on his blog and during meetups and webinars.

Overcome The Challenges And Limitations of Android App Automation With Espresso

Abstract
One of the most popular automation tools for mobile apps—Appium—comes with challenges and limitations. Some of these challenges have included the following:

• Tests take a long time to run
• No support for testing activities outside the app
• Test flakiness
• False positives/negative results

Jagmit Singh found that Appium challenges/limitations were resolved on switching over to Espresso Framework. Espresso UI test automation framework is Google’s de-facto testing platform for Android app developers. There are a few key motivators to use Espresso. First of all, Espresso tests are within the application and you can mock certain layers of app like a white-box testing. Also,  Espresso has unique synchronization method in its core making the tests less flaky.

During this talk, Jagmit will demo how to:

• Set up Espresso
• Develop/execute tests and identify elements
• Execute the Espresso tests in a Continuous Integration environment using Jenkins and Firebase

Bio
Jagmit Singh has more than ten years of experience in QA, Release Engineering and Software Development. He has extensive experience in both mobile and web automation using Selenium, Appium, Espresso, Robot Framework, Java and Python. He is currently working as Senior Automation Engineer at CBC Canada, developing the automation framework for the iOS and Android native apps. He also advises other teams in implementing automation strategy and best practices. When he’s not working on automation, Jagmit enjoys playing racket sports, hiking and skiing.

Wizard of Oz Testing, Panel Speaks

Abstract

The session will feature Greg Sypolt moderating a panel of customers and industry experts who will be attending the conference.

They will address the following questions from the audience, pre-polling questions, and live polls:

• Journey to CI/CD with Docker / Kubernetes
• Continuous testing
• Best practices (test script creation, execution, optimizing code, and more)
• Organizational transformation: dedicated QA engineers or developers
• Thoughts on new open source testing tools, scriptless, and visual QA
• Leveraging analytics to measure success
• Tips, what to focus on automating the right things, not everything
• Characteristics of test script maintenance

Bio
Greg Sypolt is a Director of Quality Engineering at Gannett | USA Today Network. Greg has a blend of a developer, quality, and DevOps mindset allowing him to bridge the gaps between all the team members to achieve the desired outcomes. Greg helps shape the organization approach to testing, tools, processes, continuous integration, and support development teams. He’s an advocate for automating the right things and ensuring that tests are reusable and maintainable.

Using Headless in a CD Pipeline

Abstract
Learn how the RentPath team is using headless in their CD pipeline for rent.com. They run tests against each push to GitHub in headless, leveraging the lower cost of headless containers through Sauce Labs for greater concurrency. For multi-browser testing, the team relies on traditional Sauce Labs VMs for nightly regression runs and to check new builds more thoroughly prior to release.

In this presentation, attendees will understand:

• The speed advantage that headless testing provides
• The challenges the RentPath team has overcome and
• What tweaks they’ve had to make to the tests for headless

Bio
John Theodore has worked at RentPath as an Automation Test Engineer since 2015. In that time, he has worked with two different frameworks and languages – one based in Ruby and the current stack based in Nightwatch.js. Over the past two years, he has worked with Abdur and the rest of the Rentpath team to implement a CI/CD Pipeline. A self-taught programmer, he took an unconventional path to QA Automation, majoring theater at the University of Southern California with a minor in Video Game Design and Management. After graduation, he spent several years as a manual tester, working on major titles like Call of Duty and Guitar Hero as well as some lesser known and unreleased games. Outside of work, John has always been an avid NASCAR fan and hosts a sim racing stream on twitch – twitch.tv/john_a_theodore.

Test Armada: Evolution of a Value-based Test Automation Platform

Abstract
In this talk, Pranav Verma will discuss the ins and outs of Test Armada, an end-to-end non-opinionated test platform that the Walmart Labs team developed for web, native, and backend services with actionable reporting capabilities. Test Armada runs more than 100K tests every day at Walmart Labs, with significant external users contributing and benefiting from this open source platform.

This talk is ideal for anyone responsible for or curious about learning how to build a successful automation framework for a large organizations/open source communities. In this talk, Pranav will cover:

• The evolution of Test Armada
• Details of the Test Armada platform
• The core values of Test Armada, including case studies:
  • Actionable Insights: Test reports should provide detailed insights into the failure to reduce, if not eliminate, debugging time.
  • Non-Opinionated: Do not make choices for your users, compliment their current automation solutions.
  • Reliability: Test results should always be reliable, and only fail due to real bugs in application code or setup.
  • Speed: Test should always run fast, and scale well to fit the needs of a large organization.
• A demo of Test Armada
• Test Armada Benefits with Stats
• Test Armada roadmap

Attendees will the this talk with:

• An understanding of how and why Test Armada was developed
• Ideas for how to build an automation framework to fit a large organization
• Practical advice that will help with your current automation solution

Bio
Pranav is a Senior Engineering Manager at Walmart Labs.

How to Measure Success in Continuous Testing

Abstract
For many organizations, evaluating their progression in regards to continuous testing is mostly a subjective matter. Mainly because maturity is not well defined: is it the number of tests executing, the number of test cases per line of code, the frequency of execution, or the average time a developer spends fixing defects? As we all know, without a target and a way to measure up to that target, it’s very hard to optimize the path to achieve it.

Sauce Labs has a unique perspective on what makes an organization successful at leveraging automated testing. Using observations from running 2 billion tests for thousands of Sauce Labs customers, Fernando and Amir will outline the the key drivers of success in continuous testing and an provide an actionable way to see how your organization or team is doing relative to other successful testers. Learn why velocity, quality, efficiency, and coverage are the four pillars of continuous testing—and how to effectively measure these concepts.

In this talk, Fernando and Amir will discuss a new index, which can be used by any organization to evaluate their progression towards continuous testing. The audience will learn:

• The components of the new continuous testing index
• What makes those components such strong maturity indicators
• How can organizations improve on each of the metrics
• What happens organizationally when those metrics are improved

Bio
Fernando Vidal is a Senior Data Scientist at Sauce Labs who works on understanding how customers use Sauce Labs and what metrics best predict their testing success. Prior to Sauce Labs, he worked in quantitative investment research implementing machine learning models for trading systems.

Building The Blocks Of Trust In Automation

Abstract
Having followed best practices from previous Selenium Conference talks, the Zoll Lifevest team was able to shift from flaky tests to stable and reliable automated tests. During that time, Sneha Viswalingam learned the importance of building trust in the test suite to unite the team as a whole. Once trust is established in the automated tests, it became crucial to the overall software development lifecycle.

In this talk, Sneha will cover the following topics:

• Strategies that she used to make the tests reliable
• Trainings for the manual testers to contribute to the automation efforts
• Importance of having an Automation Test Reporting Dashboard
• Presenting the project status and test reports on a wallboard to reinforce trust

By implementing these steps in her organization, Sneha has built trust not only within the test suite but into the team as a whole.

Bio
Sneha Viswalingam is a staff web test automation engineer at Zoll Lifevest. Her passion for ‘All things Automation’ began when she started her career in Silicon Valley right after graduate school. Her experience revolves around Architecting and Developing end-to-end automation testing frameworks for web-based and Mobile applications. Currently, she resides in Pittsburgh, PA. When not coding, she is an artist who loves to create acrylic paintings.

Automating the New York Times Crossword

Abstract
The New York Times crossword grid is made up of hundreds of individual web elements. Automating game logic via the puzzle interface is a daunting technical (and logical) task. Find out how the New York Times Games team uses Webdriver.io, cheerio.js, event listeners, and Sauce Labs to deliver quality crosswords while continuously improving. Attendees in this session will leave with a few key takeaways:

• How to shave assertion-heavy tests’ running times from minutes down to seconds
• How to avoid race conditions when verifying animations on the page
• How the New York Times crossword gets made

Bio
Phil Wells is a software engineer, author, and dad from Fairview, NJ. He’s a software development engineer in test with the New York Times Games team, which pretty much means he’s been training a computer to do the crossword. Phil does not know Will Shortz.

HARgh Matey, Network Traffic Validation Made Easy

Abstract
Testing network calls manually is a tedious and time-consuming process; especially when multiple URLs or network calls need validation. Using the Sauce Labs Extended Debugging feature, USA Today Network can validate data at a much faster rate to ensure that our data is accurate. By generating a HAR (http archive) file, the team can search the network calls to find and validate needed data.

For USA Today Network, this is a valuable tool for validating that analytics data is correct. By searching the data in the HAR file, they are able to find the network calls for each of the three analytics providers and validate that the values in each call are present and exact. Incomplete or incorrect data can lead to business decisions based upon bad data. Therefore, regularly testing analytics is a business must in any industry.

Bio
Aaron Wolford is in his seventh year in Quality Engineering at Gannett | USA Today Network. He is currently a Software Development Engineer in Test supporting several products across the network. He also spent eight years at Sallie Mae as a Team Lead owning a variety of responsibilities including software testing, project management, and product ownership. His interests include test automation, process improvement, and analytics. He enjoys spending time with his son, puzzles, and sightseeing.