Beyond Selenium: Choosing the Right Library for Your Testing

Is Selenium still the best choice for your test automation needs?

In this episode of Test Case Scenario, Jason Baum sits down with Diego Molina, Staff Software Engineer for Open Source & Community at Sauce Labs, to unpack the complexities of selecting the right testing framework. They discuss the evolution of Selenium, the rise of modern tools like Playwright and Cypress, and the nuances of browser automation standards like WebDriver and CDP.

You’ll also hear about the hidden pitfalls of building custom frameworks, how collaboration is shaping the future of automation, and why focusing on value-driven testing is the key to long-term success.

Join us as we discuss: 

(00:00) Introduction

(01:12) Choosing the right testing framework

(04:46) The pros and cons of Selenium for modern automation

(07:35) You can’t compare Selenium, Playwright, and Cypress

(13:01) High-level vs. low-level approaches to browser automation

(16:20) The evolution of browser automation: CDP vs. WebDriver

(23:16) The importance of collaboration in the testing community

(20:53) Standardization and the future of cross-browser automation

(27:08) Reducing maintenance costs with modern test frameworks

(30:09) Recommendations for Java, Python, Ruby, and JavaScript frameworks

(38:14) Predictions for Playwright and the rise of WebDriver BiDi

(40:49) Key takeaways for choosing the best testing framework

We’d love to hear from you! Share your thoughts in the comments below or at community-hub@saucelabs.com.

SUBSCRIBE and visit us at https://saucelabs.com/community to dig into the power of testing in software development.

Sign up for a free account and start running tests today at https://saucelabs.com/. 

▶ Sauce YouTube channel:  / saucelabs  

💡 LinkedIn:  / sauce-labs  

🐦 X: / saucelabs

Transcript

[00:00:25] Jason Baum: Hey, everybody. Welcome back to another episode of Test Case Scenario. I'm your host, Jason Baum, and I'm super glad that you're able to make it here for another episode with us. it's just me today, but I'm going to be teeing it off, uh, and introing an amazing conversation that Diego Molina. Put together for us called Beyond Selenium, choosing the right library for your testing.

[00:00:51] Jason Baum: And it's truly informational has everything you need to know about the test frameworks and libraries, uh, that you'll need, uh, to get the job [00:01:00] done. And why to use certain ones over other ones. I know you're going to enjoy it. We sure did. And, uh, with that, I will turn it over to Diego Molina. Diego is a developer advocate at Sauce Labs.

[00:01:13] Jason Baum: He also happens to be the technical leadership committee chair for the Selenium project. He is also on the project leadership committee. So without further ado, here's Diego Molina with Beyond Selenium.

[00:01:28] Jason Baum: Uh, thank you so much for coming to the Sauce Community Office Hours.

[00:01:32] Jason Baum: Today, we're going to talk about beyond Selenium, choosing the right. Library for your testing. Like I said, I am Jason Baum. I am the senior director of DevRel at Sauce Labs. 

[00:01:44] Jason Baum: And with that, I will turn it over to Diego Molina. Diego, why don't you say a few words about yourself and then let's go straight into beyond Selenium, choosing the right library for your testing.

[00:01:55] Diego Molina: So yeah, I'm Diego. Uh, I do, community [00:02:00] work at SOS, so DevRel, but part of my responsibilities are also to support the Selenium project and other open source projects like Appium or WebDriver. io. spending a lot of time in the community, uh, helping people get started with automation, with testing, um, answering a lot of questions in the Slack for the Selenium channel.

[00:02:23] Diego Molina: anyway, there are different ways you can reach out to me and you can ask for help to get started with Selenium or Sauce. I'm, I'm, I'm here for you. and that's one of the points of having us, uh, as part of this, uh, community. Like a connection, like a bridge between open source and source, in case you have questions that goes to either side of those, um, sides of the, of the bridge.

[00:02:44] Diego Molina: I am also one of the leads of the Settling Project and the lead of the OpenSource Program Office.

[00:02:49] Diego Molina: And if you want to reach out to me, those are my social media network users, if you want to reach out to me in any [00:03:00] way. the objective of this talk is to, try to, um, Put you on a green field status, like what would you do today if you had to choose an automation tool to write tests for your website or for your mobile application, for example, right?

[00:03:21] Diego Molina: So imagine that scenario right now. You have three choices, what would you do? From the things you read, from the things you hear, try to build something in your head. And I will ask you a question pretty soon and hopefully you can, you can write something in the chat, um, based on your thoughts. So, we're going to cover different things.

[00:03:42] Diego Molina: One of them is thinking if Selenium is a good option or not, and I'm going to give you a good amount of context on why it is a good option and why it is not a good option. Plus, we're going to talk about how important it is to choose a test framework, some [00:04:00] alternatives that we have out there, open source, all of them, and some final thoughts from this presentation.

[00:04:06] Diego Molina: So my first question to you, to all of you is, should you choose Selenium as a tool for automation? What would you do?

[00:04:16] Diego Molina: And I have my answer, because the idea is to, uh, Maybe it's also a good option. Uh, it depends on the project. It depends on many things. But, um, I also have a clear answer about that question. Like, if today I had a Greenfield project, if I was starting in and to automation, what would you do?

[00:04:34] Diego Molina: What would you choose Selenium? My answer is that I would not, I would not choose Selenium because, I have my reasons, but I will share them in a moment, right? Talking from the pros and cons, uh, there are different reasons why you should choose or why you should not choose Selenium 12. Yes, because it's a tool that automates every single browser that is used by a real user out there.

[00:04:59] Diego Molina: it's based on a [00:05:00] standard. Um, you can remotely execute those tests. You can do that locally. Uh, it has a huge support from the community. So that aspect is very important because it basically ensures that the project will be living for a long time, and your tests are going to be working for a long time, but, maybe you would not choose Selenium because you don't have a bunch of nice, nice features that you have in a framework that are like automatic weighting, you don't have any reports when you just use Selenium, you don't have Nice libraries that help you assert on different things that you're testing.

[00:05:40] Diego Molina: most of the time, if you want to write a test using Selenium, you have to write a bunch of code before that. Uh, so you're like helper functions, preparing methods, like a bunch of things you have to do before actually writing your test. So I'm more on the no side because I would miss those things [00:06:00] if I would start using Selenium from the first time.

[00:06:03] Diego Molina: and I will tell you why in a few minutes, but before that, I'm going to give you a bunch of context on, on what, you should consider, what you should know before deciding to choose Selenium or not. The main thing is that Selenium is not a testing framework, right? It is not. Selenium is a low level browser automation tool, so sending this message across the community has been the failure of the Selenium project.

[00:06:32] Diego Molina: This is something we're working to improve and one, for example, this presentation, we're trying to send to the community a big message about what to choose when starting automation and, and all the context I'm going to give you here is going to help you that. to give you a different example about what a low level automation tool means in this context is that let's imagine you want to build a website, right?

[00:06:55] Diego Molina: Let's If you want to build a website, would you just use something [00:07:00] like pure JavaScript and maybe jQuery? I mean, that's a lot of work, right? You would probably use a framework that has a lot of functionalities, a lot of features that are working out of the box for you. And one of them could be like Angular or could be like React or Vue.

[00:07:17] Diego Molina: Tools that with a few clicks, a few type commands, you have a website working, right? And that's the idea of this talk. Like maybe let's look more about frame, talk about frameworks. Let's look at them instead of like using low level, tools that are made for a purpose. So. One of the things that are a bit annoying in the automation ecosystem is that people are tending to compare Selenium versus Playwright versus Cypress versus Puppeteer and so on.

[00:07:50] Diego Molina: that's annoying because you're comparing, Apples and oranges. Like Selenium is not a testing framework. Playwright is a full fledged testing framework. So you cannot compare [00:08:00] them because they don't have the same, uh, set of capabilities and features. And for that, it's good to understand all the context we have.

[00:08:08] Diego Molina: Regarding browser automation, we have been trying to build this knowledge for several months and years until the folks at Google, the DevTools team, they are starting now to invest a lot in browser automation, more than before, and they have built an excellent knowledge base that. In short, explains you the history of browser automation.

[00:08:33] Diego Molina: So I'm going to give you this context in 5 to 10 minutes. So you don't have to go and watch that video and read that blog post. So I'm going to give it to you right now. Years ago, in 2004, Selenium was created. So today it's 2024, it's the 20th anniversary of Selenium. And Jason Huggins, one of the co founders at Sauce Labs, [00:09:00] he thought it was a good idea to have a tool to automate web applications.

[00:09:05] Diego Molina: He created this. And rather soon, they realized that, okay, this approach is not that good because what Selenium was using was to inject JavaScript into the browser to automate an application. They saw different flaws, different issues, like you cannot automate like different windows. and something that has been interesting is that Other tools that are supposedly called modern tools are using that approach.

[00:09:30] Diego Molina: For example, Cypress works in the exact same way conceptually as Selenium 20 years ago. Three, four years later, Simon Stewart realized, okay, all those flaws are opportunities to improve the tool. And then he created something called WebDriver. Selenium and WebDriver were two different projects. what they decided to do was to talk to each other, stop competing, and then creating a better tool for the community.

[00:09:56] Diego Molina: And that's how Selenium WebDriver was created. Years passed by, [00:10:00] then it became a standard. And that's why many browsers can be automated in a simple way. And those automation scripts will last in a long time because they are based on a standard. this created something that we like to call the WebDriver Ecosystem.

[00:10:17] Diego Molina: The base of that is this standardization, um, that was done some years ago. And a standard is basically some text within a GitHub repository that says, WebDriver Ecosystem. What a browser should do when you click, when you type, when you load a URL, when you do any given action on a website, then there are implementations of that standard.

[00:10:42] Diego Molina: The most popular implementation is Selenium. Another popular is GeckoDriver, ChromeDriver, SafariDriver, and so on. So on the client side, the most popular one is called Selenium. And on top of Selenium, there are An infinite amount of [00:11:00] tools built, around that, uh, ecosystem. So one of them is night watch.

[00:11:04] Diego Molina: One of them is Appium, one of them is something called water, uh, selenite and so on. So those are complete testing frameworks that are built on a standardized tool like Selenium, which is based on web driver. But there is also something called WebDriver. io. This is an excellent, uh, implementation of the WebDriver standard, uh, that has coverage for browser, for mobile, and it's a complete testing framework.

[00:11:31] Diego Molina: It gives you reporting, it gives you assertion libraries, and so on. on top of that, you have something called Serenity, which is a tool that helps you abstract. the implementation that is running under the hood. So for example, you can do interesting things like you can write your test in a, agnostic language, like say, I mean, you still use JavaScript, but the definition of the steps are, is agnostic and under the hood in a configuration file, you can say execute this with playwright, [00:12:00] execute this with WebDriver IO and so on.

[00:12:03] Diego Molina: So. This is a new type of tool that is growing, but it's also part of the WebDriver ecosystem. PHP, there is an implementation of the PHP. And let's not forget that around 70 to 80 percent of the web is still PHP. For example, WordPress, I think it's around like 50 percent of the websites we have. In the world are WordPress, which is based on PHP.

[00:12:28] Diego Molina: and on top of that, there are also frameworks built. So Laravel, Dusk and Panther, they are frameworks made for automation and testing. So we have this ecosystem. And then at some point over the years, we decided that JavaScript was going to be. The language we're gonna use to build the web. So we started with Node and then what came along Appium.

[00:12:53] Diego Molina: Um, then we have Protector and so on until the most recent one, which is [00:13:00] playwright. So this brings us to the point where we can talk about the different automation approaches we have out there. One of them is the high level approach, which is the one that Cypress and Tes use. And then we have another one, which is the low level approach that is used by most of the tools we use out there.

[00:13:21] Diego Molina: This could be subdivided into like the WebDriver tools and then the ones that are using extensively CDP, the Chrome DevTools Protocol. So let's talk first about this high level approach. perhaps you remember that a moment ago I was mentioning that this is how Selenium used to work 20 years ago. And let's mention Cypress in this case.

[00:13:44] Diego Molina: The way Cypress works is that when you write your JavaScript test or your TypeScript test, uh, what happens is that this code gets injected into the browser and everything runs inside it. So, [00:14:00] What happens there is that browsers have an event loop that is used to execute all the commands that your web application has.

[00:14:09] Diego Molina: So when your test is injected in the middle of that, what is happening is that your application is not executing itself in the normal flow. So this means that if your application has to execute this command and then your test comes on then the event loop is executing your web application and your test.

[00:14:28] Diego Molina: In between, and that's why the application is not executing itself in the, in the, in the normal, regular environment, a user would use it in the end. Um, that's one of the first flaws I think is that this approach has, other people see this as a positive thing, because this tends to have, to make your web application more stable, because it's not, you're able to freeze it with your, with your test.

[00:14:56] Diego Molina: So this means it. Could be, uh, interpreted as, as, as being [00:15:00] more, uh, like less flaky, for example. Uh, and it's easier to look at elements and all that. But in that sense, the approach, when you're thinking about end to end testing. You need to let the application behave as it normally would do, and manipulating the application to stop and freeze at any given point, that's not really accurate, and it's not the way we should be automating end to end applications.

[00:15:24] Diego Molina: So one of the interesting things that Cypress has is this process that is Node. js that basically states that. Serving as a, man in the middle to pass along all the connections that are done via HTTP. So everything you do in Cypress, like, mocking a network call and everything is, is manipulated by this network process.

[00:15:45] Diego Molina: so that's how Cypress works. And I think this is a valid approach for testing components, testing other types of things. But if you want to focus on end to end testing, I don't think this is the right approach. [00:16:00] TestCafe works very similarly, I'm not going to go through all the details because Conceptually it's the same.

[00:16:07] Diego Molina: What I want to mention about, TestCafe is that at some point they started to see that this proxy, this man in the middle, was a high overload in their tests. And they started to use the CDP protocol, the Chrome DevTools protocol, when they were using Chrome, Chromium based browsers. And that has been a relief for them, and I'm going to give you some more information about that in a moment.

[00:16:34] Diego Molina: Now let's switch to the low level approach where we have two variants. One is the WebDriver and the other one is the CDP variant. And the difference between them is that one is based on HTTP calls, the WebDriver one, the classic WebDriver one, and the CDP one is based on WebSockets. A simple explanation of this difference is that If you think about HTTP, it's [00:17:00] like writing an email or writing a letter to someone, So ideally you cannot communicate fluently because when you send a message, you need to wait for a reply. If you don't have the reply, you cannot continue the flawless communication. And that's basically the concept of HTTP communication. When you talk about WebSockets, this is more like a phone call that you can say something and you can hear back a lot of messages and you don't need to do this reply request mode.

[00:17:31] Diego Molina: You can just hear back the other way around. You can send a message and you can continue sending messages without the need to wait for a reply. why is this relevant? Because websites have evolved enormously and they are not just like the simple application with static code that we can manipulate easily.

[00:17:54] Diego Molina: Web applications today are based on web frameworks that have like service [00:18:00] workers on the back, API calls and different things. So just asserting that the website looks good and when you click or when you type is doing the right thing is not enough today. There are a bunch of low level use cases that we need also to assert to make sure that our web application works well.

[00:18:20] Diego Molina: For example, if your web application is working well on like, if you, if that's what you can see when you play with it, but if you check the console messages and you see a bunch of errors. Then actually it's not working well because you could be leaking something like credentials if the developer didn't write the error in the correct way.

[00:18:41] Diego Molina: Maybe you could be doing something like using the incorrect endpoint. Maybe the user is trying to log to some place but the UI looks good but there is an error in the console. but there are many things you still need to check in your web application, even though the front part looks good. some other use [00:19:00] cases that are interesting is that, um, today in a microservices world, we have a lot of APIs in our web application.

[00:19:07] Diego Molina: We need to make, we need to make a lot of API calls to make it work properly. But when we're testing, most of the time, those API services are either not up, Or they're really hard to start up, or it takes a long time to, to make it available for you, or maybe there is a team making a change. So it's, it's complicated to have all microservices up so you can test your web application properly.

[00:19:32] Diego Molina: So one nice thing that you could do is actually intercept all those network requests, mock them. Instead API, you can supply a network call, uh, a reply, a response that works for your application, And you don't need to start all those API services to your test properly. 

[00:19:51] Diego Molina: If you're talking about something like responsive design, if you need to check that your, web application is working well on a, on a widescreen, [00:20:00] on a mobile device, It's good to do for early testing, simulate how the user viewport is looking, so you can iterate fast at the beginning, but when you actually want to do proper testing, maybe a good way to move from there is to use emulators and simulators and then do the final test with real devices, but it's good to have a way to check that the That's it.

[00:20:24] Diego Molina: Viewport looks good on an early stage. Also things like simulating your location, for example, here in Europe, there are websites behaving a different way depending on the country where you are, so that's very important for you as well. having said all that, there is a bunch of use cases that something like, um, CDP with WebSockets cover, but they're not covered in the WebDriver classic, approach, right?

[00:20:53] Diego Molina: So, so what is happening these days in the, in the standardization and in the automation world? There is something called WebDriver [00:21:00] byline, so WebDriver bidirectional. And here comes again the Google DevTools team with their support by documenting and making this thing popular. So basically what is happening is that And I shouldn't be saying this in this way, but what the working group of WebDriver, which is Apple, Mozilla, Microsoft and Google, uh, Salesforce is there as well, one of the things that is happening is that they're all sitting together to understand the different use cases that are good for automation, take them from CDP, understand how they work in CDP and make them standard.

[00:21:40] Diego Molina: Um, so every single browser can do them properly. so you can do, for example, in the future, geolocation in Safari. You can mock network responses in Firefox. You can all do these things in a way that is standardized across browsers. One of the cool things is that, [00:22:00] instead of being attached to a proprietary standard, which is, From DevTools, like the thing to do in with the Chrome DevTools protocol, they only work on Chromium browsers.

[00:22:12] Diego Molina: So moving from that to a place where you will be able to actually automate. All browsers in the same way, like the way you do with Web browser Classic, it's a phenomenal advancement in, in browser automation. And one of the first steps that is happening right now is that puppeteer, what many people called the Selenium Killer in 20 18 20 19.

[00:22:36] Diego Molina: it's now becoming a cross browser automation tool because they are moving very fast. They're now implementing the WebDriver by day standard. And now you can, automate with Puppeteer all the Chromium browsers, but also Firefox. And when Apple is done with Safari, they will be able to also automate Safari with Puppeteer.

[00:22:57] Diego Molina: So if we go back to this, [00:23:00] ecosystem that I was talking a moment ago, then we can say that there's a new standard, that is the Bidi, and there is now a new member of the ecosystem, which is Puppeteer. So to summarize this high and low level context that I'm giving you, one interesting thing that I want to leave here as a message is that there is a lot of collaboration happening, and TestCafe is already noticing that.

[00:23:26] Diego Molina: TestCafe To give you more context, TestCafe belongs to a company called DevExpress and they produce actually web components for other companies to reuse, uh, for their websites. So they thought there is no good tool out there to do testing. We're going to create our own one and we're going to call it TestCafe.

[00:23:47] Diego Molina: They implemented the tool. They tested all browsers with their tool and eventually they started to realize This is a lot of work, we're not getting any profit from this, we need to start [00:24:00] collaborating with the rest of the community. The first step they're taking into this is to use the native CDP protocol to automate comprehensive browsers, and then they also have this TestCafe proxy for the rest of the browsers.

[00:24:16] Diego Molina: But then they realize it. That all the things that we're using for their particular solution, they were creating an overhead for, for all their tests. So what is happening there is that they realize that, okay, by using something that the community uses, something that is Fairly standardized in Chromium browsers, they can reduce their work by a huge extent.

[00:24:44] Diego Molina: And that means that they can actually focus on creating a better tool for testing, and my prediction is that now they see The benefit of collaborating and by using CDP and eventually they will move as well to, to Baidai. [00:25:00] And this was very interesting, a very interesting insight from the test cafe folks.

[00:25:05] Diego Molina: So they have to run tests. Every single time they make a commit with the web components and they use the framework to test that, right? So now they have two versions of their, framework. One is the one that is their classic approach with the test cafe proxy. And one is the new approach that is using, they call it native automation with, with CDP.

[00:25:30] Diego Molina: So you can see that by using a tool that is, Better specialized for the browser, working better with the browser and automating it, they're seeing that their test suite is gaining speed and obviously reducing their speed, the, the total execution time between 40 and 70%. And that's a huge benefit because basically they realize that their framework.

[00:25:57] Diego Molina: was adding a lot of overhead [00:26:00] to all their tests. So imagine your test suite being 40 to 70 percent faster just by switching the automation tool you're using under the hood. And eventually, uh, as I said, they're gonna see the benefit of using a tool that is inside the browser for automation, and they're gonna eventually move to ByteDyne.

[00:26:20] Diego Molina: My, my guess, my prediction is that at some point, Cypress will see the same thing and they will eventually see the need to move to a standardized approach as well. that was the, context I want to share with you And having that context, let's put in our heads, okay, now I have, I know all this information that I can use to make a good decision about choosing a test framework.

[00:26:46] Diego Molina: What would be a good test framework for me? Knowing that there is a high level approach, there is a low level approach, that in some cases this is suited for like component testing, the high level approach, but then it's not so good for an end [00:27:00] to end testing approach. So, with all this information, how would I choose a test framework?

[00:27:06] Diego Molina: So let's think about that. One of the main objectives of choosing a test framework is because we want to reduce maintenance costs. So for example, if for some reason you're using Selenium or you're using a tool based on Selenium, and maybe you decide that you want to do now mobile automation, you only have like browser tests, you want to do mobile automation, you don't need to write a bunch of extra code to do that.

[00:27:32] Diego Molina: If you have a framework that is already implementing that, most likely you're. effort will be reduced and therefore your maintenance will be reduced as well. And also you can be faster because there is shorter time to have something working. And then when the browsers change, when the standards change, you can be reactive because most of the time those tools will have a solution for you already.

[00:27:58] Diego Molina: Some of the things we look when [00:28:00] we are trying to find a framework, is features that they're helping us have something working out of the box, and most of all, they are helping us to focus on writing tests. One of the most common errors I've seen in customers and also in my, in my professional life is that people spend a lot of time writing custom code, Helper methods, reporting tools in their own personal or company's framework.

[00:28:32] Diego Molina: And they haven't even had the time to write a single test. So that means the more time you invest in your own framework, in your personal framework, in your company's framework, that is less time that you can invest in making your product better for your company and for your customers. So when you have things that are working out of the box, like.

[00:28:53] Diego Molina: Assertions, that you can do mobile and browser automation out of the box, that you can have reporting [00:29:00] that works out of the box in your framework, that, for example, you can use this framework to execute things locally or remotely in SauceLabs that has things that allow you to make the framework configurable.

[00:29:12] Diego Molina: For example, you don't need to hard code the browser and the browser version in every single test, but rather. You have a configuration file that lets you configure all those things. I mean, you are releasing a bunch of time that you can actually dedicate to make your application better. Some additional things that you consider when choosing a framework is obviously the programming language, how active the project is.

[00:29:40] Diego Molina: How well documented, how well they support you, and one of the cool things is that if you're using SOS as a cloud provider, as a place to execute your tests, you have extra support because there is a bunch of knowledge that we have for you in case you're running into issues, getting something working, and to be honest, [00:30:00] the amount of people we have at SOS and the amount of quality of knowledge we have at SOS really to help you get successful in automation.

[00:30:07] Diego Molina: It's just incredible. So that's one of the things that you should also consider when choosing a tool, like how well does this tool work with my preferred vendor? it's also very recommendable that you should do a proof of concept before deciding, like just not do. Like a hello world type of test, maybe just use real use cases, uh, from your existing tests or from your new application.

[00:30:31] Diego Molina: Try to do like spend a few weeks doing a real, uh, proof of concept. So to start rounding up this presentation, some alternatives out there that I find very trustable are the following ones. So these tools, they focus on reducing the effort you need to do. To do before writing a test. So they're going to give you, a configuration, a ground base, a place where you [00:31:00] can just start writing tests.

[00:31:01] Diego Molina: You focus on making your application better. I'm going to start with Java because it's my preferred language. and the example I'm going to give in an extended way is Selenite. I think this is the best. Framework for browser and mobile automation in Java out there.

[00:31:17] Diego Molina: so let me give you some examples why. So if you have done some sort of testing in Java, um, you realize that writing a single test could be easily 15 to 20 code lines because you have to write, you know, Open the browser, uh, set up some connections, then stop the browser, quit the browser and so on.

[00:31:39] Diego Molina: And in this case, most of that, you know, like setup and teardown is handled by the framework. In this case, You could see that in four lines, there are different methods, in the API that this, framework has, that anyone who doesn't really need or knows how to code.

[00:31:59] Diego Molina: And [00:32:00] if they read this, they could understand basically what is happening in the test. So if you see open, you understand what is happening there. You understand that there is like a user name field and the value being said is Tom Smith. You could see the name. Then you're like pressing enter, and at the end you could see clearly that there is an element that should have the text you are logged into secure area.

[00:32:21] Diego Molina: So those are the nice things about using a framework that you are abstracting yourself from the implementation of the automation tool and you're writing code that is much more. simple and easy to understand. Boilerplate code is one of the pitfalls that we have in automation. You write too much boilerplate code to get started to write actual tests.

[00:32:46] Diego Molina: For example, uh, in Selenium, if you want to find an element, this is what you would do. Like a long line, but with Selenite, this is what you do. That's enough. You don't need to do more. You don't need to do less. If you have to [00:33:00] wait for something, Selenite has, got you covered. It has automatic waiting. In Selenium, this is what you would do.

[00:33:07] Diego Molina: An extremely long line, and if you want to assert that something is there by the visibility, this is how you would need to wait, right? In Selenite, you make it more clear, more understandable. Um, anyone who can read English would be able to understand what is happening there. That the element, h1, should be visible, and that's it.

[00:33:30] Diego Molina: The error messages are also clear, they give you the complete element that you were trying to assert on, and they also take a screenshot for you, which is very useful when you want to debug. And this is just working out of the box, you don't need to do any extra configuration. It works, on parallel, so all the tests you have there, they work on parallel out of the box.

[00:33:51] Diego Molina: And this is great because one of the Talking about pitfalls, one of the main pitfalls when people want to start scaling up their tests [00:34:00] is that you write maybe 10, 20 tests, but the way you wrote the tests are maybe, hindering from executing them in parallel for different reasons that are not, um, particular to this case.

[00:34:12] Diego Molina: But with Selenite, you just write your tests, To set a flag and they run in parallel out of the box. And that's when you can actually have great benefits from like having an icon at source, for example, because you can scale up your tests, finish your test suites faster, and that's how you can get more benefits from, from this type of tools.

[00:34:34] Diego Molina: It has a embedded, natural out of the box integration with Appium. If you want to do mobile testing and there are a bunch, there is a bunch of, um, examples out there for you as well. If we talk about Python, we have Selenium Base. It's a framework that is, I think it's one of the best maintained out there.

[00:34:56] Diego Molina: there is Michael Mintz, who is the main maintainer and basically [00:35:00] his paid job is to maintain the framework for his company and for the community. you can do a bunch of things there. You have, Reporting out of the box, it's a very interesting use case. Many people use this to do tutorials.

[00:35:13] Diego Molina: Like when somebody joins your company and you want to show them how to use the website or how the website works. This has a tutorial mode that when you write your tests, it actually highlights all the elements that is, that are being clicked and that they're being interacted with. So it's very easy to understand what is happening when your test is being executed.

[00:35:35] Diego Molina: If you like Ruby, uh, Watcher is one of the best frameworks out there. this framework existed already before Selenium was created. So it's older than 20 years old. And that only means that there is a lot of good knowledge for you. Instead of you, Figuring out everything on your own, writing all that code on your own to make your test suite, work better and faster and reliable in a remote configuration [00:36:00] when you're using Sauce.

[00:36:01] Diego Molina: Basically you can use a framework like this that has all this knowledge to make it work, in a swift way, either locally or remote. So it's a great, framework, made for Ruby. And if you're into NET, There is a lot of people still writing tests in C sharp. There is a Tata. It's a framework that has all the features I mentioned from the other ones.

[00:36:23] Diego Molina: So, uh, reporting, automatic weighting, and so on. So, so have a look at that if you are into C sharp. And let's leave JavaScript for the last. let's first start talking about WebWR IO. Personally, I think this is like the hidden gem of the, of the WebDriver ecosystem. It's one of the best tools out there to do browser and mobile automation.

[00:36:49] Diego Molina: it has everything there for you. It has an incredible amount of plugins. So if you are just getting started, it has a CLI that asks you maybe five to [00:37:00] seven questions, and it's, basically figuring out where you want to run your tests. What reports do you want? And that's it. It gives you a configuration file that you can use and you can just start running tests from there.

[00:37:13] Diego Molina: One of the cool things there is that if you are using SauceLabs, it has a plugin that sets all the right data for your test. If it passed, if it failed, if you want to annotate it, if you want to retrieve like, for example, different values like the video afterwards or screenshots. It's just a great tool and I think, if you are looking into automate, with JavaScript, this should be your, your first, first choice.

[00:37:43] Diego Molina: And yeah, let's also mention Playwright because. we cannot deny that PlayReady is a tool that is growing a lot. It's very important. very easy to use. It has a great usability, great UX. but one of the things that we have been emphasizing is that it's using CDP and [00:38:00] it is, patching browsers in a way that makes the browser not the actual browser that it is.

[00:38:07] Diego Molina: And users have in their laptops. So let's make a parenthesis there and let's, let's talk about that. So, so Playwright is based on CDP. So you can do browser automation in Chromium, basic browsers like Chrome and Edge, and they work really well. But if you want to do Firefox, what Playwright does is that they take the code base from Firefox, they patch it, they put CDP on top, and they release a new version of Firefox that only works for Playwright.

[00:38:39] Diego Molina: With Safari, they don't use Safari because Safari is closed at source, so they use WebKit, which is the base for Safari. They take Safari They patch it, they put CTP on top, and they do a new release of WebKit that works only for Playwright. So when you're using, for [00:39:00] example, WebKit to say that you're doing Safari, it's not actually the same thing.

[00:39:04] Diego Molina: so what is happening here? The Playwright team has a lot of work to do. Every time there is a release, they have to patch the browsers. They have to do a new release. If there is an error, because in the main code base of those browsers, they've made something that is like conflicting with their CTP patches, then it's an issue for them.

[00:39:25] Diego Molina: And similar to TestCafe, some minutes ago, I was mentioning Playwright starting to understand that collaboration is the key to move forward. So a few weeks ago, The PlayRed folks started to identify ByteDyne as a way to move forward because they would like to automate Safari. They would like to automate Firefox without the need to do patches, without the need to do extra releases.

[00:39:52] Diego Molina: And is great news because there are, Seeing that collaborating with a community that is established, that is [00:40:00] open, that is working together to move automation forward, it's a way to go. So, I'm adding here as a recommendation player as well, because they are looking into Safari, into, into ByteDyne as a way to automate browsers as well.

[00:40:16] Diego Molina: In the future, I don't know if this will be like six months or one year, we're going to see that this ecosystem is going to grow by one, by adding playwright to them, to the mix. so one of the biggest messages of this. Presentation for you all is collaboration. Like this is how we all move together, collaborate, and make the automation for browser and mobile better for you all, the community, who would like to have tests that work not just for like a few months, but for years as well.

[00:40:48] Diego Molina: To wrap up, the takeaways I would like to share with you are basically that when you want to choose a tool, get your context, understand how the tool works, what approach does the tool [00:41:00] have, And then do an evaluation to see how that tool works. Is performing for your specific use case and don't stop there, like check the tool every now and then understand the tool, how the changes that the tool has and how the tool is changing over time and try to adapt to those changes.

[00:41:20] Diego Molina: Right? And the idea. So if. If you would need to take one thing from this presentation, it would be this third bullet point. Please focus on testing as much as you can. Focus on delivering value to your team. Make your web and mobile application better. And try to avoid Building Frameworks, because this is the pitfall that many people fall into.

[00:41:45] Diego Molina: And ideally, if you focus on making your website and mobile application better, then we're all going to win. my wish in the future is that we would have something like Playwright because it's, it's very nice. The usability is great. But [00:42:00] with the WebDriver by DAI stability. And last but not least, if you want to learn more about all these Selenium conferences happening together with the Appian conference next year, so have a look at seleniumconf.

[00:42:14] Diego Molina: com and join Solace, join the Selenium community, the Appian community, and learn more about all these things, in the conference. Thank you very much.

[00:42:25] Jason Baum: Thank you, Diego. Really appreciate it. 

[00:42:27] Jason Baum: Thank you so much for participating in the Sauce Community Office Hours, uh, with Diego. hope you learned something today.

[00:42:35] Jason Baum: Uh, please hit up that email if you have further questions. Also talk to your CSMs about topics you might want to see. We love to hear from them what you want to hear and then we'll put it back and we'll give them the feedback that you guys have, 

[00:42:50] Jason Baum: and we will see you for our visual office hours. Take care. Bye.