Playwright MCP vs Playwright CLI vs Skills | Future of Agentic QA Automation

0:00 Hello everyone, this is Absari. Welcome back to the channel. In today's video, we're going to talk about a very important topic for the future of QA automation. I would rather say it's agentic automation, right? And we'll see uh specifically playrite MCP, playrite uh CLI and skills how combining all these three uh we can use within our agentic uh workflow. So if you are a manual QA or automation QA SDT or QA lead so this video will be very very helpful for you. Actually we have uh already been uh doing a lot of automation. Currently we stand here most of the team I'm not saying all but most

0:45 of the team currently doing this manually writing code. this very heavy code automation right QA or SGT they understand the requirement they write the automation code locator steps assertions do manually right finding the locator right locator X path then uh semantic uh locator those all details most of us most of us are still doing and running the test and finding the bugs and some education scenario this is the traditional way of uh doing so now uh This shift is being happening in automation as well. Right? So now what what we are doing is uh or maybe in tomorrow or in the future we'll be doing

1:31 is intelligent intent based uh testing right where QA engineer or SGT provides the intent or goal then AI agent will with a right MCP understand the plan understand then plan then execute and validate and fix any um any locators uh during the uh executions then it try to heal then run the process script. So this is how the all agentic flow that will be the future right. But why we are shifting toward this complete agentic uh AI based solutions? Because a couple of challenges within uh this current approach right time

2:17 consuming one high maintenance definitely many timing you have to fix manually uh when there is some changes right still you need to change in if you're using AI also you need to go and fix it but it will be easier than manually you are fixing something right and this test case writing are very flaky and brittle sometimes It depends on who is writing the test. If you are junior, if you are still uh learning or beginner level, then definitely you'll uh have this flaky and uh brittle tests. That's how we learn, right? Then locator issues definitely um this is also based on your understanding

3:02 on uh various locators. Then uh finding the uh right locator is the important thing, right? So in that case we see those locator issues and limited coverage definitely you don't have enough time to write all kind of coverages sometime we just go ahead with the happy path all the positive scenarios those kind of things right so now with the agents with the AI this challenges somehow will be fixed right so you just ask your agents or your um let's say uh GitHub copiler or a code, you just ask them and it will just try to um run it its own. You don't have to

3:49 uh wait for it to fix something if your intent is something to okay go ahead and check whether we're able to log in with the correct username and password or not. Right? So that that's my intent. So the AI what it will do it's just validate the intent right you enter the correct password enter the correct uh username then you should see your login should be successful intermittent if you see some popup some blocker screen come some ad scam so agent will intelligently handle those details okay so now let's talk about what is playright MCP so MCP stand for model context protocol. So it's kind of like server running within

4:36 your local machine or if you are already hosted somewhere in your server then you just access those. So basically it's provides certain tools to you and AI agent can directly communicate with those tool to um perform certain browser actions right then it iterate also till certain goal is not achieved right let's say this is how it works like you prompt something like you said okay open Google then search for some website Then um click on certain link then open and verify that link is working correct or not. Right? So this is your prompt uh the LLM or coding agent let's say in

5:23 this case your VS code with GitHub copilot then what it will do it will basically um understand your intent right if your intent is to open a browser basically you're asking to perform certain action right so it will check within its capability does it have any tools which can open the browser or which can uh open the browser and click or search on the web pages, right? So what uh tools uh that can perform those actions is a brightite MCP or selenium MCP or any browser MCP server or tools uh that can perform click operation navigate something like

6:09 this. If you see we have in play we have various tools basically list of tools available which basically this LLM agents uh need to perform certain action right let's say we said okay go to Google and try to do certain things so we'll see the demo also um just now uh in in a minute to understand more about it right so let's see how um to write MC so we can Let's directly go to playride. Um let's say I go to MCP and here is the documentation for playright right play MCP. So before that let's say I want to install and do uh

6:57 from the beginning right get started installation init provide test I let's quickly create one project here right and let's say npm init okay so I just enter enter all the default I should have say hyphen s okay so now I just write in it within here right okay TypeScript brow JavaScript doesn't matter um GitHub action or maybe no browser yes then it will install certain details over here

7:43 okay then we'll see how we can use MCP uh those details as well here right so it's trying to download some browser. So let's quickly test it whether this is all working correct with its uh some sample test I believe. So if it is working then all good and I believe this is working okay so far. Yes. So it's working in a headless mode and all pass now. Cool. Uh so now let's uh try to uh install MCP server right. So what I want to do here go to the installation here. So this is

8:30 the one um MCP server command I need to use in my um VS code uh MCP setting file or JSON file you can say. So a couple of way uh either you can set here right you can directly go to this particular thing or you can also create uh one MCP folder here I can create uh repo level file here okay dot uh vs code mcp uh json right and here you can see I can add the servers right if you have already some

9:16 server here. I can use that or I can just use this particular thing here. So this is my MCP server. Okay. So I can just remove this one to just a server ident. Okay. So now the error gone. We can also directly do I believe here if you click on this client setup and for VS code you can uh use from the CLI as well. So if you just add this it will also do the same thing. uh whatever we are doing manually just click on it and you'll see all 23 tools um ping configure right so how you can check those tools is configured you can click on it here this

10:04 uh icon then after that you can see play right already uh available here right uh okay so this one is previously somewhere available able I need to just remove that right and let me see how we can remove that as well otherwise it's okay I believe so this is already there somewhere uh in my user directory that's why it's saying that to so I just stop from there I'll just use this one okay within my uh VS code directory okay so Now if you see the MCP tools example, browser navigate, browser

10:51 snapshot, browser click, browser type, those uh all the details like take screenshot or snapshot whatever right. So those details we can see here the same uh these are the different different tools that is being developed to perform a certain actions right. So whatever you can think of uh for playright uh to perform on the browser you can find most of the details over here right then we'll see how we can use it uh let's say I have a uh orange SRM this is uh demo portal let's see how we can use um playright mcp server that just now installed right to

11:37 uh write a automation and test. Right? So I have a prompt already. So I created this prompt orange uh for uh navigating to this orange HRM demo website. Then some credential I have provided and what is the objective? Valid the basic login and d workflow like a QA automation engineer. Then it will uh open. So these are the steps basically the more uh detailed steps that you provide would be the better result that you will find in the MCP or agent TKI right the better the prompt better the result so that's the thumb rule okay so let's use this uh prompt to uh check how it is doing I'm

12:24 using uh GPT 4.5 mini or you can use any cloud uh sonet 4.5 4.6 six anything that we can use here. But yeah, better model is the better result sometimes, right? And make sure you're using the agent mode, plan mode, ask mode also you can use if you are just trying to understand or make or want to feed the information to the agent and later you want to create or uh do or perform the actions. So later you can select also. So this is also better if you want to just plan first everything just dump into it within this session. Then after that you just uh ask the agent you first review everything then after that you ask agent to perform uh the details right so in this case it's

13:10 very straightforward for me I just added all the prompts let's get started okay um this prompt to perform UI uh UI test. Okay. So let's see and how it is doing. Okay. And we can see here what exactly it's doing. It's evaluating certain things. It's uh going to call MCP server seems if we see all the details here. The prompt outline it read everything. Then it's saying MCP provide to uh

13:55 browser navigate browser snapshot those details uh it will try to perform right let's see okay so now it's trying to open the browser using playride MCP server right you can also click here to understand uh from where the server is coming you can see the servers output those details this you can say allow in this session. Okay. So now it should open certain t here. Okay. Let me search the details here. If you see the browser new browser is already opening and allow without review. I want to

14:43 review as well. Right. Perfect. I loaded the player MCP tool. So certain details is also providing and you can see here it's already open the browser and try to navigate to that particular thing. Okay. So now I say okay just skip the review part and let's do and uh let's allow without review let's for this particular thing let's just allow without review this uh snapshot as well. So let's see what exactly it's trying to do. Let's say allowing this session allow you.

15:28 Okay, let's say whether this is able. Okay, it's entering the details. So this is very basic. So far we know this how the play MCP works, right? So this will try to perform certain details here. Okay. I can say allow this without the review. Uh click also for each of the actions. It will try to ask her certain details. Okay. So now you can see it's already logged in and try to trying to validate certain details. Orange HRM UI test validation summary. Now it's giving us the summary whatever you have done like login page. So these are the details it verified. Then

16:16 dashboard page some verification pim module right whatever we ask right pretty much it did right is user non and record count display one report. So whatever it can find right it it found all the details from here. Right. So now also it can generate the script for me as well. Right. Next thing what I can do normally this is how we do. Okay. Now we generate the script and try to run it. Okay. Now in generate um type script right

17:04 code and try to run it and it's able to generate aspect file then after that it's trying to run or not and if it is part then we are good to go. So now it's uh generated some file here or in the JM now you can see uh the code here right and after you can review then just keep it then allow running also so now we'll see something here uh hopefully I can so here something happening already in different different browser modes all So the same test whatever we just now

17:51 did it's failed and let's say whether this is trying to uh okay so now as the server is already running for the test report is running that's why it's not going to the next so we can just click manually or you can just uh disable the um your reporting opening thing right let's say after fix it did try to fix certain thing here then after that it'll try to run it multiple times right it's uh running now and let's see whether this is able to fix or not okay

18:38 it's trying again column header still matching some multiple element So I need to use first right sometime what happens is um somehow it's able to generate or perform certain actions but when it generates the code the code is not the same what it did write right so let's say for performing certain action on the browser it tried uh to use uh let's say um some JavaScript executor then when we ask it to generate the code it just generated

19:25 some semantic code uh from the prewrite it didn't generate the way that it uh perform the action on the browser sometime when you do is uh explicitly ask it to generate the code whatever it tried and try to make it work right so that way it can generate the actual code. We'll see later um how we can do that and ask your agent to do certain uh things based on whatever you want. So that's where the skills comes into the picture where you need to specifically say okay you have to do whatever you have done during the exploration of the application the same thing you do uh when you uh try to generate the code as

20:12 well. All right. So this is generated already um all paths now you can use this code you can create a page object model whatever you want to do from here right so all pass that means all locator is good now let's try to do the same thing using uh MCP server right so let's try to install the MCP server as well here uh if I go to same uh documentation here you can go to here play right agent CLI right and come to the installation part and I can install globally basically if you uh so this is just a CLI and this CLI is being uh driven by the skills

21:00 we'll talk about it a little bit uh in a while let me uh I just install it globally playright cla it's already there and I should Okay, I can just use this plate help and as you see all the available um commands for this particular P cla right all the run command. So basically what happens is if you want to say P CLI install browser those detail also it can do I believe we already installed the browser so we don't need it for now. Uh and this is the important thing. There are CLI skills. Uh let me try to do in here.

21:48 Let's see whether it's able to do or not. Okay. So hopefully it did the skills as well. So if you see here and skills provide C cli and within that we got the skills. So this skills file basically it contains uh this MD file. I believe I can just use sip control v and you'll see like this. Okay. So this is about the skills file. So skills file is nothing but just the um context about uh this particular uh basically you can say it's a knowledge base right to do anything. So knowledge base of what? So knowledge base of

22:36 playright CLI basically whatever we saw here. So we can do uh certain details. Let's say if you want to open the browser also you can do from the command line like C cli open browser or whatever. Uh you can see here let's say I I'll just ask uh let me just quickly right let me see here one of the example here. Right. go to displayite.de tab right and it's also open and it will just open the browser first let's say this one right I mean not using any AI in this for now okay CLI is not a AI tool I mean it's AI but not the way that we

23:23 are looking into so what is saying after that play CLI open it's browser default open on the P ID So it's open some browser definitely I will see here. All right. So this is the browser is opening. Uh actually what happening is the browser is opening but uh in headless mode. Let's try to close this and okay I can just close it. Okay. And now let's say I open it with a headed mode. So my new browser should open now. Yes, it it is opening here new browser and now I'll say okay go ahead and perform certain things here. So

24:09 basically this is just a terminal um CLA application right. So now I see whatever you do it will perform that action click on some element right. So now you know and manually you can do that details and it click on this NodeJS and certain things right. So now the same thing if we wanted to do from agent agent need to know what this playright cla tool it can do for us right so that's the exactly all the detail about this playright CLI right so the the one that we saw just now

24:54 uh using the help okay I just go and help So whatever you see different different features available within the playright CLI it is being documented basically in the uh skills.md right so basically it's saying okay so the this is a playright CLI and what it can do and various commands that it can do um the way it can perform certain actions. So some examples also it is being given here so that it can follow when agent try to understand about play CLI first it will ask within this u uh within this um

25:41 local repository is there any information available about this playright cla because it's already uh it's completely new to uh agent let's say if I just ask all right and okay maybe we already have this um skill. So it already know about it. Let's say if I just um delete it then will it know about it? Let's say let's try to uh make it delete right cli delete. Okay. Do you okay maybe okay it will try to get the information from internet but let's see do you know

26:28 about uh play CLI tools. Oh okay let's see. Um so basically it will go and search internet and to get the information. Um but usually okay let's see so from where it's getting the information okay it executed play right cla close command certain details indeed and it it got the information but the thing is it will not get the exact information whatever we already have within the skills right so that's the benefit of having the skills here so if

27:14 I just click here and install So basically this um different different ways of interacting with the element. You can also create your own um um markdown file file to ask this playright CLI to do this way. Right? So that's the the whole purpose of using this uh knowledge base of this particular play MCP. Okay. So this is how I as I said playright CLI it uses uh skills right installable skill just now we installed this skills uh with the various examples you can have uh different examples and

28:00 link it with the skills boom in your um this local repository uh it has all the links to it right or you can explicitly if you um have if you have any agent.m MD or cloud.md file then you can also link these skills uh in into your u main entry point as well. So now let's use playrite uh CLI. Okay, I'll say playright cla to validate the orange hrm demo application and let's try to use a new session here. Okay, now try to use

28:48 C cli uh to perform on this prompt. Okay, let's see whether this is trying to uh call this playrite CLI and execute exact same steps or not. So you can see he's trying to read a playright CLI basically uh reading this particular CLI folder. Then after that it understood what are the details. So now if you see here basically it's going to the skill proite C cli here local file right then after that it understood this skills and where exactly this open uh uh

29:36 open browser things there right and I allow it let's see the next step what is happening we can see over here okay so it's opening in the headless mode so we don't see the browser Now but we can also see so it's trying to get the stamp set. Okay, for now I'll just uh allow it uh allow CLI for now. So it's it will try to do certain things its own same way just the different difference uh here is it's using playrite cla tool using whatever the reference that you have example that you have you are giving knowledge to uh AI agent this is claude solid 4.5 lm and using this

30:27 examples is trying to uh perform certain actions right? All right. So now it's understood. So it's also trying to get the snapshots. You can see here those snapshots also being saved somewhere and it's hierarchy and everything is being stored and is it will be also used um as a reference when you try to generate the code from it. Right? When you ask it to generate the code, it'll also do the same thing using from here it can get all the informations about the page. Right? So now it understood the

31:13 um complete flow. It executed it and it's saying successful now and it will also do the same thing. Okay. Now generate the typescript code and try to execute it. Okay. So let's see whether it's uh trying to execute it or not correctly. Okay. So now the code is being generated here. Okay. Now let's see and a browser is been opening here. I can see not that one here is opening something to browser.

32:02 Okay. So yeah let's see whether this is passing all the validation pass successfully. Yes. So now all the validation passed correctly, right? You can see all the report as well. All right. So it's already I think running some report here. That's why it's not able to run. All right. So let's see here. Now let's understand the basic difference between MCP and the skills. Right? Some people think like MCP and skills are competing or maybe alternative to each other but

32:47 no right. So it will not uh replace the MCP right some people say also MCP is only for tools and skills are only for prompts is also not correct but for key automation the best way to understand it is very simple. MCB gives the AI agent access and skills give AI agent the expertise. Just now we saw how uh playright CLI uh can perform using its skills or examples uh which we already have in the markdown files then based on that it perform all the actions because the LLM already knows okay for this

33:32 playright CLI right it it has certain um uh working example so we just follow the same and try to execute it. So okay so in our case when we try to delete the skill also the LLM already know about the play CLI let's say on or the lot of um information or knowledge available online so it knows it can search online as well because this is a complete agentic uh uh way right it also have different tools if you don't know uh if you can see here um basically there are lot of built-in tools available within this VS code editor. So it can directly search online agents the browser then it

34:20 can uh fetch the detail from various web right. So that's why it's able to get a lot of information about playright uh CLI. Let's say you created instead of playright CLI uh let's say you just created some uh automation CLI right. So that automation CLI uh agent will not or the LLM will not have information about it. It cannot search those information on online also. Right? Let's say I I try to uh ask do you uh know about let's say I ask a very like this app CLI

35:06 okay will it okay it will try let's say what it's trying to do it right user is asking I don't have information about it which you live all right so because it doesn't have any information uh locally or in public also right so that's why if I create certain information about it right I can create a tool uh let me uh ask agent to create a officer CLI tool here let's see how it works okay okay so I have written down here to create a offser CLI uh new tool and create uh uh skill about

35:52 it. Let's see how it works. Uh I'm just asking it to generate uh some fake uh tool here and uh with all details possible. Let's see whether it's able to generate me uh new skills. Okay. So let's see. Okay. So now I see it's generating some uh CLA related thing here. Did it create? Let me see here. Skills I go here CLI and we have the skills right. Okay. Uh let's try to understand it CLI automation architecture advisor and

36:37 repository analyzer. So you can create any uh tools available and if you have um this kind of uh tool and it can be anything right. It can you can create your own um uh CLI which can perform certain execution of uh the um of the test or maybe analyze anything or run the performance testing anything you can create and you just ask and you you just explain about each of the prompts or the commands that it uh supports. Let's say this socializ support search and you can give some example about it what exactly it will do and those thing this is just a fake I ask it to generate a fake uh

37:24 tool for me and let's see whether this is uh working or not let's see okay so what do you know about op right so that we see okay it doesn't know anything about it. Okay. What do you know about? Let's see whether this trying to understand trying to get the information from it. I can see it. So this is uh going to read the skills of CCLI then it's trying to understand about it and then it will ask if I say of CLI then um uh it can also try to

38:10 um it can also try to execute that if already available because this is just a browser command right so you can see now I have all the information about this tool so same concept same principle works on the playright CLI as well, right? So the skills and uh MCP is something a little bit different but it's not a competitor you would say, right? So this is just a example of uh you know how you can create a skills but you can create a skills for anything not only for the tools you can create uh knowledge base as well for any application. Let's say this is my orange SRM application just now we open and you can create login page detail. You can

38:58 create all the manual test cases also. You can um you can write within this uh skills and later if you want to generate or reuse the same skills you can also um use the same and it will able to read those knowledge base basically and try to use it in your automation or whatever the task you want to do. Let's say you have multiple hundreds of microservices then you can also create a skills for each of the microservices and and each of the m uh skills uh of each of the microservices will contain all about the schemas let's say it's it's the payload details it's uh um various architecture

39:44 diagram everything it has content then you can just make it as a charge GP your internal charg for uh middleware team or be your internal whatever um backend team is using right so you'll have all the knowledge at a single place as well if you want like centralization of your knowledge base for your team you can do that as well so this is the very high level uh you know diagram where we can see where exactly the skill fits user prompts first then the main agent uh like in our case a vs code agent Or if you have cloud code right then uh your context basically you're giving the system prompts or whatever the um let's

40:32 say uh agent uh agents uh has its own instructions as well. So those details it already have. Let's say if you're using VS code it's little bit different. If you're using um let's say GitHub profiler it has some uh different format a little bit but it's it's work the same. Then you have a specialized agent. If you have multiple agent personas, right, for specific task, you created a multiple agents, that agent also can use its uh specific uh skills, right? As I just now I gave one example, if you want to create a um skill or knowledge base for complete backend team, right? You can create a

41:17 one agent backend uh specialist. then that backend specialist will have all the access to that particular backend skills for all the microservices. So whenever you ask to that agent, it can give you all the answers about the back end microser related any questions and you can use also this for the tested aggregation if you wanted to. Right? So this is the skills where you have all the details more the detail more the context better the result. Right? You can have a template, a reference document. You can have uh script also inside it, right? Where you can have Python or uh let's uh just uh PowerShell script, right? Or you can have your

42:04 TypeScript any code that you can execute it, right? You can also uh keep within it. So if you wanted to perform certain uh action using those script it's also uh you can keep it within that um folder right and as many example you can provide that will be really helpful when you try to perform some using that tool you can create any tools right just now I created apps cla so it doesn't matter so you can create your own um uh tool or CLI command line tool and that will have all the information written within this skill file and you are good to go. No need the payite MCP at all if you want it. Yes, if you have

42:51 a tool plus the skills then you can say you can replace with it but that's a specific use case for the MCP as well right we will talk about it in the end when to use MCP when to use playright CLI right so this is how you can place skills and tools in between right so this is your complete agent uh agentic workflow Right. Let's say uh when to use which one? So we have a playright MCP, we have a playright uh CLI. So when to use the MCP? If you want to explore the complete application, if you just say okay, go

43:37 ahead and explore the application using playrite uh MCP it is really good exploric QA testing autonomous uh testing with a uh various loops right so that is good at it. This consume very less tokens as well. So this is very useful when you have um some prompt you have you you don't want to uh you know um maybe uh write the code and then execute it some sometime you want to just go ahead and perform certain um com prompts you have and just try to execute it then then in that case play CL is uh it will do better Right? Because you

44:25 have a proper prompts then it will execute uh really nice and with the less uh you can say the uh tokens. Okay, it's a token efficient workflow with a low context overhead and a skill based guidance and reusable workflows great for reportric test generations. Right. Okay. So this was uh very brief about what is the MCP, what is uh skills, playrite CLI, playride MCP, when to use uh those detail. We'll definitely come up with some more exciting topic around it, right? Yep. Thanks for watching. See you soon.