Full transcript of “Cross-Platform App Testing Just Got Stupid Simple with AI”. Click a timestamp to open the video at that moment. Need another video? Use the YouTube transcript generator.
0:00 Look, we're stuck with two terrible choices. Write endless brittle tests on Android, iOS, and web, or just skip testing entirely and hope our app doesn't crash in production. Both options suck. And at this point, we're just waiting for AI to replace us so we can escape this testing hell in the first place. But luckily, there's a new solution that can solve this problem entirely. Now, we can write a few lines of code and let AI agents test the software for us on literally every platform. And to make this happen, we're going to use a tool called Ask UI. It's an AI agent that can scan your screen, find elements, and interact with pretty much anything you can see. And we can do all of this with just plain English commands, which I think is pretty cool. So to get started, we just need to run the command pip install aski. So let's
0:48 jump into a demo. Okay, so we're just going into this really simple example here of how Ask UI works. What we can do is run this simple script and all it's going to do is click on screen the text notepads. You can see within the text editor itself I have the word notepads down here. So let's see what happens when I execute this file. You can see ASUI starts to scan the screen. It finds the text notepads and the mouse moves over to it and clicks it. So ASUI is constantly scanning the screen reading that information and then working out the position of elements on screen and then interacting with it. And we can do that either through natural language or a bunch of tools that are provided through the SDK. So here on agent for example, we have act, click, close, get,
1:34 a bunch of different tools. So aside from this simple example, let's see what else we can do. So we have another file here called test example and we're doing a similar thing. We're clicking on the text notepads and then we're also using natural language to check if a certain piece of text exists on screen. We're saying agent.get get is the text create new notepad visible reply with yes or no and then assert that the answer is yes in this example we're actually running it through pi test so we've defined a test here at the top you can see inside the notepads tab we have the text create a new notepad so let's close this and then we'll do pi test example py and you can see this time running a test again aski monitors the
2:20 screen detects notepads click exit and then let's see if it detects the text. And there we have it. The test has passed. So you can see here that we can use natural language or any of the commands provided. For example, instead of using click, we could use agent.act and then just describe the action that we want to perform. Click on the notepads button. So I'll comment this out, minimize this, and then let's just run this file one more time. And you can see the same action is performed. And again the test will pass. So this just shows that you can use both natural language or any of the methods provided through the SDK. Now this is just a super simple example. So how would this actually work in reality? Well, I've got an example application here showing a
3:06 subscription screen for an Android app. So we've got the input here to enter a promo code. We can click apply. For example, let's enter save 20. We press apply and you can see a promo code has been applied. So I click great and you see we now have 20% off for the first month. Now how could we automate this testing? Well, we can do that again with Ask UI. So in this code here, we've got a new test called test app.py. And what I'll do first is we'll run this test app.py and we'll pull up the emulator and then see what happens. So aski first finds the text types in save 20 clicks apply we click
3:52 great and then finally we detect if 20% off first month is on screen and you can see here that the test has passed. So let's just look into how this test works. First we run the command agent.click and we're locating the text enter promo code. So if we look on the app again here, we're just looking for this piece of text enter promo code. And by clicking here, we can actually focus the field. The next is to type in the text save 20 and then to click the button apply. So let's demonstrate what would happen here. Save 20. Apply. This modal appears at which point we tell the agent to click the word great, which is the button here. And then finally, we say in natural language, is the text 20% off the first month visible? reply with
4:37 yes or no and then we can just use a simple assertion with piest assert answer equals yes. So if for example we just look for a random piece of text that's not actually on screen. Let's see what happens if we run this test again. So again the test is running. We're checking what's on screen. We enter the promo code save 20. We move over, press apply. We then again scan the screen. look for the word great. We click great, but then notably the text we're looking for won't be found. So in this case, the test actually failed. Now, because this code is platform agnostic, it means it can run on both Android, iOS, or actually anything that exists on your computer.
5:22 You can actually even run Ask UI on physical devices. So running these kind of tests means that you can be pretty much platform independent, which is really cool. So there we have it, a single test suite which can handle multiple platforms without changing a single line of code. How cool is that? And of course guys, if you do want to check this out yourself, you can download it today. Just head over to askui.com. Thanks for watching.
