How AI Can Help UI Automation

 

How AI Can Help UI Automation: A Beginner’s Guide for Manual Testers

Manual testers have a strong foundation for automation: they understand user journeys, business rules, risks, edge cases, and expected behavior. The challenge is often technical—learning code, selecting tools, writing locators, and maintaining scripts.

AI can make that transition easier. It does not replace testing judgment, but it can work as a helpful assistant while you learn UI automation.

What Is UI Automation?

UI automation uses a tool to perform actions that a user would perform in an application, such as:

• Opening a web page
• Entering text into a form
• Clicking a button
• Checking an error message
• Verifying that a dashboard is displayed

For example, instead of manually testing login on every release, an automated test can enter credentials, select Login, and verify that the user reaches the correct page.

Common UI automation tools include Playwright, Selenium, Cypress, and Appium. While these tools often require some programming knowledge, AI can help manual testers get started faster.

1. AI Can Convert Manual Test Cases into Automation Scenarios

Your existing manual test cases are a great starting point.

Consider this manual test case:

Verify that a registered user can log in with valid credentials.

1. Open the login page.
2. Enter a valid email address.
3. Enter a valid password.
4. Click Login.
5. Verify that the dashboard is displayed.

You can ask AI to turn it into an automation-ready outline or starter script.

Example prompt:

Convert this manual test case into a beginner-friendly Playwright test in JavaScript. Add comments explaining each step.

AI can generate a first draft, saving time and helping you see how manual steps map to code. You should still review the result carefully, because the AI may make incorrect assumptions about URLs, element names, or expected behavior.

2. AI Can Explain Code in Simple Language

Code can be intimidating when you are new to automation. AI can explain individual lines, error messages, and automation concepts in plain language.

For example, this Playwright statement:

await page.getByRole('button', { name: 'Login' }).click();

means:

• Find a page element with the role of a button.
• Find the button named “Login.”
• Click it.

You can ask AI questions such as:

• “Explain this test script line by line.”
• “What is the difference between a locator and an assertion?”
• “Explain this error message for a beginner.”
• “Why did this test fail?”

This turns AI into a learning partner rather than simply a code generator.

3. AI Can Generate Starter Scripts for Common Test Flows

Many UI tests follow familiar patterns. AI can help create starter scripts for scenarios such as:

• Login and logout
• Registration forms
• Search functionality
• Form validation
• Add-to-cart flows
• Password reset
• Navigation checks
• Profile updates

For example:

Create a Playwright TypeScript test that verifies an error message is shown when a user submits a registration form with an empty email field. Use clear comments for a beginner.

The generated script may not be ready for production immediately, but it can give you a useful starting point. You can then run it, adjust the page details, and learn from each change.

4. AI Can Help You Create Better Locators

A locator is how an automation tool finds an element, such as a button, text field, or message.

Poor locators make tests fragile. For example, a long CSS selector based on page layout may break when the UI changes. AI can suggest more stable locator strategies.

A good locator preference is usually:

1. Accessible roles and names, such as a button named “Submit”
2. Form labels, such as “Email address”
3. Stable test IDs, such as data-testid
4. Meaningful unique attributes
5. CSS or XPath only when necessary

You can provide an HTML snippet and ask:

Suggest reliable Playwright locators for these elements. Prefer accessible roles, labels, and test IDs. Explain why each locator is appropriate.

AI suggestions should be validated against the real application and your team’s standards.

5. AI Can Help Diagnose Test Failures

Automated tests can fail because of a real product defect, but they can also fail because of timing issues, missing test data, environment problems, or unstable locators.

Suppose you see this error:

Timeout exceeded while waiting for locator('text=Welcome')

AI can help you investigate possible causes:

• Is the expected text correct?
• Did login actually succeed?
• Is the element visible only after a page load or API response?
• Is the locator incorrect?
• Is the test looking in the wrong frame or modal?
• Is the environment slow or unavailable?

Do not blindly apply AI-suggested fixes. For example, adding a long fixed wait may hide the real problem and make tests slower. Instead, make the test wait for a meaningful condition, such as a visible message, expected URL, or completed action.

6. AI Can Suggest Test Data and Edge Cases

Manual testers are already skilled at thinking about unusual user behavior. AI can help expand your coverage by suggesting cases for:

• Required fields
• Invalid formats
• Boundary values
• Special characters
• Duplicate records
• Error messages
• Permission-based behavior
• Accessibility checks

For a registration page, you might ask:

Suggest positive, negative, boundary, and accessibility test scenarios for a registration form with name, email, password, and confirm-password fields.

Use only safe, fictional test data in AI prompts. Do not share customer data, production credentials, confidential source code, or sensitive business information unless your organization has approved the AI tool and data-sharing process.

7. AI Can Help Reduce Repeated Code

As your automation suite grows, you may repeat steps such as logging in before every test. AI can help identify repeated code and suggest reusable functions.

For example:

async function login(page, email, password) {
  await page.goto('/login');
  await page.getByLabel('Email').fill(email);
  await page.getByLabel('Password').fill(password);
  await page.getByRole('button', { name: 'Login' }).click();
}

This can be reused in multiple tests. If the login page changes, you update the shared function instead of changing every test separately.

Keep your test design simple at first. Ask AI to explain any suggested reusable structure before adding it to your project.

8. AI Can Help Create Documentation

Automation projects need understandable documentation. AI can assist with:

• README files
• Setup instructions
• Test execution steps
• Test naming conventions
• Comments for complex scripts
• Defect summaries
• Test result summaries

For example:

Write a simple README section for beginners explaining how to install Playwright, run tests, and view the test report.

Clear documentation makes it easier for you and your teammates to maintain the automation suite.

What AI Cannot Replace

AI can accelerate your work, but it cannot replace a tester’s judgment. You still need to decide:

• Which scenarios are valuable to automate
• Whether requirements are complete and clear
• Whether a generated test checks the correct business outcome
• Whether a failure is a product defect or a test problem
• Whether test data is safe to use
• Whether generated code follows team security and quality standards

Think of AI output as a draft, not as a final answer. Review it, run it, and verify that it tests the intended behavior.

A Simple Learning Path for Manual Testers

You do not need to automate an entire regression suite on day one. Start small.

Step 1: Select one repetitive, stable test

Choose a simple scenario such as login, search, or required-field validation.

Step 2: Use the automation tool chosen by your team

If your organization already uses Playwright, Selenium, Cypress, or another framework, begin there. Following existing team standards will make learning easier.

Step 3: Ask AI to explain generated code

Do not only ask for scripts. Ask what each line does, why a locator was selected, and what the assertion verifies.

Step 4: Run and validate the test

Confirm that the test performs the expected user journey. Also confirm that it fails when the behavior is intentionally changed.

Step 5: Add scenarios gradually

After one positive test works, add a negative test. Then improve locators, add reusable functions, and learn reporting and continuous integration over time.

Final Thoughts

AI can help manual testers begin UI automation by converting test cases into starter scripts, explaining code, suggesting locators, diagnosing failures, and generating documentation. It lowers the barrier to entry, but it does not eliminate the need for testing knowledge, review, and practice.

Start with one small test case you already understand. Use AI to learn from the automation draft, validate every step, and improve it gradually. Your manual testing expertise is not being replaced—it is becoming even more valuable when combined with automation skills.

Handle custom tags using WebDriver

Sometime while doing automation we have to identify the element which have custom tags like <g> <svg> etc. For this we normally use * that will identify all elements tags in case we have to identify only specified custom tag then we can use below example for this.

<svg height="600" width="910" xmlns:xlink="http://www.w3.org/1999/xlink"  
   id="svgcontent" overflow="visible" x="910" y="600" viewBox="0 0 910 600">
    <g style="pointer-events:all">
      <text fill="rgb(0,0,0)" font-family="Sans-Serif" font-size="13" id="80-155-126" 
        transform="rotate(-90, 168, 126)" y="126" x="168" style="pointer-
         events:inherit">A100A</text>
   </g>
  </svg>

Following Xpath will work here

By.xpath("//*[local-name()='svg']")

By.xpath("//*[local-name()='text']")

By.xpath("//*[local-name()='g']")

Webdriver/Selenium Difference between methods .isDisplayed() and .isEnabled()?

Hi All,

I have seen so many posts where people asked the differences between .isDisplayed() and .isEnabled() in Selenium/Webdriver automation and found mostly people are still confused.

The methods .isDisplayed() and .isEnabled() have nothing in common.

 Here I am trying to explain the read differences between .isDisplayed() and .isEnabled(). As name itself clearly says .isDisplayed() means is my object are visible or not on my webpage means Am I able to see my element through my eyes.

Method .isDisplayed() :

An element is considered displayed when it is perceptually visible to the human eye.

The element displayed algorithm is a boolean state where true signifies that the element is displayed and false signifies that the element is not displayed.

To compute the state on element:

• ·         If the attribute hidden is set, return false.
• ·         If the computed value of the display style property is "none", return false.

Like

<input type="hidden" name="abc" value="10" id="hiddenFiled1" />

Observe the output, since element with id hiddenField1 is hidden from web page, so isDisplayed method return false, whereas isEnabled() method return true.

OR

Refer below link

http://www.w3schools.com/css/tryit.asp?filename=trycss_display_block

Try to run with display: block;  and display: none;

Method .isEnabled() :

An element is considered enabled if it's not a form control (button, input, textarea, select or option) or when the user interactions and focus are not blocked with the disabled attribute/property.

Is Element Enabled determines if the referenced element is enabled or not. This operation only makes sense on form controls.

Like

Last name: <input type="text" name="lname" disabled><br>

Last name: <input type="text" name="lname" enabled><br>

 

Refer below link

http://www.w3schools.com/tags/tryit.asp?filename=tryhtml_input_disabled

Last name: <input type="text" name="lname" disabled><br>

https://www.w3.org/TR/webdriver/#is-element-enabled

XPath unable to find text with or using webdriver /

Hi,

Some times we face issues to create XPath of  WebElement which have space in starting of it and then we check the element html source where we found &nbsp; or &nbsp is added before the element text while it display as space on UI. Normally we create the XPath to identify this element by adding space in the starting of it but his will not identify the element. There are another workaround to identify this element by contains text method. But if we have to identify the element with exact text having starting space or ending space  then we can use following way to do this.

lets see this example

<div class="myclass">
<label>AgendaShowCapacity&nbsp;</label>
</div>

OR

<div class="myclass">
<label>&nbspAgendaShowCapacity</label>

</div>

Following XPath will not work for this

//label[text()='AgendaShowCapacity '] doesn't work

Here we have to use following text to identify the element

&nbsp; = \u00a0

&nbsp = \u0160

So XPath will be like this

//label[text()='AgendaShowCapacity\u00a0']


//label[text()='\u0160AgendaShowCapacity']

Verify Broken Links

Verify Broken Links

Hi All,

Now we will learn how to validate broken links on page using WebDriver and java http methods. There are so many open source free tools available in market. Like Xenu we can validate the same but if we have to validate multiple web pages like >300 then it will be very tiresome task to enter each page URL.

Approach:

First we will open the webpage using web driver and get all the links having <A> tag. For finding the link URL, we will get the “href” property value of all the links present in webpage. In this way we will store the all the links URL present on webpage. We  have to exclude those links which having href property value like java script, hash(#) or mail to etc… because these links either referring to same page or refer to outlook mail like mail to link present in contact section.

Now we have the entire link’s URL, one way to open each URL using WebDriver but it is not advisable to use this approach because it will take too much time to perform. Instead of this we can use http method technique to test this. If my response code is 200 that means it is valid request else not. Note: Sometimes URL asks for certification acceptance, for handling this we are using SSL manager. For authentication URLs we can also use authentication methods

Below is the code where we getting the page URLs from excel sheet and validating the broken links. Results are saving in notepad.  Please do the modifications as per your need.

##################################################################

import java.io.File;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;

import org.apache.poi.xssf.usermodel.XSSFRow;
import org.apache.poi.xssf.usermodel.XSSFSheet;
import org.apache.poi.xssf.usermodel.XSSFWorkbook;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;

public class BrokenLinks {
 public static void main(String[] args) throws Exception {
  WebDriver driver = new FirefoxDriver();
  String urlxpath = "//a";
  List<WebElement> elements;
  Map<String, String> AllLinksURL = new HashMap<String, String>();
  Map<Integer, String> m = new HashMap<Integer, String>();
  try {
   // Open the Excel file
   FileInputStream fis = new FileInputStream(
     "D:\\url.xlsx");
   // Access the required test data sheet
   XSSFWorkbook wb = new XSSFWorkbook(fis);
   XSSFSheet sheet = wb.getSheet("url");
   // Loop through all rows in the sheet
   // Start at row 1 as row 0 is our header row
   File file = new File("Output.txt");
   // creates the file
   file.createNewFile();
   // creates a FileWriter Object
   FileWriter writer = new FileWriter(file);
   int i = 0;
   try {
    for (int count = 0; count <= sheet.getLastRowNum(); count++) {

     XSSFRow row = sheet.getRow(count);
     String page = row.getCell(0).toString();
     m.put(i, page);
     i++;
    }
   } catch (Exception e) {
    System.out.println(e);
   }
   fis.close();

   Collection<String> keyset = m.values();
   System.out.println(keyset);
   try {
    for (String page : keyset) {

     driver.get(page);
     Thread.sleep(5000);
     elements = driver.findElements(By.xpath(urlxpath));
     System.out.println(elements.size());
     // System.out.println(elements);
    
      for (WebElement ele : elements) {
       try {
        String temp = ele.getAttribute("href");
        // System.out.println(temp);

        if (temp != null) {
         if (!temp.contains("javascript")
           && (!temp.contains("tel://"))&& (!temp.contains("mailto:"))&& (!temp.equalsIgnoreCase("#")))
          
         AllLinksURL.put(temp, temp);

        }
       } catch (Exception e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
       }
      }
     
     

     
      for (String urlVal : AllLinksURL.keySet()) {
       try {
        String res = check(urlVal);
        // System.out.println(res);
        if (!res.equals("OK")) {

         // Writes the content to the file
         writer.write(page + "\t" + urlVal + "\t" + res
           + "\n");
         writer.flush();

        }
       } catch (Exception e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
       }
      }
      AllLinksURL.clear();

    }
   } catch (Exception e) {
    System.out.println(e);
   }
   writer.close();
  } catch (IOException e) {
   System.out.println(e);
   System.out.println("Url file not found");
  }
 }

 public static String check(String urlString) throws Exception {
  String response = "";
  TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
   public java.security.cert.X509Certificate[] getAcceptedIssuers() {
    return null;
   }

   public void checkClientTrusted(X509Certificate[] certs,
     String authType) {
   }

   public void checkServerTrusted(X509Certificate[] certs,
     String authType) {
   }
  } };

  // Install the all-trusting trust manager
  SSLContext sc = SSLContext.getInstance("SSL");
  sc.init(null, trustAllCerts, new java.security.SecureRandom());
  HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

  // Create all-trusting host name verifier
  HostnameVerifier allHostsValid = new HostnameVerifier() {
   public boolean verify(String hostname, SSLSession session) {
    return true;
   }
  };

  // Install the all-trusting host verifier
  HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);

  URL url = new URL(urlString);
  URLConnection con = url.openConnection();
  HttpURLConnection connection = (HttpURLConnection) con;
  connection.connect();

  response = connection.getResponseMessage();
  System.out.println("Response : " + response);
  connection.disconnect();
  return response;
 }
}

########################################################################

########################################################################

WebDriver compare data/css between two pages

Hi All,

In Web based project like implemented in CQ5 where we have to check same site for different country. For this we have to rollout/replicate some/all pages for different countries OR I can say downloaded/uploaded the production data in testing environment or vice versa.
After this we have to validate page static data (like text) with production site Data, page wise validation.
E.g.  One page of US site and replicate the same page for another country then verify page content or css
For this we have to check manually, we can automate the same. I assume both pages having same text content (English).
Logic to implement this:
• First we have to analyze page objects properties (what are the common page objects (elements) properties are being used) like class, id, type etc and create array of this.
• Then we have to get all elements from the base page(expected) using xpath(//*)
• For each element get all properties and value using java script and store only those element properties/value which are matching with common page objects (elements) properties stored in array.
• Create locators for those elements by generating xpath for them.
• Now same locator(xpath) should be exist in actual page
• open two driver instances driver1 for page1 , driver2 for another country page2
• Using WebDriver find element by xpath(locator) from base page and same for another page, match the text using gettext() method if  same element present in both pages
• We can use same for CSS attribute values check for two pages having same element with same type of properties(class, ID etc)

Here is the code to implement this.


public static List<String> getPageLocatorsxPath(WebDriver driverbase,String pageURL )
{
List<String> strLocatorxPath = new ArrayList<>();
/// Common attributes from base site pages
String[] attributes = { "class", "data-class", "data-class-name",
"data-tab", "id", "name", "data-countrycode" };

HashMap<String, String> mapWeblementAttributes = new HashMap<String, String>();

JavascriptExecutor js = (JavascriptExecutor) driverbase;

/// get all elements from page
List<WebElement> childElementsGSAM = driverbase.findElements(By
.xpath("//body//*"));
int counter=0;
for (WebElement webElement : childElementsGSAM) {

// check only those element which are visible and displayed
if (webElement.isDisplayed() & webElement.isEnabled()) {

mapWeblementAttributes.clear();
// get element tag name use for creating Xpath
String strTagName = webElement.getTagName();
String strAttributes = (String) js
.executeScript(
"var str = '', obj = arguments[0].attributes; for (var key in obj){if(obj.hasOwnProperty(key)){str = str + '~' + obj[key].name + '=' + obj[key].value;}}; return str;",
webElement);
// store element properties and value in map
String[] arrAttValues = strAttributes.split("~");

for (String attribute : attributes) {
for (int k = 0; k < arrAttValues.length; k++) {
String[] arrAttrValue = arrAttValues[k].split("=");
if (arrAttrValue.length == 2) {
String strAttr = arrAttValues[k].split("=")[0];
String strAttrValue = arrAttValues[k].split("=")[1];

// store only if element property match with defined properties
if (attribute.equals(strAttr)) {
mapWeblementAttributes.put(strAttr,
strAttrValue);

}
}
}
}
// create element locators only if attribute size >1 like class and id etc..

if (mapWeblementAttributes.size() > 1) {
Set<String> setUIKeys = mapWeblementAttributes.keySet();
String strLocatorStart = "//" + strTagName + "[";
String strLocator = "";
Boolean bAnd = false;
for (String key : setUIKeys) {
if (bAnd) {
strLocator = strLocator + "and@" + key + "='"
+ mapWeblementAttributes.get(key) + "'";
} else {
strLocator = strLocator + "@" + key + "='"
+ mapWeblementAttributes.get(key) + "'";
}
bAnd = true;
}
String finalxpath = strLocatorStart +strLocator + "]";
System.out.println(finalxpath);
strLocatorxPath.add(finalxpath);
counter++;
} else {
System.out.println("not able to create locator for "
+ strTagName
+ mapWeblementAttributes.values().toString());
}


///######################
//// only for class attribute if element has only one class attribute
if (mapWeblementAttributes.size() == 1) {
Set<String> setUIKeys = mapWeblementAttributes.keySet();
String strLocatorStart = "//" + strTagName + "[";
String strLocator = "";
Boolean bAnd = false;
for (String key : setUIKeys) {
if (key.equalsIgnoreCase("class"))
{
if (mapWeblementAttributes.get(key).matches(".*\\d.*"))
{
System.out.println("not able to create locator for "
+ strTagName
+ mapWeblementAttributes.values());


}else
{
if (bAnd) {
strLocator = strLocator + "and@" + key + "='"
+ mapWeblementAttributes.get(key) + "'";
} else {
strLocator = strLocator + "@" + key + "='"
+ mapWeblementAttributes.get(key) + "'";
}
bAnd = true;
String finalxpath = strLocatorStart + strLocator + "]";
System.out.println(finalxpath);
strLocatorxPath.add(finalxpath);
}
}

}

}
///#####################
}
}
        System.out.println("Total xpath locator created from base page " + strLocatorxPath.size());
return strLocatorxPath;
}

------------------------------------------------------#############---------------------

WebDriver driverbase = new FirefoxDriver(profile);
driverbase.manage().window().maximize();
WebDriver driverStatic = new FirefoxDriver(profile);
driverbase.manage().window().maximize();
String page1 = "http://www.google.com/";
String page2 = "http://www.google.com/";
driverbase.get(page2);
driverStatic.get(page1);

///######## Getting elements locator from base page ###################
List<String> strLocatorxPathofBasepage = getPageLocatorsxPath(driverbase, page1);
////##########################
        System.out.println("Total xpath locator created from base page " + strLocatorxPathofBasepage.size());
for (String xpathExpression : strLocatorxPathofBasepage) {
System.out.println("Checking for xpath " + xpathExpression);
        String strLocator = xpathExpression;
try {
                 boolean elementFoundinBasePage = (driverbase.findElements(By.xpath(xpathExpression))
.size() > 0);
                 boolean elementFoundinGivenPage = (driverStatic.findElements(By.xpath(xpathExpression))
.size() > 0);
               
if ((elementFoundinBasePage==true)
& (elementFoundinGivenPage==true)) {
strLocator = xpathExpression;
WebElement childElementGSAM = driverbase.findElement(By
.xpath(xpathExpression));
WebElement childElementStatic = driverStatic.findElement(By
.xpath(xpathExpression));
// call verify css method for css validation
verifyCSSAttributes(WebElement childElementGSAM,WebElement childElementStatic);

System.out.println(driverbase.findElement(
By.xpath(xpathExpression)).getText());
System.out.println(driverStatic.findElement(
By.xpath(xpathExpression)).getText());
/// call verify text method
verifyText(WebElement childElementGSAM,WebElement childElementStatic);
}
}}


##############################################


In this way we can check for multiple pages