Selenium for Victims of Manual Testing (Java Track)
Selenium for Victims of Manual Testing (Java Track)
Introduction to Selenium
Selenium consists of a set of tools
- Selenium IDE
- Firefox add-on
- Simple record and play back of browser interactions
- Selenium WebDriver
- Create programs to automate web browser's
- There are programmatic drivers for most of the common Browsers
- i.e. Firefox, Internet Explorer, Safari, Chrome, etc.
- Given the various web browser drivers - thus the name WebDriver
Selenium IDE Introduction
- Selenium IDE is useful for building starting test cases or test suites
- It allows recording of user interactions with the browser (Firefox)
- It allows building of test cases by adding commands and HTML elements they will act upon
- It allows the test cases to be run (to make sure they work)
- It allows the test cases to be saved in various languages such as Java
For a further discussion of Selenium IDE see Introduction to Selenium (Java Track)
Selenium WebDriver Introduction
- Selenium WebDriver is a tool that automates browsers
- Automation refers to giving commands to a browser
- For example: browser open, browser go to web site page, browser give me the tags of this type, browser close.
- When given the commands the browser can be watched as it opens, goes to pages, closes, etc.
- Selenium WebDriver is a tool that can make assertions about web page data and tags
- Assertions when using Java are from the Java JUnit modules
- Assertions are questions that that can be answered as true or false
- Does this web page have the following data?
- Does this web page have the following HTML tag?
- Does this web page exist?
- Assertions can be failed
- Selenium WebDriver is a tool that can put data into form inputs on a web page
- Put in a username and password
- Put in a search term
- Fill out a profile for a user
- This makes Selenium WebDriver a very powerful tool for the testing of web pages and content
Understanding the Testing Environment
Types of Testing
- Performance Testing
- Identify performance bottlenecks
- Determine throughput
- Determine performance for various system and server configurations
- Functional Testing
- Does system meet original requirement specifications
- Testing is only as good as the test plan
- If requirements are not complete then tests may fail to catch all bugs
- Set up tests to test JIRA reported bugs
- Regression Testing
- rerun sets of standard testing scripts
- determine if system still functions to meet requirements when new versions are to be released
- White Box and Black Box testing
- White Box - find out about the internal structures of the software
- For Example database connections, queries, and inserts should be tested.
- Connections to other systems should be tested.
- Complex algorithms should be tested
- Black box - ignore the internals and just act like a user
- What do users do with the system?
- What types of errors can they make as the use the system?
- Any ECommerce that can go wrong, multiple clicks on the buy button?
- White Box - find out about the internal structures of the software
- Testers built test plans and scripts
- Close cooperation with developers is needed
- Testers must ask questions
- Are there hidden requirements?
- Show us what is supposed to be happening with the system?
- What happens if bad input is put into the system?
- Can we break the system?
Functional Testing
- Testing that the systems meets the requirements
- Does the system behave as it should
- Act like a user even making errors
- Black box and white box
- Black box - act as thought system internals are unknown, i.e. HTML Request where response is built from database
- White box - system internals are known, i.e. test a complex algorithm
- References
Regression Testing
- Determine whether the system still meets requirements after changes or bug fixes
- Rerun tests that deal with parts of the system not included in the changes
- References
Wikipedia on Regression Testing Whatis on Regression Testing
Stress Testing
- Putting a server or the software under stress to see whether it behaves with a large number of users
- Does the server get overloaded and response time suffers?
- Does a database get overloaded and fail or response time suffers?
- Does the software fail due to the load?
- Does the web server fail due to the load?
- Tools must be available to gauge the performance of the hardware and software
Agile and SCRUM
Agile was proposed as an alternative to the Waterfall method. It employs some of the concepts used in the Unified Method.
Agile - These are software development methods in which the software solutions are arrived at through collaborative cross-functional, self-organizing teams. It features iterative development, early delivery, rapid response to changing requirements, and continuous improvement of processes.
Scrum - Refers to a particular Agile methodology as shown in the image below. It places a number of named concepts within the Agile methodology. The word scrum refers to scrum in Rugby where the team attempts to move the ball forward through a rugged collaborative effort (bruises and all).
Agile SCRUM can be said to have a number of organizing principles. These have developed over a number of years through experimentation with the processes of Agile. This list comes from the book Essential Scrum by Kenneth Rubin.
- Prediction and Adaptation - Waterfall is a predictive methodology (predict up front). Scrum is an Adaptive methodology (adapt to changing requirements)
- Validated Learning - validate important assumptions fast, learn through many learning loops
- Work in Progress (WIP) - implement small sized slices of the project so that a large amount of work in progress is not disrupted by changes in requirements or designs. Focus on idle work and not idle workers.
- Progress - measure progress by what has been delivered and validated not by how far we are along a predefined plan
- Performance
- build in quality by moving forward but not hurrying.
- If quality is built in then there will be less costly rework required.
- Value is built in to a high level of confidence validated by testing.
- Have the minimal sufficient amount of ceremony, amount of documentation, and amount of sign off ceremony
- The purpose of documentation
- Part of the product, user manual, user guide, etc.
- Capture important discussions, decisions, or agreements
- Help new team members come on board
- Regulatory requirements
- The purpose of documentation
- In Class Exercise
- Discuss project management methodologies that you are aware of. Where do they fit in terms of their characteristics of Prediction vs Adaptation, Validated Learning, WIP, Measurement of Progress, and Performance of the team.
- Discuss any projects that you have worked on that you are willing to share and any problems and solutions that were encountered.
- Note that Projects to be discussed can be either software, hardware, construction, or mechanical
- References
- Wikipedia on Scrum Software Development)
- Wikipedia on Agile Software Development
- Essential Scrum by Kenneth Rubin, Addison Wesley, January, 2015
Selenium Java Primer
- Selenium WebDriver uses a set of programmatic bindings
- These allow automation of various web browsers
- Such as Firefox, Internet Explorer, Safari, Chrome, etc.
- The bindings are written for several programming languages
- Such as Phython, Java, Ruby, JavaScript, PHP, C#, etc.
- To use Java the language must be understood
- Which is the purpose of the following References from the web
- Depending upon the programming skills of those in attendance go over
- Classes
- Packages
- Methods
- Constructors
- Eclipse Debugging (walk through the code)
- The public static void main method
- Variables (Local and Class Instance)
- Control statements if, if/else, while, for, switch
- Standard Libraries
- Arrays, linked lists, hashmaps
- Inheritance
- Method overloading
- Selenium WebDriver Classes
- JUnit Tests
- Exporting Selenium IDE commands as Java
- References
- TutorialsPoint Java Tutorial
- Jenkov Java Tutorial
- Java Beginners Tutorial
- Java Tutorial
- Oracle Java Tutorials
- Selenium JavaDocs
WebDriver - Using The Java Bindings
- The Java Bindings are a set of classes and functions that enable WebDriver in a Java program
- To use them generally it is best to program in an IDE such as Eclipse
- When stuck with WebDriver, solutions can be found:
- By recording a session with Selenium IDE and then export to Java
- By building a session in Selenium IDE manually while testing each step
- Does each step work?
- Try different locators when the commands with locators are failing
- Put in echo statements to debug
- Put in pauses to see what is happening
- Reduce the complexity of the HTML so it is easier to work with a page
- You May need a web server of your own to test the commands
- By web searches for solutions to the problem
- The general steps in using the bindings are listed in an approximate order below
Choose which WebDriver browser to use
- There are drivers for most common browsers
- The driver can be set up in the code of a Java application
- The driver can be set up in the setup method of a JUnit test (see section below)
- In the image below the various Browser classes can be seen in Eclipse
Load a Page
- To load a page the webdriver Browser classes have get methods
For example in the Firefox driver the code is:
driver.get("http://localhost:8080/Web_Project_1/seleniumTest.jsp");
- Generally it is a good practice to wait until the web page has loaded before using it
- This can be done using either an implicit or explicit wait
Explicit or Implicit Wait
- Mixing explicit and implicit waits can cause unpredictable waits
Explicit Waits
- Explicit waits makes the web driver wait for a certain condition to occur
- This can be done using WebDriverWait along with an ExpectedCondition
- It should be noted that the locator is very important.
- XPath seems to be a good way of specifying the locator
- Note that this example is one where a page is loaded
- It is not using AJAX
Example in Java
WebDriver driver = new FirefoxDriver(); driver.get("http://somedomain/url_that_delays_loading"); WebElement myDynamicElement = (new WebDriverWait(driver, 10)) .until(ExpectedConditions.presenceOfElementLocated(By.id("myDynamicElement")));
- Common Expected Conditions
- Most of these work with a page being loaded
- Only some of them work with AJAX
- For example the title of the page will not change with AJAX
- However, some elements, their attributes, or their child text contents will change with AJAX
titleIs titleContains presenceOfElementLocated visibilityOfElementLocated visibilityOf presenceOfAllElementsLocated textToBePresentInElement textToBePresentInElementValue frameToBeAvailableAndSwitchToIt invisibilityOfElementLocated elementToBeClickable - it is Displayed and Enabled. stalenessOf elementToBeSelected elementSelectionStateToBe alertIsPresent
See the topic further below on AJAX
References
Implicit Waits
- Implicit waits make the web driver wait for a certain amount of time
- Once set in the WebDriver it stays until the WebDriver quits
Example of Implicit Wait
import org.openqa.selenium.firefox.FirefoxDriver; public class BuildFormsTestCase { private WebDriver driver; @Before public void setUp() throws Exception { driver = new FirefoxDriver(); driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS); }
Locating UI Elements
- For HTML pages the elements are the HTML tags i.e. div, a, span, input, iframe, etc.
- A tag usually needs to be located so that it can be clicked, have text entered, be selected, etc.
- There are several ways to locate an element on the page
- The most useful of these are listed below
By id
- Find an element by using its id attribute
- For example the html and its locator would be
The HTML: <div id="idOfElement">some text or other HTML</div> The locator examples: element = driver.findElement(By.id("idOfElement")) driver.findElement(By.id("div1")).click()
Note: pages often have the same ids for numerous elements so the use of this locator may be limited
By Name
- Find an element using the name attribute
The HTML: <input type="text" name="FirstName" value="First Name"> The locator example: driver.findElement(By.name("FirstName")).clear();
ByXPath
- XPath is a very useful locator since it can distinguish most elements
- For AJAX modified or created elements it works very well
The HTML: <input type="checkbox" name="vehicle" value="Bike"> I have a bike<br> The locator example: driver.findElement(By.xpath("//input[@value='Bike']")).click();
By Link Text
- Locate a link ("a" tag> element by the link text
The HTML: <a href="formsDemo.jsp">Forms Demo</a> The locator: driver.findElement(By.linkText("Forms Demo")).click();
By Partial Link Text
- Find a link ("a" tag) by partial link text
The HTML <a href="formsDemo.jsp">Forms Demo</a> The locator: driver.findElement(By.partialLinkText("Forms Demo")).click();
- References
User Input in Forms
- HTML Forms take input from users
- In general they have:
- Textboxes
- Checkboxes
- Radio Buttons
- Select dropdowns with options
- Testing these requires
- locating the correct element
- Entering text
- Clicking on elements and options
- Clicking on submit
- Checking the data on the action page
- The action page is the page that submit invokes
- One of the most useful ways of locating an element is by XPath
# Example of entering text driver.findElement(By.name("FirstName")).clear(); driver.findElement(By.name("FirstName")).sendKeys("Hermione"); # Example of clicking checkboxes if (driver.findElement(By.xpath("//input[@value='Bike']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Bike']")).click(); # Example of click radio buttons driver.find_element_by_xpath("(//input[@name='gender'])[2]").click() # Example of choosing an entry in a drop down select list new Select(driver.findElement(By.xpath("//select[@name='selectBox1']"))).selectByVisibleText("Audi"); # Example of click the submit button findElement(By.xpath("//input[@value='Submit']")).click();
- Hands on Exercise
- Go to the page on the Tomcat server
http://ServerAddress:8080/Web_Project_1/userForm.jsp
- Write a WebDriver program that will:
- fill the username and password
- Click on the submit button
- Assert whether the login has succeeded
- Prove that the login effects the page "anotherPage.jsp"
Handle Alerts
- Alerts are handled with the Alert class
- Accept and dismiss alerts
- Enter Text into prompt alerts
- The alert can be switched_to using the webdriver object
Example of code that popups up an Alert
<p>Click the button to display a confirm box.</p> <button onclick="myFunction1()">Try it</button> <p id="demo"></p> <script> <!-- function myFunction1() { var x; if (confirm("Press a button!") == true) { x = "You pressed OK!"; } else { x = "You pressed Cancel!"; } document.getElementById("demo").innerHTML = x; } //--> </script>
Examples of Java Selenium JUnit Alert Testing code
package com.example.tests; import java.util.concurrent.TimeUnit; import org.junit.*; import static org.junit.Assert.assertEquals; import static org.junit.Assert.fail; import org.openqa.selenium.*; import org.openqa.selenium.firefox.FirefoxDriver; public class AlertTestCase { private WebDriver driver; private boolean acceptNextAlert = true; private StringBuffer verificationErrors = new StringBuffer(); @Before public void setUp() throws Exception { driver = new FirefoxDriver(); driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS); } @Test public void testAlertTestCase() throws Exception { driver.get("http://localhost:8080/Web_Project_1/seleniumTest.jsp"); driver.findElement(By.linkText("Popup Window Demos")).click(); driver.findElement(By.xpath("//button[@onclick='myFunction1()']")).click(); Alert alert = driver.switchTo().alert(); String alertText = alert.getText(); assertEquals("Press a button!", alertText); String pageSource = driver.getPageSource(); int index = pageSource.indexOf("You pressed OK!"); boolean contains = index >= 0; assertEquals(contains, true); } @After public void tearDown() throws Exception { driver.quit(); }
- Note that when using Selenese the Alerts do not actually popup
- Although they test correctly
- Note that when using the Java bindings the Alerts do popup
Hands on Exercise
- If possible work in groups to do this hands on exercise
- Using the example code for putting up an Alert on a web page
- Using the example Selenium code for testing an alert
- Do the following
- Write a simple set of HTML code that will put up an Alert
- Put it onto the test Tomcat server
- Prove that it works by accessing it
- Write a Selenium program in Java that will test the Alert
- Run the Selenium test against the web page
- References
Selenium Alert Handling JavaScript Alerts
Interact with Popup Windows
- Popup Windows can be invoked by:
- JavaScript (see the example below)
- Targets on a link ("a" tag)
- To test these windows it is necessary to switch to them and then test
Example HTML code that will be in a popup window
<html> <head> <meta charset="ISO-8859-1"> <title>New Popup Window</title> </head> <body> <h1>This is a popup</h1> <h2>Please fill in the form</h2> <br/> <form action="seleniumTestAction.jsp"> First name: <input type="text" name="FirstName" value="First Name"><br> <br/><br/> <input type="submit" value="Submit"> </form> </body> </html>
Code that pops up the window
<a href="smallPopup.html" onClick="return popup(this, 'notes')">Display a popup</a> <SCRIPT TYPE="text/javascript"> <!-- function popup(mylink, windowname) { if (! window.focus)return true; var href; if (typeof(mylink) == 'string') href=mylink; else href=mylink.href; window.open(href, windowname, 'width=400,height=200,scrollbars=yes'); return false; }
Selenium test code for popup window
- Note that this test uses the window title to find its handle
- The window url could also be used since the driver is being asked for title (or URL) of each open window
public void testPopupWindowTestCase() throws Exception { driver.get("http://localhost:8080/Web_Project_1/seleniumTest.jsp"); driver.findElement(By.linkText("Popup Window Demos")).click(); driver.findElement(By.xpath("//a[contains(@href, 'smallPopup.html')]")).click(); String newWindowHandle = switchToWindowUsingTitle("New Popup Window") driver.findElement(By.name("FirstName")).clear(); driver.findElement(By.name("FirstName")).sendKeys("John"); driver.findElement(By.xpath("//input[@value='Submit']")).click(); } private String switchToWindowUsingTitle(String windowTitle) { Set<String> windows = driver.getWindowHandles(); for (String window : windows) { driver.switchTo().window(window); if (driver.getTitle().contains(windowTitle)) { return driver.getWindowHandle(); } } return "Window Not Found"; }
- Node that when using Selenese the window does not actually popup
- Although they test correctly
- In Java WebDriver it does popup correctly
Hands on Exercise
- Build a test program using the Selenium Test Code above
- Test the popup window that can be seen at:
- Go to http://servername:8080/Web_Project_1/seleniumTest.jsp
- Go To link Popup Window Demos
- Go to link Display a Test Popup
- Moving backwards and forwards in history is accomplished with commands
- driver.navigate().forward();
- driver.navigate().back();
Cookies
- With Selenium Cookies can be added, deleted, and retrieved
- For example, making sure that a cookie was correctly added can be tested
The following example of cookie use is from the Selenium website
- Reference See Selenium Site on WebDriver
# Go to the correct domain driver.get("http://www.example.com") # Now set the cookie. Here's one for the entire domain # the cookie name here is 'key' and its value is 'value' Cookie cookie = new Cookie("key", "value"); driver.manage().addCookie(cookie); # And now output all the available cookies for the current URL Set<Cookie> allCookies = driver.manage().getCookies(); for (Cookie loadedCookie : allCookies) { System.out.println(String.format("%s -> %s", loadedCookie.getName(), loadedCookie.getValue())); } # You can delete cookies in 2 ways # By name driver.manage().deleteCookieNamed("CookieName"); # Or all of them driver.manage().deleteAllCookies();
Compare Python to Java Bindings
- Here is the code for testing the same page in Python and Java
- The first uses the Python Selenium Bindings
- The second uses the Java Selenium Bindings
- Both sets of code came from using the Selenium IDE to prototype and then export to the two languages
- The code was corrected after export to eliminate small errors made by Selenium in translating from Selenese
- Note that both sets of code have several methods that are courtesy methods that can be used by the programmer
- For example to work with JavaScript Alerts
- The methods that are not test, setup, or teardown
Python Bindings code
class BuildFormsTestCase(unittest.TestCase): def setUp(self): self.driver = webdriver.Firefox() self.driver.implicitly_wait(30) self.base_url = "http://localhost:8080/Web_Project_1/seleniumTest.jsp" self.verificationErrors = [] self.accept_next_alert = True def test_build_forms_test_case(self): driver = self.driver driver.get("http://localhost:8080/Web_Project_1/seleniumTest.jsp") driver.find_element_by_link_text("Forms Demo").click() print("going to write in text boxes") driver.find_element_by_name("FirstName").clear() driver.find_element_by_name("FirstName").send_keys("Hermione") driver.find_element_by_name("LastName").clear() driver.find_element_by_name("LastName").send_keys("") driver.find_element_by_name("LastName").send_keys("Gran") driver.find_element_by_name("LastName").send_keys("ger") print("going to check boxes") if not driver.find_element_by_xpath("//input[@value='Bike']").is_selected(): driver.find_element_by_xpath("//input[@value='Bike']").click() if driver.find_element_by_xpath("//input[@value='Bike']").is_selected(): driver.find_element_by_xpath("//input[@value='Bike']").click() print("Going to click on I have a bike again") if driver.find_element_by_xpath("//input[@value='Car']").is_selected(): driver.find_element_by_xpath("//input[@value='Car']").click() if not driver.find_element_by_xpath("//input[@value='Car']").is_selected(): driver.find_element_by_xpath("//input[@value='Car']").click() if driver.find_element_by_xpath("//input[@value='Car']").is_selected(): driver.find_element_by_xpath("//input[@value='Car']").click() if not driver.find_element_by_xpath("//input[@value='Car']").is_selected(): driver.find_element_by_xpath("//input[@value='Car']").click() if driver.find_element_by_xpath("//input[@value='Car']").is_selected(): driver.find_element_by_xpath("//input[@value='Car']").click() if not driver.find_element_by_xpath("//input[@value='Car']").is_selected(): driver.find_element_by_xpath("//input[@value='Car']").click() print("going to select dropdown") Select(driver.find_element_by_xpath("//select[@name='selectBox1']")).select_by_visible_text("Audi") print("going to radio buttons") driver.find_element_by_xpath("(//input[@name='gender'])[2]").click() driver.find_element_by_xpath("//input[@name='gender'][1]").click() driver.find_element_by_xpath("(//input[@name='gender'])[2]").click() driver.find_element_by_xpath("//input[@value='Submit']").click() def is_element_present(self, how, what): try: self.driver.find_element(by=how, value=what) except NoSuchElementException as e: return False return True def is_alert_present(self): try: self.driver.switch_to.alert() except NoAlertPresentException as e: return False return True def close_alert_and_get_its_text(self): try: alert = self.driver.switch_to.alert() alert_text = alert.text if self.accept_next_alert: alert.accept() else: alert.dismiss() return alert_text finally: self.accept_next_alert = True def tearDown(self): self.driver.quit() self.assertEqual([], self.verificationErrors) if __name__ == "__main__": unittest.main()
Java Bindings code
package com.example.tests; import java.util.regex.Pattern; import java.util.concurrent.TimeUnit; import org.junit.*; import static org.junit.Assert.*; import static org.hamcrest.CoreMatchers.*; import org.openqa.selenium.*; import org.openqa.selenium.firefox.FirefoxDriver; import org.openqa.selenium.support.ui.Select; public class BuildFormsTestCase { private WebDriver driver; private String baseUrl; private boolean acceptNextAlert = true; private StringBuffer verificationErrors = new StringBuffer(); @Before public void setUp() throws Exception { driver = new FirefoxDriver(); baseUrl = "http://localhost:8080/Web_Project_1/"; driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS); } @Test public void testBuildFormsTestCase() throws Exception { driver.get("http://localhost:8080/Web_Project_1/seleniumTest.jsp"); driver.findElement(By.linkText("Forms Demo")).click(); System.out.println("going to write in text boxes"); driver.findElement(By.name("FirstName")).clear(); driver.findElement(By.name("FirstName")).sendKeys("Hermione"); driver.findElement(By.name("LastName")).clear(); driver.findElement(By.name("LastName")).sendKeys(""); driver.findElement(By.name("LastName")).sendKeys("Gran"); driver.findElement(By.name("LastName")).sendKeys("ger"); System.out.println("going to check boxes"); if (!driver.findElement(By.xpath("//input[@value='Bike']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Bike']")).click(); }; if (driver.findElement(By.xpath("//input[@value='Bike']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Bike']")).click(); }; System.out.println("Going to click on I have a bike again"); if (driver.findElement(By.xpath("//input[@value='Car']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Car']")).click(); }; if (!driver.findElement(By.xpath("//input[@value='Car']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Car']")).click(); }; if (driver.findElement(By.xpath("//input[@value='Car']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Car']")).click(); }; if (!driver.findElement(By.xpath("//input[@value='Car']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Car']")).click(); }; if (driver.findElement(By.xpath("//input[@value='Car']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Car']")).click(); }; if (!driver.findElement(By.xpath("//input[@value='Car']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Car']")).click(); }; System.out.println("going to select dropdown"); new Select(driver.findElement(By.xpath("//select[@name='selectBox1']"))).selectByVisibleText("Audi"); System.out.println("going to radio buttons"); driver.findElement(By.xpath("(//input[@name='sex'])[2]")).click(); driver.findElement(By.xpath("//input[@name='sex'][1]")).click(); driver.findElement(By.xpath("(//input[@name='sex'])[2]")).click(); driver.findElement(By.xpath("//input[@value='Submit']")).click(); } @After public void tearDown() throws Exception { driver.quit(); String verificationErrorString = verificationErrors.toString(); if (!"".equals(verificationErrorString)) { fail(verificationErrorString); } } private boolean isElementPresent(By by) { try { driver.findElement(by); return true; } catch (NoSuchElementException e) { return false; } } private boolean isAlertPresent() { try { driver.switchTo().alert(); return true; } catch (NoAlertPresentException e) { return false; } } private String closeAlertAndGetItsText() { try { Alert alert = driver.switchTo().alert(); String alertText = alert.getText(); if (acceptNextAlert) { alert.accept(); } else { alert.dismiss(); } return alertText; } finally { acceptNextAlert = true; } } }
WebDriver - Using Java JUnit
- JUnit specifies a series of test cases that are to be run against a web site
- It allows a setUp method and a tearDown method
- These two are run before and after each test case in the Unit Test
- Each test must be a method that starts with "test"
- Assert methods are provided by the Testcase base class
- Tests can be skipped by using appropriate decoration
- i.e. @unittest.skip("demonstrating skipping")
Example of UnitTest
package com.example.tests; import java.util.concurrent.TimeUnit; import org.junit.*; import static org.junit.Assert.*; import org.openqa.selenium.*; import org.openqa.selenium.firefox.FirefoxDriver; import org.openqa.selenium.support.ui.Select; public class BuildFormsTestCase { private WebDriver driver; private boolean acceptNextAlert = true; private StringBuffer verificationErrors = new StringBuffer(); @Before public void setUp() throws Exception { driver = new FirefoxDriver(); driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS); } @Test public void testBuildFormsTestCase() throws Exception { driver.get("http://localhost:8080/Web_Project_1/seleniumTest.jsp"); // the test code goes here } @After public void tearDown() throws Exception { driver.quit(); String verificationErrorString = verificationErrors.toString(); if (!"".equals(verificationErrorString)) { fail(verificationErrorString); } } }
- JUnit has a testSuite class which can aggregate tests.
- Hands on Exercise - Install Java JDK:
- Go over JUnit at the the following url TutorialsPoint JUnit Tutorial
- Create a test that will go to the Google homepage and check for some text on the site (assertion)
- The test should have Before, After and Test methods
- Create a second test that will go to the Yahoo website and check for some text (assertion)
- Create a test suite that includes these two tests.
WebDriver - Using TestNG
- Testing code in various ways - unit, functional, regression, and integration
- Use thread pools in various ways
- Use dependencies between test methods, i.e. method 2 depends upon method 1 being successful
@Test public correctVM() {} @Test public serverStartedOk() {} @Test(dependsOnMethods = { "correctVM", "serverStartedOk" }) public method1() {} ...
- Test Groups and group dependencies
@Test(groups = { "init" }) public correctVM() {} @Test(groups = { "init" }) public serverStartedOk() {} @Test(dependsOnGroups = { "init.* }) public method1() {} ...
Simple Example of Selenium using TestNG
- An example of a TestNG Selenium test
- Note the BeforeTest, AfterTest and Test annotations
- Note the usage of the Selenium WebDriver bindings
package com.example.tests; import org.openqa.selenium.firefox.FirefoxDriver; import org.openqa.selenium.support.ui.Select; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; import static org.testng.Assert.*; //used for assertions import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import java.util.concurrent.TimeUnit; public class BuildFormsTestNG { private WebDriver driver; String verificationErrorString = ""; //used to set up any verification errors @BeforeTest public void setUp() throws Exception { driver = new FirefoxDriver(); driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS); } @Test public void testBuildFormsTestNG() throws Exception { driver.get("http://localhost:8080/Web_Project_1/seleniumTest.jsp"); driver.findElement(By.linkText("Forms Demo")).click(); driver.findElement(By.name("FirstName")).clear(); driver.findElement(By.name("FirstName")).sendKeys("Hermione"); driver.findElement(By.name("LastName")).clear(); driver.findElement(By.name("LastName")).sendKeys(""); driver.findElement(By.name("LastName")).sendKeys("Gran"); driver.findElement(By.name("LastName")).sendKeys("ger"); if (!driver.findElement(By.xpath("//input[@value='Bike']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Bike']")).click(); }; if (driver.findElement(By.xpath("//input[@value='Car']")).isSelected()) { driver.findElement(By.xpath("//input[@value='Car']")).click(); }; new Select(driver.findElement(By.xpath("//select[@name='selectBox1']"))).selectByVisibleText("Audi"); driver.findElement(By.xpath("(//input[@name='gender'])[2]")).click(); driver.findElement(By.xpath("//input[@value='Submit']")).click(); } @AfterTest public void tearDown() throws Exception { driver.quit(); if (!"".equals(verificationErrorString)) { fail(verificationErrorString); } } }
Install the Eclipse TestNG plugin
- There is a TestNG Eclipse Plugin
- Install is done from within Eclipse
- See the link Install TestNG in Eclipse
Page Object Model
- This is a design pattern where the automation of a page is separated from the verification of the page.
WebDriver - Read and Write Files
- File can be written and read from within the Java Test code
- The standard Java techniques of opening, reading, and writing files are used
Example of writing a file
PrintWriter writer = new PrintWriter("filename.txt", "UTF-8"); writer.println("The first line"); writer.println("The second line"); writer.close();
Hands on Exercise Write a test that does the following
- read a username and password from a file
- Use those to test the user login on the web page:
- http://nameOfServer:8080/Web_Project_1/userForm.jsp
- This is a page in the Web Project 1 WAR file
- See the Section on Selenium Sample Zip Files
- References
Ways of writing a file Reading and writing files Read a file
WebDriver - Parse a Web Page
- Parsing a web page is done using Java code within your test
- It can be done by manipulating Strings using String methods i.e. substrings and indexOf etc.
- It can be done by using xpath or css locators with the driver findElements method
An Example of Parsing a web page to find many elements
- Find all of the top stories on the Boston Globe
- Check to see if the urls have content on their pages
@Test public void testPopupWindowTestCase() throws Exception { driver.get("http://www.boston.com/?refresh=true"); List<WebElement> urlList = driver.findElements(By.xpath("//ul[@class=\"headline-list__list js-split-list js-add-p1\"]/li[@class=\"headline-list__item\"]/a")); for (WebElement listItem :urlList){ String href = listItem.getAttribute("href"); System.out.println("the top story url is " + href); // used to debug //get the pages here and check them for content with assert statements } }
WebDriver with AJAX Modified Pages
- What is AJAX
- AJAX stands for Asynchronous JavaScript and XML
- In JavaScript communication occurs with a server
- The server data is then used to modify a portion of a page
- A page reload does not occur
- Thus portions of a page can be modified with data from a server
- Testing AJAX
- Page loading does not occur
- The tester must determine what constitutes a change on the page
- What are the characteristics of the change
- Text change?
- Elements are added?
- Element contents change?
- What are the characteristics of the change
- Once the change characteristics are determined
- A test must be devised that can recognize the changes
- The test must wait for the changes to occur
- Network and server delay time is possible
An Example of AJAX from a JavaServer Page (JSP)
<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"> <title>Test AJAX functionality</title> <script> function loadDoc() { // set up to make an ajax request to a server var httpRequest; if (window.XMLHttpRequest) { // Mozilla, Safari, IE7+ ... httpRequest = new XMLHttpRequest(); } else if (window.ActiveXObject) { // IE 6 and older httpRequest = new ActiveXObject("Microsoft.XMLHTTP"); } //define the function that will handle the response from the server // note that this is a javascript shorthand method of defining an anonymous function httpRequest.onreadystatechange = function() { if (httpRequest.readyState==4 && httpRequest.status==200) { document.getElementById('div1').innerHTML=httpRequest.responseText; } } // Make the request to the server once the response occurs it will jump to the handler function httpRequest.open('GET', 'http://localhost:8080/Web_Project_1/ajaxResponseServlet', true); httpRequest.send(null); } </script> </head> <body> <div id="div1" style="cursor: pointer" onclick="loadDoc()">Click Here for AJAX response</div> </body> </html>
Changes caused by the AJAX call are
The div tag with id="div1" has its innerHTML (i.e. Click Here for AJAX response) replaced with: <div>Ajax response from Servlet</div> <div>Hello Glad to hear from you</div> <div>It is a nice day</div> This gives a different a structure to the html of the page along with different content
- How to test AJAX changes to a page
- Load the page
- Perform an action that invokes the AJAX code to change the page
- Wait for the change to occur
- Check that the changes is correct with an Selenium WebDriver assertion
- Or a Selenium IDE assert or verify command (when prototyping your code)
Example of WebDriver Code to test an AJAX change
@Test public void testAjaxWaitForElementPresent() throws Exception { driver.get("http://localhost:8080/Web_Project_1/ajaxTest.jsp"); driver.findElement(By.id("div1")).click(); //wait for a div within div1 to show up WebDriverWait wait = new WebDriverWait(driver,30); wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//div[@id='div1']/div"))); /* code as generated from Selenium IDE was for (int second = 0;; second++) { if (second >= 60) fail("timeout"); try { if (isElementPresent(By.xpath("//div[@id='div1']/div"))) break; } catch (Exception e) {} Thread.sleep(1000); } */ try { assertTrue(driver.findElement(By.cssSelector("BODY")).getText().matches("^[\\s\\S]*response from Servlet[\\s\\S]*$")); } catch (Error e) { verificationErrors.append(e.toString()); } }
- Hands on Exercise
- Start the Tomcat server that has Web_Project_1 on it as an application
- Determine how to write a test that will check for the contents of the second div tag that is a child of the //div[@id='div1'] tag
- Use an XPath to do the above
- Use JUnit tests to specify your test case
- Write an assertion
- Make the unit test both succeed and fail by changing your Java code
- Most likely changing the assertion makes the most sense
- See what happens if you introduce an error in the XPath
References
Running WebDriver JUnit Test Suite manually
- From Eclipse export the project as a Runnable Jar File.
- Put the jar file in a location that you can easily find
- In Windows open a command prompt
- Move to the folder that has the Runnable Jar File
- execute the command java given below
java -jar NameOfTheJarFile.jar NameOfTheTestRunner
- NameOfTheJarFile is the name given to the runnable jar
- NameOfTheRestRunner is the test runner class created to run the test suite
- See the section in this document on Building a Test Suite
Reusing Code with JAR files
- Common code used in tests can be reused by packaging it in JAR files
- For example reading files or writing files
- To use JAR files
- The JAR files are placed in the classpath
- The tests can then use the JAR file classes by putting them into the -cp argument if running Java manually
- Note that in Eclipse the JAR files would be included in the Build Path for the project
Building a test suite
- Java JUnit can be used to build test suites
- A test Suite runs several java files with test cases at the same time
- A Test Suite class and a Runner class are required
Example of building and running a test suite
The Test Suite class
- Note that it includes two of the test cases that were already written
package com.example.tests; import org.junit.runner.RunWith; import org.junit.runners.Suite; @RunWith(Suite.class) @Suite.SuiteClasses({ AlertTestCase.class, BuildFormsTestCase.class }) public class JunitTestSuite { }
The Test Runner class
- It is just a main method that will run and start the test suite
package com.example.tests; import org.junit.runner.JUnitCore; import org.junit.runner.Result; import org.junit.runner.notification.Failure; public class TestRunner { public static void main(String[] args) { Result result = JUnitCore.runClasses(JunitTestSuite.class); for (Failure failure : result.getFailures()) { System.out.println(failure.toString()); } System.out.println(result.wasSuccessful()); } }
Selenium with Chrome
- To use Selenium with Chrome the chromeDriver.exe file must be downloaded to a well known location
- Its location must be put into a system property
- The Java code is changed to the following
Note the addition of the System.setProperty for the location of the chrome driver
public class BuildFormsTestCaseChrome { private WebDriver driver; private boolean acceptNextAlert = true; private StringBuffer verificationErrors = new StringBuffer(); @Before public void setUp() throws Exception { System.setProperty("webdriver.chrome.driver", "C:\\WellKnownFolderPath\\chromeDriver.exe"); driver = new ChromeDriver(); driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS); }
- References
Selenium Grid
- Selenium Grid is use to run multiple tests in a distributed environment
- The computer controlling the tests is called the hub
- The computers being controlled are called the nodes
- Using a distributed environment allows:
- Testing faster depending on the number on nodes used
- Note that this is not intended to stress test as you would with a tool like JMeter
- For example JMeter slave nodes can simulate between 200 and 800 users per node
- Testing with varying operating systems on the various nodes
- Testing with varying browsers being used on the various nodes
- Testing with various test sets per node
- Testing faster depending on the number on nodes used
Example of Installing and Running the Grid
- Download the Selenium Server from Selenium Downloads
- The hub and all nodes should have the same Selenium Server version
- The download is a jar file which will be run using java
- Note that you will need to have Java installed on the hub and node computers
- It is best to have the same Java version on each computer
- The Java JDK (Java Development Kit) is the most useful since it has most tools
- It also has the JRE (Java Runtime Environment)
- It can be downloaded from Oracle Java Standard Edition (SE)
- See the Section in the Contents "Install the Java SE JDK"
- Place the Selenium Server jar file in a location that is well known
- Such as C:\SeleniumServer on Windows
- It is best to place the jar files in approximately the same location on each computer
- With slight variations caused by the various operating systems
- Start the hub on the computer where you intend to control the grid
- Open a command prompt and move to the directory where the jar file exists
- Start the hub using the command java -jar selenium-server-standalone-2.45.0.jar -role hub
- The hub by default starts on port 4444
- Check that the hub is running using a browser
- The Selenium Documentation recommends turning off the Firewall on the hub and nodes
- With the Firewall you may get connection errors
- Go to URL http://localhost:4444/grid/console or
- Click this link once the hub is started Hub Web Interface
- The Grid Console runs in a Browser
- The Grid Console will look like the following image once the remote host is started and it is refreshed
- The Selenium Documentation recommends turning off the Firewall on the hub and nodes
- Check that the node computer can see the hub by doing the same command on the node
- Replace the "localhost" with the IP address of the hub
- Find the hub IP address using on Windows ipconfig in a command prompt
- Look for IPv4 Address
- Start a node computer using the following command:
- java -jar selenium-server-standalone-2.45.0.jar -role node -hub http://ipaddressOfHub:4444/grid/register -port 5555
- The java -jar tells Java to run the main method within the indicated jar file
- The name of the jar file is next selenium-server-stand...
- The -role node (can also be webdriver) indicates that this is a node not the hub
- The ipaddressOfHub should be the IP address of the computer running the hub
- The 4444 is the port that the hub came up on, see the hub command above
- The -port 5555 is the port that the node will come up on
- java -jar selenium-server-standalone-2.45.0.jar -role node -hub http://ipaddressOfHub:4444/grid/register -port 5555
- Check that the node has registered by looking at the Hub's grid console
- It should look like the prior image and show the connected nodes
- Seeing the nodes and their configuration is the consoles main purpose (at this time)
- The Hub's configuration can also be seen in the console
Example of a Java JUnit that is run on the Selenium Grid
- To get a Java unit test to run on a node it driver code must be changed
The code as run locally is:
@Before public void setUp() throws Exception { driver = new FirefoxDriver(); driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS); }
The changed code to run on a Grid node is:
- Note that the IP address and port is the hub location and not the node
- The hub determines which node is to be used for the test and transfers the testing on to that node
@Before public void setUp() throws Exception { nodeURL="http://10.0.0.5:4444/wd/hub"; DesiredCapabilities capability = DesiredCapabilities.firefox(); capability.setBrowserName("firefox"); capability.setPlatform(Platform.XP); URL theURL = new URL(nodeURL); driver = new RemoteWebDriver(theURL, capability); driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS); }
- The command executor tells the location of the hub
- The hub determines which node will run this set of tests
- The desired capabilities (platform and browserName) comes from a roll-over of the browser icons in the node's Grid Console entry
- For example, for Firefox roll the mouse over the Firefox icons under WebDriver in the console
- For some reason the roll over says the platform is VISTA even though the node had Windows 7 in this example
- Note that desired capabilities is a dictionary and therefore entries are in that form
- For example, for Firefox roll the mouse over the Firefox icons under WebDriver in the console
- Once this change is made to the code the code is run on the hub computer
- The results (success or failure) come back to the program that is run on the hub
- Not to the console window (which might be expected)
- The results (success or failure) come back to the program that is run on the hub
- The test run can be seen in the command prompt on the node computer.
- The browser window will open and walk through the test on the node computer.
- The command window will look similar to the one below as the test is run
Hands on Exercise
- Using the instructions above start a hub and a node and run some tests
- Use the test files from the Selenium Sample Zip Files section at the end of this document
- The zip file to download and extract is under link "Eclipse Selenium Project"
- Use Eclipse and the files in the zipped folder EclipseSelenium.zip
- Import the project into Eclipse
- Use the BuildFormsTestCaseGrid.java file as the test case
- Use the test files from the Selenium Sample Zip Files section at the end of this document
Reporting Tools for Selenium
- Selenium runs as either JUnit or TestNG test
- Organizing the test results is useful
- Methods of organizing the results are
- Using HTML and web pages
- Using Excel
- Writing results to a file in some other format
ExtentReports and HTML
- ExtentReports is a Web Page building framework
- Fairly easy to use
- Open Source Reporting Library
- Used with JUnit or TestNG testing
- Works well with Selenium testing in Java
Install ExtentReports
- Download the ExtentReports zip file and extract it
- Install the ExtentReports jar file and the lib jar files into the Eclipse Build Path for your Selenium Project
- Or the build path for the IDE that you are using
Using ExtentReports
- The first task is to create a ExtentReport object
- It will be used by the various tests
- Generally in a single test class new report entries are appended to the report
public class ExtentReportsTest { // create the extent report with a file path, to append to the file // a parameter can also be put in to force the newest tests to be first ExtentReports extent; @Before // could use an @BeforeClass instead public void setUp() throws Exception { extent = new ExtentReports("some file path\\extentreport1.html", false);
- Configure the report
- Look at the classes in the JavaDocs to see the various configurations
@Before // could use an @BeforeClass instead public void setUp() throws Exception { extent = new ExtentReports("some file path\\extentreport1.html", false); extent.config().documentTitle("Test on AJAX and other tests"); extent.config().reportHeadline("This is the headline"); extent.config().reportName("The Report Name");
- Start the test report entry within each @Test method
- The @Test methods are JUnit annotated methods
@Test public void testAjaxWaitForElementPresent() throws Exception { ExtentTest test = extent.startTest("TestAjaxWaitForElement", "A test of Ajax creating an element");
- Within each @Test method log test results to the report
- Note that if the Assert statement fails in this case it jumps to the catch statement
- It skips over the LogStatus.PASS statement
try { assertTrue(driver.findElement(By.cssSelector("BODY")).getText() .matches("^[\\s\\S]*response from Servletxx[\\s\\S]*$")); test.log(LogStatus.PASS, "Step Name", "Test Passed"); } catch (Error e) { test.log(LogStatus.FAIL, "Step Name", "Error in AJAX, AJAX did not build page correctly"); }
- Add a screenshot if so desired
- Note that the screenshot is taken by Selenium
- Then passed into the report log
- This takes two lines of code
- Then passed into the report log
- Note that the path used by FileUtils.copyFile places the screenshot into the file system
- test.addScreenCapture uses the image in the report that is placed at the path by copyFile
- Note that the screenshot is taken by Selenium
} catch (Error e) { test.log(LogStatus.FAIL, "Step Name", "Error in AJAX, AJAX did not build page correctly"); File src= ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE); FileUtils.copyFile(src, new File("Screenshot path\\screenshot1.png")); test.log(LogStatus.INFO, "Snapshot below: " + test.addScreenCapture("Screenshot Path\\screenshot1.png")); }
- End the test report
- The entire @Test method looks like the following
- Note that this sample code was from a test that was on a web page doing AJAX calls
- Therefore it used the explicit wait rather than an implicit wait
- The last statement in this method is the extent.endTest method
- Note that the Selenium driver creation code was removed from the code above
- So that it could focus on the ExtentReports
- It does show up however in the code below
@Test public void testAjaxWaitForElementPresent() throws Exception { ExtentTest test = extent.startTest("TestAjaxWaitForElement", "A test of Ajax creating an element"); driver.get("http://some url to be tested"); driver.findElement(By.id("div1")).click(); // wait for a div within div1 to show up WebDriverWait wait = new WebDriverWait(driver, 30); wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//div[@id='div1']/div"))); try { assertTrue(driver.findElement(By.cssSelector("BODY")).getText() .matches("^[\\s\\S]*response from Servletxx[\\s\\S]*$")); test.log(LogStatus.PASS, "Step Name", "Test Passed"); } catch (Error e) { test.log(LogStatus.FAIL, "Step Name", "Error in AJAX, AJAX did not build page correctly"); File src= ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE); FileUtils.copyFile(src, new File("Screenshot Path\\screencast1.png")); test.log(LogStatus.INFO, "Snapshot below: " + test.addScreenCapture("Screenshot Path\\screencast1.png")); verificationErrors.add(e.toString()); } extent.endTest(test); }
- The ExtentReport has been created and report entries have been logged
- The ExtentReport must now be flushed to the disk
- This is done in either the @After or @AfterClass annotated method
- Note that the extent.flush is done before the fail can occur which fails the test
- Fail seems to not execute the flush statement if it comes after it.
- Note that this code is similar to the code produced by the Selenium IDE
- When you export the IDE code as Java
- Errors are gathered in verificationErrorStrings and then printed out
- Assets that fail produce Error exceptions which can be caught
- Then all fails are done in the @After method
@After public void tearDown() throws Exception { driver.quit(); extent.flush(); for (String verificationErrorString : verificationErrors) { if (!"".equals(verificationErrorString)) { fail(verificationErrorString); } } }
- Hands on Exercise
- Create a test suite for the userform.jsp, actionPage.jsp and anotherPage.jsp that is on the Tomcat server
- Create an HTML report for the tests of the pages using screenshots and ExtentReport log entries
References
Excel Reports using Apache POI
- Apache POI is an open source project
- It allows the creation files that are in the various Microsoft formats
- Excel, Word, etc.
- It is used with Java programming
Install Apache POI
- download the POI zip file from Apache POI download page
- Unzip the zip file to get the POI folder (i.e. poi-3.13 or similar)
- Open Eclipse
- In the Selenium project that will use POI go into "Configure Build Path"
- Using "Add External Jars" add the poi jars as well as the jars in lib and ooxml-lib
- Make sure that jars that are added do not duplicate any jars already in the External Jars
- Also make sure that older or newer versions of the jars already present are not added
- This completes the install of Apache POI, its classes can now be used in the project
Use Apache POI
- Create an Excel workbook
Workbook wb = new XSSFWorkbook();
- Create a sheet in the workbook
XSSFSheet sheet = (XSSFSheet) wb.createSheet();
- Create rows and columns with data in the sheet
//write a Excel spreadsheet with 3 columns int rowIndex = 1; int colIndex = 1; XSSFRow theRow = sheet.createRow(rowIndex++); XSSFCell theCell = theRow.createCell(colIndex++); theCell.setCellValue("Failurexxx"); theCell = theRow.createCell(colIndex++); theCell.setCellValue("testUserFormFailxxx"); theCell = theRow.createCell(colIndex++); theCell.setCellValue(message); ExcelWorkbook excelWorkbook = new ExcelWorkbook(); excelWorkbook.saveWorkbook(wb, "File Path\\test.xlsx");
References
Other Reporting Methods
- It is possible to create other homegrown reporting methods
References
- http://www.guru99.com/pdf-emails-and-screenshot-of-test-reports-in-selenium.html
- http://www.ontestautomation.com/create-your-own-html-report-from-selenium-tests/
- http://seleniumeasy.com/selenium-tutorials/take-screenshot-with-selenium-webdriver
- http://seleniumeasy.com/
Selenium Sample Zip Files
- The following zip files are used in the hands on projects in this document
- Selenium IDE Testcases Zip File
- Web Project WAR for Tomcat Server with Examples to be Tested
- Eclipse Selenium Project These are only the source files for the Eclipse Project