Tutorial: Writing XPath selectors for Selenium tests

Selenium is the most commonly used web testing framework. It allows you to write tests in Java that can perform standard “web” actions such as filling in forms, clicking buttons etc. Usually, a test will take the following form:

  • Open a web page.
  • Locate an html element.
  • Interact with it in some way. i.e. click it, type into it, select an option
  • Locate another html element.
  • Interact with it.
  • etc

Locating elements is obviously key to writing a test, and Selenium offers several ways you can do this:

  • By name.
  • By id.
  • Using XPath.

Using the name or id is okay for simple tests, but for more complex situations, you need to know how to write XPath queries. Even though you will typically start off by using the Selenium IDE Firefox plugin to record a test, which will automatically generate selectors for you, in practice you will usually find that you need to change some of them. This will often happen when you are dealing with a web page that is dynamically generated. For example, suppose you are searching a website for a shirt to buy. If there are various sizes and colours available for the shirt, you could get back multiple “Add to bag” buttons. When you click the button for the white shirt in collar size regular, the Selenium IDE won’t know that this is what you are doing. It will do its best to write an expression that picks out the button, but it will probably be incorrect. For example, if the button you click is the second one on the page, the Selenium IDE could use that to pick out the button. However, if you rerun the test and some of the sizes or colours are out of stock, the button you want might not be the second one. What you really want to do is write a test that knows to “click the button for the white shirt in regular size”.  XPath allows you to do this, but XPath queries can be quite complex to begin with, so the purpose of this tutorial is to guide you through writing a query, step by step.

Example

Step 1 – Navigate to the problematic web page

The first step is to manually navigate to the web page which your test is having a problem with. I’ve recently been working on a test that purchases a product by selecting a customer, then a product, then entering address details, then finally credit card details. I used the Selenium IDE to record the test and then saved it as Java code for playback. However, when running, it failed at the stage of entering the credit card details, specifically the card number. Have a look at the following screenshot and see if you can tell why:

Did you spot it? There are two “card number” fields on the page. The recorded Selenium test was picking up the one for the gift card. You can see that we need the one in the credit card section, so how do we go about changing the selector to pick this out?

Step 2 – Write and test your XPath expression

There are a number of ways to write and test an XPath expression:

  1. Using the Selenium IDE itself – it has a “test” option.
  2. Using Firebug and a separate XPath tester.
  3. Using Firebug and the FirePath tester, which is a separate add-on but integrates with Firebug.

All these methods work, but the third method is probably the most powerful, so it is the one I will explain here. The first thing to do is to identify the html element that you are interested in, and see how it fits into the structure of the page. If I open up Firebug, I’ll see something like this:

Firebug has the “selector” tool,  which allows you to select an html element on the page and see it in the HTML view. It is the arrow icon in the top left of the FireBug panel:

Once you have added the FirePath extension, you’ll get a separate FirePath view and you’ll also be able to use the selector tool to bring up an element in that view. So, if I switch to the FirePath view and then use the selector tool to pick out the card number input box, I’ll see the following:

An XPath expression that picks out the element will automatically be generated and displayed at the top of the panel. However, this is the expression that needs to be rewritten. I find the easiest way to write an XPath expression is to gradually build it up. What is the simplest thing to start with? Well, I’m looking for an input, so a good start might be to write something that will select all inputs:

//input

Sure enough, FirePath highlights all of the inputs on the page:

I know that I need the card number input within the “Credit Card” area of the page, but how do I know what XPath expression to write to pick this out? Well, you can use Firebug to inspect the DOM to understand the structure:

Looking at this structure, you can see that conceptually I’m looking for the input with the id of cardNumber that is within the div that has an h3 heading of “Credit Card”. To build up an expression for this, you can start by just trying to pick out that div. For example:

//div[h3/text()='Credit Card']

This correctly picks out the div, but how do you navigate down to the input? Well, in XPath you use axes to navigate up and down the DOM. The input is within the div, so we’ll need to use an axis like “child” or “descendant”. You could test that with:

//div[h3/text()='Credit Card']/descendant::*

Sure enough, that picks out all of the nodes beneath the div. I know that I want an input with an id of “cardNumber”, so I change that to:

//div[h3/text()='Credit Card']/descendant::input[@id='cardNumber']

This works, but actually it is overspecified. What if the heading was changed to an h2 or a legend? It is more flexible for the expression just to specify that the div must have some kind of child node with the text “Credit Card”, rather than specifically an h3 heading. Hence the final XPath expression is:

//div[*/text()='Credit Card']/descendant::input[@id='cardNumber']

Step 3 – put the XPath expression back into your test and rerun

Once you have worked out the XPath expression, you can put it back into your test and rerun it. I’m using the Selenium 2 API for my test, so the code becomes:

webDriver.findElement(
  By.xpath(
    "//div[*/text()='Credit Card']/descendant::input[@id='cardNumber']"
  )
).sendKeys("1234123412341234");

If you’re using Selenium 1, your syntax will be more like:

selenium.type(
  "//div[*/text()='Credit Card']/descendant::input[@id='cardNumber']",
  "1234123412341234"
);

Hopefully you’ll find that by using the above technique with Firebug and FirePath, you can build up and test even complex XPath expressions easily. If you want to read up on XPath, a good place is the w3schools tutorial:

http://www.w3schools.com/xpath/default.asp

Firebug can be downloaded from:

http://getfirebug.com/

and FirePath from:

https://addons.mozilla.org/en-us/firefox/addon/firepath/

If you want to automate your Selenium tests, you might also be interested in my blog post Automating Selenium testing with TestNG, Ant and CruiseControl

This entry was posted in Selenium, Testing and tagged , . Bookmark the permalink.

53 Responses to Tutorial: Writing XPath selectors for Selenium tests

  1. George Young says:

    Nice! Thanks

  2. Ramdas Singh says:

    Great!!!!

  3. li zhaokun says:

    Cool guide!

  4. sue says:

    Very helpful…Thanks!

  5. tester says:

    Hi, very nice and helpful!
    But I have a question: You wrote Selenium can only select by id, name or xpath. What about css selectors, tagname, linkname or class?
    Another question: why is using IDs only for simple tests ok? you suggest to use xpath in complex tests. why? IDs are unique so I can’t see any problems with that.

    • hedleyproctor says:

      Sorry, my list wasn’t exhaustive – you can use css selectors and links. With regards to ids, if all of the elements on the page have sensible ids, that is usually the best option. However, sometimes this isn’t the case. Firstly, some applications won’t have ids for all elements. Secondly, if elements of a page are dynamically generated, then the ids will be too, and you can end up writing a test that relies on understanding hidden application logic for generating ids, rather than relying on the information visible to the user. Consider the example I give of searching for a shirt in a particular colour and getting back a list of all of the colour variants of that shirt. If the application generates ids for the elements representing these colour variants, what format will they take? If they are labelled “shirt-blue”, “shirt-green”, “shirt-yellow” then you might be okay – your test knows it is looking for the yellow shirt and it can pick it out using the ids. However, if they were labelled “shirt-1″, “shirt-2″, “shirt-3″ then you can’t use the id to locate the element. However, you know that that part of the web page must be visibly labelled “yellow” for the customer to see. By writing an XPath expression you can make use of this visible label to locate the correct element.

  6. phani says:

    Excellent!! fire path makes my life easy now!!
    Thanks so much mate, for such nice article.

    Regards,
    Phani

  7. Viru says:

    Hi, the direction to use Firepath was excellent and helped me learn and practice xpath directly on a web page.

    Thanks
    Viru Kulkarni.

  8. khoi le says:

    what a great example

  9. Tripp says:

    Firebug/FirePlay is the bomb.

  10. sravan says:

    thanks, a very good example. :)

  11. santhosh says:

    Perfect explanation. It resolved my doubt. Thanks!!!

  12. deepankur tyagi says:

    Hi Hedley,
    I have a problem with xpath. I have a local variable “end” which is essentially a date, I am creating an element with value( “testauto” + end).
    Now while trying to assert this value using xpath variale I am able to locate it with this expression
    assertTrue(driver.findElement(By.xpath(“//div[\'testauto13/09/2012\']“)).isDisplayed());
    Now I want to assert it using variable “end” I am trying with something like this but it’s not recognising my local variable “end”
    assertTrue(driver.findElement(By.xpath(“//div[\'testauto\'+ end]“)).isDisplayed());
    Thanks
    Deep

  13. deepankur tyagi says:

    found the answer
    assertTrue(driver.findElement(By.xpath(“//div[\'testauto\'\"+ end\"]“)).isDisplayed());

  14. vivek sinha says:

    nice post..

  15. deepankur tyagi says:

    Hi Hedley,
    I need your help with following Xpath example
    I want to select the elemet with “testautocomms27 Sep 2012″ this is the only way my automated tests will recognise my elemnet dynamically. So In my xpath expression I want “testautocomms27 Sep 2012″

    testautocomms27 Sep 2012

    The firepath generates this Xpath for me .//*[@id=\'storeCommsTable_cell_14_1\']
    But it’s not of any use for two reasons, one the id is dynamic and second it hasn’t got “testautocomms27 Sep 2012″

    I tried to go through the W3 schools xpath tutorial but could not find a way to do this.
    Will aprreciate if you can help.
    Thanks

  16. Dwarika Dhish Mishra says:

    This post has helped me in understanding Xpath selector and i have enriched it more in my new post on my blog. Hope
    http://abodeqa.wordpress.com/2012/09/27/xpath-in-automation-mainly-in-selenium/
    this will help you all equally as this blog has helped me in coming out with this post
    Thanks hedleyproctor for paving my way in learning Selenium.

  17. deepankur tyagi says:

    Trying again
    testautocomms27 Sep 2012

    Trying without td
    id=”storeCommsTable_cell_14_1″ class=”comm14 announcementgreen” onclick=”loadRecord_storeCommsTable(‘storeCommsTable’, ’219′, ‘comm_title’, ‘testautocomms27 Sep 2012′)” style=” “> testautocomms27 Sep 2012

    • deepankur tyagi says:

      So just assume it has td tag…as it’s not letting me paste with td tag.
      Please reply.
      Thanks

      • hedleyproctor says:

        Hi Deep. If you want to get the text content of a node you can use the text() xpath function. Off the top of my head I would guess something like //td[text()=\"testautocomms27 Sep 2012\"] should pick out that td.

  18. Umamaheshwar Thota says:

    Hi Hedley,
    I have xpath written in Selenium RC & now I want to write the same xpath in selenium Webdriver. below is the xpath:
    “//table[@id=\'lgnLogin\']/tbody/tr/td/table/tbody/tr[4]/td”
    by using this xpath, i am capturing the error message displayed on my application like “Please check your Password”.
    Now how can I write it in Webdriver. I have different ways but, not worked out.
    “String msg= driver.findElement(By.xpath(“//*[@td=\'error2\']“)).toString();” – This is what i did in Webdriver.
    Please help me out on this…

    • hedleyproctor says:

      I’m a little confused – if the XPath expression works in Selenium RC the same expression should work in the WebDriver. Are you really trying to write the same expression or is the second expression testing a different page of your application?

  19. Dwarika Dhish Mishra says:

    Hi
    I need one help with you and this is related to writing xpath for case insensitive text()
    I am asking this for Google Search button on Google home page
    xpath for Google Search
    //span[text()=\'Google Search\']
    But I want to use google search as text() to find the same element that is being searched by above xpath
    so please let me know the right xpath if I am using
    google search
    Google search
    google Search
    in place of Google Search
    Since I have tried translate function but that haven’t work for me.
    So please help me

    • hedleyproctor says:

      You should be able to use the XPath upper-case() or lower-case() functions. i.e. //span[lower-case(text())=\'google search\']

  20. hina says:

    thanks for explaining with so much detail and screenshots. its really helped me a lot

  21. Hamid Assous says:

    Hi,
    Would you recommend a Selenium IDE with xPath instructor-led training class in the Mid-Atlantic area?
    Thanks!

    • hedleyproctor says:

      Sorry, I’m not really familiar with Selenium training courses. The documentation is pretty good so I’d recommend working through that.

  22. Dhruba says:

    Hi

    Don’t know how do I thank you. Great elaboration and you are a great teacher too.
    Thanks a lot again.

    Regards
    Dhruba

  23. Manoj Singh says:

    Great explanation.
    But it would have been more better if you would have used a dynamic id for the element which changes with every page load . In that case how to manage the id attribute.

  24. xpath says:

    Hi!
    A really beautiful tutorial!!thanks a lot!!as a beginner it was really very helpful..

  25. kevin says:

    I am curious, why can’t you use the xpath that FirePath generates for you? I was using just Firebug and so far when I select an element, which is then highlighted below, I then rightclick below and use Copy XPath. From that, I’ve so far not had any problem finding exact elements when testing (or automating). It seems FirePath generates an even more exact xpath than the Firebug did. Why do you need to modify what it already provides for you?

    • hedleyproctor says:

      Sometimes this will work, but the problem is that the XPath that FirePath generates doesn’t necessarily match what you are conceptually trying to achieve, which can make the XPath brittle if the page changes. Imagine a payment page that has a credit card payment area at the top, and a gift card payment area at the bottom that has similar html elements. If you try to pick out the credit card number field with FirePath, it might decide that the best way to pick out the credit card field is to specify that it is the first of the two similar fields. What happens if the application developers change the design of the page to put the credit card at the bottom and the gift card at the top? The XPath will break. By contrast, if you have an xpath expression that says something like “pick out the input element that is inside a div which also contains the text ‘credit card”, even if that section of html is moved around on the page, the xpath will still match. Similar reasoning applies if you are testing an application that can be configured in different ways for different customers.

      A second consideration is trying to make your xpath expressions as readable as possible for other developers or testers who might need to maintain the tests. An expression like /h3/div/input doesn’t really tell someone what it is trying to pick out, whereas something that says /div[@title=\'Credit Card\'] makes the intent of the xpath much clearer.

      Of course, I can’t emphasis enough that really all applications should use distinct ids on their html elements wherever possible, so that no xpath is necessary, and you should encourage the application developers to add them if they are missing!

  26. Martin Powell says:

    Hi there and thanks for a very informative article, that I thought was going to be my saviour, but thus far I’m still stuck at the same point

    I’m using selenium to parse through a dynamically loaded table of links
    The links all start 3-??????????? so I initialy tried :-

    List links = driver.findElements(By.partialLinkText(“3-”));

    I thought this was working, but on a table bigger that 25 rows, it seemed to cap the liks.size() to 25

    So I used Firepath to help me use xpath instead and the elements a found as follows

    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-0\']/div[1]/table/tbody/tr/td[2]/a
    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-0\']/div[2]/table/tbody/tr/td[2]/a
    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-0\']/div[3]/table/tbody/tr/td[2]/a

    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-0\']/div[24]/table/tbody/tr/td[2]/a
    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-0\']/div[25]/table/tbody/tr/td[2]/a
    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-1\']/div[1]/table/tbody/tr/td[2]/a
    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-1\']/div[2]/table/tbody/tr/td[2]/a
    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-1\']/div[3]/table/tbody/tr/td[2]/a

    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-1\']/div[7]/table/tbody/tr/td[2]/a
    .//*[@id=\'34c0ca44-f1d3-4d58-8972-40d67bee0dae-page-1\']/div[8]/table/tbody/tr/td[2]/a

    Interestingly, the div section seems to cap at 25, then the @id= section increments it’s page by 1
    Unfortunatley , the ID seems to be dynamic as when I run the page again, I have a completely different @id value:-

    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-0\']/div[1]/table/tbody/tr/td[2]/a
    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-0\']/div[1]/table/tbody/tr/td[2]/a
    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-0\']/div[2]/table/tbody/tr/td[2]/a
    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-0\']/div[3]/table/tbody/tr/td[2]/a

    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-0\']/div[24]/table/tbody/tr/td[2]/a
    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-0\']/div[25]/table/tbody/tr/td[2]/a
    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-1\']/div[1]/table/tbody/tr/td[2]/a
    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-1\']/div[2]/table/tbody/tr/td[2]/a
    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-1\']/div[3]/table/tbody/tr/td[2]/a

    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-1\']/div[7]/table/tbody/tr/td[2]/a
    .//*[@id=\'2df518f0-b01f-412b-8ee4-15fe8f44f108-page-1\']/div[8]/table/tbody/tr/td[2]/a

    Everything else is static, but I can’t get to the items that are on “page-1″ of the result set.

    List srlink = driver.findElements(By.xpath(“//*/div/table/tbody/tr/td[2]/a”));
    has a size() of 25, but there are a further 7 rows that seem to be on a separate ID.

    The HTML of the page is like this:-

    I’ve found if I set a breakpoint in the IDE at the point I make this call to list elements, and scroll down to the bottom of the table, and then click the next step, it does return 33 as a size, but not sure how I can scroll down a table at runtime in selenium.

    Any help on this greatly appreciated.

    • hedleyproctor says:

      It sounds like the table is being dynamically loading using javascript. If this is the case, then you only really have two options. If testing this dynamic loading is not crucial to you, and all you really want to do is access the links, then I’d see if you can just reconfigure the number of links it gets in a single chunk to be large enough for the list you have. If you really want to test the applications ability to dynamically load the list, then I’m afraid you would have to scroll the list in your test. It’s difficult to comment much about this without knowing how you are doing the scolling. If you are clicking a button it should be fairly straightforward in selenium. If not, how are you doing the scrolling?

  27. Jéssica says:

    Congratulations , turotial great .

  28. sreenath says:

    great!!!thank you so much

  29. Good Line says:

    Thank you! Great tutorial.

  30. Juan Garcia says:

    Just Curious:
    Why not just use the “id” attribute… is it because “id” values are not enforced to be unique?

    i.e.
    webDriver.findElement.By.id(“//input[@id=\'\']“);

    • hedleyproctor says:

      Yes, you should use the id wherever you can. However, if you are dealing with dynamically generated page elements, you need to be happy that you understand how the ids will be generated. As long as you can talk to the developers of the app and make sure you understand how the ids are generated, you should be able to specify them correctly.

  31. Georgey says:

    I think this is the second time I’ve referred to this tutorial to brush up on xPath. Good job Hedley. Hope you and yours are all blooming.

    George

  32. Phil says:

    Hi Hedley this is a great guide but I am new to selenium and I am having problems implementing this on a page I am testing.

    I am trying to select the element engine-code-nid=”CHPB” as shown below
    Engine code

    with the following xpath
    element = driver.findElement(By.xpath(“//div[h3/text()=\'Engine code\']/descendant::engine-code-nid[@id=\'CHPB\']“));
    but get
    Unable to locate element: {“method”:”xpath”,”selector”:”//div[h3/text()=\'Engine code\']/descendant::engine-code-nid[@id=\'CHPB\']“}

    can you help and see what I am doing wrong.

    Many Thanks

    Phil

  33. karunanidhi says:

    hi

    Really good explanation of Xpath manual writing, it’s give up to me a more confident to write the xpath.

    Thanks

  34. Ritika says:

    Very Good Explaination..

  35. David J McClelland says:

    Thanks – very good instructions to get someone going with xpath in Selenium. I could paste xpaths right from the selector in Firepath into my findElement clause in Selenium java – nice!

  36. Andre says:

    Thanks a lot man. the “descendant::” bit was new to me and it helped me a lot. Cheers

  37. Ivo Nikolov says:

    Hi, Hartley

    very useful and helpful explanation.Thanks a lot.

  38. ebix says:

    You could use the $x(“”) in chrome console (F12 > Console tab).
    Instead of using descendant, why not use:
    //div[h3/text()=\'Credit Card\']//input[@id=\'cardNumber\']

  39. ebix says:

    Additionally helpful stuff:
    -following-sibling
    -preceding-sibling
    -count

    These are specially helpful when checking tables.
    example:
    ITEM-NAME | COL1 | COL2
    ITEM 1 | HELLO | THERE
    ITEM 2 | HI | YOU
    //td[text()=\'ITEM1\']/following-sibling::td[count(//td[text()=\'COL1\']/preceding-sibling::td)]

  40. vijay says:

    Knowledge about the xpath is clearly mentioned and shared in this article. Really more informative

Leave a Reply

Your email address will not be published. Required fields are marked *

501,820 Spambots Blocked by Simple Comments

HTML tags are not allowed.