Selenium RC Architecture

Note:

This topic tries to explain the technical implementation behind Selenium RC. It’s not fundamental for a Selenium user to know this, but could be useful for understanding some of the problems you might find in the future.

To understand in detail how Selenium RC Server works and why it uses proxy injection and heightened privilege modes you must first understand the same origin policy.

The Same Origin Policy

The main restriction that Selenium faces is the Same Origin Policy. This security restriction is applied by every browser in the market and its objective is to ensure that a site’s content will never be accessible by a script from another site. The Same Origin Policy dictates that any code loaded within the browser can only operate within that website’s domain. It cannot perform functions on another website. So for example, if the browser loads JavaScript code when it loads , it cannot run that loaded code against even if that’s another of your sites. If this were possible, a script placed on any website you open would be able to read information on your bank account if you had the account page opened on other tab. This is called XSS (Cross-site Scripting).

To work within this policy, Selenium-Core (and its JavaScript commands that make all the magic happen) must be placed in the same origin as the Application Under Test (same URL).

Historically, Selenium-Core was limited by this problem since it was implemented in JavaScript. Selenium RC is not, however, restricted by the Same Origin Policy. Its use of the Selenium Server as a proxy avoids this problem. It, essentially, tells the browser that the browser is working on a single “spoofed” website that the Server provides.

Note

You can find additional information about this topic on Wikipedia pages about Same Origin Policy and XSS.

Proxy Injection

The first method Selenium used to avoid the The Same Origin Policy was Proxy Injection. In Proxy Injection Mode, the Selenium Server acts as a client-configured HTTP proxy , that sits between the browser and the Application Under Test. It then masks the AUT under a fictional URL (embedding Selenium-Core and the set of tests and delivering them as if they were coming from the same origin).

[1]	The proxy is a third person in the middle that passes the ball between the two parts. It acts as a “web server” that delivers the AUT to the browser. Being a proxy gives Selenium Server the capability of “lying” about the AUT’s real URL.

[2]	The browser is launched with a configuration profile that has set localhost:4444 as the HTTP proxy, this is why any HTTP request that the browser does will pass through Selenium server and the response will pass through it and not from the real server.

Here is an architectural diagram.

As a test suite starts in your favorite language, the following happens:

1.   The client/driver establishes a connection with the selenium-RC server.

2.   Selenium RC server launches a browser (or reuses an old one) with a URL that injects Selenium-Core’s JavaScript into the browser-loaded web page.

3.   The client-driver passes a Selenese command to the server.

4.   The Server interprets the command and then triggers the corresponding JavaScript execution to execute that command within the browser. Selenium-Core instructs the browser to act on that first instruction, typically opening a page of the AUT.

5.   The browser receives the open request and asks for the website’s content from the Selenium RC server (set as the HTTP proxy for the browser to use).

6.   Selenium RC server communicates with the Web server asking for the page and once it receives it, it sends the page to the browser masking the origin to look like the page comes from the same server as Selenium-Core (this allows Selenium-Core to comply with the Same Origin Policy).

7.   The browser receives the web page and renders it in the frame/window reserved for it.

Heightened Privileges Browsers

This workflow in this method is very similar to Proxy Injection but the main difference is that the browsers are launched in a special mode called Heightened Privileges, which allows websites to do things that are not commonly permitted (as doing XSS, or filling file upload inputs and pretty useful stuff for Selenium). By using these browser modes, Selenium Core is able to directly open the AUT and read/interact with its content without having to pass the whole AUT through the Selenium RC server.

Here is the architectural diagram.

As a test suite starts in your favorite language, the following happens:

1.   The client/driver establishes a connection with the selenium-RC server.

2.   Selenium RC server launches a browser (or reuses an old one) with a URL that will load Selenium-Core in the web page.

3.   Selenium-Core gets the first instruction from the client/driver (via another HTTP request made to the Selenium RC Server).

4.   Selenium-Core acts on that first instruction, typically opening a page of the AUT.

5.   The browser receives the open request and asks the Web Server for the page. Once the browser receives the web page, renders it in the frame/window reserved for it.

Handling HTTPS and Security Popups

Many applications switch from using HTTP to HTTPS when they need to send encrypted information such as passwords or credit card information. This is common with many of today’s web applications. Selenium RC supports this.

To ensure the HTTPS site is genuine, the browser will need a security certificate. Otherwise, when the browser accesses the AUT using HTTPS, it will assume that application is not ‘trusted’. When this occurs the browser displays security popups, and these popups cannot be closed using Selenium RC.

When dealing with HTTPS in a Selenium RC test, you must use a run mode that supports this and handles the security certificate for you. You specify the run mode when your test program initializes Selenium.

In Selenium RC 1.0 beta 2 and later use *firefox or *iexplore for the run mode. In earlier versions, including Selenium RC 1.0 beta 1, use *chrome or *iehta, for the run mode. Using these run modes, you will not need to install any special security certificates; Selenium RC will handle it for you.

In version 1.0 the run modes *firefox or *iexplore are recommended. However, there are additional run modes of *iexploreproxy and *firefoxproxy. These are provided for backwards compatibility only, and should not be used unless required by legacy test programs. Their use will present limitations with security certificate handling and with the running of multiple windows if your application opens additional browser windows.

In earlier versions of Selenium RC, *chrome or *iehta were the run modes that supported HTTPS and the handling of security popups. These were considered ‘experimental modes although they became quite stable and many people used them. If you are using Selenium 1.0 you do not need, and should not use, these older run modes.

Security Certificates Explained

Normally, your browser will trust the application you are testing by installing a security certificate which you already own. You can check this in your browser’s options or Internet properties (if you don’t know your AUT’s security certificate ask your system administrator). When Selenium loads your browser it injects code to intercept messages between the browser and the server. The browser now thinks untrusted software is trying to look like your application. It responds by alerting you with popup messages.

To get around this, Selenium RC, (again when using a run mode that support this) will install its own security certificate, temporarily, to your client machine in a place where the browser can access it. This tricks the browser into thinking it’s accessing a site different from your AUT and effectively suppresses the popups.

Another method used with earlier versions of Selenium was to install the Cybervillians security certificate provided with your Selenium installation. Most users should no longer need to do this however; if you are running Selenium RC in proxy injection mode, you may need to explicitly install this security certificate.

Supporting Additional Browsers and Browser Configurations

The Selenium API supports running against multiple browsers in addition to Internet Explorer and Mozilla Firefox. See the SeleniumHQ.org website for supported browsers. In addition, when a browser is not directly supported, you may still run your Selenium tests against a browser of your choosing by using the “*custom” run-mode (i.e. in place of *firefox or *iexplore) when your test application starts the browser. With this, you pass in the path to the browsers executable within the API call. This can also be done from the Server in interactive mode.

cmd=getNewBrowserSession&1=*custom c:\Program Files\Mozilla Firefox\MyBrowser.exe&2=http://www.google.com

Running Tests with Different Browser Configurations

Normally Selenium RC automatically configures the browser, but if you launch the browser using the “*custom” run mode, you can force Selenium RC to launch the browser as-is, without using an automatic configuration.

For example, you can launch Firefox with a custom configuration like this:

cmd=getNewBrowserSession&1=*custom c:\Program Files\Mozilla Firefox\firefox.exe&2=http://www.google.com

Note that when launching the browser this way, you must manually configure the browser to use the Selenium Server as a proxy. Normally this just means opening your browser preferences and specifying “localhost:4444” as an HTTP proxy, but instructions for this can differ radically from browser to browser. Consult your browser’s documentation for details.

Be aware that Mozilla browsers can vary in how they start and stop. One may need to set the MOZ_NO_REMOTE environment variable to make Mozilla browsers behave a little more predictably. Unix users should avoid launching the browser using a shell script; it’s generally better to use the binary executable (e.g. firefox-bin) directly.

Troubleshooting Common Problems

When getting started with Selenium RC there’s a few potential problems that are commonly encountered. We present them along with their solutions here.

Unable to Connect to Server

When your test program cannot connect to the Selenium Server, Selenium throws an exception in your test program. It should display this message or a similar one:

"Unable to connect to remote server (Inner Exception Message:

    No connection could be made because the target machine actively

    refused it )"

    (using .NET and XP Service Pack 2)

If you see a message like this, be sure you started the Selenium Server. If so, then there is a problem with the connectivity between the Selenium Client Library and the Selenium Server.

When starting with Selenium RC, most people begin by running thier test program (with a Selenium Client Library) and the Selenium Server on the same machine. To do this use “localhost” as your connection parameter. We recommend beginning this way since it reduces the influence of potential networking problems which you’re getting started. Assuming your operating system has typical networking and TCP/IP settings you should have little difficulty. In truth, many people choose to run the tests this way.

If, however, you do want to run Selenium Server on a remote machine, the connectivity should be fine assuming you have valid TCP/IP connectivity between the two machines.

If you have difficulty connecting, you can use common networking tools like ping, telnet,ifconfig(Unix)/ipconfig (Windows), etc to ensure you have a valid network connection. If unfamilar with these, your system administrator can assist you.

Unable to Load the Browser

Ok, not a friendly error message, sorry, but if the Selenium Server cannot load the browser you will likley see this error.

(500) Internal Server Error

This could be caused by

·         Firefox (prior to Selenium 1.0) cannot start because the browser is already open and you did not specify a separate profile. See the section on Firefox profiles under Server Options.

·         The run mode you’re using doesn’t match any browser on your machine. Check the parameters you passed to Selenium when you program opens the browser.

·         You specified the path to the browser explicitly (using “*custom”–see above) but the path is incorrect. Check to be sure the path is correct. Also check the user group to be sure there are no known issues with your browser and the “*custom” parameters.

Selenium Cannot Find the AUT

If your test program starts the browser successfully, but the browser doesn’t display the website you’re testing, the most likely cause is your test program is not using the correct URL.

This can easily happen. When you use Selenium-IDE to export your script, it inserts a dummy URL. You must manually change the URL to the correct one for your application to be tested.

Firefox Refused Shutdown While Preparing a Profile

This most often occurs when your run your Selenium RC test program against Firefox, but you already have a Firefox browser session running and, you didn’t specify a separate profile when you started the Selenium Server. The error from the test program looks like this:

Error:  java.lang.RuntimeException: Firefox refused shutdown while

preparing a profile

Here’s the complete error message from the server:

16:20:03.919 INFO - Preparing Firefox profile...

16:20:27.822 WARN - GET /selenium-server/driver/?cmd=getNewBrowserSession&1=*fir

efox&2=http%3a%2f%2fsage-webapp1.qa.idc.com HTTP/1.1

java.lang.RuntimeException: Firefox refused shutdown while preparing a profile

        at org.openqa.selenium.server.browserlaunchers.FirefoxCustomProfileLaunc

her.waitForFullProfileToBeCreated(FirefoxCustomProfileLauncher.java:277)

...

Caused by: org.openqa.selenium.server.browserlaunchers.FirefoxCustomProfileLaunc

her$FileLockRemainedException: Lock file still present! C:\DOCUME~1\jsvec\LOCALS

~1\Temp\customProfileDir203138\parent.lock

To resolve this, see the section on Specifying a Separate Firefox Profile

Versioning Problems

Make sure your version of Selenium supports the version of your browser. For example, Selenium RC 0.92 does not support Firefox 3. At times you may be lucky (I was). But don’t forget to check which browser versions are supported by the version of Selenium you are using. When in doubt, use the latest release version of Selenium with the most widely used version of your browser.

Error message: “(Unsupported major.minor version 49.0)” while starting server

This error says you’re not using a correct version of Java. The Selenium Server requires Java 1.5 or higher.

To check double-check your java version, run this from the command line.

java -version

You should see a message showing the Java version.

java version "1.5.0_07"

Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_07-b03)

Java HotSpot(TM) Client VM (build 1.5.0_07-b03, mixed mode)

If you see a lower version number, you may need to update the JRE, or you may simply need to add it to your PATH environment variable.

404 error when running the getNewBrowserSession command

If you’re getting a 404 error while attempting to open a page on “http://www.google.com/selenium-server/”, then it must be because the Selenium Server was not correctly configured as a proxy. The “selenium-server” directory doesn’t exist on google.com; it only appears to exist when the proxy is properly configured. Proxy Configuration highly depends on how the browser is launched with *firefox, *iexplore, *opera, or *custom.

·         *iexplore: If the browser is launched using *iexplore, you could be having a problem with Internet Explorer’s proxy settings. Selenium Server attempts To configure the global proxy settings in the Internet Options Control Panel. You must make sure that those are correctly configured when Selenium Server launches the browser. Try looking at your Internet Options control panel. Click on the “Connections” tab and click on “LAN Settings”.

o    If you need to use a proxy to access the application you want to test, you’ll need to start Selenium Server with “-Dhttp.proxyHost”; see the Proxy Configuration for more details.

o    You may also try configuring your proxy manually and then launching the browser with *custom, or with *iehta browser launcher.

·         *custom: When using *custom you must configure the proxy correctly(manually),

otherwise you’ll get a 404 error. Double-check that you’ve configured your proxy settings correctly. To check whether you’ve configured the proxy correctly is to attempt to intentionally configure the browser incorrectly. Try configuring the browser to use the wrong proxy server hostname, or the wrong port. If you had successfully configured the browser’s proxy settings incorrectly, then the browser will be unable to connect to the Internet, which is one way to make sure that one is adjusting the relevant settings.

·         For other browsers (*firefox, *opera) we automatically hard-code the proxy for you, and so ther are no known issues with this functionality. If you’re encountering 404 errors and have followed this user guide carefully post your results to user group for some help from the user community.

Permission Denied Error

The most common reason for this error is that your session is attempting to violate the same-origin policy by crossing domain boundaries (e.g., accesses a page fromhttp://domain1 and then accesses a page from http://domain2) or switching protocols (moving from http://domainX to https://domainX).

This error can also occur when JavaScript attempts to find UI objects which are not yet available (before the page has completely loaded), or are no longer available (after the page has started to be unloaded). This is most typically encountered with AJAX pages which are working with sections of a page or subframes that load and/or reload independently of the larger page.

This error can be intermittent. Often it is impossible to reproduce the problem with a debugger because the trouble stems from race conditions which are not reproducible when the debugger’s overhead is added to the system. Permission issues are covered in some detail in the tutorial. Read the section about the The Same Origin Policy, Proxy Injectioncarefully.

Handling Browser Popup Windows

There are several kinds of “Popups” that you can get during a Selenium test. You may not be able to close these popups by running selenium commands if they are initiated by the browser and not your AUT. You may need to know how to manage these. Each type of popup needs to be addressed differently.

·         HTTP basic authentication dialogs: These dialogs prompt for a username/password to login to the site. To login to a site that requires HTTP basic authentication, use a username and password in the URL, as described in RFC 1738, like this: open(“http://myusername:myuserpassword@myexample.com/blah/blah/blah”).

·         SSL certificate warnings: Selenium RC automatically attempts to spoof SSL certificates when it is enabled as a proxy; see more on this in the section on HTTPS. If your browser is configured correctly, you should never see SSL certificate warnings, but you may need to configure your browser to trust our dangerous “CyberVillains” SSL certificate authority. Again, refer to the HTTPS section for how to do this.

·         modal JavaScript alert/confirmation/prompt dialogs: Selenium tries to conceal those dialogs from you (by replacing window.alert, window.confirm and window.prompt) so they won’t stop the execution of your page. If you’re seeing an alert pop-up, it’s probably because it fired during the page load process, which is usually too early for us to protect the page. Selenese contains commands for asserting or verifying alert and confirmation popups. See the sections on these topics in Chapter 4.

On Linux, why isn’t my Firefox browser session closing?

On Unix/Linux you must invoke “firefox-bin” directly, so make sure that executable is on the path. If executing Firefox through a shell script, when it comes time to kill the browser Selenium RC will kill the shell script, leaving the browser running. You can specify the path to firefox-bin directly, like this.

cmd=getNewBrowserSession&1=*firefox /usr/local/firefox/firefox-bin&2=http://www.google.com

Firefox *chrome doesn’t work with custom profile

Check Firefox profile folder -> prefs.js -> user_pref(“browser.startup.page”, 0); Comment this line like this: “//user_pref(“browser.startup.page”, 0);” and try again.

Is it ok to load a custom pop-up as the parent page is loading (i.e., before the parent page’s javascript window.onload() function runs)?

No. Selenium relies on interceptors to determine window names as they are being loaded. These interceptors work best in catching new windows if the windows are loaded AFTER the onload() function. Selenium may not recognize windows loaded before the onload function.

Problems With Verify Commands

If you export your tests from Selenium-IDE, you may find yourself getting empty verify strings from your tests (depending on the programming language used).

Note: This section is not yet developed.

Safari and MultiWindow Mode

Note: This section is not yet developed.

Firefox on Linux

On Unix/Linux, versions of Selenium before 1.0 needed to invoke “firefox-bin” directly, so if you are using a previous version, make sure that the real executable is on the path.

On most Linux distributions, the real firefox-bin is located on:

/usr/lib/firefox-x.x.x/

Where the x.x.x is the version number you currently have. So, to add that path to the user’s path. you will have to add the following to your .bashrc file:

export PATH="$PATH:/usr/lib/firefox-x.x.x/"

If necessary, you can specify the path to firefox-bin directly in your test, like this:

"*firefox /usr/lib/firefox-x.x.x/firefox-bin"

IE and Style Attributes

If you are running your tests on Internet Explorer and you cannot locate elements using theirstyle attribute. For example:

//td[@style="background-color:yellow"]

This would work perfectly in Firefox, Opera or Safari but not with IE. IE interprets the keys in@style as uppercase. So, even if the source code is in lowercase, you should use:

//td[@style="BACKGROUND-COLOR:yellow"]

This is a problem if your test is intended to work on multiple browsers, but you can easily code your test to detect the situation and try the alternative locator that only works in IE.

Error encountered - “Cannot convert object to primitive value” with shut down of *googlechrome browser

To avoid this error you have to start browser with an option that disables same origin policy checks:

selenium.start("commandLineFlags=--disable-web-security");