Testing & Automation Banner Image
September 30, 2016

Testing & Automation

Testing plays an important role in any industry. Any product or material which is developed or made needs to go through the various testing phases to achieve a proper quality product and expected outcomes. A poorly tested product does not fulfill the customer satisfaction and it may lead to the losses in many ways.

Testing = Checking + Investigating + Exploring.

What Is Software Testing?

As we target to IT industry, Testing plays a key important role. Any software developed needs to be tested properly, which can windows, web, android, iOS or any other. In software industry testing can be simply defined as precise quality of the product.

It can also be defined as:

  • Process of executing a program or application with the intent of finding the software bugs.
  • Process of validating and verifying that a software program or application or product.

Why Testing Is Required?

We all make mistakes. Mistakes are done unintentionally. Mistakes can be unimportant or perhaps sometimes it can be dangerous and risky in some situations. We need to check, verify everything and anything we produce as Humans makes mistakes all the time. So to tackle the mistakes properly and overcome it we need testing. Any product quality can be measure with its testing efforts.

In Software industry, testing helps to improve the quality, reliability, and performance of the system or product produced. Importance of testing is the major part of Software Development Life Cycle and it is better to introduce testing in the early stage of SDLC phases so it helps to identify the defects in the early stage and try to avoid the bug findings & get resolve in the last critical stage.

Few Manual & Automation Testing Types:

  • Black Box Testing
  • White Box Testing
  • Unit Testing
  • Incremental Integration Testing
  • Integration Testing
  • Functional Testing
  • System Testing
  • Sanity Testing
  • Regression Testing
  • Acceptance Testing
  • Load Testing

Manual Testing:

Manual Testing is a process to find out the defects. In this process the tester plays the role of end user and performs all the operations, verify all the features of the application,behavior of the application. It is preliminary step carried out with software development. As 100% Automation is not possible at all so manual testing is a need. In testing of software application manual testing makes plays a vital role. Here tester needs to creative and patient. Here main Goal is to make sure that application under testing is defect free and it works as per the requirement specification document.

PROS:

  • Cost is lower.
  • Find real user issues.
  • Testing is flexible.

CONS:

  • Certain tasks are difficult to do manually.
  • Can’t reuse manual tests

Automation Testing:

Automation testing is the process, in which paid or free automated tools run the tests which are predefined comparing expected and actual outcomes. If the expectations and actual outcomes don’t match then it is refer to as bug or issue. At this moment concern person needs to look into the code and debug what is wrong. Automation testing is good to use when the project is large, there are many system users, or when filling out forms.

PROS:

  • Run tests quickly and effectively.
  • Can be Cost Effective.
  • Results can be viewed by all.

CONS:

  • Expensive.
  • Time consuming.
  • Limitations of tools.

Why Automation Testing Is Necessary?

Test automation can improve the development process of software application in many cases. Automation of tests is initially associated with the increased effort but the related benefits will quickly pay off. Automated tests run fast and frequently which is very cost effective for software applications with a long maintenance life.

When testing in an agile environment, the ability to quickly react to ever-changing software systems and requirements is necessary. New test cases are generated continuously and can be added to existing automation in parallel with the development of the software itself. In both manual and automated testing environments test cases need to be modified for extended periods of time as the software project progresses. It is important to be aware that complete coverage of all tests using test automation is unrealistic. When deciding what tests to automate first, their value vs. the effort to create them needs to be considered. Test cases with high value and low effort should be automated first. Subsequently test cases with frequent use, changes, and past errors; as well as test cases with low to moderate effort in setting up the test environment and developing the automation project are best suited for automation.

The main goal in software development processes is a timely release. Automated tests run fast and frequently, due to reused modules within different tests. Automated regression tests which ensure the continuous system stability and functionality after changes to the software were made lead to shorter development cycles combined with better quality software and thus the benefits of automated testing quickly outgain the initial costs. Manual testing can be mundane, error-prone and therefore become exasperating. Test automation alleviates testers’ frustrations and allows the test execution without user interaction while guaranteeing repeatability and accuracy. Instead testers can now concentrate on more difficult test scenarios.

Where Does Test Automation Fit In Software Life Cycle?

Automation Testing Vs Manual Testing:

  • Automated testing is cheaper (Using open sources tool).
  • Automated testing is faster to execute.
  • Automated testing is more reliable.
  • Automated testing reduces the human and technical risks.
  • Automated testing is more powerful and versatile.

Few Open Source Automation Tools:

  • Selenium
  • Watir
  • Windmill
  • SoapUI
  • Sahi
  • TestNG
  • Appium

Few Commercial Automation Tools:

  • Ranorex
  • Tellurium
  • Squish
  • TestComplete
  • TestStudio
  • QTP
  • eggPlant

Selenium Introduction:

Selenium is a portable software testing framework for web applications. Selenium provides a record / playback tool for authoring tests without learning a test scripting language (Selenium IDE). It also provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages, including Java, C#, Groovy, Perl, PHP, Python and Ruby. The tests can then be run against most modern web browsers. Selenium deploys on Windows, Linux, and Macintosh platforms. It is open-source software, released under the Apache 2.0 license, and can be downloaded and used without charge.

History of Selenium:

Selenium was originally developed by Jason Huggins in 2004 as an internal tool at ThoughtWorks. Huggins was later joined by other programmers and testers at ThoughtWorks,before Paul Hammant joined the team and steered the development of the second mode of operation that would later become “Selenium Remote Control” (RC). The tool was open sourced that year. In 2005 Dan Fabulich and Nelson Sproul (with help from Pat Lightbody) made an offer to accept a series of patches that would transform Selenium-RC into what it became best known for. In the same meeting, the steering of Selenium as a project would continue as a committee, with Huggins and Hammant being the ThoughtWorks representatives.
In 2007, Huggins joined Google. Together with others like Jennifer Bevan, he continued with the development and stabilization of Selenium RC. At the same time, Simon Stewart at ThoughtWorks developed a superior browser automation tool called WebDriver. In 2009, after a meeting between the developers at the Google Test Automation Conference, it was decided to merge the two projects, and call the new project Selenium WebDriver, or Selenium 2.0.[1]
In 2008, Philippe Hanrigou (then at ThoughtWorks) made “Selenium Grid”, which provides a hub allowing the running of multiple Selenium tests concurrently on any number of local or remote systems, thus minimizing test execution time. Grid offered, as open source, a similar capability to the internal/private Google cloud for Selenium RC. Pat Lightbody had already made a private cloud for “HostedQA” which he went on to sell to Gomez, Inc.

The name Selenium comes from a joke made by Huggins in an email, mocking a competitor named Mercury, saying that you can cure mercury poisoning by taking selenium supplements. The others that received the email took the name and ran with it.

Components of Selenium:

  • Selenium IDE
  • Selenium client API
  • Selenium Remote Control
  • Selenium WebDriver
  • Selenium Grid

Selenium IDE:

Selenium IDE is a Firefox extension plugin. It is easy to record and play the recorded scripts. We can debug and set the breakpoints and save the test in html, Webdriver and other formats. Selenium saves all the recorded information into html table format. As Selenium IDE is available only for Firefox browser it proves as its limitation. User cannot add manual scripts. So overcome all these Selenium WebDriver evolved.

Selenium WebDriver: (Selenium 2)

Biggest change in Selenium recently has been the inclusion of the WebDriver API. WebDriver is a web automation framework that allows you to execute your tests against different browsers, not just Firefox (unlike Selenium IDE).It a piece of program which intelligently control the browser by programming. It is more flexible and powerful. SeleniumWebDriver supports multiple browsers in multiple platforms.

  • Google Chrome 12.0.712.0+
  • Internet Explorer 6+
  • Firefox 3.0+
  • Opera 11.5+
  • Android – 2.3+ for phones and tablets
  • iOS 3+ for phones
  • iOS 3.2+ for tablets

WebDriver is designed to providing a simpler and uniformed programming interface/Same WebDriver script runs for different platforms. It supports multiple programming language such as Java, C#, Python, Ruby, PHP, Perl.

POWER of Selenium WebDriver:

  • Simulate user actions.
  • Functional testing.
  • Create regression tests to verify functionality and user acceptance.
  • Browser compatibility testing.
  • The same script can run on any Selenium platform.
  • Load testing.
  • Stress testing.

Selendroid Introduction:

Selendroid is a test automation framework which drives off the UI of Android native and hybrid applications (apps) and the mobile web. Tests are written using the Selenium 2 client API. Selendroid can be used on emulators and real devices and can be integrated as a node into the Selenium Grid for scaling and parallel testing.

Features of Selendroid:

  • Full compatibility with the JSON Wire Protocol/Selenium 3 Ready.
  • No modification of app under test required in order to automate it.
  • Testing the mobile web using built in Android driver webview app.
  • Same concept for automating native or hybrid apps.
  • UI elements can be found by different locator types.
  • Gestures are supported: Advanced User Interactions API.
  • Selendroid can interact with multiple Android devices (emulators or hardware devices)at the same time.
  • Existing Emulators are started automatically.
  • Selendroid supports hot plugging of hardware devices.
  • Full integration as a node into Selenium Grid for scaling and parallel testing.
  • Multiple Android target API support (10 to 19).
  • Built in Inspector to simplify test case development.
  • Selendroid can be extended at runtime with your own extensions.

Appium Introduction:

Appium is an open-source tool for automating native, mobile web, and hybrid applications on iOS and Android platforms. Native apps are those written using the iOS or Android SDKs. Mobile web apps are web apps accessed using a mobile browser (Appium supports Safari on iOS and Chrome or the built-in ‘Browser’ app on Android). Hybrid apps have a wrapper around a “webview” — a native control that enables interaction with web content. Projects like Phonegap, make it easy to build apps using web technologies that are then bundled into a native wrapper, creating a hybrid app.

Appium Concepts:

  • Client / Server Architecture
  • Session
  • Desired Capabilities
  • Appium Server
  • Appium Clients

Squish Introduction:

Squish is the leading cross-platform/cross-technology GUI test automation tool for functional
GUI regression tests.

Features:

  • Cross-platform, cross-technology GUI Test Automation Tool.
  • Creating a GUI test case.
  • Record a test by point and click Playback.
  • Edit the test script to make it more robust (JavaScript, Python, Perl, Tcl, Ruby).
  • Put into continuous integration.

What Is Automation Testing Frameworks?

A testing framework or more specifically a testing automation framework is an execution environment for automated tests. It is the overall system in which the tests will be automated. It is defined as the set of assumptions, concepts, and practices that constitute a work platform or support for automated testing. It helps to structure our code and make maintenance process easy. Use of automation framework helps to increase code reusage, and reduce the script maintenance cost.

Testing framework is responsible for:

  • Defining the format in which to express expectations.
  • Creating a mechanism to hook into or drive the application under test
  • Executing the tests
  • Reporting results

Properties of a testing framework:

  • It is application independent.
  • It is easy to expand, maintain and preserve.

Why We Need An Automation Testing Framework?

  • If we have a group of testers and suppose if each project implements a unique strategy then the time needed for the tester become productive in the new environment will take long.
  • To handle this we cannot make changes to the automation environment for each new application that comes along. For this purpose we use a testing framework that is application independent and has the capability to expand with the requirements of each application.
  • An organized test framework helps in avoiding duplication of test cases automated across the application. Test frameworks helps teams organize their test suites and in turn help improve the efficiency of testing.

Types of Automation Testing Framework:

  • Modular Testing Framework.
  • Data Driven Testing Framework.
  • Keyword Driven Testing Framework.
  • Hybrid Testing Framework.

Modular Testing Framework:

The Modularity testing framework is built on the concept of abstraction.This involves the creation of independent scripts that represent the modules of the application under test. These modules in turn are used in a hierarchical fashion to build large test cases. Thus it builds an abstraction layer for a component to hide that component from the rest of the application. Thus the changes made to the other part of the application do not affect that component. This framework introduces a high level of modularization. So when there is a change in the functionality we can change the bottom level script without affecting all the other test cases that test that control.

PROS:

    • Modular division of scripts leads to easier maintenance and also the scalability of the automated test suites.
    • The functionality is available in easy to use test libraries so creating new driver scripts for different tests is easy and fast.

CONS:

      • Main problem with modular frameworks is that the test scripts have test data embedded in them. So when the test data needs to be updated we need to change the code of the script. This becomes a big problem when the test script is large.

Data Driven Testing Framework:

Data driven testing is where the test input and the expected output results are stored in a separate data file (normally in a tabular format) so that a single driver script can execute all the test cases with multiple sets of data. The driver script contains navigation through the program, reading of the data files and logging of the test status information. Data file contains the different types of input data which will be given to the driver script. Driver script reads the data from the data file computes the value and compares it with the expected result from the data file.

PROS:

      • This framework reduces the number of overall test scripts needed to implement all the test cases.
      • Less amount of code is required to generate all the test cases.
      • Offers greater flexibility when it comes to maintenance and fixing of bugs.
      • The test data can be created before test implementation is ready or even before the system to be tested is ready.

CONS:

      • Test cases created are similar and creating new kind of tests requires creating new driver scripts that understand different data. Thus the test data and driver scripts are strongly related that changing one requires changing the other too.

Key Driven Testing Framework:

Keyword driven testing is an application independent framework utilizing data tables and self-explanatory keywords to explain the actions to be performed on the application under test. Not only is the test data kept in the file but even the directives telling what to do which is in the test scripts is put in external input data file. These directives are called keywords. The keyword based testing is an extension to the data driven testing.

PROS:

      • It has all the advantages that data driven testing has.
      • Automation expertise is not required to maintain or create a new set of test cases.
      • Keywords are reused across multiple test cases.

CONS:

      • The main problem is that this requires a more complicated framework than the data driven framework.
      • With the keyword driven approach the test cases get longer and complex and this is due to the greater flexibility that this approach offers.
      • So in order to combine the strengths of all the frameworks and their weaknesses hybrid testing framework evolved.

Hybrid Testing Framework:

Hybrid testing framework is the combination of modular, data-driven and keyword driven testing frameworks. This combination of frameworks helps the data driven scripts take advantage of the libraries which usually accompany the keyword driven testing.

Waterfall to Agile:

Agile life cycles are characterized by short and rapid test cycles and frequent change requests. Test automation plays a crucial role in software testing. Any type of testing framework can be implemented in the agile environment but with the short iterations and rapidly changing requirements it becomes difficult to maintain the test automation suite.

In the agile environments, testing plays a crucial role through the different phases of iterations. It involves continuous integration, unit testing (which is usually done using test driven development) and constant regression testing which is difficult to accomplish using testing frameworks. Also, achieving maximum code and functionality coverage using testing frameworks is difficult. Hence testing frameworks are not a good fit for the agile environment.

Why Automation Testing In Siddhatech?

In Siddhatech we work on all types of technologies and develop Web, Android & iOS applications. I work in Siddhatech as a Quality Analyst or Software tester. I have to act as end user and perform the entire testing scenario and verify how the product performance is, find out the bugs and get it resolve and obtain a good quality product. On filtering the product, application and user requirements, all the scenarios have to be designed into test cases. All the test cases are then executed manually as manual testing.

Manual testing is the best method when testing any new functionality, but it becomes difficult or a boring job when performing the same operation while doing regression testing. Verifying that updates of new code didn’t break the old working functionality,checking each functionality, User Interface in all the multiple browsers becomes a tedious work. All these demands and needs, urged to use the automation process. Automation is not a new practice. Most software teams are trying to automate testing in some way or another, especially as a replacement for long, manual regression testing cycles. Automation often takes care of the repetitive steps. If we monitor the software industry and market, few years back automation was an expensive process. Companies need to invest and purchase all the expensive software packages and needed to provide costly training to all the QA team. But Selenium Webdriver, Appium which are free and open source tools which can be used with Java, Python or Ruby brought a new revolution in software testing industry.

These open source library helps to simulate, user actions such as pressing buttons, entering text into textboxes, selecting web and mobile elements. These all conditions made a necessity for the software testers to code and understand the code. Agile software development process has also boomed the life of a software tester. Now the development team and software team need to work hand in hand for each task. Adding the regression testing in agile workflow was also a need for the software testers. With the advent of continuous integration tool like Jenkins it became easier for the QA team. Learning automation is a challenging task for every tester. Implementing manual testing on the applications and performing the same manual actions into automation is a challenging work.

Automation Testing Implementation:

In Siddhatech Automation testing is implemented on one of the android and iOS applications. Application is core banking application. Banking application consist of process Registration, login, pin creation, performing transfers from one account to other, adding account beneficiario. All these processes have repetitive scenarios which needs to tackle and end user or tester needs to perform the same operations again and again. To overcome this automation process is implemented.

Automation includes from installing the fresh application on emulator or real device,launching the application, and performing all the manual steps with robotics actions. Selenium and Appium plays a vital role in automation. Appium allows native, hybrid and supports automation test on physical devices as well as on emulator or simulator both. Appium has framework which translate Selenium Webdriver commands into UIAutomation (iOS) or UIAutomator (Android) commands depending on the device type not any OS type.

Following are the important software’s used to implement automation for android or iOS app:

        • Eclipse IDE
        • Selenium & Selendroid Jars
        • Appium
        • TestNg
        • Maven
        • Jenkins
        • Extent Reports
        • Auto IT

With Integration of above mentioned software’s we have achieve the automation for the android and iOS applications. Tester has to create a project in eclipse ide and write the respective code for all the expected test cases. Addition of the required jars in projects helps to integrate required features. Example – To send an alert mail if any functionality fails we need mail and activation jars. With the help of Appium we can run the application apk on the real device or emulator. TestNg helps to prioritize the test cases priority accordingly. Maven saves all the required repository jars to the local machine and can be downloaded if the project is moved to other locations. With Jenkins we can run continuously the scripts with proper time interval. Extent Reports helps to create fancy reports, easy to understand the test results. Auto It is used for uploading the reports on the server.