Archive

Archive for March, 2009

Test automation frameworks

March 27th, 2009 2 comments

Recently I researched many testing frameworks in order to choose one to use at work. I spent few days practically testing the following frameworks:

I chose Robot Framework. It is not perfect, but promising, extensible, and actively developed by the open source community. The following features made me choose this framework over others:

  • Keyword driven
  • Ability to use re-usable keywords
  • Large collection of built-in keywords
  • Test cases and suites are HTML documents
  • RobotIDE test case editor
  • Extendable in Python and Jython
  • Reporting using HTML and XML files

All this gives a solid base to extend and wrap around in order to create state-of-the art testing framework. Robot framework is easily installable. Examples and documentation are more than enough to get automation engineers and testers started. It requires some learning and getting used to (especially RobotIDE) but once grasped, creating test cases is easy. Additionally, new keywords can be simply programmed in Python or Jython, or created with existing keywords (macros). Tests are started with a command line tool and can be very simply scheduled using cron.

FitNesse framework is widely used and popular, but is not as extendable as robot framework. It’s a testing wiki. Test cases are created as tables in wiki documents and are started by clicking a button on a page. The results show up immediately, but there is no way to store them. Each test table must have a code fixture written in Java. Other languages can be used with a help of additional Fit servers. Being a wiki, FitNesse cannot be used for automatic and non-interactive testing. Automation can be achieved by using command line runner or ant tasks.

Cucumber is a testing environment that allows testers to write tests in a a domain-specific language based on a spoken language. It’s behavior-driven, but all used sentences must be mapped in underlying programming language (Ruby by default). It’s easy to transcribe users stories into test cases, but it requires automation engineer to work with testers writing every test case. On the other hand, it’s a very interesting and amazing way to write unit tests.

Categories: Work Tags:

My new job …

March 21st, 2009 No comments

Recently I started a new job in a agile company as Quality Assurance Automation Engineer. My task is to design and implement testing framework from scratch. The software that will be tested is under development and parts of it will be ready to test this month but as a whole it’s months from completion. The testing system must be in place soon because we have to be ready to start testing soon and prepare automated test cases.

The jobs is a great challenge. I will have to prioritize the tasks and decide what parts of the framework will need to be ready first and what can wait. We’ll have to decide how to test the system, but don’t have the system yet. The framework will need to be, therefore, very universal and extendable.

I’ve done testing automation before and I’ll use what I’ve learned so far. I designed and developed a distributed regression system for an existing manual testing framework. I’ve used Python and Django – Python for job distribution, Django for web-based result browsing. It worked great and helped the QA team to speed up automated testing.

The new job will also require a distributed testing. But here I will need to come up with a testing system: a language to write the test cases, test execution, result comparison, and so on. Due to time constraints I will most likely use an existing open-source testing framework and extend it to our needs. Can’t wait to start working on it, but for now I have a lot to learn about the system that my company is working on.

Categories: Work Tags:

Opening

March 14th, 2009 No comments

I have been thinking about starting a blog for the last few years. I have followed many professional and personal blogs getting different perspectives on things. Reading them I always thought that I should share my experiences as well.

To start, I asked my brother for a login to his hosting site and installed and configured WordPress. First I wanted to start with blogger.com, but I like to control things and I chose to maintain the blog myself. I try not to use google’s services too much (maybe I’ll explain that in a future post) – except search maybe.

Now I’ll start thinking about the next post …

Categories: Uncategorized Tags: