What Engineering Has in Common With Manufacturing and Why It Matters:
- Software engineering is more like manufacturing than most people expect. Once we spot their similarities, we can apply the lessons learned over the last 50 years in manufacturing to software development. This article by Dr. Alistair Cockburn picks six lessons to apply to software development gleaned from the manufacturing industry. (9 pages)
Why Software Fails:
- The business and societal costs of software failures - in terms of wasted taxpayer and shareholder dollars as well as investments that can't be made - are now well into the billions of dollars a year. Of the IT projects that are initiated, from 5 to 15 percent will be abandoned before or shortly after delivery as hopelessly inadequate. Many others will arrive late and over budget or require massive reworking. The biggest tragedy, however, is that software failure is for the most part predictable and avoidable. Unfortunately, most organizations don't see preventing failure as an urgent matter, even though that view risks harming the organization and maybe even destroying it. Understanding why this attitude persists has tremendous implications for business and society. Read more in this IEEE Spectrum article by Dr. Robert Charette. (13 pages)
The Challenge of "Good Enough" Software:
- This article by James Bach is a slightly updated version of the the original that was published in American Programmer magazine in 1995. The author's ideas are rooted in his experiences at Apple Computer in the late 80's, and Borland in the early nineties. This article was treated as a counter-culture rant in 1995. But today, the agile development movement, which exemplifies these ideas, is well established. (11 pages)

Product	Vendor	Comments
AdaTEST	IPL	Coverage, static, and dynamic software testing for Ada Apps
AQtime	AutomatedQA	Profiling toolkit for thorough analysis of Delphi, Visual Basic, Visual C++, C++ Builder, Intel C++, GCC and Visual Fortran applications. It offers over two dozen performance and memory usage profilers and productivity tools that work in unison to give you an unrivaled level of information on the state of your software projects - from inception to delivery.
BoundsChecker	Compuware	Run-time error detection and debugging tool for C++ developers. Supports C/C++,.net,ASP,ASP.net.
Bullseye Coverage	Bullseye Testing Technology	C/C++ code coverage
CMT++	Testwell	Static analysis/metric tool for C and C++ code
Code Coverage	DMS	DCC reinvents code coverage. Without recompiling or relinking, function, line, decision and branch coverage information is gathered. Full source code annotation is given. Information from multiple runs can be aggregated.
CodeCheck	Abraxas Software	Measures maintainability, portability, complexity, and standards compliance of C and C++ source code
CodeWizard	ParaSoft	C++ analysis tool
CTA++, CTB	Testwell	Test harnessing tools for unit testing C and C++ code
CTC++	Testwell	Test coverage analyzer and execution profiling tool for C++ code
devAdvantage	Anticipating Minds	Create custom C# coding standards. Detects and corrects for .NET best practices.
Diversity Analyzer	Vidak Quality	Implements a unique patented technology aimed at measuring the control and data diversity given by a test suite for C/C++/C# and VB code. Higher control and data diversity higher the probability of problem detection. Also gives conditional coverage, as a special case of conditional diversity.
GlowCode	Electric Software	Memory and resource leak detection, code profiler, function call trace, and report tools for Win32 programs
Insure++	Parasoft	Runtime error detection tool for C and C++
LDRA Testbed	LDRA	Static and code coverage analysis tool for C/C++, C#, Ada83, Ada95, Java, Visual Basic, Cobol, Coral66, Fortran, Pascal, PL/Mx86, PL/1, Algol and Intel, Motorola, Texas Instruments and PowerPC Assemblers. Static analysis features include code visualization, programming standards enforcement and complexity metrics. Code coverage is available for different coverage levels including MC/DC level A for D0-178B standard, regression testing and availability on any host/target environment.
Leak Check	DMS	Quality tool used for finding memory leaks. No recompile or relink is required! Application code and 3rd party code can be checked. Innovative Leak Pinpointing gives both the allocation stack and the spot of the leak
Logiscope	Telelogic	Source code analysis tools
OSPC	Knowledge Software	Checks applications, written in C for conformance to company coding standards, International standards, and conformance to the C standard
Panorama	ISA Inc	Test Coverage Analysis & Metrics Testing for C & C++
McCabe TQ	McCabe Software	Using McCabe IQ, you will identify, objectively measure, and report on the complexity and quality of your code at the application and enterprise level. McCabe IQ's robust and easy to understand test coverage solution shows you what has been tested, how thoroughly, and if the code has been tested after changes have been made.
PolySpace Suite	PolySpace	Detects run-time errors and non-deterministic constructs in applications at compilation time. The Suite is available for ANSI C, C++ and Ada 83/95.
Predictive Lite	ism	Code analyzer that predicts defects in C, C++, Java or C# source code, based on proven industry standards.
Prevent	Coverity	Static source cod analysis tool to automate the detection of catastrophic software defects at compile time. Developed at Stanford University, Coverity's technology is used at nearly 80 companies to ensure that code is secure and high quality.
Purify	Rational Software	Run-time error and memory leak detector
TBGEN	Testwell	Test harnessing tool for unit testing Ada code
TCAT C/C++	Software Research	TCAT C/C++ is a fully automated tool for static analysis and code coverage at the unit, sub-system, system and integration testing levels. With TCAT C/C++ available testing efforts can focus on untested code, rather than generating redundant tests. The resulting data is displayed in coverage reports, color-annotated call-trees and digraphs with capabilities of viewing the source code. Using recursive descent compiler technology, TCAT C/C++ produces many reports from which you can access current, past and cumulative test results.
TCMON	Testwell	Test coverage and execution profiling tool for Ada code
Test Coverage	Semantic Designs	Family of test coverage tools for a wide variety of languages (C, C++, C#, COBOL, Java, more...). Very low probe overhed. Independent of compiler. Useful for application and embedded contexts. Graphical display of coverage over source code; text and XML summaries.

Product

Vendor

Comments

IPL

Coverage, static, and dynamic software testing for Ada Apps

AutomatedQA

Profiling toolkit for thorough analysis of Delphi, Visual Basic, Visual C++, C++ Builder, Intel C++, GCC and Visual Fortran applications. It offers over two dozen performance and memory usage profilers and productivity tools that work in unison to give you an unrivaled level of information on the state of your software projects - from inception to delivery.

BoundsChecker

Compuware

Run-time error detection and debugging tool for C++ developers. Supports C/C++,.net,ASP,ASP.net.

Bullseye Coverage

Bullseye Testing Technology

C/C++ code coverage

CMT++

Testwell

Static analysis/metric tool for C and C++ code

Code Coverage

DMS

DCC reinvents code coverage. Without recompiling or relinking, function, line, decision and branch coverage information is gathered. Full source code annotation is given. Information from multiple runs can be aggregated.

CodeCheck

Abraxas Software

Measures maintainability, portability, complexity, and standards compliance of C and C++ source code

CodeWizard

ParaSoft

C++ analysis tool

CTA++, CTB

Testwell

Test harnessing tools for unit testing C and C++ code

CTC++

Testwell

Test coverage analyzer and execution profiling tool for C++ code

devAdvantage

Anticipating Minds

Create custom C# coding standards. Detects and corrects for .NET best practices.

Diversity Analyzer

Vidak Quality

Implements a unique patented technology aimed at measuring the control and data diversity given by a test suite for C/C++/C# and VB code. Higher control and data diversity higher the probability of problem detection. Also gives conditional coverage, as a special case of conditional diversity.

GlowCode

Electric Software

Memory and resource leak detection, code profiler, function call trace, and report tools for Win32 programs

Insure++

Parasoft

Runtime error detection tool for C and C++

LDRA Testbed

LDRA

Static and code coverage analysis tool for C/C++, C#, Ada83, Ada95, Java, Visual Basic, Cobol, Coral66, Fortran, Pascal, PL/Mx86, PL/1, Algol and Intel, Motorola, Texas Instruments and PowerPC Assemblers. Static analysis features include code visualization, programming standards enforcement and complexity metrics. Code coverage is available for different coverage levels including MC/DC level A for D0-178B standard, regression testing and availability on any host/target environment.

Leak Check

DMS

Quality tool used for finding memory leaks. No recompile or relink is required! Application code and 3rd party code can be checked. Innovative Leak Pinpointing gives both the allocation stack and the spot of the leak

Logiscope

Telelogic

Source code analysis tools

OSPC

Knowledge Software

Checks applications, written in C for conformance to company coding standards, International standards, and conformance to the C standard

Panorama

ISA Inc

Test Coverage Analysis & Metrics Testing for C & C++

McCabe TQ

McCabe Software

Using McCabe IQ, you will identify, objectively measure, and report on the complexity and quality of your code at the application and enterprise level. McCabe IQ's robust and easy to understand test coverage solution shows you what has been tested, how thoroughly, and if the code has been tested after changes have been made.

PolySpace Suite

PolySpace

Detects run-time errors and non-deterministic constructs in applications at compilation time. The Suite is available for ANSI C, C++ and Ada 83/95.

Predictive Lite

ism

Code analyzer that predicts defects in C, C++, Java or C# source code, based on proven industry standards.

Prevent

Coverity

Static source cod analysis tool to automate the detection of catastrophic software defects at compile time. Developed at Stanford University, Coverity's technology is used at nearly 80 companies to ensure that code is secure and high quality.

Purify

Rational Software

Run-time error and memory leak detector

TBGEN

Testwell

Test harnessing tool for unit testing Ada code

TCAT C/C++

Software Research

TCAT C/C++ is a fully automated tool for static analysis and code coverage at the unit, sub-system, system and integration testing levels. With TCAT C/C++ available testing efforts can focus on untested code, rather than generating redundant tests. The resulting data is displayed in coverage reports, color-annotated call-trees and digraphs with capabilities of viewing the source code. Using recursive descent compiler technology, TCAT C/C++ produces many reports from which you can access current, past and cumulative test results.

TCMON

Testwell

Test coverage and execution profiling tool for Ada code

Test Coverage

Semantic Designs

Family of test coverage tools for a wide variety of languages (C, C++, C#, COBOL, Java, more...). Very low probe overhed. Independent of compiler. Useful for application and embedded contexts. Graphical display of coverage over source code; text and XML summaries.

Product	Vendor	Comments
.TEST	Parasoft	Unit-testing tool that automatically tests classes written on Microsoft's .NET Framework without requiring developers to write a single test scenario or stub.
AberroTest	Aberro Software	AberroTest is a functional test and verification product that works at the user interface level.
AETG Web	Telcordia	The Telcordia Efficient Test Case Generation Service, formerly known as AETG Web, makes it easy to use AETG, the Automatic Efficient Test Generation system. The AETG algorithms employ combinatorial design techniques to create minimal sets of tests that cover all the pairwise interactions among input values. Free trial account (14 days) available. Implemented as a service delivered via our site, so there is no software to install.
Automate!Test Manager	Newmerix	Automate!Test Manager is the first solution specially designed for PeopleSoft that automates the functional testing process. By capturing and replaying the most important business processes in your application, any time a change is made to the application, the most important processes can be quickly and cost effectively tested.
Automated Test Designer	AtYourSide Consulting	Tool for creating Test Cases based on Functional Requirements, which can be used in a collaborative environment by all project team members and can handle changes and requirements of any complexity. It uses an advanced and rigorous Neural Network Optimization algorithm and reduction methods, in order to generate the minimum number of test cases to certify 100% of the Requirements rules.
AutoTester One	AutoTester	Functional, regression, and systems integration testing of Windows, Client Server, Host/Legacy, or Web software. Also offers a version enhanced to work with SAP applications and a load and stress testing engine that drives AutoTester ONE test scripts to recreate normal and peak load conditions in order to assess system performance.
Avignon Acceptance Testing System		Acceptance test system that allows you to write executable tests in a language that you define. It uses XML to define the syntax of the language but, if you choose to extend the language, leaves the semantics of the tests up to you.
BugHuntress	BugHuntress QA	BugHuntress Test Suite (BHTS) is a client-server system which is intended for full-scale automated testing of applications on handhelds and/or emulators with Palm OS 3.0-4.1. PC and Palm parts of BHTS interact through the COM-port, by using the USB-bus and via the TCP/IP protocol. Automated testing is performed using special scenarios, scripts and control points (e.g. screen CRC, controls states, databases states, etc.) which basically represent software test plan written in Java-script with BHTS commands. BHTS requires no additional hardware.
CAPBAK/X, CAPBAK/MSW	Software Research	Capture/playback system for X, and Windows which allows the user to record mouse mouvements, keyboard activities, widget calls and verification information into test scripts. Fully editable and programmable "C" language scripts make CAPBAK adaptable to your testing requirements. Its automatic synchronization features discover minor application changes during playback. CAPBAK has a realistic Load Generation engine built-in and handles all kinds of GUI testing: Local, Client/Server, Remote, Host/target. Test can run unsupervised, as often as required. Used together with SMARTS, the sophisticated test management component that controls test execution and verification you have the perfect Regression Test Suite.
Certify	Worksoft	Platform neutral solution that allows development of tests without programming for Web, client/server and mainframe applications.
CitraTest	Tevron LLC	Automated software testing solution for all Citrix Environments
Code Testing Tool Pro	RHiTech	This tool is a convenient wrapper for your testing code and test cases. It allows doing code testing (software testing) in the next modes: single-threaded, multithreaded and stress-mode (emulating the peak overloads). Support the next development environments: MS VC++, Borland C++ Builder and Borland Delphi. Installation with examples sources is included.
Eggplant	Redstone Software	Running on MacOSX, Eggplant can test and automate processes on virtually any platform. Leveraging open source VNC server software, Eggplant achieves platform independence, "seeing" the remote screen and acting as a virtual user to generate and run test scripts and to automate other tasks on the remote machine.
Eventcorder	CMS	Eventcorder records and plays back Windows events. It searches for 'Clickviews' and focused windows during playback. E-Scripter is a scripting utility (VBScript) to drive Eventcorder.
FERRET	Azor Inc	Automated Software Testing System for Macintosh, Windows, OS/2 and X-Windows software
GUITAR	University of Maryland	A tool that generates new test cases using a new technology of event-flow graphs. Pre/postconditions are used to generate expected response. Coverage (event and code) is evaluated. Free.
Haven	Lecando	Automated acceptance testing of applications with parallel execution in multiple environments. The tests in Haven are JUnit based so you can continue to use your JUnit enabled IDE when you write, debug and run your acceptance tests. Since the test process in Haven is defined using Apache Ant, you can get help from your IDE here too. In Haven you can test as many products as you want. Haven can test all branches of your products simultaneously.
Holodeck	Security Innovations	Uses fault simulation to emulate real-world application and system errors. This allows testers and developers to work in a controlled, repeatable environment to analyze and debug error-handling code in hostile environments.
JPdfUnit	OIO	Integrates PDFBox as a PDF API with the JUnit framwork for the test of pdf documents so JPdfUnit is a high level api. Simple ready-to-use assertions help to compare the expected data to the concrete data of the pdf document.
MITS.GUI		Has an intelligent state machine engine that makes real-time decisions for navigating through the GUI portion of an application without scripts. Testers simply enter test data into a spreadsheet which is used to populate the objects that appear for the particular test scenario defined.
PETA	verit	Eclipse-based platform for automated software testing and covers the software life cycle from the specification up to the maintenance phase. The components of message based software systems like Client-/Server-, Multitier- and Service-Oriented-Architectures (SOA) may be simulated as well as tested in isolation and in their supposed interaction.
PyUnit		Unit testing framework. Python language version of JUnit. Open source.
QACenter	Compuware	Test development and execution tool for GUI and character-based client-server software testing
Replay Xcessory	Integrated Computer Solutions	Automation tool for Xt/Motif applications
Repro	Repro Software	Records software process execution at multiple levels to provide unmatched visibility into process execution. With Repro you can record, annotate, and playback: Video capture of user actions System level traces of operations and data streams System resource usage and performance graphs A complete summary of the PC's configuration at the time the bug occurred Each captured component is viewed simultaneously during playback to give the software professional an integrated and organized view of the recording along each dimension.
SAP Software Quality Assurance Testing Tools	Sucid	The leading provider of SAP software quality assurance testing tools and products.
ScriptMap® for Siebel	Serpico Software	Planning and functional testing tool which focuses exclusively on Siebel applications. Serpico's proprietary ScriptMap technology delivers functional 'testing with context'. The key to 'testing with context' is the removal of serial scripts as the primary mode of navigation, application control and data entry. The absence of scripts for all aspects of the testing process enables accelerated creation of test scenarios and delivers a measurable reduction in test asset maintenance. Effective testing throughout the whole Siebel lifecycle finds more defects, thus lowering the risk and reducing the cost of the project.
ScriptTech	TMX	Generates test narratives and fully executable scripts for WinRunner, TestPartner, SilkTest and other automation engines with out the need for understanding or knowing the underlying programming language. Provides a single point of maintenance using global "find and replace" features for test data, object definitions and action definitions.
Smalltalk Test Mentor	SilverMark, Inc.	Automated GUI and domain object software testing tool for IBM's
Squish	froglogic	Cross platform software testing framework for Qt/C++ applications. With no modifications to the application, it is possible to test the application using recorded or manually written test scripts in different scripting languages.
TALC2000	Tallecom Software	Test automation for applications on mainframe and mid range proprietary and UNIX platforms.
TestArchitect	LogiGear	Keyword-driven test automation framework. It includes automation support for the most common platforms in use today, along with powerful version-control features to support team-based test automation.
TestComplete	AutomatedQA	Automated test manager, with unmatched support for unit, functional, regression, distributed and HTTP performance testing at the project level. Designed for application developers and testers alike, TestComplete will help you to achieve thorough Quality Assurance in development from the first line of code right through delivery and maintenance, with no surprises along the way. Ship superior applications, and ship them on time. Supports Delphi, Win32, .NET, Java, and Web applications.
TestWorks	Software Research	Integrated suite of Automated Regression Test and Coverage Analyzers for UNIX, Windows/2000//NT/XP with multiple reports and re-usable test scripts in C, C++ and JAVA.
Unified Test Pro	Software Development Technologies	A "3rd generation" test automation solution, which uses a roles-based software test automation approach to Design, Build and Run test cases.
Vermont HighTest Plus	Vermont Creative Software	. Windows-based software regression-testing tool for testing stand-alone or browser-based applications. With our direct integration into Internet Explorer, testing a web-based application or website is as easy as testing any other application. Just use the application as you normally would, and HighTest will take care of the rest. Or you can customize the generated scripts to harness the full power of HighTest. HighTest gives you the flexibility to work the way you want to, in a product that is easier to use than ANY other automated software testing tool on the market.
VNCRobot	VNCRobot	Free tool for automated software testing based on VNC technology which enables to program, verify and reproduce behavior of a remote computer via RFB protocol and generate a comprehensive report.
X-Unity	MIIK Ltd.	Unit testing environment for the Microsoft .NET framework

Product

Vendor

Comments

.TEST

Parasoft

Unit-testing tool that automatically tests classes written on Microsoft's .NET Framework without requiring developers to write a single test scenario or stub.

AberroTest

Aberro Software

AberroTest is a functional test and verification product that works at the user interface level.

AETG Web

Telcordia

The Telcordia Efficient Test Case Generation Service, formerly known as AETG Web, makes it easy to use AETG, the Automatic Efficient Test Generation system. The AETG algorithms employ combinatorial design techniques to create minimal sets of tests that cover all the pairwise interactions among input values. Free trial account (14 days) available. Implemented as a service delivered via our site, so there is no software to install.

Automate!Test Manager

Newmerix

Automate!Test Manager is the first solution specially designed for PeopleSoft that automates the functional testing process. By capturing and replaying the most important business processes in your application, any time a change is made to the application, the most important processes can be quickly and cost effectively tested.

Automated Test Designer

AtYourSide Consulting

Tool for creating Test Cases based on Functional Requirements, which can be used in a collaborative environment by all project team members and can handle changes and requirements of any complexity. It uses an advanced and rigorous Neural Network Optimization algorithm and reduction methods, in order to generate the minimum number of test cases to certify 100% of the Requirements rules.

AutoTester One

AutoTester

Functional, regression, and systems integration testing of Windows, Client Server, Host/Legacy, or Web software. Also offers a version enhanced to work with SAP applications and a load and stress testing engine that drives AutoTester ONE test scripts to recreate normal and peak load conditions in order to assess system performance.

Avignon Acceptance Testing System

Acceptance test system that allows you to write executable tests in a language that you define. It uses XML to define the syntax of the language but, if you choose to extend the language, leaves the semantics of the tests up to you.

BugHuntress

BugHuntress QA

BugHuntress Test Suite (BHTS) is a client-server system which is intended for full-scale automated testing of applications on handhelds and/or emulators with Palm OS 3.0-4.1. PC and Palm parts of BHTS interact through the COM-port, by using the USB-bus and via the TCP/IP protocol. Automated testing is performed using special scenarios, scripts and control points (e.g. screen CRC, controls states, databases states, etc.) which basically represent software test plan written in Java-script with BHTS commands. BHTS requires no additional hardware.

CAPBAK/X, CAPBAK/MSW

Software Research

Capture/playback system for X, and Windows which allows the user to record mouse mouvements, keyboard activities, widget calls and verification information into test scripts. Fully editable and programmable "C" language scripts make CAPBAK adaptable to your testing requirements. Its automatic synchronization features discover minor application changes during playback. CAPBAK has a realistic Load Generation engine built-in and handles all kinds of GUI testing: Local, Client/Server, Remote, Host/target. Test can run unsupervised, as often as required. Used together with SMARTS, the sophisticated test management component that controls test execution and verification you have the perfect Regression Test Suite.

Certify

Worksoft

Platform neutral solution that allows development of tests without programming for Web, client/server and mainframe applications.

CitraTest

Tevron LLC

Automated software testing solution for all Citrix Environments

Code Testing Tool Pro

RHiTech

This tool is a convenient wrapper for your testing code and test cases. It allows doing code testing (software testing) in the next modes: single-threaded, multithreaded and stress-mode (emulating the peak overloads). Support the next development environments: MS VC++, Borland C++ Builder and Borland Delphi. Installation with examples sources is included.

Eggplant

Redstone Software

Running on MacOSX, Eggplant can test and automate processes on virtually any platform. Leveraging open source VNC server software, Eggplant achieves platform independence, "seeing" the remote screen and acting as a virtual user to generate and run test scripts and to automate other tasks on the remote machine.

Eventcorder

CMS

Eventcorder records and plays back Windows events. It searches for 'Clickviews' and focused windows during playback. E-Scripter is a scripting utility (VBScript) to drive Eventcorder.

FERRET

Azor Inc

Automated Software Testing System for Macintosh, Windows, OS/2 and X-Windows software

GUITAR

University of Maryland

A tool that generates new test cases using a new technology of event-flow graphs. Pre/postconditions are used to generate expected response. Coverage (event and code) is evaluated. Free.

Haven

Lecando

Automated acceptance testing of applications with parallel execution in multiple environments. The tests in Haven are JUnit based so you can continue to use your JUnit enabled IDE when you write, debug and run your acceptance tests. Since the test process in Haven is defined using Apache Ant, you can get help from your IDE here too. In Haven you can test as many products as you want. Haven can test all branches of your products simultaneously.

Holodeck

Security Innovations

Uses fault simulation to emulate real-world application and system errors. This allows testers and developers to work in a controlled, repeatable environment to analyze and debug error-handling code in hostile environments.

JPdfUnit

OIO

Integrates PDFBox as a PDF API with the JUnit framwork for the test of pdf documents so JPdfUnit is a high level api. Simple ready-to-use assertions help to compare the expected data to the concrete data of the pdf document.

MITS.GUI

Has an intelligent state machine engine that makes real-time decisions for navigating through the GUI portion of an application without scripts. Testers simply enter test data into a spreadsheet which is used to populate the objects that appear for the particular test scenario defined.

PETA

verit

Eclipse-based platform for automated software testing and covers the software life cycle from the specification up to the maintenance phase. The components of message based software systems like Client-/Server-, Multitier- and Service-Oriented-Architectures (SOA) may be simulated as well as tested in isolation and in their supposed interaction.

PyUnit

Unit testing framework. Python language version of JUnit. Open source.

QACenter

Compuware

Test development and execution tool for GUI and character-based client-server software testing

Replay Xcessory

Integrated Computer Solutions

Automation tool for Xt/Motif applications

Repro

Repro Software

Records software process execution at multiple levels to provide unmatched visibility into process execution. With Repro you can record, annotate, and playback:

Video capture of user actions
System level traces of operations and data streams
System resource usage and performance graphs
A complete summary of the PC's configuration at the time the bug occurred

Each captured component is viewed simultaneously during playback to give the software professional an integrated and organized view of the recording along each dimension.

SAP Software Quality Assurance Testing Tools

Sucid

The leading provider of SAP software quality assurance testing tools and products.

ScriptMap® for Siebel

Serpico Software

Planning and functional testing tool which focuses exclusively on Siebel applications.

Serpico's proprietary ScriptMap technology delivers functional 'testing with context'. The key to 'testing with context' is the removal of serial scripts as the primary mode of navigation, application control and data entry. The absence of scripts for all aspects of the testing process enables accelerated creation of test scenarios and delivers a measurable reduction in test asset maintenance.

Effective testing throughout the whole Siebel lifecycle finds more defects, thus lowering the risk and reducing the cost of the project.

ScriptTech

TMX

Generates test narratives and fully executable scripts for WinRunner, TestPartner, SilkTest and other automation engines with out the need for understanding or knowing the underlying programming language. Provides a single point of maintenance using global "find and replace" features for test data, object definitions and action definitions.

Smalltalk Test Mentor

SilverMark, Inc.

Automated GUI and domain object software testing tool for IBM's

Squish

froglogic

Cross platform software testing framework for Qt/C++ applications. With no modifications to the application, it is possible to test the application using recorded or manually written test scripts in different scripting languages.

TALC2000

Tallecom Software

Test automation for applications on mainframe and mid range proprietary and UNIX platforms.

TestArchitect

LogiGear

Keyword-driven test automation framework. It includes automation support for the most common platforms in use today, along with powerful version-control features to support team-based test automation.

TestComplete

AutomatedQA

Automated test manager, with unmatched support for unit, functional, regression, distributed and HTTP performance testing at the project level. Designed for application developers and testers alike, TestComplete will help you to achieve thorough Quality Assurance in development from the first line of code right through delivery and maintenance, with no surprises along the way. Ship superior applications, and ship them on time. Supports Delphi, Win32, .NET, Java, and Web applications.

TestWorks

Software Research

Integrated suite of Automated Regression Test and Coverage Analyzers for UNIX, Windows/2000//NT/XP with multiple reports and re-usable test scripts in C, C++ and JAVA.

Unified Test Pro

Software Development Technologies

A "3rd generation" test automation solution, which uses a roles-based software test automation approach to Design, Build and Run test cases.

Vermont HighTest Plus

Vermont Creative Software

. Windows-based software regression-testing tool for testing stand-alone or browser-based applications. With our direct integration into Internet Explorer, testing a web-based application or website is as easy as testing any other application. Just use the application as you normally would, and HighTest will take care of the rest. Or you can customize the generated scripts to harness the full power of HighTest. HighTest gives you the flexibility to work the way you want to, in a product that is easier to use than ANY other automated software testing tool on the market.

VNCRobot

Free tool for automated software testing based on VNC technology which enables to program, verify and reproduce behavior of a remote computer via RFB protocol and generate a comprehensive report.

X-Unity

MIIK Ltd.

Unit testing environment for the Microsoft .NET framework

Product	Vendor	Comments
BugTimer	BugStomper Software	BugTimer was designed to streamline the entire process of timing and documenting Performance Test results into one Application. BugTimer is a timer application that records, displays, saves, sorts, and prints Performance Test results.
DB Stress	DTM	Utility for stress testing the server parts of information systems and applications, as well as DBMSs and servers themselves.
LoadeaTest	Loadea	Loadea is composed with 3 modules: Capture module to create a test project, use predefined capture packages/plug ins, model your test as a C# script and modelize data in XML. Control module to define test and deploy it on several machines: number of virtual users, performance system, monitor test. Analyze module to analyze test using proven mathematical formalism which helps classify information from test, generate reports, print and export data.
Monitor Master	Argogroup	Monitor Master is a quality management solution for wireless data services - WML/XHTML, SMS and MMS. Monitor Master helps Operators and Content Developers ensure that end-users of wireless applications will have the best quality of experience possible - regardless of the handset being used.
IxLoad	Ixia	A "Load Appliance" for running performance testing applications that can emulate millions of users. Ixia's portable and rack-mounted chassis provide a managable, scalable, repeatable platform that eliminates the need for large numbers of PCs. Massive test results can be displayed in real-time, logged and analyzed with built-in graphical reporting. Emulated protocols include HTTP, SSL, FTP, SMTP, POP3, Streaming/VoIP, IPSEC, PPPoX and others.
QACenter Performance Edition	Compuware	Load testing
Scapa StressTest for Citrix MetaFrame	Scapa Technologies	Scapa StressTest for Citrix MetaFrame is a performance testing tool that tests how well MetaFrame deployments perform under variable load. The tool is highly scalable and can generate indepth analysis of systems running up to 10's of thousands of users. Available globally through Citrix partners under affordable licensing and support packages, its unique n-Tier Resonance technology quickly provides accurate client-server timings through direct use of the ICA channel.
Shunra\Storm	Shunra Software	Shunra's solutions enable IT professionals to pre-empt application performance problems and avoid downtime. By creating a mirror image of the production network environment in the lab, ShunraÁ\|s products enable customers to accurately evaluate the functionality and scalability of networked applications - before and after deployment.
SSW Performance PRO! 97	Superior Software for Windows Pty Ltd	Essential for performance tuning your Access frontend or upsizing to SQL Server. SSW Performance PRO! analyses every form, combo and list box in your Access 97, 2000 or 2002 app. It checks the SQL statements and reports on the slowest features. It tells you exactly where to spend time that will make a difference to the speed of your application. You can also use SSW Performance PRO! after you've upsized an application to SQL Server to pinpoint where tuning is required.
TestLoad	Origsoft	iSeries-specific automated load testing solution. Rather than placing artificial load on the network, it runs natively on the server, simulating actual system performance, monitoring and capturing Batch activity, server jobs and green-screen activity. TestLOAD synchronises, launches and monitors hundreds of virtual users concurrently, running multiple versions of different scripts to simulate how each might use different parts of the application and carry out different transactions.
Vantage	Compuware	Integrated suite of application performance management products. Each product deals with a different aspect of the application's performance across the network infrastructure.
WinFeedback	Beson Data	WinFeedback is a Windows scripting extension for testing, monitoring and automation purposes, like response timing, up-timing, functional testing, stress testing, health monitoring, task automation.
XtremeLoad	US Computer Software	XtremeLoad is a software framework for load testing of distributed software systems. It provides an extensible, scalable, and easy-to-use foundation on which you can build a comprehensive and cost-effective load testing solution tailored to your product.

Product

Vendor

Comments

BugTimer

BugStomper Software

BugTimer was designed to streamline the entire process of timing and documenting Performance Test results into one Application. BugTimer is a timer application that records, displays, saves, sorts, and prints Performance Test results.

DB Stress

DTM

Utility for stress testing the server parts of information systems and applications, as well as DBMSs and servers themselves.

LoadeaTest

Loadea

Loadea is composed with 3 modules:

Capture module to create a test project, use predefined capture packages/plug ins, model your test as a C# script and modelize data in XML.
Control module to define test and deploy it on several machines: number of virtual users, performance system, monitor test.
Analyze module to analyze test using proven mathematical formalism which helps classify information from test, generate reports, print and export data.

Monitor Master

Argogroup

Monitor Master is a quality management solution for wireless data services - WML/XHTML, SMS and MMS. Monitor Master helps Operators and Content Developers ensure that end-users of wireless applications will have the best quality of experience possible - regardless of the handset being used.

IxLoad

Ixia

A "Load Appliance" for running performance testing applications that can emulate millions of users. Ixia's portable and rack-mounted chassis provide a managable, scalable, repeatable platform that eliminates the need for large numbers of PCs. Massive test results can be displayed in real-time, logged and analyzed with built-in graphical reporting. Emulated protocols include HTTP, SSL, FTP, SMTP, POP3, Streaming/VoIP, IPSEC, PPPoX and others.

QACenter Performance Edition

Compuware

Load testing

Scapa StressTest for Citrix MetaFrame

Scapa Technologies

Scapa StressTest for Citrix MetaFrame is a performance testing tool that tests how well MetaFrame deployments perform under variable load. The tool is highly scalable and can generate indepth analysis of systems running up to 10's of thousands of users. Available globally through Citrix partners under affordable licensing and support packages, its unique n-Tier Resonance technology quickly provides accurate client-server timings through direct use of the ICA channel.

Shunra\Storm

Shunra Software

Shunra's solutions enable IT professionals to pre-empt application performance problems and avoid downtime. By creating a mirror image of the production network environment in the lab, ShunraÁ|s products enable customers to accurately evaluate the functionality and scalability of networked applications - before and after deployment.

SSW Performance PRO! 97

Superior Software for Windows Pty Ltd

Essential for performance tuning your Access frontend or upsizing to SQL Server. SSW Performance PRO! analyses every form, combo and list box in your Access 97, 2000 or 2002 app. It checks the SQL statements and reports on the slowest features. It tells you exactly where to spend time that will make a difference to the speed of your application. You can also use SSW Performance PRO! after you've upsized an application to SQL Server to pinpoint where tuning is required.

TestLoad

Origsoft

iSeries-specific automated load testing solution. Rather than placing artificial load on the network, it runs natively on the server, simulating actual system performance, monitoring and capturing Batch activity, server jobs and green-screen activity. TestLOAD synchronises, launches and monitors hundreds of virtual users concurrently, running multiple versions of different scripts to simulate how each might use different parts of the application and carry out different transactions.

Vantage

Compuware

Integrated suite of application performance management products. Each product deals with a different aspect of the application's performance across the network infrastructure.

WinFeedback

Beson Data

WinFeedback is a Windows scripting extension for testing, monitoring and automation purposes, like response timing, up-timing, functional testing, stress testing, health monitoring, task automation.

XtremeLoad

US Computer Software

XtremeLoad is a software framework for load testing of distributed software systems. It provides an extensible, scalable, and easy-to-use foundation on which you can build a comprehensive and cost-effective load testing solution tailored to your product.

Product	Vendor	Comments
Abbot		Abbot is a JUnit extension for testing Java GUIs. It supports recording and playback at both low and high levels of abstraction.
AdaptiveCells/J		Open source J2EE benchmarking system. It enables the development of complex artificial J2EE test-beds without requiring a single line of code. Such test-beds can be used for: Performance testing : if you want to compare the performance of different application servers or if you need to test the functionality/performance of you server in particular contexts. Middleware infrastructure testing : if you are developing some infrastructure software (eg. server adaptation software or a monitoring solution). Creating working J2EE applications when you just need to have some running systems (maybe you are learning J2EE or you work on some other aspects of a larger system and you need a working J2EE application for testing everything together).
AgileTest	Polygenix	Mock object implementation for unit testing both Java interfaces and classes. AgileTest works at the bytecode level, enabling non-intrusive dynamic mock generation - without the generation of additional source artefacts or the constraints of the Java proxy mechanism. Free for non-commercial use.
Agitator	Agitar	Helps developers create and maintain unit tests for all their code.
AppPerfect DevSuite	AppPerfect	Suite of products designed to help you design and build high-performance applications during the development phase of the product-development life-cycle. The AppPerfect DevSuite consists of Unit Tester, Code Analyzer J2EE / JVM Profiler, Functional Tester, Load Tester.
Bugkilla		Set of java tools (open source) for the functional test of J2EE Web Applications. Specification and execution of tests will be automated for web front end and business logic layer. One goal is to integrate with existing frameworks and tools. We started with eclipse struts and junit integration.
Cactus	Jakarta Project	A simple test framework for unit testing server-side java code (Servlets, EJBs, Tag Libs, Filters, ...). The intent of Cactus is to lower the cost of writing tests for server-side code. It uses JUnit and extends it.
GJ-Coverage	Tester's Edge	Coverage tool for Java programs of all types: J2ME, J2SE, and J2EE. Features ease of use, low overhead, HTML reports, branch level coverage, test case breakdowns, and more.
GJTester	TreborSoft	Java testing tool that provides a GUI to aid developers in building test cases and test scripts. It allows the testers to accomplish unit and regression test without programming effort. The tool is useful for testing CORBA, RMI and other server technologies as well.
GUIdancer	Bredex GmbH	Automated GUI test tool for programs written with Java and Swing, which requires no programming. Created tests are highly reusable and easily maintainable. Because of the nature of the object recognition, tests are resilient to changes in the application under test. Additional features include an observation mode, generation of HTML result reports, multi-user capacity, and user-defined hierarchical organization. Platform independent; runs as standalone application or as an Eclipse plugin.
JCover	Codework	Java code coverage testing
JCover	Man Machine Systems	Java code coverage
Jemmy		An open source library that can be used to create tests for Java GUI applications.
JMeter	Apache	Java desktop application designed to load test functional behavior and measure performance. Open source.
JStyle	Man Machine Systems	Java source code analyzer.
JSystem		Framework for writing and running automated tests, based on JUnit. Its main goal is to support automation of functional/system testing.
jtest	Parasoft	Java class testing
JUnit	junit.org	Regression testing framework for the developer who implements unit tests in Java. Open Source
JVerify	Man Machine Systems	Java unit/integration testing.
KCC	Koalog	Code coverage computation application written in Java. Its main features are: in-process or remote coverage computation, capability of working directly on Java binaries (no recompilation), predefined (XML, HTML, LaTeX, CSV, TEXT) or custom report generation, session merging, portability, ease-of-use, Ant integration.
LISA	iTKO	Uses "inline testing" technology to talk to every component within your infrastructure. One-click wizards enable even non-developers to connect to, analyze and interact with live EJBs, databases, messaging layers and web services/SOAP objects.
Panorama	ISA	Visual environment for Java/C/C++/VB software testing, quality assurance, documentation, and maintenance on Unix/Linux/Windows 95/98/NT/2000
Marathon		Testing framework for GUI applications developed using Java/Swing. Marathon composes of recorder, runner and editor. The testscripts are composed of python code. Marathon focuses on end-user testing. One need not know Java or Swing to record test scripts using Marathon. Marthon is kept intentionally simple. The aim to produce test scripts that are readable by everyone on the project. This includes the developers, testers and the customers. Marathon is flexible and provides facilities to extend the functionality available through python scripts or through writing ones own component resolvers. Marathon captures the semantic actions on components instead of the mouse & keyboard events. This results in scripts that are simple and at a higher granular level. Marathon suites well for applications that mostly depend on forms-paradigm. Open source.
QEngine	AdventNet	Automated testing tool that helps in building high-quality J2EE/J2SE applications with consistent and predictable behavior. It supports server-side unit, functional, performance, stability and regression testing.
qftestJUI	QFS	Tool for the creation, execution and management of tests for Java/Swing applications with a graphical user interface (GUI). It runs on Windows and all major Unix systems with JDKs from Sun, IBM and Blackdown from 1.1 through 1.4.
QStudio Enterprise	QA Systems GmbH	A tool to objectively assess the quality of your Java and .NET software. It works by integrating state-of-the-art 3rd party code analyzers with a sophisticated ISO 9126 Standard software quality model.
TCAT/Java	Software Research	Code coverage analyzer for Java. Features true 32-bit native executables, advanced compiler-based source-language scanning technology, improved runtime support, and full GUI access to a project's function and method digraphs, and current and past-test coverage data at the branch (C1) level.

Product

Vendor

Comments

Abbot

Abbot is a JUnit extension for testing Java GUIs. It supports recording and playback at both low and high levels of abstraction.

AdaptiveCells/J

Open source J2EE benchmarking system. It enables the development of complex artificial J2EE test-beds without requiring a single line of code. Such test-beds can be used for:

Performance testing : if you want to compare the performance of different application servers or if you need to test the functionality/performance of you server in particular contexts.
Middleware infrastructure testing : if you are developing some infrastructure software (eg. server adaptation software or a monitoring solution).
Creating working J2EE applications when you just need to have some running systems (maybe you are learning J2EE or you work on some other aspects of a larger system and you need a working J2EE application for testing everything together).

AgileTest

Polygenix

Mock object implementation for unit testing both Java interfaces and classes. AgileTest works at the bytecode level, enabling non-intrusive dynamic mock generation - without the generation of additional source artefacts or the constraints of the Java proxy mechanism. Free for non-commercial use.

Agitator

Agitar

Helps developers create and maintain unit tests for all their code.

AppPerfect DevSuite

AppPerfect

Suite of products designed to help you design and build high-performance applications during the development phase of the product-development life-cycle. The AppPerfect DevSuite consists of Unit Tester, Code Analyzer J2EE / JVM Profiler, Functional Tester, Load Tester.

Bugkilla

Set of java tools (open source) for the functional test of J2EE Web Applications. Specification and execution of tests will be automated for web front end and business logic layer. One goal is to integrate with existing frameworks and tools. We started with eclipse struts and junit integration.

Cactus

Jakarta Project

A simple test framework for unit testing server-side java code (Servlets, EJBs, Tag Libs, Filters, ...). The intent of Cactus is to lower the cost of writing tests for server-side code. It uses JUnit and extends it.

GJ-Coverage

Tester's Edge

Coverage tool for Java programs of all types: J2ME, J2SE, and J2EE. Features ease of use, low overhead, HTML reports, branch level coverage, test case breakdowns, and more.

GJTester

TreborSoft

Java testing tool that provides a GUI to aid developers in building test cases and test scripts. It allows the testers to accomplish unit and regression test without programming effort. The tool is useful for testing CORBA, RMI and other server technologies as well.

GUIdancer

Bredex GmbH

Automated GUI test tool for programs written with Java and Swing, which requires no programming. Created tests are highly reusable and easily maintainable. Because of the nature of the object recognition, tests are resilient to changes in the application under test.

Additional features include an observation mode, generation of HTML result reports, multi-user capacity, and user-defined hierarchical organization.

Platform independent; runs as standalone application or as an Eclipse plugin.

JCover

Codework

Java code coverage testing

JCover

Man Machine Systems

Java code coverage

Jemmy

An open source library that can be used to create tests for Java GUI applications.

JMeter

Apache

Java desktop application designed to load test functional behavior and measure performance. Open source.

JStyle

Man Machine Systems

Java source code analyzer.

JSystem

Framework for writing and running automated tests, based on JUnit. Its main goal is to support automation of functional/system testing.

jtest

Parasoft

Java class testing

JUnit

junit.org

Regression testing framework for the developer who implements unit tests in Java. Open Source

JVerify

Man Machine Systems

Java unit/integration testing.

KCC

Koalog

Code coverage computation application written in Java. Its main features are: in-process or remote coverage computation, capability of working directly on Java binaries (no recompilation), predefined (XML, HTML, LaTeX, CSV, TEXT) or custom report generation, session merging, portability, ease-of-use, Ant integration.

LISA

iTKO

Uses "inline testing" technology to talk to every component within your infrastructure. One-click wizards enable even non-developers to connect to, analyze and interact with live EJBs, databases, messaging layers and web services/SOAP objects.

Panorama

ISA

Visual environment for Java/C/C++/VB software testing, quality assurance, documentation, and maintenance on Unix/Linux/Windows 95/98/NT/2000

Marathon

Testing framework for GUI applications developed using Java/Swing. Marathon composes of recorder, runner and editor. The testscripts are composed of python code.

Marathon focuses on end-user testing. One need not know Java or Swing to record test scripts using Marathon. Marthon is kept intentionally simple. The aim to produce test scripts that are readable by everyone on the project. This includes the developers, testers and the customers. Marathon is flexible and provides facilities to extend the functionality available through python scripts or through writing ones own component resolvers.

Marathon captures the semantic actions on components instead of the mouse & keyboard events. This results in scripts that are simple and at a higher granular level. Marathon suites well for applications that mostly depend on forms-paradigm. Open source.

QEngine

AdventNet

Automated testing tool that helps in building high-quality J2EE/J2SE applications with consistent and predictable behavior. It supports server-side unit, functional, performance, stability and regression testing.

qftestJUI

QFS

Tool for the creation, execution and management of tests for Java/Swing applications with a graphical user interface (GUI). It runs on Windows and all major Unix systems with JDKs from Sun, IBM and Blackdown from 1.1 through 1.4.

QStudio Enterprise

QA Systems GmbH

A tool to objectively assess the quality of your Java and .NET software. It works by integrating state-of-the-art 3rd party code analyzers with a sophisticated ISO 9126 Standard software quality model.

TCAT/Java

Software Research

Code coverage analyzer for Java. Features true 32-bit native executables, advanced compiler-based source-language scanning technology, improved runtime support, and full GUI access to a project's function and method digraphs, and current and past-test coverage data at the branch (C1) level.

Product	Vendor	Comments
Message Magic	Elvior	Tests embedded software components - processes, subsystems, etc. With MessageMagic, one can test a certain component by simulating its neighbour components. Contains tools for visualization of messages traffic (also ASN.1 messages) between the testable component and their simulated counterparts.
Reactis Tester	Reactive Systems	Generates comprehensive test suites from Simulink/Stateflow models. The test suites exercise large portions of the software under test while avoiding redundancy, thereby maximizing the probability of finding defects within the time available for software testing. The generated tests store model-generated outputs as well as inputs so that Tester-generated test harnesses can automatically check the correctness of source code implementations of models.
TBrun	LDRA	Automated unit testing tool for C/C++, C#, Ada83, Ada95, Java and Visual Basic. TBrun's functionality includes automatic test harness generation, automatic test stub generation and test documentation reporting, automatic generation of test data vectors for a robust test scenario, support of black-box and white-box testing, automatic host/target deployment on any embedded target board and across environments.
Tessy	RazorCat Development	Offers module testing of C code directly on the target system using standard debugging technology. It supports the whole unit testing cycle and works transparently on all supported target debuggers.
TestQuest Pro	TestQuest	Automated testing of any mobile device or application, tests the actual end-user experience, and allows testing of multiple devices, enabling end-to-end testing.
USBTester	Jungo	Software-only USB testing application tool that enables USB developers to quickly validate their USB devices for USB compatibility, Chapter-9 conformance, standard class driver functionality, performance and other relevant parameters.
VectorCAST	Vector Software	Vector Software's VectorCAST is a world-class integrated software test solution that automates the tasks associated with testing software components for C/C++, Embedded C++, and Ada83/Ada95 programs. Automation includes: complete test harness construction, test generation, test execution, code coverage analysis, regression testing and static measures for code complexity and basis path analysis. VectorCAST enables companies to significantly reduce the time, effort and cost to validate safety (DO-178B), mission, and business critical systems. VectorCAST is customized to the target CPU, cross compiler and run-time environments for leading embedded development tools from: Wind River Systems, Green Hills, TI, Rational, Cosmic, Tasking, Mentor Graphics, Analog Devices, and MetaWare.

Product

Vendor

Comments

Message Magic

Elvior

Tests embedded software components - processes, subsystems, etc. With MessageMagic, one can test a certain component by simulating its neighbour components. Contains tools for visualization of messages traffic (also ASN.1 messages) between the testable component and their simulated counterparts.

Reactis Tester

Reactive Systems

Generates comprehensive test suites from Simulink/Stateflow models. The test suites exercise large portions of the software under test while avoiding redundancy, thereby maximizing the probability of finding defects within the time available for software testing. The generated tests store model-generated outputs as well as inputs so that Tester-generated test harnesses can automatically check the correctness of source code implementations of models.

TBrun

LDRA

Automated unit testing tool for C/C++, C#, Ada83, Ada95, Java and Visual Basic. TBrun's functionality includes automatic test harness generation, automatic test stub generation and test documentation reporting, automatic generation of test data vectors for a robust test scenario, support of black-box and white-box testing, automatic host/target deployment on any embedded target board and across environments.

Tessy

RazorCat Development

Offers module testing of C code directly on the target system using standard debugging technology. It supports the whole unit testing cycle and works transparently on all supported target debuggers.

TestQuest Pro

TestQuest

Automated testing of any mobile device or application, tests the actual end-user experience, and allows testing of multiple devices, enabling end-to-end testing.

USBTester

Jungo

Software-only USB testing application tool that enables USB developers to quickly validate their USB devices for USB compatibility, Chapter-9 conformance, standard class driver functionality, performance and other relevant parameters.

VectorCAST

Vector Software

Vector Software's VectorCAST is a world-class integrated software test solution that automates the tasks associated with testing software components for C/C++, Embedded C++, and Ada83/Ada95 programs. Automation includes: complete test harness construction, test generation, test execution, code coverage analysis, regression testing and static measures for code complexity and basis path analysis. VectorCAST enables companies to significantly reduce the time, effort and cost to validate safety (DO-178B), mission, and business critical systems. VectorCAST is customized to the target CPU, cross compiler and run-time environments for leading embedded development tools from: Wind River Systems, Green Hills, TI, Rational, Cosmic, Tasking, Mentor Graphics, Analog Devices, and MetaWare.

Product	Vendor	Comments
AETG	Telcordia	The AETG algorithms employ combinatorial design techniques to create minimal sets of tests that cover all the pairwise interactions among input values.
Data Generator	GSApps	Test data generator, used for creating intelligent data in almost any database or text file. GS DataGenerator enables users to: Complete application testing by inflating a database with meaningful data Create industry-specific data that can be used for a demonstration Protect data privacy by creating a clone of the existing data and masking confidential values Accelerate the development cycle by simplifying testing and prototyping
Data Generator	DTM	Fully customizable utility that generates data, tables (views, procedures etc) for database testing (performance testing, QA testing, load testing or usability testing) purposes.
Datatect	Banner Software	Generate a variety of realistic test data to ASCII flat files or directly to RDBMS including Oracle, Sybase, SQL Server, and Informix.
DTM DB Stress	DTM Soft	Utility for stress testing the server parts of information systems and applications, as well as DBMSs and servers themselves. This tool allows you to create and configure a continuous set of requests to the server of the OLAP (query execution) and OLTP (adding, modifying and deleting data in the database) types.
ER/Datagen	Banner Software	Produces "model-driven" test data in support of Allfusion ERwin and ER/Studio (enterprise editions).
Jenny		JENNY is a free tool similar to AETG and ALLPAIRS. Given many dimensions of a piece of software, with several possible features per dimension, and some restrictions saying which features cannot be used together, JENNY will recommend tests that cover all pairs or triples of allowed feature combinations. For example, given 20 dimensions, 10 features each, JENNY recommends 210 testcases.
Jumpstart	Talking Frog	SQL Server test data generation tool. Lookup values as well as SPs, ad hoc SQL and configurable random values.
SQL DB Validator	C and C Productions	Performs database and data cube verification and validation testing. Tests the content of a database extract against its original data store using the specific SQL syntax for each of the data sources. Tests can be contained in a single file and each test can be configured to connect to different data sources.
SQL Profiler	Embaradero	Identify, isolate, and correct coding mishaps that could lead to costly performance problems.
SQS/Test Professional	SQS	Develops business-oriented requirements for test data - usually for individual functions in an application - so that the necessary data is input (e.g. in a dialog) to ensure that all critical processes in an application are called.
TestIt!	TdgTeam	Test your programs with data that 'feels' right. Simple yet powerful way to generate great quantity of random but real-life looking data, real fast. Generates meaningful people names, companies, streets, cities, IDs, etc. Quick-start tutorial with examples.
TurboData	Canam Software	Test data generator. Generate very realistic test data with foreign keys resolved automatically. Also generates Select, Update and Delete SQL. TurboData includes an SQL Builder, Grid Data Editor and has full Version Control. Free trial version never expires.
utPLSQL		Open source, unit-testing framework for Oracle PL/SQL developers.

Product

Vendor

Comments

AETG

Telcordia

The AETG algorithms employ combinatorial design techniques to create minimal sets of tests that cover all the pairwise interactions among input values.

Data Generator

GSApps

Test data generator, used for creating intelligent data in almost any database or text file. GS DataGenerator enables users to:

Complete application testing by inflating a database with meaningful data
Create industry-specific data that can be used for a demonstration
Protect data privacy by creating a clone of the existing data and masking confidential values
Accelerate the development cycle by simplifying testing and prototyping

Data Generator

DTM

Fully customizable utility that generates data, tables (views, procedures etc) for database testing (performance testing, QA testing, load testing or usability testing) purposes.

Datatect

Banner Software

Generate a variety of realistic test data to ASCII flat files or directly to RDBMS including Oracle, Sybase, SQL Server, and Informix.

DTM DB Stress

DTM Soft

Utility for stress testing the server parts of information systems and applications, as well as DBMSs and servers themselves. This tool allows you to create and configure a continuous set of requests to the server of the OLAP (query execution) and OLTP (adding, modifying and deleting data in the database) types.

ER/Datagen

Banner Software

Produces "model-driven" test data in support of Allfusion ERwin and ER/Studio (enterprise editions).

Jenny

JENNY is a free tool similar to AETG and ALLPAIRS. Given many dimensions of a piece of software, with several possible features per dimension, and some restrictions saying which features cannot be used together, JENNY will recommend tests that cover all pairs or triples of allowed feature combinations. For example, given 20 dimensions, 10 features each, JENNY recommends 210 testcases.

Jumpstart

Talking Frog

SQL Server test data generation tool. Lookup values as well as SPs, ad hoc SQL and configurable random values.

SQL DB Validator

C and C Productions

Performs database and data cube verification and validation testing. Tests the content of a database extract against its original data store using the specific SQL syntax for each of the data sources. Tests can be contained in a single file and each test can be configured to connect to different data sources.

SQL Profiler

Embaradero

Identify, isolate, and correct coding mishaps that could lead to costly performance problems.

SQS/Test Professional

SQS

Develops business-oriented requirements for test data - usually for individual functions in an application - so that the necessary data is input (e.g. in a dialog) to ensure that all critical processes in an application are called.

TestIt!

TdgTeam

Test your programs with data that 'feels' right. Simple yet powerful way to generate great quantity of random but real-life looking data, real fast. Generates meaningful people names, companies, streets, cities, IDs, etc. Quick-start tutorial with examples.

TurboData

Canam Software

Test data generator. Generate very realistic test data with foreign keys resolved automatically. Also generates Select, Update and Delete SQL. TurboData includes an SQL Builder, Grid Data Editor and has full Version Control. Free trial version never expires.

utPLSQL

Open source, unit-testing framework for Oracle PL/SQL developers.

WEB TEST TOOLS

Product	Vendor	Comments
AccVerify/AccRepair	HiSoftware	Verify, Correct, Monitor and Manage your WebSite and Web Based Applications for W3C and Section 508 compliance.
ChangeAgent	Web site repair and maintenance application.
CSE HTML Validator	AI Internet Solutions	HTML, XHTML, CSS, link, spelling, and accessibility checker available. Windows application
Cyber Spyder Link Test	Aman Software	Web site management program to be used for verifying that the URLs on a site are not broken and for analyzing site content. Shareware.
Dead Links	Dead Links	Free online spider based link checker. You only have to input the index page and the spider will crawl your website looking for broken links. Also gives advices for search engines.
HTML Candy	Anetto Software	New-age software for final preparation of HTML pages. It is able to fix up a wide range of problems with HTML syntax, including tags, styles, attributes, attributes' values, deprecated and obsolete elements and attributes.
HTML PowerTools	Talicom	Suite of Windows tools for HTML checking, spelling, etc.
HTML Tidy	W3C	A very nice tool that fixes common errors and pretty prints HTML. Free
InFocus	SSB Technologies	Tool for making Web sites compliant with Section 508 requirements for access by the individuals with disabilities.
Link Checker Pro		Link Checker Pro is a link checking tool for websites and has been tested on sites containing more than 100,000 links. It can export results in a number of formats and provide a graphical view of the website structure.
LinkRunner	Viable Software Alternatives	Dead link detector for Windows
LinkScan	Elsop	Dead link detector
LinkSleuth	Xenu	Dead link detector. Free
Link Validator	REL Software	Link checker and site management tool for webmasters to check links for accuracy and availability, find broken links and links containing syntactic errors. A free "lite" version is also offered.
MOMspider	UCI	Robot link checker. Free
Ramp Ascend		Testing and remediating websites and web based applications for integrated accessibility for Section 508 and WCAG compliance.
Real Validator		HTML syntax checker for Windows
Truwex website QA tool	Erigami	Free online tool for evaluating website accessibility, privacy, performance, quality, broken links. Shows issues on a web page map and in the HTML code.
WebLight	Illumit	Finds broken links and markup problems, and identifies relationships between URL's.
WebQA	Watchfire	Report on over 40 errors including, Search engine optimization, Site inventory, Web accessibility (now includes Bobby reports, Section 508, W3C's WCAG), Corporate standards. Can also manage meta data and create automtated test scripts to test site functionality.

Product

Vendor

Comments

AccVerify/AccRepair

HiSoftware

Verify, Correct, Monitor and Manage your WebSite and Web Based Applications for W3C and Section 508 compliance.

ChangeAgent

Web site repair and maintenance application.

CSE HTML Validator

AI Internet Solutions

HTML, XHTML, CSS, link, spelling, and accessibility checker available. Windows application

Cyber Spyder Link Test

Aman Software

Web site management program to be used for verifying that the URLs on a site are not broken and for analyzing site content. Shareware.

Dead Links

Free online spider based link checker. You only have to input the index page and the spider will crawl your website looking for broken links. Also gives advices for search engines.

HTML Candy

Anetto Software

New-age software for final preparation of HTML pages. It is able to fix up a wide range of problems with HTML syntax, including tags, styles, attributes, attributes' values, deprecated and obsolete elements and attributes.

HTML PowerTools

Talicom

Suite of Windows tools for HTML checking, spelling, etc.

HTML Tidy

W3C

A very nice tool that fixes common errors and pretty prints HTML. Free

InFocus

SSB Technologies

Tool for making Web sites compliant with Section 508 requirements for access by the individuals with disabilities.

Link Checker Pro

Link Checker Pro is a link checking tool for websites and has been tested on sites containing more than 100,000 links. It can export results in a number of formats and provide a graphical view of the website structure.

LinkRunner

Viable Software Alternatives

Dead link detector for Windows

LinkScan

Elsop

Dead link detector

LinkSleuth

Xenu

Dead link detector. Free

Link Validator

REL Software

Link checker and site management tool for webmasters to check links for accuracy and availability, find broken links and links containing syntactic errors. A free "lite" version is also offered.

MOMspider

UCI

Robot link checker. Free

Ramp Ascend

Testing and remediating websites and web based applications for integrated accessibility for Section 508 and WCAG compliance.

Real Validator

HTML syntax checker for Windows

Truwex website QA tool

Erigami

Free online tool for evaluating website accessibility, privacy, performance, quality, broken links. Shows issues on a web page map and in the HTML code.

WebLight

Illumit

Finds broken links and markup problems, and identifies relationships between URL's.

WebQA

Watchfire

Report on over 40 errors including, Search engine optimization, Site inventory, Web accessibility (now includes Bobby reports, Section 508, W3C's WCAG), Corporate standards. Can also manage meta data and create automtated test scripts to test site functionality.

On-line Link and HTML Test Services

Product	Vendor	Comments
Audit Blossom	Blossom Software	Low cost web site audit service.
Bobby	CAST	Help make pages accessible to people with disabilities (WAI and Section 508 compliance). It will also find HTML compatibility problems that prevent pages from displaying correctly on different browsers. Free
CSSCheck	Web Design Group	On-line cascading Style sheet checker. Free
CSS Validation Service	W3C	Free service that checks Cascading Style Sheets (CSS) in (X)HTML documents or standalone for conformance to W3C recommendations
Dr. Watson		HTML checking service. Free.
HTML Validator	WDG	Online service that validates HTML documents. Free
HTML Validation Service	W3C	Online service that checks HTML documents for conformance to W3C HTML and XHTML Recommendations and other HTML standards. Free
Link Alarm	LinkAlarm	Online service for detecting broken links
NetMechanic	Monte Sano Software	Service which searches your site to find broken links, spot bad HTML tags, and rates your server's response time. Commercial but offers free trials
Site Check	UITest.com	Enter a URL and it runs HTML and stylesheet validators, accessibility assessment, link check, load time check, and more using free online web test tools.
SiteTechnician	SiteTechnician	Hosted management solutions. Identifies broken links, analyzes accessibility, reports on search engine optimization, monitors page load times, and helps you manage changes to your site over time.
Validation Spider	Dead Links	Free dead link checker
W3C Link Checker	W3C	Free dead link checker
Weblint Gateway	San Francisco State University	Performs an HTML error scan and creates an HTML listing with error messages in context and tags, URLs, and error messages highlighted in color
Web Page Backward Compatibility Viewer	Delorie Software	On-line utility that serves a page based on how various browsers would render it.
Web Page Purifier	Delorie Software	On-line utility that maps a page to HTML 2.0, HTML 3.2, HTML 4.0, or WebTV 1.1. standards. Free.
XML Validation	Scholarly Technology Group	Validates XML documents. Free.

Product

Vendor

Comments

Audit Blossom

Blossom Software

Low cost web site audit service.

Bobby

CAST

Help make pages accessible to people with disabilities (WAI and Section 508 compliance). It will also find HTML compatibility problems that prevent pages from displaying correctly on different browsers. Free

CSSCheck

Web Design Group

On-line cascading Style sheet checker. Free

CSS Validation Service

W3C

Free service that checks Cascading Style Sheets (CSS) in (X)HTML documents or standalone for conformance to W3C recommendations

Dr. Watson

HTML checking service. Free.

HTML Validator

WDG

Online service that validates HTML documents. Free

HTML Validation Service

W3C

Online service that checks HTML documents for conformance to W3C HTML and XHTML Recommendations and other HTML standards. Free

Link Alarm

LinkAlarm

Online service for detecting broken links

NetMechanic

Monte Sano Software

Service which searches your site to find broken links, spot bad HTML tags, and rates your server's response time. Commercial but offers free trials

Site Check

UITest.com

Enter a URL and it runs HTML and stylesheet validators, accessibility assessment, link check, load time check, and more using free online web test tools.

SiteTechnician

Hosted management solutions. Identifies broken links, analyzes accessibility, reports on search engine optimization, monitors page load times, and helps you manage changes to your site over time.

Validation Spider

Dead Links

Free dead link checker

W3C Link Checker

W3C

Free dead link checker

Weblint Gateway

San Francisco State University

Performs an HTML error scan and creates an HTML listing with error messages in context and tags, URLs, and error messages highlighted in color

Web Page Backward Compatibility Viewer

Delorie Software

On-line utility that serves a page based on how various browsers would render it.

Web Page Purifier

Delorie Software

On-line utility that maps a page to HTML 2.0, HTML 3.2, HTML 4.0, or WebTV 1.1. standards. Free.

XML Validation

Scholarly Technology Group

Validates XML documents. Free.

Product	Vendor	Comments
actiWATE	Actimind	Freeware web application testing environment. actiWATE Framework is written in Java and Java is used for writing tests.
AutoTester One	AutoTester	Functional, regression, and systems integration testing of Windows, Client Server, Host/Legacy, or Web applications.
Badboy	Badboy Software	Provides an enhanced browser interface to aid in building and testing dynamic applications. Combines capture/replay ability with performance testing and diagnostic features to allow developers to closely monitor and understand the interactions between the browser and the server. Free for small users.
Canoo WebTest	Canoo	Canoo WebTest is a free open source tool for testing of web applications. It calls pages and verifies the results, giving comprehensive reports on success and failure.
eValid	Software Research	Test enabled Web Browser. Provides Browser-Based Client-Side WebSite quality checking for Functional Testing, LoadTest, Site Analysis (SiteMap), Performance check and page tuning.
IeUnit		Simple framework to test logical behaviors of web pages. It helps users to create, organize and execute functional unit tests. IeUnit is implemented in JavaScript for the Windows XP platform with Internet Explorer.
Imprimatur		Web application testing tool. The tests are described in a simple XML file. Along with the standard GET and POST actions, Imprimatur handles HTTP sessions and file uploads. The responses can be validated using regular expressions and response code checks. Open source.
Internet Macros	Opus	Low cost automation testing
HTTP::Recorder	Opus	Browser-independent recorder that records interactions with web sites and produces scripts for automated playback. Open source (Perl module)
iRise Application Simulator	iRise	Allows the definition, testing and approval of Web-based business software before developers write the first line of code. Users validate requirements from a visually accurate, functionally rich and interactive simulation of the application that can be modified on the fly during feedback sessions.
ITP	Incanica	Web application testing harness. Lightweight, yet powerful! Test scripts written in XML. No programming required! No changes required to your application. Supports sessions/cookies, POST form data. Command line based for integration into other tools. Open source.
LISA	iTKO	No-code, enterprise-strength automated testing solution for J2EE applications, websites and web services. LISA uses "inline testing" technology to talk to every component within your infrastructure. New one-click wizards enable even non-developers to connect to, analyze and interact with live EJBs, databases, messaging layers and web services/SOAP objects.
MaxQ	Bit Mechanic	MaxQ is a free functional testing tool. It includes an HTTP proxy that records your test script, and a command line utility that can be used to playback tests. The proxy recorder automatically stores variables posted to forms, so you don't have to write that stuff by hand. The paradigm of MaxQ is similar to commercial tools like Astra QuickTest or Empirix e-Test. These products are quite expensive. MaxQ hopes to provide the essential features: HTTP test recording, scripting, and playback without the huge cost. Source code to MaxQ is provided as part of the distribution.
Netvantage Functional Tester	Netvantage Technologies	Functional and regression automated web testing tool.
PesterCat	PesterCat	Web testing tool that was designed to perform functional testing for web applications. PesterCat features an integrated proxy recorder that enables you to record scripts using your favorite web browser. PesterCat runs on Linux, Mac OSX, and Windows.
QA Wizard	Seapine	Automated functional testing tool for Web-based apps. Automated scripting allows the creation of more scripts in less time. Advanced object binding greatly reduces script changes when Web-based apps change. FREE QA Wizard Demo and Test Script!
Ranorex	Ranorex Software	Windows GUI test and automation framework for C++, Python and for the .Net languages
Rational Robot	Rational Software	Automated functional, regression, and smoke tests for e-applications
Sahi		Automation and testing tool for web applications, with the facility to record and playback scripts. Developed in java and javascript, this tool uses simple javascript to execute events on the browser. Features include, in-browser controls, text based scripts, ant support for playback of suites of tests, and multi threaded playback. It supports HTTP and HTTPS.
SAMIE		Simple Automated Module For Internet Explorer. Perl module (SAM.pm) that allows a user to automate Internet Explorer. This free tool is designed for quality assurance engineers that need to run tests for their browser applications. (Windows only)
Selenium	ThoughtWorks	Selenium is a test tool for web applications. Selenium tests run directly in a browser, just as real users do. And they run in Internet Explorer, Mozilla and Firefox on Windows, Linux, and Macintosh.
SoapTest	Parasoft	Automated tool for testing Web services. SOAPtest helps you to prevent errors by performing server functional testing, load testing, and client testing with just the click of a button. SOAPtest facilitates server functional testing by automatically creating a test suite from a WSDL document that tests every operation associated with that document. The same test suite used for server functional testing can also be used as a load test suite that not only monitors the servers response rate with the specified number and mixture of simultaneous requests, but also verifies whether the test loads cause functionality problems. You can also have SOAPtest emulate a server and verify that the client sends appropriate requests to the appropriate services, and that the client handles the services responses as expected.
soapui	eviware	Desktop application for inspecting, invoking and testing (functional and load) of web services over HTTP. It is mainly aimed at developers/testers providing and/or consuming web services (java, .net, etc). Functional and Load-Testing can be done both interactively in soapui or within a automated build/integration process using the soapui command-line tools. soapui currently requires java 1.5 and is licensed under the LGPL license.
Solex	NEOMAlogic	Solex is an open source Web application testing tool built as a plug-in for the Eclipse IDE. It provides functions to record a client session, adjust it according to various parameters and replay it later typically in order to ensure non regression of a Web application's behaviour.
Squish	froglogic GmbH	Professional test tool which allows to create, debug and run automated GUI tests for Qt, Tk, XView and HTML/Web applications.

Product

Vendor

Comments

actiWATE

Actimind

Freeware web application testing environment. actiWATE Framework is written in Java and Java is used for writing tests.

AutoTester One

AutoTester

Functional, regression, and systems integration testing of Windows, Client Server, Host/Legacy, or Web applications.

Badboy

Badboy Software

Provides an enhanced browser interface to aid in building and testing dynamic applications. Combines capture/replay ability with performance testing and diagnostic features to allow developers to closely monitor and understand the interactions between the browser and the server. Free for small users.

Canoo WebTest

Canoo

Canoo WebTest is a free open source tool for testing of web applications. It calls pages and verifies the results, giving comprehensive reports on success and failure.

eValid

Software Research

Test enabled Web Browser. Provides Browser-Based Client-Side WebSite quality checking for Functional Testing, LoadTest, Site Analysis (SiteMap), Performance check and page tuning.

IeUnit

Simple framework to test logical behaviors of web pages. It helps users to create, organize and execute functional unit tests. IeUnit is implemented in JavaScript for the Windows XP platform with Internet Explorer.

Imprimatur

Web application testing tool. The tests are described in a simple XML file. Along with the standard GET and POST actions, Imprimatur handles HTTP sessions and file uploads. The responses can be validated using regular expressions and response code checks. Open source.

Internet Macros

Opus

Low cost automation testing

HTTP::Recorder

Opus

Browser-independent recorder that records interactions with web sites and produces scripts for automated playback. Open source (Perl module)

iRise Application Simulator

iRise

Allows the definition, testing and approval of Web-based business software before developers write the first line of code. Users validate requirements from a visually accurate, functionally rich and interactive simulation of the application that can be modified on the fly during feedback sessions.

ITP

Incanica

Web application testing harness. Lightweight, yet powerful! Test scripts written in XML. No programming required! No changes required to your application. Supports sessions/cookies, POST form data. Command line based for integration into other tools. Open source.

LISA

iTKO

No-code, enterprise-strength automated testing solution for J2EE applications, websites and web services. LISA uses "inline testing" technology to talk to every component within your infrastructure. New one-click wizards enable even non-developers to connect to, analyze and interact with live EJBs, databases, messaging layers and web services/SOAP objects.

MaxQ

Bit Mechanic

MaxQ is a free functional testing tool. It includes an HTTP proxy that records your test script, and a command line utility that can be used to playback tests. The proxy recorder automatically stores variables posted to forms, so you don't have to write that stuff by hand. The paradigm of MaxQ is similar to commercial tools like Astra QuickTest or Empirix e-Test. These products are quite expensive. MaxQ hopes to provide the essential features: HTTP test recording, scripting, and playback without the huge cost. Source code to MaxQ is provided as part of the distribution.

Netvantage Functional Tester

Netvantage Technologies

Functional and regression automated web testing tool.

PesterCat

Web testing tool that was designed to perform functional testing for web applications. PesterCat features an integrated proxy recorder that enables you to record scripts using your favorite web browser. PesterCat runs on Linux, Mac OSX, and Windows.

QA Wizard

Seapine

Automated functional testing tool for Web-based apps. Automated scripting allows the creation of more scripts in less time. Advanced object binding greatly reduces script changes when Web-based apps change. FREE QA Wizard Demo and Test Script!

Ranorex

Ranorex Software

Windows GUI test and automation framework for C++, Python and for the .Net languages

Rational Robot

Rational Software

Automated functional, regression, and smoke tests for e-applications

Sahi

Automation and testing tool for web applications, with the facility to record and playback scripts. Developed in java and javascript, this tool uses simple javascript to execute events on the browser. Features include, in-browser controls, text based scripts, ant support for playback of suites of tests, and multi threaded playback. It supports HTTP and HTTPS.

SAMIE

Simple Automated Module For Internet Explorer. Perl module (SAM.pm) that allows a user to automate Internet Explorer. This free tool is designed for quality assurance engineers that need to run tests for their browser applications. (Windows only)

Selenium

ThoughtWorks

Selenium is a test tool for web applications. Selenium tests run directly in a browser, just as real users do. And they run in Internet Explorer, Mozilla and Firefox on Windows, Linux, and Macintosh.

SoapTest

Parasoft

Automated tool for testing Web services. SOAPtest helps you to prevent errors by performing server functional testing, load testing, and client testing with just the click of a button. SOAPtest facilitates server functional testing by automatically creating a test suite from a WSDL document that tests every operation associated with that document. The same test suite used for server functional testing can also be used as a load test suite that not only monitors the servers response rate with the specified number and mixture of simultaneous requests, but also verifies whether the test loads cause functionality problems. You can also have SOAPtest emulate a server and verify that the client sends appropriate requests to the appropriate services, and that the client handles the services responses as expected.

soapui

eviware

Desktop application for inspecting, invoking and testing (functional and load) of web services over HTTP. It is mainly aimed at developers/testers providing and/or consuming web services (java, .net, etc). Functional and Load-Testing can be done both interactively in soapui or within a automated build/integration process using the soapui command-line tools. soapui currently requires java 1.5 and is licensed under the LGPL license.

Solex

NEOMAlogic

Solex is an open source Web application testing tool built as a plug-in for the Eclipse IDE. It provides functions to record a client session, adjust it according to various parameters and replay it later typically in order to ensure non regression of a Web application's behaviour.

Squish

froglogic GmbH

Professional test tool which allows to create, debug and run automated GUI tests for Qt, Tk, XView and HTML/Web applications.

General

General - Tools

Publications

Testing Related Web Sites

White Papers - Test Automation

White Papers - Manual Testing

White Papers - Agile Testing

White Papers - Requirements Management

White Papers - Other

QA Tester Certifications

APPLICATION TEST TOOLS

Source Test Tools

Functional Test Tools

Performance Test Tools

Java Test Tools

Embedded Test Tools

Database Test Tools

WEB TEST TOOLS

Link and HTML Test Tools

On-line Link and HTML Test Services

Functional Test Tools