Page History

...

On Testing

Wiki MarkupDue to the extension of the subject a new page has been created with more details. Please check \[[Testing|Testing]\] for more information on unit Testing for more information on unit testing

Existing approaches

So far there's been some approaches to the problem. As far as we know, the following:

• Elliot Metsger created some unit testing framework as shown at http://presentations.dlpe.gatech.edu/proed/or09/or09_052009_4/index.html
• Scott Phillips did a set of functional tests for DSpace 1.5 as seen at http://scott.phillips.name/2010/04/dspace-functional-tests/
• Gareth Waller has built a set of Selenium tests as commented recently in the developer's list
• Aaron Zeckoski's effort added JUnit-based testing into DSpace Services e.g. http://scm.dspace.org/svn/repo/modules/dspace-services/tags/dspace-services-2.0.1/impl/src/test/java/org/dspace/test/DSpaceAbstractKernelTest.java

Next there is a table that compares these and some other approaches to the problem:

...

Both would benefit DSpace, as unit testing would ensure code quality while functional testing would ensure the application behaves as expected with standard usage.

Proposals and Scope

The proposals original proposal for the project arewere to:

• Create a framework for Functional Tests, Integration Tests, Unit Tests and Performance Tests that run automatically on build (when possible)
  • I will create Create all the background require scaffolding required (mockups, database connections, configs, etc) for the tests to be runI will integrate Cobertura to create reports on the testing coverage

• Integrate with a Continuous Integration server
• (Optional but recommended) Integrate with a code quality management tool

  ...

  • • I will integrate Cobertura to create reports on the testing coverage

  The main proposed scope of the project is was to apply the proposals tests to the API core. Once done, work would be extended in the remaining time to Manakin and then other subprojects (Sword, LNI, OIA). JSPUI will be left for last, as is an interface that will disappear in the future.

  Project Plan

  A work plan has been decided in an IRC meeting on May 26th, 2010 (log: [http://duraspace.org/irclogs/index.php?date=2010-05-26|http://duraspace.org/irclogs/index.php?date=2010-05-26]). The work has been divided in 3 parts:

  1. Single class Unit Tests. Ideal candidates would be classes such a DCDate. These can be easily tested in isolation. This sort of test, along with an appropriate way for maven to execute them will give the project, and DSpace, and early win. Hopefully it will encourage further tests to be created by other developers, and we could move towards an expectation of new code requiring tests like this before they are committed.
  2. Integration Testing, which requires a database to be present. This would ideally work with an embedded database such as hqsql or derby. We need to think about this a little more to decide how best to do this, and how it would interact with the new services architecture.
  3. Functional Testing using a tool such as selenium.

  It's been decided they will be done in the same order. The aim is not to do tests for all the existing classes, but to create an infrastructure the community can benefit from and can expand with its own tests.

  The first two points will be finished before the mid-term evaluation. The third point will be done after that.

  Integration with a continuous integration environment will be possible out-of-the-box,a s we will use Maven for points 1 and 2 and Ant for point 3. The capability of running the functional tests in the continuous integration server depends on configuration as it will require running ant tasks created after the generation of code.

  Integration with Cobertura and similar is left to the usage of a quality management tool, as the current state of the Maven plugins creates issues when trying to merge reports from different sub-projects.

  What has been Implemented

  At the end of the project the following components have been implemented:

  • Structure to allow the implementation of unit tests
  • Structure to allow the implementation of integration tests
    • Integration tests allows for performance checks
  • Unit tests for package org.dspace.content, usable as example
  • Two Integration tests usable as example to create new ones
  • Added some static analysis tools (like FindBugs) to the build process

  No integration with a Continuous Integration environment or a Quality Management tool has been done. This is left for a later phase after the end of GSoC

  The Unit Tests created (due to complications during development) have been restricted to those in package org.dspace.content.

  Technical description

  A tecnical description of the tools used and steps followed is available at GSOC 2010 Unit Tests - Technical documentation

  Considerations

  DSpace code suffers from testability issues. In the long term it might be advisable to refactor it. Refactoring is dangerous right now as we don't have any test and we risk introducing new bugs to the code base, so it should be avoided until we have enough tests working that makes us confidant we can take that road.

  Project Description

  At first three basic objects will be created: an Abstract Functional Test, an Abstract Integration Test, and if needed an Abstract Unit Tests. These basic abstract tests would be able to preform the basic setup and teardown operations to get those resources needed: database, etc. The structure will include an embedded webserver, database and temporary file system.

  Performance Testing will be achieved through the use of ContiPerf, a tool that reuses JUnit tests as performance tests.

  Project Plan

  A work plan has to established that will outline:

  • Work to be done by mid-term evaluation
  • Work to be done by end of GSOC
  • Documentation to be generated

  Tools

  ...

  • Wiki Markup
    SONAR \[[http://www.sonarsource.org/|http://www.sonarsource.org/] \] will be the code quality management tool

  ...

  • Embedded webserver will be Jetty

  • Wiki Markup
    Embedded database will be HyperSQL \[[http://hsqldb.org/|http://hsqldb.org/] \]

  ...

  • JMock will be used as mocking framework
  • HTMLUnit may be used if required

  Source Code

  The code is held in an SVN branch: http://scm.dspace.org/svn/repo/sandbox/gsoc/2010/testing/

  ...

  Thanks

  This page has been created with help from Stuart Lewis, Scott Phillips and Gareth Waller. I want to thank them all for their comments. Some information has been taken from Wikipedia to make the text more complete. I'm to blame for errors in the text.

  ...