Testing guidelines

Preparing environment

Install phpunit

To run Magento 2 tests you need to have latest phpunit library.

Simplest way to install latest version looks as follows:

wget https://phar.phpunit.de/phpunit-5.2.9.phar
chmod +x phpunit.phar
sudo mv phpunit.phar /usr/local/bin/phpunit
phpunit --version

This will output:

PHPUnit 5.2.9 by Sebastian Bergmann and contributors.

Create additional database

You need to create additional empty database for integration testing purpose. 

When it's done copy dev/tests/integration/etc/install-config-mysql.php.dist file to dev/tests/integration/etc/install-config-mysql.php and fill it with database access details.

Running tests

Unit tests

To run unit tests execute following command:

phpunit -c dev/tests/unit/phpunit.xml.dist app/code/RocketWeb/ShoppingFeeds/Test/Unit

Expected output:

PHPUnit 5.2.9 by Sebastian Bergmann and contributors.
..................                                                18 / 18 (100%)
Time: 577 ms, Memory: 19.00Mb
OK (18 tests, 27 assertions)

Integration tests

To run integration tests execute following commands:

cd dev/tests/integration
phpunit -c phpunit.xml.dist ../../../app/code/RocketWeb/ShoppingFeeds/Test/Integration

Expected output:

PHPUnit 5.2.9 by Sebastian Bergmann and contributors.
..                                                                  2 / 2 (100%)
Time: 1.2 minutes, Memory: 426.00Mb
OK (2 tests, 2 assertions)
=== Memory Usage System Stats ===
Memory usage (OS):      436.34M (102.49% of 425.75M reported by PHP)
Estimated memory leak:  10.59M (2.43% of used memory)

Static tests

To run static tests execute following commands:

cd dev/tests/static/
phpunit

What to test

Unit tests

Cover following:

• all methods (100% lines of code) in a class
• if possible cover 100% of lines and argument combinations.

Use mocks to make sure only single class is being tested at a time.

Integration tests

Cover following:

• model parts which interacts with database indirectly (eg. model saves, loads, collections loads)
• other classes than model if it's not possible to test them using unit tests
• test if feeds are properly generated for different product types and their configuration 

Static tests

Static tests shipped with Magento cover all code, including custom extension code.

There is no need to write additional static tests.

Existing tests should be run to make sure custom code follows current Magento 2 standards.

Functional tests

Currently there is no plan to implement functional tests.

Code coverage

Unit and integration tests run separately so we can generate two separate code coverage reports. However, it's possible to generate merged code coverage report.

To have code coverage generated you need to have xdebug installed and create custom phpunit.xml files based on phpunit.xml.dist for both integration and unit tests. Follow these steps:

1. Prepare configuration file for unit tests
   1. Copy dev/tests/unit/phpunit.xml.dist to dev/tests/unit/phpunit.xml
   2. Uncomment out coverage-html node and change target attribute to ../../../var/unit-test-reports/coverage
   3. Uncomment out coverage-php node and change target attribute to ../../../var/unit-test-reports/coverage.cov
2. Prepare configuration file for integration tests
   1. Copy dev/tests/integration/phpunit.xml.dist to dev/tests/integration/phpunit.xml
   2. Copy logging node from unit tests configuration file to integration tests configuration file
   3. Adjusts paths to use ../../../var/integration-test-reports/coverage and ../../../var/integration-test-reports/coverage.cov
   4. In whitelist node change ../../../app/code/Magento directory to ../../../app/code/*

Then to generate code coverage reports, follow these steps:

1. Generate unit tests code coverage in cov format:
   phpunit -c dev/tests/unit/phpunit.xml app/code/RocketWeb/ShoppingFeeds/Test/Unit
2. Generate integration tests code coverage in cov format:
   cd dev/tests/integration
   phpunit -c phpunit.xml ../../../app/code/RocketWeb/ShoppingFeeds/Test/Integration
3. Merge results and generate merged html:
   mkdir var/test-reports
   cp var/integration-test-reports/coverage.cov var/test-reports/coverage-integration.cov
   cp var/unit-test-reports/coverage.cov var/test-reports/coverage-unit.cov
   phpcov merge --html var/test-reports/ var/test-reports/

Resources

• http://devdocs.magento.com/guides/v2.0/config-guide/cli/config-cli-subcommands-test.html
• https://github.com/magento/magento2/wiki/Magento-Automated-Testing-Standard
• https://leanpub.com/b/grumpy#grumpy-phpunit
• https://leanpub.com/grumpy-testing