Writing tests¶
Tests are defined within a test case file. e.g login_test.py
.
It is recommended to store these files under a ui_tests
folder:
- ui_tests
- boarding_test.py
- login_test.py
- mainpage_test.py
- feature_test.py
- other_feature_test.py
Test cases inherit from BaseTestCase class and utilize features such as setup, teardown and assertions.
Defining a test¶
Example:
from magento.base import BaseTestCase
class ExampleTest(BaseTestCase):
"""
Tests example functionality
"""
def test_example_1(self):
...
def test_example_2(self):
...
def test_example_3(self):
...
Tests are defined as functions prefixed with test_ and usually include some kind of assertion.
Example:
def test_example(self):
# scroll fast to bottom
self.magneto(scrollable=True).fling()
# get element by id
el = self.magneto(text=ids.foo)
# assert that the element exists in the current view
Assert.true(el.exists)
Results¶
Test results are available via Magneto logs:
ui_tests/ftu_test.py .s
ui_tests/cards_test.py F
ui_tests/discovery_test.py .
ui_tests/folders_test.py ..s..
ui_tests/homescreen_test.py .
ui_tests/magneto_test.py ....
ui_tests/search_test.py .F....
In the log above, each line represents a test case and its tests results.
.
= passeds
= skippedF
= failed
A summary at the end:
=============== 2 failed, 16 passed, 2 skipped in 348.66 seconds ===============
Failed tests log more information, pointing to where an error or assertion failure occurred:
self = <magneto.test.cards_test.CardsTestCase testMethod=test_browse_news_cards>
def test_browse_news_cards(self):
"""
Test browse through cards in News smart folder
"""
cards = self.magneto(resourceId=ids.card)
folder = Folder(folders.news)
folder.click()
Assert.true(cards.exists)
folder.menu.click()
self.magneto(text=names.hide_cards).click()
self.magneto.press.home()
folder.click()
Assert.false(cards.exists)
folder.menu.click()
self.magneto(text=names.show_cards).click()
self.magneto.press.home()
folder.click()
> Assert.true(cards.exists)
E AssertionError: False is not true
magneto/test/cards_test.py:29: AssertionError
Even more data about failed tests¶
Magneto can be instructed to capture adb logcat logs, element hierarchy and screen image at the moment the fail was determined.
When all test runs are over, these files are made available in the dedicated folder (usually tmp/magneto_test_data
unless specified differently with the --magneto_failed_data_dir
parameter) as one bundled file.
This file could be made available in CI systems as a build artifact.
Example:
- Nexus4-01acd7ef4c3d12d4 4.53.24 PM
- 7-test_example_1-201503081428-1.video.mp4
- 7-test_example_1-201503081428.hierarchy.uix
- 7-test_example_1-201503081428.logcat.log
- 7-test_example_1-201503081428.screenshot.png