Refactored TestExpectations out of Telemetry and into content/test/gpu.
commitdf4f51769e3c1eb43734db6f5360e0c4908e2c5f
authorkbr <kbr@chromium.org>
Thu, 16 Jul 2015 17:47:02 +0000 (16 10:47 -0700)
committerCommit bot <commit-bot@chromium.org>
Thu, 16 Jul 2015 17:47:58 +0000 (16 17:47 +0000)
tree1ef8ba319a1e15487a11f6e75f673fa0cfb47673
parenta71701a195fb434b02cacf095b28219858daa15e
Refactored TestExpectations out of Telemetry and into content/test/gpu.

(Re-land #2 with fixes for handling crashed renderers and other bug fixes)

(Original CL: https://codereview.chromium.org/1224233005/ )
(Re-land #1: https://codereview.chromium.org/1231863008/ )

The GPU tests are the primary user of Telemetry's TestExpectations,
and it turns out they can be implemented entirely in the code using
Telemetry, rather than Telemetry itself. For this reason, move them
into content/test/gpu.

Refactored the GPU tests to all derive from a common set of base
classes implementing support for test expectations. Verified that
skipping tests, expected failures, and flaky test handling all work.
More tests of test expectations are needed, and will be added in later
CLs; this one is too large already, but can't be broken up further.

Fixed up two WebGL perf tests that were relying on test expectations
to skip the pages on specific GPUs, and added a hook to Telemetry's
browser_info to enable this.

This CL loses one valuable test,
testHandlingOfCrashedTabWithExpectedFailure, from
page_run_end_to_end_unittest.py. It will be reintroduced in a later CL.

A subsequent CL will redesign TestExpectations. This CL is intended to
be a refactoring with no behavioral changes.

Ran telemetry_unittests, telemetry_gpu_unittests,
telemetry_perf_unittests, and all GPU tests locally to verify.

BUG=495870
CQ_EXTRA_TRYBOTS=tryserver.chromium.perf:linux_perf_bisect;tryserver.chromium.perf:mac_perf_bisect;tryserver.chromium.perf:win_perf_bisect
TBR=bajones@chromium.org,nednguyen@google.com,dtu@chromium.org,zmo@chromium.org

Review URL: https://codereview.chromium.org/1233133003

Cr-Commit-Position: refs/heads/master@{#339069}
43 files changed:
content/test/gpu/gpu_tests/cloud_storage_test_base.py
content/test/gpu/gpu_tests/context_lost.py
content/test/gpu/gpu_tests/exception_formatter.py [new file with mode: 0644]
content/test/gpu/gpu_tests/gpu_process.py
content/test/gpu/gpu_tests/gpu_rasterization.py
content/test/gpu/gpu_tests/gpu_test_base.py [new file with mode: 0644]
content/test/gpu/gpu_tests/gpu_test_base_unittest.py [new file with mode: 0644]
content/test/gpu/gpu_tests/gpu_test_expectations.py
content/test/gpu/gpu_tests/gpu_test_expectations_unittest.py
content/test/gpu/gpu_tests/hardware_accelerated_feature.py
content/test/gpu/gpu_tests/maps.py
content/test/gpu/gpu_tests/memory_test.py
content/test/gpu/gpu_tests/pixel.py
content/test/gpu/gpu_tests/screenshot_sync.py
content/test/gpu/gpu_tests/test_expectations.py [moved from tools/telemetry/telemetry/page/test_expectations.py with 90% similarity]
content/test/gpu/gpu_tests/test_expectations_unittest.py [moved from tools/telemetry/telemetry/page/test_expectations_unittest.py with 97% similarity]
content/test/gpu/gpu_tests/trace_test.py
content/test/gpu/gpu_tests/webgl_conformance.py
content/test/gpu/page_sets/gpu_process_tests.py
content/test/gpu/page_sets/gpu_rasterization_tests.py
content/test/gpu/page_sets/memory_tests.py
content/test/gpu/page_sets/pixel_tests.py
tools/perf/benchmarks/blink_perf.py
tools/perf/benchmarks/smoothness.py
tools/perf/benchmarks/webgl_expectations.py [deleted file]
tools/perf/page_sets/android_screen_restoration_shared_state.py
tools/perf/page_sets/diagonal_scrolling_supported_shared_state.py
tools/perf/page_sets/fling_gesture_supported_shared_state.py
tools/perf/page_sets/maps.py
tools/perf/page_sets/tough_webgl_cases.py
tools/perf/page_sets/webgl_supported_shared_state.py
tools/telemetry/telemetry/android/shared_android_state.py
tools/telemetry/telemetry/benchmark.py
tools/telemetry/telemetry/benchmark_unittest.py
tools/telemetry/telemetry/internal/browser/browser_info.py
tools/telemetry/telemetry/internal/story_runner.py
tools/telemetry/telemetry/internal/story_runner_unittest.py
tools/telemetry/telemetry/page/page_run_end_to_end_unittest.py
tools/telemetry/telemetry/page/page_test.py
tools/telemetry/telemetry/page/shared_page_state.py
tools/telemetry/telemetry/record_wpr.py
tools/telemetry/telemetry/story/shared_state.py
tools/telemetry/telemetry/testing/page_test_test_case.py