Mobile Development

Mobile Testing: Unit, UI, and E2E

Shopify's mobile team publishes that a 1% increase in checkout crash rate costs millions in lost revenue during peak sales. Instagram's iOS team runs 80,000+ unit tests on every PR, completing in under 4 minutes on CI. Facebook's mobile infrastructure team developed snapshot testing internally before open-sourcing it because pixel regressions in their design system were reaching production undetected. Mobile testing is not a bureaucratic checkbox - it is the difference between a 3am pager alert and a quiet night.

  • **Airbnb's Paparazzi** (open-sourced 2022) runs Android screenshot tests without an emulator, cutting their visual regression CI time from 45 minutes (emulator-based) to under 5 minutes - enabling snapshot testing on every PR rather than only on release branches.
  • **Wix's Detox** was built after their React Native app had persistent E2E test flakiness with Appium (30-40% flake rate on CI). After migrating to Detox's gray-box synchronization, flake rate dropped below 2%, making E2E tests reliable enough to block merges.
  • **Spotify's iOS team** maintains a test pyramid of 70% unit, 20% integration, 10% UI tests. Their CI runs unit tests in 3 minutes and full test suite in 25 minutes for an app with 400+ screens - achieved through strict component isolation with protocols and dependency injection.

XCTest and iOS Unit Testing

XCTest is Apple's native testing framework, integrated into Xcode. Unit tests subclass XCTestCase and run in a separate process from the app. Each test method name starts with 'test' and can assert values with XCTAssert*, XCTEqual, XCTThrowsError, and expectation-based async assertions. XCTest measures both correctness and performance via measureBlock, which runs a closure 10 times and reports mean, deviation, and baseline regressions.

0

1

Sign In

Dependency injection via protocols is the iOS testing pattern: CartService takes a CartStorage protocol, MockCartStorage implements it. This avoids touching the real file system or network in unit tests. Swift's @testable import makes internal-visibility types accessible to test targets without exposing them publicly.

Why does iOS test code use @testable import instead of a regular import?

Espresso and Android UI Testing

Espresso is Android's official UI testing framework developed at Google. It runs on-device in the same process as the app and uses an IdlingResource mechanism to automatically wait for async operations (network calls, animations, background threads) before performing actions or assertions. This eliminates the flakiness of sleep()-based tests. Espresso tests follow a ViewMatchers → ViewActions → ViewAssertions pattern.

Espresso's synchronization with the main thread uses an internal queue inspector - it checks that the main looper's message queue is idle and no AsyncTasks are running. For Retrofit/OkHttp, registering an OkHttpIdlingResource makes Espresso wait for all HTTP calls to complete before asserting, preventing the most common source of UI test flakiness.

What problem does Espresso's IdlingResource mechanism solve in UI testing?

Snapshot Testing

Snapshot testing captures a rendered component's output (view hierarchy or pixel image) to a reference file, then compares future renders against it. Any visual change - intentional or accidental - fails the test. iOS uses Point-Free's swift-snapshot-testing library (or iOSSnapshotTestCase), Android uses Paparazzi (Airbnb, 2022) for off-device screenshot testing without an emulator. Paparazzi renders views using LayoutInflater + a custom Canvas, running on the JVM 10x faster than emulator tests.

Snapshot tests are brittle by design: font rendering differences across OS versions, anti-aliasing changes, and pixel-rounding variations cause false failures. Best practice is to snapshot only stable design-system components (buttons, cards, typography), not full screens with dynamic content. Storing snapshot files in git adds PR diff visibility - reviewers see exactly what changed visually.

Why does Paparazzi run Android snapshot tests faster than emulator-based UI tests?

End-to-End Testing with Detox

Detox (Wix, 2017) is a gray-box E2E testing framework for React Native and native mobile apps. Unlike black-box E2E tools (Appium), Detox instruments both the app and the test runner - it can observe React Native's bridge activity and JavaScript event loop to know when the app is idle, eliminating arbitrary wait() calls. Tests run on real simulators/emulators and execute user-level actions: tap, scroll, typeText, swipe.

The mobile testing pyramid: many unit tests (ms each, no simulator), fewer snapshot tests (ms, JVM), even fewer Espresso/XCTest UI tests (seconds, simulator), few Detox E2E tests (minutes, simulator). Detox E2E tests cost 2-5 min per test on CI; running all E2E tests on every PR is expensive. The standard pattern is unit + snapshot on every commit, UI tests on every PR, E2E on merge to main.

E2E tests are the most valuable tests because they test the whole app

E2E tests are the most expensive and slowest tests; they should verify critical user journeys only (login, checkout, core feature), while unit tests verify business logic and snapshot tests catch visual regressions

A Detox E2E test takes 2-5 minutes and requires a running simulator; the same logic verified by a unit test takes milliseconds. Inverting the test pyramid (few unit tests, many E2E) leads to CI pipelines that take 2+ hours and developers who skip running tests locally

What makes Detox 'gray-box' testing and why does it reduce test flakiness compared to black-box tools like Appium?

Key Ideas

  • **XCTest (iOS) and Espresso (Android)** are the native unit/UI testing frameworks. XCTest uses async/await and measureBlock; Espresso uses IdlingResource to synchronize with async operations, eliminating sleep()-based flakiness.
  • **Snapshot testing** captures rendered component output and fails on any visual change. Paparazzi (Android) and swift-snapshot-testing (iOS) run on the JVM without a simulator, making them fast enough for every PR.
  • **Detox** is gray-box E2E: it instruments React Native's bridge and JS event loop to know when the app is truly idle, reducing flake rates from 30-40% (Appium) to under 2%. E2E tests should cover critical user journeys only, not replace the unit/snapshot layers.

Related Topics

Mobile testing connects to CI/CD and architecture topics:

  • Mobile CI/CD — Test suites only provide value when run automatically; mobile CI/CD pipelines with Fastlane and GitHub Actions are the infrastructure that runs XCTest, Espresso, and Detox on every commit
  • Mobile Architecture at Scale — Testable mobile architecture depends on dependency injection and modularization - the same patterns that make XCTest and Espresso fast also enable feature-team independence

Вопросы для размышления

  • A React Native app has a checkout flow with 6 screens. Design the testing strategy: which parts get unit tests, which get snapshot tests, which get Detox E2E tests, and what are the tradeoffs at each layer?
  • An iOS UI test that was passing for 6 months starts failing intermittently (50% pass rate). What are the most likely causes and how would the diagnosis process proceed?
  • Paparazzi tests on Android fail on CI but pass locally. What environment differences between a developer's machine and a Linux CI runner could cause snapshot pixel mismatches?

Связанные уроки

  • stat-05-hypothesis
Mobile Testing: Unit, UI, and E2E