My name is Dmitry, I work as a tester in a company . Most recently, I finished dealing with a relatively fresh feature from - namely, with instrumental testing of iOS applications using the native testing framework XCUITest.
Before that, I already tried Firebase Test Lab for Android and I really liked everything, so I decided to try to put the project's iOS test infrastructure on the same rails. I had to google a lot and not everything worked out the first time, so I decided to write a tutorial article for those who still have to do it.
So, if you have UI tests on an iOS project, you can already try running them on real devices today, kindly provided by the Good Corporation. Interested - welcome under cat.
In the story, I decided to build on some source data - a private repository on GitHub and the CircleCI build system. Application name is AmazingApp, bundleID is com.company.amazingapp. I quote these data immediately, to reduce subsequent confusion.
If you implemented certain solutions in your project differently, share your experience in the comments.
1. The tests themselves
Create a new project branch for UI tests:
$ git checkout develop
$ git pull
$ git checkout -b “feature/add-ui-tests”
Let's open the project in XCode and create a new Target with UI tests [XCode -> File -> New -> Target -> iOS Testing Bundle], give it the telling name AmazingAppUITests.
Go to the Build Phases section of the created Target and check for Target Dependencies - AmazingApp, in Compile Sources - AmazingAppUITests.swift.
It is good practice to separate the various build options into separate Schemes. We create a scheme for our UI tests [XCode -> Product -> Scheme -> New Scheme] and give it the same name: AmazingAppUITests.
Build of the created scheme should include the Target of the main application - AmazingApp and Target UI tests - AmazingAppUITests - see screenshot
Next, we create a new build configuration for the UI tests. In Xcode, click on the project file, go to the Info section. Click on the “+” and create a new configuration, for example XCtest. We will need this in the future in order to avoid dancing with a tambourine when it comes to code signing.
There are at least three Targets in your project: the main application, unit tests (there are some, right?) and the Target UI of tests we created.
Go to Target AmazingApp, Build Settings tab, Code Signing Identity section. For the XCtest configuration, select iOS Developer. In the Code Signing Style section, select Manual. We have not yet generated a provisioning profile, but we will definitely return to it a little later.
For Target AmazingAppUITests, we do the same, but enter com.company.amazingappuitests in the Product Bundle Identifier column.
2. Setting up a project in the Apple Developer Program
We go to the Apple Developer Program page, go to the Certificates, Identifiers & Profiles section and then to the App IDs column of the Identifiers item. Create a new App ID named AmazingAppUITests and bundleID com.company.amazingappuitests.
Now we have the opportunity to sign our tests with a separate certificate, but ... The build procedure for testing involves building the application itself and building the test runner. Accordingly, we are faced with the problem of signing two bundle IDs with one provisioning profile. Fortunately, there is a simple and elegant solution - Wildcard App ID. We repeat the procedure for creating a new App ID, but instead of Explicit App ID, select Wildcard App ID as in the screenshot.
At this point, we're done with developer.apple.com, but we won't minimize the browser window. Let's go to and read about the Match utility from cover to cover.
An attentive reader will have noticed that to use this utility we will need a private repository and an account with access to both the Apple Developer Program and Github. Create (if you don't have one) an account like InfrastructureAccount@your.company.domain, come up with a strong password, register it at developer.apple.com, and assign it as a project administrator. Next, give the account access to your company's github repository and create a new private repository with a name like AmazingAppMatch.
3. Setting up Fastlane and the match utility
Open the terminal, go to the folder with the project and initialize fastlane as indicated in . After entering the command
$ fastlane inityou will be prompted to select the available usage configurations. We select the fourth item - manual configuration of the project.
A new fastlane directory has appeared in the project, in which there are two files - Appfile and Fastfile. In a nutshell - in Appfile we store service data, and in Fastfile we write jobs, in Fastlane terminology called lanes. I recommend reading the official documentation: , .
Open the Appfile in your favorite text editor and bring it to the following form:
app_identifier "com.company.amazingapp" # Bundle ID
apple_dev_portal_id "infrastructureaccount@your.company.domain" # Созданный инфраструктурный аккаунт, имеющий право на редактирование iOS проекта в Apple Developer Program.
team_id "LSDY3IFJAY9" # Your Developer Portal Team ID
We return to the terminal and, according to the official manual, start setting up match.
$ fastlane match init
$ fastlane match development
Next, enter the requested data - repository, account, password, etc.
Important: the first time you run the match utility, it will ask you to enter a password to decrypt the repository. It is very important to save this password, we will need it at the stage of setting up the CI server!
A new file has appeared in the fastlane folder - Matchfile. Open it in your favorite text editor and bring it to the form:
git_url("https://github.com/YourCompany/AmazingAppMatch") #Созданный приватный репозиторий для хранения сертификатов и профайлов.
type("development") # The default type, can be: appstore, adhoc, enterprise or development
app_identifier("com.company.amazingapp")
username("infrastructureaccount@your.company.domain") # Your Infrastructure account Apple Developer Portal username
We fill it in this way if we want to use match in the future to sign builds for uploading to Crashlytics and / or AppStore, i.e. to sign the bundle ID of your application.
But, as we remember, we created a special Wildcard ID to sign the test build. Therefore, open Fastfile and enter a new lane:
lane :testing_build_for_firebase do
match(
type: "development",
readonly: true,
app_identifier: "com.company.*",
git_branch: "uitests" # создаем отдельный бранч для development сертификата для подписи тестовой сборки.
)
end
Save, enter into the terminal
fastlane testing_build_for_firebaseand see how fastlane created a new certificate and put it in the repository. Great!
Open XCode. Now we have the required provisioning profile of the Match Development com.company.* type, which must be specified in the Provisioning profile section for the AmazingApp and AmazingAppUITests targets.
It remains to add lane to build tests. Let's go to fastlane plugin project that makes it easy to set up export to Firebase Test Lab and follow the instructions.
Copy-paste from the original example so that our lane testing_build_for_firebase ends up looking like this:
lane :testing_build_for_firebase do
match(
type: "development",
readonly: true,
app_identifier: "com.company.*",
git_branch: "uitests"
)
scan(
scheme: 'AmazingAppUITests', # UI Test scheme
clean: true, # Recommended: This would ensure the build would not include unnecessary files
skip_detect_devices: true, # Required
build_for_testing: true, # Required
sdk: 'iphoneos', # Required
should_zip_build_products: true, # Must be true to set the correct format for Firebase Test Lab
)
firebase_test_lab_ios_xctest(
gcp_project: 'AmazingAppUITests', # Your Google Cloud project name (к этой строчке вернемся позже)
devices: [ # Device(s) to run tests on
{
ios_model_id: 'iphonex', # Device model ID, see gcloud command above
ios_version_id: '12.0', # iOS version ID, see gcloud command above
locale: 'en_US', # Optional: default to en_US if not set
orientation: 'portrait' # Optional: default to portrait if not set
}
]
)
end
For complete information on setting up fastlane in CircleCI, I recommend reading the official documentation .
Don't forget to add a new task to our config.yml:
build-for-firebase-test-lab:
macos:
xcode: "10.1.0"
working_directory: ~/project
shell: /bin/bash --login -o pipefail
steps:
- checkout
- attach_workspace:
at: ~/project
- run: sudo bundle install # обновляем зависимости
- run:
name: install gcloud-sdk # на mac машину необходимо установить gcloud
command: |
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null ; brew install caskroom/cask/brew-cask 2> /dev/null
brew cask install google-cloud-sdk
- run:
name: build app for testing
command: fastlane testing_build_for_firebase # запускаем lane сборки и отправки в firebase
4. What about our test stand? Setting up Firebase.
Let's proceed, in fact, to what the article was written for.
Your app may be using Firebase on a free plan, or it may not be using Firebase at all. There is absolutely no fundamental difference, because for the needs of testing we can create a separate project with a year of free use (cool, huh?)
We log in to our infrastructure account (or any other, it doesn’t matter), and go to . Create a new project named AmazingAppUITests.
Important: In the previous step, in the Fastfile in lane firebase_test_lab_ios_xctest, the gcp_project parameter must match the name of the project.
The default settings suit us just fine.
We do not close the tab, we register under the same account in is a necessary measure, since communication with Firebase takes place using the gcloud console interface.
Google is giving away $300 for a year, which in the context of performing autotests is equivalent to a year of free use of the service. We enter payment data, wait for the test write-off of $1 and get $300 to the account. After a year, the project will be automatically transferred to a free tariff plan, so you should not worry about the possible loss of money.
Let's return to the tab with the Firebase project and transfer it to the Blaze tariff plan - now we have something to pay if the limit is exceeded.
In the gcloud interface, select our Firebase project, select the "Catalogue" main menu item and add the Cloud Testing API and Cloud Tools Result API.
Then go to the menu item "IAM and administration" -> Service accounts -> Create a service account. Grant permission to edit the project.
Create an API key in JSON format
We will need the downloaded JSON a little later, but for now we will consider the Test Lab setup complete.
5. Setting CircleCI
A reasonable question is brewing - what to do with passwords? To keep our passwords and other sensitive data securely, the mechanism of environment variables of our build machine will help us. In the CircleCI project settings, select Environment Variables
And set the following variables:
- key: GOOGLE_APPLICATION_CREDENTIALS
value: contents of the gcloud service account key json file - key: MATCH_PASSWORD
value: password to decrypt github repository with certificates - key: FASTLANE_PASSWORD
value: Apple Developer Portal infrastructure account password
We save the changes, create a PR and send it to our team leader for review.
Conclusion
As a result of these simple manipulations, we got a good, stable working stand with the ability to record video on the device screen at the time of testing. In the test case, I specified the iPhone X device model, but the farm provides a rich selection from a combination of different models and iOS versions.
The second part will be devoted to setting up Firebase Test Lab for an Android project step by step.
Source: habr.com
