iOS CI/CD using Jenkins and Fastlane

Why you need continuous delivery

We often want to deploy our mobile applications more frequently without compromising the stability of our applications and to do that we need the process of building and distributing our mobile apps predictable and on demand.

By the end of this article I will guide you through setting up a CI/CD pipeline using Fastlane, Jenkins, Gitlab and SonarQube for code quality. Hold on tight, it's going to be a long way.

Before we are able to do that, we must make sure that our code is always ready for deployment so that we can deliver with confidence. We can do this by building a deployment pipeline that can help us discover bugs in minutes with the help of unit, integration and UI tests. Because of this, the team’s focus will be on delivering small, incremental changes that are low risk and help the product with a faster time to market.

Continuous delivery can even drive the development costs down even though they require an up-front effort in building those pipelines and writing those unit tests by reducing and even completely eliminating the so called hardening and smoke test phases as well as the unnecessary “code freezes”.

By being able to deploy smaller changes faster while at the same time being able to continue working on other small, incremental improvements and getting feedback faster from users, development teams are more efficient and happier because it enables them to do what they care about most: building apps and features that users love.

Delivering with confidence

The first and most important step in building a valuable continuous delivery pipeline is having confidence in the code you deploy. How many times have you heard the following excuses for not deploying an app to the AppStore or even for submitting a build for testing?

“We cannot release the application until our lead developer returns from vacation.”

“I have to recreate the provisioning profiles before I send a new testing build but I am currently working on something else, sorry”

“This fix is not critical, we can submit it in our next release, later this year”

Even though if some of this excuses have a seed of truth in them, the fact is that these teams do not have any confidence in the code they are about to deliver or the process is just too tedious to do that they loath going through it.

On top of this, Apple’s reviewing process, although necessary, doesn’t encourage implementing a continuous delivery process.

The first step a team should take towards gaining trust in the code they are delivering is making sure it works according to the specs and nothing helps more than having an extensive unit test suite. Integration and UI tests are also more than welcomed.

Investing time in writing and maintaining automated tests not only helps the team identify and fix bugs faster and easier, it also lowers the team’s dependency on manual testing which can very often be time consuming therefore increasing the confidence in the code it delivers.

Automate the development and release process

Once you have a great suite of automated tests it is time to put it to good use by building a continuous delivery pipeline around it. There are two main approaches to this:

Using Xcode Server - that allows you to configure bots that execute your test suites
Using Fastlane and a CI environment such as Jenkins, Bitrise, CircleCI, etc.

I personally prefer the latter one. Even though it involves relatively more effort in setting up and maintaining, fastlane comes with more flexibility and a much broader set of plugins and actions that will allow your team to do much more with your continuous delivery pipeline.

Fastlane is a suite of open source Ruby scripts that allows your team to build, test and distribute your application. And is one of the most widely used tools iOS developers rely on in their development process and if you didn’t knew about it until now I recommend going to their website to find out more.

If you’re not currently using fastlane, your development process probably looks something like this:

Run your test suite and make sure not a single one is failing.
Build the app for release.
Run into code signing issues.
Update your certificate and provisioning provisioning profile.
Try to build the app again and realise you forgot to update some of the application screenshots so you go and do that.
Update your push notification certificate or key.
Build the app again and upload it to AppStore Connect.
Send a build for smoke testing on TestFlight.
Spot a small typo on the first screen and realise you have to start this process all over.

With fastlane, all you would have to do is push a single button. Your release process could look as simple as this:

lane :appstore do
  cocoapods # Installs Cocoapods dependencies
  scan # Tests the app
  snapshot # Creates automated app screenshots
  match # Installs the certificate and provisioning profile
  gym # Builds the app
  deliver # Uploads the build to AppStore Connect
end

Simple, isn’t it? And more important than that: predictable. Every single time you would run this, the result will be the same under the same conditions, no human error involved.

Now that I’ve got your attention, let’s jump right in and see how we can make your project benefit from what fastlane has to offer.

Setting up Jenkins

Before we begin, we have to setup the continuos integration environment, and to do that we will rely on Jenkins. Jenkins is an open source automation server that will allow you to create your own jobs that run different fastlane lanes.

You can download and install on a mac machine Jenkins from the official website but we recommend using brew:

brew update && brew install jenkins

To start Jenkins, just run the following command in your terminal:

jenkins

You should now be able to access Jenkins as http://localhost:8080 and the first thing I recommend you to do is install a couple of plugins. To do that, go to “Manage Jenkins” and then “Manage Plugins”.

The plugins I recommend are the following:

HTML Publisher Plugin - to publish test reports
AnsiColor Plugin - to color the build logs output
Rebuild Plugin - trust me, you will need this.
Git Plugin - For accessing your Git repo.
Gitlab - This one allows Gitlab to trigger Jenkins builds.

You can go on and add more anytime you like or think some plugin can ease the process of building your continuous delivery pipeline such as SSH Slaves Plugin, Slack plugin, etc.

We’ll come back to creating our first Jenkins job right after we write our fastlane files.

Setting up Fastlane

Let’s start by installing fastlane. As I previously mentioned, fastlane is a set of open source ruby gems. For your ruby environment it is recommended to use rbenv but rvm (that comes as a default on Mac OS) is also fine.

Because you do not want to worry about what version of fastlane is installed on your machine and what version you have on the Jenkins machine, or simply because you do not want to install all the dependencies when moving to another machine we recommend using bundler to manage the fastlane installation and not only. If you do not have bundler install, go on and install it.

In the root of your project create a file named Gemfile that should have the following contents for starters:

source "https://rubygems.org"

gem 'fastlane'
gem 'cocoapods'
gem ‘CFPropertyList' gem ‘slather'

plugins_path = File.join(File.dirname(__FILE__), 'fastlane', 'Pluginfile')
eval_gemfile(plugins_path) if File.exist?(plugins_path)

As you can see, we also have the Cocoapods dependency in this file along with slather - that is used for generating code coverage and CFPropertyList - that can be used to update .plist files. This means that we don’t have to worry about having cocoapods installed on the machine that we want to run this build on.

To install fastlane, save this file and then just run the following command in your terminal:

bundle install

After it finishes installing all the dependencies, you should be able to use fastlane, so let’s start using it. First run:

fastlane init

After this, you should be presented with 4 options that look like this:

What would you like to do?
1. 📸 Automate screenshots
2. 👩‍✈️ Automate beta distribution to TestFlight
3. 🚀 Automate App Store distribution
4. 🛠 Manual setup - manually setup your project to automate your tasks

Even though fastlane suggests trying at least one automation if you are a beginner, we recommend going with the fourth option: manual setup.

After this, fastlane will guide you through some frequent asked questions. We recommend reading through them so you get a better understanding of the concepts behind it.

Once finished, you should now have a faslane folder that contains two files: Fastfile and Appfile. We will add a couple more to this folder as we move along.

Setting up Fastlane

The firsts thing you should focus on is the Appfile. It should look something like this:

app_identifier "com.company.identifier" # The bundle identifier of your app
apple_id "account@developer.com" # Your Apple email address 
team_id "APPLETEAMID" # Developer Portal Team ID

We then want to write our first lane, so we open the Fastfile. It should look something like this:

default_platform(:ios)

platform :ios do
  desc "Description of what the lane does" lane :custom_lane do
      # add actions here
  end 
end

The first thing we want it to do is rename that lane and add our first action that runs our tests. To do that, we recommend using scan, an action provided by fastlane.

Scan

We open the terminal at the location of our project and run the following command:

fastlane scan init

This should create a new file in our fastlane folder named Scanfile, go on and open it. The structure of it should be pretty self explanatory so we go on and update it.

# The project scheme you want to build and test
scheme("YourAppName")

# If it should clean the project before testing
clean(true)

# The output report types
output_types("html,junit")

# If it should open the reports
open_report(false)

# The output directory for the test reports
output_directory("./reports")

# Generate code coverage files
code_coverage(true)

# Device on which the test will be ran
device("iPhone XS (12.2)")

# Enable skip_build to skip debug builds for faster test performance
skip_build(true)

This file is used by Scan to configure how Xcode command line tool should run your application tests. We will have similar config files throughout this step by step guide so we recommend carefully going through each line’s comments to better understand why we added that setting.

For more details and options, we encourage you going through the official fastlane documentation: https://fastlane.tools.

We can now go on and add this action to our Fastfile and test it. Our file should look like this now:

platform :ios do
  desc "Test the application" lane :test_app do
    # run tests
    scan 
  end
end

You can see that the lane is now called test_app and that we added the scan action. To run this lane, open the terminal and run the following command:

fastlane test

Gym

Now that we’ve added our first action, we must dive deeper to take full advantage of what fastlane has to offer.

We have to let fastlane know how it should build our project and to do that we recommend you rely on another action provided by it: Gym. To setup Gym you should run the following command in the terminal:

fastlane gym init

This should create a new file, called... you’ve guessed it, Gymfile, that you should go on and edit:

# If you use Cocoapods you surely have a workspace workspace("YourAppName.xcworkspace")
# Your default project scheme
scheme("YourAppName")

# The project default configuration to use
configuration("Ad-hoc")

# If it should clean the project before building
clean(true)

# The default export method
export_method("ad-hoc")

# The path to your build folder
build_path("./build")

This configuration file is tailored more around building ad-hoc builds because this will be the most common use of this action: providing builds for testing.

One thing you might have noticed is the Ad-Hoc configuration. We imply that your projects has three configurations:

Debug - for debugging
Ad-Hoc - for ad-hoc builds that are signed with an ad- hoc provisioning profile
Release - a configuration for your release build

This is common if you use a build distribution system such as HockeyApp (now AppCenter) or Firebase Distribution from Firebase. If you are using Testflight, the Release configuration should suffice.

That being said, let’s now jump to one of my favourite topics: managing your certificates and provisioning profile.

Match

One of the most useful tool in the fastlane arsenal is Match, a tool that helps teams manage signing certificates and provisioning profiles. Before you begin, please make sure you haven’t reached the maximum limit of signing certificates.

If you did so, please try clearing up one space up for match to use. To start using Match in your Fastfile, run the following command in the terminal:

fastlane match init

This will guide you through a setup process in which you will be asked to provide a storage option - I recommend git - and a git repository. Once you’ve done that, a Matchfile will be generated for you. It should look something like this:

# The storage method
storage_mode("git")

# The repo you want to store your cenrificates and provisioning profiles
git_url("https://gitlab.com/your-account/certificates")

# The default type, can be: appstore, adhoc, enterprise or development
type("development")

The best part about Match is that you can create a secure, shared repository of certificates and provisioning profiles that multiple teams can use.

I recommend using git branches for each team to achieve this separation. Each team will have its own branch that will contain the encrypted certificates (development and release) along with their own provisioning profiles: development, ad-hoc and release.

To do this you must first generate the certificates and provisioning profiles for one team and then clone that branch for the other ones.

# The storage method
storage_mode("git")

# The repo you want to store your cenrificates and provisioning profiles
git_url("https://gitlab.com/your-account/certificates-repo")

# Your team git branch or “master”
git_branch "myteam"

# The default type, can be: appstore, adhoc, enterprise or development
type("adhoc")

# The app identifiers for which the profiles will be needed
app_identifier ["com.company.identifier", "com.company.identifier.extension"]

# Your Apple Developer Portal username
username "account@developer.com"

Now that we’ve done this, we can go on and create the certificates and provisioning profiles:

fastlane match development
fastlane match adhoc
fastlane match release

Your branch repo should now have two folders: certs and profiles. The certs folder should contain two folders that have the development and distribution certificates while the profiles folder should contain folders with the provisioning profiles for each of your targets / app identifiers.

To add another team to this repo simply create a new branch from this one and delete the profiles folder and update the branch name in the Matchfile:

# Other team git branch
git_branch "other-team"

Then you can run the commands for generating the provisioning profiles again (fastlane match development, etc) and the new team will have its own encrypted provisioning profiles. Pretty neat, isn’t it? Just don’t forget the encryption password or you’ll have to nuke (fastlane match nuke) everything and start over again.

One last thing you should do before using match in your project is add the following line in your Matchfile:

readonly true

This will prevent match from regenerating the certificates and provisioning profiles again when you run match from your fastfile.

Now that we’ve covered the testing, building and code signing parts, we can move on to creating a lane that will build and distribute our app for testing.

Build and distribute

Let’s add another lane to our Fastfile:

desc "Build and distribute the application"
lane :build_and_distribute do
  # test the application
  test_app
  # get the last commit message
  change_log_message = changelog_from_git_commits(commits_count: 1)
  # install the certificate and provisioning profile
  match(type: "adhoc")
  # build and archive the application
  gym
  # distribute the app via Firebase Distribution
  firebase_app_distribution(app: "your-app-id",
      release_notes: "#{change_log_message}",
      debug: true,
      groups: "testing"
  )
  # upload dsyms to Firebase Crashlytics
  upload_symbols_to_crashlytics(dsym_path: "./YourAppName.app.dSYM.zip")
end

Let’s go through it line by line:

One of fastlane’s capabilities is the so-called “lane switching” and we use this to run our previously defined test_app lane. It’s just a ruby function called from another function.
The second one creates a string with the latest git commit message.
The next one installs the certificates and provisioning profiles we just created. As you can see, we specify what type of profile we want to use.
We then build our app. Gym uses the configuration in the Gymfile.
After the build and archive succeeds, we upload it to Firebase Distribution, but you can use any tool that might suit you better.
Don't forget to upload the dSYM zip file for Crashlytics to symbolicate your crash reports.

We can now run this function locally using this command:

fastlane build_and_distribute

Now that we got familiar with the basic concepts, let’s go and set up our first Jenkins jobs.

Setting up the Jenkins jobs

Go on and open Jenkins in your browser by accessing http://localhost:8080, and go “Manage Jenkins” and click on “Configure System”. Under the Gitlab section you should add your Gitlab configuration and credentials, where the host URL is the base URL of your [Gitlab instance](https:// gitlab.com/).

After you’ve done this click on “New item” and you should see a form similar to the one below.

Let’s name this one YourAppName-Test because it will be responsible for testing your app. In the “General” section select the Gitlab connection that you have just created.

Under the “Source Code Management” section please select the Git option.

Type in your repo URL in the “Repository URL”, add or select your Gitlab credentials from the “Credentials” option and specify what branch would you like to be tested - let’s go with origin/develop - which should checkout the develop branch.

You might want certain commits not to be built. To do that you could add an additional behaviour that can ignore commits with a specific message. Let say you want all build that contain skip-ci in the commit message to be ignored. You can add (?s).*skip-ci.* in the Exclude Messages and all commits containing skip-ci will be ignored (for example: [skip-ci] Bump version number).

In the “Build Environment” section please select “Delete workspace before build starts”.
Finally we got to the crucial part: building and testing the app. In the “Build section” add an “Execute Shell” build step that contains the following:

source ~/.bashrc
export LANG="en_US.UTF-8"
bundle install
SNAPSHOT_FORCE_DELETE=1 fastlane snapshot reset_simulators fastlane test_app

The first file loads your bash configuration file.

The second one is a requirement for fastlane to properly display the run logs.

Keep in mind that at this point the job has already checked out your develop branch, so the next line is for bundler instal the project dependencies: cocoapods, etc.

We then reset all the simulators, to run our tests on a fresh environment (someone might have used them before our job is ran) and after that we run the test_app lane just like we did on our machine.

It should look something like this:

You can add a Post-build Action to publish JUnit test result report. As you might remember, when we configured Scan, we added to options: one to publish JUnit and html reports and one that specifies where to publish them so let’s go on and add the path in the Test report XMLs to reports/cobertura.xml

And that’s it! Save your job and go ahead and run it and at the end of the log you should see something similar:

[08:20:04]: fastlane.tools just saved you 11 minutes! 🎉 Recording test results
Finished: SUCCESS

You can now go on and create another job to automate the distribution of the app so let’s do that right now only this time we will use what we have already configured so in the Copy from section type the name of the job you have just created.

Let’s name it YourAppName-Distribute.

This will create a copy of the job you have just set up and we’ll only change two things: enable “Build when a change is pushed to GitLab” - Push events in the “Build trigger” section and change the following line in the Execute Shell Build Phase: fastlane test_app to fastlane build_and_distribute.

And you’re done. You have a job that can build, test and distribute your app.

We’re on a roll here, so let’s add one more. This one will be a little bit more fancy but will have the same base as the previous one so let’s go on and create a new job from the test one that we have created the first time. Go on and name it YourAppName-MR because it will be responsible of building merge requests open in Gitlab.

Under the Source Code Management section change the “Branch specifier” section from: origin/develop to origin/${gitlabSourceBranch}.

This will checkout the branch specified as a source in the merge request. Now let’s add an additional behaviour to “Merge before build” that should look like this:

The name of the repostiory: origin

Branch to merge to: ${gitlabTargetBranch}

Merge strategy: default

Fast-forward mode: —ff

In the “Build trigger” section check the “Build when a change is pushed to GitLab. GitLab webhook URL” and select push events, Open merge Request events and Comments. In the Comment field you can add a comment that will trigger a re-build of the merge request, Jenkins please retry for example.

We did a lot of work in Jenkins in this chapter and to finish all up we will need to link everything up with some Gitlab webhooks.

Triggering Jenkins jobs from GitLab

Gitlab has to communicate with Jenkins when different events happen: a push, a merge request, a comment on a merge request to rebuild the merge request, etc. To do that we can use Gitlab webhooks.

To add a webhook, go to your Gitlab project and click “Integrations” under the “Settings” menu. It should look something like this:

In here you can add your webhooks, we’ll need two: one for changes pushed to the develop branch - to distribute builds, and one for opened merge request - to merge, build and test the merge request. Merge requests (to the develop branch) that are accept will then trigger the distribute job as it will result in another push to the develop branch.

Hope this flow is clear so let’s jump straight to it.

The first webhook you should add is the one that will trigger your distribute job. In the URL section you should add the URL to the Jenkins job, is should look like this:

http://localhost:8080/project/YourAppName-Test/

If you have Jenkins installed on a different machine, please use its address instead.
The only thing to check from the list of options is “Push events” and add “develop” in the branch name text input underneath it - this will only trigger for builds on the develop branch.

Save it and test it, and it should queue a job in Jenkins.

The second webhook you must create should do a POST request to the merge request job:

http://localhost:8080/project/YourAppName-MR/

And it’s triggers should be the following:

Comments
Confidential Comments
Merge request events

You can now save and to test that this is working properly open a merge request in your git repository or comment with your re-build comment phrase that you have previously set (“Jenkins please retry”).

By now you should be able to build, test and distribute your app in an automated manner and start saving precious time.

Gathering SonarQube metrics using Fastlane

This part is optional but brings a lot of value to teams that want to keep an eye on the code quality of their project and what to take action towards constantly improving it.

To install SonarQube and it’s dependencies, please follow the instructions found here.

To start running it just type sonar start in the terminal. You should be able to access it at https://localhost:9000.

Before we dive into updating the sonar-project.properties we should setup Slather - a tool that handles code coverage for your project. Please create a .slather.yml file in your project (yes, with a dot in front) that looks something like this:

coverage_service: cobertura_xml
xcodeproj: YourAppName.xcodeproj
workspace: YourAppName.xcworkspace
scheme: YourAppName
source_files:
- YourAppName/*
output_directory: reports/ ignore:
- YourAppNameTests/* - fastlane/*
- Pods/*

This will be used to generate a code coverage report that will be sent to SonarQube.

Then we should set up our SwiftLint rules. To find out more about that please go to https://github.com/realm/ SwiftLint to find out more about it and how your project’s .swiftlint.yml like should look like. Please check the full set of rules and pick what fits best your project.

Let’s head back to our Fastfile and update the test_app lane first.

desc "Test the application"
private_lane :test_app do |options|
  # get the publish_reports option
  publish_reports = options[:publish_reports]
  # run tests
  scan
  # check if it must publish reports
  if publish_reports
    publish_reports_to_sonar
  end 
end

And then add the new private lane called by the test_app lane.

desc "Publish the application reports to sonar"
private_lane :publish_reports_to_sonar
  # Run slather to generate code coverage
  slather(proj: "YourAppName.xcodeproj", 
    workspace: "YourAppName.xcworkspace",
    scheme: "YourAppName",
    configuration: "Debug",
    html: true,
    cobertura_xml: true, 
    output_directory: "./reports"
  )
  # Run swiftlint to detect code smells (you might need to specify the executable path
  swiftlint(output_file: "./reports/swiftlint.txt", ignore_exit_status: true)
  
  # Run lizard to check code complexity
  lizard(source_folder: 'YourAppName',
    export_type: 'xml',
    report_file: 'reports/lizard-report.xml')
  # Run sonar
  sonar
end

Add a new lane that will be called by our merge requests:

desc "Test a merge request"
lane :test_merge_request
  # test the app without publishing the reports
  test_app(publish_reports: false)
end

And a new lane that will be called from now on by our test job:

desc "Run tests"
lane :run_tests
  # Test the app and publish the reports
  test_app(publish_reports: true)
end

As you can see above, we have taken advantage of lane parameters to be able to add more complex logic in you Fastline.

The final configuration step is to edit the sonar- project.properties. We won’t go through it here but you should keep in mind that we keep most of the reports in the reports folder. The file provided by the Backelite team should be a good starting point.

After all of this has been done, the Jenkins test and merge request jobs should be updated to call the run_tests and test_merge_requests lanes.

When this is done, you should be able to check your project code coverage and code smells in SonarQube.