Automate Website Performance Testing for Desktop and Mobile with Lighthouse x Sitespeed.io in Jenkins

https://www.sitespeed.io/

In this article I’d like to share how we can automate measuring multiple web pages to see how it performs and how well it follows the best practices in Desktop and Mobile device using Lighthouse and Sitespeed.io in Jenkins.

“Sitespeed.io is a set of Open Source tools that makes it easy to monitor and measure the performance of your web site. Sitespeed.io have Docker images with Chrome, Firefox, Xvfb and all the software needed for recording a video of the browser screen and analyze it to get Visual Metrics.”

Let’s see how you can set up your Jenkins job,

1. Click on New Item to create new job
2. Select Freestyle project after entering Item Name and then click on OK
3. Go to Add to Build Step under Build section and select the option “Execute shell”
4. Prepare list of URLs you’d like to test in a .txt file, e.g.:

https://www.google.com
https://www.wikipedia.org

5. Prepare your budget.json file which will be the standard measurement of you site performance whether it should be marked as passed or failed, e.g.:

{
  "budget": {
    "requests": {
        "total": 100
    },
    "transferSize": {
        "total": 400000
    },
    "thirdParty": {
        "requests": 0
    },
    "score": {
      "bestpractice": 80,
      "privacy": 90,
      "performance": 80
    }
  }
}

6. You need to copy the list of URLs and the budget.json file into docker which will be set as the standard whether your test result should be marked as passed or failed. One way to do it is to add parameter in your job by ticking This project is parameterized checkbox under Job Notifications section and add urls.txt and budget.json into the File Parameter.

6. Input Command

Things to keep in mind:

• Lighthouse runs on mobile emulator by default therefore you need to set specifically the lighthouse configurations such as which User Agent and Device Emulator you’d like to use to run the performance test
• The more iteration you have to test your URLs will have better accuracy, by default it is 3 times and you can set it using e.g. -n 1

Example command to execute performance test on desktop emulator:

docker run --rm -v ${WORKSPACE}:/sitespeed.io sitespeedio/sitespeed.io:17.8.3-plus1 -n 1 --desktop urls.txt 
--outputFolder output --budget.configPath budget.json 
--lighthouse.settings.formFactor desktop 
--lighthouse.settings.screenEmulation.mobile false 
--lighthouse.settings.screenEmulation.width=1024 
--lighthouse.settings.screenEmulation.height=768 
--lighthouse.settings.screenEmulation.deviceScaleFactor=2
--lighthouse.settings.throttling.rttMs 40 
--lighthouse.settings.throttling.throughputKbps 10240 
--lighthouse.settings.throttling.cpuSlowdownMultiplier 1
--lighthouse.settings.emulatedUserAgent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4420.0 Safari/537.36 Chrome-Lighthouse"

Example command to execute performance test on mobile emulator:

docker run --rm -v ${WORKSPACE}:/sitespeed.io sitespeedio/sitespeed.io:18.0.0-plus1 -n 1 --mobile urls.txt
--outputFolder output --budget.configPath budget.json

You can explore other configuration with:

docker run sitespeed.io/sitespeed.io:18.0.1 --help

7. Add Publish HTML reports under Post-build Actions and click on Save after setting the following details:

• HTML Directory to archive: output
• Index page[s]: index.html
• Report title: Performance Report

8. You’re ready to build the job

Sitespeed.io is a really powerful tool which you can integrate with other CI tools or even a dashboard, their sample dashboard can be found in: https://dashboard.sitespeed.io/

This is my very first post. I’m really excited to record more stuff in my learning journey!! :3