Measure UI Performance Using LukeBrowser
LukeBrowser provides several ways to measure UI performance of a page, so you can continuously deliver high-quality web applications.
Obtain performance timing information
For some context, Chrome uses the RAIL model as the guideline for UI performance measurement:
- Response: Respond to user input in under 100 ms.
- Animation: Produce a frame in under 10 ms when animating or scrolling.
- Idle: Maximize main thread idle time.
- Load: Load interactive content in under 5000 ms.
There are several relevant metrics that are widely used to analyze the UI performance following the RAIL model, such as:
- First Contentful Paint: Measures the time from navigation to the time when the browser renders the first bit of content from the DOM.
- First Meaningful Paint: Measures when the page appears meaningfully complete.
- First CPU Idle: Marks the first time at which the page's main thread is quiet enough to handle input.
- Time To Interactive: Measures when a user can consistently interact with all page elements.
In addition, the following metrics are also sometimes used to measure the UI performance:
- Hero Elements: The load time for the major/key elements on the web page.
- Network Requests: The time it takes for all network requests on the web page to complete.
- Long JavaScript Task: The time for all the long-running JavaScript functions to finish so the main thread is no longer being blocked.
Developers can collect UI performance metrics from the browser through the LukeBrowser.pageLoadTime API, which utilizes the information above. All the collected performance metrics are recorded by the browser and are returned as a UiPerformanceTimingResult, which includes the following measurement results:
- Standard Page Load Time: The time it takes for the HTML document to load. Not necessarily a good indicator of page load for modern web applications like C3 applications, which utilize JavaScript to render additional elements.
- Ajax Request Load Time: The time it takes to have no in-flight AJAX requests.
- Time To Interactive: The time it takes for the page to have no long-running JavaScript tasks. When gathering this metric, Luke makes sure that there is at least a five-second quiet window before the metric is finalized. A quiet window is defined by a period of time with up to two in-flight GET requests, no POST requests, and no long-running JavaScript tasks. This helps ensure that applications do not "cheat" by delaying their long-running JavaScript tasks to later in the render time.
- Custom Page Load Time: The time it takes for a specific element(s) in your page to load. The element selected should be the longest-loading element or the most important element of the page.
Generate UI performance metrics with Jango
The recommended way to generate UI performance metrics within C3 AI is through Jango. Jango provides capabilities for both UI Functional Testing as well as UI Performance Testing. It is recommended to use Jango's Low-Code Recording feature to create UI Performance tests, but users who are more comfortable with writing code for their tests can refer to the code snippets below.
For testing the UI performance of a locally deployed application, refer to this template:
var filename = 'test_uiPerfLocal.js';
LukeBrowser.runJasmine(filename, function (suiteName) {
describe(suiteName, function () {
beforeAll(function () {
// Set up the test context so data can easily be created and torn down
this.ctx = TestApi.createContext(filename);
this.baseUrl = TestApi.setupVanityUrl(this.ctx);
// Setup Luke
this.client = LukeBrowser.init();
// Initialize some test page
this.page = LukeTestPage.make({
baseUrl: this.baseUrl,
path: 'some/path',
luke: this.client,
});
});
describe('measure test page load time', function () {
afterAll(function() {
// Remove the data that was created with the context of this test
TestApi.teardown(this.ctx);
});
it('should have performance metrics within threshold', function () {
this.client.pageLoadTime({
testPage: this.page,
keyElementSelectors: ['.some-selector'],
timeoutSeconds: 90,
customMetricNames: ['standardPageLoadTime', 'customPageLoadTime'],
}).assertPerformance('toBeLessThan', {
standardPageLoadTime: 30,
customPageLoadTime: 30,
});
});
});
});
});For testing the UI performance of a remote environment instead, refer to this template:
var filename = 'test_uiPerfRemote.js';
LukeBrowser.runJasmine(filename, function (suiteName) {
describe(suiteName, function () {
beforeAll(function () {
// Setup Luke
this.client = LukeBrowser.init();
// Authenticate if necessary
this.client.setOAuthToken('<oauthToken>', 'https://<some-url>')
});
describe('measure test page load time', function () {
it('should have performance metrics within threshold', function () {
this.client.pageLoadTime({
testUrl: 'https://<some-url>'
keyElementSelectors: ['.some-selector'],
timeoutSeconds: 90,
customMetricNames: ['standardPageLoadTime', 'customPageLoadTime'],
}).assertPerformance('toBeLessThan', {
standardPageLoadTime: 30,
customPageLoadTime: 30,
});
});
});
});
});You can alter your input to pageLoadTime in a couple different ways. An example would be if you would rather skip the generation of a test page. This could be true if you already know your path and there isn't any additional path generation logic necessary, and you know your page doesn't require additional authentication. This can be accomplished like so:
this.client.pageLoadTime({
baseUrl: this.baseUrl,
path: 'some/path',
keyElementSelectors: ['.some-selector'],
timeoutSeconds: 90,
customMetricNames: ['standardPageLoadTime', 'customPageLoadTime'],
}).assertPerformance('toBeLessThan', {
standardPageLoadTime: 30,
customPageLoadTime: 30,
});Another example of a modification to pageLoadTime would be the ability to measure your application's performance while utilizing the caching capabilities of your browser, which we will call a "hot load." By default, pageLoadTime will open a new window and measure your application's performance, which would be a "cold load", but during a hot load it is assumed there is an active window already, and pageLoadTime will then refresh the page and measure your metrics again.
Here's how to differentiate between a cold load and a hot load measurement:
var filename = 'test_cold_hot.js';
LukeBrowser.runJasmine(filename, function (suiteName) {
describe(suiteName, function () {
beforeAll(function () {
// Setup Luke
this.client = LukeBrowser.init();
// Authenticate if necessary
this.client.setOAuthToken('<oauthToken>', 'https://<some-url>')
});
describe('page load time', function () {
// Opens a new window and gets the cold load metrics
it('cold load', function () {
this.client.pageLoadTime({
testUrl: 'https://<some-url>'
keyElementSelectors: ['.some-selector'],
timeoutSeconds: 90,
customMetricNames: ['standardPageLoadTime', 'customPageLoadTime'],
}).assertPerformance('toBeLessThan', {
standardPageLoadTime: 30,
customPageLoadTime: 30,
});
});
it('hot load', function () {
// Hot load needs a "true" to be passed on the second argument for `pageLoadTime`
// Also note that this assumes a window is already active, which was done in the "cold load" it-block
this.client.pageLoadTime({
testUrl: 'https://<some-url>'
keyElementSelectors: ['.some-selector'],
timeoutSeconds: 90,
customMetricNames: ['standardPageLoadTime', 'customPageLoadTime'],
}, true).assertPerformance('toBeLessThan', {
standardPageLoadTime: 30,
customPageLoadTime: 30,
});
});
});
});
});User input for Jasmine-based UI performance tests
You can also select which performance metrics you want to persist and on which ones you want to assert requirements. The metrics you persist will be stored in the TestCaseResult Type. See UiPerformanceTimingResult to view a list of available metrics.
In this example, we want to persist our customPageLoadTime, keyElementSelector, and timeToInteractive while also asserting that the customPageLoadTime should be under 15 seconds using the assertPerformance function.
this.client.pageLoadTime({
baseUrl: this.baseUrl,
path: 'some/path',
keyElementSelectors: ['.some-selector'],
timeoutSeconds: 90,
customMetricNames: ['standardPageLoadTime', 'timeToInteractive', 'selector: .some-selector'],
}).assertPerformance('toBeLessThan', {
customPageLoadTime: 15,
});Get performance metrics for non-landing pages
There are also use cases for generating performance metrics for non-landing pages. For example, if the page you want to test is only opened through a tab button that stays on the same URL, you can use the LukeCore.timeToQuiet, LukeCore.nodePerformance, orLukeCore.lambdaPerformance functions to test performance metrics, depending on your needs.
The LukeCore.timeToQuiet function measures the time in seconds it takes for the page to "quiet down", starting from the invocation of the function. We define a page as "quiet" when there are no long-running tasks and active ajax requests within a 5 second window.
The LukeCore.timeToQuiet function consists of two arguments:
- The desired name for the persisted metric (default is 'timeToQuiet')
- The timeout (default is 20 seconds)
In the example below, we select a button in the navbar that takes us to another page. We then immediately call the timeToQuiet function, which triggers the performance measurement.
it('measures timeToQuiet', function () {
this.client.search('.nav-bar .assetpage', true, 10).click();
this.client.timeToQuiet('assetpageTimeToQuiet', 10).assertPerformance('toBeGreaterThan', {
max: 5,
});
});The LukeCore.nodePerformance function consists of three arguments:
- A lambda function
- The number of times the performance test runs (default is 1)
- The desired name for the persisted metric (default is 'defaultMetric')
NOTE: The results list how long the tail-node of the lambda function takes to resolve; therefore, include a search() or searchAll() function as the last chained node.
In the example below, we need to select some button to access the page we wish to measure. We identify some key selector that we say determines when our page is loaded, and we assert that this selector should not take longer than 10 seconds.
this.client.search('some-button', true).click();
this.client.nodePerformance(function (client) {
return client.search('some-key-selector', true, 10);
}, 1, 'myMetric').assertPerformance('toBeLessThan', {
max: 10,
});If you want to incorporate multiple actions into the timing of a metric, use LukeCore.lambdaPerformance instead.
The LukeCore.lambdaPerformance function consists of three arguments:
- The lambda function
- The metric name
- The maximum time the lambda is allowed to run for
NOTE: This function is different from LukeCore.nodePerformance because it does not just consider the tail-node of the lambda function, but rather all the actions that took place within the lambda.
In the example below, we need to find the time it takes for a user to click multiple buttons and wait for the UI to be updated with the changes.
this.client.lambdaPerformance(function (client) {
client.search('first-button', true, 5).click();
client.search('second-button', true, 5).click();
client.search('third-button', true, 5).click();
client.search('.ui-change', true, 5);
}, 'multiAction', 20).assertPerformance('toBeLessThan', {
max: 20,
});Run the Jasmine-based UI performance tests
The easiest way to run your new UI performance test is through Jango. Upload the file in Jango and run the test. Alternatively, record a new test with Jango and run it after adding your test assertions.
You can also run the test through static console, but this requires a remote server.
NOTE: Due to cluster restrictions, you can only run the code snippets below if the UI your performance test is running against is deployed on the same cluster as the cluster where you are running the code.
// This requires a remote server
Selenium.ensureService();
var file = Pkg.files('/uiDemoTest/test/js-luke-browser/test_example.js');
var test = JsBrowserLukeTester.make().runTestSuiteCode(file.readString());Alternatively, you can upload the test file through the static console in the browser memory and run the following commands:
// This requires a remote server
Selenium.ensureService();
var file = C3.console.LoadedFiles[0];
var test = JsBrowserLukeTester.make().runTestSuiteCode(file.readString());