Trackingplan | Regression testing

Regression Testing

💡

Trackingplan’s Regression Testing module allows you to validate that the analytics performed in each test session match the expected baseline without changing your current tests in your current frameworks. Exactly, just plug & play.

☝🏻

Moreover, you can also consult the results of the tests directly within your CI/CD pipeline using our API. Learn more about it here.

Here are some of the features you’ll be able to enjoy with Trackingplan’s Regression Testing:

Pages Diff
Sequence Validations
Request Log

All of them are designed to:

Ensure your test executions do not break your analytics before going into production.
Automatically document the regressions found between test sessions or specific test cases and their baseline to fix those errors before compromising your data.
Establish baselines, making it possible to confirm a difference as desired so that the test session becomes the new baseline.

After selecting the test session, you’ll be able to instantly see the compatible baselines you can compare them with.

💡

Compatibility occurs when the baseline has the same or more tests than the test session selected. If that doesn't happen, Trackingplan will automatically show it to you with a yellow icon next to each element 👇

Moreover, you also have the option to filter your tests and baselines to quickly spot them.

Once you’ve found the test session and the compatible baseline you want to compare it with, click on the green button in the bottom right of the modal. With that, Trackingplan’s regression testing module will automatically check and compare all the regressions found between the two different executions focusing on the following aspects:

Missing events: Check that all events exist as in your baseline.
Unexpected events: Check that there are no undesired events that are not present in your baseline.
Missing properties: Check that events carry all the properties they do in your baseline.
Unexpected properties: Check that your events do not carry undesired properties that are not present in your baseline.
Specification Warnings: All specification warnings are supported in Regression Testing:

Pages Diff

If you want to have a closer view of the regressions found, the Pages Diff will show you on which pages these differences can be found. To find it, just click on Pages Diff in the event you want to investigate.

This will provide you with precise information about the specific pages Trackingplan has spotted extra or missing hits to know where the problem needs to be solved exactly.

Ready to try it out? Contact us to get Trackingplan’s Regression module and easily spot where your warnings are located with our Pages Diff.

Sequence Validations

Moreover, you can also validate Sequences to check within your critical paths if certain values still match from one event to another.

The possibilities are endless. Let’s see some examples!

Validate that the property product_sku carries the same value between all the events in a critical path like PageHit -> CatalogImpression -> AddToCart -> ProceedCheckout.
Ensure that the event PurchaseConfirmed contains all the SKUs that have been added to the event AddToCart.
Check, whenever an event AddToCart occurs, if it matches with a previous ProductImpression event.
Make sure the event PurchaseConfirmed has in its "purchased items" parameter the same values as in the event AddToCart.
Check that the ‘page type’ in ProductImpression matches the ‘page type’ in PageHit (which is the event that occurs when the page is loaded).

To automatically verify that your events don’t change or any properties get lost in transitions between pages, just write us to support@trackingplan.com with the Sequences to validate and we will add them for you.

After that, we will display in your panel a new section called Sequences, where you will be able to verify the status of these custom validations created according to your needs.

Request Log

After comparing a test session or a specific test case with any of its compatible baselines and seeing that some regressions have been found between them, it’s time to fix those errors before compromising your data.

And for it, you will need meaningful contextual information about the regressions found.

Trackingplan’s Request Log provides you with key information about all the tracks generated throughout a session to get all the necessary details to analyze them locally and debug them easily.

You can either download in bulk all the tracks generated by a specific session or individually by looking at the date, the provider, and the name of the event along with the page where they are displayed.

Just contact us for more details to try it out.

Supported Test Frameworks

We support an increasing number of test frameworks such as Cypress (web), JUnit/Barista (Android), and XCTest (iOS).

Installing Trackingplan Cypress

Trackingplan Cypress adds support into Cypress to run Trackingplan as part of your e2e tests. This way, existing tests can be used to catch analytics data problems before they get into production.

Trackingplan Cypress installs a network interceptor in your app under testing before each test is executed. During test executions, the analytics data that your app generates due to user interactions will be collected by Trackingplan Cypress and sent to Trackingplan for further analysis.

This process works in the background as non-blocking and, therefore, does not interfere with the original request sent to your analytics destination.

Once your tests have finished, you can visit your Trackingplan panel and validate that the analytics performed in each test session match the expected baseline without changing your current tests.

🛠️

To install Trackingplan Cypress, follow the next instructions:

Add @trackingplan/cypress dependency to package.json:

$ # For npm
$ npm install --save-dev @trackingplan/cypress
$
$ # For yarn
$ yarn install --dev @trackingplan/cypress

Add the trackingplan command to Cypress. To do so, one option is to import the @trackingplan/cypress plugin in your support/command.js file. Note that you don't need to copy our custom command, only import the plugin. For example:

// command.js file
// Other commands
import '@trackingplan/cypress';

Setup beforeEach and afterEach for Trackingplan Cypress. One option is to include a global beforeEach in your support/e2e.js file. For example:

beforeEach(() => {
    // Initializes Trackingplan Cypress before each test starts
    cy.trackingplan(TP_ID, ENVIRONMENT, TEST_SESSION_ID, {
        // Advanced options goes here
        tracksEndPoint: "https://eu-tracks.trackingplan.com/v1/"
    })
});

afterEach(() => {
    // Ensures collected data is sent after test has finished
    cy.trackingplan_after_each();
});

⚠️

Note that in the previous example, TP_ID, ENVIRONMENT and TEST_SESSION_ID must be replaced for actual values:

TP_ID is the trackingplan identifier for tracks coming from your app.
ENVIRONMENT allows you to differentiate between production and testing environments.
TEST_SESSION_ID is the identifier of a test session/execution, so that each test execution can be differentiated from previous ones. For example, a timestamp or build number could be used as TEST_SESSION_ID.
tracksEndPoint is the endpoint where collected analytics data are sent. In the example above, it has been set to point to the tracking endpoint in Europe. See Advanced options.

Advanced options

The cy.trackingplan command can receive an optional options object as the last parameter. Please note that the cy.intercept command doesn't work as expected when fetch interceptor is enabled in Trackinglan Cypress. That is why this interceptor is disabled by default.

Parameter	Description	Default value	Example
`customDomains`	Allows to extend the list of monitored domains. Any request made to these domains will also be forwarded to Trackingplan. The format is `{"myAnalyticsDomain.com": "myAnalytics"}`, where you put, respectively, the domain to be looked for and the alias you want to use for that analytics domain.	`{}`	`{"mixpanel.com": "Mixpanel"}`
`sourceAlias`	Allows to differentiate between sources	`cypress`	`IOS App`
`tracksEndPoint`	Endpoint where collected analytics data are sent.
`debug`	Shows Trackingplan debugging information in the console	`false`	`true`

🤔

Questions? Problems? Need more info? We can help! Contact us here.

← Previous

Environments

Trackingplan Public API