top of page

UX Strategy

On the journey to raise maturity of our digital products in the portfolio, one milestone were introducing UX strategies to the products. Through observation, it was apparent that product teams did not have the right tools in place to communicate the products value. Normally it is somewhat easy to communicate value, when products are sold (e.g. through ROI, subscriptions), but not in this case as the products in the portfolio are developed for internal usage primarily.

 

Due to substantial investments in these products, the higher is the importance of reporting accurate data with value to it's stakeholders, especially the sponsoring executives. Hence why part of the UX strategy were to introduce industry standard metrics and measurements, as they reveal value for it's users. Not only would these measurements and metrics report on value, they equip the product teams to do data-driven decisions, then enabling them to measure the performance of the overall products through before plan and after plan measurements.

 

One example of that is that we see that one specific project is not using a specific feature, then we can strategically reach out to the project to understand why. This leads to targeted research, so we can focus on the problems we do have and can see through the data on usage. Let's in this case say that the users through interviews reveal that the specific feature X doesn't allow them to do once specific thing on reporting, so they can't use it. From there we can go to other users and understand whether it is the same case for them, or if it's just an edge case. If this scenario were applicable to more end-users, and this reporting feature would increase adoption, first then we can write new user stories and put them into the product roadmap. So, when designs are tested with those users and verified, we can go and build that specific feature, and invite the end-users into our product Demo's so they increase goodwill and ownership to our products. This may lead to more people in the business adopting the tools because they've heard through the most powerful marketing material, word of mouth, that they had a great experience with our team and the product in itself. This is the type of stories we aim to achieve through introducing the Digital Adoption platform.  

User Usage Tracking

Contents

  • User Story

  • Introduction

  • Metrics

  • Data Structure

    • Usage Tracking Data Structure

    • Usage Tracking Summarised Metrics Table structure

    • Modifications to the Users table

  • Calculation Methodology

    • Architecture​

      • Structure Diagram of WebJob and Main Code​

    • Times and Dates in Calculations

    • User Types

  • Display of the Metrics

User Story - Improved usage monitoring leading to user led prioritisation

Acceptance Criteria

Confirmed metrics to monitor:

 

General Notes:

  • 2 levels of reporting needed: 

    • Level 1: Application, accounting for all projects inside MIDP online

    • Level 2: Project level (one project in MIDP Online).

  • Refer to attached PDF

  • Exclude all global admins from the count and DPM team members to not influence counts.

  • Key actions "activation milestones" in MIDP Online includes:

    • Setting up the sync to CDE

    • Project Config

    • Delivery Planner

    • Sync to CDE

    • Assign Milestone

    • Create Deliverable

    • Add user to user group

 

1) Product Adoption Rate at the frequency of Daily, Weekly, Monthly, quarterly and Yearly for all pages in the product.:

 

Definitions:

  • New Active Users = The number of people who completed one or more of key actions in the stated time period. 

  • New Users = The number if people who obtained a user account in MIDP in a given period stated previously.

  • Total Users = All users who have a user profile in MIDP (active and inactive users)

  • Daily Active Users (DAU) = The number of people who performed 1 or more key actions in a day. 

  • Monthly Active Users (MAU) = The number of people who performed 1 or more key actions in a week. 

 

Formula for product adoption rate:

(New Users Divided by Total Users) x 100

Note: Exclude all global admins from the count and DPM team members to not influence counts.

NFRs:

  • Build new loggers to allow for the above

  • Build web job to pre-calculate the appropriate metrics

  • Appropriate data structures to store both logger raw entries and surmised metric values

  • Build PowerBi Dashboard to report on the above to account for: page hit and page updating

  • Update Wiki accordingly

2) User Activation Rate (Not for new users to the whole product) that interact with "Activation Milestones" at the frequency of: Daily, Weekly, Monthly, quarterly and Yearly for all pages in the product.

Formula:

(Number of users who reached one or more of the listed activation milestones divided by number of users who signed up) x 100

3) Metrics for Product Stickiness Rate: at the frequency of: Daily, Monthly, quarterly and Yearly for all pages in the product.

 

  • DAU = Number of unique users + number of unique returning users (In both cases performing more than one key action per day)

    • Daily active users = DAU​

  • MAU = Number of unique users in a month + number of unique returning users (In both cases performing more than one key action per month)

    • Monthly active users= MAU​

 

Product Stickiness Formula: (DAU divided by MAU) X 100

 

4) Metrics for Active User Growth at the frequency of: Weekly, Monthly, quarterly and Yearly for all pages in the product.

 

Calculates growth rate in weekly active users (WAU).

 

Formula: ((Current WAU - Previous period WAU) divided by Previous period WAU) x 100

 

 

6) Product Usage Frequency at the frequency of: Daily, Weekly, Monthly, quarterly and Yearly for all pages in the product.

 

Number of user sessions with a performed key action(s) within a specific time period

Introduction

To properly track both user usage of features and also tracking user usages of the whole site we need a tracking system that is robust and does some of the calculation work up front and stores it in the Database to ensure it's easy for a range of sources to consume. To that end the intent is to make a system that tracks both page hits with a unique identifier and also specific 'Activation Milestones'.

An 'Activation Milestone' is a key event that is completed within the application that marks a user doing a specific action. These can be seen in the PDF.

Metrics

 

The following section explains the metrics we are collecting, the frequency and whether they are for the whole application or specifically for a project.

Key Activation Milestones

The key activation milestones for MIDP Online are:

  • Setting up a connection to a CDE

  • Setting up rules of credit (Advanced Analytics)

  • Delivery Planner - i.e. accessing the page

  • Sync a CDE - i.e. import, export, sync or configuring a CDE schedule

  • Assign a Milestone

  • Create a Deliverable

  • Add a user to a group in the project

  • Importing a schedule

Frequency

 

​The frequency of measurements will depend on the specific metric but the periods we will be using with example date times (all times are in UTC) are:

frequency.png

Specific metrics

 

There are 8 separate metrics we are going to measure which are as follows:

  1. Product Adoption Rate

  2. User Activation Rate

  3. Product Stickiness Rate

  4. Active User Growth

  5. Retention Rate

  6. Product Usage Frequency

  7. Churn Rate

  8. Utilization Rate

Each metric is described in detail below with a calculation. The variables in each calculation are described in the table following this table. Activation Milestones and Project refer to whether this metric is also calculated for each activation milestone and for each project.

metrics.png
Variables part 1.png
variables part 2.png
Data structure.png
usagetrackingsummarisedmetrics table structure.png
usagetrackingsummarisedmetrics table structure part 2.png
calculation method.png
Introduction
User story & AC
Metrics
calculation method part 2.png
User story
Anker 1
final piece -display of metrics.png
  • Email
  • Linkedin - Grå Circle
bottom of page