Have you ever wondered why do you get frustrated when suddenly your phone call drops, even when you are connected to the network which shows full five bars (best signal quality)? Does it affect your perception about your mobile operator? You are not alone in thinking “why does this happen to me? I pay a lot for my mobile phone and the network connection fee”. I believe that you can easily relate to this example. This example illustrates that a positive user’s experience regarding a particular service (network connection) may not be guaranteed by the current state-of-the-art wireless access networks such as 4G.
In this example, there are two primary stakeholders: you, the end user or the customer; and the mobile operator. You as one of the stakeholder, might be affected by the quality of services (signal strength, price, and other value-added services) offered by the mobile operator to determine “should I change my network operator?” On the other hand, the mobile operator might be interested in understanding your experience about the services they offer to you with an intention to keep you as their customer for as long as possible, i.e., to minimize the network churn. So, how can the network operator understand your experience of their services or your (users) Quality of Experience?
So what is users’ Quality of Experience?
According to Oxford Dictionary, the word ‘quality’ means “a distinctive attribute or characteristic possessed by someone or something,” and the word “experience” means “an event or occurrence which leaves an impression on someone.” Users’ Quality of Experience (QoE) about a particular thing, for instance, technology, network, application, or any other service, is subjective. It depends on their cognitive, psychological, and behavioral states along with the characteristics of the things (e.g., signal quality and price). The users’ QoE regarding a thing, also depends on users’ situations in which these services are provided to them. Last but not the lease, users’ QoE varies over time. For example, you may start to like/dislike your mobile phone after using it for a longer period.
After not finding a clear definition of QoE, we recently tried to define it as follows: “Quality of experience (QoE) is a metric that depends on the underlying QoS along with a person’s preferences towards a particular object or service where his/her preferences are defined by his/her personal attributes related to expectations, experiences, behaviour, cognitive abilities, object’s attributes and the environment surrounding that person.”
How do w
e model, measure and predict QoE?
Typically, the stakeholders such as network operators and telecom engineers try to relate the crude quality of service (QoS) metrics as packet losses, delay, and jitter to user’s QoE. This relation is determined using statistical models that consider users subjective ratings gathered using laboratory and field tests and the corresponding QoS values. For example, the well-known ITU-T E-Model, PESQ model, PSQA, or the model proposed by Menkovski et. al. These models are limited as they model QoE as a function of QoS. As mentioned above, QoE comprises several factors related to users’ situation such as their cognitive and mental state and QoS. We, therefore, define QoE as the function of all these factors which we call context. We define context as any information that assists in determining users’ QoE. The table on the right shows several context factors that can be used to determine users’ QoE. So how do we take all these factors and measure users’ QoE? To answer this question, we proposed and developed a system called CaQoEM – Context-Aware QoE Modelling, Measurement, and Prediction.
CaQoEM: Context-Aware QoE Modelling, Measurement, and Prediction
CaQoEM considers Bayesian networks and utility theory to model complex inter-dependencies between several factors (see table) to measure users’ QoE on a single scale. It is also able to predict users’ QoE with high accuracy under multiple diverse conditions 
prevalent in mobile and distributed systems. An example, in the figure on the right you can see a decision network that comprises a Bayesian network (oval nodes) and utility nodes (diamond shape). First, this Bayesian network is created using the collected data via field tests (initial training set). For example, the data for factors such as bandwidth, delay, location, jitter and satisfaction are collected via network probes, GPS and users. The collected data (initial training set), is used to train a Bayesian network. The experts can also create a BN based on their experience or expertise. The formed Bayesian network is then rigorously tested to determine several hypotheses. Once, the experts are satisfied by the outputs the Bayesian network, this Bayesian network can be used measure users’ QoE.
To measure users’ QoE, we consider utility theory to map users’ subjective ratings, for example, “good,” “poor,” or “excellent” to an objective scale. We do so by using an interval scale. The values on the interval scale then become the utility values for each state (“good” or “poor”) belonging to QoE node such as Satisfaction and Technology Acceptance (context states). These states are then fused together (using multi-criteria decision making) to determine users QoE or the overall QoE situation.
Once we have measured QoE based on the collected user ratings, and other factors present in a dataset (initial training set), these QoE values are added as a new column in the dataset. We now call this dataset (training set_append) which can be used to create a new BN for QoE prediction on-the-fly. This Bayesian network can be deployed in applications, network entities such as routers, gateways, and load balancers to orchestrate QoE-aware operations.
For more information about how CaQoEM works, and to get a grasp the state-of-the-art in the area, you can read the following articles:
- Karan Mitra, Arkady Zaslavsky and Christer Ahlund, “Context-Aware QoE Modelling, Measurement and Prediction in Mobile Computing Systems”, IEEE Transactions on Mobile Computing, IEEE, 2015.(pdf)(all figures and tables) (bib).
- Karan Mitra, Arkady Zaslavsky, and Christer Åhlund. “QoE modelling, measurement and prediction- A review.” arXiv preprint arXiv-1410.6952 (2014). (pdf)
Please do not hesitate to contact me to discuss anything regarding QoE!