Changes in Skyscanner Partner Quality Score Calculation - FAQ

1. What is the PQS V2 score? What is the difference between V1 & V2 scores?

The PQS V2 score is a new algorithm to calculate PQS score based on travellers' 1-5 scale of emoji rating on partners, and subtract the traveller complaint score loss. A time decay factor has been applied to it, which means, the more recent data carries higher weightage compared to older data in the calculation.

Compared to the V1 score, here are the differences with the V2 score:

• First, it is based on a new survey design, which is a 1-5 scale emoji rating by travellers, rather than the old binary one (good/bad). The new survey data provides more granularity of traveller ratings and dimensions into PQS score calculation.
• Second, the algorithm is stretching the concentration of PQS scores to make the distribution of scores more differentiable. Then the partner’s score for each market will distinguish with each other, compared to V1 score that huge amount of partners having the same or very close scores.
• Also, this algorithm guarantees the score generation for smaller rating volumes but still provides a trustworthy score, rather than showing “not enough ratings” for a V1 score.

•  

2. Why are we switching to V2 scores?

There are 2 reasons for that:

1. The V1 score is too concentrated, which makes it hard for travellers to choose between partners. We want the PQS score to differentiate partners and better reflect traveller ratings. With PQS V2 scores, the score distribution is less concentrated, which makes it easier for travellers to better understand a partner’s quality score before making a booking selection.
2. The V1 score algorithm will rely on a relatively large volume of ratings. Compared to that, the V2 score algorithm can generate a reliable score with a smaller volume. Therefore, smaller partners or those who have recently launched a new market can more easily get a PQS score when the minimum volume of ratings are received. This will provide information to travellers who are more trusting of a PQS score than a “not enough ratings” sign.

3. Will the new PQS score be immediately shown to travellers?

The PQS score will be shown to travellers and available in partner portal to you when it is live in mid-late Q4, together with the revamped “Traveller Feedback” page on partner portal. Before the new PQS score is officially launched, you will have 2 months of notice period to be aware of the change of PQS score. You can get in touch with your Skyscanner Commercial Manager to understand your projected score before official launch, get prepared for it, and make the improvement before the new score is live.

4. As a partner, what can I expect for my V2 scores?

You may witness an increase or drop of your PQS score in a specific market, but we do not anticipate any drastic changes.

If you have a PQS score drop for V2, please note that this does not necessarily mean you are “becoming worse”, it is instead a result of introducing a new algorithm for score calculation. For instance, travellers in some markets have strict standards for giving a high overall PQS rating. In these cases, we expect most of our partners in these markets to experience a drop in their PQS score. Please note that this is not partner-specific behavior, but market-specific behavior.

There may also be cases where you had no PQS score before but then get a score when V2 is launched. This means your rating volume meets the minimum requirement of generating a PQS V2 score, which needs less feedback numbers than V1 score. Overall travellers are more likely to choose a partner with a PQS score than one with the “not enough ratings” message. Additionally, the new algorithm ensures that extreme ratings, when submitted in low volumes, will not significantly compromise the trustworthiness of the score.

5. As a global partner with multiple brands or markets, why do I provide similar services but end up with different scores?

This can be due to the brand awareness of different brands in different markets. Travellers’ sometimes have different rating behaviours depending on the market they are voting from. This can lead to the score differences for the same brand in different markets. Similarly, travellers have different brand awareness, which will lead to different PQS scores on multiple brands of the same partner.

6. What if I have very few ratings? Will I still get a PQS score?

The PQS V2 score algorithm makes it possible to use a smaller volume of traveller ratings to generate a trustworthy PQS score. This provides smaller partners or partners in new markets opportunities to reflect their quality through PQS score. The minimum feedback volume to generate PQS is 10 in each market over the last 91 days. If your brand feedback volume is still smaller than the minimum required to generate a score, the “not enough ratings” message will be shown instead of a score.

7. What am I supposed to do to prepare for the new PQS score?

You will be informed of the PQS score change 2 months before it is live in mid-late Q4. Before the V2 score is live, you can get in touch with your Skyscanner Commercial Manager to understand your projected V2 score. Together with the launch of PQS V2 score in mid-late Q4, we will also launch the the revamped “Traveller Feedback” page on partner portal with telling the breakdown of each theme, which will provide you how travellers feel about each aspect of your quality. And in the future, we will provide more insights on partner portal such as the trending of PQS score to help you better understand your performance.