Mysteries in the Klouts

Klout is arguably the de facto standard in social media analytics. The magic of Klout is the “Klout score“, a number between 1 and 100 originating from an algorithm that is for obvious reasons not disclosed by the company. Users are able to track their scores, and they may receive “Perks” (e.g. discounts, gifts) from Klout partners as rewards for their social activity.

klout_perk

(photo from BrianBoyd)

But what makes Klout scores valuable? In spite of the algorithm not being open, we can find some insights about the score in their Web site. The first is that it attempts to measure “influence” and no other thing:

The Klout Score is a number between 1-100 that represents your influence. The more influential you are, the higher your Klout Score.

The second is that it aggregates your data from different social platforms, but it is not a measure of central tendency:

The Klout Score isn’t the average of your influence across all your networks, it’s the accumulation.

This is a good start to understand the score. However, it says little about its importance or potential usages beyond some form of gamification in social media activity  (i.e. competing with your colleagues to get better scores), or a as a sort of “vanity metric”. Is Klout really worth considering for business matters? Is it representing something interesting for social media professionals? Well, it seems so, as solutions like Salesforce integrate Klout.

As a matter of fact, displaying high Klout scores have been found to influence (positively of course) perceptions of credibility. Yes, it has also raised a number of criticisms, for example, about inconsistencies in ranking. Nonetheless, the scores are there to stay, and I take them as something given, since questioning to what extent they capture the concept of influence will always be challenging and subject to controversy. If we look at Merriam Webster’s definition of the term “influence” we have:

: the power to change or affect someone or something : the power to cause changes without directly forcing them to happen

: a person or thing that affects someone or something in an important way

This seem to be coherent with the following hints about the score you can find in Klout’s Web site:

We believe it’s better to have a small and engaged audience than a large network that doesn’t respond to your content.

Posting a thousand times and getting zero responses is not as influential as posting once and getting a thousand responses

Interacting with people who have higher Scores will help raise yours more, but interacting with someone with a lower Score will never hurt yours.

So, our common understanding of influence seems to be properly captured, as the Klout score is considered a measure of the intensity of action you trigger via social platforms. This may or may not be correlated with the influence you have for example in the realm of ideas or in your personal or professional life. We simply don’t know. It is clear that influence as a measure of action in the Web may or may not be the same as influence in the life of people and in society in general, but this is not saying that the Klout scores have no use or importance.

Moving away from conceptual discussions, the quotes above give an impression of how the score is calculated. However, the heterogenity of sources that Klout uses makes it difficult to guess how “influence” is actually and concretely computed. Current sources (as listed in their Web page) are:

  • Facebook (likes, comments, wall posts, friends),
  • Twitter (followers, retweets, mentions, list memberships),
  • Instagram (followers, likes, comments, photographs submitted)
  • Google+ (comments, +1’s, reshares, on your personal profiles only)
  • LinkedIn (connections, recommendations, comments, on your personal profiles only)
  • WordPress (Inlinks, ratio of inlinks to outlinks, PageRank),
  • FourSquare (tips saved by others, Tips liked by others, Friends, Check-ins, Likes, Mayorships).

It also considers Klout’s own “endorsment” system, the “+K”. Quite an impressive set of inputs!

These are the “attributes” used as raw data, and from there, to complicate things further, Klout uses “signals”, e.g.:

the ratio of reactions you generate compared to the amount of content you share. For example, generating 100 retweets from 10 tweets will contribute more to your Score than generating 100 retweets from 1,000 tweets

The possible relevant different scores one can devise from those diverse sources is nearly infinite.  So, designing a “correct” measure of “influence” from these is simply not possible, as we cannot define social Web influence with a “golden rule” to compare. In consequence, it is not possible to measure the “accuracy” of Klout scores against any criterion that does not include pre-assumptions about what is influence or who is influential in the Web. But this is not a criticism of Klout, is simply the fact of recognizing that “influence” is an inherently vague concept, and measuring it with numbers will always be an imperfect endeavor.

So maybe it happens that Klout scores become right because people tend to behave in the ways that increase Klout scores?. This is nothing bad or good in itself, but something that deserves further study. In any case, I believe studying Klout and how it relates to different accounts of influence is an exciting topic for Web scientists.

Given the insights on the sources used by Klout and the hints on its internals, Klout seems to follow pretty similar ideas to the original PageRank (accidentally, it uses also for WordPress a computation of PageRank as an input). I have tried to figure out how to create my own social score computation at a toy scale, a sort of “DIY score”. I started by looking directly at a concrete PageRank implementation, that included in the NetworkX package, and the way I could model the attributes via directed graphs (using the Digraph class).

For simplicity we use only one of the signals about: retweets. Then, we can build a graph with nodes representing Twitter accounts. The problem comes with the edges. As Klout uses ratios of reactions to the amount of contents, the relevant signal is actually the retweet ratio.  The retweet ratio should then be included in the model as an input, and recomputed periodically (it should be noted that Klout says it uses a 90-day window of information).

Now the problem is guessing the distribution of retweet ratios. Suh et al. (2010) found that retweet ratios correlate linearly well with number of followers, and they also correlate with number of folowees and age of the account (but not with the number of past tweets).

So in principle, we could use number of followers as a proxy measure. But, what is the distribution of followers? Kwak et al. (2010) found followers do not follow a power law distribution for users with more than 100,000 followers, which is a sort of anomaly among social networks. But with the exception of those highly followed users, they found an unsurprising power law with exponent 2.276.

Restricting our toy example to small numbers of followers, we can then plan a simple simulation generating a scale-free network, i.e. a graph with degree distribution following a power law, and then applying the PageRank. Code snippet fragment follows.

G=nx.scale_free_graph(10000)
G = nx.DiGraph(G)
P = nx.pagerank(G)

# Transform PageRanks to 0-100 integer interval
for key in P:
    P[key]=max(0, 100-int((1-math.log(P[key]))*10))
print P.values()

Only with this we have our toy rank up and running (the transformation is in this case particularly naïve). But the science behind crafting a good rank is tailoring the parameters, weights and types of inputs to have a reasonable account of social influence. This only could come from experimentation, of the kind we assume Klout has invested significant effort.

In any case, with that simple ranking scheme and of course using relevant information (e.g. retweet ratios or something that correlates well with it) you have something roughly behaving coherently with Klout’s hints quoted above: it is not central tendency, connecting more networks (providing more data) only adds to your score and focuses on reactions and not in the amount of content postings. The distribution however does not seem coherent with the popular belief that the average Klout score is 20 (or 40 according to other sources) but it is with the fact that scoring more than 60 is really uncommon. Look at the distribution for a sample of 1000 random “users” (another problem of this computation is that you really need parallelism to make it scale, don’t expect to compute this for more than 100,000 nodes using your laptop).

toy_social_score_hist

Average is 14 and (it is hard to see in the figure) very few scores are above 60.

Well, it seems that devising a social score is both easy and hard in some sense!.

Advertisements

One thought on “Mysteries in the Klouts

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s