Bizarre Trend: Journalism Professors Using Klout Scores As Part Of Students' Grades
from the bad-metrics dept
We've raised questions in the past about the relevance of "Klout" scores. If you don't know, Klout is one of a few companies that try to measure "influence" online by looking at your social media activity. The whole process seems kind of silly, but for whatever reason, once you put a number on things, people take it seriously, no matter how bogus the number might be. Lots of companies now use Klout scores to determine who they should give special perks to, leading to plenty of people just trying to game their scores. However, should Klout scores count towards your grade as a student? Adam Singer sent over examples of two separate journalism professors who think so.In both cases, the professors admit that Klout scores may not be particularly accurate, but both have interesting defenses for including the score as part of the grades that students get. Todd Bacile, from Florida State University, argues that even if people think Klout is inaccurate or silly, since others are taking it seriously, students need to take it seriously too. Even if it's a problem-perpetuating situation, he argues that he's being something of a realist and merely preparing his students for the real world, where they'll be judged by their Klout scores whether they like it or not -- and thus one project (but not the entire grade for the course) will be graded based on Klout scores:
Most people seem to either love or hate Klout, so the notion of assigning a portion of a student’s grade to their respective Klout score may cause some to react … what’s a good word to use here … fretfully. Yet, as an educator teaching electronic marketing at the collegiate level I owe it to my students to introduce them to every and any concept that will help them land an internship or fulltime job. Klout matters to employersFor whatever it's worth, he notes that his students have massively increased their Klout scores because of this. He also claims that by having them focus on Klout scores -- something that could constantly be measured -- it actually kept students more engaged:And here is an inescapable fact. Many firms are sizing up college student’s Klout scores as a quantitative metric to use for job applicant screening. Therefore, I decided to create a class project in which the final grade earned is solely determined by a student’s Klout score.
An experiential project like this proved to be enjoyable for the students and maintained their attention and enthusiasm throughout the semester. Many students would compare scores and discuss different techniques used to engage with powerful opinion leaders within the social world. Which students had higher scores became a friendly competition causing students to work even harder at engaging others. Imagine that: students wanting to work more to develop skills that current marketing employers are searching for!Bacile has since noted that when he presented this idea at a conference, other professors told him they were going to do the same thing.
The second story Singer sent over is from Ryan Thornburg, who teaches journalism at UNC, and who is also using Klout as part of the grade (in this case, counting as 20% of it). Thornburg also admits, quite directly, that Klout is probably "overly simplistic." So why is he doing this? Because he thinks that it forces students to do actual experiments in which they can all learn how Klout works, and see if they can improve it.
How is that possibly fair to students who are struggling to raise this arbitrary number that's contrived inside a black box? It's fair because it transforms the class from a workshop on button-pushing to an exercise in hypothesis testing, strategy and critical thinking. Students -- who often approach grades with calculating economy of effort -- don't know what they have to do to boost their Klout scores, so they are forced to design simple experiments, isolate variables, and generalize their findings.In the end, he notes that perhaps his best students will actually be able to build a better Klout.
I'll admit that my initial reaction to these stories was horror. The idea of basing grades on a silly system like Klout certainly feels very, very wrong. However, the explanations and defenses from both professors have me rethinking that stance somewhat. Is it really all that different from "teaching to the test", as some teachers do for standardized testing? An SAT score may not really tell us much of anything, but it is important for many colleges, so is it a surprise that teachers help their students optimize for it? While we can quite reasonably worry that focusing on Klout has students optimizing less useful skills, from an experimental standpoint, perhaps it's not such a crazy idea.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: grades, journalism, professors, scores
Reader Comments
Subscribe: RSS
View by: Time | Thread
All of my classes have a heavy dose of social media in them these days, but Klout is among the five or six big Internet trends I dismiss outright early on. I'd much rather have them increase their clout by learning how to write and report stories that people actually care about.
[ link to this | view in chronology ]
Re:
That can be applied to wider society as well which is scary. It is no longer sufficient to be intelligent and hard working in order to succeed in life. Nowadays, it is all about marketing themselves in the right way which, let's be honest, not every one is good at. This is why you see so many arse kissers promoted to high level jobs while those who would actually be good at the job get left behind.
[ link to this | view in chronology ]
Re: Re:
iz zere zomezing wrong mit you ? ? ?
vat are you trying to hide by hiding from facebook ? ? ?
'thought-crimes' have been made real, they are working on 'pre-crime', and soon, you will be hunted down for not plugging into the matrix...
(based on a true story)
art guerrilla
aka ann archy
eof
[ link to this | view in chronology ]
Re:
What I read from this article and these professors, is that they are trying to teach students both how to write/etc. and how to understand and influence measures of their impact online.
That seems pretty comprehensive and valuable to me.
[ link to this | view in chronology ]
Re: Re:
I think his point is that any time spent working on one's Klout score is time not spent on other things, such as learning how to research or write. Obviously it's possible to do both, but not an unlimited amount of both, so it's a tradeoff.
[ link to this | view in chronology ]
Re: Re: Re:
It is my opinion that conclusion does not logically follow even if time is scarce. People are capable of promoting themselves and honing their craft at the same time and I would presume that the more you learn and experiment with promotion the better you get and the less time it may take.
In fact, I think the fundamental premise behind what Mike tries to teach is that most professions require that individual promotion and outreach to become a core part of becoming successful.
[ link to this | view in chronology ]
Re:
At least this system is impartial, repeatable, and known. Typically when being graded on how well you write, you get a heavy heavy dose of bias based on the teacher. You're not writing for the "public", you're writing for your teacher. If you do something unconventional or something they don't agree with, you'll do badly despite it perhaps being an idea the public (your future audience) would love.
I think the idea has merit. Perhaps in the future many classes that teach so to speak public service positions will be judged and graded by that very same public. The internet is a wonderful thing... embrace it
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Not Quite Like the SAT
Klout's secret formula for determining influence is only part of the problem with using it for decisions like employment and grades. The much bigger problem is that it requires giving up lots of access to personal data to a private entity that will do who knows what with it. Want a higher Klout score? Give access to Facebook and other social media channels. This will be problematic, particularly down the road as people get jobs or grades that they want, then forget about their Klout accounts, which still have access to personal and otherwise private information.
[ link to this | view in chronology ]
Enough, Already!
[ link to this | view in chronology ]
Re: Enough, Already!
This comes from a Liberal, as well :-P
[ link to this | view in chronology ]
sounds like the 'Ebonics' of the Internet
[ link to this | view in chronology ]
Re: sounds like the 'Ebonics' of the Internet
[ link to this | view in chronology ]
The "economist" concludes: "bad-metrics" are better than none.
But you don't grasp that's a mis-applied buzz-word; here's the entire definition from my plain old text dictionary of the early 90's:
metrics: n.pl. [with sing. v.] the science or art of writing in meter
"my initial reaction to these stories was horror" -- Good, you've reached "hardening of the attitudes" geezer stage.
But you talk yourself round:
"perhaps it's not such a crazy idea" -- Bad, you've never had a solid foundation, and only need to get accustomed to the crazy.
[ link to this | view in chronology ]
Re: The "economist" concludes: "bad-metrics" are better than none.
Thankfully, dictionary.com provides the perfect example sentance that pertains directly to you:
"In fact, you should maintain metrics and prove with verifiable data that you are worthy to be an authority on any point made."
I rest my case.
[ link to this | view in chronology ]
Re: The "economist" concludes: "bad-metrics" are better than none.
[ link to this | view in chronology ]
Re: The "economist" concludes: "bad-metrics" are better than none.
The definition of attitude is "the inclination of the three principal axes of an aircraft relative to the wind, to the ground, etc." So what the heck are you talking about, man?
[ link to this | view in chronology ]
Well you know what they say...
It's not what you know, it's who you blow.
Some of us have known this for years. Even more importantly some of us do not care! I don't need to have some idiot a half world away re-tweet. "LOL! Funny stuff..."
[ link to this | view in chronology ]
... and now the cynical viewpoint...
So, sure, gaming Klout scores can be a great way to learn valuable real life skills of ... gaming the system.
(Or, less cynically, they are learning to work within the construct that has been given them, and finding creative ways to gain some advantage over all the other folks who are trying to do the same thing. Isn't that pretty much business and life?)
[ link to this | view in chronology ]
Re: ... and now the cynical viewpoint...
If life is zero-sum, yes. If not, then there's much more to it than gaining advantage over those around you.
[ link to this | view in chronology ]
The problem here
Those professors might as well grade a student on how many dogs walk past their house. Certainly, a student could game the system by leaving dog food outside, but what is really being measured here?
Also, TL:DR but are any companies REALLY using Klout scores as part of their hiring decisions? As a hiring manager I have taken a number of training courses and it seems to me that Klout would fall in the *really don't do this* section of the manual.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
Good metrics, maybe. But how are bad metrics better than subjective scoring? You would rather measure the wrong thing objectively than the right thing subjectively?
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Grade Buying
[ link to this | view in chronology ]
Re: Grade Buying
[ link to this | view in chronology ]
If it is formulated as the professor using Klout as an arbiter of the quality of student work, that is bad. He should be the arbiter. His opinion is more direct, more responsive to your work, conveys more information, is more interesting, teaches you more.
If it is formulated as a specific "Klout project", an assignment where you maximize your Klout score and get a grade for doing this, then it looks like a fun competition to me.
Implicit in the "Klout project" is that the fraction of the grade it represents will not be too big. Say 10% at most. So you get 10% of your grade for your ability to manipulate the net in a certain way. That is fine as a kind of "practical assignment".
Also implicit in the "Klout project" is that the percentage is subject to critique. If it is too much, then the students would quite naturally ask, why are we spending so much time on this aspect of things.
It reminds me of Qualitative Analysis (a chemistry lab course in high school) where all that counted was identifying the chemical. You write the chemical name on a card after several weeks of work. A cool challenge, but please not too important percentagewise because there is too much luck involved.
I suppose you could even have a class purely in "manipulating the net" where the whole class is one big project to get hits or raise a google rank or hack something. But that's a pretty *specialized* class and it's not journalism per se.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
SAT scores and trainability
[ link to this | view in chronology ]
a) the subject is journalism.
b) the Klout score is only a fraction of the grade.
Look at modern journalism - how many journalists function well in their role without a plethora of social media? Not just for socializing, but for putting out some of their work (albeit in link form), garnering interest, communicating with regular readers and garnering new ones - all well recognized uses for social media in business here on Techdirt! The Klout score projects sound like they are designed to focus students on using social media in this manner, instead of solely with a social focus.
Secondly, a key aspect of journalism is not only getting your work to the public, but gathering resources. Human resources are mainly collected through networking. Developing social media skills, and online networking, is a valuable skill for journalism students, surely? I would likely feel different if the course was calculus.
Finally, as someone else pointed out, this develops their writing skills - not the traditional skills they are already learning in class, and have spent several years through out primary and secondary school developing, but a specific skill-set used online. It is quite a different task to write a compelling argument in 2000 words as opposed to writing an interesting or informative statement in under 160 characters.
These are all skills that they will need in their profession. Academia is often faulted for being too esoteric - at least these two professors are focusing on real-world skills that will assist these students in the workforce. And there is no indication they are doing it at the expense of traditional core subjects for the unit.
[ link to this | view in chronology ]
Privacy Nightmare
[ link to this | view in chronology ]
Preposterous
The business model is less about your score and more about delivering you to companies to make money. That's fine - I'm a capitalist pig small business guy - but purporting to measure influence when there is little independent research establishing that one can determine influence from social media activity... Doesn't belong in the curriculum.
@commammo -- I also adjunct at Kent State.
[ link to this | view in chronology ]
and...
[ link to this | view in chronology ]
Klout Score and Hiring
It demonstrates to me that the applicant values the wrong things and, worse, thinks I do too.
Despite the advent of the social-media-promoted rockstar marketer, I look for people who can articulate a *client's* story and make their case impactful.
That has nothing to do with a high Klout score. In fact, a high Klout score from a young applicant most likely means that the applicant will probably spend most of his/her time promoting themselves.
[ link to this | view in chronology ]
The Klout Controversy...
The grade for this assignment is a mere 10%, which is similar to a standard participation grade or attendance grade in many courses. That being the case, I am surprised how much uproar it has created! I see too many benefits to use this in my class to immediately discount it due to the argument that it is an imperfect metric. First, I have spoken directly with hiring managers who use Klout to screen applicants seeking social media jobs. These are the types of jobs my marketing students are hoping to land upon graduation. Do all companies use this? No. Do some? Yes. Second, I am teaching students how to create content other people want to engage with (comment on, retweet, 'Like", +1, re-share, etc...), similar to successful brands doing the same. I see a strong correlation with Klout and these types of activities. Can people cheat? Sure, but anything online - and in life - can be cheated or manipulated. For example, some people estimate as many as 30% of online product reviews are fake; yet, online reviews are still useful and serve a purpose. Third, as for the argument of 'it is not fair to base a grade on an unknown algorithm', such a stance would cause thousands of schools to immediately cease the use of marketing and managerial simulation software. These software programs provide a good exercise for students, yet the grading is based on an unknown algorithm analyzing several criteria. The point is we live in a world of uncertainty with useful, yet somewhat questionable metrics.
I have to run. Feel free to contact me via Twitter if you want to discuss this more. Take care everyone! @toddbacile
[ link to this | view in chronology ]
The clout of Klout
Klout is not the most accurate metric. Nonetheless, it doesn't stop companies from trying to grasp at any type of metric that's available. For example, genesys has started prioritizing customers according to their Klout score. I write about this here: http://www.blurban.org/2012/11/08/whats-your-number/
[ link to this | view in chronology ]