Monday, September 22, 2008

Featured users algorithms

Some of you may have noticed the featured users list just changed and there's been a few people either angry or confused as to why they were removed, and others asking about the algorithm so I figured I'd write a quick post about it.

Featured users was an idea I came up with one night in response to ROY4L's thoughts on the motivation of users to produce quality content as well as my near religious following of Clay Shirky's "A Group Is Its Own Worst Enemy" essay.

"Featured users" was supposed to provide two main benefits; to find users who consistently make good content that we could highlight on the front page in an effort to make the front page less overwhelming and less of a "treasure hunt". Second, and more often overlooked was to create an incentive for the "cream of the crop" users to create new content and interact with the site more often.

The goal was (and is still) ambitious; figure out who the best content producers are mathematically. I was surprised that a simple algorithm could produce fairly good results. The last featured users list was roughly 98% of generated by algorithm, with the last 2% being me adding or removing users manually.

So here, for the first time, is a run down of the incredibly simple featured users algorithm
(originally called 'user score'):

(average_site_score * 1.2)
(average_number_of_votes * 0.23)
x (number_of_favorites * 0.43)
---------------------------------------
= Your dumb score.

We calculate a score for all the users, and then take the top score and convert it to 10000, and convert all other scores to fit into that percentage. An example of how this turns out shows that the weight is very light towards the top and heavy towards the bottom. Here are some sample results from an old run

#1 nutnics 10000
#2 ROY4L 9947.16
#3 phaseblue 8328.45
#4 max 6374.33
#5 astuteNacute 5957.35
#6 krebstar 4685.71
#7 syncan 4451.45
#8 kingstefan 3757.21
#9 ALMusic 3620.85
#10 PCF 3549.24

From there, the scores went down drastically, because users near the top skew the results for everyone else. The requisite for getting on the "list" was a score over 200, which only roughly 300 people achieve.

At, I used more data to base the scores off of, number of comments (this is why whetstone made the list), number of views, etc. I then realized sites like "Blue Ball Machine" skewed the averages for everyone, so I tried to do them with the top 5% of each users sites excluded (trying to discard anomalies), but the results were still really off.

The numbers don't lie. This algorithm is working on a large enough set of data that a few up-voting alts wont make a difference. More people are viewing, favoriting and voting on the featured users than those who arent featured (even if you use a time scale of a period before featured users existed).

Now the only thing that you can really muck with here is the weight on each piece of the algorithm. Normally, you can look at the results and change the algorithm to remove results you don't like or get results you do like, but a huge part of this is opinion based. Multiple people want DarthWang to be featured, but I find I think the problem is that you can't please everyone with a single list.

This time around, after I was persuaded to let it happen, BTape and Teknorat took the generated list and then added and removed people as they saw fit, which is what caused much of the recent change. So focus your rage towards them for the next couple weeks.

I also made a quick change to the featured users content box, which filters out duplicate users, so at any one point in time you wont see more than one site by each user, which I think will deal with a lot of the spam issues.

anyway, back to work, dongs.

22 comments:

Anonymous said...
This comment has been removed by the author.
Anonymous said...

This month, I had the most commented on site of the month, a site rated high enough to make it to my top 5 all-time, and earned YTMND about $15 in donations, yet I get removed from the list. Users like me and PatienceKing, who have sites rated higher than literally 99.99% of all ytmnds, get removed to add someone like WAFFLEZ, who has only 1 site of 50 rated higher than a 4. I don't mind sharing the space, but getting removed because 1 or 2 people didn't like me despite performing well is demoralizing. I don't mind the idea of adding more featured users, but we should really just be giving people less front page time than doing a 'rotation' or removing proven users to add pet projects of 2 mods. I never had a problem with not getting "enough time" on the front page and I don't see why adding a few more people would drastically change that and force a rotation/dynamic style to things. Sure, I'm bitching, but with the disparities between me and WAFFLEZ, it's a pretty valid point.

keatonkeaton999 said...

I'm confused by what average number of votes and average favorites is. How many other people favorite your sites, and how many votes your sites get?

Dorkmo said...

hey max,

i was thinking... it would be nice if there was a way to indicate which sites the creator was proud of and which were just trash halfassed ideas. and then that data could go twords scoring as well? maybe a check box or something.

cuz like every once in a while i make a good one that i really like and then otehr times i just make lame fad stuff.

Anonymous said...

I really like the new featured user list and not because I'm on it. While there are some people not included that surprised me, for the most part the list includes some of the more creative users of YTMND. People who don't need fads to come up with ideas. While there is nothing wrong with a fad site, how many Koen and faggot sites need to be made by the same users?--hanktherapper

Anonymous said...

Listen, if my sites were rated an average of 3 or less, I'd stop making fad sites. Now that I'm unfeatured, I can STILL get a PSA site on up and coming with a high rating #15-ish of the day, beating featured users while I'm at it. It may not be funny to you, or the 20-30 so called "elite" users who think and act like they run the place and are the ultimate critics, but it's clearly still good to the average community which is what the list should focus on. Yeah, it pisses me off to see my sites perform well yet get knocked off for someone who simply doesn't have a good or funny site to begin with, just ended up tickling the fancy of 1 or 2 people. It seems alot of people who utilize the spirit of YTMND, fad evolution and humor, are getting knocked out of the way for the 'creative' types. Why the hell are NiteSky, Dmaz, Coconutman, etc. getting kicked off? They are pushed aside and replaced with users who tend to be very artistic and talented, but ignore the principle that the site should actually be funny and create things like fractals, odd juxtaposition robots, smooth motion gifs with little more to them than being smooth, or sites on bananas that are simply devoid of humor. This was and I'm pretty sure still is a comedic website, right? It still should be. Is it really impossible to share that space between the formulaic and spontaneous? I can't co-exist alongside gr33nscr33n? Are fads really being destroyed on YTMND? Bring on the creativity, I'm truly waiting to laugh. But if the list is now going to be a steady stream of MoMA bullshit I'm going to downvote without regret. I can make it on Up and Coming with my "lesser" sites all the same. If anything, I'll fight to make original sites that get me back on the list, and personally hank, reward you with 3,000 PSA sites.

Anonymous said...

I'm not going to fight on Max's blog. I don't look at YTMND as a comedic website, but an entertainment site. I don't have to laugh to enjoy a site. I'll even admit here most of my sites are more amusing than out-right funny. I like dmaz. I like yourthecoconutman. I don't know why they were removed. I don't know why you were removed either. I never said you deserved to get removed. As far as why your PSA sites are so popular. I guess you've figured out why there are so many Earnest movies. You can make as many of the same one-note joke sites as you want. But then maybe you've just stated why you were removed in the first place.

Anonymous said...

Yeah comparing me to Ernest..pretty low..that's where I don't bother to argue any further.

Anonymous said...

Although I am glad to be added to the featured user's list, and glad to see people like Korf there. (His sites are rated so low primarily because he gets kinda nasty in comments and gets revenge downvoted, which i think he should be able to do without seeing major repercussions. His sites wouldnt have really high ratings otherwise but he would have at least a few over 4) I don't see why people who seem to play well to the average ytmnd crowd are being removed, i personally don't care for their content (for the most part) but alot of people do. I think the idea of featuring people like myself/korf/laundry/gr33nscr33n/hank was to give some of the people who put more effort into their sites, despite them not appealing to the head paste lawlz crowd, some more front page time so more casual viewers may get a chance to see the sites, without having to sift through pages of sites made in 30 seconds.

And for the most part didn't you already 1 most "art" sites goneja?

Anonymous said...

I mostly give them 2 or 3s. I reserve 1's for complete dicks and the 90% of sites which are stolen/complete garbage and the like.. My voting practices should not be contested, check out my votespread.. and for the record, I've only seen one of your sites, and I 5'd it.

Anonymous said...

I think the pure math is a better solution... it's perfectly fair to everyone.

Anonymous said...

Featured Artists

Users that have an average site rating of 4.0 (or whatever you determine) with a minimum of 20 sites (again can be changed) are featured in this main page tab. Sites that have been created in the past 3 days by these users will cycle thru this tab, randomly showing any 5 eligible sites on each refresh. How often do the "Ytmnd superstars" produce a crappy site?

This section could also require other users to "vote in" Featured Artists. Each account could have X ammount of "Artist Favorites" that could be used similiar to the Favorite site vote--if a user receives Y ammount of Favorite votes, there sites are eligible for the Featured section
--Ryuudo 20:40, April 15, 2007 (CDT)

Was that before or after the comment that you read from ROY4L?

Anonymous said...

Pure math simply allows people room to abuse their position or artificially inflate themselves up.
Ideally a solution would involve active monitoring rather than a few nights of general guesswork and quick evaluations on many hundreds of users.

The point of the system is to feature good content and content that the average visitor to the site may enjoy. Users who consistently post poor sites or extremely repetitive content go against this point. They drive away more traffic and interest than they are worth. Simply put, appreciate the featured status or you risk losing it.

Hopefully the right balance will be struck between proper algorithms rewarding users and watchful eyes propping up potentially valuable content as well as preventing abuse.

Anonymous said...

I agree with everything Goneja said - especially the part about 5'ing Sanvarin sites. 5'd.

Anonymous said...

In my eyes, the new pinky list is pretty much what it was before. A list of people who are around the site relatively often and who participate freely. To make a mathematical formula to discern a "good" user from the "bad" ones is impossible. We ALL make SHIT sites, because humor is subjective and so is... and I hesitate to call it this... our creative process.

I'm happy to be on the list, but when I thought I was off, it didn't really phase me that much. I'd still make site, my friends would still see them, and I'd still (occasionally) make the U+C. And while I wouldn't call two users picking out names "fair", I'd love to see someone compile an exact list of attributes that WOULD be "fair." I would then like max to post that list in the news and ask for feedback.

As far as I'm concerned, you guys are doing the best you can at improving the site, keeping it true to its roots, and being even-handed. That is until you do something I disagree with, of course =P

-MrStump

Joel Green said...
This comment has been removed by the author.
Anonymous said...

Just get rid of featured users.

Anonymous said...

Yes, get rid of featured users, for fucks sake. It's alot funner when your sites (and you) aren't subjected to that level of scrutiny. And it's just better that everyone is equal and has the same color usernames.

Replace it with "User Favorites", which would be a randomized list of sites that have been favorited at least once.

Anonymous said...

Why's YTMND down?

Anonymous said...
This comment has been removed by the author.
Anonymous said...

You're the blog now dog

Anonymous said...

Fearcondom - I think a better way to hilight FU created content is by changing the title color of sites appearing in the recently created section. That eliminates the need for a featured user section and still allows users to prioritize their browsing of new sites.