Posts Tagged facebook
FeedMod: Quantifying news feed contributions of Facebook friends
Posted by Matt in development, general on January 17, 2011
Previously, I proposed FeedMod, a Slashdot-style moderation system for Facebook to weed out all the garbage posts while allowing the interesting stuff to stand out. After a basic implementation and a month of gathering data, I present some results. Through post moderation, I quantify the contribution of each Facebook friend to my news feed and identify the Facebook friends who provide the most positive and negative contributions. Such information can quickly identify which friends to “defriend” and which to pat on the back for their good posts. Such a moderation system will also allow for fine grained filtering of the Facebook news feed to provide a more enjoyable news feed experience.
While Facebook provides a “like” button to indicate approval of news feed posts, there is no indication that this feature is used to filter news feed content. Facebook does provide an option to block news feed posts of designated friends, but this is a very coarse grained filter. More recently, Facebook presented an interesting analysis of news feed posts. This study identifies post topics that are more likely to be “liked” or commented on by friends as well as identifying post topics that are correlated with high friend counts. While the Facebook study analyzes post content, it only compares against metrics inherent to Facebook itself, such as “likes” per post, number of comments per post, age of posters, and friend counts. The study neither attempts to quantify the news feed topic preferences of Facebook users, nor does it attempt to identify a Facebook user’s best or worst friends based on friends’ news feed posts.
Design
As previously mentioned, the main idea is for each FeedMod user to moderate each news feed post in a manner similar to Slashdot post moderations. Each news feed post is assigned a moderation which consists of an integer value score and descriptor string. For this experiment, I chose the following moderations:
Value Descriptor Examples
-2 Troll Rants or politically charged arguments; I really can't stand these, hence the -2
-1 Spam Advertisements, such as requests to attend an event, donate money, or to come to a friend's place of business to buy something
-1 Redundant Endless duplicate announcements, typically on holidays and birthdays
-1 Indifferent Stuff I really don't care about, such as random song lyrics or sports posts about teams that I don't follow
-1 Unintelligible Something posted out of context, usually by someone with whom I'm out of touch
0 Normal I assign my posts to this, otherwise I use it rarely
+1 Funny Self explanatory, but I found myself only using it when I was about to fall out of my chair laughing
+1 Insightful Anything that gives insight into the highlights of a friend's life or a well formed argument on a topic
+1 Informative Usually updates on what someone is doing
+1 Inquisitive Someone posing a question to his or her friends
+2 Interesting Usually news articles or a post about something I hadn't heard of or thought about before (these posts are the best, hence the +2)
By moderating the status updates of all of my Facebook friends, I am then able to assign a score to each friend based on those moderations. I can even moderate status update comments made by friends and non-friends alike. The score, similar to Slashdot’s “karma,” is created by summing the moderation values for all of a friend’s posts. In this way, I can see the balance of how positive or negative a friend’s posts tend to be. The advantage of this method is that friends with a large number of good or bad posts will stick out.
Implementation and Facebook API
My original idea was to use the Facebook API to modify the news feed and allow me to moderate friends’ posts. A month ago when I started this, I had no experience with the Facebook API and did not know if it would even allow me to do such a thing. While I did achieve what I wanted in being able to moderate friends’ posts, I can say conclusively that the Facebook API is seriously lacking. I have two major issues with the API: 1) an inability to customize the Facebook website, forcing developers to reinvent the wheel for even the smallest applications, and 2) limited access to information easily accessed on the main Facebook website.
What I really wanted to do was to be able to modify the existing Facebook news feed and insert a moderation select list next to each status update. When a FeedMod user selected a moderation, the post-moderation pair would be stored in a database. Unfortunately, Facebook does not allow developers to modify any portion of the Facebook website, including the news feed. It would be much easier if Facebook allowed this since I wouldn’t have to recreate the entire news feed. This lack of customization is a serious barrier to developers who have an idea for a simple application that makes use of the existing Facebook website.
The Facebook API does, however, allow you to query status updates and friend information for use on your own website. What I learned was that apps within Facebook are actually remote websites running within an iframe on the main Facebook website. Unfortunately, Facebook limits the information you can get: news feed items are restricted to status updates, links, videos, and photos — you cannot retrieve time ordered taggings, newly added friends, profile picture updates, and likes, among other things.
Interface
While I found the Facebook API disappointing, I was able to recreate the Facebook news feed on my own webserver, using the Facebook PHP toolkit. Finally, with a database backend to store moderations, I was able to produce a customized news feed with the ability to moderate friends’ posts:
To the right of each status update, I included a select list to moderate the post. I also display the current moderation total for the friend, the friend’s moderated post count, and the percentile rank out of all friends (the friend with the most negative moderation total is 0%, the most positive is 100%). This way, at a quick glance it is easy to see how many good or bad posts a friend has made to date.
I also created a statistics page, which ranks friends by the number of posts as well as lists of the top ten best and worst friends based on their moderation totals. I also breakdown the moderation total and display the leaders of each moderation attribute (for example, who has the most “spam” posts).
Evaluation
After finishing the implementation, I then moderated posts for a month and now present an analysis of the results. During a one month period, I moderated over 2,000 posts with 250 individual posters. Some other interesting tidbits:
- 67% of my friends posted at least once within this one month period
- 64% of all moderated posters were friends; the remaining are non-friends commenting on a friend’s post
- 93% of all moderated posts were made by friends
Here is a breakdown of all moderated posts by their assigned attributes:
From the figure, a clear majority of posts are positive, while 35% are negative. Out of all negative posts, the “indifferent” and “unintelligible” categories are used the most. On the whole, I find most posts that I don’t like to be something I don’t care much about (and don’t want to read) or something that I don’t understand (usually it’s something taken out of context), hence the pervasiveness of the “indifferent” and “unintelligible” categories. The “informative” category dominates the positive posts, followed by “insightful” and “interesting.” My experience is that a significant number of posts (almost 40% to be exact) are status updates announcing what a particular friend is doing. While most of these are mundane, I don’t really find them to be bothersome, hence the positive score.
Next, we look at how many posts each Facebook friend created during my one month evaluation:
This figure shows the number of posts made by each friend that posted at least once. A little less than 40% of my friends posted once, while about 3% posted more than 40 times in the past month. One friend posted almost 120 times, or roughly four times per day! Clearly, my news feed is dominated by posts from the same handful of friends.
Next, we look at the distribution of moderation scores across all of my Facebook friends:
This figure shows that most of my Facebook friends (the ones that posted at least once) have a slightly positive moderation total. Since most of my friends do not post very much, the lack of extreme scores makes sense. A small number of friends have very bad or very good scores — 3% have have a total less than -6 and another 3% have a total score greater than +20. Those 3% that have the worst scores clearly stand out from the majority of my friends and are definitely good candidates for defriending. Conversely, the top 3% go a long way in keeping my news feed enjoyable.
Lastly, we look at both post count and moderation total:
Here, we see the quantity and quality of posts made by each Facebook friend. Some friends have identical (post count, moderation total) values, hence the z-axis indicating the number of overlapping friends. Like the previous two figures, this shows that most friends have slightly positive moderation totals and low post counts. It also appears as though friends with higher post counts tend to be more positive. However, there are some friends that buck the trend and have very negative scores for relatively high post counts (as identified previously, these are the ones that should be defriended).
Conclusions and Future Work
After struggling with the limitations of the Facebook API, I was able to implement a moderation system for Facebook news feed posts. This moderation system quantifies the contributions of each friend to the news feed and clearly identifies the friends with very good and very bad posts. Using this system, it is possible to filter the news feed based on friends’ moderation scores as well as filter the news feed based on previous moderation of similar content. It is also possible to extend the moderation system across a user’s social network, identifying friends whose posts are well liked by everyone.
Introducing FeedMod: Clean out your Facebook News Feed
In my last post, I wrote about the shortcomings of email and why I think it should be replaced to reduce spam and misuse. Someone then commented that he found Facebook spam to be much more of a pain than email spam, considering that Gmail did a solid job of spam filtering. While the focus of my post was more on curbing misuse rather than spam, the comment got me thinking. First, I wondered how this guy got spam on Facebook. I never get any poorly written posts on my news feed or in my inbox begging me to buy some Viagra knockoff. Then, I realized that perhaps the commenter’s definition of spam is different than my own.
Within the past few days, my news feed has been jammed with these “numbers” posts. For the most part, it’s been people who I haven’t seen or talked to in years posting a status update with a number followed by a random out of context comment that makes no sense to me. It’s been getting so bad that one of my Facebook friends posted that he would defriend anyone that posted another “numbers” status update. While these people aren’t trying to sell me Viagra, the effect is the same as email spam: annoyance and frustration.
It’s no surprise that the top reason for defriending is “frequent, unimportant posts.” Others, as well as myself, are getting frustrated with junk like these “numbers” posts and are on the brink of defriending the perpetrators. To solve this, I propose FeedMod, a Facebook news feed moderation system to remove the garbage and push the good stuff to the top.
Disclaimer
I have absolutely zero experience with the Facebook API, so I have no idea how much of this is possible. We’ll see.
How it works
My proposal is to borrow from the Slashdot moderation system. Facebook already has a “like” button, but it’s inconclusive that by clicking it, similar posts will be given priority in the news feed. There is also an “x” for each post, but clicking this seems heavy-handed, since it appears to blot out most posts from that particular friend. Instead, FeedMod participants will moderate each post with a positive, neutral, or negative descriptor. The results of this moderation is to flush out the good posts and hide the bad ones.
Unlike the Slashdot system, every news feed post can always be moderated by a FeedMod user. With Slashdot, the goal is to have publicly acceptable posts float to the top, hence the need for randomly selected moderators to avoid gaming the system. With FeedMod, however, the goal is to have posts acceptable to individual users float to the top, so users should be able to moderate everything in order to provide as much training information as possible. Perhaps later the moderation can extend across groups of friends so that certain Facebook users can be recognized as having posts liked or disliked by all of their friends.
Label each post as good, bad, or neutral
I would keep all the positive Slashdot post descriptors: informative, insightful, interesting, and funny. I would keep one neutral descriptor, such as “normal” or “neutral.” Also, I would modify the negative descriptors from what is used on Slashdot so that they are more appropriate for social networking and individual preferences: out of context, spam, unintelligible, and troll. Most news feed posts that annoy me either fall into the “unintelligible” or “out of context” categories: posts that make no sense due to poor writing skills or lack of background information since I don’t live every minute of my life with the poster. Stuff like the “numbers” posts would fall into the “spam” category and posts from friends looking to incite others (usually politics) would get labeled “troll.”
Customized News Feed
As a FeedMod user labels each news feed post with a descriptor, the news feed becomes customized to the user’s preferences. This is the tricky part, because I’m not sure if the Facebook API will allow me to rip out the text of each post and store it in a database. If I’m able to do that, I can try to model a user’s feed preferences based on the friends they mod as mostly positive or negative and the likelihood that a new post from a particular friend will be acceptable. I could go a step further and find keywords or topics that are found in good and bad posts, allowing new posts to be shown or hidden in the feed based on their content.
If all else fails, defriend
I should definitely be able to rank a user’s friends by the number of positive, neutral, and negative posts. That way, after a few weeks of modding posts in the news feed, a user can have a top 5 and bottom 5 list of the most liked and most offensive friends. Then, the user can just defriend those bottom 5 and see an increase in the news feed quality. There can also be rankings by each specific positive or negative descriptor as well as ranking by post volume.
What’s Next?
When time permits, I’ll try to figure out what the Facebook API is capable of and hopefully be able to attach moderation descriptors to each news feed post. With that, I’ll be able to find the friends whose posts I like the most as well as the least. Learning preferences will be more difficult, especially if the API has limitations on what can be accessed in a user’s news feed. Why Facebook hasn’t already implemented a better news feed moderation system is beyond me. I’m sure they’ve at least considered it.
Email is Broken: A Case Study
Facebook’s “email” announcement is generating quite the buzz. Some speculate that Gmail and Hotmail will soon be on their deathbeds. While Mark Zuckerberg describes email as “slow and formal,” there is a much more pertinent problem: email abuse and misuse. Anyone with an email account gets bucketfuls of spam every day. I know I do.
With email nearly as old as the internet, the protocol still in use today assumes that: 1) All users trust each other, and 2) All users have a moderate level of skill to send and receive email properly. The failure of the first assumption has led to the high levels of spam and phishing emails we see today. There is no trust: how does anyone know that a link supposedly send by a friend isn’t a botnet client? The failure of the second assumption is just as bad: how many jobs have been lost due to a hasty “reply all” click? The remainder of this post will focus on the second assumption: email misuse.
Mailing List Misuse
The timing of Facebook’s email announcement coincided with an incident that really demonstrates that it’s well past time for email to die. I have subscribed to a fair number of mailing lists throughout the past 15 years or so I’ve been using email. As per Wikipedia, electronic mailing lists fall into two categories: 1) Discussion lists, and 2) Announcement lists. Discussion lists, such as the TinyOS programming help mailing list, allow all subscribers to send questions to everyone else in the list. Someone poses a question in the email, puts the mailing list address in the “to” header, and once sent, all other subscribers get the email. Anyone with an answer “replies all” to the list so that everyone can benefit from the discussion.
Announcement lists, on the other hand, are only a one-way broadcast. Only a single owner is permitted to send announcements to all subscribers on the list with the recipients unable to reply to all. That’s the idea, anyway.
In September, I joined an alumni mailing list which I would estimate has at least several hundred subscribers. The list owner sent out regular updates at an interval of roughly once per two weeks, as illustrated in the figure below. However, following the most recent announcement, chaos ensued. On September 15th (see figure), a subscriber learned that he could “reply all” to the list and not only email the list owner, but all of the other subscribers as well. For whatever reason, the announcement list was set up like a discussion list, allowing any subscriber to reply. It only got worse from there.
Soon, a torrent of emails followed in the steps of the first subscriber as one person after another typed up a response and hit “reply all.” Being an alumni list, the discussion circulated among a group of individuals of about the same class year, discussing subjects that made no sense to anyone else. The responses quickly got out of hand: through the afternoon of September 15th, the sending rate soared to well past 1 email per minute.
Following a mad surge of random discussion on the 15th and another resurgence on the afternoon of the 16th, subscribers began to email the list complaining about the onslaught. One after another, again surging past the rate of 1 email per minute, subscribers demanded that they be unsubscribed and that the discussion be taken to a message board. Finally, the message was heard: everything died down and stopped completely by the evening of the 16th.
So what went wrong? First, the list owner should have ensured that he was the only one able to send emails to the list. Second, a significant number of subscribers emailed the list in an attempt to unsubscribe instead of emailing the list owner. Both mishaps plus the breach of protocol for announcement lists made things quite a mess in only a few hours.
Some other interesting statistics can be computed in the aftermath. Since the list’s inception, about 10% of the total volume can be attributed to announcements, 50% for misuse/discussion, and 40% for complaints about the discussion. Next, we show the age distribution of senders:
Since this is an alumni list, I was able to determine the age of just about every sender within two or three years, however, beyond some quick Googling, about 2% of the senders have an undetermined age. The above figure shows that those who abused the mailing list for discussion are all older than 45. Conversely, there is a roughly even distribution of age for those complaining. There is also slightly lower percentage of users under 45. From the figure, we can conclude that the younger generations are savvy enough to avoid either: 1) Misusing the announcement list as a discussion list, or 2) Sending the list an unsubscribe email instead of the owner.
Another interesting fact is that 37% of all senders used a business email address to reply to the list. 37% of senders used their business email for personal reasons! That’s insane. Another 9% had nasty hundred word “disclaimer” signatures at the end of their messages. Stuff like: “THIS IS A LEGALLY PRIVILEGED AND CONFIDENTIAL COMMUNICATION THAT IS INTENDED TO BE VIEWED ONLY BY THE INTENDED RECIPIENT…ANY DISSEMINATION, DISTRIBUTION, OR COPYING OF THIS TRANSMISSION IS STRICTLY PROHIBITED.” I think the distribution and copying prohibition went out the window the minute they hit “reply all.” It’s a wonder more people don’t get busted for stuff they put in emails.
In conclusion, this case study demonstrates exactly why email ought to be abandoned. There are too many avenues for abuse, misuse, and unintended consequences. It would be nice to see Facebook come up with something appropriate for today’s internet.
Rage Against the Cloud
I’ve posted previously about how much control third party and cloud computing services have over your information and how it will only get worse. Guess what: it just got worse.
For well over a year, I’ve imported the RSS feed from my blog to Facebook via the Notes application. This particular feature has been flaky in the past and now appears to be completely broken despite cries to fix it. I’m betting my bottom dollar that the poor implementation and maintenance of the blog import feature is deliberate on Facebook’s part: they want you to stay within their walled garden and keep all of your content solely within it.
Before last week, anything I posted on my blog would take up to three days to show up on Facebook. Occasionally, I would make several posts and they would all show up out of order several days later. To make posts show up immediately, I had to log in to Facebook and manually update the blog import. Several days is an eternity for a service that depends on real time information, especially when I get 20 or 30 wall posts every hour. Why should my blog posts be treated differently than wall posts? Google has real time search for the entire internet, but Facebook can’t keep up with a handful of RSS feeds for new blog posts? It’s obvious that Facebook could easily make an import feature that functions in real time and would allow you to import anything from anywhere. So why haven’t they done this?
Now, however, the Facebook blog import feature appears to be completely broken. After writing a blog post last week and then trying to manually import it, I got the following obtuse error from Facebook: “The blog/rss url you entered is not valid. “ I got no such errors from Feed Validator. I also thought it could be because I had upgraded to WordPress 3.0, but a discussion thread revealed that plenty of people with other blogging services were also having trouble importing to Facebook. After a week and a half, I have a hard time believing that Facebook would allow a bug like this to go ignored without some kind of acknowledgment or fix. What are they up to?
My guess is that Facebook deliberately broke the blog import feature, thinking that those who used it would just forget about it and start posting to Facebook directly. If true, it’s quite the subversive attempt to gain even more control over my information. I’m certainly not going to abandon my blog just because I can’t import it into Facebook. My guess is that the more restrictions like this that Facebook imposes, the more incentivized people will be to abandon their accounts. Changing privacy policies at the drop of a hat may not be enough to convince Facebook users to leave, but I’m betting that restricting users’ control of their own data will be the last straw.
Force Fed by Broadcast Media
If I see one more thing about Michael Jackson I’m going to puke.
As I was heading home from Rome, a guy in the seat in front of me had a USA Today with a full page spread and giant picture of Michael Jackson. I couldn’t read the text, but it was pretty obvious that he had died. Ever since I got home, every newspaper front page and every TV news station has been droning endlessly about Michael Jackson and rehashing every last detail pertaining to his death.
I don’t care and I don’t want to hear about it. It doesn’t interest me.
If I were biking somewhere outside of Williamsburg and crashed into a ditch and died where nobody could immediately see me, it could be a week before people even noticed I was missing. It might be up to two weeks before someone would think to file a police report and even longer before anyone would find my body. Few would care that I was missing or died, and such is the case for the thousands of people that die every day. None of them get news of their death plastered all over newspapers and television for days on end, and most of the world really wouldn’t care to hear about it.
The problem with this is that this is difficult to get away from when relying on broadcast media alone. Turn on the TV and it’s Michael Jackson. Turn on AM talk radio and it’s Michael Jackson. Open to the front page of the newspaper and it’s Michael Jackson. The good thing is that with the power of the Internet, and social networking, it’s easier to find alternative sources that give you the news you are interested in. The Internet is the great equalizer in that you no longer have to mindlessly read, watch, or hear what media executives want you to see. You can read about the topics you are interested in from a wide range of sources.
Those in broadcast media don’t like this paradigm shift. They file suits complaining of copyright infringement, claiming that they are losing money due to DVRs and piracy. The real issue isn’t as much monetary as it is control: broadcast media wants you to watch only what they feed you and only at the time they want you to see it.
Since broadcast media isn’t willing to embrace the Internet, they even try to restrict the freedom it brings in ways much more backhanded than traditional lawsuits. Recently, the New York Times worked with Wikipedia founder Jimmy Wales to stifle edits about a Times journalist being captured in Afghanistan. While hiding the news about the capture was meant to save the journalist’s life, if it were anything but a journalist that was captured, news of the capture would be blasted all around the world by the same news outlets trying to keep this case quiet. It’s as if those in the media industry think they are on a plane above everyone else. They want to think our thoughts for us and only show us what they want us to see. Most people just blindly follow along as evidenced by comments in blogs and Facebook. It’s time to stop being sheep.
I find myself watching almost no TV and reading the newspaper less and less, skipping through most of the stories and only reading the things I’m interested in. Personalized Google News and Digg do a halfway decent job of filtering out broadcast media’s force feeding. Maybe soon everyone will realize that plenty of other things happened today than the decision of who is receiving custody of Michael Jackson’s kids.
Losing interest in games
Posted by Matt in Uncategorized on October 29, 2008
It used to be that I would sacrifice hours of sleep in high school to play computer games every night. Many nights I would stay up past 11 playing Counter-Strike or the latest Command and Conquer. I could stay on the computer forever and then be a zombie in class the next day, nodding off as early as 9 or 10 in the morning. During the summers, it would be the same thing: suffer through a run in the late morning heat and then play games the rest of the day. I don’t think my mom cared for that, doing nothing most of the day, and was probably relieved when I started working after my junior year.
Those days are over and I don’t think they are coming back. Something happened when I started college that has really changed my interests in nearly everything. I’m not sure what it was or exactly how it happened, but it seems to have been a slow, gradual process that probably completed its transformation by the time I finished undergrad. I’m different now and I wish I knew why.
I want to play video games, but I just can’t get into it. I’ve started playing stuff occasionally, such as the latest episodes of Half-Life 2 and Portal. Those I finished because of their brevity, but longer stuff, like Call of Duty 4 and some recent RTS games I just started on, got stuck in one place and then just said: "screw it". None of them are nearly as exciting as they would have been eight or ten years ago. The most recent stuff that’s out there now would have been mind-blowing to me back then: the latest GTA, Fallout 3, and possibly Spore. I would have spent most of my life playing them, but not anymore.
It appears as though I’m not the only one that this has happened to. Some of the research says it’s to do with social networking; that everyone would rather play games with people they know rather than against unknown opponents online or a computer in single player. I do notice a correlation between the time I spend on sites like Digg and Facebook and the decrease in time I spent playing games. However, I still notice tons of people about my age and older that still play a fair amount: the assistants at PT talk occasionally about meeting up after work to play Halo, the physical therapist (in his 40s) even says he plays Madden on the weekends, my roomate plays plenty of PC games, and even my dad finished Crysis when I just dropped it after the zero-grav part.
In some ways, programming replaced the satisfaction I got from playing video games. It’s better because I have complete control over the environment. In video games, you are forced to live in the environment the developers create and play by their rules. It’s like the first Matrix movie: why play by the rules when you can bend them or even create your own? Why should I be restricted to use a shotgun to solve the problem of a developer-imagined monster when I can solve much more interesting problems of my own creation with tools of my own creation? Yesterday, I spent all day thinking about a job-scheduling type scenario for wireless sensor networks. Last night, I finally figured out a solution. While the problem wasn’t entirely my doing, coming up with the solution was much more in-depth and satisfying than blasting the crap out of a few enemy soldiers with a C-130 in CoD 4. And, my solution may not be the best. My adviser or the other guys in our group may come back with some constructive criticism to help with a better answer.
Again, I would really like to get back into gaming. Doing work can be satsifying, but it is also draining and it would be great to have some kind of release since I can’t run. Some of the stuff that’s out there right now sounds like it could be fun, but I don’t even know if my PC can handle it. Possibly, I’d have to upgrade the video card at the least, and of course that’s a slippery slope to just upgrade the whole desktop that’s been rusting under my desk. It’s too loud when it’s on and generates a ton of heat, so there’s a massive incentive to figure out how to make the whole thing quieter. Furthermore, I’ve only got one monitor that I use with my laptop, so I’d have to unplug it, the keyboard, and the headphones and put them all into the PC whenever I want to play anything. There’s also the cost issue: right now I’ve got to save since my surgery/rehab is expensive.
Maybe I’ll try once again to get back into gaming, and maybe I won’t. Maybe there will be another sea change that will shift my attitude and fun factor when I play, but I’m not so sure.
Back on Facebook
Posted by Matt in Uncategorized on March 31, 2008
I’ve decided to reactivate my Facebook profile. It seems that the big brother style ad “feature” died a quick death due to all the backlash, which was the main reason I left. Besides, it’s not like it was adding that much more time to my already wasteful internet browsing routine.
What’s more, I seem to be isolated from a lot of the things I used to do (practice) and am focusing more on other activities (school). Going back, it seems that I will be able to follow what’s going on in my old life a little better.
I think it’s going to be an interesting match between Facebook and OpenSocial with the focus being on user-generated content and custom apps/plugins. I wish I had some ideas to make some of my own. At this point, I can’t think of any itches that need scratching.








Recent Comments