While Facebook provides a “like” button to indicate approval of news feed posts, there is no indication that this feature is used to filter news feed content.  Facebook does provide an option to block news feed posts of designated friends, but this is a very coarse grained filter.  More recently, Facebook presented an interesting analysis of news feed posts.  This study identifies post topics that are more likely to be “liked” or commented on by friends as well as identifying post topics that are correlated with high friend  counts.  While the Facebook study analyzes post content, it only compares against metrics inherent to Facebook itself, such as “likes” per post, number of comments per post, age of posters, and friend counts.  The study neither attempts to quantify the news feed topic preferences of Facebook users, nor does it attempt to identify a Facebook user’s best or worst friends based on friends’ news feed posts.

Design

As previously mentioned, the main idea is for each FeedMod user to moderate each news feed post in a manner similar to Slashdot post moderations.  Each news feed post is assigned a moderation which consists of an integer value score and descriptor string.  For this experiment, I chose the following moderations:

Value	Descriptor	Examples
-2	Troll	Rants or politically charged arguments; I really can't stand these, hence the -2
-1	Spam	Advertisements, such as requests to attend an event, donate money, or to come to a friend's place of business to buy something
-1	Redundant	Endless duplicate announcements, typically on holidays and birthdays
-1	Indifferent	Stuff I really don't care about, such as random song lyrics or sports posts about teams that I don't follow
-1	Unintelligible	Something posted out of context, usually by someone with whom I'm out of touch
0	Normal	I assign my posts to this, otherwise I use it rarely
+1	Funny	Self explanatory, but I found myself only using it when I was about to fall out of my chair laughing
+1	Insightful	Anything that gives insight into the highlights of a friend's life or a well formed argument on a topic
+1	Informative	Usually updates on what someone is doing
+1	Inquisitive	Someone posing a question to his or her friends
+2	Interesting	Usually news articles or a post about something I hadn't heard of or thought about before (these posts are the best, hence the +2)

By moderating the status updates of all of my Facebook friends, I am then able to assign a score to each friend based on those moderations.  I can even moderate status update comments made by friends and non-friends alike.  The score, similar to Slashdot’s “karma,” is created by summing the moderation values for all of a friend’s posts.  In this way, I can see the balance of how positive or negative a friend’s posts tend to be.  The advantage of this method is that friends with a large number of good or bad posts will stick out.

Implementation and Facebook API

My original idea was to use the Facebook API to modify the news feed and allow me to moderate friends’ posts.  A month ago when I started this, I had no experience with the Facebook API and did not know if it would even allow me to do such a thing.  While I did achieve what I wanted in being able to moderate friends’ posts, I can say conclusively that the Facebook API is seriously lacking.  I have two major issues with the API: 1) an inability to customize the Facebook website, forcing developers to reinvent the wheel for even the smallest applications, and 2) limited access to information easily accessed on the main Facebook website.

What I really wanted to do was to be able to modify the existing Facebook news feed and insert a moderation select list next to each status update.  When a FeedMod user selected a moderation, the post-moderation pair would be stored in a database.  Unfortunately, Facebook does not allow developers to modify any portion of the Facebook website, including the news feed.  It would be much easier if Facebook allowed this since I wouldn’t have to recreate the entire news feed.  This lack of customization is a serious barrier to developers who have an idea for a simple application that makes use of the existing Facebook website.

The Facebook API does, however, allow you to query status updates and friend information for use on your own website.  What I learned was that apps within Facebook are actually remote websites running within an iframe on the main Facebook website.  Unfortunately, Facebook limits the information you can get: news feed items are restricted to status updates, links, videos, and photos — you cannot retrieve time ordered taggings, newly added friends, profile picture updates, and likes, among other things.

Interface

While I found the Facebook API disappointing, I was able to recreate the Facebook news feed on my own webserver, using the Facebook PHP toolkit.  Finally, with a database backend to store moderations, I was able to produce a customized news feed with the ability to moderate friends’ posts:

News Feed with Moderation Select Lists

To the right of each status update, I included a select list to moderate the post.  I also display the current moderation total for the friend, the friend’s moderated post count, and the percentile rank out of all friends (the friend with the most negative moderation total is 0%, the most positive is 100%).  This way, at a quick glance it is easy to see how many good or bad posts a friend has made to date.

I also created a statistics page, which ranks friends by the number of posts as well as lists of the top ten best and worst friends based on their moderation totals.  I also breakdown the moderation total and display the leaders of each moderation attribute (for example, who has the most “spam” posts).

Evaluation

After finishing the implementation, I then moderated posts for a month and now present an analysis of the results.  During a one month period, I moderated over 2,000 posts with 250 individual posters.  Some other interesting tidbits:

• 67% of my friends posted at least once within this one month period
• 64% of all moderated posters were friends; the remaining are non-friends commenting on a friend’s post
• 93% of all moderated posts were made by friends

Here is a breakdown of all moderated posts by their assigned attributes:

Moderation attributes by percentage of all moderated posts.

From the figure, a clear majority of posts are positive, while 35% are negative.  Out of all negative posts, the “indifferent” and “unintelligible” categories are used the most.  On the whole, I find most posts that I don’t like to be something I don’t care much about (and don’t want to read) or something that I don’t understand (usually it’s something taken out of context), hence the pervasiveness of the “indifferent” and “unintelligible” categories.  The “informative” category dominates the positive posts, followed by “insightful” and “interesting.”  My experience is that a significant number of posts (almost 40% to be exact) are status updates announcing what a particular friend is doing.  While most of these are mundane, I don’t really find them to be bothersome, hence the positive score.

Next, we look at how many posts each Facebook friend created during my one month evaluation:

Most friends post very little, but a few post all the time.

This figure shows the number of posts made by each friend that posted at least once.  A little less than 40% of my friends posted once, while about 3% posted more than 40 times in the past month.  One friend posted almost 120 times, or roughly four times per day!  Clearly, my news feed is dominated by posts from the same handful of friends.

Next, we look at the distribution of moderation scores across all of my Facebook friends:

Most of my friends have slightly positive moderation scores.

This figure shows that most of my Facebook friends (the ones that posted at least once) have a slightly positive moderation total. Since most of my friends do not post very much, the lack of extreme scores makes sense.  A small number of friends have very bad or very good scores — 3% have have a total less than -6 and another 3% have a total score greater than +20.  Those 3% that have the worst scores clearly stand out from the majority of my friends and are definitely good candidates for defriending.  Conversely, the top 3% go a long way in keeping my news feed enjoyable.

Lastly, we look at both post count and moderation total:

Friends that post more tend to be more positive, but there are some outliers.

Here, we see the quantity and quality of posts made by each Facebook friend.  Some friends have identical (post count, moderation total) values, hence the z-axis indicating the number of overlapping friends.  Like the previous two figures, this shows that most friends have slightly positive moderation totals and low post counts.  It also appears as though friends with higher post counts tend to be more positive.  However, there are some friends that buck the trend and have very negative scores for relatively high post counts (as identified previously, these are the ones that should be defriended).

Conclusions and Future Work

After struggling with the limitations of the Facebook API, I was able to implement a moderation system for Facebook news feed posts. This moderation system quantifies the contributions of each friend to the news feed and clearly identifies the friends with very good and very bad posts.  Using this system, it is possible to filter the news feed based on friends’ moderation scores as well as filter the news feed based on previous moderation of similar content.  It is also possible to extend the moderation system across a user’s social network, identifying friends whose posts are well liked by everyone.

I debated the pros and cons of paid vs. self hosting for awhile before I settled on hosting everything myself.  Unless I get a serious amount of traffic, it just isn’t worth the money to me when I can just run the whole thing on my own.  However, after manually updating the DNS records for my domain thanks to a service outage and new IP address assignment, I’m starting to wonder.

There are a few problems with self-hosting.  First, service outages are a real pain, especially in the summer.  The IT guys schedule weekend maintenance quite often in the summer and take the whole network offline for hours at a time.  Power outages due to storms compound this and even an uninterruptible power supply may not prevent any storm-related service outages.  Second, my IP address changes occasionally (usually due to the maintenance outages) and I have to update the DNS records.  This makes for even more downtime since it may take awhile before I realize that the IP has changed.  Lastly, I’m running this on a NAS, which makes access times pretty slow.  If you’ve wondered why it takes awhile for the page to load, this is why.  Sometimes the system will unspool the hard drive when it hasn’t been accessed in awhile, so it can take several seconds to spool back up and process the incoming page requests.  This, on top of the fact that the machine can just barely run Apache/PHP/MySQL.

The good thing about self-hosting is that it’s free with no bandwidth restrictions.  I can post as many massive pictures and videos as I want and it doesn’t matter.  I can use the domain name and the NAS as a massive storage archive accessible from anywhere.  I have complete control over the system and its configuration, so if I want something installed, I can do it without begging a hosting company to provide some feature (e.g. few hosts I know of provide Subversion repositories).  Of course I also have to take responsibility to ensure everything is managed properly so that someone doesn’t install a rootkit and/or deface the site.

So the balance between noticeable downtime and no restrictions continues.  For now, it’s acceptable, but I wonder if people ever get disappointed when they can’t get to my blog.  Probably not.

I use VLC as my default media player. During an Ubuntu upgrade (I think it was 8.04 to 8.10), video playback in VLC started flickering.

In VLC, go to Tools->Preferences and click the Video button. Change output to X11 video output. This fixed the problem for me. There may be some way to change the output system-wide if you use other players, but I’ve had no reason to do this. Also, I don’t think I’ve had any issues with Internet Flash videos.

The shop said to come back in a week or so to make sure it’s adjusted right.  I’ll have to do that since my IT bands on both sides have always tightened up when I bike.  I don’t want to take any chances with that.  I’ve also got to figure out how to clean everything, especially the drivetrain.  Every few months of riding I would try to clean up all the crap that accumulated on the Bianchi and relube the chain, but I was never able to get it as clean as when I would take it to the shop.  I’ve found some tips on the internet, but there wasn’t anything I hadn’t really been doing before.  One issue is that I don’t have access to a hose at the apartment.  Maybe if I get a chain cleaning machine it will be easier — sometimes I’ve just taken off the chain completely.  A bike stand would help too, but the expenses are piling up.  Regardless, it won’t be long before dirt and mud starts building up on the frame and drivetrain.

I’ve forgotten what it’s like to bike (or run) outside.  There are only a small number of good routes around Williamsburg, so I wind up doing the same thing almost every day.  I remember a lot of my teammates would start to complain about this in terms of running routes, especially towards the end of the spring semester.  We would wind up running the same old route a lot.  Someone would offer a bunch of suggestions of where to go, but they would always get shot down, and we would head out to the same trail that we had done for the last zillion maintenance runs.  Right now, it doesn’t really matter since it’s a relief to go away somewhere for awhile.

Unless you live in an area with subsidized broadband and/or wifi, you’ve got to pay an access fee to an ISP. At this point, dial-up is more or less useless, so you’ve got to fork over $40 every month just to get online. Like most people, your ISP is probably a large telecom, like Verizon, Comcast, or AT&T, so the true cost is probably a lot more than that since it’s almost impossible to buy internet access by itself. You’re forced into signing up for a “bundle” which is some combination of land line phone, cell phone, TV, and internet all provided by the same company.  A lot of the junk (and added cost) that comes along with the bundle is probably unwanted, like bizarre foreign language TV channels, call waiting, and hardware rental fees since the option isn’t often given to buy your own modem or cable boxes.  By purchasing a bundle from an ISP, you’re subsidizing the weirdos who actually watch some of those high-numbered TV stations, listen music on their TV, or enjoy being nagged by call waiting that can’t be turned off.  Since the number of providers available for a given location are usually pretty limited, you’re forced into paying monopolistic prices as well as paying for services that you don’t even want.

It’s bad enough that you’ve got to fork over the dough for services that you don’t even want in order to get internet access, but in truth, it’s a lot worse.  Access is merely a base cost for using the internet.  In nearly all cases, there’s also a cost for accessing content.  Somebody has to pay for hosting and generation of that website you frequent or the videos you watch, and again, it’s going to be you.  Advertising is one commonly used method to shift the cost onto consumers.  Nearly every website has it.  Until around five years ago with the advent of pop-up blockers, Javascript removal tools, and ad-blocking tools, every website I visited spammed me with a torrent of flashing banner ads, pop-ups, and keyword advertising.  Somehow, the notion goes, enough people would actually click on that crap and buy whatever it was that was being advertised.  The advertising revenue would keep the servers running and provide the blog/newspaper authors with a small paycheck.

Personally, I can’t stand advertising in general.  It gets in the way of whatever it is I’m trying to do, whether it be searching or browsing the internet, reading a print newspaper, or watching TV.  Reading the paper or watching TV probably adds about fifteen minutes of sifting through all the ads or waiting through commercials to get to the rest of the TV show I want to watch or the newspaper article I want to read.  I’ve never clicked on a banner ad, much less even bought something that was advertised in this manner.  On the internet,  it’s hard to manually filter out all the crap to get to what you want.  Fortunately, current ad-blocking tools do a pretty good job.  I’ve even forgotten how good a job they really do, for when I sit down in front of a computer without any ad-blocking software, the harassment of flashing banners and keyword ads drive me up the wall.

Though internet advertising revenue has increased in the last year, it is predicted to fall in 2009.  Everyone is finally getting sick of all the junk constantly being pushed at them as advertising approaches levels seen in “Idiocracy” and methods used in “Minority Report.”  A great argument against internet advertising states that it is “not trusted, not wanted, and not needed.”  While print newspapers are folding due to declining subscriptions, content providers on the internet are worried about a similar fate due to declining ad revenue.  Ironically, the print newspapers mainly blame their plight on the shift to electronic media.  To that end, content providers are considering increasing their usage of a second tool to provide revenue: yet another access cost.

Most online newspaper websites and online offerings of network TV shows do so with fairly low restrictions on who accesses their content. The front page stories for nearly every newspaper are available online as well as last night’s Lost episode.  This freedom of access is what makes the internet so great: once you get in, there are few barriers to access anything.  However, the same people that brought you bundled internet access want to change that.  One media executive says, “We want to change consumer behavior somewhat, so the expectation that everything online is free has to change.”  If this expectation changes, the internet as we know it is finished.  It is the freedom and openness of the internet that makes it as valuable as it is.  You can find anything or anyone and learn about nearly any topic available.  On the internet, you can collaborate with people half a world away.  The introduction of a second access cost for some internet content will most likely remove that openness.  The effort to preserve network neutrality has been in place for several years, but now it may be coming to an end.  Several ISPs are currently proposing pay access for channels and TV shows online.  The movement of this business model may force providers of other media to do the same.  Original sources will become locked down, leaving open and collaborative efforts to rot without them, such as Wikipedia, or IMDB.  Someone may want you to look at a video posted to YouTube, but you can’t look at it because you don’t have the money to pay the access fee.

Nobody really wants to pay a pile of access fees to get the latest news, or even to watch videos on the internet, especially when an ISP is charging a significant monthly fee just to get online (though some say they are).  The problem is that the annoyance and unprofitability of internet advertising is forcing a shift to another solution to prop up content providers.  If the shift to selling access to content goes too far, the internet may become segmented into a large number of tiers, causing the digital divide to span both the physical and electronic worlds.

Around the end of January, everything comes in and I start to set it up.  Lots of headaches ensue:

Problem 1: The Xbox 360 Pro analog and HDMI jacks are right next to each other, preventing the simultaneous use of HDMI and analog audio.  Both plugs don’t fit at the same time since they bump into each other.  I guess the MS designers thought that most people would use exclusively digital or analog AV, but not digital for video and analog other for sound.  There were plenty of complaints about this on the internet, and there are a few posts about tearing apart the Xbox analog plug so both HDMI and analog outputs can be used at the same time.  As it turns out, the Xbox 360 elite has space between the HDMI and analog jacks so both can be used simultaneously, but to me this “feature” isn’t worth the extra $100.  So, I’m using component instead, since the HP monitor has component input and I can’t tell the difference between that and HDMI.  On the internet, there are also arguments abound about HDMI versus component, so it seems that one is not necessarily better than the other.  I’ve heard that HDMI is noticeably better for really large TVs, but at 24 inches, my monitor doesn’t even come close.

Problem 2: I got the wrong adapter for the analog audio to mini-stereo.  I ordered the adapter I thought I would need to convert the analog output from the Xbox to the line in on my PC sound card.  As it turned out, I needed an additional analog AV female-female connector to make the whole thing work.  I also got a Sennheiser DSP 360 to act as a standalone amplifier, but so far I haven’t used it since the computer seems to be working fine.

Problem 3 (The Big One): The monitor came with a bent stand.  After I set it up and began using it, I noticed the monitor had a slight lean to it and the whole thing shook when I typed.  This was really annoying.  Provantage required that I call HP to try to resolve the problem first, and then go back to them for an exchange if necessary.   So, after two hours of phone tree hell and calls transferred between India and the US, I finally reach someone who handles the monitor I got.  Fortunately, the monitor I got is serviced by the small business department (not home), so the support I got was pretty good.  I got a case number to give to Provantage, since the HP tech said it would be faster if Provantage did the exchange and not HP.  I filled out an RMA request on the Provantage website, hoping this would be the case.  The monitor with the bent stand had no stuck or dead pixels and I asked the Provantage customer service agent if I could just replace the stand, but I wasn’t permitted to do that.  The whole thing had to be returned, which would come back to kick me.

Fortunately, the RMA went through and I didn’t have to pay for return shipping, but the whole process took forever.  It took about a week for the RMA to go through from the day I got the monitor, another week for the RMA to be processed, and another week still for the replacement monitor to arrive.  Last Friday, the replacement came and I picked it up, set it up (fortunately, the stand was not bent on the replacement), and turned it on.  As soon as the backlight came on, a red splotch appeared in the middle of the screen about the size of a quarter.  A smaller one the size of a dime was right below it.  On light backgrounds, I couldn’t see the splotch, but with anything dark or black, it was plainly visible, like glare from the sun.  It was really bad playing GTA IV, where it seems to be night almost all the time.

This time I decide to deal just with HP and forget about screwing around with another return from Provantage.  I’d read on the Hard Forum thread that a few other people who bought the monitor had recieved exchanges from HP, with a tech coming right to their door with a replacement.  I went throught the HP phone tree mess once again and talked to someone who said a tech would be dispatched with a replacement on Monday.  Monday came and went, but today, the guy showed.

The tech wheeled the monitor into the apartment and opened the box.  As soon as he opened the box, I knew something was amiss.  The stand was different.  When the tech took all the foam and packaging off the screen, I saw it was a different model: a 2465, not LP2475w, like what I had.  The specs were clearly inferior and didn’t have all the inputs of the 2475, so the guy said he would order the correct model and come back. I wonder how long that will be…

So, overall my new setup is great: with two monitors I have tons of desktop space for work, and the monitor works great with the Xbox, except for the red spot, of course.  Aside from the spot, the monitor is way brighter than the 20 inch Dell, even at 75% brightness.  The colors are much more vibrant.

This has been the biggest mess I’ve dealt with regarding ordering stuff online.  Poorly designed audio jacks and monitor defects have gotten the better of me, but I’m not giving up.

There is a more heavyweight solution, if you like Eclipse.  Since I’ve been using Subclipse with Eclipse on several machines to keep all my code in sync, I’ve found Eclipse to be pretty useful.  A TinyOS 2.x Eclipse plugin is available on this site that provides syntax highlighting and it appears as though it will compile and install source code as well (though I haven’t tried that part).

2/21/2009: Since I’m on the topic of Eclipse, I’ve noticed that the Ubuntu repository Eclipse version is 3.2, but 3.4 is available from the Eclipse website.  Following this guide, it seems that you can override the /usr/eclipse directory with the new version if you want to upgrade.  Plugins will have to be reinstalled, though.

I am aware that many contracts with corporate employers make you sign over any ideas or inventions you come up with using company time or resources. I’ve even heard of instances where things you come up with on your own time and own resources are still property of your employer. From what I’ve heard, life as a researcher in industry basically requires you to clam up completely about your work. Anything you work on is considered a trade secret of sorts and if it’s useful, the employer patents it. Little or nothing at all gets published for the outside world to see — most likely you won’t be making a name for yourself in industry. I can understand this mentality to a point: it’s important for a company to protect its ideas. Otherwise, a company might lose out big time if a rival gets hold of something and brings it to market first or changes it in an interesting or unplanned way to make a product better. However, I do think that a company claiming hold on an employee’s idea that was developed on his or her own time and resources is a bit overboard.

If industry guards its research, then academia provides its polar opposite. The concept of academia is to share and build ideas as a collective. In academia, you try to publish everything, to get ideas into the open and to leave your mark. It seems, however, that many academic administrations are clamping down on this free-thinking, open source mentality. In the New York Times article, several examples are cited of schools that are demanding royalties and claiming IP rights to their students’ ideas.

I can understand that without the school, the ideas you come up with as a student may not have been possible. To that end, if you come up with some killer app as a student, using school resources, the school could be entitled to some form of compensation if you try to market your idea. But, unlike the corporate world, it isn’t like the school stands to lose in some way if you try to sell whatever it is that you came up with. I suppose if your idea becomes successful and you leave school prematurely as a consequence, the school may lose out on publications you may produce as a student or tuition you may have paid. However, it’s not like you are leaving the school to become its direct competitor.

The article gives two contrasting examples of students who attempted to market their ideas. In the first example, an MIT student, while interning at NASA during a semester break, came up with some kind of posture detection system. He later tried to market the idea to help elderly people keep good posture and prevent falling. MIT demanded a $75,000 up front payment and royalties if the student wanted to use the idea. There must be more to the story than what is in the article, since if the idea was developed on break while at NASA, then I would imagine NASA would own the rights and any royalties would go there. Also, I suppose if any faculty members were involved then the MIT demands would make more sense. Otherwise, the school provided opportunity and resources, but I would argue the bulk of the effort in development came from the student, not the school.

A more reasonable example in the article references a group of students at RPI that developed a design for water bottles that could snap together to provide housing in undeveloped countries. RPI claimed 25% of any profit and $250 per year if the students decided to sell the idea. This seems to be more on par with the role of the school in the development of such an idea, again as long as it was the students that developed the idea and design.

All of this seems to boil down to the problem that if you want to break ties with your affiliated school or employer and go it alone, you’re going to get screwed. How do you start your own business if you were ever a student or employee? As soon as you try to sell your ideas, your past will come back and demand compensation. An IP lawyer in the article states: “The mission of a student is to research, get their degree, and move on with their life, and if you start fooling around with this stuff too much, it’s a total distraction.” This really gets me — obviously the lawyer has no idea what it means to be a graduate student. The mission isn’t to get your degree and move on with your life: the mission is to gain an understanding in some research area and develop your own ideas. You develop these ideas so that when you do get your degree, you have something to say for yourself. You use your research ideas as a selling point to employers, in academia or industry. But, it seems that no longer can you use your research in school as a starting point for going it alone without expecting to be penalized.