Posts Tagged work

FeedMod: Quantifying news feed contributions of Facebook friends

Previously, I proposed FeedMod, a Slashdot-style moderation system for Facebook to weed out all the garbage posts while allowing the interesting stuff to stand out.  After a basic implementation and a month of gathering data, I present some results.  Through post moderation, I quantify the contribution of each Facebook friend to my news feed and identify the Facebook friends who provide the most positive and negative contributions.  Such information can quickly identify which friends to “defriend” and which to pat on the back for their good posts.  Such a moderation system will also allow for fine grained filtering of the Facebook news feed to provide a more enjoyable news feed experience.

While Facebook provides a “like” button to indicate approval of news feed posts, there is no indication that this feature is used to filter news feed content.  Facebook does provide an option to block news feed posts of designated friends, but this is a very coarse grained filter.  More recently, Facebook presented an interesting analysis of news feed posts.  This study identifies post topics that are more likely to be “liked” or commented on by friends as well as identifying post topics that are correlated with high friend  counts.  While the Facebook study analyzes post content, it only compares against metrics inherent to Facebook itself, such as “likes” per post, number of comments per post, age of posters, and friend counts.  The study neither attempts to quantify the news feed topic preferences of Facebook users, nor does it attempt to identify a Facebook user’s best or worst friends based on friends’ news feed posts.

Design

As previously mentioned, the main idea is for each FeedMod user to moderate each news feed post in a manner similar to Slashdot post moderations.  Each news feed post is assigned a moderation which consists of an integer value score and descriptor string.  For this experiment, I chose the following moderations:

ValueDescriptorExamples
-2TrollRants or politically charged arguments; I really can't stand these, hence the -2
-1SpamAdvertisements, such as requests to attend an event, donate money, or to come to a friend's place of business to buy something
-1RedundantEndless duplicate announcements, typically on holidays and birthdays
-1IndifferentStuff I really don't care about, such as random song lyrics or sports posts about teams that I don't follow
-1UnintelligibleSomething posted out of context, usually by someone with whom I'm out of touch
0NormalI assign my posts to this, otherwise I use it rarely
+1FunnySelf explanatory, but I found myself only using it when I was about to fall out of my chair laughing
+1InsightfulAnything that gives insight into the highlights of a friend's life or a well formed argument on a topic
+1InformativeUsually updates on what someone is doing
+1InquisitiveSomeone posing a question to his or her friends
+2InterestingUsually news articles or a post about something I hadn't heard of or thought about before (these posts are the best, hence the +2)

By moderating the status updates of all of my Facebook friends, I am then able to assign a score to each friend based on those moderations.  I can even moderate status update comments made by friends and non-friends alike.  The score, similar to Slashdot’s “karma,” is created by summing the moderation values for all of a friend’s posts.  In this way, I can see the balance of how positive or negative a friend’s posts tend to be.  The advantage of this method is that friends with a large number of good or bad posts will stick out.

Implementation and Facebook API

My original idea was to use the Facebook API to modify the news feed and allow me to moderate friends’ posts.  A month ago when I started this, I had no experience with the Facebook API and did not know if it would even allow me to do such a thing.  While I did achieve what I wanted in being able to moderate friends’ posts, I can say conclusively that the Facebook API is seriously lacking.  I have two major issues with the API: 1) an inability to customize the Facebook website, forcing developers to reinvent the wheel for even the smallest applications, and 2) limited access to information easily accessed on the main Facebook website.

What I really wanted to do was to be able to modify the existing Facebook news feed and insert a moderation select list next to each status update.  When a FeedMod user selected a moderation, the post-moderation pair would be stored in a database.  Unfortunately, Facebook does not allow developers to modify any portion of the Facebook website, including the news feed.  It would be much easier if Facebook allowed this since I wouldn’t have to recreate the entire news feed.  This lack of customization is a serious barrier to developers who have an idea for a simple application that makes use of the existing Facebook website.

The Facebook API does, however, allow you to query status updates and friend information for use on your own website.  What I learned was that apps within Facebook are actually remote websites running within an iframe on the main Facebook website.  Unfortunately, Facebook limits the information you can get: news feed items are restricted to status updates, links, videos, and photos — you cannot retrieve time ordered taggings, newly added friends, profile picture updates, and likes, among other things.

Interface

While I found the Facebook API disappointing, I was able to recreate the Facebook news feed on my own webserver, using the Facebook PHP toolkit.  Finally, with a database backend to store moderations, I was able to produce a customized news feed with the ability to moderate friends’ posts:

News Feed with Moderation Select Lists

To the right of each status update, I included a select list to moderate the post.  I also display the current moderation total for the friend, the friend’s moderated post count, and the percentile rank out of all friends (the friend with the most negative moderation total is 0%, the most positive is 100%).  This way, at a quick glance it is easy to see how many good or bad posts a friend has made to date.

I also created a statistics page, which ranks friends by the number of posts as well as lists of the top ten best and worst friends based on their moderation totals.  I also breakdown the moderation total and display the leaders of each moderation attribute (for example, who has the most “spam” posts).

Evaluation

After finishing the implementation, I then moderated posts for a month and now present an analysis of the results.  During a one month period, I moderated over 2,000 posts with 250 individual posters.  Some other interesting tidbits:

  • 67% of my friends posted at least once within this one month period
  • 64% of all moderated posters were friends; the remaining are non-friends commenting on a friend’s post
  • 93% of all moderated posts were made by friends

Here is a breakdown of all moderated posts by their assigned attributes:

Moderation attributes by percentage of all moderated posts.

From the figure, a clear majority of posts are positive, while 35% are negative.  Out of all negative posts, the “indifferent” and “unintelligible” categories are used the most.  On the whole, I find most posts that I don’t like to be something I don’t care much about (and don’t want to read) or something that I don’t understand (usually it’s something taken out of context), hence the pervasiveness of the “indifferent” and “unintelligible” categories.  The “informative” category dominates the positive posts, followed by “insightful” and “interesting.”  My experience is that a significant number of posts (almost 40% to be exact) are status updates announcing what a particular friend is doing.  While most of these are mundane, I don’t really find them to be bothersome, hence the positive score.

Next, we look at how many posts each Facebook friend created during my one month evaluation:

Most friends post very little, but a few post all the time.

This figure shows the number of posts made by each friend that posted at least once.  A little less than 40% of my friends posted once, while about 3% posted more than 40 times in the past month.  One friend posted almost 120 times, or roughly four times per day!  Clearly, my news feed is dominated by posts from the same handful of friends.

Next, we look at the distribution of moderation scores across all of my Facebook friends:

Most of my friends have slightly positive moderation scores.

This figure shows that most of my Facebook friends (the ones that posted at least once) have a slightly positive moderation total. Since most of my friends do not post very much, the lack of extreme scores makes sense.  A small number of friends have very bad or very good scores — 3% have have a total less than -6 and another 3% have a total score greater than +20.  Those 3% that have the worst scores clearly stand out from the majority of my friends and are definitely good candidates for defriending.  Conversely, the top 3% go a long way in keeping my news feed enjoyable.

Lastly, we look at both post count and moderation total:

Friends that post more tend to be more positive, but there are some outliers.

Here, we see the quantity and quality of posts made by each Facebook friend.  Some friends have identical (post count, moderation total) values, hence the z-axis indicating the number of overlapping friends.  Like the previous two figures, this shows that most friends have slightly positive moderation totals and low post counts.  It also appears as though friends with higher post counts tend to be more positive.  However, there are some friends that buck the trend and have very negative scores for relatively high post counts (as identified previously, these are the ones that should be defriended).

Conclusions and Future Work

After struggling with the limitations of the Facebook API, I was able to implement a moderation system for Facebook news feed posts. This moderation system quantifies the contributions of each friend to the news feed and clearly identifies the friends with very good and very bad posts.  Using this system, it is possible to filter the news feed based on friends’ moderation scores as well as filter the news feed based on previous moderation of similar content.  It is also possible to extend the moderation system across a user’s social network, identifying friends whose posts are well liked by everyone.

, , , , , , , , ,

No Comments

Another Weekend in Stockholm

It’s still doubtful that we’ll be able to leave again on Tuesday, but there is hope, since the Swedish airport authority said that they may allow flights into and out of Arlanda tomorrow.  Furthermore, the BBC weather maps predict the volcano ash will pass to the south of Norway and Sweden by the beginning of the week.  It’s a better than nothing chance that we could leave, but I’m not holding my breath.  Furthermore, the air crews and aircraft all have to be in place by Tuesday.

Yesterday, we took a trip to the Vasa Museum, which houses an absolutely massive wooden warship.  The ship was built by Sweden in the 1600s in its fight against Poland and sank as soon as it put out to sea.  It was recovered in the 1950s and restored over a 20 year period before being placed inside the museum.

I’ve seen plenty of wooden sailing ships at home, most were very small, but the Vasa is a giant.  Its size was probably part of the reason for its downfall — the prevailing argument is that it was too top heavy and did not have enough ballast.  From the keel to the top of the mainmast must be well over 100 feet: the ship seems like the Spruce Goose of its time.

Last night I went out with more stuck conference attendees to a bar near the waterfront.  The average price was about $10/pint, but “pints” here are a noticeably larger 0.5L (16.9oz), not 12oz as in the United States.  The place I went to had a large selection of beers, but not too many I hadn’t heard of.

Today we went out to the Royal Palace, meeting up with Alexandra, but this time we saw the changing of the guard, toured the insides of the palace, and the crown jewels.  Interestingly, the woman operating the ticket office commented on my William and Mary ID, for it seems the concept of royalty has a different meaning in places that have monarchs.  The changing of the guard involved a military marching band playing a few songs and marching around.  The Royal Apartments were ornate, but were no Versailles, although a room in the Royal Palace was modeled after the Hall of Mirrors.  The Treasury, which contained some of the crown jewels, was most interesting, with a guide explaining the historical significance of different crowns, maces, and orbs.

Since Alexandra was also stuck here with a canceled flight, we walked back to the train station to see if she could get a ticket back in the direction towards her school.  She estimated the travel time would be about 30 hours.  The train station in Stockholm was packed, with the ticket office handing out numbers to those who wanted to purchase tickets.  The line was probably several hours long, so we just left.  More than likely, trains towards Copenhagen were sold out for several days.  It’s fortunate that Europe has the rail system it does, for if air travel was stopped in the United States for more than a few days, it would be much worse.

Later, we went back over towards where the Vasa museum was to find the zoo.  We circled around it but could not find a way in.  Finally, when we reached the entrance, it was 10 minutes from closing.  We walked back along the waterfront and I headed back to the hotel.

While I was out today, I ran into several people from the conference.  I saw one guy from the hotel who mentioned that he knew of people heading south into Spain or Morocco to fly to South Africa and then catch a flight from there to the United States.  I ran into some others from the University of Illinois in a grocery store who predicted the winds would shift just in time to allow their Thursday flight to leave.

At this point, we will wait and see what happens with our scheduled Tuesday flight.  If it doesn’t go, then we will probably start to think about alternatives.  A group from the conference is considering chartering a bus to Madrid, but their airspace has been closed intermittently and I’ve heard the hotels there are booked solid.  There is also a ferry that runs from Denmark to Iceland: we could take a train to Denmark, take the ferry to Iceland, and fly back home from there.  Surprisingly, Iceland’s airport is allowing flights to and from the United States.

If we do consider finding an alternate route home, we will have to consider the costs of leaving a city that we know well with residents that speak good English and are pretty tolerant of foreigners.  We also have to consider the probability of something else going wrong when traveling overland long distances, connecting on different trains or even buses.  Bus and especially rail routes are extremely packed because of all that’s happened, with many being sold out for days.  By the time we travel a week and a half to get home via land and sea, the airspace may have opened up anyway.

At this point, time is starting to pass.  I haven’t done any work and I really should, at least to keep my mind off of being stuck here.  My adviser and I may go to KTH to find an office and get something done.  At the end of the week I am supposed to meet with my fiancee and the pastor for our wedding but I might not be able to go.  I wonder what would happen if I were stuck here for months or even for the rest of my life (I’m sure this won’t happen, but I am thinking hypothetically).  However, my sister has lived abroad for quite some time, so I’m sure I could adapt like she has.

, , , , , , ,

No Comments

Stockholm: CPS Week

While my Stockholm trip to attend CPS Week has gone well until today, I suddenly find that my travels are far from over.  Thanks to the ash cloud ransacking the UK, Europe, and Scandinavia, Sweden may become my new home for awhile.  I really seem to be a marked man when it comes to air travel.

Day 1: Saturday 4/10/2010

When I arrived in Norfolk, there was a mix up at the ticket counter – the web check-in boarding pass I printed the day before was invalid and it took some work for the ticket agent to print a new one.  When we arrived in Chicago that afternoon, it was a very long trip from the B concourse island to the international terminal.  We had to go under the ramp, into the main terminal, and take a train to the international terminal, a good 15 or 20 minutes.  Fortunately, the terminal was not busy and we were able to check in at the SAS counter without any issues.

The TSA agent checking my boarding pass at the O’Hare International Terminal commented about me being from Virginia; she said she had never been there.  When I travel to other parts of the country, I always get interesting comments from the locals about me being from Virginia.  In Palo Alto, a waitress commented that it was cold in Virginia, when at the time I remember it was really much warmer at home than in Silicon Valley.  When in Gloucester visiting an old roommate, I got a comment from his neighbor that I was from “Virginny.”  I’ve also heard another Massachusetts resident (who attends school at William and Mary) refer to “Virginny,” as well.  Hollywood always portrays Virginians with thick backcountry accents when the reality is nothing of the sort.  Some have the Tidewater accent, but it’s unnoticeable to the untrained ear.

Also, there were no restaurants in the O’Hare International terminal: 8 bucks for a pre-made sandwich.  Fortunately, we didn’t have to wait long until the Stockholm flight departed.  I noticed that nearly everyone waiting was Swedish, for there were no blue American passports that I could see.  Once on the plane, I realized we lucked out on the seat assignments and got extra legroom since we were just behind the premium economy section.

On the flights, my adviser and I talked about the direction of my career in school and afterwards.  Placement at a faculty job would probably require more time in school, perhaps up to another year, while a research lab may require less.  It’s all about gaining enough experience to do what you want to do.  It was interesting to talk to him about the possibilities since he has experience in both industry and academia.  Both have benefits and drawbacks and there is no way I want to close the door on any particular track just yet.  Just this week I talked with some other students in our department about the time required to complete a Ph.D., and those Master’s students involved in the discussion were wary of spending years in school to graduate.  I would not be opposed to spending another two years in school as long as I worked hard enough to get some good papers published in the hope that I could get a good faculty position at a research university.  From the looks of it, I have plenty of time to think about it and decide.

Day 2: Sunday 4/11/2010

As usual, I didn’t sleep much on the plane, only short bouts of 10 or 20 minutes.  The sun came up quickly and the clouds parted, giving us a view of jagged mountains and snow and little sign of life.  Before landing, the ground was covered up again by thick fog, which we soon descended into.

Though it was 7:30 in the morning, the airport was almost completely deserted.  Immigration and customs was a breeze considering we were the only flight arriving at the time.  A high speed train took us from the airport to the Stockholm Central terminal; a speedometer read well over 200kph as trees and buildings flew by.  I doubt any train at home travels anywhere near that fast.  From the train terminal, we took a subway two stops until we were near the hotel.   One thing I found odd was that the subway tickets had to be purchased from one of several convenience stores surrounding the subway entrance.  In the subway terminal, there were no ticket machines or windows and no signs indicating where tickets could be bought.  Instead, we had to ask someone at the entrance looking for freeriders where we could purchase tickets.

The city was mostly deserted, though it was a Sunday.  Stores were closed and nearly nobody was on the subway or walking around on the streets.  Even the school where the conference was to be held was nearly empty of students with the library closed.  Weatherwise, the day started out cloudy but later the sun came out, with temperatures in the lower 40s – pretty much a December or January day at home.  It’s a good thing I brought my heavy coat.

The hotel had a room available as soon as we arrived, which was surprising since it was only 9 in the morning.  My adviser and I slept for a few hours and then headed out in the afternoon, exploring Gamla stan and the area around the conference.  We walked by the parliament building and the Stockholm Palace, taking a ton of pictures.

To me, it seemed as if everyone was pretending that the weather was nice.  People at coffee shops sat outside at tables, wearing winter coats while being punished by the wind.  Others stood in lines at ice cream stands, despite the 40 degree weather.  Plenty more were out running and biking.  I suppose weather can be much worse in Stockholm.

The crowds picked up some in the afternoon, but there were few restaurants around our hotel so we settled on a Mongolian BBQ place.  I’ve been to similar places at home, but in the brief time I’ve been here there wasn’t anything interesting that caught my eye.  Just the usual McDonald’s and a few other places labeled “American Grill,” or “Steakhouse.”  I travel thousands of miles to get away from American food, among other things, but it seems I can’t escape it.

Day 3: Monday 4/12/2010

Though sleep was better than the previous night, I still had trouble.  I almost never sleep well on trips since it’s always hard for me to adjust to new environments.  I got up before 7AM and went out for a run beyond the university where the conference was, noticing there was a large park I could check out, so I headed that way and was surprised.  City blocks now packed with commuter traffic and sidewalks packed with people suddenly gave way to a vast forest with tons of dirt trails.  Plenty of other people were out running and biking.

I tooled around on the trails for awhile and headed back.  I was surprised that traffic would stop even if it looked like I was about to cross the street.   In addition to the trails, there were large paved paths with marked lanes for bikes and pedestrians.  Sweden seems to have solved the bike path problem so prevalent in the United States.  By making the paths wider and by separating pedestrians and cyclists, cyclists can cruise the bike paths without having to dodge pedestrians.  In the city, sections of curb separated bike lanes from vehicle traffic, also decreasing the chance of a bicycle accident.

On Monday, there were a handful of workshops; I attended one on “Cooperating Objects,” which appears to be a new buzzword in the embedded/sensor networks community.  A few people spoke about event detection and machine learning, with ideas similar to my research.  There were easily several hundred people, much more than at previous conferences I attended, people from all over the world.

At the reception Monday evening, I met Alexandra from Romania, who attended school in Slovenia.  It was interesting talking to her, considering that we are from very different and faraway places, yet we work in the same field.  We hung out quite a bit between breaks throughout much of the conference and through her I was introduced to a few others from Eastern Europe.  At school, it can feel very insular with few people to share your work and ideas with, but at a place like this, everybody is doing the same thing, and they come from everywhere.

Day 4: Tuesday 4/13/2010

Tuesday was a long one.  I was up at 6:30 to run, with plenty of daylight.  I explored a different part of the park than before, but ended up by a factory.  I’m still surprised at the number of trails in the city.

The plenary speaker seemed to be more of a biologist, speaking about human and animal brains as a control system.  There were quite a few talks in IPSN about machine learning and/or event detection so it was interesting to see others’ approaches to similar problems.  Later on, there was a poster and demo session which lasted until nearly 7:30 at night.

During the poster/demo session, I met a guy from the University of Utah which had concocted a sort of “x-ray” vision with 802.15.4 radios – I remember reading about this on the internet a few months ago.  Link quality between radios would change due to people moving about in the room, and with enough links, the moving people could be localized.  Apparently, his paper based on this was rejected mostly because nobody believed it.

Day 4: Wednesday 4/14/2010

On Wednesday, I hit the motherlode for trails when out on my run.  On previous runs I seemed to hit dead ends – running into roads, office buildings, or even factories, but today I found a trail that took me out to the rest of the park.  There were plenty more trails and open space by the time I had to turn around.  It was also sunny right from the start instead of morning fog.

The IPSN tracks were pretty interesting, again with a few on event detection and machine learning.   One group took a twist with a technique I used in my paper that I hadn’t thought of.  I also went to the CPS conference sensor network track, but didn’t really see too much differences compared with the typical sensor network research.

Day 5: Thursday 4/15/2010

I ran again in the morning, out to all the new trails I found the day before.  Unfortunately, the clouds returned.  At the conference, my adviser’s adviser, Jack Stankovic, was the plenary speaker.  During his talk, he used the example of a storm in Chicago as a reference to real time job scheduling.  The next day, I was supposed to return home via Chicago – hearing about any airline problem in Chicago was the last thing I wanted to think about.   This example eerily foreshadowed the problems to come for my return flights (through Chicago).

There were a few other sessions in the IPSN track that were interesting – at least one other event detection paper that was related to my work.  Then, in the afternoon, it was time for my presentation.  Since I was in the sensor networks track of RTAS, most people were in the concurrent IPSN track, so I didn’t get a huge audience.

Just before the presentation, I found out about the ash cloud coming from Iceland.  One of the other people in the room mentioned about flying back through Chicago the next day, as were my adviser and I.  He then mentioned something about maybe not getting back, and that’s when I found out about the ash cloud causing a huge mess in the UK and northern Europe.  It was headed to Sweden next.

With this on my mind, it was my turn to present.  I thought I did reasonably well and finished on time, except that I rearranged my slides just before the presentation and wound up having a backup slide placed ahead of my last slide, causing me to skip through it.

Following my presentation, my adviser and I locked ourselves in a discussion room in the university library and proceeded to call the airline reservation number to get a new booking.  The earliest we could get was the following Tuesday, over four days away.  Weather reports stated that the cloud might stay for days or weeks.  I walked back to the hotel and extended the hotel reservation until Tuesday while in the lobby other conference attendees scrambled to adjust their plans.

At the very least, I’ll get to see more of Stockholm and Sweden, though the weather isn’t supposed to hold up.  Fortunately for this week it has been fairly warm and sunny.  We’ll have to take it day by day to see if the ash cloud will dissipate enough to allow us to go home.  If Tuesday comes and goes and there’s no sign of the cloud letting up, it might be worth a try to head south via train or bus through Denmark to somewhere where I can fly out.

, , , , , , , , , , , , ,

No Comments

Going Mobile: Network Neutrality

The traditional approach for implementing solutions in wireless sensor networks is to use academically designed sensor motes, which provide an open hardware and software platform.  Such devices have been good for addressing fundamental problems, like radio irregularity, routing, data aggregation, and power savings, all of which require fine-grained control (open source) over the software and hardware.  While sensor motes provide a good proof of concept solution to some problems, more practical solutions are needed that are more attention-grabbing than a collection of weak, low power devices with limited sensing capabilities.  Basically, there aren’t a whole lot of applications for sensor motes that would be useful to most people.  A more recent approach is to use cell phones, which are increasingly becoming equipped with GPS, accelerometers, and microphones, providing a platform for interesting and practical wireless sensor solutions.  However, the hardware and software of most cell phones are extremely restrictive, clamped down by manufacturers and wireless providers, effectively limiting the research possibilities.  For example, one can’t just perform a clear channel assessment using a cell phone’s WiFi radio without less restrictive or open source drivers (even with Android).  Such restrictiveness is hindering improvements in mobile devices and many of these improvements would be welcomed by everyday consumers.

Like landline phones, Internet users will eventually drop wired ISP connections in favor of wireless.  This shift towards mobile and ubiquitous computing emphasizes that wireless infrastructure and usage policies will be critical in the coming years.  With few exceptions, ISPs have followed the principles of network neutrality with respect to wired networks, permitting wired customers to use any device with any software using any communication protocol.  However, such openness is not reflected in wireless networks with ISPs placing heavy restrictions on devices, software, and means of communication.  To ensure competitive pricing, hardware innovations, exciting software applications, and available bandwidth for the surge in wireless traffic, wireless providers must follow the principles of network neutrality.  Since wireless providers are hesitant do do so, the FCC’s decision to enforce network neutrality is a step in the right direction.  The enforcement of the FCC’s network neutrality principles will allow researchers to push mobile computing to new and exciting levels and will allow consumers to get more functionality at lower prices.

The openness of the wired Internet has seldom been encroached upon by ISPs and rarely regulated by government, making the Internet the world that it is today.  This freedom has been defined by the FCC’s four principles of network neutrality:

  • A user can access any content over the network.  All who access the Internet are provided access to everything on the Internet, whether it be an AP news report or the Unabomber Manifesto.  No ISP restricts content: everything is available, even if it may be morally objectionable or illegal.
  • A user can run any application or use any service over the network.  Anyone can use any web service or application (Google, Mapquest, Facebook) without restriction from an ISP.   No ISP prevents users from making Skype calls even if that ISP also sells landline telephone service.
  • A user can connect any device to the network, given it does no harm.  An Internet user can connect with any hardware, whether it be with a ten year old piece of junk running Linux or with a $10,000 top-of-the-line quad core processor running the latest Windows 7 beta.  No ISP restricts a user’s Internet access because his or her computer is a piece of junk.  Nor do ISPs force users to connect only with proprietary computers sold by the ISP.
  • Competition among network, content, and application providers.  A user is open to choose from one of several ISPs (if more than one are even available), and also has choice over competing web services and content providers.

This freedom has not always been maintained, even with the wired Internet.  Two years ago, Comcast deliberately throttled the bandwidth of peer-to-peer protocols, limiting the upload and download speed of file sharing applications.  With peer-to-peer using customers enraged over this finding, the FCC forced Comcast to abandon this policy.  Other than this incident and a few others, the Internet has always been delivered equally to all who have access.

Unfortunately, such open policies of the wired Internet are not followed by wireless providers.  From the very birth of cell phones, wireless providers have controlled everything:

  • Unrestricted content.  Wireless providers block (through restricted software) bandwidth-heavy content, such as streaming videos, voice/video communication, as well as file sharing.  Such restrictions are in place to limit or prevent wireless network congestion.  Improvements in wireless infrastructure could alleviate such congestion, but wireless providers have decided to restrict, rather than improve.
  • Use of any application or service.  Nearly all providers restrict the operating system and applications available on connecting mobile devices.  Apple blocks the use of Skype when the phone is not connected to a WiFi network.  Similarly, the Google Voice application was also removed by Apple and AT&T.  Both applications provide features that could be seen as subversive to standard cellular voice calls.  Instead of providing competing applications (maybe with better features), the ISP and application providers just block any competition.
  • Connection of any device.  Verizon’s network may be good, but their phones are terrible.  A customer cannot just build their own tricked out phone and connect it to Verizon’s network.  Instead, users are forced to choose from a handful of phones with menial features (only sold by Verizon, of course).
  • Competition. Since content, applications, and hardware are restricted on most wireless networks, competition is limited.  Only a handful of wireless providers offer large coverage areas, especially for data communication.  Since the United States pays more for wireless than any other western country, it is clear that less restriction would provide more options and more competition for consumers.

Despite the crackdown on allowable hardware, software, and access, wireless executives somehow manage to argue that their industry is “perhaps the most competitive consumer market in America.”  While unregulated competition would be great for consumers in terms of increasing wireless freedom, there simply isn’t enough competition to go around.  There are only four major carriers in the United States, effectively forming a cartel that can set prices artificially high and extensively limit consumer freedom.  With barriers to entry so high (infrastructure), few new carriers are likely to emerge.  In cases like these, government intervention is the only solution.  Fortunately, the FCC is investigating the lack of competition in the wireless market with respect to insane billing rules and lack of consumer freedom in comparison to the wired Internet.  Of course, their latest decision to enforce network neutrality will also provide a strong push.

The end result should allow anyone to build their own phone, install a custom operating system, and connect it to the wireless provider of their choice without issue.  Such a user should be allowed unrestricted access to the entire Internet without throttling or restriction of streaming video, large file downloads, or VoIP calls.  Ultimately, text messaging and voice calls will be merged into wireless data plans, removing such unreal and confusing costs like $.25 to send a 160 byte message or “anytime” minutes, which are anything but anytime.  Unrestricted hardware, software, and access will bring increased competition among existing wireless providers, fostering the development of better mobile hardware and cool applications.

, , , , , ,

No Comments

Force Fed by Broadcast Media

If I see one more thing about Michael Jackson I’m going to puke.

As I was heading home from Rome, a guy in the seat in front of me had a USA Today with a full page spread and giant picture of Michael Jackson. I couldn’t read the text, but it was pretty obvious that he had died. Ever since I got home, every newspaper front page and every TV news station has been droning endlessly about Michael Jackson and rehashing every last detail pertaining to his death.

I don’t care and I don’t want to hear about it. It doesn’t interest me.

If I were biking somewhere outside of Williamsburg and crashed into a ditch and died where nobody could immediately see me, it could be a week before people even noticed I was missing. It might be up to two weeks before someone would think to file a police report and even longer before anyone would find my body. Few would care that I was missing or died, and such is the case for the thousands of people that die every day. None of them get news of their death plastered all over newspapers and television for days on end, and most of the world really wouldn’t care to hear about it.

The problem with this is that this is difficult to get away from when relying on broadcast media alone. Turn on the TV and it’s Michael Jackson. Turn on AM talk radio and it’s Michael Jackson. Open to the front page of the newspaper and it’s Michael Jackson. The good thing is that with the power of the Internet, and social networking, it’s easier to find alternative sources that give you the news you are interested in. The Internet is the great equalizer in that you no longer have to mindlessly read, watch, or hear what media executives want you to see.  You can read about the topics you are interested in from a wide range of sources.

Those in broadcast media don’t like this paradigm shift. They file suits complaining of copyright infringement, claiming that they are losing money due to DVRs and piracy.  The real issue isn’t as much monetary as it is control: broadcast media wants you to watch only what they feed you and only at the time they want you to see it.

Since broadcast media isn’t willing to embrace the Internet, they even try to restrict the freedom it brings in ways much more backhanded than traditional lawsuits.  Recently, the New York Times worked with Wikipedia founder Jimmy Wales to stifle edits about a Times journalist being captured in Afghanistan.  While hiding the news about the capture was meant to save the journalist’s life, if it were anything but a journalist that was captured, news of the capture would be blasted all around the world by the same news outlets trying to keep this case quiet.  It’s as if those in the media industry think they are on a plane above everyone else.  They want to think our thoughts for us and only show us what they want us to see.  Most people just blindly follow along as evidenced by comments in blogs and Facebook.  It’s time to stop being sheep.

I find myself watching almost no TV and reading the newspaper less and less, skipping through most of the stories and only reading the things I’m interested in.  Personalized Google News and Digg do a halfway decent job of filtering out broadcast media’s force feeding.  Maybe soon everyone will realize that plenty of other things happened today than the decision of who is receiving custody of Michael Jackson’s kids.

, , , ,

No Comments

The Coast Guard and location prediction using Monte Carlo simulation

In the paper this morning, the front page article was on debris tracking and prediction for Air France Flight 447.  Software developed by the Coast Guard in Portsmouth is being used in the recovery effort to predict where debris is located.  Ocean currents are used along with the last known location of the plane to predict the most likely places where debris will be found.  A density map of location probabilities is shown in the article and I knew without reading it they were using Monte Carlo simulation.  This was confirmed in the article.

Using water current and Monte Carlo simulation to predict object positions in water?  This sounds really similar to our Sidewinder paper.  It’s strange enough that the people who did this debris location prediction practically live right down the block.

There are plenty of differences, though.  I would bet that the Coast Guard’s current model is much more advanced than a general group velocity with random deviations for all objects involved.  Instead of predicting debris location, we use Sequential Monte Carlo simulation to predict the location of a sink node in a mobile wireless sensor network.  The prediction is refined over multiple hops to make routing more reliable and efficient in a highly mobile environment, such as floating sensors routing data in a flood tracking application.  A similar density plot to the one provided in the paper is created at each hop for the estimated sink location.  This density plot becomes darker and smaller with each hop as the refinement occurs.  I’m guessing that the Coast Guard doesn’t use such sequential refinement.

So where’s my front page newspaper article on SMC prediction and flood tracking?

, , , , ,

No Comments

The mediocre call it obsession

But I call it passion.

Today, there was an article in the local paper about a guy, Paul Boyette, from Chesapeake who had a running streak that lasted over six years.  He ran at least two miles every day, running through injuries, terrible respiratory infections, and all kinds of weather.  His streak was finally ended by a torn meniscus which proved too painful for him to run through.  Oddly, this happened last summer, within a month of the pain in my knee becoming too great to run.  The article was really vague as to what happened after he stopped because of the torn meniscus, but one of the photo captions describes him running last month.  So, it appears as though he is back to doing what he loves.

The author of the article really makes this guy out to be an eccentric, hounding him because of his massive music and beer stein collections.  The author describes how Boyette’s family and friends gave him endless grief about maintaining the running streak and running every day though illness and injury.  Boyette is compared with a drug addict, as someone who has to get his fix or else he can’t function.  His stacks of meticulously kept running logs and piles of worn out shoes are referenced as paraphernalia, aiding the addiction.  A few commenters for the article say the guy is addicted to endorphins.

It isn’t addiction.  It isn’t obsession. It is passion.  Over the years, I’ve been hounded by people saying the same things, over and over:

“Why are you running in this weather? It’s too [cold/hot/rainy].”

“If you’re tired, take a day off.”

“Why don’t you try doing something else instead of running?”

The mediocre, the average, the run-of-the-mill Joe Six Pack only cares enough to go through the motions with any activity.  These people, which make up a large majority of the population, see those who put all their effort into something as obsessive.  The mediocre only wish to do good enough and to quit early.  They will never feel good about any of their accomplishments, yet they simultaneously envy and criticize those who work extremely hard and become successful.  The mediocre are everywhere and would love nothing more than to witness the failure of those who actually try.  To that end, the mediocre spread their negativity and criticism to those they know that are successful, secretly wishing their endeavors will implode and their work will be for nothing.  The author of the article and the commenters that follow are prime examples of those who just don’t get it.

People scoff at Bill Gates for sleeping in his office during the early years of Microsoft, working at his computer until he collapsed onto the floor with exhaustion.  Though the mediocre label him as obsessive and wish to see Microsoft wiped from the earth, every single one of them has used a Microsoft product.  If it weren’t for Gates’s efforts, this large scale success would never have been realized.  Computing would never be what it is today without those “obsessives” working until they drop on the floor.  Still more people can’t believe that the efforts Adam Savage of Mythbusters undertook to recreate an exact duplicate of the Maltese Falcon.  He spent months of work researching the prop from the film, drawing sketches, making several mockups, and eventually getting his hands on the original prop to create a near flawless duplicate.  In the video, Savage’s passion for his work really comes out as he speaks quickly and excitedly about every last detail of his quest.  You don’t see that with the mediocre.  You can tell when someone is passionate about their work when they speak of it like Savage, and you can tell when someone really just doesn’t care.  I’ve seen professors, other grad students, running teammates, and internship co-workers speak with the same excitedness as Savage when they speak about their latest accomplishments.  The passionate can pull you right into the hype.

It disappoints me to see that most others criticize the concentrated efforts of those like Bill Gates and Adam Savage while it is efforts such as those that keep the world in one piece.  Without them, this world would be a bunch of slackers.

So why run until the pain becomes unbearable?  Why work until you fall asleep at your desk?  It is because not doing so would be a failure.  I ran every day I could because I wanted to make the best of what I had.  I ran every day and through all weather and many illnesses and injuries of my own because I knew there would be a day where the problems would be too great to run.  On that day when I finally couldn’t run, I could look back and know that I had done my best.  On the day I couldn’t run, I would have no regrets about the past.  I would not have to wish that I had gone out for a run on a day that I was only too lazy to go.  That day came last summer when my knee made running unbearable.  Like Boyette, I had to stop.  I had no regrets: I knew that I had made the best with what I had.  Those who are mediocre will regret the day that they are unable to do something because of circumstances beyond their control.  They had the chance to give it their all, but they didn’t make use of it.

I hope that I never again get criticised for biking or running as much as I can or for doing work when someone wants me to go out.  It is what makes me who I am and it is not a disorder that should be treated with magic pills.

, , , , , , , , , , , ,

No Comments

One reason grad school is different

My sister (and others) have been complaining about the gobs of work they’ve got to get done during exam period. There are papers, projects, and of course, exams. My case is somewhat different. I remember how it seemed all kinds of massive projects and papers were due right before or during exams, and then I would have to suffer through a whole ton of exams. It was a huge weight off my back when it was done.

But now, it’s almost worse. I started working on my current project in September, right after my knee surgery. It’s been going for nine months and now I am finally cramming all of those nine months into ten pages or less, double column.

To put this in perspective for an undergrad: take all the papers, all the projects, all the homework assignments, and all the exams, tests, and quizzes in the last two semesters and make them all due next week. That is the weight of what I am working on. It nags at me every day that I have been working for so long and have nothing to show for it. I think about it in bed before I go to sleep, I think about it when I wake up, I think about it when I’m in the shower, and I think about it when I’m on the bike. As an undergrad and even for the first part of grad school, I got closure incrementally: with periodic assignments and tests and at the finish of each semester when classes end. Closure is now when I get a paper out, and the time span for that seems indefinite.

My first project and paper was faster — it took a semester and a summer. However, I’m still dealing with that project now — I’m going to present it at SECON and when I do, it will almost be a year since the first version of the paper went out.

It seems most undergrads treat papers lightly: “Oh, I can crank out a ten page paper in a few hours and still get an A…” I was the same way — it was spit something out as fast as possible to get it over with, but put enough effort into it to get a decent grade. Now, papers are everything. The paper is how everyone else sees your work. I may have spent the past nine months creating something that could have huge implications for the future of wireless sensor networks, but nobody but my adviser would know about it unless I tell them in a paper. The reviewers will lay the smack down on you if you try to whip out a paper in a matter of hours. Yesterday, I spent six hours writing and got out about five paragraphs. They were five critical paragraphs about the core of what I did, and they had better be comprehensive and understandable from the perspective of an outsider.

I find it hard to tell the story of my project, but I think the difficulty is from inexperience. There are key things that reviewers look for that I must give special attention in addressing. Because of this, there is a pretty rigid way to write a research paper, but even then I find it difficult. It is hard to convey specific algorithmic details, yet be concise and easy to understand. I have to remember the main selling points of my work and refer back to them throughout the paper. For these projects, I find the initial problem discovery, solution design, and implementation to be interesting and sometimes even fun. Enough experience programming and thinking about solutions to problems has helped with that, but I haven’t done much writing.

With time, I imagine writing will get easier. I will know exactly what to do. With my current paper, I’m almost there. I’m almost to the point where I feel good about what I have written and know that everything will turn out okay. I just have to keep working and get it done. Then there will be some closure.

, , , , , , ,

No Comments

Ubuntu 9.04: Rough Upgrade

I recently upgraded from 8.10 to 9.04 on my desktop machine, and so far, it’s been a mess.  This was not the painless upgrade from 7.10 to 8.04 or from 8.04 to 8.10.  So far:

1. ATI/AMD is no longer providing drivers for my X1900XTX video card, so I’m using the free drivers.  The problem is that I can’t use two monitors without the mouse flickering like crazy and some strange window focusing problem where an unfocused window “jumps out” at me occasionally and then reverts to normal.  Disabling compiz and visual effects didn’t solve this.  Only when I disable the second monitor do the flickering and focusing issues go away.  I’m still messing with this one to try and find a solution, but there isn’t anyone else on the internet that’s complained about this.  I find it funny that people switch to Linux so they can run older hardware, but without driver support, that can’t happen.  The free drivers work well, but only if you’ve got one monitor.

2.  Something hosed my ability to play MP3s with Amarok. From this thread, I learned that running the following solved the problem:
sudo apt-get install phonon-backend-xine

3.  Flash stopped working in Firefox.  Any page with Flash would crash Firefox.  This can be solved by uninstalling and reinstalling Flash:
sudo aptitude remove flashplugin-nonfree flashplugin-installer
sudo apt-get install flashplugin-nonfree

4.  My Eclipse installation randomly crashes.  Also, the splash screen never goes away after starting up.  I’m in the process of reinstalling Eclipse to see if this does anything. Edit: reinstalling appears to have fixed the crashing and splash screen issues.

This hasn’t been an enjoyable experience so far — I was under the opinion that Linux is reaching maturity, especially Ubuntu, but this is definitely not the case.  The problems I’m experiencing seem to be a step backward from the last upgrade cycle.  As I just mentioned, the ability to run fast on older hardware is a huge enticement to switch to Linux, and if my three year old desktop can’t handle it, there’s a lot more work to be done.  I’m certainly not going to buy a new video card that’s worth much more than the rest of the computer.

, , , , , , , ,

No Comments

Published

My past year and a half of research-oriented work has resulted in a conference acceptance.  This summer, I’ll go to IEEE SECON in Italy to give a presentation on:

Sidewinder: A Predictive Data Forwarding Protocol for Mobile Wireless Sensor Networks

In the coming weeks, I’ll have to revise the paper somewhat for publication and then prepare a presentation.  As a M.S. student, a lot of my classes involved reading papers and giving class presentations on them.  I hadn’t thought much about it at the time, but it really sets everything up for the next phase: presenting your own research.  It will be interesting (and challenging) to face down some tough questions on my paper from people all over the world.

In the meantime there’s a whole bunch of other work I’ve got to take care of on a current project.  Things are about to get interesting.

, , , , ,

No Comments