In the book, Kurzweil says that soon computers will be everywhere, even in the walls.  Within the sensor network community, I’ve heard similar arguments dozens of times.  The real question is: why do we need computers in the walls?  What is the underlying motivation for having computing intelligence in your clothes, in your desk, or in your dishwasher?  What can sensor networks do to improve your life?  Right now, it’s really hard to say.

Another researcher in the sensor network community likened sensor network research ideas as “hammers looking for nails.”  That is, solutions are invented before the problems are defined.  With emerging technologies, its hard to say immediately what their benefits are.  An interesting quote from a guy in the augmented reality community depicts how a new technology can sound cool but has little practical value:

“The first movies that ran could show anything, like an elephant in the zoo,” Meier said. “100 years ago it wasn’t about the elephant, it was ‘that thing is moving!’ Eventually it became more about the content…”

I would say that the current state of sensor networks is much like the above movie analogy.  Computing in the walls sounds cool, but what can small, networked devices in your walls do that really matters?  Few research papers offer any practical applications, since most sensor network papers focus on small fundamental problems: radio interference and MAC protocols, routing issues, and primitive event detection with sensors.  The few applications that exist in research papers aren’t exciting: activity recognition with body sensor networks (e.g. sitting, standing, walking), vehicle detection and tracking, and human health monitoring.

A few of these sensor network applications have made it into the mainstream media.  Lightweight health monitoring sensors may sound boring, but could provide motivation to stay in shape when your daily living habits are compared online with your peers.  Another project aims to put health monitoring sensors in soldiers’ underwear, with the aim that the underwear can release drugs and treat wounds.  One idea from IBM promises to have a computer at road intersections notify cars to turn off their engines and save fuel at lights.  Lastly, the smart home concept promises to network household appliances together to notify users of energy usage and to reduce energy consumption during peak time periods.

Whether the above ideas are really exciting or constitute the limits of sensor network capability is another matter.  I’m hoping that some really cool stuff will emerge in the future.  My bet is on participatory sensing and peer to peer collaboration with mobile phone-based systems, but like most people, I don’t have any specifics.

Monday 4/19/2010

On Monday I did my last run through the paths and trails behind KTH.  I caught up to another guy who was running at a pretty good clip.  We ran together for a bit, but then it was time for me to turn around.

The rest of the day was spent in a classroom at KTH set up by the faculty for those at the conference who were stuck.  In the room, I got some work done and checked the forecasts for the ash cloud.  The cloud had left Scandinavia for the time being, allowing Monday’s Stockholm – Chicago flight to leave, the same flight I was scheduled to take Tuesday.  Another guy I knew from the conference had stopped by and said his mom was able to leave that afternoon on an Icelandair flight to New York.  I took those two things as a good sign, but I was still skeptical that we would be able to leave.

Throughout the conference and the following days, I was quite impressed with the KTH campus.  The lecture halls were very nice with stadium-style seating.  Smaller classrooms provided large desks with plenty of workspace.  With 8,000 undergraduates and 7,000 graduate students, it accounts for nearly a third of Sweden’s university-level education.  It was quite larger than William and Mary, with many large lecture halls we used for the conference sessions, as well as large hi-rise style buildings.  Still, it was condensed into a space much smaller than many U.S. universities of its size.  I have no doubt that the excellent facilities were made possible in part due to generous government funding.

That night, my adviser and I ate at a Thai place down the street from the hotel.  When we returned, we packed everything up and noticed on the airline website that we were supposed to report to the airport by 7AM for check in.  Since it appeared that web check in was disabled, the airline probably wanted everyone at the airport early to check everyone in with plenty of time.

Tuesday 4/20/2010

On Tuesday I awoke at 5:30AM and miraculously discovered that Stockholm Arlanda was open and that our flight was still scheduled to leave.  I emailed Sarah back home that we were heading to the airport and she hadn’t even gone to sleep yet since it wasn’t even midnight.  My adviser and I left the hotel by 6AM.

The sun woke me up nearly every morning during the trip and we gained at least 30 minutes of daylight in the time I was in Stockholm.  There were automatic awnings that would cover the hotel room window in the afternoons and I realized that their purpose was to block the midnight sun during the summer months.  On the day we left, the sun woke me up at 5AM and the night before, it was twilight well after 8PM.

Another interesting note is that everyone in Stockholm seemed to follow the same schedule.  When we left for the airport yesterday at 6AM, there was barely anyone out on the streets.  However, during the conference the week before, when I would go out to run at 7AM, the streets were packed with commuters.  On the weekends, the city and KTH were deserted.  Even the university library was closed on the weekends and nearly empty after 6PM.

We made our way to the subway and then to the high speed airport train, which got us to the airport just before 7AM.  The international terminal had rows of check in counters, all but two were deserted: our flight and a Continental flight.  A TV crew was interviewing a few sparse passengers about the crisis.  Fortunately, unlike the massive Continental check-in line, our line was much shorter.  Apparently, many booked on our flight were Swedes who decided to stay home.

After being issued boarding passes, we hung out in the deserted gate area.  My adviser and I talked to a guy sitting next to us who worked for a company that produces industrial-grade robots and sensors.

Amazingly, we got on the plane and it left within 30 minutes of its scheduled departure time.  There were plenty of open seats and I was able to get a nice window seat with nobody sitting next to me.  There was plenty of legroom again, which made the 9 hour flight bearable.  The pilot announced that we passed over 100km from the volcano, but nobody could see anything out the window.  We passed over Norway, Iceland, Greenland, and Labrador, all of which consisted of rock, snow, and ice.  We passed over so much snow and rock, it made me realize how much of the world is probably uninhabitable and how fortunate I am to live somewhere relatively warm.

Though we left late, we arrived in Chicago only a few minutes after our scheduled landing time.  We had less than an hour and a half for our connection to Norfolk.  I rushed through immigration and customs and got on the train to the main United terminal since I had no boarding pass for the next flight.  Once in the main terminal, I tried to check in using an automated kiosk, which told me to see an agent.  After several minutes in line and little time left before the flight departed, I got to see agent.  The agents were interested to hear firsthand what it was like in Europe and Scandinavia.  When I got my boarding pass, I realized it was at a gate at a faraway concourse and ran with my bags for what seemed like a mile and a half to one of the last gates in the terminal.  With the clock ticking down, I raced around people in the jammed terminal passing gate after gate until I finally got to the Norfolk flight.

Oddly, when I got to the gate, my adviser was already there.  He had told me to go ahead after leaving the Stockholm flight because immigration might be slow.  Apparently, there was another check-in desk for United in the arrival hall for the international terminal, which would have sped things up for me.  Regardless, we both made the Norfolk flight, and by yesterday evening, I was home.

Sarah came to pick me up and on the way home I realized how nice the weather was: warm sun and leaves on the trees with green everywhere.  On the national news, there was coverage of people stuck in airports in Europe and in the United States, but more flights were running.

Overall, it was a good trip.  The conference went very well and I saw many good presentations relevant to my field.  I met a lot of new people from everywhere, and I saw a lot of Stockholm.  I really liked Stockholm — except for the cold, I could see myself living there, which is saying a lot since I don’t really want to live in a city.  The excellent infrastructure with subways, high-speed trains, fast and cheap internet, separated bike lanes, and large parks made Stockholm very appealing.  The people were very friendly and accommodating to visitors.

The getting stuck part was frustrating, but as evidenced by the news, it could have been much worse.  There are many others who are still stranded, but hopefully everyone will be back where they want to be soon.  I think a few things happened that really saved the day: 1) not booking any connecting flights in Europe, 2) using Skype to call the U.S. airline reservation number immediately after hearing the news about the volcano to get a new flight, 3) sticking to the rebooked flight and not attempting to take a train or bus to another airport in Europe, and 4) getting lucky with the ash cloud in Scandinavia.  Skype was a real help since we had no working mobile phone and international calls were cheap.

As my adviser and I discussed, this trip is one we’ll remember for quite some time.

Day 1: Saturday 4/10/2010

When I arrived in Norfolk, there was a mix up at the ticket counter – the web check-in boarding pass I printed the day before was invalid and it took some work for the ticket agent to print a new one.  When we arrived in Chicago that afternoon, it was a very long trip from the B concourse island to the international terminal.  We had to go under the ramp, into the main terminal, and take a train to the international terminal, a good 15 or 20 minutes.  Fortunately, the terminal was not busy and we were able to check in at the SAS counter without any issues.

The TSA agent checking my boarding pass at the O’Hare International Terminal commented about me being from Virginia; she said she had never been there.  When I travel to other parts of the country, I always get interesting comments from the locals about me being from Virginia.  In Palo Alto, a waitress commented that it was cold in Virginia, when at the time I remember it was really much warmer at home than in Silicon Valley.  When in Gloucester visiting an old roommate, I got a comment from his neighbor that I was from “Virginny.”  I’ve also heard another Massachusetts resident (who attends school at William and Mary) refer to “Virginny,” as well.  Hollywood always portrays Virginians with thick backcountry accents when the reality is nothing of the sort.  Some have the Tidewater accent, but it’s unnoticeable to the untrained ear.

Also, there were no restaurants in the O’Hare International terminal: 8 bucks for a pre-made sandwich.  Fortunately, we didn’t have to wait long until the Stockholm flight departed.  I noticed that nearly everyone waiting was Swedish, for there were no blue American passports that I could see.  Once on the plane, I realized we lucked out on the seat assignments and got extra legroom since we were just behind the premium economy section.

On the flights, my adviser and I talked about the direction of my career in school and afterwards.  Placement at a faculty job would probably require more time in school, perhaps up to another year, while a research lab may require less.  It’s all about gaining enough experience to do what you want to do.  It was interesting to talk to him about the possibilities since he has experience in both industry and academia.  Both have benefits and drawbacks and there is no way I want to close the door on any particular track just yet.  Just this week I talked with some other students in our department about the time required to complete a Ph.D., and those Master’s students involved in the discussion were wary of spending years in school to graduate.  I would not be opposed to spending another two years in school as long as I worked hard enough to get some good papers published in the hope that I could get a good faculty position at a research university.  From the looks of it, I have plenty of time to think about it and decide.

Day 2: Sunday 4/11/2010

As usual, I didn’t sleep much on the plane, only short bouts of 10 or 20 minutes.  The sun came up quickly and the clouds parted, giving us a view of jagged mountains and snow and little sign of life.  Before landing, the ground was covered up again by thick fog, which we soon descended into.

Though it was 7:30 in the morning, the airport was almost completely deserted.  Immigration and customs was a breeze considering we were the only flight arriving at the time.  A high speed train took us from the airport to the Stockholm Central terminal; a speedometer read well over 200kph as trees and buildings flew by.  I doubt any train at home travels anywhere near that fast.  From the train terminal, we took a subway two stops until we were near the hotel.   One thing I found odd was that the subway tickets had to be purchased from one of several convenience stores surrounding the subway entrance.  In the subway terminal, there were no ticket machines or windows and no signs indicating where tickets could be bought.  Instead, we had to ask someone at the entrance looking for freeriders where we could purchase tickets.

The city was mostly deserted, though it was a Sunday.  Stores were closed and nearly nobody was on the subway or walking around on the streets.  Even the school where the conference was to be held was nearly empty of students with the library closed.  Weatherwise, the day started out cloudy but later the sun came out, with temperatures in the lower 40s – pretty much a December or January day at home.  It’s a good thing I brought my heavy coat.

The hotel had a room available as soon as we arrived, which was surprising since it was only 9 in the morning.  My adviser and I slept for a few hours and then headed out in the afternoon, exploring Gamla stan and the area around the conference.  We walked by the parliament building and the Stockholm Palace, taking a ton of pictures.

To me, it seemed as if everyone was pretending that the weather was nice.  People at coffee shops sat outside at tables, wearing winter coats while being punished by the wind.  Others stood in lines at ice cream stands, despite the 40 degree weather.  Plenty more were out running and biking.  I suppose weather can be much worse in Stockholm.

The crowds picked up some in the afternoon, but there were few restaurants around our hotel so we settled on a Mongolian BBQ place.  I’ve been to similar places at home, but in the brief time I’ve been here there wasn’t anything interesting that caught my eye.  Just the usual McDonald’s and a few other places labeled “American Grill,” or “Steakhouse.”  I travel thousands of miles to get away from American food, among other things, but it seems I can’t escape it.

Day 3: Monday 4/12/2010

Though sleep was better than the previous night, I still had trouble.  I almost never sleep well on trips since it’s always hard for me to adjust to new environments.  I got up before 7AM and went out for a run beyond the university where the conference was, noticing there was a large park I could check out, so I headed that way and was surprised.  City blocks now packed with commuter traffic and sidewalks packed with people suddenly gave way to a vast forest with tons of dirt trails.  Plenty of other people were out running and biking.

I tooled around on the trails for awhile and headed back.  I was surprised that traffic would stop even if it looked like I was about to cross the street.   In addition to the trails, there were large paved paths with marked lanes for bikes and pedestrians.  Sweden seems to have solved the bike path problem so prevalent in the United States.  By making the paths wider and by separating pedestrians and cyclists, cyclists can cruise the bike paths without having to dodge pedestrians.  In the city, sections of curb separated bike lanes from vehicle traffic, also decreasing the chance of a bicycle accident.

On Monday, there were a handful of workshops; I attended one on “Cooperating Objects,” which appears to be a new buzzword in the embedded/sensor networks community.  A few people spoke about event detection and machine learning, with ideas similar to my research.  There were easily several hundred people, much more than at previous conferences I attended, people from all over the world.

At the reception Monday evening, I met Alexandra from Romania, who attended school in Slovenia.  It was interesting talking to her, considering that we are from very different and faraway places, yet we work in the same field.  We hung out quite a bit between breaks throughout much of the conference and through her I was introduced to a few others from Eastern Europe.  At school, it can feel very insular with few people to share your work and ideas with, but at a place like this, everybody is doing the same thing, and they come from everywhere.

Day 4: Tuesday 4/13/2010

Tuesday was a long one.  I was up at 6:30 to run, with plenty of daylight.  I explored a different part of the park than before, but ended up by a factory.  I’m still surprised at the number of trails in the city.

The plenary speaker seemed to be more of a biologist, speaking about human and animal brains as a control system.  There were quite a few talks in IPSN about machine learning and/or event detection so it was interesting to see others’ approaches to similar problems.  Later on, there was a poster and demo session which lasted until nearly 7:30 at night.

During the poster/demo session, I met a guy from the University of Utah which had concocted a sort of “x-ray” vision with 802.15.4 radios – I remember reading about this on the internet a few months ago.  Link quality between radios would change due to people moving about in the room, and with enough links, the moving people could be localized.  Apparently, his paper based on this was rejected mostly because nobody believed it.

Day 4: Wednesday 4/14/2010

On Wednesday, I hit the motherlode for trails when out on my run.  On previous runs I seemed to hit dead ends – running into roads, office buildings, or even factories, but today I found a trail that took me out to the rest of the park.  There were plenty more trails and open space by the time I had to turn around.  It was also sunny right from the start instead of morning fog.

The IPSN tracks were pretty interesting, again with a few on event detection and machine learning.   One group took a twist with a technique I used in my paper that I hadn’t thought of.  I also went to the CPS conference sensor network track, but didn’t really see too much differences compared with the typical sensor network research.

Day 5: Thursday 4/15/2010

I ran again in the morning, out to all the new trails I found the day before.  Unfortunately, the clouds returned.  At the conference, my adviser’s adviser, Jack Stankovic, was the plenary speaker.  During his talk, he used the example of a storm in Chicago as a reference to real time job scheduling.  The next day, I was supposed to return home via Chicago – hearing about any airline problem in Chicago was the last thing I wanted to think about.   This example eerily foreshadowed the problems to come for my return flights (through Chicago).

There were a few other sessions in the IPSN track that were interesting – at least one other event detection paper that was related to my work.  Then, in the afternoon, it was time for my presentation.  Since I was in the sensor networks track of RTAS, most people were in the concurrent IPSN track, so I didn’t get a huge audience.

Just before the presentation, I found out about the ash cloud coming from Iceland.  One of the other people in the room mentioned about flying back through Chicago the next day, as were my adviser and I.  He then mentioned something about maybe not getting back, and that’s when I found out about the ash cloud causing a huge mess in the UK and northern Europe.  It was headed to Sweden next.

With this on my mind, it was my turn to present.  I thought I did reasonably well and finished on time, except that I rearranged my slides just before the presentation and wound up having a backup slide placed ahead of my last slide, causing me to skip through it.

Following my presentation, my adviser and I locked ourselves in a discussion room in the university library and proceeded to call the airline reservation number to get a new booking.  The earliest we could get was the following Tuesday, over four days away.  Weather reports stated that the cloud might stay for days or weeks.  I walked back to the hotel and extended the hotel reservation until Tuesday while in the lobby other conference attendees scrambled to adjust their plans.

At the very least, I’ll get to see more of Stockholm and Sweden, though the weather isn’t supposed to hold up.  Fortunately for this week it has been fairly warm and sunny.  We’ll have to take it day by day to see if the ash cloud will dissipate enough to allow us to go home.  If Tuesday comes and goes and there’s no sign of the cloud letting up, it might be worth a try to head south via train or bus through Denmark to somewhere where I can fly out.

Like landline phones, Internet users will eventually drop wired ISP connections in favor of wireless.  This shift towards mobile and ubiquitous computing emphasizes that wireless infrastructure and usage policies will be critical in the coming years.  With few exceptions, ISPs have followed the principles of network neutrality with respect to wired networks, permitting wired customers to use any device with any software using any communication protocol.  However, such openness is not reflected in wireless networks with ISPs placing heavy restrictions on devices, software, and means of communication.  To ensure competitive pricing, hardware innovations, exciting software applications, and available bandwidth for the surge in wireless traffic, wireless providers must follow the principles of network neutrality.  Since wireless providers are hesitant do do so, the FCC’s decision to enforce network neutrality is a step in the right direction.  The enforcement of the FCC’s network neutrality principles will allow researchers to push mobile computing to new and exciting levels and will allow consumers to get more functionality at lower prices.

The openness of the wired Internet has seldom been encroached upon by ISPs and rarely regulated by government, making the Internet the world that it is today.  This freedom has been defined by the FCC’s four principles of network neutrality:

• A user can access any content over the network.  All who access the Internet are provided access to everything on the Internet, whether it be an AP news report or the Unabomber Manifesto.  No ISP restricts content: everything is available, even if it may be morally objectionable or illegal.
• A user can run any application or use any service over the network.  Anyone can use any web service or application (Google, Mapquest, Facebook) without restriction from an ISP.   No ISP prevents users from making Skype calls even if that ISP also sells landline telephone service.
• A user can connect any device to the network, given it does no harm.  An Internet user can connect with any hardware, whether it be with a ten year old piece of junk running Linux or with a $10,000 top-of-the-line quad core processor running the latest Windows 7 beta.  No ISP restricts a user’s Internet access because his or her computer is a piece of junk.  Nor do ISPs force users to connect only with proprietary computers sold by the ISP.
• Competition among network, content, and application providers.  A user is open to choose from one of several ISPs (if more than one are even available), and also has choice over competing web services and content providers.

This freedom has not always been maintained, even with the wired Internet.  Two years ago, Comcast deliberately throttled the bandwidth of peer-to-peer protocols, limiting the upload and download speed of file sharing applications.  With peer-to-peer using customers enraged over this finding, the FCC forced Comcast to abandon this policy.  Other than this incident and a few others, the Internet has always been delivered equally to all who have access.

Unfortunately, such open policies of the wired Internet are not followed by wireless providers.  From the very birth of cell phones, wireless providers have controlled everything:

• Unrestricted content.  Wireless providers block (through restricted software) bandwidth-heavy content, such as streaming videos, voice/video communication, as well as file sharing.  Such restrictions are in place to limit or prevent wireless network congestion.  Improvements in wireless infrastructure could alleviate such congestion, but wireless providers have decided to restrict, rather than improve.
• Use of any application or service.  Nearly all providers restrict the operating system and applications available on connecting mobile devices.  Apple blocks the use of Skype when the phone is not connected to a WiFi network.  Similarly, the Google Voice application was also removed by Apple and AT&T.  Both applications provide features that could be seen as subversive to standard cellular voice calls.  Instead of providing competing applications (maybe with better features), the ISP and application providers just block any competition.
• Connection of any device.  Verizon’s network may be good, but their phones are terrible.  A customer cannot just build their own tricked out phone and connect it to Verizon’s network.  Instead, users are forced to choose from a handful of phones with menial features (only sold by Verizon, of course).
• Competition. Since content, applications, and hardware are restricted on most wireless networks, competition is limited.  Only a handful of wireless providers offer large coverage areas, especially for data communication.  Since the United States pays more for wireless than any other western country, it is clear that less restriction would provide more options and more competition for consumers.

Despite the crackdown on allowable hardware, software, and access, wireless executives somehow manage to argue that their industry is “perhaps the most competitive consumer market in America.”  While unregulated competition would be great for consumers in terms of increasing wireless freedom, there simply isn’t enough competition to go around.  There are only four major carriers in the United States, effectively forming a cartel that can set prices artificially high and extensively limit consumer freedom.  With barriers to entry so high (infrastructure), few new carriers are likely to emerge.  In cases like these, government intervention is the only solution.  Fortunately, the FCC is investigating the lack of competition in the wireless market with respect to insane billing rules and lack of consumer freedom in comparison to the wired Internet.  Of course, their latest decision to enforce network neutrality will also provide a strong push.

The end result should allow anyone to build their own phone, install a custom operating system, and connect it to the wireless provider of their choice without issue.  Such a user should be allowed unrestricted access to the entire Internet without throttling or restriction of streaming video, large file downloads, or VoIP calls.  Ultimately, text messaging and voice calls will be merged into wireless data plans, removing such unreal and confusing costs like $.25 to send a 160 byte message or “anytime” minutes, which are anything but anytime.  Unrestricted hardware, software, and access will bring increased competition among existing wireless providers, fostering the development of better mobile hardware and cool applications.

A holistic computing solution

Everyone splits their computing time between a whole pile of devices: cell phones, media players, laptops, netbooks, desktops, and gaming consoles.  Even typical cable TV boxes have a hard drive and user interface.  The purposes and capabilities of such devices is becoming increasingly diverse and will probably continue to do so in the short term.  However, it’s becoming a real pain to use a standalone device for a different task.   Carry a cell phone for voice and text messages.  Carry a laptop or netbook for working on the go.  Carry a media player to listen to music.  When at home, many people switch from a laptop to a separate desktop PC.  It would be great to see some consolidation in order to reduce frustration with dragging around multiple devices and learning the quirks of each.

Since most people aren’t running weather prediction simulations, a single small mobile device would be sufficient for most people’s computing needs.  Ideally, something the size of a cell phone would combine the functionalities of a phone, camera, media player, laptop/netbook PC, and even desktop PC.  No longer would people with multiple devices need to synchronize information or become familiar with multiple interfaces.  A single device would provide a user with most of the computing power and capability that he or she would need without the hassle of dragging around a separate phone, media player, and laptop.  The device would be designed such that it could easily support and interface with different user input and output methods.

Improved user interfaces

Improved interfaces in mobile devices would make for a better user experience and improve productivity.  With multiple gadgets to haul around, each comes with a different interface with varying levels of usability.  For the most part, the smaller the device, the worse the user experience becomes.  A desktop computer with a keyboard, mouse, and giant monitor provides a solid experience for most, with both ease of input and output.  A netbook, however, may satisfy a user’s computing needs, but may cramp usability and productivity with its tiny screen and uncomfortable keyboard.  Even worse, web browsing and writing emails or text messages on some cell phones can be nearly impossible.  Personally, my experience with multi-touch phones has been horrible, since most of the time the phone selects something other than what I intended.  Rethinking and improving the physical and software interfaces would permit a shift to a cell phone-sized holistic computing device.

Improvements to both physical and software interfaces would provide huge benefits for the end user.  With respect to physical interfaces, improving multi-touch surfaces would be a big step in the right direction.  Input methods that are simple and accurate would make phone calls, text messaging, and web browsing on mobile devices much more enjoyable.   Output methods should extend beyond a tiny three inch screen, such as a wearable HUD or projector similar to that used in MIT’s SixthSense.  With regards to software interfaces, most cell phone operating systems provide horrible user interfaces which make the simplest tasks a real pain.  Most of these software designs inherit from heavyweight PC interfaces where input is made easy with keyboard and mouse.  Mobile developers should focus on increasing usability by making tasks require the fewest amount of user inputs possible.

Behavior and activity recognition

Computing systems of tomorrow could predict a user’s intentions and act upon these predictions.  With the introduction of accelerometers, GPS receivers, light sensors, cameras, and microphones in cell phones, plenty of research has provided ways to recognize user behavior and activities.  Such research can help provide an augmented reality for users, pointing out suggestions as to what a user could do based on his or her surroundings, current activity and learned preferences and behavior.  For example, a user traveling in an unfamiliar city could get instant suggestions as to where to eat when he or she normally takes a meal, with a mobile device providing directions to restaraunts that match the kinds of foods the user normally eats.  A HUD would allow the device to paint a path directly on the streets to take without the user staring down at his or her phone.  A device could automatically perform Internet searches and return data relevant to what a user is doing, whether it be retrieving a weather report before a user heads to the beach or providing real-time flight delay information as a user drives to the airport.  Such intelligent systems could interact with the physical world and turn on the lights or adjust the temperature at a user’s home before he or she arrives.    Also, behavioral and activity recognition would eliminate the need for user-generated Twitter and Facebook posts, performing automatic updates whenever a user changes activities or does something unusual.

Better inter-device collaboration

Figuring out how to get cell phone pictures off the phone and onto a computer can be a monumental task.  It’s even more enjoyable to get a projector to correctly display a presentation on a laptop.  Nearly every slideshow presentation I’ve witnessed, in classes and in conferences requires each presenter to wage war with his or her laptop and the projector to get the presentation to display properly.  Improving inter-device communication would make everyday computing more seamless and a lot less frustrating.   For example, a slideshow presentation could be loaded on a mobile device and a user could walk into a room with a projector, with the mobile device automatically connecting wirelessly to the projector and displaying the presentation.  There would be no cables to plug in, no display settings to modify, and no buttons and inputs to fidget with on the projector.  Synchronizing and moving data between different devices stands to gain significant improvement, for people are constantly upgrading their cell phones and laptops as well as sharing their data with others.

Longer design lifetimes

I’m guessing I’m not the only one with a pile of old hardware that’s worn out from too much use or discarded due to obsolescence.  More robust devices with longer expected lifetimes would reduce the pile of useless junk in the closet.  This would be especially helpful for mobile devices, which often wind up in mud puddles, toilets, or under someone’s steel-toed boot.  Modular construction would allow for periodic upgrades without throwing away the whole device.  An effort towards longer lifetimes and upgradeability would also significantly cut down on the amount of toxic e-waste.

Longer battery life

Most cell phones don’t last past  a couple days of standby or two hours of talk time.  Most laptops don’t make it past three or four hours.  With an increased focus towards mobile and ubiquitous computing, improvements in battery technology, power savings, and battery recharging would do wonders.  Apple has a new battery design in its latest laptops that double battery life, but more strides in this direction are needed.  Efforts on power-conscious radio communication and CPU utilization will contribute to power savings through better software.  Lastly, harnessing available energy sources such as motion, body heat, and the sun will also allow mobile devices to run unplugged for longer periods.

In general, these issues are what I think mobile computing needs in order to really take off.  The current experience is haphazard and lacking, requiring a mobile user to carry multiple devices with poor interfaces and short battery life.  With future applications, mobile devices will do a lot more than allow phone calls, web browsing, and text messaging.  They will further the integration of the cyber and the physical world, helping a user interact with the environment and the Internet in ways currently unimaginable.

The most visible privacy issues have been raised with the introduction of social networking: Facebook’s Beacon being one of the most infamous, with Facebook secretly collecting your online activities to generate targeted advertisements.  More recently, software designed to control and monitor children’s internet usage was revealed to actually transmit all internet activity, including instant messages, back to the companies that designed the programs.  The list of abuses and potential abuses goes on and on and will only get worse.  The introduction of cloud computing and the shift to thin clients mean that third parties are handling more and more personal data. The more opportunities others are given to handle our personal information, preferences, photos, browsing habits, and documents, the more opportunities exist for the abuse of that control.

Social networking and the monitoring of internet habits is only the tip of the iceberg.  The social networking concept is quickly moving beyond the confines of the Internet and integrating itself with the physical world.  The concepts of participatory sensing, body sensor networks, and smart homes/offices will see the introduction of internet-linked sensors placed everywhere.  Cameras and microphones will soon be on every street corner.  Temperature, humidity, and other energy monitoring sensors will be commonplace in every home, all linked to the internet.  Even now, most cell phones come equipped with GPS, accelerometers, and microphones, allowing for activity recognition and localization.  This “information saturation” will allow any developer to design an application that makes all kinds of weird discoveries: with body sensor networks and smart phones, rush hour traffic can be monitored in real time, local nightlife hotspots can be easily discovered, and suggestions can be made on how to save energy based on water and electricity usage in your home.

While the integration of the internet into the real world sounds cool, imagine what someone could do if your sensing information got into the wrong hands.  Would you really want your health insurance company to know your heart rate or blood pressure at any time of day?  Would you really want everyone to know that you waste the most water out of everyone who lives on your block?  Would you want a crazed stalker to know where you were at any moment or to learn your daily activities or routines?

Privacy and security is going to be an increasing concern as sensor networks become more commonplace and integrated with the internet.  While I was at UVA two weeks ago, Prof. John Stankovic mentioned that security and privacy in sensor networks is a huge problem and unfortunately, little is being done in this area.

A recent article by a UCLA student illustrated some of the problems with participatory sensing and presented some general solutions.  Disclaimer:  as part of the lead-in on her article, she says: “the developers I work with might say [my research area] is about telling them what they should be doing—which I must admit is the goal of this article.”  When someone says they know better than you, it’s time to run, not walk, to the nearest exit.  That said, I do agree with some of the things she says.  The author argues that a general framework should be designed for all participatory sensing applications that allow for user privacy management.  The frameworks should allow a user to easily understand how the systems work and how to control the release to the outside world of personal sensor data and inferences.  Ultimately, the author argues that data generated by a sensor network that you own is yours to control and distribute.  Third parties must respect the wishes of the content generators.

The author doesn’t go into fine-grained detail about how to ensure the preservation of an end user’s privacy wishes.  I could imagine some kind of CA could help verify that those third parties which access a user’s content/sensor data are who they say they are.  Some kind of feedback mechanism could allow a user to see where his or her sensor data went and how it was used.  If a third party abused a user’s data, the CA certificate could be revoked, effectively tarnishing the reputation of that third party.  Most likely, encryption would have to be introduced to ensure nobody but the permitted third parties could access a user’s data.  Unfortunately, encryption is very energy and bandwidth heavy for low power wireless sensors.

We all know the wrong way to go about privacy with the experiences of Facebook and its tacked-on privacy measures.  Initially, Facebook provided little privacy controls with no framework at all to allow a user to control access to his or her information.  Even now, the privacy controls for Facebook are hard to access and even harder to understand what they do.  A user has no fine-grained control over adjust who can access what content and exactly how your information is being shared with third party applications and advertisers.  A comprehensive and user-oriented privacy framework installed from the ground up would help stop the mishaps that are common with social networking.  Such a framework would come into its own with the increase in participatory sensing.

On that note, it’s time to get a few cell phones and start a participatory sensing application of my own to provide motivation for some research problems.  The possibilities are limitless and I can throw privacy to the wind (for now).  Unfortunately, as the author of the ACM article mentioned: researchers and developers don’t think about privacy, they think about research problems and cool applications.  In a small-scale research environment, the information is mine to control since I control the sensors, the application, and the release of any data or inferences from the data.  But, if anything were to be released into the wild and become popular, a framework for end user privacy control would be indispensable.  I certainly don’t want some nutcase knowing when and where I’m sleeping.

That said, it was an interesting trip, or sequence of trips, covering Boston for a paper presentation and then to Charlottesville for a Wireless Sensor Networks “retreat”. The first thing that happened started several days before I left to Boston, tripping on a downed branch in the street which nuked my ankle and plantar fascia, as I later found out. I got up when it was still dark out the morning I left for Boston and went running only to find my foot in incredible pain. It was the worst plantar fascia pain I’ve ever had, despite having trouble in that area plenty of times. So after getting back I found myself unable to walk without searing pain in my heel and barely able to limp around. This, right before a day of airports, subways, and walking around Boston.

Fortunately, the flights did go well this time around. Everything was basically on time going out and coming back from Boston. I left last Monday morning, connected through Washington, and took a shuttle flight to Boston. I had to take a bus from the terminal to the T station which was tricky since there were several buses that all looked the same but went to different places. The subway was straightforward but I did have to change trains once. It dumped me out two blocks from the hotel and I just walked limped from there.

In an effort to save the department some money, I went a day late.  It turned out that when I got to the hotel in Boston on Monday afternoon that there were no sessions scheduled for the rest of the day.  I had hoped to sit in on a few to see what everyone else was doing and to be sure that I had about 20 minutes to present.  Since nothing was going on, I dropped off most of my junk at the hotel and went outside to walk around, despite my heel pain.  I figured I might as well take advantage of being somewhere else besides school for a change.

Clearly, people in Boston know a good day when they see one and get out when it’s nice.  This was the first time in awhile that I had been there when it wasn’t winter.  The previous times, the Charles was iced over, snow piled up, and winds whipped through the streets.  Not many people were out then.  This time, I walked through Boston Common and it was packed.  I didn’t realize there was a pond that you could swim in.  I went up and down the streets near the hotel, found some food, and went back to the hotel to crash.  As tired as I was, I didn’t sleep.

My presentation was first thing Tuesday morning so I practiced the night before and hoped that the length would be about right.  I wanted to get up early and run in Boston before I presented, but I realized that my heel pain was too great to do that.  When I got to the conference room where I was to present, there was only one guy in there in charge of the projector.  Nobody else came, not even the session chair, until five minutes before I was to start.  After I got going more people showed up including another professor from William and Mary who also had a paper.  Fortunately, everything went well and I handled the questions without any real issues.

The next trick was getting back since I had to go to UVA the next day.  I was booked on a 3 PM shuttle flight from Boston to LaGuardia, but that would give me less than an hour to connect.  If there were delays, I could miss the connection and get stuck, as I experienced little more than a month ago.  I wanted to get on an earlier shuttle flight but when I booked the ticket, the system wouldn’t let me.  Interestingly, the book I was reading had a character that took a shuttle flight to LaGuardia and then was able to return anytime without charge since it was considered an “open” ticket.  I’ve also heard elsewhere that you could just change your ticket on shuttle flights if there was room.  Since the flight out had few people on it and the Boston shuttle terminal was nearly empty when I arrived, it seemed that changing my ticket wouldn’t be limited because of full flights.  I got the subway and bus back to the airport and got there early to see if I could leave sooner.  Nope, the gate agent demanded 50 bucks.  So much for all the hype over “open” tickets.  Another guy asked the same question, but apparently arriving an hour or two early was worth the money to him.

I sat in the empty Boston airport for several hours while two other shuttle flights left for New York, both with very few passengers.  Finally, when I got on my plane, I was assigned an exit row where the seat in front of me was removed.  I had double legroom.  I guess I was rewarded for waiting.  The flight back to Norfolk was a little late and I managed to get home at 8 that Tuesday night.  This time I really did crash and slept for over 11 hours.  Yet another big day was coming up.

On Wednesday I left home for Charlottesville and picked up the other student in our group, Zhen, from Williamsburg.  We got there Wednesday night and stayed at a hotel that was hopefully within walking distance of where the meeting was the next day and Friday.  Since I couldn’t run and was still in pain, I took my bike with me so I could explore Charlottesville in the early morning hours before everything started.  Biking was interesting since it was dark and there were a lot of big hills compared to Williamsburg.  On Thursday morning, my headlight came off its mount and shattered all over the pavement.  I had to get a new one.

My adviser told us that the retreat was to be held in the rotunda, which motivated the choice of hotel since it was within walking distance.  I had been to the rotunda before and it seemed weird since it was only really one room where tours were given.  I asked him several times if that was where it was, but when we got there Thursday morning the doors were locked and nobody was around.  I had the number of a UVA grad student which told us that it was at the business school, which was nowhere near the rotunda or the Computer Science building.  He couldn’t say where in the business school it was, and after looking at a map, the business school complex was huge.  We might never find where we were to go.  Somehow we managed to find the right room and building after running into a knowledgeable receptionist right after we walked in the door of the first business school building we found.  We got to the right room 45 minutes late.  Other UVA students laughed when I told them that my adviser said the meeting was in the rotunda.

The retreat was more or less an informal series of conference-style presentations.  Most everyone was a UVA student presenting on a work in progress.  There were lots of unsolved problems which led to all kinds of nasty questions.  It appeared as though some students didn’t practice since they went way over time or were cut off entirely with zillions of slides left.  The few that finished early were given plenty of discussion to fill in the gaps.  There were lots of interruptions, especially from the professors, and usually the adviser of the student would cut in with some difficult question about something that hadn’t been fleshed out yet.

The retreat went all day Thursday and then Friday until just after noon.  There were a lot of demonstrations, including that of a fall detection system for the elderly where the user would wear several accelerometers and gyroscopes.   One of the students with sensors taped to his shirt and legs fell down on the floor five or six times, which was pretty amusing, especially since the system didn’t indicate a fall until the last attempt.  There was a breakout session where my group discussed future applications and system designs.  An interesting thing that came out of that was the release of more iPhone/Google Maps-style APIs for third parties to make use of the deluge of data from sensor networks.  Someone commented that very soon you’ll find a microphone, panoramic cameras, among other sensors on every street corner, essentially providing a live Google Street View.  Imagine what someone could do with that: follow people remotely as they go about their day or generate a tag cloud of things people are discussing the most and break it down by location.  When I got home, I noticed that Google Maps now has a traffic congestion tool that uses GPS and speed readings from mobile users to determine if a road segment is congested.  Stuff like this that people currently consider to be invasive is going to be more commonplace and acceptable.

After all that I managed to get back home last Friday afternoon.  Since the traffic was such a mess in the tunnels, I wound up going through some nasty parts of Portsmouth to get around everything, something I hadn’t done before.  It took me as long to get from Charlottesville to Williamsburg as it did to get from Williamsburg back home.

Overall, the trip was probably worth some of the fatigue and heel pain since I gave a conference presentation and got a fair amount out of the retreat.  With the start of the fall semester, most the travelling is done for awhile and it’s time to get going on work that I haven’t been able to get to since I was gone.

Given the state of the art in machine learning, there is no way for an AI to learn on its own.  There is reinforcement learning, but even in this case a human must decide the conditions and the amount of a reward or penalty for each AI decision or output.  This is why AI in games is terrible: when there are a large number of non-deterministic game states and a large number of non-deterministic actions to take, it is almost impossible to determine the correct action to take at every decision point.  This means that it is difficult or impossible to provide the AI with labeled ground truth or a reward for training.  More to the point, labeling each output with the correct value would be a real headache.  Instead, game developers resort to a rule-based system that still has trouble covering every possible scenario.  As a result, NPC characters still wind up doing something weird, like running into walls.

I’ve been working with this stuff for awhile, realizing its capabilities and especially its limitations.  Then, this weekend I see a headline reading: “Scientists Worry Machines May Outsmart Man.”  Overblown media hype at its best.  The article concerns a conference on machine learning attended by the aforementioned Tom Mitchell and futurist Ray Kurzweil.  From what I gather, the conference dealt with mostly philosophical issues with respect to advancing technology and its integration with everyday life.  There wasn’t much about strong AI taking over the world, yet it was plastered all over the news that a Skynet-esque entity would rise from the Internet and doom us all.  In some ways, simpler systems have already taken over our lives: GPS tells us where to drive, automated tools read MRI scans and provide diagnoses, and viruses wreak havoc on our personal computers.  However, strong AI has quite a ways to go, with most machine learning research peaking decades ago.  As one Slashdot commenter on the NYT article writes:

Any computer scientist who is worried about AI taking over no longer deserves to be referred to as a computer scientist. The state of “artifiical intelligence” can be best described as “a pipe dream.”

All of this comes on the heels of a TED talk on the development of a brain simulator.  The speaker indicates his current brain “implementation” is running on a 10,000 core Blue Gene system.  The article gives few details, but it sounds like a large scale artificial neural network, which still needs supervised training data to learn properly.  10,000 nodes is still way too small, since the average human brain has 100 billion neurons with 7,000 connections each.  Maybe in ten years the requisite computing horsepower will be in place, but I’m guessing the algorithms and the intelligence will not.

In a similar light, a team of scientists recently used DNA computing to solve the NP-Complete Hamiltonian Path problem.  Instead of using some artificial construct or model, billions of DNA sequences, each representing a possible path, were randomly constructed such that those having a correct solution would glow a different color.  While massive parallelism makes this a relatively fast solution to an NP-Complete problem, this approach really isn’t a doomsday AI either.

While specific solutions continue to be discovered for our technological problems, development of strong AI (and the development of Skynet) will sit on the back burner.  Until then (and it’ll be awhile), everyone can take off their tinfoil hats.

Fortunately, the drive was mostly unused and had nothing really useful in it, while most everything remained safe on the primary WD Raptor.  The really important stuff is already backed up on my network drive or on my department disk space, so if anything else blows up I’ll still be okay.  When I removed the offending drive and tried to boot, only the power supply and CPU fans started spinning, leaving me with only a blank screen.  Somehow I managed to have unseated the video card from its connection to the motherboard and after I reseated it, everything returned to normal.

Quite a few people try to salvage failing computer components, but I don’t think it’s worth it.  The main idea is to patch stuff up just enough to transfer off any important data.  There’s the legend of  sticking the drive in the freezer overnight to get it working again.  Someone even got their video card working by baking it in the oven.  The fear of losing data is so great to some that there even exists a sound library of hard drive failures.

I don’t think my computer has much life left in it.  I don’t want to switch to my laptop for everything since it only has a VGA output and no digital connection, rendering my monitor useless.  I suppose I could get a docking station, but I can’t believe that Lenovo still makes laptops without digital outputs for external monitors.  Despite buying it last year, it even came with a pile of serial ports, a dial-up modem, and a PCMCIA slot that nobody would ever use.  A new desktop would be good for games but that was the purpose of buying an Xbox, and besides, high end desktops are still pretty expensive.  Of course something beefy would also be good for my schoolwork, since the last two projects had implementations that were extremely CPU intensive.

Regardless, I think I’m running on borrowed time.