Interviewed on MSDN Radio.

I have been interviewed on Swedish MSDN Radio! Last week Dag König, who is a Product Technology Specialist at Microsoft Sweden, talked to me about out Surface in general, Surface development and out plans with Surface at Connecta. The interview is in Swedish and is entitled “We put all the cards on the table with Microsoft Surface.” The show is over an hour and the Surface-stuff starts at about 27 minutes into the show and last for 20 minutes.

The interview

Surface training

When me and Jouni were at the Surface training in Munich Jouni wrote an application that uses the raw images from the cameras. The table consists of five cameras, four cameras that take pictures of the corners and one that takes a picture in the center. Then there is a process that goes from the raw images to a binary version of the image with each pixel either black or white for faster processing. The screen is divided into a grid containing one-inch squares. The next step is to place each frames contact data into shared memory and tell the SDK that data is available and ready to use.

The fact that cameras are used and not just a touch screen is what makes Microsoft Surface so special. Things that happen on the surface are recognized and even things that happen above the surface are noticed. About one inch above things starts to happen.

The course that we attended took place at one of Microsoft’s Technology Centers (MTC). There are three MTC in Europe, Dublin, Paris and Munich. On these centers Microsoft have three types of offerings, Strategy Briefing, Architecture Design Session and Proof of Concept Workshop. The center was really great even if the lunch was a logistical challenge. You can read more about the centers here: http://www.microsoft.com/mtc/default.mspx

I had to try Jounis application and the quality of the images that you get from the cameras with my face. "Thank god they can be cleaned was Dr Niels diagnosis."


The resulting image

Jouni in action

Love at first sight (at least attraction)

A very important new concept with Microsoft Surface is "Attract applications". These types of applications are screen-saver like, and are designed to entice people to the table. A good (and by all means fun) attract application should welcome people to touch and use the Surface table.

Water
The standard attract application that is shipped with Surface is called “Water”. Water might seem a a bit laggy when it’s run through the simulator, but on the table itself it’s beautiful and runs smoothly. The Water application can be customized a tad by using the Water configuration tool that comes with the Surface SDK.


Water Configuration Tool

Custom attract applications
It's pretty challenging to create your own attract applications with the Microsoft Surface Interaction Design Guidelines in mind. However, if you go for it strive for an application with very high interactivity and 360 degrees usability. You have to be able to entice people from all sides of the table - and keeping them there. To create a custom attract application, follow these steps:

1. Develop a Surface application.
2. Modify the application’s description file. (This XML file most likely has the same title as your project, at least if you’ve created the project by using the Visual Studio templates that comes with the Surface SDK). Change the file to look something like this:


Description file for an attract application. (Note: “ExcecutableFile” is the path to your build Surface application).

Another option is to just comment out the "application" section in the description XML file and un-comment the "attractapplication" section.

3. Deploy the XML file to "%ProgramData%\Microsoft\Surface\Programs". %ProgramData% is a pre-defined path.
4. The next step involves creating a registry key, so if you’re not familiar with handling the Windows registry make sure you create a back-up of the registry before moving on.
5. Add the following registry key "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Surface\v1.0\AttractMode"
6. Create a new String value to the key AttractMode (from step 5), and call it "CurrentAttractApplication". Set the value to your new attract application’s xml file name, without the file extension. I.e. the above “MyFirstAttractApplication.exe” would be the value “MyFirstAttractApplication”.

When you restart the Surface shell or Surface simulator your new attract application should load right way. Don’t worry if it doesn’t. If the registry key points to the wrong file, or you're having any other hard-to-guess error, the default application is loaded (normally “Water”).

Note: The Surface SDK tells you to “Open the registry and navigate to the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Surface\v1.0\ModeProfiles\Xxxx\AttractMode key (where Xxxx is the appropriate mode profile).” I can't get this to work, and also Dr. Neil talked about this at the Surface training I attended. Below you got an example of the suggested SDK (wrong) way, and one that works.


Doesn't work


Works

Surface training

On Wednesday last week, me and Jouni (the other writer on this blog) went to Munich in Germany to attend one of the first Surface developer courses outside of Redmond. Teacher was Dr Neil Roodyn from Australia. Dr Neil has been working with the Surface team for almost a year, developing applications and giving training. This was one of three occasions in Europe.


Dr. Neil Roodyn


The course was titled Building Applications for Microsoft Surface and lasted two days. We were 16 attendees and had two lovely Surface tables at our disposal.

Dr Neil talked a lot about the Surface vision and the history about the table, how it all started and how it came to be what it is today. The work with the Surface table started back in 2001 and in 2005 it got the current look and feel. In 2007 the first public pictures appeared and at PDC in 2008 it was made available to the public. If it took Microsoft seven years to get Surface to the public I can’t help thinking about what kind of projects they are starting right now that we will se more of in 2015!

It is really important to think of Microsoft Surface as something else then just a big touch enabled screen. Cause if touch is all that you are looking for then there are much cheaper solutions! Now the unique thing with Microsoft Surface is the interaction with what happens on and above the surface of the Surface.

Thanks to the vision recognition system Surface is able to detect things on top of the actual surface. About an inch from the surface things are starting to get picked up by the cameras. This information for instance used to detect fingers and the orientation of the finger.

As usual when you attend a course and have some knowledge of the subject, the first hours or the first day is not really challenging. Although Dr Neil managed to keep my interest and soon we got to start writing code and play with the tables.

One interesting thing to see on the table compared to the simulator is when you have recognized a tagged object and move it fast over the surface. The vision recognition system is not able to track that it is a tagged object that moves and it becomes a blob instead. But when you stop moving it is recognized as a tag again and the TagVisualizer is animated to the new spot. On the simulator you can move a tagged object with a TagVisualizer at any speed with no problem.

One part of the course was focused on unpacking, configuring and setting up the table. This part was really good since we haven’t gotten our table yet. It was also great to talk to Dr Neil about different deployment and development environment strategies. He had some interesting thoughts about how to use Live Mesh with Surface that I will try.

Anyway a really great course but I would have enjoyed more time coding and hands-on Surface-time!

Touchability by Contact Events

An event describes an action which occurs in a specific behavior. To add ‘touchability’ to your Surface controls, contact events has to be registered for them. There are a couple of ways to add such contact handlers. The following examples show how to register a handler for capturing a press-and-hold event:

XAML:
<s:SurfaceSlider x:Name="MySurfaceSlider" ContactHoldGesture="MySurfaceSlider_ContactHoldGesture" />

C#, first option:
MySurfaceSlider.ContactHoldGesture += new ContactEventHandler(MySurfaceSlider_ContactHoldGesture);

C#, second option:
Contacts.AddContactHoldGestureHandler(MySurfaceSlider, MySurfaceSlider_ContactHoldGesture);

The implementation of the contact event handler:
private void MySurfaceSlider_ContactHoldGesture(object sender, ContactEventArgs e)
{ /* TODO: Handle Contact Hold Gesture event */ }

There are several events that are possible to capture. The following section lists the contact events that are supported in the Surface SDK (version 1.0):

• ContactDown, which triggers when a contact is placed on a Surface control.
• ContactUp, which triggers when a contact is removed from a Surface control.
• ContactHoldGesture, which triggers when a Surface control is subject of a press-and-hold action. Note: Is only raised when the contact is of type Finger.
• ContactTapGesture, which triggers when a Surface control is being tapped on.
• ContactChanged, which triggers when a contact that is in touch with a Surface control changes its attributes, i.e. position.
• ContactEnter, which triggers when a contact enters a control’s boundaries, i.e. if a finger (that is already in touch with the Surface) is moved onto a control.
• ContactLeave, which triggers when contacts leaves a control’s boundaries.

I feel the ScatterViewItems’ ContactUp and ContactDown events needs a bit of extra information. ContactUp/Down are handled internally, which prevents you from raising any of the events. If you forget about this while working with ScatterViewItem this might be somewhat confusing. Another topic to keep in mind is the ScatterViewItems’ property ‘StaysActive’. If the property is set to ‘true’ this may affect the ScatterViewItem when a contact is released from it by the control not being deactivated.

Lastly I’ve posted a C# example of how to check what kind of contact is triggering a contact event.