PeterKellner.net

Tonight, at the Microsoft office (see this announcement for more details: http://www.meetup.com/The-San-Francisco-NET-User-Group/events/21982451/?a=cr1p_grp&rv=cr1p ) I’m going to be talking about what it is really like to build a full service application using Microsoft’s Windows Azure Cloud Computing Platform.

For those that don’t know what I’m going to talk about, here is a prelude.

First, I’m going to go through about 10 slides talking briefly about what is azure.  That is, Azure is basically a family of hosted services that Microsoft provides in the cloud.  I’m going to talk about:

• Compute
• Storage
• Cache
• Service Bus
• SqlAzure

Then, I’m going to got through and actually show all the things I do on a daily basis while I’m doing my development work.  That includes using their development environment (called the dev fabric) as well as how I deploy and test.  The product I developed can be found at http://connectionroad.com if you are interested.  It’s basically Microsoft SyncToy to Cloud Storage.

Bring your questions!  This could be very fun.

OK, I think this is the longest post title I’ve ever made, but if you understand it, you’ll know why it needs to be so long.  I discovered this totally by accident.  I would never ever have pressed F11 (step into) from a client side proxy and expect to get into anything but a bunch of ugly machine generated proxy code (especially with Azure).  To my total surprise, I landed right inside my WCF service as if I had started the Windows Azure Developer fabric in debug mode and set a break point.

So, I’ll step through the process an show screen shots on the outside chance I was dreaming and can’t reproduce it. If I can, now I will have proof so I can do it again.  Sorry for the work in progress code you will see.  The point here is really the debugger and not the code I’m showing so try and ignore that.

OK, here we go.

Let’s assume you have a completely configured azure web role that is hosting a simple WCF service.  When you start that web role and point at the service, you’ll get something like this: (just a note that I started the app fabric by deploying directly from Microsoft Visual Studio 2010 with the start/run.  I am using a debug profile but am NOT running in debug mode.  If I do run the app fabric in debug mode, this does not work.

Microsoft’s Windows Azure is truly a revolutionary way to have web sites hosted.  The beauty of it is practically all the IT type stuff is just taken care of for you.  The kind of thing you do not have to worry about are

Security patches to the operating system

1. Worrying about what happens if the server goes down and you are not around
2. Database backups if your system crashes unexpectedly
3. Scaling servers to meet high demands
4. Other platform provided features such as Cache

The penalty for this unfortunately is that you have to spend more time up front specifying your configuration exactly so that Azure knows how to exactly boot and provision your web site in the cloud.

The Windows Azure I imagine is one where, from Visual Studio, I simply could say “Publish To Azure” , and as long as I have a Microsoft Live sign on, it would basically publish my web site to the Azure Cloud.  Seems to me that as long as the web site can run in my local development environment with either the built in visual studio web server (aka Cassini) or IIS Express, Azure should be able to make this into a cloud web site with no other steps.  I’m not saying don’t offer all the other configuration options, just give me a choice to make it easy and let me go with the defaults.

Am I wrong?  I totally get that there are a ton of configuration options that can be specified, but it seems to be they should not have to be.  If you feel the same or different, let me know your opinion, post your comment below and let the discussion begin now.

Again, I can’t express how much I like having Azure as a destination for deploying web sites.  Even with the current extra work to deploy, it is a huge benefit and worth it.  I strongly recommend taking a serious look at Azure for your web site if you have not already.

MIX11 In Las Vegas has been a really fun and exciting couple of days.  For me, the sessions are interesting, but my feeling is I can always go back and see them on the internet.  The most interesting things happen during the side conversations with Microsoft Team members and friends I don’t see often.

In this video, I’m having a little Fun with Steve Marx and Tina Dawson of the Windows Azure Team.  I’m not sure of Tina’s role, but Steve is undoubtedly my best advocate on the Windows Azure team.  Over the past 6 months or so, since I’ve been working building my first major Azure product (Soon to be found at http://connectionroad.com), Steve has been an immense help in sorting out tough issues.  Patiently explaining concepts as well as taking my “constructive” feedback to the team for further analysis.

or:   http://www.youtube.com/watch?v=kbF5lwqaLNI

(BTW, just a little background on the wardrobe consulting comment at the beginning.  About a year ago, I discovered, well, maybe someone mentioned to me, that wearing white socks with my dark running shoes and long pants was not exactly a fashion statement.  Noticing Steve in the same condition, obviously landing there as I did, we asked Tina for a consult.  Steve asked Tina if white socks with dark shoes and dark pants were reasonable.  Tina’s response was “Only if you are Michael Jackon”.  Later, she tried to mellow her response however by that point every had heard it her hopes of what is said in Las Vegas stays in Las Vegas were totally dashed.)

This is one of those things that you do once every year or so and it is very easy to forget how to do it the next time. I’d almost prefer my Cert’s to expire every week so I get in the swing of fixing it (just kidding).

So, I tried do a web deploy to my Windows Azure Instance this morning and I got a cryptic error that I blogged about here:  http://social.msdn.microsoft.com/Forums/en-US/windowsazuredevelopment/thread/3b1b6204-2a5f-4fe0-aebc-5245edf46c82

It looks something like this:

Error 1 Web deployment task failed.(Remote agent 
(URL https://xxx.cloudapp.net:8172/MsDeploy.axd?site=Web_IN_0_Web) could not be 
contacted.  Make sure the remote agent service is installed and started on the target computer.)
Make sure the site name, user name, and password are correct. If the issue is not resolved, 
please contact your local or server administrator.
Error details:
Remote agent (URL https://xxx.cloudapp.net:8172/MsDeploy.axd?site=Web_IN_0_Web)
 could not be contacted.  Make sure the remote agent service is 
installed and started on the target computer.
An unsupported response was received. The response header 'MSDeploy.Response' was 
'' but 'v1' was expected.
The remote server returned an error: (401) Unauthorized. 
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets 3847 5 Web

The way I do Windows Azure development is that I have OneClick Deployment setup on my Azure host.  This means I can simply and quickly update what I have on my Azure WebRole and test it without going through the 10 to 20 minute deployment process.  Sometimes, when I’m tired, I accidentally press the “Debug” or “Run” button on my Visual Studio 2010 cloud project.  This launches a 2 or 3 minute process that I have trouble killing.

Today, I accidentally figured out a way to stop it.  Simply go to the web project and right mouse button on your project and say “Publish”. You will be immediately presented with an option to stop the deployment!

That’s it.  Simple, but pretty undiscoverable IMHO.

The Problem

In my scenario, I have the same web project that I want to host in two different Windows Azure Data Centers (BTW, Steve Marx let me know it’s “Windows Azure” and not “Azure” at MVP Summit so I’ll try and keep my terminology right as much as I can).  Each Windows Azure Data Center has it’s own azure account (Azure Credential).  It resolves to a different domain name and as part of that scenario, has different properties in the ServiceConfiguration.csfg file.

Bruno Terkaly, the meeting organizer and local Microsoft Evangelist, did a great job of organizing as well as presenting.  Over the holidays, Bruno built a really cool end to end lab on how to build an app (both client and server) that takes advantage of Azure’s AppFabric (the service bus).  Basically, he showed the steps (and demonstrated) what it takes to have to windows PC’s talk to each other over the service bus.  One acts like a server to lots of clients. No firewalls, just communication!

As part of the meetup, Robin (@robindotnet) and I both did short presentations on how to  work SqlAzure, basically based around the problem of Azure (by design) dropping connections.  Robin talked about her real world experiences, and I talked about a method for how to deal with the problem elegantly in ado.net.  I did a blog post explaining what I presented here:  http://peterkellner.net/2011/01/21/sqlazure-connection-retry-problem-using-best-practices/

Of course ,what would a meet up be without pictures.  Feel free to add comments to the post about Ward Bell and the others.

That’s it for now!  Looking forward to the next Bruno Meetup

Introduction

If you’ve started using SqlAzure for your SqlServer with your Azure application, you’ve probably discovered that you get a reasonable number of connection failures.  The advice from the Azure team is add retry logic to all your connections to SqlAzure. There is a long discussion posted by the Azure team here.

The key paragraph states the problem as follows:

The Problem
One of the things that SQL Azure does to deliver high availability is it sometimes closes connections. SQL Azure does some pretty cool stuff under the covers to minimize the impact, but this is a key difference in SQL Azure development vs. SQL Server development.

Basically, what this means is that you must be able to deal with connections failing when you call SqlAzure.  Something that all of probably should have been doing forever, but because most of the time SqlServer is running on your local LAN and the likelihood if a connection failing was next to zero unless something else was going terribly wrong.  Certainly not something we had to do on regular basis.  To emphasize that even more, most of the controls built into asp.net that open connections to sqlserver don’t even do this and that’s from Microsoft itself.

The solution proposed in the thread mentioned above basically has you add tons of code to everyplace you access a connection object.  Personally, I don’t like that because I have hundreds if not thousands of places I open connections and inserting tens of thousands of lines of extra new untested code is a little scary.

So, what to do?

Fortunately, another team at Microsoft, known as the Windows Server AppFabric Customer Advisory Team published a general purpose solution using Extension Methods and some darn clever coding wrote a great article and published code including azure examples that solves this problem very elegantly without requiring a lot of changes to your existing code base.

In this article I plan on giving an example and publishing a sample project that uses this code with SqlAzure to solve the connection retry problem.  My goal here is not to simply restate what they published but to simply have a very simple concrete example of using their library.

I’ve been slowly building up my Azure experience over the past few months and actually plan to release a product using Azure during the next month or two.  Programming Windows Azure has been a huge value to me in learning both the basics of the Azure platform as well as the details.  It has a great balance of theory verses practice.  I strongly recommend this book if you are new to Azure or even if you have experience with Azure.  I often find myself going back and re-reading sections to better understand things.

Two of the sections I feel are particularly well written or the ones that talk about Storage and Tables (chapters 7 and 8).  The application I’ve been working on heavily uses blob storage and I spend lots of times re-reading those sections.  Azure tables is a hard topic to get a sql server programmer like me to get my head wrapped around.  Sriram does a great job of talking directly to someone like me to help me understand how and when to use tables.

Again, I whole heartedly recommend this book.

Just a quick shout out to the makers of Cloud Storage Studio Cerebrate Software.   Thanks for a great product offering!  I’ve been doing quite a bit of work recently with Microsoft Windows Azure Blob Storage and have really appreciated the insight into that storage Cloud Studio gives me.  I had been using another product to do the same thing which I had though was easier and faster, but after just a couple hours of working with Cloud Studio, I’m finding I’ve really been missing out. 

I’m attaching a screen shot below which shows viewing what is actually in the storage (not a hierarchical false view) as well as a display of all the meta data.  Either of those two features are show starters for me and enough to switch.

If you have not used it, give it  try (and have some patience, it takes a little bit of practice to find the true value).

Good luck!

Understanding how Azure Blob Storage can be used to simulate directory structures is a little tricky to say the least.  I’ve got a long forum thread on the Windows Azure Community site  now discussing the details.  As always, Steve Marx has been a big help here with a bunch of code. Steve’s got a great blog where he provides lots of examples and insights.  Neil Mackenzie has also contributed here to getting to the answer.

Just so we now have an example, I’ve put together a simple windows form app that let’s you set a few variables in your app.config to point at your azure storage and container, let you view your app as a tree as well as see the code how it can be done.  I have not commented the code much, just thought it would be good to get it out there.  The running application shows you the data as follows.

I just spend about 2 hours wrestling with what turned out to be an invalid blob storage container name.  I named a container “MyTest1” which did not meet the criteria.  For your information, the criteria is as follows:

1. 3 to 63 Characters
2. Starts With Letter or Number
3. Letters, Numbers, and Dash (-)
4. Every Dash (-) Must Be Immediately Preceded and Followed by a Letter or Number

So, I decided to hunt for a regular expression to do this and I found one on the web. I’m not sure if it’s correct, but here is what I found at http://social.msdn.microsoft.com/Forums/en-GB/windowsazuredata/thread/d364761b-6d9d-4c15-8353-46c6719a3392 from Gaurav Mantri (method that includes the regex).

internal static bool IsBlobContainerNameValid(string name)
    {
      if (name.Equals("$root"))
      { 
        return true; 
      }
      string validBlobContainerNameRegex = @"^([a-z]|\d){1}([a-z]|-|\d){1,61}([a-z]|\d){1}$";
      Regex reg = new Regex(validBlobContainerNameRegex);
      if (reg.IsMatch(name))
      {
        return true;
      }
      return false;
    }

I’m just starting to use Windows Azure for a project and plan on using the Azure Blob Storage part.  I won’t go into the details here, but let’s say I figured it out far enough so that I have pushed some piles of data into the blog storage.   Now, I want to see them.  I assumed that from the Azure portal, there would be some interface where I could see what I actual did and am being billed from.  I posted to the forums and basically was told that’s not really the case.  Here is the post with the answer saying you need some other software to do it (details below).

My friend RobinDotNet suggested I look at his article about how to use Visual Studio 2010 to view the blob storage.  Sure enough, in VS, I can set my account and key in server explorer and see the blob storage!  Thanks Robin.

For those of you that have been sleeping for the past year, Cloud computing is Hot!  Amazon more or less lead the way with their cloud computing platforms for the longest time with new comers like GoGrid, Google and Microsoft coming along with their own offerings more recently.  The offerings are quite complex, both licensing, capabilities, costs etc.  One of the nice things about this book (among others) is it gives detailed discussions in the beginning of several different offerings as well as their licensing models (which do change often so check directly with the companies).

On to the review..