The Daily Parker

Politics, Weather, Photography, and the Dog

Continuing migrations to Azure

I've finished two complete migrations from my living room the Inner Drive Technology Worldwide Data Center to Microsoft Windows Azure web sites. Astute readers may remember that in one case I moved to the Web site offering and then moved it to a full-fledged Web role. Well, today, I moved it back. Even though I'm still on the free trial, it turned out that the Web role would cost $15 per month, which, for a site that gets one or two visitors per day, simply wasn't worth it.

Moving the second site, a silly thing from 2004 created to share photos and commentary about a 10-year series of Presidents Day parties a friend of mine hosted back in the day, went a lot more smoothly:

0. I archived the project after its last deployment to the Web, which was in November 2006, when the Web server went online. (The age of this particular server is one of the reasons I'm moving everything to Azure, in fact.) I use SourceGear Vault for source control. So all the source code still lived in the Obsolete repository.

1. The old source code was C# 1.1 code from 2004, so the old project and solution files were in Visual Studio 2003 format. Forget upgrading; I just created a new solution.

2. Some of the code files in the Obsolete area were shared with other, obsolete projects. I branched them.

3. In source control, I moved files from the obsolete folder to new folder, except for obsolete configuration and code files. Vault doesn't automatically fetch the files when you do this, to prevent disasters. That was helpful.

4. Here I ran into a minor Visual Studio annoyance with its project naming schemes. I want the Web project to have a different, standard name for Web projects, so I had to rename it by hand in the file system and in the .sln file. No big deal.

5. Now I bound the new solution to source control. The procedure has to do with my source control setup, and may not be typical:

  • Got latest files from Vault in their new locations.
  • Added the files to the solution and project.
  • Closed the solution
  • In Vault, manually added the .sln and .csproj files
  • Reopened solution and changed the source control bindings

6. Added references to current version of Inner Drive Extensible Architecture:

  • Wound up replacing global.asax.cs with the version from Boxer's Shorts. The party site's was too old to bother updating.
  • Added references to InnerDrive.Azure, the Azure mail provider, etc.

7. Reviewed the code throughout the project to ensure nothing in them should prevent a quick compile:

  • Converted the project files to Web application. This adds designer files to all Web forms with code-behind files, so Visual Studio can compile the site down to a deployable assembly.
  • Went through each page to make sure the designers and the pages match
  • Built the project (without running it): OK.

At this point I'd spent about 50 minutes on it. So far, so good.

8. Updated configuration files (web.config, email configuration, etc.) to match the new versions of Inner Drive code files.

9. All Inner Drive websites get page metadata from a data tables, so I had to add the site's metadata to the SQL Database running in my Azure account. This is scripted, and the scripts, moreover, are idempotent to allow for easy automation.

10. It turned out that half the photos and two of the pages on the site weren't in source control after all. Short pause to fetch them from the current Web server and add them to the repository.

11. Compile and run in Debug mode: Yay! Evertyhing works! Time elapsed: 90 minutes (including getting my salad out of the fridge and answering a colleague's questions about something unrelated.)

12. In the Windows Azure portal, created a new Web site, and dowloaded its publish profile to the Visual Studio solution.

13. Right-clicked the Web project and clicked "Publish..."

14. While the publication went on (there were 3.6 Mb of photos to copy up to Azure), updated the production Pages tables with the site's page metadata.

15. Publication completed and...crashed immediately. Crap.

16. Uploaded a new web.config file with <customErrors mode="Off">

"The page cannot be displayed because an internal server error has occurred." That's not helpful. And it led to some thrashing before I realized the PEBCAK:

  • Removed the local-only sections of web.config
  • Stripped down the MessageConfig.xml to do absolutely nothing
  • Check the /LogFiles folder on the virtual machine about 20 times only to see it's empty
  • Check the Azure portal's "Configure" tab and turned on diagnostics
  • Oops. There it is: "Configuration file is not well-formed XML"
  • Uploaded a new web.config file with <customErrors mode="Off" /> (notice the terminating slash).

17. "Could not load file or assembly 'Microsoft.WindowsAzure.ServiceRuntime, Version=1.7.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified." No problem, I've seen this before. Fixed it.

18. Reloaded the page: success! Except all of my messaging was turned off from the debugging at step 16, so I just had to add them back.

19. Success!

And it only took 2½ hours, including lunch and talking to colleagues about something else. Next time, I hope to do it in a lot less time.

Unfortunately, I can't share the URL, because the content is old and unflattering. Those of us who attended the parties may want to remember them, or figure out what happened in cases where we can't actually remember. But it's not for general consumption.

Hot hot hot

The temperature at O'Hare just hit 35°C, and it's still rising. This is the 7th day in a row of above-32°C temperatures, and it looks like we're in for another 4 more days of it—including 38°C tomorrow and 39°C Thursday.

But that isn't the main weather story of the day; the drought is:

The latest USDA Illinois Weather and Crops report was released this afternoon. The topsoil conditions were rated at 52 percent “very short” and 37 percent “short” and only 11 percent adequate. Soil moisture conditions were best in northern Illinois, and deteriorated southward. Hardest hit was southeastern Illinois with 100 percent of the topsoil and 100 percent of subsoil rated as “very short”.

Corn and soybeans—Illinois produces 18% of the country's corn and 16% of the county's soybean— make up 73% of the state's gross agricultural product, amounting to $7.2bn per year. This summer is bad, but possibly not as bad as 1988. Yet.

It is finished.

I have successfully ported my first (existing) application to the Microsoft Windows Azure platform, and have shut down the running instance on my local Web server. I hope the second one takes less than a week.

It's a funny little site called Boxer's Shorts. Dr. Bob Boxer is a local allergist who likes puns. He worked with a local illustrator, Darnell Towns, and self-published the five paperback pun compilations advertised on the site.

Local web designer Lauren Johnson (née Liss) did the look and feel, and I provided the platform. I think we completed the site in two weeks or so. I've hosted it since it went live in September 2006—just a few days after I got Parker, in fact.

And now it's in the cloud, the first Inner Drive site to be ported. From what I learned doing it, I hope to get two more of my older sites deployed to Azure this week.

Why I haven't finished deploying to Azure

I've spent much of the past week trying to get a single, small website up into the cloud on the Windows Azure platform. Much of this effort revolved around the Azure Website product, mainly because it's free. Well, I got the application up as an Azure website...and there's a big problem with it that means I'll have to redeploy it as a Web role after all.

First, let me just outline how much fun I've had today, starting from this morning when I first tried to publish the application to the cloud:

  • For the first hour or so, I dealt with a missing patch that prevented publishing entirely.
  • My next task, after I got the bits up to Azure, was to track down why the application failed on the method RoleEnvironment.IsAvailable, which you need if you want to deploy something to an Azure Web role. First I got this:
    Could not load file or assembly 'Microsoft.WindowsAzure.ServiceRuntime, Version=1.7.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
  • The solution was simply to go to the Solution Explorer and mark the ServiceRuntime assembly as "copy local" instead of "do not copy." That fixed it. Until the next time I tried to run, when I got this:
    Could not load file or assembly 'msshrtmi, Version=1.7.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
  • Fortunately, developer Jake Ginnivan had exactly this problem yesterday. (Wow, who doesn't love Google?) Of course, that required me to compile the whole site as x86, not x64:

At this point, the entire site seemed to work. Except now I wasn't getting any exception messages. Nor was I able to get a "contact us" email. Some digging showed that my code—yes, my code—was swallowing exceptions, and throwing the wrong exceptions.

This is because the Inner Drive Extensible Architecture message system loads configuration if you don't provide it, getting the configuration file spec from the web.config file. Only, in every single instance where the thing is in production, I control the file system, so I've never had a problem knowing where the configuration file actually lives. (I also solved the problem of how to get configuration from Azure storage, as a pain I described earlier today.) The configuration setting looks like this:

<appSettings>
   <add key="messageConfigFile" value="c:\SomeActualPath\MessageConfig.xml"/>
</appSettings>

You can also use a relative path if you know the root. On Azure web sites, it turns out the root is D:\Windows. Seriously. On my little slice of a shared server, the actual path to the application is something like C:\DWASFiles\Sites\Site-Name\VirtualDirectory0\site\wwwroot\. The "virtual" part scares me; this path is not guaranteed. This is what we call in programming "brittle."

So I wound up using a relative path from the application root and adding this code to the application start method:

var configurationFile = ConfigurationManager.AppSettings["messageConfigFile"];
var fileSpec = Server.MapPath(configurationFile);
Publisher.Instance.Load(fileSpec);

I deployed it again and yay! After adding that bit of code, and spending a total of 6.9 non-billable hours on this, I finally got the application to run as a free Azure web site with its azurewebsites.net address. So all I have to do now is add a CNAME record pointing the site's real URL to this one and...

You're kidding me.

Free Azure websites running on shared instances only allow a single hostname per site:

This means that I can't use www.mysitename.com to point to mysitename.azurewebsites.net, which means the whole effort is kind of wasted, except to use the thing as a staging site. Moving to a reserved instance solves the problem, but costs 8c per hour of compute time. But for only 2c and hour, I can use a traditional Web role in an extra-small instance. I'm pretty sure this particular site won't run continuously, but still, why pay any more than required?

Almost everything I did today also applies to publishing the application to a Web role, but now I have to pay money for it. Dang.

Why Parker is disappointed in me right now

Yes, I just said I was taking Parker out for a walk, but I cut it short after five minutes. Here's why:

Just as we got back home the gust front hit. Trees are now moving in ways that trees probably shouldn't. This should be a lot of fun to watch.

...but Parker is sulking. Tant pis, mon bête noir.

Update, 1:25 pm: Huh. The storm just missed us, though reports have come in of 145 km/h gusts in Elmhust and Lombard, which "looks like a war zone" according to the Tribune.

Who may I strangle, please?

In the past week, I've been "on the bench" at work, so I've take the time to get deeply familiar with Microsoft Windows Azure. My company, 10th Magnitude, is a 100% cloud-computing shop, and a Microsoft partner. I've been developing for Azure Web applications for a year, but I haven't had to deal with migrating existing sites, pricing, or configuration on my own; this is why we're a team, right?

So, anyway, I've taken what I've learned at work, and:

  • Selected a simple website to migrate; in this case, Boxer's Shorts, a project I completed in 2006 that has five rows of data and five content pages;
  • Worked for about 16 hours this week creating Azure implementations of the Inner Drive Extensible Architecture features that won't work on Azure—but only the two features (messaging and logging) that Boxer's Shorts needs;
  • Spent 6 hours yesterday and an hour today preparing the application for Azure; and
  • Set up an Azure Web site this morning into which I was about to publish the ready-to-roll Web site.

Remember the messaging and logging services I spent lots of time migrating? Well, it looks like I made the right choice in the long run. The services use Azure tables and blobs for logging, holding the site configuration data, and in this site's case, for holding the list of books. Azure storage is really, really cheap, less than 10c per gigabyte for locally-redundant storage or 12.5c per gigabyte for geo-redundant storage. This is de rigeur for a traditional Azure Web role deployment for everything that can run without relational data. (For relational data you need SQL Server.)

I thought—mistakenly, it turns out—that if it worked in a Web role it would work in a Web site. No, not so much. In the short run, I just discovered this:

Ah. No blob storage for Azure web sites. So now I have to strip out all of the stuff that uses blob storage on this web site, and modify the book list to use SQL Server instead of Azure tables. Two more hours.

So why not just publish the site to a Web role instead? Price. With Azure, you get 10 free Web sites with your subscription; but each Web role costs at least 2c per hour for the smallest possible footprint. There are 720 hours in a month, so even though you only pay for the time the application is actually doing something, you have to plan for about $14 per month. For a site that gets 100 page-views per week, has five content pages, and five pieces of data, that's really a lot of money. And it's infinitely more than free.

All right then. I hope that Azure websites get access to Azure storage soon. For now, I'm just going to rip the logging out of the site and fix the rest of it. But first I'm going to walk the dog.

Azure build error messages aren't helpful

When working with Microsoft Windows Azure, I sometimes feel like I'm back in the 1980s. They've rushed their development tools to market so that they can get us developers working on Azure projects, but they haven't yet added the kinds of error messages that one would hope to see.

I've spent most of today trying to get the simplest website in my server rack up into Azure. The last hour and a half has been spent trying to figure out two related error messages that occurred when trying to debug a Web application project that I converted from a Web site project:

  • Failed to debug the Windows Azure Cloud Service project. The output directory ' { path }\csx\Debug' does not exist.
  • Windows Azure Tools: Can't locate service descriptions.

The first error message seems straightforward enough: when the project got created, it never added the \csx\Debug folder. After creating the empty \csx\Debug folder, the second message occurs.

When an Azure project builds, it's supposed to create the \csx\Debug folder under the Cloud Service project root. It then copies the service definition (.csdef) and configuration (.cscfg) projects into the folder, which the Azure compute emulator can hook into.

In my project, this wasn't happening. So I created a new Cloud solution to see if this was a system problem or a configuration problem. (First I uninstalled and reinstalled all the Azure tools...which wasn't as big a time-suck as it could have been because I walked Parker while that was going on.)

The Deleteme solution built fine; mine still had the problem. So then I started comparing the configuration, project, and solution files...and completely missed the significance of this:

...except it gnawed at me for a few minutes, until I looked at this:

Why it created a configuration and then decided not to build it I just don't know. The solution to my hours of pain is simply to do change the solution platform to Any CPU (or check "build" on the .NET platform):

I am now going to fix the hole in my desk where I've been pounding my head.

When I started getting these messages, I Googled and I Googled, but the technology is so new that no one else appears to have had exactly this problem. I hope this post pays back some of the Karmic debt I've taken on from all the times when someone else had the right answer.

Have the GOP always been like this?

The Texas Republicans published their 2012 platform this week, vowing to stop teaching children critical reasoning skills in the next four years. I was curious about other GOP platforms, to see if Texas was an aberration, and I found this one:

Resolved, That we, the delegated representatives of the Republican electors of the United States, in convention assembled, in discharge of the duty we owe to our constituent and our country, unite in the following declarations:

...

8. That the normal condition of all the territory of the United States is that of freedom; that as our republican fathers, when they had abolished slavery in all our national territory, ordained that no "person should be deprived of life, liberty or property, without due process of law," it becomes our duty, by legislation, whenever such legislation is necessary, to maintain this provision of the constitution against all attempts to violate it; and we deny the authority of congress, of a territorial legislature, or of any individuals, to give legal existence to slavery in any territory of the United States.

12. That while providing revenue for the support of the general government by duties upon imports, sound policy requires such an adjustment of these imposts as to encourage the development of the industrial interests of the whole country, and we commend that policy of national exchanges which secures to the workingmen liberal wages, to agriculture remunerating prices, to mechanics and manufacturers an adequate reward for their skill, labor and enterprise, and to the nation commercial prosperity and independence.

14. That the Republican Party is opposed to any change in our naturalization laws, or any state legislation by which the rights of citizenship hitherto accorded by emigrants from foreign lands shall be abridged or impaired; and in favor of giving a full and efficient protection to the rights of all classes of citizens, whether native or naturalized, both at home and abroad.

The rest of it is pretty interesting, and also short. Obviously I'm quoting their first presidential-election-year platform, from 1860, so much of it applies to the situation that existed right before the Civil War.

Today's Illinois GOP platform, while saner (only just) than the Texas platform, still has these planks:

  • A call to "meet the contractual obligations of our state by properly funding the various state pension systems" without raising revenue to do it;
  • An assertion that kids are better off "within a two-parent family based on the principle of marriage between one man and one woman;" and
  • The anti-science position that "The Illinois Republican Party opposes the fostering of utilitarian experiments which sacrifice human embryos in what appears to be a futile search for medical cures."

On the other hand, some of their planks really surprised me:

  • "We call on the Federal Government to streamline the task of citizenship for legal immigrants to assimilate and complete the process of becoming Americans."
  • "We call for the granting of full citizenship rights to be granted to any immigrant upon the completion of service to the armed forces of the United States."
  • "[W]e endorse...[t]he use of criminal and mental background checks by licensed firearms dealers...."

Finding three planks to support out of the entire platform took some effort, though. A lot of it repeats the right-wing policies of the national GOP, like opposition to taxes in general, support for a concealed-carry firearms law, and one of my favorites, "We call on the United States Senate to reject treaties which cede the powers and rights of the American people to the United Nations and other international agencies." (Because of the black helicopters following some of the more, ah, committed party members, you see.)

I'm sure a careful reading of my party's platform (which, unfortunately, is spread across 14 pages of the party website rather than being written down in any one document) would uncover a few planks I don't support. I expect, though, I've chosen the right bunch for now. When the Republicans shed themselves of the right-wing nutters currently running things for them, maybe I'll take another look.

Texas GOP: To One Over Infinity and Beyond!

The Texas Republican Party has published their platform after their recent convention, and...well:

We decry the appointment of unelected bureaucrats, and we urge Congress to use their constitutional authority to defund and abolish these positions and return authority to duly elected officials, accountable to the electorate.

We strongly support the immediate repeal of the Endangered Species Act. We strongly oppose the listing of the dune sage brush lizard either as a threatened or an endangered species. We believe the Environmental Protection Agency should be abolished.

We oppose the teaching of Higher Order Thinking Skills (HOTS) (values clarification), critical thinking skills and similar programs that are simply a relabeling of Outcome-Based Education (OBE) (mastery learning) which focus on behavior modification and have the purpose of challenging the student’s fixed beliefs and undermining parental authority.

We urge that the Voter Rights Act [sic] of 1965 codified and updated in 1973 be repealed and not reauthorized.

We affirm that the practice of homosexuality tears at the fabric of society and contributes to the breakdown of the family unit. Homosexual behavior is contrary to the fundamental, unchanging truths that have been ordained by God, recognized by our country’s founders, and shared by the majority of Texans.

If you're happy and you know it, clap your eyes.

This makes me curious, actually. I might check out the platforms of other states' Republican parties, particularly Illinois'. And the Democratic Party platform here. Updates as warranted.

Officially hotter than hell

The temperature in downtown Chicago edged up to 100°F (38°C) this afternoon:

At 2:23 pm the temperature at Midway Airport as measured by observer Frank Wachowski reached 37.83°C . Last summer Midway recorded triple-digits on two occasions: July 20, 37.8°C, and July 21, 38.3°C. On both of those days the city's official thermometer at O'Hare International Airport peaked at 37.2°C.

So far today O'Hare's temperature has peaked at 37.2°C but should reach 37.8°C or higher later this afternoon. The city's last official 37.8°C day was on July 24, 2005 when the mercury hit 38.9°C.

I really don't want to go outside...but I have to go home at some point. Maybe I'll do an impromptu pub crawl to and from the El...