Using Visual Studio configuration transforms with Azure
Environment-specific configuration is a pain.
An ASP.net app holds configuration settings like connection strings and debug flags in Web.config. Most apps have small but important configuration differences between our development, integration, and production environments. For example, I enable ASP.net debugging during development but disable it in production (as the Secure Development Lifecycle requires.)
Previously, I'd been been hand-tuning our configurations before each deployment, but this is painful (and I screw it up pretty often.) Visual Studio 2010 includes a new feature called Web.config Transformation that makes maintaining Web.config files easier. With this feature, Web.config acts as a shared base configuration. Configuration overlays (okay, probably not the official name) can override or replace values in the shared configuration. Each overlay is tied to a specific configuration; for example, the Debug configuration uses Web.Debug.config. The overlays use XML Document Transformations to change Web.config; at build time, the build system (MSBuild) applies the transformations to create Web.config.
This works great for Web.config, but my applications now live in Windows Azure. Windows Azure also has a service configuration system. Azure apps are packaged as cloud service packages. To deploy a cloud service, I upload a cloud service package and a configuration file to Azure. The configuration file, usually named ServiceConfiguration.cscfg, specifies the number of instances for a role and a set of key-value pairs for configuration:
Azure lets us change the configuration for a running service using the developer web portal or the REST API. Only the service configuration can be changed; it's not possible to change embedded files like Web.config with this interface. When a role configuration is changed, Azure fires an event.
Since the Azure configuration is independent of the cloud service package, I can deploy a particular cloud service package with one of many configuration files. This is good: to avoid errors, I want to deploy the same package to each environment and change only the Azure configuration.
After some hacking, I ended up with these MSBuild directives. To apply them, right-click on your cloud service package, choose "Unload Project", right-click again, and choose "Edit." You'll now be editing the project file, so make sure you have a backup.
For this sample, we'll use a cloud service called BogusService with three environments (test, integration, and production) and a single datacenter (North Central US). Create a new cloud service project with two worker roles following this service definition:
ServiceDefinition.csdef
We'll transform ServiceConfiguration.cscfg using three environment-specific overlays corresponding to our test, integration, and production environments. (You'll need to create these files.) Just before
add to the cloud project file (ccproj):
I also had to explicitly specify namespace prefixes in my configuration files. ServiceConfiguration.cscfg will look like:
ServiceConfiguration.cscfg:
and ServiceConfiguration.BogusService-Test-NorthCentralUS.cscfg looks like:
ServiceConfiguration.BogusService-Test-NorthCentralUS.cscfg
and so on for ServiceConfiguration.BogusService-Int-NorthCentralUS.cscfg and ServiceConfiguration.BogusService-Prod-NorthCentralUS.cscfg.
Now, you should be able to place your environment-specific configuration in the overlay files. When you publish the package, the environment-specific configuration files will appear in the Publish directory.
This imports the Web publishing targets to borrow the transformation support implemented by the TransformXml target. I replaced the ValidateServiceFiles target from the cloud service library. With this change, our base service configuration doesn't need to be valid (since it may lack certain options that are filled in by overlays) but it still ensures that the generated files are valid. After the package is built, it copies the generated configuration files to the same directory as the cloud service package.
If you want to use the development fabric, you'll need to have a valid ServiceConfiguration.cscfg; I haven't found a way to change this yet. Groveling through Microsoft.CloudService.targets might shed light on it.
If you're using Team Foundation Server, you'll need to manually add the overlays to source control. (I explicitly specify BaseConfiguration so that I could use several base configurations in the future.) To debug build problems, turn on verbose MSBuild logging using Tools/Options/Projects and Solutions/Build and Run/MSBuild Verbosity. Search for TransformXml in the (probably huge) MSBuild log in the Output pane.
Feedback welcome...I'd never played with MSBuild before, so if I'm doing something awful, let me know.
References: "Making Visual Studio 2010 Web.config Transformations Apply on Every Build", "Using config transforms outside web projects", "How To: Get MSBuild to run a complete Target for each Item in an ItemGroup", Microsoft.CloudService.targets, and Microsoft.Web.Publishing.targets.
An ASP.net app holds configuration settings like connection strings and debug flags in Web.config. Most apps have small but important configuration differences between our development, integration, and production environments. For example, I enable ASP.net debugging during development but disable it in production (as the Secure Development Lifecycle requires.)
Previously, I'd been been hand-tuning our configurations before each deployment, but this is painful (and I screw it up pretty often.) Visual Studio 2010 includes a new feature called Web.config Transformation that makes maintaining Web.config files easier. With this feature, Web.config acts as a shared base configuration. Configuration overlays (okay, probably not the official name) can override or replace values in the shared configuration. Each overlay is tied to a specific configuration; for example, the Debug configuration uses Web.Debug.config. The overlays use XML Document Transformations to change Web.config; at build time, the build system (MSBuild) applies the transformations to create Web.config.
This works great for Web.config, but my applications now live in Windows Azure. Windows Azure also has a service configuration system. Azure apps are packaged as cloud service packages. To deploy a cloud service, I upload a cloud service package and a configuration file to Azure. The configuration file, usually named ServiceConfiguration.cscfg, specifies the number of instances for a role and a set of key-value pairs for configuration:
<?xml version="1.0"?>
<ServiceConfiguration serviceName="CloudService4"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
<Role name="WebRole1">
<Instances count="1" />
<ConfigurationSettings>
<Setting name="DiagnosticsConnectionString" value="UseDevelopmentStorage=true" />
</ConfigurationSettings>
</Role>
</ServiceConfiguration>
Azure lets us change the configuration for a running service using the developer web portal or the REST API. Only the service configuration can be changed; it's not possible to change embedded files like Web.config with this interface. When a role configuration is changed, Azure fires an event.
Since the Azure configuration is independent of the cloud service package, I can deploy a particular cloud service package with one of many configuration files. This is good: to avoid errors, I want to deploy the same package to each environment and change only the Azure configuration.
After some hacking, I ended up with these MSBuild directives. To apply them, right-click on your cloud service package, choose "Unload Project", right-click again, and choose "Edit." You'll now be editing the project file, so make sure you have a backup.
For this sample, we'll use a cloud service called BogusService with three environments (test, integration, and production) and a single datacenter (North Central US). Create a new cloud service project with two worker roles following this service definition:
ServiceDefinition.csdef
<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="BogusService" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
<WorkerRole name="WorkerRole1">
<ConfigurationSettings>
<Setting name="HelloWorldString" />
</ConfigurationSettings>
</WorkerRole>
<WorkerRole name="WorkerRole2">
<ConfigurationSettings>
<Setting name="HelloWorldString" />
</ConfigurationSettings>
</WorkerRole>
</ServiceDefinition>
We'll transform ServiceConfiguration.cscfg using three environment-specific overlays corresponding to our test, integration, and production environments. (You'll need to create these files.) Just before
</Project>, I added:add to the cloud project file (ccproj):
<!-- begin added settings -->
<ItemGroup>
<EnvironmentConfiguration Include="ServiceConfiguration.BogusService-Test-NorthCentralUS.cscfg">
<BaseConfiguration>ServiceConfiguration.cscfg</BaseConfiguration>
</EnvironmentConfiguration>
<EnvironmentConfiguration Include="ServiceConfiguration.BogusService-Int-NorthCentralUS.cscfg">
<BaseConfiguration>ServiceConfiguration.cscfg</BaseConfiguration>
</EnvironmentConfiguration>
<EnvironmentConfiguration Include="ServiceConfiguration.BogusService-Prod-NorthCentralUS.cscfg">
<BaseConfiguration>ServiceConfiguration.cscfg</BaseConfiguration>
</EnvironmentConfiguration>
<!-- make our environment configurations appear in VS -->
<None Include="@(EnvironmentConfiguration)" />
</ItemGroup>
<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.targets" />
<!--
Previously, our custom target hooked in with BeforeTargets="BeforePackageComputeService"
to run the transformation before packaging. However, this meant that we had to have a valid
ServiceConfiguration.cscfg which was not technically necessary (and perhaps dangerous because
it could be accidentally deployed.) Now, we **replace** the ValidateServiceFiles target in
Microsoft.CloudService.targets so that our own validation logic is used.
-->
<Target Name="ValidateServiceFiles" Inputs="@(EnvironmentConfiguration);@(EnvironmentConfiguration->'%(BaseConfiguration)')"
Outputs="@(EnvironmentConfiguration->'%(Identity).transformed.cscfg')">
<Message Text="APL: ValidateServiceFiles: Transforming %(EnvironmentConfiguration.BaseConfiguration) to %(EnvironmentConfiguration.Identity).tmp via %(EnvironmentConfiguration.Identity)" />
<TransformXml Source="%(EnvironmentConfiguration.BaseConfiguration)" Transform="%(EnvironmentConfiguration.Identity)"
Destination="%(EnvironmentConfiguration.Identity).tmp" />
<Message Text="APL: ValidateServiceFiles: Transformation complete; starting validation" />
<ValidateServiceFiles ServiceDefinitionFile="@(ServiceDefinition)" ServiceConfigurationFile="%(EnvironmentConfiguration.Identity).tmp" />
<Message Text="APL: ValidateServiceFiles: Validation complete; renaming temporary file" />
<Move SourceFiles="%(EnvironmentConfiguration.Identity).tmp" DestinationFiles="%(EnvironmentConfiguration.Identity).transformed.cscfg" />
</Target>
<Target Name="CopyTransformedEnvironmentConfigurationXml" AfterTargets="AfterPackageComputeService">
<Copy SourceFiles="@(EnvironmentConfiguration->'%(Identity).transformed.cscfg')" DestinationFolder="$(OutDir)Publish" />
</Target>
<!-- end added settings -->
I also had to explicitly specify namespace prefixes in my configuration files. ServiceConfiguration.cscfg will look like:
ServiceConfiguration.cscfg:
<?xml version="1.0"?>
<sc:ServiceConfiguration serviceName="BogusService"
xmlns:sc="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration">
<sc:Role name="WorkerRole1">
<sc:Instances count="1" />
<sc:ConfigurationSettings>
<!--
The base ServiceConfiguration.cscfg file is intentionally invalid.
-->
</sc:ConfigurationSettings>
</sc:Role>
<sc:Role name="WorkerRole2">
<sc:Instances count="1" />
<sc:ConfigurationSettings>
<sc:Setting name="HelloWorldString"
value="This is ServiceConfiguration.cscfg" />
</sc:ConfigurationSettings>
</sc:Role>
</sc:ServiceConfiguration>
and ServiceConfiguration.BogusService-Test-NorthCentralUS.cscfg looks like:
ServiceConfiguration.BogusService-Test-NorthCentralUS.cscfg
<?xml version="1.0"?>
<sc:ServiceConfiguration
xmlns:sc="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<sc:Role name="WorkerRole1" xdt:Locator="Match(name)">
<sc:ConfigurationSettings>
<sc:Setting name="HelloWorldString"
value="Hello world - WorkerRole1 in ServiceConfiguration.BogusService-Test-NorthCentralUS.cscfg" xdt:Transform="Insert" />
</sc:ConfigurationSettings>
</sc:Role>
<sc:Role name="WorkerRole2" xdt:Locator="Match(name)">
<sc:ConfigurationSettings>
<sc:Setting name="HelloWorldString"
value="Hello world - WorkerRole2 in ServiceConfiguration.BogusService-Test-NorthCentralUS.cscfg" xdt:Transform="Replace" xdt:Locator="Match(name)" />
</sc:ConfigurationSettings>
</sc:Role>
</sc:ServiceConfiguration>
and so on for ServiceConfiguration.BogusService-Int-NorthCentralUS.cscfg and ServiceConfiguration.BogusService-Prod-NorthCentralUS.cscfg.
Now, you should be able to place your environment-specific configuration in the overlay files. When you publish the package, the environment-specific configuration files will appear in the Publish directory.
This imports the Web publishing targets to borrow the transformation support implemented by the TransformXml target. I replaced the ValidateServiceFiles target from the cloud service library. With this change, our base service configuration doesn't need to be valid (since it may lack certain options that are filled in by overlays) but it still ensures that the generated files are valid. After the package is built, it copies the generated configuration files to the same directory as the cloud service package.
If you want to use the development fabric, you'll need to have a valid ServiceConfiguration.cscfg; I haven't found a way to change this yet. Groveling through Microsoft.CloudService.targets might shed light on it.
If you're using Team Foundation Server, you'll need to manually add the overlays to source control. (I explicitly specify BaseConfiguration so that I could use several base configurations in the future.) To debug build problems, turn on verbose MSBuild logging using Tools/Options/Projects and Solutions/Build and Run/MSBuild Verbosity. Search for TransformXml in the (probably huge) MSBuild log in the Output pane.
Feedback welcome...I'd never played with MSBuild before, so if I'm doing something awful, let me know.
References: "Making Visual Studio 2010 Web.config Transformations Apply on Every Build", "Using config transforms outside web projects", "How To: Get MSBuild to run a complete Target for each Item in an ItemGroup", Microsoft.CloudService.targets, and Microsoft.Web.Publishing.targets.