Tuesday, April 13, 2010

SharePoint on SQL Server Mirrors?!?!?!

Yes, while SharePoint does not have the native ability to recognize a SQL Server 2008 Database Mirroring Session, you can configure it in such a way that you can protect your SharePoint installation with high availability for a lower cost. (It should be noted that SharePoint is recommended to run on a SQL Server cluster)

I recently implemented SharePoint on SQL Mirrors for a client, and the following is the method which I chose to use. (Some images have been redacted) 

Background

Setting up mirroring on SQL Server to be utilized by a SharePoint installation requires some additional steps to achieve automatic failover. SharePoint is not a mirroring aware product, it does not allow for capturing the FAILOVER PARTNER attribute for connection strings. In order to accomplish this you have a couple of options.

First you can manually alter the connection string settings in the configuration files for SharePoint. As this is not the recommended or selected approach it will not be discussed, other than to state that the danger in this is any upgrade or service pack could cause configuration files to be overwritten and the addition of the FAILOVER PARTNER attribute in the connection strings could be lost.

The selected method of running SharePoint against databases that are mirrored is described in detail below.

As a starting point we used the Microsoft White Paper “Using SQL Server Mirroring with Office SharePoint Server® and Windows SharePoint Services®”. This document can be accessed via the link below.

Using Database Mirroring with Office SharePoint Server and Windows SharePoint Services

The white paper describes how to set up the mirroring, how to set up a SQL Alias for SharePoint to use as a data source, and how to set up SharePoint to use that SQL Alias. When the database mirroring session fails over to the mirror server, the SQL Alias on the SharePoint servers can be modified to point to the mirror server, which has become the principal server, thereby removing the need to reconfigure SharePoint. Because the above white paper does not provide a method for automating the failover from the principal database server to the mirror server, we had to derive how to do this on our own. Following is the detailed documentation of the steps involved in automating the switch over for SharePoint to use the alternate database server.

Remotely updating the SQL Server Alias

The first step to achieving automated failover for SharePoint is to automate the update to the SQL Server Alias on each SharePoint Web Front End server and Application server. The SQL Alias configuration is held within the Windows Registry on each individual server, therefore a process must be run to update the value of the SQL Alias to change the Server attribute portion to match the currently active database principal. The registry value is held at HKLM\SOFTWARE\Microsoft\MSSQLServer\Client\ConnectTo\<SQL Alias Name>. <SQL Alias Name> is a REG_SZ key with a comma separated value that represents the connection type and server and if necessary the port number.

We created A pair of batch files to reside on each server. The pair of batch consists of one file for each database server in the mirroring session. For development we have chosen to place the files within a folder named “SQLServerMirrorScripts” on the root of drive C:. Note, we could have chosen different options for this, such as PowerShell scripts but due to client standards and preference we were limited.

Note the content of one of the batch files:

set DBSERVERALIAS=SQLAlias

set DBSERVER=DBSrv2

reg add HKLM\Software\Microsoft\MSSQLServer\Client\ConnectTo /v %DBSERVERALIAS% /t REG_SZ /d "DBMSSOCN,%DBSERVER%" /f

reg query HKLM\Software\Microsoft\MSSQLServer\Client\ConnectTo

The next step is to find a way to execute the batch files from a remote server, the database servers, but within the process space of the target server, the MOSS servers. In order to accomplish this we are used a utility called psexec.exe. You can attain this utility from Microsoft at Microsoft PSTools Page. This utility is a DOS based program that will remotely execute a process on a server under the currently logged in user context or under the user provided by command line argument. Each database server has another pair of batch files that encapsulate the execution command for the appropriate SQL Alias editing batch file on the remote server. These files, along with the psexec utility executable reside on each database server in a folder called “SQLServerMirrorScripts” in the root of drive C:

Again, each batch file is named for the database server it is going to change the SQL Alias too. Each batch file executes the remote batch file appropriate, designated in its name, to the operation. Below is the content of these batch files. These files exist in the same location on each of the database servers.

C:\SQLServerMirrorScripts\psexec.exe \\SharePointServer C:\SQLServerMirrorScripts\SQLAlias1.bat -d -e

At this point we have a process that will allow us to change the SQL Server Alias value from a remote location. When using psexec.exe for the first time from a server, you must manually execute the application and accept the license agreement that will popup. If this step is skipped the process will fail, because it will always wait for license acceptance.

Automating the SQL Server Alias Update

While the previous steps will allow us to modify the SQL Server Alias from each database server, they do not provide for the automatic updating of the alias in response to a database failing over from the principal server to the mirror server. In order to accomplish the automation, several additional steps are required.

Enabling Service Broker for each database

In order to enable automation, there must be a mechanism in place to respond to an event. In order to provide event notification in SQL Server, Service Broker must be enabled for all the mirrored databases and the msdb database. If you are unsure if Service Broker is enabled for your database you can execute the following query to find out.

USE master

Go

SELECT name, is_broker_enabled

FROM sys.databases

This will list the databases and a binary value for the state of is_broker_enabled, 0 is no, and 1 is yes.

image

By default, SharePoint databases are created with Service Broker enabled, but if there is a need to enable it after the database is created, the following command can be issued in SQL Server Management Studio;

ALTER DATABASE <DATABASE NAME> SET ENABLE_BROKER

You must also ensure that SQL Server Agent is running. If it is not, you must start the service, either via the services control snap in

image

or command line.

net start SQLSERVERAGENT


Setting Up the Event Notification

DATABASE_MIRRORING_STATE_CHANGE is not one of the events that can be automatically trapped. The event is available as a manually created Event Notification.
You will need to create an event notification named DBMirrorStateChange to raise a notification for the DATABASE_MIRRORING_STATE_CHANGE event, on both database servers in the mirroring session (not the witness):

If Not Exists (Select 1 From sys.server_event_notifications Where name = 'DBMirrorStateChange')

Begin

Create Event Notification DBMirrorStateChange

On Server

For DATABASE_MIRRORING_STATE_CHANGE

To Service 'DBMirrorService', 'current database';

End

This code need only be execute for a single database, denoted above as ‘current database’, in order to create the event notification that will be used for any database the has a mirroring state change.

If it ever becomes necessary to remove an event notification you can use the code below;

DROP EVENT NOTIFICATION notification_name [,...n ]

ON { SERVER | DATABASE | QUEUE queue }[;]

The DATABASE_MIRRORING_STATE_CHANGE event notification provides a property that denotes what type of state change has happened. This “state” property will inform us of what type of state change has occurred. The possible states for the event are as follows;

State

Name

Description

0

Null Notification

This state occurs briefly when a mirroring session is started.

1

Synchronized Principal with Witness

This state occurs on the principal when the principal and mirror are connected and synchronized and the principal and witness are connected. For a mirroring configuration with a witness, this is the normal operating state.

2

Synchronized Principal without Witness

This state occurs on the principal when the principal and mirror are connected and synchronized but the principal does not have a connection to the witness. For a mirroring configuration without a witness, this is the normal operating state.

3

Synchronized Mirror with Witness

This state occurs on the mirror when the principal and mirror are connected and synchronized and the mirror and witness are connected. For a mirroring configuration with a witness, this is the normal operating state.

4

Synchronized Mirror without Witness

This state occurs on the mirror when the principal and mirror are connected and synchronized but the mirror does not have a connection to the witness. For a mirroring configuration without a witness, this is the normal operating state.

5

Connection with Principal Lost

This state occurs on the mirror server instance when it cannot connect to the principal.

6

Connection with Mirror Lost

This state occurs on the principal server instance when it cannot connect to the mirror.

7

Manual Failover

This state occurs on the principal server instance when the user fails over manually from the principal, or on the mirror server instance when a force service is executed at the mirror.

8

Automatic Failover

This state occurs on the mirror server instance when the operating mode is high safety with automatic failover (synchronous) and the mirror and witness server instances cannot connect to the principal server instance.

9

Mirroring Suspended

This state occurs on both partner instances when the user suspends (pauses) the mirroring session or when the mirror server instance encounters an error. It also occurs on the mirror server instance following a force service command. When the mirror comes online as the principal, mirroring is automatically suspended.

10

No Quorum

If a witness is configured, this state occurs on the principal or mirror server instance when it cannot connect to its partner or to the witness server instance.

11

Synchronizing Mirror

This state occurs on the mirror server instance when there is a backlog of unsent log. The status of the session is Synchronizing.

12

Principal Running Exposed

This state occurs on the principal server instance when the operating mode is high protection (synchronous) and the principal cannot connect to the mirror server instance.

13

Synchronizing Principal

This state occurs on the principal server instance when there is a backlog of unsent log. The status of the session is Synchronizing.

Creating an Alert for Automatic Failover capture

We are interested in State 8 for this step. State 8 indicates an automatic fail over has occurred. This event is raised on the mirror server when a database that is acting as the mirror, changes state to acting as the principal.

The mechanism we use for responding to an event notification is an alert. An alert can be created via SQL Server Management Studio (SSMS), or via T-SQL script. The SSMS method will be shown here. Locate and expand the SQL Server Agent node in the SSMS object tree view.

image

Right click on Alerts and select “New Alert”. Enter a name for the Alert in the Name: attribute box, select WMI event alert in the Type drop down list. The Namespace: value will be populated automatically for the server you are on. Note: you will have to do this on both servers, but you can script the alerts and jobs from the first server and use those scripts on the second server as shown later. In the Query: box you will write the query based on the DATABASE_MIRRORING_STATE_CHANGE event.

DB_MIRROR_ALERT1_redact

Select the Response page and check the box for “Execute Job”, then select the “New Job” button.

DB_MIRROR_ALERT2_redact

This will create a new job and open the Job properties dialog for you. Enter a job name, owner and select the category as Database Mirroring. If the category “Database Mirroring” is not available select “Uncategorized (Local)”. Enter a description to detail the purpose of the job, and ensure Enabled is checked.

DB_MIRROR_JOB1_redact

Select the Steps page on the left next. And select the “New” Button on the bottom. This will launch the Job Step properties page.

Enter a name for the job step you are creating, select the type, in this case we are executing a batch file so select “Operating System (CmdExec)”, and chose the Run As account from the drop down. This will be the user that the process will run under. In the Command box input the batch file for the server you are working on and click “OK”.

DB_MIRROR_JOB3_redact

Back on the Alert properties page now, click “OK” to finish creating the Alert.

We now have an Alert that will capture the event that a database has gone from Mirror to Principal and take appropriate action. When the event occurs, the Alert is activated if the state value is 8. The Alert then executes the Job that was created, and the job executes the batch file which executes the psexec utility to execute the batch file on the SharePoint server, which modifies the registry to change the server value for the SQL Server Alias.

Creating an Alert for Manual Failover capture

A manual fail over is different than an automatic fail over, in that it fires an event with state 7 that occurs on the Principal server where the fail over command originated. Because SharePoint is a multi-database application, it is desired that all databases reside on the same server at all times. For automatic fail overs, it would be rare that the fail over would happen at a database level. Normally it will occur because the server went down, the network card failed, the disks failed, or some other catastrophic event occurred which rendered the Principal server inoperable. Manual fail over occurs when the Fail Over button is pressed in the Mirroring properties of the database or when certain T-SQL commands are issued. Manual fail over occurs at the database level, therefore a mechanism to automate manual fail over for the SharePoint instance, must not only change the SQL Server Alias, but also cause the other SharePoint databases to fail over to the Mirror. We will follow the same steps as for Automatic fail over, but we will add a new step to the job before the alias update step.

Locate and expand the SQL Server Agent node in the SSMS object tree view.

image

 

Right click on Alerts and select “New Alert”. Enter a name for the Alert in the Name: attribute box, select WMI event alert in the Type drop down list. The Namespace: value will be populated automatically for the server you are on. Note: you will have to do this on both servers, but you can script the alerts and jobs from the first server and use those scripts on the second server as shown later. In the Query: box you will write the query based on the DATABASE_MIRRORING_STATE_CHANGE event. Note that we are looking for State 7 in this event response.

DB_MIRROR_ALERT3_redact

Select the Response page and check the box for “Execute Job”, then select the “New Job” button.

DB_MIRROR_ALERT4_redact

This will create a new job and open the Job properties dialog for you. Enter a job name, owner and select the category as Database Mirroring. If the category “Database Mirroring” is not available select “Uncategorized (Local)”. Enter a description to detail the purpose of the job, and ensure Enabled is checked.

DB_MIRROR_JOB4_redact

Select the Steps page on the left next. And select the “New” Button on the bottom. This will launch the Job Step properties page.

DB_MIRROR_JOB5_redact

Enter a name for the job step you are creating, select the type, in this case we are a SQL Server query so select “Transact-SQL script (T-SQL)”. Leave the “Run As” box empty. In the Command box input the code shown and click “OK”.

DB_MIRROR_JOB6_redact

Code from above step:

SET NOCOUNT OFF

DECLARE @strSQL NVARCHAR(4000)

DECLARE @strDatabasename NVARCHAR(2000)

DECLARE MyCursor CURSOR FOR

SELECT name FROM master.sys.databases a

INNER JOIN master.sys.database_mirroring b

ON a.database_id=b.database_id

WHERE NOT mirroring_guid IS NULL

AND mirroring_role_desc='PRINCIPAL'

OPEN MyCursor

FETCH Next FROM MyCursor INTO @strDatabasename

WHILE @@Fetch_Status = 0

BEGIN

SET @strSQL = 'ALTER DATABASE [' + @strDatabaseName + '] SET PARTNER FAILOVER'

EXEC sp_executesql @strSQL

FETCH Next FROM MyCursor INTO @strDatabasename

END

CLOSE MyCursor

DEALLOCATE MyCursor

This code will force all databases on the server to become the “FailOver Partner” or Mirror, which causes the existing Mirror, or “FailOver Partner” to become the principal.

Back on the Job properties Steps page now, click “New” to create another step. This step will be identical to the job step for automatic failover, with the exception of which batch file it calls. Enter the values as below and click “Ok”.

DB_MIRROR_JOB3_redact

 

Click “OK” on the Job Properties pages. We now have a job with two steps to force all databases to fail over if one does, and to update the SQL Server Alias after that.

Back on the Alert Property page, click “Ok” to finish creating the alert.

We now have an Alert that will capture the event that a database has been manually failed over and take appropriate action. When the event occurs, the Alert is activated if the state value is 7. The Alert then executes the Job that was created, and the job executes the T-SQL code to fail over the remaining databases and then the batch file which executes the psexec utility to execute the batch file on the SharePoint server, which modifies the registry to change the server value for the SQL Server Alias.

Of course you have to do the same for the other server in the mirroring set. The procedure is the same, just in reverse of what is shown here, as for which batch file to call when.

That’s all for this post. Feel free to leave comments or ask questions. I’ll be happy to answer any. I’ve also included some useful links below.

Useful Links

SQL Server Central Blog

http://msdn.microsoft.com/en-us/library/ms188712.aspx

http://www.mssqltips.com/tip.asp?tip=1564


Technet Blog On Mirroring

Technorati Tags: ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Thursday, February 25, 2010

Database Mirroring on SQL Server 2008

I know it has been a very long time since I posted, and for that I am sorry. Sometimes work and personal life conspire to evaporate any free time you might have. But here I am with a new and hopefully useful bit of hard won experience to share with you.

I was recently engaged at a client to assist them with implementing database mirroring on SQL Server 2008. Everything seemed straight forward enough, they were conducting a proof of concept (POC) around using SQL Server to host their new ERP system and needed to prove the high availability features of automatic failover. No problem so far. They had a primary and secondary server and a virtual server to act as the witness already in place.

 

The wrench got thrown in at our kick-off meeting when the client informed me that since they are already implementing SharePoint 2007, they want the POC to use SharePoint as the application that will utilize the mirrored database set up. I’m not sure how many of you know this, but in order for an application to be natively ready to utilize a SQL Server mirrored environment and facilitate automatic fail-over, it must be capable of implementing the FAILOVER PARTNER attribute of a mirroring aware data source. SharePoint does not. There are many posts around the net on DB Mirroring, and SharePoint on mirrors, but I failed to find a comprehensive source that provided all the information I needed. Hence, my contribution to the community via this post. Actually this and the next post. I plan to break it up into a pair of posts, one that deals with the setting up of the mirroring, and one that deals specifically with the steps necessary to implement SharePoint on the mirroring session and provide automatic fail-over.

So in this post we will go through the process of creating a mirroring session and implementing mirroring for a database. We’ll use the AdventureWorks2008 sample database that Microsoft was so kind to create for us.

Any time you embark on configuring something in SQL Server, it is a good idea to be sure all the prerequisites are met ahead of time. There is a good source for prerequisites on SQL Server database mirroring at Microsoft TechNet. Here is the link Prerequisites, Restrictions, and Recommendations for Database Mirroring

Some things to note: Make sure both partners are using the same version and EDITION of SQL Server 2008. If you are running Enterprise on the primary, you should run Enterprise on the secondary. Both servers should also utilize the same default collation. You can not implement mirroring on database servers that have implemented the FILESTREAM option. Use of a third server, which can be a different Edition of SQL Server 2008, even EXPRESS, is required to fill the role of witness if you are implementing high availability mode (high safety with automatic fail-over).

So let’s begin. The first step is to ensure we have 3 server instances available to us. In a production application, these will be 3 separate servers, even if any or all are virtual. If you are doing this on your own, you can install a named instance of SQL Server along with your default instance and install SQL Server Express to act as the witness.

 

Once you have established your servers, you will need to ensure you have met all the requirements and pre-requirements as laid out in the above link. When you are ready to begin, follow the steps below.

Step 1:
Set Recovery Model to FULL.

Mirroring_AdvWrks1

 Mirroring_AdvWrksRecoveryFull

Step 2:
Backup the database.

Mirroring_AdvWrksBackup

Step 3:
Copy the backup to the designated mirror instance.
Step 4:
Restore the backup on the designated mirror instance with the options selected as below.

 

Mirroring_AdvWrksRestore1

Mirroring_AdvWrksRestore2

You must choose the “Leave the database non-operational, and do not roll back uncommitted transactions. Additional transaction logs can be restored (RESTORE WITH NORECOVERY)” option for database mirroring.

Step 5:
On the designated principal instance, right click the database and select “Tasks” and then “Mirror”. This will launch the Database Mirroring Wizard.

Mirroring_AdvWrksMirrorWiz2

 

Click “Next” and move through the wizard, selecting the Principal or primary server, the secondary or Mirror server, and the Witness server. In order to achieve automatic fail over a witness server is required.

Mirroring_AdvWrksMirrorWiz3

Mirroring_AdvWrks4

There will be an identical screen on which to select the Witness server. The wizard creates and configures the endpoints for you.

Mirroring_AdvWrksMirrorWiz5

These accounts are used for communication between the three servers.

Mirroring_AdvWrksMirrorWiz6

 

Final check before finishing.

Mirroring_AdvWrksMirrorWiz7

 

This screen shows the configuration progress for each server. Any warnings or errors will be caught and displayed here. When this screen finishes, you will be prompted to start mirroring.

Mirroring_AdvWrksStartMirror1

 

Once the process concludes and the databases have synchronized, the mirroring session is started and all transactions made at the principal will be duplicated on the mirror. In the event of a failure of the primary/principal the Witness and the Mirror form a quorum and determine to switch control to the mirror, which exits the restoring state and becomes a disconnected principal. When the mirroring session is started the Mirror page of the database properties is as shown below.

Mirroring_AdvWrksMirroringComplete

 

You will see in the database object tree view in SQL Server Management Studio, that one database shows Principal, synchronized, and one shows Mirror, synchronized.

Mirroring_DB_View

That’s it. You have set up mirroring on your database. Now when a problem occurs with the principal, the mirror will take over operation as the principal database. Any mirroring aware applications can implement the FAILOVER PARTNER attribute of the connection string to ensure continuous operation through a failed database server.

You can manual fail over the database from primary server to the secondary server in two ways.

One: Open the properties for the database and select the Mirror page. There is a button “Failover” that if you press, will manually fail over the database from the primary to the secondary.
Two: You can issue a T-SQL Query to fail over the database. On the primary server with the principal database execute the following code;
ALTER DATABASE AdventureWorksDW SET PARTNER FAILOVER

That’s all there is too setting it up. Of course there is a ton more information on Microsoft MSDN and TechNet as well as in BOL. Look it up, there’s much, much more to learn about database mirroring.

In my next post we will continue from here and implement automatic failover for a SharePoint installation. Until thin, keep slinging the SQL.