SharePoint on SQL Server Mirrors?!?!?!

Yes, while SharePoint does not have the native ability to recognize a SQL Server 2008 Database Mirroring Session, you can configure it in such a way that you can protect your SharePoint installation with high availability for a lower cost. (It should be noted that SharePoint is recommended to run on a SQL Server cluster)

I recently implemented SharePoint on SQL Mirrors for a client, and the following is the method which I chose to use. (Some images have been redacted) 

Background

Setting up mirroring on SQL Server to be utilized by a SharePoint installation requires some additional steps to achieve automatic failover. SharePoint is not a mirroring aware product, it does not allow for capturing the FAILOVER PARTNER attribute for connection strings. In order to accomplish this you have a couple of options.

First you can manually alter the connection string settings in the configuration files for SharePoint. As this is not the recommended or selected approach it will not be discussed, other than to state that the danger in this is any upgrade or service pack could cause configuration files to be overwritten and the addition of the FAILOVER PARTNER attribute in the connection strings could be lost.

The selected method of running SharePoint against databases that are mirrored is described in detail below.

As a starting point we used the Microsoft White Paper “Using SQL Server Mirroring with Office SharePoint Server® and Windows SharePoint Services®”. This document can be accessed via the link below.

Using Database Mirroring with Office SharePoint Server and Windows SharePoint Services

The white paper describes how to set up the mirroring, how to set up a SQL Alias for SharePoint to use as a data source, and how to set up SharePoint to use that SQL Alias. When the database mirroring session fails over to the mirror server, the SQL Alias on the SharePoint servers can be modified to point to the mirror server, which has become the principal server, thereby removing the need to reconfigure SharePoint. Because the above white paper does not provide a method for automating the failover from the principal database server to the mirror server, we had to derive how to do this on our own. Following is the detailed documentation of the steps involved in automating the switch over for SharePoint to use the alternate database server.

Remotely updating the SQL Server Alias

The first step to achieving automated failover for SharePoint is to automate the update to the SQL Server Alias on each SharePoint Web Front End server and Application server. The SQL Alias configuration is held within the Windows Registry on each individual server, therefore a process must be run to update the value of the SQL Alias to change the Server attribute portion to match the currently active database principal. The registry value is held at HKLM\SOFTWARE\Microsoft\MSSQLServer\Client\ConnectTo\<SQL Alias Name>. <SQL Alias Name> is a REG_SZ key with a comma separated value that represents the connection type and server and if necessary the port number.

We created A pair of batch files to reside on each server. The pair of batch consists of one file for each database server in the mirroring session. For development we have chosen to place the files within a folder named “SQLServerMirrorScripts” on the root of drive C:. Note, we could have chosen different options for this, such as PowerShell scripts but due to client standards and preference we were limited.

Note the content of one of the batch files:

set DBSERVERALIAS=SQLAlias set DBSERVER=DBSrv2 reg add HKLM\Software\Microsoft\MSSQLServer\Client\ConnectTo /v %DBSERVERALIAS% /t REG_SZ /d "DBMSSOCN,%DBSERVER%" /f reg query HKLM\Software\Microsoft\MSSQLServer\Client\ConnectTo

The next step is to find a way to execute the batch files from a remote server, the database servers, but within the process space of the target server, the MOSS servers. In order to accomplish this we are used a utility called psexec.exe. You can attain this utility from Microsoft at Microsoft PSTools Page. This utility is a DOS based program that will remotely execute a process on a server under the currently logged in user context or under the user provided by command line argument. Each database server has another pair of batch files that encapsulate the execution command for the appropriate SQL Alias editing batch file on the remote server. These files, along with the psexec utility executable reside on each database server in a folder called “SQLServerMirrorScripts” in the root of drive C:

Again, each batch file is named for the database server it is going to change the SQL Alias too. Each batch file executes the remote batch file appropriate, designated in its name, to the operation. Below is the content of these batch files. These files exist in the same location on each of the database servers.

C:\SQLServerMirrorScripts\psexec.exe \\SharePointServer C:\SQLServerMirrorScripts\SQLAlias1.bat -d -e

At this point we have a process that will allow us to change the SQL Server Alias value from a remote location. When using psexec.exe for the first time from a server, you must manually execute the application and accept the license agreement that will popup. If this step is skipped the process will fail, because it will always wait for license acceptance.

Automating the SQL Server Alias Update

While the previous steps will allow us to modify the SQL Server Alias from each database server, they do not provide for the automatic updating of the alias in response to a database failing over from the principal server to the mirror server. In order to accomplish the automation, several additional steps are required.

Enabling Service Broker for each database

In order to enable automation, there must be a mechanism in place to respond to an event. In order to provide event notification in SQL Server, Service Broker must be enabled for all the mirrored databases and the msdb database. If you are unsure if Service Broker is enabled for your database you can execute the following query to find out.

USE master Go SELECT name, is_broker_enabled FROM sys.databases

This will list the databases and a binary value for the state of is_broker_enabled, 0 is no, and 1 is yes.

By default, SharePoint databases are created with Service Broker enabled, but if there is a need to enable it after the database is created, the following command can be issued in SQL Server Management Studio;

ALTER DATABASE <DATABASE NAME> SET ENABLE_BROKER

You must also ensure that SQL Server Agent is running. If it is not, you must start the service, either via the services control snap in

or command line.

net start SQLSERVERAGENT

Setting Up the Event Notification

DATABASE_MIRRORING_STATE_CHANGE is not one of the events that can be automatically trapped. The event is available as a manually created Event Notification.
You will need to create an event notification named DBMirrorStateChange to raise a notification for the DATABASE_MIRRORING_STATE_CHANGE event, on both database servers in the mirroring session (not the witness):

If Not Exists (Select 1 From sys.server_event_notifications Where name = 'DBMirrorStateChange') Begin Create Event Notification DBMirrorStateChange On Server For DATABASE_MIRRORING_STATE_CHANGE To Service 'DBMirrorService', 'current database'; End

This code need only be execute for a single database, denoted above as ‘current database’, in order to create the event notification that will be used for any database the has a mirroring state change.

If it ever becomes necessary to remove an event notification you can use the code below;

DROP EVENT NOTIFICATION notification_name [,...n ] ON { SERVER | DATABASE | QUEUE queue }[;] The DATABASE_MIRRORING_STATE_CHANGE event notification provides a property that denotes what type of state change has happened. This “state” property will inform us of what type of state change has occurred. The possible states for the event are as follows; State Name Description 0 Null Notification This state occurs briefly when a mirroring session is started. 1 Synchronized Principal with Witness This state occurs on the principal when the principal and mirror are connected and synchronized and the principal and witness are connected. For a mirroring configuration with a witness, this is the normal operating state. 2 Synchronized Principal without Witness This state occurs on the principal when the principal and mirror are connected and synchronized but the principal does not have a connection to the witness. For a mirroring configuration without a witness, this is the normal operating state. 3 Synchronized Mirror with Witness This state occurs on the mirror when the principal and mirror are connected and synchronized and the mirror and witness are connected. For a mirroring configuration with a witness, this is the normal operating state. 4 Synchronized Mirror without Witness This state occurs on the mirror when the principal and mirror are connected and synchronized but the mirror does not have a connection to the witness. For a mirroring configuration without a witness, this is the normal operating state. 5 Connection with Principal Lost This state occurs on the mirror server instance when it cannot connect to the principal. 6 Connection with Mirror Lost This state occurs on the principal server instance when it cannot connect to the mirror. 7 Manual Failover This state occurs on the principal server instance when the user fails over manually from the principal, or on the mirror server instance when a force service is executed at the mirror. 8 Automatic Failover This state occurs on the mirror server instance when the operating mode is high safety with automatic failover (synchronous) and the mirror and witness server instances cannot connect to the principal server instance. 9 Mirroring Suspended This state occurs on both partner instances when the user suspends (pauses) the mirroring session or when the mirror server instance encounters an error. It also occurs on the mirror server instance following a force service command. When the mirror comes online as the principal, mirroring is automatically suspended. 10 No Quorum If a witness is configured, this state occurs on the principal or mirror server instance when it cannot connect to its partner or to the witness server instance. 11 Synchronizing Mirror This state occurs on the mirror server instance when there is a backlog of unsent log. The status of the session is Synchronizing. 12 Principal Running Exposed This state occurs on the principal server instance when the operating mode is high protection (synchronous) and the principal cannot connect to the mirror server instance. 13 Synchronizing Principal This state occurs on the principal server instance when there is a backlog of unsent log. The status of the session is Synchronizing. Creating an Alert for Automatic Failover capture We are interested in State 8 for this step. State 8 indicates an automatic fail over has occurred. This event is raised on the mirror server when a database that is acting as the mirror, changes state to acting as the principal. The mechanism we use for responding to an event notification is an alert. An alert can be created via SQL Server Management Studio (SSMS), or via T-SQL script. The SSMS method will be shown here. Locate and expand the SQL Server Agent node in the SSMS object tree view. Right click on Alerts and select “New Alert”. Enter a name for the Alert in the Name: attribute box, select WMI event alert in the Type drop down list. The Namespace: value will be populated automatically for the server you are on. Note: you will have to do this on both servers, but you can script the alerts and jobs from the first server and use those scripts on the second server as shown later. In the Query: box you will write the query based on the DATABASE_MIRRORING_STATE_CHANGE event. Select the Response page and check the box for “Execute Job”, then select the “New Job” button. This will create a new job and open the Job properties dialog for you. Enter a job name, owner and select the category as Database Mirroring. If the category “Database Mirroring” is not available select “Uncategorized (Local)”. Enter a description to detail the purpose of the job, and ensure Enabled is checked. Select the Steps page on the left next. And select the “New” Button on the bottom. This will launch the Job Step properties page. Enter a name for the job step you are creating, select the type, in this case we are executing a batch file so select “Operating System (CmdExec)”, and chose the Run As account from the drop down. This will be the user that the process will run under. In the Command box input the batch file for the server you are working on and click “OK”. Back on the Alert properties page now, click “OK” to finish creating the Alert. We now have an Alert that will capture the event that a database has gone from Mirror to Principal and take appropriate action. When the event occurs, the Alert is activated if the state value is 8. The Alert then executes the Job that was created, and the job executes the batch file which executes the psexec utility to execute the batch file on the SharePoint server, which modifies the registry to change the server value for the SQL Server Alias. Creating an Alert for Manual Failover capture A manual fail over is different than an automatic fail over, in that it fires an event with state 7 that occurs on the Principal server where the fail over command originated. Because SharePoint is a multi-database application, it is desired that all databases reside on the same server at all times. For automatic fail overs, it would be rare that the fail over would happen at a database level. Normally it will occur because the server went down, the network card failed, the disks failed, or some other catastrophic event occurred which rendered the Principal server inoperable. Manual fail over occurs when the Fail Over button is pressed in the Mirroring properties of the database or when certain T-SQL commands are issued. Manual fail over occurs at the database level, therefore a mechanism to automate manual fail over for the SharePoint instance, must not only change the SQL Server Alias, but also cause the other SharePoint databases to fail over to the Mirror. We will follow the same steps as for Automatic fail over, but we will add a new step to the job before the alias update step. Locate and expand the SQL Server Agent node in the SSMS object tree view.   Right click on Alerts and select “New Alert”. Enter a name for the Alert in the Name: attribute box, select WMI event alert in the Type drop down list. The Namespace: value will be populated automatically for the server you are on. Note: you will have to do this on both servers, but you can script the alerts and jobs from the first server and use those scripts on the second server as shown later. In the Query: box you will write the query based on the DATABASE_MIRRORING_STATE_CHANGE event. Note that we are looking for State 7 in this event response. Select the Response page and check the box for “Execute Job”, then select the “New Job” button. This will create a new job and open the Job properties dialog for you. Enter a job name, owner and select the category as Database Mirroring. If the category “Database Mirroring” is not available select “Uncategorized (Local)”. Enter a description to detail the purpose of the job, and ensure Enabled is checked. Select the Steps page on the left next. And select the “New” Button on the bottom. This will launch the Job Step properties page. Enter a name for the job step you are creating, select the type, in this case we are a SQL Server query so select “Transact-SQL script (T-SQL)”. Leave the “Run As” box empty. In the Command box input the code shown and click “OK”. Code from above step: SET NOCOUNT OFF DECLARE @strSQL NVARCHAR(4000) DECLARE @strDatabasename NVARCHAR(2000) DECLARE MyCursor CURSOR FOR SELECT name FROM master.sys.databases a INNER JOIN master.sys.database_mirroring b ON a.database_id=b.database_id WHERE NOT mirroring_guid IS NULL AND mirroring_role_desc='PRINCIPAL' OPEN MyCursor FETCH Next FROM MyCursor INTO @strDatabasename WHILE @@Fetch_Status = 0 BEGIN SET @strSQL = 'ALTER DATABASE [' + @strDatabaseName + '] SET PARTNER FAILOVER' EXEC sp_executesql @strSQL FETCH Next FROM MyCursor INTO @strDatabasename END CLOSE MyCursor DEALLOCATE MyCursor This code will force all databases on the server to become the “FailOver Partner” or Mirror, which causes the existing Mirror, or “FailOver Partner” to become the principal. Back on the Job properties Steps page now, click “New” to create another step. This step will be identical to the job step for automatic failover, with the exception of which batch file it calls. Enter the values as below and click “Ok”.   Click “OK” on the Job Properties pages. We now have a job with two steps to force all databases to fail over if one does, and to update the SQL Server Alias after that. Back on the Alert Property page, click “Ok” to finish creating the alert. We now have an Alert that will capture the event that a database has been manually failed over and take appropriate action. When the event occurs, the Alert is activated if the state value is 7. The Alert then executes the Job that was created, and the job executes the T-SQL code to fail over the remaining databases and then the batch file which executes the psexec utility to execute the batch file on the SharePoint server, which modifies the registry to change the server value for the SQL Server Alias. Of course you have to do the same for the other server in the mirroring set. The procedure is the same, just in reverse of what is shown here, as for which batch file to call when. That’s all for this post. Feel free to leave comments or ask questions. I’ll be happy to answer any. I’ve also included some useful links below. Useful Links SQL Server Central Blog http://msdn.microsoft.com/en-us/library/ms188712.aspx http://www.mssqltips.com/tip.asp?tip=1564 Technet Blog On Mirroring

Technorati Tags: SharePoint,Server,Mirrors,Database,Session,installation,cost,client,method,Some,Background,product,FAILOVER,PARTNER,connection,options,settings,configuration,danger,addition,databases,Microsoft,White,Paper,Office,Windows,Services,Alias,data,servers,documentation,Front,Application,Registry,HKLM,SOFTWARE,MSSQLServer,ConnectTo,Name,REG_SZ,port,batch,development,folder,SQLServerMirrorScripts,Note,PowerShell,scripts,preference,DBSERVERALIAS,SQLAlias,DBSERVER,DBMSSOCN,MOSS,PSTools,Page,user,context,argument,execution,Again,location,SharePointServer,agreement,acceptance,Update,response,automation,Service,Broker,mechanism,event,notification,SELECT,FROM,Management,Studio,ALTER,ENABLE_BROKER,Agent,SQLSERVERAGENT,DATABASE_MIRRORING_STATE_CHANGE,events,DBMirrorStateChange,Exists,Where,Begin,Create,DBMirrorService,DROP,QUEUE,State,Description,Null,Principal,Mirror,Lost,instance,Manual,Automatic,mode,instances,error,Quorum,backlog,status,protection,Alert,SSMS,script,Locate,node,tree,Alerts,Enter,Type,Namespace,jobs,Query,Execute,properties,dialog,owner,category,Local,purpose,Steps,Button,Step,System,CmdExec,account,Command,Back,action,card,disks,Fail,Over,Transact,Leave,Code,NOCOUNT,DECLARE,NVARCHAR,MyCursor,CURSOR,INNER,JOIN,OPEN,FETCH,INTO,Fetch_Status,EXEC,CLOSE,DEALLOCATE,exception,Click,procedure,Feel,Useful,Links,Central,Blog,library,Technet,psexec,synchronous,strSQL,strDatabasename,database_id

Database Mirroring on SQL Server 2008

I know it has been a very long time since I posted, and for that I am sorry. Sometimes work and personal life conspire to evaporate any free time you might have. But here I am with a new and hopefully useful bit of hard won experience to share with you.

I was recently engaged at a client to assist them with implementing database mirroring on SQL Server 2008. Everything seemed straight forward enough, they were conducting a proof of concept (POC) around using SQL Server to host their new ERP system and needed to prove the high availability features of automatic failover. No problem so far. They had a primary and secondary server and a virtual server to act as the witness already in place.

 

The wrench got thrown in at our kick-off meeting when the client informed me that since they are already implementing SharePoint 2007, they want the POC to use SharePoint as the application that will utilize the mirrored database set up. I’m not sure how many of you know this, but in order for an application to be natively ready to utilize a SQL Server mirrored environment and facilitate automatic fail-over, it must be capable of implementing the FAILOVER PARTNER attribute of a mirroring aware data source. SharePoint does not. There are many posts around the net on DB Mirroring, and SharePoint on mirrors, but I failed to find a comprehensive source that provided all the information I needed. Hence, my contribution to the community via this post. Actually this and the next post. I plan to break it up into a pair of posts, one that deals with the setting up of the mirroring, and one that deals specifically with the steps necessary to implement SharePoint on the mirroring session and provide automatic fail-over.

So in this post we will go through the process of creating a mirroring session and implementing mirroring for a database. We’ll use the AdventureWorks2008 sample database that Microsoft was so kind to create for us.

Any time you embark on configuring something in SQL Server, it is a good idea to be sure all the prerequisites are met ahead of time. There is a good source for prerequisites on SQL Server database mirroring at Microsoft TechNet. Here is the link Prerequisites, Restrictions, and Recommendations for Database Mirroring

Some things to note: Make sure both partners are using the same version and EDITION of SQL Server 2008. If you are running Enterprise on the primary, you should run Enterprise on the secondary. Both servers should also utilize the same default collation. You can not implement mirroring on database servers that have implemented the FILESTREAM option. Use of a third server, which can be a different Edition of SQL Server 2008, even EXPRESS, is required to fill the role of witness if you are implementing high availability mode (high safety with automatic fail-over).

So let’s begin. The first step is to ensure we have 3 server instances available to us. In a production application, these will be 3 separate servers, even if any or all are virtual. If you are doing this on your own, you can install a named instance of SQL Server along with your default instance and install SQL Server Express to act as the witness.

 

Once you have established your servers, you will need to ensure you have met all the requirements and pre-requirements as laid out in the above link. When you are ready to begin, follow the steps below.

Step 1:
Set Recovery Model to FULL.

 

Step 2:
Backup the database.

Step 3:
Copy the backup to the designated mirror instance.
Step 4:
Restore the backup on the designated mirror instance with the options selected as below.

 

You must choose the “Leave the database non-operational, and do not roll back uncommitted transactions. Additional transaction logs can be restored (RESTORE WITH NORECOVERY)” option for database mirroring.

Step 5:
On the designated principal instance, right click the database and select “Tasks” and then “Mirror”. This will launch the Database Mirroring Wizard.

 

Click “Next” and move through the wizard, selecting the Principal or primary server, the secondary or Mirror server, and the Witness server. In order to achieve automatic fail over a witness server is required.

There will be an identical screen on which to select the Witness server. The wizard creates and configures the endpoints for you.

These accounts are used for communication between the three servers.

 

Final check before finishing.

 

This screen shows the configuration progress for each server. Any warnings or errors will be caught and displayed here. When this screen finishes, you will be prompted to start mirroring.

 

Once the process concludes and the databases have synchronized, the mirroring session is started and all transactions made at the principal will be duplicated on the mirror. In the event of a failure of the primary/principal the Witness and the Mirror form a quorum and determine to switch control to the mirror, which exits the restoring state and becomes a disconnected principal. When the mirroring session is started the Mirror page of the database properties is as shown below.

 

You will see in the database object tree view in SQL Server Management Studio, that one database shows Principal, synchronized, and one shows Mirror, synchronized.

That’s it. You have set up mirroring on your database. Now when a problem occurs with the principal, the mirror will take over operation as the principal database. Any mirroring aware applications can implement the FAILOVER PARTNER attribute of the connection string to ensure continuous operation through a failed database server.

You can manual fail over the database from primary server to the secondary server in two ways.

One: Open the properties for the database and select the Mirror page. There is a button “Failover” that if you press, will manually fail over the database from the primary to the secondary.
Two: You can issue a T-SQL Query to fail over the database. On the primary server with the principal database execute the following code;
ALTER DATABASE AdventureWorksDW SET PARTNER FAILOVER

That’s all there is too setting it up. Of course there is a ton more information on Microsoft MSDN and TechNet as well as in BOL. Look it up, there’s much, much more to learn about database mirroring.

In my next post we will continue from here and implement automatic failover for a SharePoint installation. Until thin, keep slinging the SQL.

Change Tracking (Not CDC)

Change Tracking has a similar name to Change Data Capture and has caused some minor confusion. Change Tracking, however, is entirely different and serves a separate purpose.

CDC is an asynchronous process that captures row level changes and stores them in special change tables. This information is available as relational data and can be queried by client applications.

Change Tracking is synchronous and tracks data changes but stores only the fact that they were changed and the last value for the row. Both one-way and two-way synchronization is supported, but remember that with two-way conflict detection is supported but the client  is responsible for handling it.

Change Tracking uses tracking tables to store the primary key of the modified rows along with version numbers. It is easy to detect version conflicts.

1. An application requests the version number for a row it intends to modify.
2. If the version has changed since the last request, there is a conflict.
3. Solving the conflict is up to the requesting application.

In order to work with Change Tracking there are a few steps to do.

1. Enable Change Tracking at the database level.
2. Enable Change Tracking at the table level.
3. Use CHANGE_TRACKING_MIN_VALID_VERSION function to get the minimum version, which is an integer. This value can be used to get the changes for a specific table.
4. Use CHANGETABLE function to get information , such as changes, types of changes, columns that changed, etc.
5. Use CHANGE_TRACKING_CURRENT_VERSION function to get the current version. This is set by the last transaction committed in the database.
6. Use the WITH CHANGE_TRACKING_CONTEXT() hint to specify a context for data modifications. This allows data modifications to be grouped according to client or application.

Let's look at some code that will show some of the basics.

USE master;
GO
--Create a test database if necessary
IF NOT EXISTS(SELECT * FROM sys.databases WHERE [name]=N'MyDB')
    CREATE DATABASE MyDB;
GO
--Configure the database to allow change tracking
IF NOT EXISTS(SELECT * FROM sys.change_tracking_databases 
WHERE database_id = (SELECT database_id FROM sys.databases WHERE [name]=N'MyDB'))
    ALTER DATABASE MyDB SET CHANGE_TRACKING = ON 
    (CHANGE_RETENTION = 2 DAYS, AUTO_CLEANUP = ON);
GO
--Configure the database to use snapshot isolation
ALTER DATABASE MyDB SET ALLOW_SNAPSHOT_ISOLATION ON;
GO
USE MyDB;
GO
--Create a test table
IF NOT EXISTS(SELECT * FROM sys.tables WHERE [name]=N'testTbl')
    CREATE TABLE testTbl(testID INT IDENTITY(1,1) PRIMARY KEY 
    NOT NULL, VAL VARCHAR(64));
GO
--Enable change tracking on the table
IF NOT EXISTS(SELECT * FROM sys.change_tracking_tables WHERE [object_id] = OBJECT_ID(N'testTbl'))
    ALTER TABLE testTbl ENABLE CHANGE_TRACKING;
GO
--Examples of using the Change Tracking functions.
SELECT CHANGE_TRACKING_CURRENT_VERSION();
SELECT CHANGE_TRACKING_MIN_VALID_VERSION(OBJECT_ID(N'testTbl'));INSERT INTO testTbl(VAL) VALUES('V01');
SELECT CHANGE_TRACKING_CURRENT_VERSION();
SELECT CHANGE_TRACKING_MIN_VALID_VERSION(OBJECT_ID(N'testTbl'));
INSERT INTO testTbl(VAL) VALUES('V02'),('V03'),('V04');
SELECT CHANGE_TRACKING_CURRENT_VERSION();
SELECT CHANGE_TRACKING_MIN_VALID_VERSION(OBJECT_ID(N'testTbl'));

The example code above shows the initial values for Change Tracking as well as the changes when you update a row's data. To use Change Tracking you need to reconnect to check that you have a valid version of the data before you process it. Below shows the one-way synchronization scenario.

--This shows the one way sync scenario
SET TRANSACTION ISOLATION LEVEL SNAPSHOT;
DECLARE @Last_sync_version INT = 0;
DECLARE @sync_version INT;
--Use transaction for consistency
BEGIN TRANSACTION
--Make sure the last_sync_version is valid
IF (@Last_sync_version < CHANGE_TRACKING_MIN_VALID_VERSION(OBJECT_ID(N'testTbl')))
    RAISERROR(N'Invalid value', 10, 1);
ELSE
BEGIN
    --Get the current for the next sync
    SET @sync_version = CHANGE_TRACKING_CURRENT_VERSION();
    --Show the current tracking version and change version
    --for each row.
    SELECT testTbl.testID, testTbl.VAL, CT.SYS_CHANGE_VERSION CV,
        CT.SYS_CHANGE_CREATION_VERSION CCV, CT.SYS_CHANGE_OPERATION CO,
        CT.SYS_CHANGE_COLUMNS CC, CT.SYS_CHANGE_CONTEXT CContext
    FROM CHANGETABLE(CHANGES testTbl, @Last_sync_version) AS CT
    LEFT JOIN testTbl ON testTbl.testID = CT.testID;
    SELECT @sync_version;
END
COMMIT TRANSACTION
GO

--Show another session updating the data
UPDATE testTbl SET VAL='New V01' WHERE testID = 1;
INSERT INTO testTbl(VAL) VALUES('V05');
DELETE FROM testTbl WHERE testID = 3;

--The initial session reconnects to get
--changes
SET TRANSACTION ISOLATION LEVEL SNAPSHOT;
DECLARE @Last_sync_version INT = 2;
DECLARE @sync_version INT;
BEGIN TRANSACTION
--Check that you have a valid version
IF(@Last_sync_version < CHANGE_TRACKING_MIN_VALID_VERSION(OBJECT_ID(N'testTbl')))
    RAISERROR(N'Invalid value', 10, 1);
ELSE
BEGIN
    SET @sync_version = CHANGE_TRACKING_CURRENT_VERSION();
    SELECT testTbl.testID, testTbl.VAL, CT.SYS_CHANGE_VERSION CV,
        CT.SYS_CHANGE_CREATION_VERSION CCV, CT.SYS_CHANGE_OPERATION CO,
        CT.SYS_CHANGE_COLUMNS CC, CT.SYS_CHANGE_CONTEXT CContext
    FROM CHANGETABLE(CHANGES testTbl, @Last_sync_version) AS CT
    LEFT JOIN testTbl ON testID.Tbl.MyID = CT.MyID;
    SELECT @sync_version;
END
COMMIT TRANSACTION
GO

There is also two-way synchronization, which is shown below. In the scenario shown, an attempt is made to update the first row. If it can't update the row, the execution path follows the conflict resolution code. This example only uses a table with 2 columns, but in real-world scenarios a table will have far more columns, some of which may have been updated. In this situation, if you are updating columns in a table that haven't been updated, you may choose to proceed.

--Two-way synchronization scenario.
--Assumes the last sync occurred when the sync value was 2
SET TRANSACTION ISOLATION LEVEL SNAPSHOT;
DECLARE @Last_sync_version INT = 2;
DECLARE @sync_version INT;
DECLARE @current_row_version INT;
--Find the current row version for testID=1
SELECT @current_row_version = ISNULL((SELECT CT.SYS_CHANGE_VERSION 
FROM CHANGETABLE(VERSION testTbl, (testID), (1)) CT),0)
SELECT @current_row_version;
BEGIN TRANSACTION
IF (@Last_sync_version < CHANGE_TRACKING_MIN_VALID_VERSION(OBJECT_ID(N'testTbl')))
    RAISERROR(N'Invalid value',10,1);
ELSE
BEGIN
    --Attempt to update the 1st row
    UPDATE testTbl SET VAL = 'NewAppValue'
    FROM testTbl WHERE testID = 1 AND @Last_sync_version >= @current_row_version;
END
IF (@@ROWCOUNT = 0)
BEGIN
    --A conflict is indicated by the execution path (@@ROWCOUNT = 0)
    --Gather more information to see if the VAL column has been updated.
    IF EXISTS(SELECT CHANGE_TRACKING_IS_COLUMN_IN_MASK(2, CT.SYS_CHANGE_COLUMNS) CC
    FROM CHANGETABLE(CHANGES testTbl,@Last_sync_version) CT
    WHERE CT.testID = 1 AND 
    CHANGE_TRACKING_IS_COLUMN_IN_MASK(2,CT.SYS_CHANGE_COLUMNS)=1)
        RAISERROR(N'The column VAL was changed', 10, 1);
        RAISERROR(N'An update conflict has occurred', 10, 1);
END
COMMIT TRANSACTION

Change Tracking fits into Microsoft's "Connect to your data from any device" theme for SQL Server 2008, which is made up of the Microsoft Sync Framework, Sync Services for ADO.Net, SQL Server Compact Edition 3.5. It provides for conflict detection and easy data retrieval of changes with minimum performance impact, and automatic setting for data retention. There are several advantages, such as no schema changes on the tracked tables (no need for a timestamp column), security is at the table level and prevention of loopbacks is easy.

As always, please leave your comments or send email to sql.slinger@gmail.com. I'm always interested in your feedback.

Until next time, happy slinging.

Technorati Tags: SQL Server,SQL Server 2008,Change Tracking,Versioning,Version,Row Version,Tracking,Conflict Resolution

Beyond Auditing

Sometimes capturing the fact that someone changed something and who that someone is, isn't enough. Sometimes, if the values in a row of a table were changed, you need to know what the original values were along with who changed them. Into this space we introduce SQL Server 2008 Change Data Capture (CDC).
This new feature of SQL Server records Data Modification Language operations (INSERT, UPDATE, DELETE) on user tables. The changes are exposed using table-valued functions. The CDC asynchronously reads changes in the source tables from the transaction log and inserts those changes into separate change tables. This feature is available in Enterprise, Developer, and Evaluation editions only.

Before we can use this feature we have to configure the database for it. A member of the sysadmin server role must enable CDC on the database by running the sys.sp_cdc_enable_db stored procedure. You can also enable CDC for individual tables only by running sys.sp_cdc_enable_db_change_data_capture.

--First, let's create a demonstration database with a table in it
USE [master];
GO
IF NOT EXISTS (SELECT name FROM sys.databases WHERE name = N'DemoDB')
    CREATE DATABASE [MyDB];
GO
USE [MyDB];
GO
CREATE TABLE MyTable
(MyID INT PRIMARY KEY NOT NULL,
 MyItem VARCHAR(128),
 MyQty INT);
 GO
 
--Next, let's enable CDC on the database
EXEC sys.sp_cdc_enable_db;
GO
--Enable CDC for the Products tableEXEC sys.sp_cdc_enable_table
@source_schema = N'dbo',
@source_name   = N'MyTable',
@role_name     = NULL,
@capture_instance = N'MyTbl_1',
@captured_column_list = N'MyID, MyItem, MyQty',
@filegroup_name = N'PRIMARY',
@supports_net_changes = 1
GO

When you run the two system stored procedures you will see results like these;

Job 'cdc.MyDB_capture' started successfully.
Job 'cdc.MyDB_cleanup' started successfully.

Notice that two jobs are created to monitor/capture and to cleanup the data capture tables.
Now, we have our table, let's put some data into it and modify that data so we can see CDC in action.

--Now let's add some initial data
INSERT INTO MyTable(MyID,MyItem,MyQty) 
Values(1,'Item 1',20),(2,'Item 2',10),(3,'Item 3',1),(4,'Item 4',3);
GO--And change it
UPDATE MyTable SET MyQty+=10 WHERE MyID=1;
GO



Command(s) completed successfully.

Now, when we enabled CDC for the database and the table in it, SQL Server created a pair of Table Valued Functions for us. The functions are named as cdc.fn_get_all_changes_'capture_instance', where 'capture_instance' is the value we specified for @capture_instance in our sp call to enable cdc on the table. In our case they are cdc.fn_cdc_get_all_changes_MyTbl_1. There are also several built-in functions for CDC that make querying information about CDC tables and instances easier. (See BOL for details). We will use two of the built-in functions in our example below.
Here they are in action;

DECLARE @from_lsn binary(10), @to_lsn binary(10);
SELECT @from_lsn = sys.fn_cdc_get_min_lsn(N'MyTbl_1');
SELECT @to_lsn = sys.fn_cdc_get_max_lsn();
SELECT * FROM cdc.fn_cdc_get_all_changes_MyTbl_1(@from_lsn,@to_lsn,N'all');
SELECT * FROM cdc.fn_cdc_get_net_changes_MyTbl_1(@from_lsn,@to_lsn,N'all');
GO

And the results from the above query;
 
Notice the difference in the results from the two functions. The first function tracks history for every row in the table. It shows Item 1 as it was insert and again after it's update. The second function however, doesn't show the history, but just the net changes. Notice that the initial insert value for Item 1 is missing, only the current value is shown.

To read more on this topic check out the section in SQL Server 2008 Books Online. See all the available system functions associated with it. BOL topic is Change Data Capture.

Until next time, happy slinging.

Technorati Tags: SQL Server 2008,CDC,Change Data Capture,Auditing,Audit,Tracking,T-SQL,Database audit,database tracking,change tracking,data history

Productivity enhancements for SSMS

SQL Server 2008 includes several enhancements geared toward improving productivity and customer satisfaction. Some of the enhancements have been long sought after by the SQL Server database developer and administrator.

IntelliSense!
Yes we have long awaited the ability to write T-SQL code with IntelliSense helping code faster. IntelliSense with SQL 08 provides word completion and displays parameter information for functions and stored procs. In the XML editor, IntelliSense can completely show an element declaration. It also indicates errors with red squiggly lines and references them in the Error List window, so you can quickly navigate to the error line by double-clicking the entry. IntelliSense is provided in the Query editor as well as the XML editor, and supports most, but not all, T-SQL syntax. Below are some screen-shots of IntelliSense at work.

Collapsible Regions 
Similar functionality is now provided in the SMS08 Query Editor to regions delimited by BEGIN...END blocks and multi-line statements, such as SELECT statements that span two or more lines. This is similar to what Visual Studio has offered for a few releases now. Here's an example (note the '-' sign on the left, click to collapse text):

Delimiter Matching
Also similar to Visual Studio, SQL Server 2008 Management Studio now offers delimiter matching and highlighting. When you finish typing the second delimiter in a pair, the editor will highlight both the delimiters, or you can press CTRL + ] to jump to the matching delimiter when you are on one of them. This will help you keep track of parenthesis' and nested blocks. Automatic delimiting will recognize these delimiters;
(...), BEGIN...END, BEGIN...END TRY, BEGIN...END CATCH. Brackets and quotes are not recognized for delimiter highlighting.

SMS Object Explorer
More context menu choices have been provided for your right-click menu in Object Explorer. These choice include options for changing table design, to opening the table with a certain number of returning records, to getting some of the new reports available. Options for partitioning (Storage), Policies, and Indexing are provided as well.

You will notice a new "Start PowerShell" option. Look for a future post to cover the new Windows PowerShell integration.

Integrated T-SQL Debugging
Debugging has now been integrated into the Query Editor! You can set breakpoints, step through code, step into code a particular location, and even set watches up to monitor variable values, locals and the call stack. Woohoo!!!!

Other improvements
Multi-Server queries, allow you to run a query across numerous servers and return the results with the server name prepended.
Launch SQL Profiler directly from Management Studio.
Customizable tabs, allow you to modify the information shown and the layout from the tools\options dialog.
Object Explorer Detail Pane has been improved for better functionality and productivity with navigational improvements, detailed object information pane at the bottom, and integrated object search.
Object Search, allow you to search within a database on a full or partial string match and return objects to the Object Explorer Details pane.
Activity Monitor, you have to see it to appreciate it. Built brand new from scratch, it is based on the Windows Resource Monitor and allows you to see graphs of processor wait time, waiting processes, database I/O and batch requests. Detail grids are provided for Processes, Resource Waits, Data File I/O, and Recent Expensive Queries. It's a vast improvement for the DBA.
Performance Studio, new performance tuning tool that tracks historical performance metrics and stores them using drill-through reports.
Partitioning Setup GUI, at last a way to create and manage table partitions graphically. This is accessed from the context menu for a table under Storage. A wizard will launch allowing you to create or manage partitions.
Service Broker Hooks, new context menu items centralize access to the Service Broker T-SQL templates for messages, contracts, queues, etc. Read only property pages are provided for each of these objects as well.

The improvements in the toolset provided with SQL Server 2008 are vast and far sweeping. There are more for you to discover and the best way is to get your hands on it. You can download SQL Server Express 2008 from Microsoft if you don't have a developers edition license.

Please, as always, post your comments and questions. Until next time, happy slinging.

Technorati Tags: SQL Server 2008,SQL Server,SSMS,SMS,SQL Server Management Studio,T-SQL,Query Editor,Enhancments,Object Explorer,Object Explorer Detail view,Results Grid,Activity Monitor,SQL Tool Enhancemenets.