tim laqua dot com » sql

Uncivil UNIONs

Tim — Thu, 02 Feb 2012 16:12:43 +0000

If you asked any SQL Server Developer or DBA what UNION does, they would tell you it combines two result sets. Then if you were to ask them if they knew it removed duplicates without the ALL argument, they would say yes. That's academic, but I dare you to go look at their and your code and see how often UNION without the ALL argument is used. UNION is evil. Why do I say that? Because DISTINCT is evil and blocking operations are evil so it follows that UNION is evil.

Would you be OK with the following query as the source for a fact table?

SELECT DISTINCT *
FROM
(
  SELECT DISTINCT [c1],[c2] FROM [t1]
  UNION ALL
  SELECT DISTINCT [c1],[c2] FROM [t2]
) a

No? Then why are you OK with this?

SELECT [c1],[c2] FROM [t1]
UNION
SELECT [c1],[c2] FROM [t2]

Personally, I tend to use UNION to combine data from two different sources. For example, if I wanted to combine the sales transactions from two JDE instances - the query might look like:

SELECT [c1],[c2] FROM [JDE1].[dbo].[F42119]
UNION
SELECT [c1],[c2] FROM [JDE2].[dbo].[F42119]

Each line here in a sales detail represents a line on an order, so assuming you selected all the PK columns in your query, a UNION won't remove any rows within either set, but when combined - it's possible the same PK could exist in both systems. Now you could split hairs and say it's highly unlikely the rest of the row would be identical, but that's not the point - it's wrong to ever DISTINCT a set of transactions intended for creating a fact table. Sure, in this particular case you could add a "SourceSystem" column that would stop that, but then you still have to accept that the UNION is blocking and expensive. Just because a DISTINCT doesn't modify results doesn't mean it didn't do any work - it actually did a great deal of work. Now consider that we're usually not selecting 2 columns, we're selecting 10-20. Do you want to wait for it to figure out all the rows are distinct when you already knew that? Further, if they're not distinct you probably screwed up a JOIN somewhere.

As far as coding standards regarding UNION go, I would suggest never using UNION without the ALL clause and doing your DISTINCT elsewhere if that's actually what you intended. Does that seem silly? Yes - but I would wager you won't ever use the 2nd one outside of building Dimensions.

SELECT [c1],[c2] FROM [JDE1].[dbo].[F42119]
UNION ALL
SELECT [c1],[c2] FROM [JDE2].[dbo].[F42119]
 
--OR
 
SELECT DISTINCT * FROM
(
  SELECT [c1],[c2] FROM [JDE1].[dbo].[F42119]
  UNION ALL
  SELECT [c1],[c2] FROM [JDE2].[dbo].[F42119]
)

And don't get me started on unioning subqueries with GROUP BY's - Why don't you just throw half your processers in the garbage, light tempdb on fire and put all your data in heaps.

SQL Schema Source Control 2.0

Tim — Thu, 21 Apr 2011 11:30:26 +0000

SQL Schema Source Control is, at it's core, just another database scripting app. Where this app differs from others is that the sole purpose of scripting out the database is to check it in to source control (sorry, it still only supports SVN).

I think this may have once gone under the name "SQL2SVN," but was rebranded before being checked in to Codeplex for the first time. This is a re-write of most of the application to be more modular and extensible in the future. Additional features found in this release are:

Scripting of Configuration settings via sp_configure
Scripting of Logins
Scripting of Indexes
Scripting of Users
Scripting of Views
Scripting of Keys (Primary and Foreign)
Scripting of Constraints
Scripting of Defaults
Scripting of Stored Procedures
Scripting of Functions
Scripting of Partition Schemes
Scripting of Partition Functions
Regular Expression support for database names, allows you to script all databases excluding system databases... or including them... whatever

The application and source code are available on Codeplex: SQL Schema Source Control

Monitoring Failed Report Server Subscriptions

Tim — Tue, 19 Apr 2011 11:30:51 +0000

While we love to empower our users as much as possible, we still need to pay attention to them and what they're up to. The first place I usually see users running amok as Microsoft BI adoption grows in organization is the Report Subscriptions in SSRS. They go nuts with them. I was at one company where 50% of the report subscriptions went to users outside of the company - that means our customers were depending on these subscriptions! Which brings up an interesting licensing debacle... but that's another story. Needless to say, when people set up a subscription, they expect it to work. If it doesn't, we REALLY need to let someone know.

Here, we have a procedure to monitor for Reporting Services subscription failures. Specifically, email subscription failures. When a failed subscription is detected, an email is sent to both the subscription owner and a digest of failures is sent to the specified admin group.

This procedure has saved me so many times it's not even funny. The last thing you want is your end users knowing about a failure before you.

/************************************************************************
***
*** Procedure:       Alert_FailedReportingServicesSubscriptions
*** Purpose:         Notifies administrators and subscription owners that
***                  subscriptions were not delivered as expected
***      
*** Author:          tl
*** Date Created:    2009-10-28
*** 
*** Revision History
*** Date        Author   Description
*** 2009-10-28  tl       Created
*** 
************************************************************************/
CREATE PROCEDURE [dbo].[Alert_FailedReportingServicesSubscriptions]
   @ReportServerBaseURL VARCHAR(1024)
  ,@AdminEmail VARCHAR(255)
  ,@FromAddress VARCHAR(255)
  ,@MailDomain VARCHAR(255)
  ,@NTDomain VARCHAR(255)
  ,@ReportServerDatabase VARCHAR(255) = 'ReportServer'
  ,@ReportServerSchema VARCHAR(255) = 'dbo'
  ,@HoursToCheck INT = 1
  ,@HourSchedule INT = 16777215
AS
BEGIN
  SET NOCOUNT ON;
 
  IF @HourSchedule & POWER(2, DATEPART(HOUR, GETDATE())) > 0
  BEGIN
    DECLARE @ReportTableOpen VARCHAR(MAX) =
    '
    The following subscription(s) that you created was/were not sent to specified recipients at the date/time you scheduled.  Database Administrators has been notified.  Please reply to this email if you have any questions.
    '
 
    DECLARE @ReportTableClose VARCHAR(MAX)='
    Owner Report Last Run Intended Recipients Status
  
'
 
    CREATE TABLE #FailedReport
      ([Id] INT IDENTITY(1,1)
      ,[Name] VARCHAR(255)
      ,[Link] VARCHAR(2048)
      ,[LastRunTime] DATETIME
      ,[OwnerUsername] VARCHAR(255)
      ,[IntendedRecipients] VARCHAR(1024)
      ,[Message] VARCHAR(1024))
 
 
    DECLARE @SQL VARCHAR(MAX) = '
    INSERT INTO #FailedReport
      ([Name]
      ,[Link]
      ,[LastRunTime]
      ,[OwnerUsername]
      ,[IntendedRecipients]
      ,[Message])
    SELECT
       b.Name
      ,''+ @ReportServerBaseURL + ''' + b.Path + ''">'' + b.Name + '''' AS [Link]
      ,LastRunTime 
      ,REPLACE(c.UserName, ''' + @NTDomain + CHAR(92) + ''', '''') AS [OwnerUserName]
      ,REPLACE(CAST(ExtensionSettings AS XML).value
        (''(//ParameterValue[Name="TO"]/Value)[1]''
        ,''VARCHAR(1024)'')
      + '';'' 
      + ISNULL(CAST(ExtensionSettings AS XML).value
        (''(//ParameterValue[Name="CC"]/Value)[1]''
        ,''VARCHAR(1024)''), ''''),'';;'','';'') AS [IntendedRecipients]
      ,LEFT(ISNULL(LastStatus, ''''), 1024) AS [Message]
    FROM ' + @ReportServerDatabase + '.' + @ReportServerSchema + '.Subscriptions a
      INNER JOIN ' + @ReportServerDatabase + '.' + @ReportServerSchema + '.Catalog b WITH(NOLOCK) 
        ON a.Report_OID=b.ItemId
      INNER JOIN ' + @ReportServerDatabase + '.' + @ReportServerSchema + '.Users c WITH(NOLOCK)
        ON a.OwnerID = c.UserID
    WHERE 
      a.DeliveryExtension = ''Report Server Email''
      AND LastStatus NOT LIKE ''Mail Sent%'' 
      AND LastStatus NOT LIKE ''% 0 errors.''
      AND LastStatus NOT LIKE ''New Sub%''
      AND LastStatus NOT LIKE ''Pending%''
      AND LastRunTime >= DATEADD(HOUR, -1 * ' + CAST(@HoursToCheck AS VARCHAR(10)) + ', GETDATE())
    ORDER BY REPLACE(c.UserName, ''' + @NTDomain + CHAR(92) + ''', ''''), b.Name'
 
    EXECUTE(@SQL)
 
    IF (SELECT COUNT(1) FROM #FailedReport) > 0
    BEGIN
      DECLARE 
         @FullReportTable VARCHAR(MAX) = ''
        ,@LastOwnerId INT = 0
        ,@Username VARCHAR(255)
 
      DECLARE @Owner TABLE
          ([OwnerId] INT IDENTITY(1,1)
          ,[Username] VARCHAR(255))
 
      DECLARE @Notification TABLE
        ([NotificationId] INT IDENTITY(1,1)
        ,[TO] VARCHAR(255)
        ,[Body] VARCHAR(MAX))
 
      INSERT INTO @Owner
        ([Username])
      SELECT DISTINCT OwnerUsername
      FROM #FailedReport
      ORDER BY OwnerUsername
 
      DECLARE @UserFailedReport TABLE
        ([Id] INT IDENTITY(1,1)
        ,[Name] VARCHAR(255)
        ,[Link] VARCHAR(2048)
        ,[LastRunTime] DATETIME
        ,[OwnerUsername] VARCHAR(255)
        ,[IntendedRecipients] VARCHAR(1024)
        ,[Message] VARCHAR(1024))
 
      WHILE EXISTS (SELECT TOP 1 1 FROM @Owner WHERE [OwnerId] > @LastOwnerId)
      BEGIN
        DELETE FROM @UserFailedReport
 
        SELECT TOP 1 
           @Username = [Username]
          ,@LastOwnerId = [OwnerId]
        FROM @Owner 
        WHERE [OwnerId] > @LastOwnerId
        ORDER BY [OwnerId]
 
        DECLARE 
           @UserReportTable VARCHAR(MAX) = ''
          ,@LastId INT = 0
 
        INSERT INTO @UserFailedReport
          ([Name]
          ,[Link]
          ,[LastRunTime]
          ,[OwnerUsername]
          ,[IntendedRecipients]
          ,[Message])
        SELECT
           [Name]
          ,[Link]
          ,[LastRunTime]
          ,[OwnerUserName]
          ,[IntendedRecipients]
          ,[Message]
        FROM #FailedReport
        WHERE [OwnerUsername] = @Username
 
        WHILE EXISTS (SELECT TOP 1 1 FROM @UserFailedReport WHERE [Id] > @LastId)
        BEGIN
 
          SELECT TOP 1
             @UserReportTable += '' + ISNULL('+ [OwnerUsername] + '@' + @MailDomain + '">' + [OwnerUsername] + '', 'Unknown') + '' + ISNULL([Link], 'Unavailable') + '' + ISNULL(CONVERT(VARCHAR, [LastRunTime], 120), 'Never') + '' + ISNULL([IntendedRecipients], 'Unknown') + '' + ISNULL([Message],'Unknown') + '' + CHAR(13)
            ,@LastId = [Id]
          FROM @UserFailedReport
          WHERE [Id] > @LastId
          ORDER BY [Id]
        END
 
        SET @FullReportTable += @UserReportTable + CHAR(13)
 
        SET @UserReportTable = @ReportTableOpen + @UserReportTable + @ReportTableClose
 
        INSERT INTO @Notification ([TO], [Body]) VALUES (@Username + '@' + @MailDomain, @UserReportTable)
      END
 
 
      SET @FullReportTable = @ReportTableOpen + @FullReportTable + @ReportTableClose
 
      INSERT INTO @Notification ([TO], [Body]) VALUES (@AdminEmail, @FullReportTable)
 
      DECLARE 
         @LastNotificationId INT = 0
        ,@TO VARCHAR(255)
        ,@Body VARCHAR(MAX)
 
      WHILE EXISTS (SELECT TOP 1 1 FROM @Notification WHERE [NotificationId] > @LastNotificationId)
      BEGIN
 
        SELECT TOP 1 
           @TO = [TO]
          ,@Body = [Body]
          ,@LastNotificationId = [NotificationId]
        FROM @Notification
        WHERE [NotificationId] > @LastNotificationId
        ORDER BY [NotificationId]
 
        EXEC msdb.dbo.sp_send_dbmail
           @recipients = @TO
          ,@from_address = @FromAddress
          ,@subject = 'Failed Report Manager Subscription(s)'
          ,@body = @Body
          ,@importance = 'High'
          ,@body_format = 'HTML'  
 
        --DECLARE @ProcName VARCHAR(255) = OBJECT_NAME(@@PROCID)
 
        --EXEC [dbo].[Utility_Alert_SendMail] 
        --   @AlertSubject = 'Failed Report Manager Subscription(s)'
        --  ,@AlertRecipients = @To
        --  ,@AlertBody = @Body
        --  ,@AlertFormat = 'HTML'
        --  ,@AlertSource = @ProcName
        --  ,@FromAddress = @FromAddress
 
      END
    END
 
    DROP TABLE #FailedReport
  END
END

Now to run it:

EXEC [Alert_FailedReportingServicesSubscriptions] 
   @ReportServerBaseURL = 'http://reports.yourcompany.com/Reports/Pages/Report.aspx?ItemPath='
  ,@AdminEmail = 'Business Intelligence '
  ,@FromAddress = 'Business Intelligence '
  ,@MailDomain = 'yourcompany.com'
  ,@NTDomain = 'AWESOMECO'

Set paramaters as follows:

@ReportServerBaseURL VARCHAR(1024)
-- The url string needed view a report (including the ItemPath querystring variable
-- An example is 'http://reports.yourcompany.com/Reports/Pages/Report.aspx?ItemPath='
-- You can obtain this by simply viewing a report and looking at the address bar	
 
@AdminEmail VARCHAR(255)
-- Email that will receive digest emails
 
@FromAddress VARCHAR(255)
-- From email, they should be able to reply to this with questions
 
@MailDomain VARCHAR(255)
-- Domain that you can send email to by appending their user name to
-- So if you set this to 'example.com' and your username is EXAMPLE\tlaqua
-- it would attempt to send an email to tlaqua@example.com
 
@NTDomain VARCHAR(255)
-- NTDomain for usernames.  If your username is EXAMPLE\tlaqua, 'EXAMPLE'
-- is the @NTDomain
 
@ReportServerDatabase VARCHAR(255) = 'ReportServer'
-- Name of ReportServer database - MUST be on the server procedure runs on 
 
@ReportServerSchema VARCHAR(255) = 'dbo'
-- Schema for ReportServer tables
 
@HoursToCheck INT = 1
-- This should conicide with how often you run the procedure
-- If you run the procedure hourly, set this to 1 so it checks 1 hour back
-- If you run the procedure daily, set it to 24
 
@HourSchedule INT = 16777215
-- Bitmask for hours of the day to run.  If you want it to only run if
-- it is 9AM, 6PM, and 11PM, set this value to 8651264:
-- POWER(2, 9) + POWER(2, 18) + POWER(2, 23) = 8651264

Locating Rampant Database Growth

Tim — Fri, 19 Mar 2010 17:40:02 +0000

Every now and then you get a helpful alert from one of your database servers about disk space being low (you're monitoring that, right?), or a bunch of autogrowth alerts (you're monitoring that too, right?) - but what happens when you get these for a database that you don't expect growth in? Further, what happens when that database is growing rampantly (say like 1GB/hr in my case) and it's a canned database from a 3rd party product? This time it was the database that SolarWinds uses for collection - and apparently it was collecting a lot of something that it wasn't collecting before.

you send out an email asking the end users of the system (IT in this case) if anything changed
adjust so said system stops trying to fill up your drives

Unfortunately, there's often quite a few possibilities for "why" a database is growing - and when it's a canned product, you don't always have the best understanding of why it does what it does when it does it. As a Database Admin, you can help diagnose the problem by letting everyone know what exactly is growing:

First, figure out what table is growing:
spSpaceUsed.sql

CREATE TABLE #Temp_SpSpaceUsed
	([name] VARCHAR(255)
	,[rows] INT
	,[reserved] VARCHAR(255)
	,[DATA]	VARCHAR(255)
	,[index_size] VARCHAR(255)
	,[unused] VARCHAR(255)
	,[CreatedTime] DATETIME DEFAULT GETDATE())
 
EXEC sp_msforeachtable 'INSERT INTO #Temp_SpSpaceUsed ([name],[rows],[reserved],[data],[index_size],[unused]) EXEC sp_spaceused [?]'
 
WAITFOR DELAY '00:01:00'
 
EXEC sp_msforeachtable 'INSERT INTO #Temp_SpSpaceUsed ([name],[rows],[reserved],[data],[index_size],[unused]) EXEC sp_spaceused [?]'
 
;WITH SpSpaceUsed AS
(
	SELECT 
		[name]
		,CAST(LEFT([DATA], LEN([DATA]) - 2) AS INT) AS [DATA]
		,[CreatedTime]
		,ROW_NUMBER() OVER (PARTITION BY [name] ORDER BY [CreatedTime] ASC) AS [RowNumber]
	FROM #Temp_SpSpaceUsed	
)
SELECT 
	 a.[name]
	,b.DATA - a.DATA AS [GrowthKB]
	,(CAST(b.DATA - a.DATA AS FLOAT) / DateDiff(MILLISECOND, a.CreatedTime, b.CreatedTime)) * 1000.0 * 60.0 * 60.0 AS [GrowthKBPerHour]
FROM SpSpaceUsed a
	INNER JOIN SpSpaceUsed b
		ON a.[name] = b.[name]
			AND a.RowNumber = 1
			AND b.RowNumber = 2
ORDER BY
	GrowthKBPerHour DESC
 
DROP TABLE #Temp_SpSpaceUsed

Next, look at the culprit (assuming there's one table doing all the damage - I've found that's usually the case) and try to decipher the column names. Finally, run a few queries against the table - WITH(NOLOCK) of course - hey, this table is pretty busy. Now try and see where all the action is coming from. In this case, one of the columns was [IP] so I just did a COUNT(1) by IP for the last hour's records. Sure enough, 99% of the records were coming from a single IP, sent that IP off to the guys who know what IPs mean, and they knew exactly which change caused the growth.

Bottom line - nobody knows everything about everything, but we can learn pretty quick with the right information.

Script sp_configure Output To Migrate Settings Between SQL Servers

Tim — Sat, 27 Feb 2010 23:22:30 +0000

Sometimes when setting up a new server, all you want is for it to work and be configured exactly like the old server. In some cases, the new server is almost identical in every way already. If you've worked much with SQL Server, you know very well that there are a billion switches and knobs and every time you do this, you forget at least one. I had a similar situation recently and figured I'd be lazy about it and just turned the output of sp_configure in to a script I could execute on the new server:

scriptSpConfigure.sql - run this on the old/source server

EXEC SP_CONFIGURE 'show advanced options' , 1;
GO
RECONFIGURE;
GO
DECLARE @spConfigureOutput TABLE
	([name] VARCHAR(255)
	,[minimum] INT
	,[maximum] INT
	,[config_value] INT
	,[run_value] INT)
 
INSERT INTO @spConfigureOutput
EXEC SP_CONFIGURE
 
SELECT	'EXEC sp_configure ''' + name + ''', ' + CAST(config_value AS VARCHAR)
FROM	@spConfigureOutput
GO
EXEC SP_CONFIGURE 'show advanced options' , 0;
GO
RECONFIGURE;
GO

Copy the selected rows and paste in to another SSMS window, add the show advanced options switch above and below - should end up with something like this after you remove all the settings you don't care about or don't want on the new server:

EXEC SP_CONFIGURE 'show advanced options' , 1;
GO
RECONFIGURE;
GO
 
EXEC SP_CONFIGURE 'Ad Hoc Distributed Queries', 1
EXEC SP_CONFIGURE 'Agent XPs', 1
EXEC SP_CONFIGURE 'clr enabled', 1
EXEC SP_CONFIGURE 'cost threshold for parallelism', 5
EXEC SP_CONFIGURE 'cursor threshold', -1
EXEC SP_CONFIGURE 'Database Mail XPs', 1
EXEC SP_CONFIGURE 'default full-text language', 1033
EXEC SP_CONFIGURE 'default trace enabled', 1
EXEC SP_CONFIGURE 'fill factor (%)', 80
EXEC SP_CONFIGURE 'ft crawl bandwidth (max)', 100
EXEC SP_CONFIGURE 'ft notify bandwidth (max)', 100
EXEC SP_CONFIGURE 'max full-text crawl range', 4
EXEC SP_CONFIGURE 'max server memory (MB)', 10000
EXEC SP_CONFIGURE 'max text repl size (B)', 65536
EXEC SP_CONFIGURE 'min memory per query (KB)', 1024
EXEC SP_CONFIGURE 'min server memory (MB)', 10000
EXEC SP_CONFIGURE 'nested triggers', 1
EXEC SP_CONFIGURE 'network packet size (B)', 4096
EXEC SP_CONFIGURE 'PH timeout (s)', 60
EXEC SP_CONFIGURE 'query wait (s)', -1
EXEC SP_CONFIGURE 'remote access', 1
EXEC SP_CONFIGURE 'remote login timeout (s)', 20
EXEC SP_CONFIGURE 'remote query timeout (s)', 900
EXEC SP_CONFIGURE 'scan for startup procs', 1
EXEC SP_CONFIGURE 'server trigger recursion', 1
EXEC SP_CONFIGURE 'SMO and DMO XPs', 1
 
EXEC SP_CONFIGURE 'show advanced options' , 0;
RECONFIGURE;
GO;

Script RESTORE DATABASE … WITH MOVE Stub

Tim — Sat, 27 Feb 2010 18:29:26 +0000

I think most people setup the drives on their Development servers to match their Production servers - this is so that restores go smoothly and files go where they're supposed to, things match up nicely, etc. Unfortunately, when you create a full backup all the backup file contains is the logical name of all the files - no physical paths. This means that if the database doesn't exist on the destination server yet, the engine has absolutely no idea where to put the files or what to name them.

The following script can be executed in the context of the source (original) database that was backed up to script out the MOVE statements if your plan is to put the files in the same place on the destination server.

scriptRestoreWithMove.sql - run this in the context of the source database

SET NOCOUNT ON
 
DECLARE @MoveOption AS TABLE
	([Id] INT IDENTITY(1,1)
	,[MoveOption] VARCHAR(MAX))
 
PRINT 'RESTORE DATABASE ' + DB_NAME(DB_ID())
-- Edit this to match where you're restoring from
PRINT 'FROM DISK = ''' + DB_NAME(DB_ID()) + '.bak'''  
PRINT 'WITH'
 
INSERT INTO @MoveOption ([MoveOption])
SELECT
    'MOVE ''' + a.name + ''' TO ''' + a.FILENAME + '''' AS [MOVE OPTION]
FROM
    dbo.sysfiles a
 
DECLARE 
	 @LastId INT = 0
	,@MoveOptionText VARCHAR(MAX)
 
WHILE EXISTS (SELECT TOP 1 1 FROM @MoveOption WHERE [Id] > @LastId)
BEGIN
	SELECT 		TOP 1
			 @MoveOptionText = [MoveOption] 
			,@LastId = [Id]
	FROM		@MoveOption
	WHERE		[Id] > @LastId
	ORDER BY	[Id] ASC
 
	PRINT CASE WHEN @LastId = 1 THEN '' ELSE ',' END + @MoveOptionText
END
 
PRINT 'GO'
 
PRINT 'ALTER DATABASE ' + DB_NAME(DB_ID()) + ' SET MULTI_USER'
PRINT 'GO'

And the output looks like this:

RESTORE DATABASE AdventureWorksDW
FROM DISK = 'AdventureWorksDW.bak'
WITH
MOVE 'AdventureWorksDW_Data' TO 'D:\AdventureWorksDW\AdventureWorksDW_Data.mdf'
,MOVE 'AdventureWorksDW_Log' TO 'T:\AdventureWorksDW\AdventureWorksDW_Log.LDF'
GO
ALTER DATABASE AdventureWorksDW SET MULTI_USER
GO

If you end up wanting to do something similar with existing databases, make sure to add a command before the RESTORE to set SINGLE_USER mode.

Clone Analysis Services Partitions with PowerShell

Tim — Fri, 15 Jan 2010 20:45:20 +0000

Most of us with large Analysis Services cubes partition our cubes by month or year or some other time-based slice and we have all, at one point or another, developed some way to create partitions for new months on-demand. Often, the solution to this seems to be a C# console application or SSIS package using AMO to create a new partition based off an existing partition. The problem I see with this is that maintaining it requires opening up the project or package, making changes, re-compiling, deploying, testing, deploying to production, verifying, etc. It also requires that whoever is going to maintain it is comfortable with C#.

To simplify the maintenance and get rid of the "black box" factor that utility apps like this tend to have, I put together a PowerShell script to do the same thing and a stored procedure to call the script. Really, it doesn't matter what you use as you're most likely using an almost identical chunk of code to get your new partition created - my argument is that using PowerShell instead of C# or SSIS reduces the cost of maintenance, improves readability, and facilitates better understanding throughout your team.

The PowerShell Script: cloneSSASPartition.ps1

param(
   [string] $ServerName, 		<# The name of the Analysis Services Instance #>
   [string] $DatabaseName, 		<# The name of the Database our Cube is in #>
   [string] $CubeName,     		<# The name of the Cube that our Measure Group is in #>
   [string] $MeasureGroupName, 	<# The name of the Measure Group that our Partition is in #>
   [string] $SourcePartitionName, 	<# The name of the Partition we want to merge in to another partition #>
   [string] $NewPartitionName, 	<# The desired name of the new partition #>
   [string] $NewPartitionQuery 	<# The Query source for the new partition (will use the same datasource as the SourcePartition #>
)
 
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.AnalysisServices") >$NULL
 
$server = New-Object Microsoft.AnalysisServices.Server
$server.Connect($ServerName)
 
$database = $server.Databases.GetByName($DatabaseName)
$cube = $database.Cubes.GetByName($CubeName)
$measureGroup = $cube.MeasureGroups.FindByName($MeasureGroupName)
$sourcePartition = $measureGroup.Partitions.GetByName($SourcePartitionName)
$sourceQueryBinding = $sourcePartition.Source
 
$newPartition = $sourcePartition.Clone()
$newPartition.ID = $NewPartitionName
$newPartition.Name = $NewPartitionName
$measureGroup.Partitions.Add($newPartition) > $NULL
$newPartition.Source = New-Object Microsoft.AnalysisServices.QueryBinding($sourceQueryBinding.DataSourceID, $NewPartitionQuery)
$newPartition.Update()
 
$server.Disconnect()

The stored procedure to call the script: [Utility_CloneSSASPartition_S01] - assumes you placed cloneSSASPartition.ps1 at the root of the C: drive. If you put it elsewhere, update the proc to reflect the actual location.

SET ANSI_NULLS ON
GO
 
SET QUOTED_IDENTIFIER ON
GO
 
/*************************************************************************************
***
*** Procedure:  		[Utility_CloneSSASPartition]
*** Purpose:		Executes PowerShell script to clone a SSAS partition
***			
***			
*** Author:		tl
*** Date Created:		2010-01-15
*** 
*** Revision History
*** Date		Author			Description
*** 2010-01-15	tl			Created
*************************************************************************************/
 
CREATE PROCEDURE [dbo].[Utility_CloneSSASPartition_S01]
(
	@ServerName AS VARCHAR(255),
	@DatabaseName AS VARCHAR(255),
	@CubeName AS VARCHAR(255),
	@MeasureGroupName AS VARCHAR(255),
	@SourcePartitionName AS VARCHAR(255),
	@NewPartitionName AS VARCHAR(255),
	@NewPartitionQuery AS VARCHAR(255)
)
AS 
 
SET NOCOUNT ON
DECLARE @cmd VARCHAR(8000)
SET @cmd = 
	'powershell.exe -Command "& {'
	+ ' C:\cloneSSASPartition.ps1'
	+ ' -ServerName:\"' + @ServerName + '\"'
	+ ' -DatabaseName:\"' + @DatabaseName + '\"'
	+ ' -CubeName:\"' + @CubeName + '\"'
	+ ' -MeasureGroupName:\"' + @MeasureGroupName + '\"'
	+ ' -SourcePartitionName:\"' + @SourcePartitionName + '\"'
	+ ' -NewPartitionName:\"' + @NewPartitionName + '\"'
	+ ' -NewPartitionQuery:\"' + @NewPartitionQuery + '\"}"'
 
PRINT @cmd
EXEC xp_cmdshell @cmd

And, finally, we execute it

EXEC [dbo].[Utility_CloneSSASPartition_S01]
	 @ServerName = 'TESTSERVER'
	,@DatabaseName = 'TestDatabase'
	,@CubeName = 'TestCube'
	,@MeasureGroupName = 'Test Measure Group'
	,@SourcePartitionName = 'Test Partition - 200912'
	,@NewPartitionName = 'Test Partition - 201001'
	,@NewPartitionQuery = 'SELECT * FROM dbo.FactTableMonth(''1/1/2010'', ''2/1/2010'')'

Again, I'm not saying there's anything "wrong" with using C# or SSIS Packages to do this sort of thing, I'm just pointing out that C# isn't always the answer - especially when someone other than you needs to maintain this thing in the future. Keep it simple.

Trending SQL Server Agent Job Duration by Hour

Tim — Wed, 23 Dec 2009 03:07:38 +0000

Earlier today I noticed a SQL Server Agent job taking a little longer than usual (or what I thought was longer than usual). Let's face it, we're not staring at the Job Activity monitor all day, so unless you've written a report to monitor job run times - on occasion you ask yourself "is that a normal run time for this thing?" The job I was curious about happened to be a job that runs throughout the day and should only have real work to do once or twice an hour - and it should run roughly the same amount of time on any given business day for a given hour (i.e. at 1:00 PM on any given business day, this thing should do the same amount of work).

So I came up with the following query to PIVOT the run duration on the hour the job executed:

DECLARE @JobName VARCHAR(255) = 'Agent Job Name'
 
;WITH JobHistory AS
(
  SELECT
     a.run_date
    ,a.run_time / 10000 AS [HOUR]
    ,(a.run_duration / 10000 * 60 * 60	-- Hours
     + a.run_duration % 10000 / 100 * 60	-- Minutes
     + a.run_duration % 100			-- Seconds
     ) / 60.0 AS [DurationMinutes]
  FROM
    msdb.dbo.sysjobhistory a WITH(NOLOCK)
    INNER JOIN msdb.dbo.sysjobs b WITH(NOLOCK)
    ON 
      a.[job_id] = b.[job_id]
      AND b.[name] = @JobName
      AND step_id = 0
      AND run_status = 1
)
SELECT *
FROM
  JobHistory	
  PIVOT
  ( SUM(DurationMinutes)
    FOR [HOUR] 
    IN  ([00],[01],[02],[03],[04],[05]
        ,[06],[07],[08],[09],[10],[11]
        ,[12],[13],[14],[15],[16],[17]
        ,[18],[19],[20],[21],[22],[23])
  ) AS p

UPDATE: Added AND run_status = 1

Then made a table (not sure why, but I can't resist doing the heatmap thing):

click the image to enlarge

And, of course, the chart so you can point at a picture should you have to explain something to somebody in the future:

click the image to enlarge

Determining when RESTORE DATABASE command will complete (SQL Server 2008)

Tim — Fri, 25 Sep 2009 17:48:22 +0000

Ah, I see you just started restoring that 1TB monster and now everyone wants to know when it's going to be finished, where you're at in the process, etc. Fear not, Microsoft is very good at making up fictional numbers for us to use as rough estimates! I usually add 10-20% on top of these estimates just incase the database gremlins wander by to ruin your day again. Or incase you encounter "storage issues."

SELECT 
	 percent_complete AS [PercentComplete]
	,estimated_completion_time/1000.0/60.0 AS [RemainingMinutes]
	,total_elapsed_time/1000.0/60.0 AS [ElapsedMinutes]
	,(estimated_completion_time+total_elapsed_time)/1000.0/60.0 AS [TotalMinutes]
	,DATEADD(MILLISECOND, estimated_completion_time, GETDATE()) AS [EstimatedTimeOfCompletion]
	,st.TEXT AS [CommandSQL]
FROM sys.dm_exec_requests r
	cross apply sys.dm_exec_sql_text(r.sql_handle) st
WHERE command LIKE '%RESTORE DATABASE%'

Determining how long a database will be IN RECOVERY (SQL Server 2008)

Tim — Fri, 25 Sep 2009 14:16:16 +0000

So, your MSSQL service crashed in the middle of a big transaction? Or you bumped the service while it was rolling back some gigantic schema change (like say a column add on a 800 million row table)? Well, as you prepare your resume in preparation for the fallout from this debockle, you can use the following query to see how much time you have left. Or, I should say, how much time it thinks you have left... which seems to swing wildly up and down... microsoft math ftw.

 
DECLARE @DBName VARCHAR(64) = 'databasename'
 
DECLARE @ErrorLog AS TABLE([LogDate] CHAR(24), [ProcessInfo] VARCHAR(64), [TEXT] VARCHAR(MAX))
 
INSERT INTO @ErrorLog
EXEC sys.xp_readerrorlog 0, 1, 'Recovery of database', @DBName
 
SELECT TOP 5
	 [LogDate]
	,SUBSTRING([TEXT], CHARINDEX(') is ', [TEXT]) + 4,CHARINDEX(' complete (', [TEXT]) - CHARINDEX(') is ', [TEXT]) - 4) AS PercentComplete
	,CAST(SUBSTRING([TEXT], CHARINDEX('approximately', [TEXT]) + 13,CHARINDEX(' seconds remain', [TEXT]) - CHARINDEX('approximately', [TEXT]) - 13) AS FLOAT)/60.0 AS MinutesRemaining
	,CAST(SUBSTRING([TEXT], CHARINDEX('approximately', [TEXT]) + 13,CHARINDEX(' seconds remain', [TEXT]) - CHARINDEX('approximately', [TEXT]) - 13) AS FLOAT)/60.0/60.0 AS HoursRemaining
	,[TEXT]
 
FROM @ErrorLog ORDER BY [LogDate] DESC