These are the queries used for this test:

Query A

SELECT 
	{Actual} ON 0
FROM [GL] 
CELL PROPERTIES VALUE

Query B

SELECT 
	{Actual} ON 0,
	NON EMPTY Hierarchize({DrilldownLevel({[Account].[Accounts].[All]},,,INCLUDE_CALC_MEMBERS)}) ON 1
FROM [GL] 
CELL PROPERTIES VALUE

Query C

SELECT 
	{Actual} ON 0,
	NON EMPTY Hierarchize({DrilldownLevel({[Date].[Fiscal].[All]},,,INCLUDE_CALC_MEMBERS)}) ON 1
FROM [GL] 
CELL PROPERTIES VALUE
GO

Results (Cache was cleared with ClearCache on the entire Database before each Series)

Series 1	Series 2	Series 3

So basically the Grand Total of this GL cube is a random number based on whoever sneaks the first query in after the cache is cleared (processing, sync, etc).

And for all of you that think the MDX script breaks everything everywhere - I did comment out my entire MDX script before running these tests.

A quick note on what that "Update FTP Connection" task is doing - it's modifying the properties of the "FTP" connection manager to the appropriate Server/Username/Password for this particular client:

1
2
3
4
5
6
7
8
9
10

        public void Main()
        {
            ConnectionManager ftp = Dts.Connections["FTP"];
            ftp.Properties["ServerName"].SetValue(ftp, (string)Dts.Variables["FTPServer"].Value);
            ftp.Properties["ServerUserName"].SetValue(ftp, (string)Dts.Variables["FTPUser"].Value);
            ftp.Properties["ServerPassword"].SetValue(ftp, (string)Dts.Variables["User::FTPPassword"].Value);
 
            Dts.TaskResult = (int)ScriptResults.Success;
 
        }

The problem: Anyone who has tried to upload multiple files to multiple FTP sites in an SSIS package very quickly ran in to FTP errors (that's what happens when you let users tell you what their FTP url and authentication is). You can handle these errors and log them via the standard OnError handler dumping the ErrorCode, ErrorDescription, and usually SourceName out to a flat file or table.

Now, when you look at the ErrorDescription for an FTP Task - you'll be disappointed. It usually states "Unable to connect to FTP server..." or something to that effect. Looking at the execution results in the progress tab, you'll see that the FTP Connection Manager threw a more detailed description of the error:

Good luck getting at that detailed error description - My first attempt here was to acquire a connection and connect via the connection manager and trap the error that came back. While that worked, the error that was trapped was HRESULT 0xC001602A, which basically means that something bad happened.

Now we move on to phase two: Retry the FTP operation via .NET FtpWebRequest and trap that error. This can either flat out replace the FTP Task as a script task or, not sure why I did it this way - I suppose I like to try and use SSIS built in tasks as much as possible - you can retry the operation in the OnError handler assuming you have the requisite information (url, username, and password) available.

Create the OnError handler (either at the package level for everything or just on the executable you care about - I chose the latter):

Then add a data flow task - inside that add a Script Component (as a source) and an appropriate destination (Here it's going to an OleDb destination):

Configure the Script Component for ReadOnly access to the requisite variables:

And configure the Output buffer:

Now go edit the actual script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;
using System.Net;
using System.IO;
using System.Text;
 
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
    public override void CreateNewOutputRows()
    {
        string errorDescription = Variables.ErrorDescription;
 
        if (Variables.ErrorDescription.Contains("Unable to connect to FTP server"))
        {
            try
            {
                FtpWebRequest ftp = (FtpWebRequest)WebRequest.Create("ftp://" + Variables.FTPUrl);
                ftp.Method = WebRequestMethods.Ftp.UploadFile;
                ftp.Credentials = new System.Net.NetworkCredential(Variables.FTPUser, Variables.FTPPassword);
 
                StreamReader sourceStream = new StreamReader(Variables.FTPLocalPath);
                byte[] fileContents = Encoding.UTF8.GetBytes(sourceStream.ReadToEnd());
                sourceStream.Close();
                ftp.ContentLength = fileContents.Length;
 
                Stream requestStream = ftp.GetRequestStream();
                requestStream.Write(fileContents, 0, fileContents.Length);
                requestStream.Close();
 
                FtpWebResponse response = (FtpWebResponse)ftp.GetResponse();
 
                response.Close();
 
                errorDescription = "Upload retry attempt via .NET was successful.";
            }
            catch (Exception e)
            {
                errorDescription = e.Message;
            }
        }
 
 
        ErrorSourceBuffer.AddRow();
        ErrorSourceBuffer.ErrorCode = Variables.ErrorCode.ToString();
        ErrorSourceBuffer.ErrorDescription = errorDescription;
        ErrorSourceBuffer.ErrorSource = Variables.SourceName;
        ErrorSourceBuffer.FTPUrl = Variables.FTPUrl;
        ErrorSourceBuffer.FTPUser = Variables.FTPUser;
        ErrorSourceBuffer.FTPPassword = Variables.FTPPassword;
    }
}

And bob's your uncle. Now you have meaningful text in the ErrorDescription column of the output buffer instead of some vague "something's wrong" message.

Here's a flat file destination walkthrough for the OnError handler: http://vsteamsystemcentral.com/cs21/blogs/applied_business_intelligence/archive/2007/04/08/ssis-design-pattern-custom-error-handling.aspx

"Excel freezes (or hangs) when I try to change this filter"

Most of us have seen this at one point or another and shrugged it off as a busy time or processing is going on or there are cats clogging up the tubes, etc. Tonight, I finally decided to figure out what's causing it.

When you have an OLAP cube filter in your pivot table and expand the items list to select your filter items, Excel fires a bunch of Discover commands at the cube (these are pretty light in most cases). If you don't have anything selected yet (so you just have the default member selected) Excel isn't doing much work here and the list pops right up. Here's two basic flow examples for what discover commands are fired off when you DO have an item selected:

If you have one level 1 member selected

1. MDSCHEMA_CUBES (whatever cube you're connected to)
2. MDSCHEMA_CUBES (again, same thing)
3. MDSCHEMA_MEMBERS (LEVEL_UNIQUE_NAME of the filter), no TREE_OP
4. MDSCHEMA_MEMBERS (MEMBER_UNIQUE_NAME of the selected member) TREE_OP 4, only immediate parent
5. MDSCHEMA_MEMBERS (MEMBER_UNIQUE_NAME of the selected member's parent) TREE_OP 8, returns itself
6. MDSCHEMA_MEMBERS (MEMBER_UNIQUE_NAME of the selected member's parent) TREE_OP 1, returns all children

If you have one level 2 member selected in a hierarchy

1. MDSCHEMA_CUBES (whatever cube you're connected to)
2. MDSCHEMA_CUBES (again, same thing)
3. MDSCHEMA_MEMBERS (LEVEL_UNIQUE_NAME of the filter), no TREE_OP
4. MDSCHEMA_MEMBERS (MEMBER_UNIQUE_NAME of the selected member) TREE_OP 4, only immediate parent
5. MDSCHEMA_MEMBERS (MEMBER_UNIQUE_NAME of the selected member's parent's parent) TREE_OP 4, immediate parent
6. MDSCHEMA_MEMBERS (MEMBER_UNIQUE_NAME of the selected member's parent's parent) TREE_OP 8, returns itself
7. MDSCHEMA_MEMBERS (MEMBER_UNIQUE_NAME of the selected member's parent's parent) TREE_OP 1, all children
8. MDSCHEMA_MEMBERS (MEMBER_UNIQUE_NAME of the selected member's parent) TREE_OP 8, returns itself
9. MDSCHEMA_MEMBERS (MEMBER_UNIQUE_NAME of the selected member's parent) TREE_OP 1, returns all children

So, that's six requests for a level 1 member and 9 requests for a level 2 member - both of these take milliseconds to complete the whole flow, so no big deal. When does this become a big deal? When you do something like click the "Select multiple items" checkbox, expand a hierarchy and UNCHECK something... like if you opened the Adventure Works 2008 cube, added Delivery Date.Date as a filter, expanded the "All Periods" level and deselected the first day:

Now click OK and expand the items box - we get a familiar popup:

It comes back after a few seconds with the expected item list, but why did it take so long? Well, based on our flows above, we have a whole bunch of level 1 members selected - 1187 of them to be exact. That means there will be 1187 requests for their parent member (TREE_OP 4) which will all return the [All Periods] member. To illustrate this, I traced that exact operation:

The profiler captured 1192 discover requests - 1187 of them were getting the parents of selected members, two were MDSCHEMA_CUBES requests, one was getting [All Periods], one was getting all the children of [All Periods], and one was getting the Unique Name of the root level. Not cool. Imagine what happens when your talking about tens of thousands of members at a particular level! Granted, that seems unlikely but understand that the logic behind this is "I don't want to see this", "hide just this member", and so forth - in some cases people have no idea what actually happened was they unchecked one of 6,000 members.

So what to do about this? Educate. To fix the immediate problem of a seemingly broken Excel pivot table - just wait for the filter to load, it'll come back eventually. In the future when working with Excel filters, always favor selecting vs. deselecting and filtering at higher levels in the hierarchy whenever possible.

To simplify the maintenance and get rid of the "black box" factor that utility apps like this tend to have, I put together a PowerShell script to do the same thing and a stored procedure to call the script. Really, it doesn't matter what you use as you're most likely using an almost identical chunk of code to get your new partition created - my argument is that using PowerShell instead of C# or SSIS reduces the cost of maintenance, improves readability, and facilitates better understanding throughout your team.

The PowerShell Script: cloneSSASPartition.ps1

param(
   [string] $ServerName, 		<# The name of the Analysis Services Instance #>
   [string] $DatabaseName, 		<# The name of the Database our Cube is in #>
   [string] $CubeName,     		<# The name of the Cube that our Measure Group is in #>
   [string] $MeasureGroupName, 	<# The name of the Measure Group that our Partition is in #>
   [string] $SourcePartitionName, 	<# The name of the Partition we want to merge in to another partition #>
   [string] $NewPartitionName, 	<# The desired name of the new partition #>
   [string] $NewPartitionQuery 	<# The Query source for the new partition (will use the same datasource as the SourcePartition #>
)
 
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.AnalysisServices") >$NULL
 
$server = New-Object Microsoft.AnalysisServices.Server
$server.Connect($ServerName)
 
$database = $server.Databases.GetByName($DatabaseName)
$cube = $database.Cubes.GetByName($CubeName)
$measureGroup = $cube.MeasureGroups.FindByName($MeasureGroupName)
$sourcePartition = $measureGroup.Partitions.GetByName($SourcePartitionName)
$sourceQueryBinding = $sourcePartition.Source
 
$newPartition = $sourcePartition.Clone()
$newPartition.ID = $NewPartitionName
$newPartition.Name = $NewPartitionName
$measureGroup.Partitions.Add($newPartition) > $NULL
$newPartition.Source = New-Object Microsoft.AnalysisServices.QueryBinding($sourceQueryBinding.DataSourceID, $NewPartitionQuery)
$newPartition.Update()
 
$server.Disconnect()

The stored procedure to call the script: [Utility_CloneSSASPartition_S01] - assumes you placed cloneSSASPartition.ps1 at the root of the C: drive. If you put it elsewhere, update the proc to reflect the actual location.

SET ANSI_NULLS ON
GO
 
SET QUOTED_IDENTIFIER ON
GO
 
/*************************************************************************************
***
*** Procedure:  		[Utility_CloneSSASPartition]
*** Purpose:		Executes PowerShell script to clone a SSAS partition
***			
***			
*** Author:		tl
*** Date Created:		2010-01-15
*** 
*** Revision History
*** Date		Author			Description
*** 2010-01-15	tl			Created
*************************************************************************************/
 
CREATE PROCEDURE [dbo].[Utility_CloneSSASPartition_S01]
(
	@ServerName AS VARCHAR(255),
	@DatabaseName AS VARCHAR(255),
	@CubeName AS VARCHAR(255),
	@MeasureGroupName AS VARCHAR(255),
	@SourcePartitionName AS VARCHAR(255),
	@NewPartitionName AS VARCHAR(255),
	@NewPartitionQuery AS VARCHAR(255)
)
AS 
 
SET NOCOUNT ON
DECLARE @cmd VARCHAR(8000)
SET @cmd = 
	'powershell.exe -Command "& {'
	+ ' C:\cloneSSASPartition.ps1'
	+ ' -ServerName:\"' + @ServerName + '\"'
	+ ' -DatabaseName:\"' + @DatabaseName + '\"'
	+ ' -CubeName:\"' + @CubeName + '\"'
	+ ' -MeasureGroupName:\"' + @MeasureGroupName + '\"'
	+ ' -SourcePartitionName:\"' + @SourcePartitionName + '\"'
	+ ' -NewPartitionName:\"' + @NewPartitionName + '\"'
	+ ' -NewPartitionQuery:\"' + @NewPartitionQuery + '\"}"'
 
PRINT @cmd
EXEC xp_cmdshell @cmd

And, finally, we execute it

EXEC [dbo].[Utility_CloneSSASPartition_S01]
	 @ServerName = 'TESTSERVER'
	,@DatabaseName = 'TestDatabase'
	,@CubeName = 'TestCube'
	,@MeasureGroupName = 'Test Measure Group'
	,@SourcePartitionName = 'Test Partition - 200912'
	,@NewPartitionName = 'Test Partition - 201001'
	,@NewPartitionQuery = 'SELECT * FROM dbo.FactTableMonth(''1/1/2010'', ''2/1/2010'')'

Again, I'm not saying there's anything "wrong" with using C# or SSIS Packages to do this sort of thing, I'm just pointing out that C# isn't always the answer - especially when someone other than you needs to maintain this thing in the future. Keep it simple.

But I digress - asking cubes questions is still pretty fun and some of the random things it returns when you point it at your own cubes can be flat out hilarious.

Here's a few questions thrown at the Adventure Works cube in the Adventure Works DW 2008 Analysis Services database

what was sales for nichole nara
SELECT {[Measures].[Internet Sales Amount]} ON 0  FROM (SELECT {[Customer].[Customer].[Nichole Nara]} ON 0 FROM [Adventure Works])
$13,295.38
what were sales for nichole nara in cy 2004
SELECT {[Measures].[Internet Sales Amount]} ON 0  FROM (SELECT {[Date].[Calendar Year].[CY 2004]} ON 0 FROM (SELECT {[Customer].[Customer].[Nichole Nara]} ON 0 FROM [Adventure Works]))
$2,419.06
what were sales for nichole nara in cy 2004 in north america
SELECT {[Measures].[Internet Sales Amount]} ON 0  FROM (SELECT {[Sales Territory].[Sales Territory Group].[North America]} ON 0 FROM (SELECT {[Date].[Calendar Year].[CY 2004]} ON 0 FROM (SELECT {[Customer].[Customer].[Nichole Nara]} ON 0 FROM [Adventure Works])))
No Results Found
what were sales for nichole nara in cy 2004 for bikes
SELECT {[Measures].[Internet Sales Amount]} ON 0  FROM (SELECT {[Product].[Category].[Bikes]} ON 0 FROM (SELECT {[Date].[Calendar Year].[CY 2004]} ON 0 FROM (SELECT {[Customer].[Customer].[Nichole Nara]} ON 0 FROM [Adventure Works])))
$2,384.07
what were sales in north america in fy 2003 for bikes
SELECT {[Measures].[Internet Sales Amount]} ON 0  FROM (SELECT {[Product].[Category].[Bikes]} ON 0 FROM (SELECT {[Date].[Fiscal Year].[FY 2003]} ON 0 FROM (SELECT {[Sales Territory].[Sales Territory Group].[North America]} ON 0 FROM [Adventure Works])))
$1,739,306.95
sales for cy 2004 in cy q1, cy q2, and cy q4 in north america and canada
SELECT {[Measures].[Internet Sales Amount]} ON 0  FROM (SELECT {[Sales Territory].[Sales Territory Group].[North America],[Customer].[Country].[Canada]} ON 0 FROM (SELECT {[Date].[Calendar Quarter of Year].[CY Q1],[Date].[Calendar Quarter of Year].[CY Q2],[Date].[Calendar Quarter of Year].[CY Q4]} ON 0 FROM (SELECT {[Date].[Calendar Year].[CY 2004]} ON 0 FROM [Adventure Works])))
I don't know - Query Failure: Members belong to different hierarchies in the  function.
sales for cy 2004 in cy q1, cy q2, and cy q4 in north america
SELECT {[Measures].[Internet Sales Amount]} ON 0  FROM (SELECT {[Sales Territory].[Sales Territory Group].[North America]} ON 0 FROM (SELECT {[Date].[Calendar Quarter of Year].[CY Q1],[Date].[Calendar Quarter of Year].[CY Q2],[Date].[Calendar Quarter of Year].[CY Q4]} ON 0 FROM (SELECT {[Date].[Calendar Year].[CY 2004]} ON 0 FROM [Adventure Works])))
$3,967,371.16
sales between cy 2003 and cy 2004 for large resellers
SELECT {[Measures].[Internet Sales Amount]} ON 0  FROM (SELECT [Large Resellers] ON 0 FROM (SELECT {[Date].[Calendar Year].[CY 2003]:[Date].[Calendar Year].[CY2004]} ON 0 FROM [Adventure Works]))
$19,561,960.04
sales for top 50 customers in fy 2004 in fy q1 and fy q3
SELECT {[Measures].[Internet Sales Amount]} ON 0  FROM (SELECT {[Date].[Fiscal Quarter of Year].[FY Q1],[Date].[Fiscal Quarter of Year].[FY Q3]} ON 0 FROM (SELECT {[Date].[Fiscal Year].[FY 2004]} ON 0 FROM (SELECT [Top 50 Customers] ON 0 FROM [Adventure Works])))
$177,263.41

The above questions returned pretty much exactly what I was looking for except for the one that threw us an error about members belonging to different hierarchies. That one was because "Canada" was in [Customer].[Country] and "north america" was in [Sales Territory].[Sales Territory Group]. Yeah, it was a dumb question because canada is in north america, but it illustrates that when you don't qualify things, you can get some pretty confusing results.

How's it work?

1. Enumerate all the Attribute Values for each Attribute Hierarchy and store them as "keywords"
2. Get some input
3. Look for the aforementioned "keywords" in the input string
4. Construct an MDX query based on what "keywords" were found in the string
5. Make sure there aren't any extra (non-noise) words in the string
6. Run Query
7. Return Results

How the queries are constructed

• Attribute Values just create a subcube - it creates a new nested subcube for each list, range, or named set it finds
• It only supports one measure as written and it just selects that from whatever quagmire of subcubes it's created

There are tons of issues with this methodology when we have role playing dimensions because the attribute values will be the same for these dimensions - so only the first one enumerated will get in (there's a way to ignore certain dimensions in the app.config - ignore all but one role playing dimension). It throws away context and calls it "noise" - now that's dangerous; if you throw away the context and just look for keywords, it's pretty easy to answer a question that wasn't really asked (oops).

As far as configuring this thing goes - take a look at the app.config and throw in appropriate values for your setup. You'll have to tweak the *Pattern values to fit things you want to match or exclude in your particular cube. It's configured for the Adventure Works cube right now, but I didn't work with it much - I just flat out excluded a few dims and hierarchies that would obviously confuse it.

Also note that, as written, it needs an Analysis Services 2008 cube.

Let me know if you want to give it a try and I'll be more than happy to work with you on configuring/modifying it for your setup. Also post comments on interesting observations you have when asking cubes unqualified questions

Download the VS 2008 Project: SmartCube.zip

Also note, the project uses an interface called IODoodad for getting questions and responding - this has two public methods, GetInput() and Respond(). The original theory here is that we could make an old-school IRC bot type thing. We did manage to hook it to Yammer for its IO, respond to private messages and public messages prefixed with @cube, etc. I didn't really explain the config for YammerIO (there's a custom config section for that that needs a bunch of OAuth stuff). Again, if you're interested in getting that working, let me know and I'll post instructions. The project above is set to use ConsoleIO.

I don't plan on doing any more development here, it was just a quick PoC to see what it looked like and get some initial feedback. If anyone's interested in moving forward with it and making it smarter (the project arguably should have been titled DumbCube at this point), let me know so I can post a link to your work and fork away

• Red: Cube Processing on a 2 Minute schedule
• Green: Cube Processing on a 1 hour schedule
• Blue: Cube Processing on a 1 hour schedule
• Black: Query Activity

Most of the activity here is from SSRS subscriptions firing around 8AM


This is the report filtered to only show events that had a duration greater than 10,000 MS (10 seconds) - Red cube is a trouble maker

And this is general SSAS activity over a few hours with the nearly instant events filtered out

The following query assumes that the trace data ended up in a table:

• @StartMillisecond: INT; Minimum relative start time (millisecond value, kinda have to guess and check)
• @EndMillisecond: INT; Maximum relative start time (millisecond value, kinda have to guess and check)
• @MinDuration: INT; Minimum duration in milliseconds

;WITH [ProcessEvent] AS
(
SELECT
	'Process' AS [Event]
	,cast([TextData] AS XML).value
		('declare namespace B="http://schemas.microsoft.com/analysisservices/2003/engine";           
		(//B:DatabaseID)[1]'
		,'VARCHAR(1024)') AS [DatabaseID]
	,cast([TextData] AS XML).value
		('declare namespace B="http://schemas.microsoft.com/analysisservices/2003/engine";           
		(//B:DimensionID)[1]'
		,'VARCHAR(1024)') AS [DimensionID]
	,cast([TextData] AS XML).value
		('declare namespace B="http://schemas.microsoft.com/analysisservices/2003/engine";           
		(//B:MeasureGroupID)[1]'
		,'VARCHAR(1024)') AS [MeasureGroupId]
	,[StartTime]
	,[Duration]
FROM dbo.zTrace_MSAS_20091023_2 T
WHERE 
	T.[EventClass] = 16 
	AND T.[EventSubclass] IN (3, 12)
	AND [TextData] LIKE '%Process%'
),
[QueryEvent] AS
(
SELECT
	'Query' AS [Event]
	,[DatabaseName] AS [DatabaseID]
	,NULL AS [DimensionID]
	,NULL AS [MeasureGroupId]
	,[StartTime]
	,[Duration]
FROM dbo.zTrace_MSAS_20091023_2 T
WHERE T.[EventClass] = 10
),
[Event] AS
(
	SELECT * FROM [ProcessEvent]
	UNION
	SELECT * FROM [QueryEvent]
)
SELECT
	 ROW_NUMBER() OVER (ORDER BY [StartTime] ASC) AS [EventId]
	,DATEADD(HOUR, -5, [StartTime]) AS [StartTime]
	,DATEDIFF(MILLISECOND, (SELECT MIN([StartTime]) FROM Event), [StartTime]) [RelativeStartTime]
	,[Duration]
	,[Event]
	,[DatabaseID]
	,CASE
		WHEN [DimensionID] IS NOT NULL THEN 'Dimension'
		WHEN [MeasureGroupID] IS NOT NULL THEN 'Measure Group'
		ELSE 'None'
	 END AS [ObjectType]	,CASE
		WHEN [DimensionID] IS NOT NULL THEN [DimensionID]
		WHEN [MeasureGroupID] IS NOT NULL THEN [MeasureGroupID]
		ELSE 'None'
	 END AS [ObjectId]	
FROM [Event]
WHERE
	DATEDIFF(MILLISECOND, (SELECT MIN([StartTime]) FROM Event), [StartTime]) >= @StartMillisecond
	AND DATEDIFF(MILLISECOND, (SELECT MIN([StartTime]) FROM Event), [StartTime]) < @EndMillisecond
	AND Duration >= @MinDuration

A few notes on the report:

• [EventId] is the Series Group
• This is a Range chart type, the Bottom value was [RelativeStartTime] and the Top value was [RelativeStartTime] + [Duration]
• Coloring the series and markers uses the following expression

  =Iif(Fields!Event.Value = "Process"
  	,Iif(Fields!DatabaseID.Value = "SSASDB"
  		,"Red"
  		,Iif(Fields!DatabaseID.Value = "OtherSSASDB"
  			,"Green"
  			,Iif(Fields!DatabaseID.Value = "YetAnotherSSASDB"
  				,"Blue"
  				,"Pink"
  			)
  		)
  	)
  	,"Black"
  )

And that's about it - if you have any more questions about it just let me know. Just seemed like an interesting way to look at trace data It also wouldn't be that difficult to generate events for other things, like your ETL processing and such so you could see that on top of everything else.

In the ReportServerService log, you'll see something like this:
Microsoft.AnalysisServices.AdomdClient.AdomdErrorResponseException: Query (..., ...) Set used with the complement operator must have all members from the same level.

Original filter
Dimension: Time
Hierarchy: Calendar Date
Operator: Not In
Filter Expression: [Today]

New filter
Dimension: Time
Hierarchy: Calendar Date
Operator: MDX
Filter Expression: Except([Time].[Calendar Date].[Calendar Date].MEMBERS, [Today])

Wait a tick - why would we give the rows to SSIS if all it is going to do insert one set and update the other? Let's just do it all in T-SQL:

The following tables and dim are fictional - I just need to make up a star schema DIM to illustrate the approach

CREATE TABLE #Temp_SCDInserts ([PartId] INT)
 
-- INSERTS for new Parts
INSERT INTO #Temp_SCDInserts
SELECT   
	[PartId]
FROM 
	Part a WITH(NOLOCK)
	INNER JOIN PartType b WITH(NOLOCK) 
		ON a.PartTypeId = b.PartTypeId
	INNER JOIN PartSupplier c WITH(NOLOCK) 
		ON a.PartSupplierId = c.PartSupplierId
	INNER JOIN PartSupplierCategory d WITH(NOLOCK) 
		ON c.PartSupplierCategoryId = d.PartSupplierCategoryId
EXCEPT
SELECT
	[PartId]	
FROM
	DW.dbo.Dim_Part WITH(NOLOCK)
 
-- UPDATES
SELECT 
	 dim.[PartKeyId]
	,a.[PartName]
	,b.[PartTypeId]
	,b.[PartTypeName]
	,c.[PartSupplierId]
	,c.[PartSupplierName]
	,d.[PartSupplierCategoryId]
	,d.[PartSupplierCategoryName]
INTO #Temp_SCDUpdates
FROM DW.dbo.Dim_Part dim WITH(NOLOCK)
	INNER JOIN Part a WITH(NOLOCK)
		ON dim.PartId = a.PartId -- Business Key
	INNER JOIN PartType b WITH(NOLOCK) 
		ON a.PartTypeId = b.PartTypeId
	INNER JOIN PartSupplier c WITH(NOLOCK) 
		ON a.PartSupplierId = c.PartSupplierId
	INNER JOIN PartSupplierCategory d WITH(NOLOCK) 
		ON c.PartSupplierCategoryId = d.PartSupplierCategoryId
WHERE
	dim.PartName <> a.PartName
	OR dim.PartTypeId <> b.PartTypeId
	OR dim.PartTypeName <> b.PartTypeName
	OR dim.PartSupplierId <> c.PartSupplierId
	OR dim.PartSupplierName <> c.PartSupplierName
	OR dim.PartSupplierCategoryId <> d.PartSupplierCategoryId
	OR dim.PartSupplierCategoryName	<> d.PartSupplierCategoryName
 
-- INSERT new records
INSERT INTO [DW].[dbo].[Dim_Part]
	([PartId]
	,[PartName]
	,[PartTypeId]
	,[PartTypeName]
	,[PartSupplierId]
	,[PartSupplierName]
	,[PartSupplierCategoryId]
	,[PartSupplierCategoryName])
SELECT 
	 a.[PartId]
	,a.[PartName]
	,b.[PartTypeId]
	,b.[PartTypeName]
	,c.[PartSupplierId]
	,c.[PartSupplierName]
	,d.[PartSupplierCategoryId]
	,d.[PartSupplierCategoryName]
FROM #Temp_SCDInserts i
	INNER JOIN Part a WITH(NOLOCK)
		ON i.PartId = a.PartId -- Business Key
	INNER JOIN PartType b WITH(NOLOCK) 
		ON a.PartTypeId = b.PartTypeId
	INNER JOIN PartSupplier c WITH(NOLOCK) 
		ON a.PartSupplierId = c.PartSupplierId
	INNER JOIN PartSupplierCategory d WITH(NOLOCK) 
		ON c.PartSupplierCategoryId = d.PartSupplierCategoryId
 
-- UPDATE existing records
UPDATE [DW].[dbo].[Dim_Part]
SET
	 [PartName] = b.[PartName]
	,[PartTypeId] = b.[PartTypeId]
	,[PartTypeName] = b.[PartTypeName]
	,[PartSupplierId] = b.[PartSupplierId]
	,[PartSupplierName] = b.[PartSupplierName]
	,[PartSupplierCategoryId] = b.[PartSupplierCategoryId]
	,[PartSupplierCategoryName] = b.[PartSupplierCategoryName]
FROM 
	[DW].[dbo].[Dim_Part] AS a
	INNER JOIN #Temp_SCDUpdates AS b 
		ON a.PartKeyId = b.PartKeyId  -- Join on Business Keys or Surrogate Key
 
 
DROP TABLE #Temp_SCDInserts
DROP TABLE #Temp_SCDUpdates

The results? Our 15 minute SSIS package has been replaced with a few lines of T-SQL and it now runs in less than 90 seconds. Nice.

This is really just an example of the implementation that worked for my purposes - I'm sure some of you would prefer to log elsewhere. This should be enough to get you going. Please drop a line in the Comments if you post alternative approaches.

Usage

SSISForcedLogging.Console.exe "Z:\SSIS Packages"
or
SSISForcedLogging.Console.Exe "Z:\SSIS Packages\Package1.dtsx"

For the following code, be sure to add References to:

• Microsoft.SqlServer.ManagedDTS
• System.configuration

Program.cs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186

using System;
using System.Configuration;
using Microsoft.SqlServer.Dts.Runtime;
using System.Reflection;
using System.Collections;
using System.IO;
using System.Collections.Generic;
 
namespace SSISForcedLogging.Console
{
    class ColumnFilter
    {
        public bool Computer;
        public bool DataBytes;
        public bool ExecutionID;
        public bool MessageText;
        public bool Operator;
        public bool SourceID;
        public bool SourceName;
 
        public DTSEventColumnFilter GetDTSEventColumnFilter()
        {
            DTSEventColumnFilter dtsEventColumnFilter = 
                new DTSEventColumnFilter();
            dtsEventColumnFilter.DataBytes = this.DataBytes;
            dtsEventColumnFilter.Computer = this.Computer;
            dtsEventColumnFilter.ExecutionID = this.ExecutionID;
            dtsEventColumnFilter.MessageText = this.MessageText;
            dtsEventColumnFilter.Operator = this.Operator;
            dtsEventColumnFilter.SourceID = this.SourceID;
            dtsEventColumnFilter.SourceName = this.SourceName;
            return dtsEventColumnFilter;
        }
    }
 
    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length != 1)
            {
                System.Console.WriteLine(
                    "\nPlease specify a package path or directory\n");
                Environment.Exit(-1);
            }
 
            string arg = args[0];
            List<string> packages = new List<string>();
 
            if (File.Exists(arg))
                packages.Add(arg);
            else if (Directory.Exists(arg))
                packages = GetPackagesFromDirectory(arg);
            else
            {
                System.Console.WriteLine(
                    "\nInvalid package path or directory specified\n");
                Environment.Exit(-1);
            }
 
            foreach (string pkgPath in packages)
            {
                System.Console.WriteLine(
                    string.Format("Updating: {0}", pkgPath));
                Application dts = new Application();
                Package pkg = dts.LoadPackage(pkgPath, null);
 
                RemoveOldLogsFromPackage(ref pkg);
                AddLogToPackage(ref pkg);
 
                System.Console.WriteLine(
                    string.Format("Saving: {0}", pkgPath));
                dts.SaveToXml(pkgPath, pkg, null);
            }
        }
 
        public static void AddLogToPackage(ref Package Pkg)
        {
            // Add new provider
            Pkg.LoggingMode = DTSLoggingMode.Enabled;
 
            LogProvider log = Pkg.LogProviders.Add(
                ConfigurationManager.AppSettings["logType"]);
            log.Name = 
                ConfigurationManager.AppSettings["logName"];
            log.Description = 
                ConfigurationManager.AppSettings["logDesc"];
            Pkg.LoggingOptions.SelectedLogProviders.Add(log);
 
            LoggingOptions logOptions = Pkg.LoggingOptions;
            logOptions.EventFilterKind = DTSEventFilterKind.Inclusion;
 
            string[] events = 
                ConfigurationManager
                    .AppSettings["eventsToLog"]
                        .Split(',');
            logOptions.EventFilter = events;
 
            ArrayList columnList = new ArrayList();
            columnList.AddRange(
                ConfigurationManager
                    .AppSettings["columnsToLog"]
                        .Split(','));
 
            ColumnFilter eventColumnFilter = new ColumnFilter();
 
            foreach (FieldInfo field in 
                     eventColumnFilter.GetType().GetFields())
                if (columnList.Contains(field.Name))
                    field.SetValue(eventColumnFilter, true);
 
            logOptions.SetColumnFilter(
                "OnError", 
                eventColumnFilter.GetDTSEventColumnFilter());
 
        }
 
        public static void RemoveOldLogsFromPackage(ref Package Pkg)
        {
            // Remove existing log provider selections
            int count = Pkg.LoggingOptions.SelectedLogProviders.Count;
            for (int i = 0; i < count; i++)
                Pkg.LoggingOptions.SelectedLogProviders.Remove(i);
 
            // Remove existing log providers
            count = Pkg.LogProviders.Count;
            for (int i = 0; i < count; i++)
                Pkg.LogProviders.Remove(i);
 
            RemoveOldLogsFromExecutable(Pkg.Executables);
 
        }
 
        public static void RemoveOldLogsFromExecutable(Executables Execs)
        {
            // reflection ftw
            int count;
            foreach(Executable exec in Execs) 
            {
                foreach (PropertyInfo prop in 
                         exec.GetType().GetProperties())
                {
                    switch(prop.Name) {
                        case "LoggingMode":
                            prop.SetValue(
                                exec, 
                                DTSLoggingMode.UseParentSetting, 
                                null);
                            break;
                        case "LoggingOptions":
                            LoggingOptions opts = 
                                prop.GetValue(
                                    exec, 
                                    null) as LoggingOptions;
 
                            count = opts.SelectedLogProviders.Count;
                            for (int i = 0; i < count; i++)
                                opts.SelectedLogProviders.Remove(i);
                            break;
                        case "Executables":
                            Executables execs = 
                                prop.GetValue(
                                exec, 
                                null) as Executables;
                            RemoveOldLogsFromExecutable(execs);
                            break;
                    }
                }
            }
        }
 
        public static List<string> GetPackagesFromDirectory(string Path)
        {
            List<string> files = new List<string>();
 
            foreach (string d in Directory.GetDirectories(Path))
                foreach (string f in GetPackagesFromDirectory(d))
                    files.Add(f);
 
            foreach (string f in Directory.GetFiles(Path, "*.dtsx"))
                files.Add(f);
 
            return files;
        }
    }
}

App.config

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
	<appSettings>
		<add key="logType" value="DTS.LogProviderEventLog.1"/>
		<add key="logName" value="Windows Event Log Logging Provider (Forced)"/>
		<add key="logDesc" value="Logs Event info to Windows Event Log (Forced)"/>
		<add key="eventsToLog" value="OnError"/>
		<add key="columnsToLog" value="Computer,Operator,SourceName,SourceID,ExecutionID,MessageText"/>
	   </appSettings>
</configuration>