Estimating the Size of a Table in SQL Server 2008

Tim — Fri, 16 Oct 2009 02:35:00 +0000

I have read this (http://msdn.microsoft.com/en-us/library/ms178085.aspx) article at least 6, maybe 7 times in the past - and every time I say to myself "this is ridiculous - someone has to have written a script to do this by now" and every time, I google for hours and fail to find anything. So I finally gave up and wrote something to do it. Note, I've only verified it on 100% fixed width tables. I compared its output to a 600+ million row table and it came out somewhere around 3% higher - fine with me as I'd rather over-estimate space requirements than under-estimate.

First, we have to create a table type to hold our table column definitions. the datatype is required, length is required for anything that allows length (MAX is not handled for any types), and the worst case scenario is always used so if you know you won't be filling those VARCHAR(255)s, feel free to use a smaller, more average, number. IsKey is a BIT value that tells the proc to use those columns as part of the index key. The rest of it is all calculated.

CREATE TYPE TableColumnDefinition AS TABLE
	([DataType] VARCHAR(16) NOT NULL
	,[Length] INT
	,[IsKey] BIT
	,[DataLength] AS
		CASE 
			WHEN [DataType] = 'BIT' THEN 0.125
			WHEN [DataType] = 'TINYINT' THEN 1
			WHEN [DataType] = 'SMALLINT' THEN 2
			WHEN [DataType] IN ('INT', 'SMALLMONEY', 'REAL', 'SMALLDATETIME') THEN 4
			WHEN [DataType] IN ('BIGINT', 'DATETIME', 'MONEY') THEN 8
			WHEN [DataType] = 'UNIQUEIDENTIFIER' THEN 16
			WHEN [DataType] = 'NVARCHAR' THEN [Length] * 2 + 2
			WHEN [DataType] = 'VARCHAR' THEN [Length] + 2	-- DOESN'T SUPPORT MAX
			WHEN [Datatype] = 'DATE' THEN 3 
			WHEN [Datatype] = 'TIME' THEN 5 -- Worst case
			WHEN [Datatype] = 'DATETIME2' THEN 8 -- Worst case
			WHEN [Datatype] = 'DATETIMEOFFSET' THEN 10 -- Worst case
			WHEN [DataType] = 'CHAR' THEN [Length]
			WHEN [DataType] = 'NCHAR' THEN [Length] * 2 + 2
			WHEN [DataType] = 'BINARY' THEN [Length]
			WHEN [DataType] = 'VARBINARY' THEN [Length] + 2
			WHEN [Datatype] = 'TIMESTAMP' THEN 8
			WHEN [DataType] = 'XML' THEN [Length]	-- Estimate an average size in bytes, 2GB max
			WHEN [DataType] IN ('DECIMAL', 'NUMERIC') THEN 
				CASE 
					WHEN [Length] BETWEEN 1 AND 9	THEN 5
					WHEN [Length] BETWEEN 10 AND 19 THEN 9
					WHEN [Length] BETWEEN 20 AND 28 THEN 13
					WHEN [Length] BETWEEN 29 AND 38 THEN 17
				END
			WHEN [DataType] = 'FLOAT' THEN
				CASE
					WHEN [Length] BETWEEN 1 AND 24 THEN 4
					WHEN [Length] BETWEEN 25 AND 53 THEN 8
					WHEN [Length] IS NULL THEN 8	-- Default n is 53
				END
			ELSE 0
		END
	,[IsVariableLength] AS
		CASE 
			WHEN [DataType] IN ('VARCHAR', 'NVARCHAR', 'VARBINARY') THEN 1
			ELSE 0
		END)

Now for the proc that attempts to replicate the methodology outlined on the aforementioned MSDN article. The results appeared to be accurate or over the actual value, which was exactly what I needed, so I went with it and moved on. I'm completely open to corrections, suggestions, improvements, etc. Just make a comment and I'll integrate them (assuming they work .

/*************************************************************************************
***
*** Procedure:  		[Utility_EstimateClusteredIndexSize]
*** Purpose:		    Estimates the amount of space needed for a given clustered index
*** Methodology:		http://msdn.microsoft.com/en-us/library/ms178085.aspx
***			
***			
*** Author:		      Tim Laqua
*** Date Created:	  2009-10-15 
*** 
*** Revision History
*** Date		Author			Description
*** 2009-10-15	tl				Created
*************************************************************************************/
ALTER PROCEDURE [dbo].[Utility_EstimateClusteredIndexSize]
(
	 @Num_Rows FLOAT
	,@ClusteredIndexIsUnique BIT
	,@Fill_Factor FLOAT
	,@TableColumn TableColumnDefinition READONLY
)
AS 
 
SET NOCOUNT ON
 
DECLARE
	 @Num_Cols FLOAT = 0					-- total number of columns (fixed-length and variable-length)
	,@Fixed_Data_Size FLOAT = 0			-- total byte size of all fixed-length columns
	,@Num_Variable_Cols FLOAT	= 0			-- number of variable-length columns
	,@Max_Var_Size FLOAT	= 0				-- maximum byte size of all variable-length columns
	,@Variable_Data_Size FLOAT = 0
 
DECLARE @VAR TABLE
	([Name] VARCHAR(64)
	,[VALUE] FLOAT
	,[DisplayValue] AS
		CASE
			WHEN LOWER([Name]) LIKE '%size%' OR LOWER([Name]) LIKE '%space%' THEN
				CASE 
					WHEN [VALUE] < 1024 THEN CAST(CAST(ROUND([VALUE], 3) AS NUMERIC(16,0)) AS VARCHAR(16)) + ' B'
					WHEN [VALUE] < 1024*1024 THEN CAST(CAST(ROUND([VALUE]/1024, 3) AS NUMERIC(16,3)) AS VARCHAR(16)) + ' KB'
					WHEN [VALUE] < 1024*1024*1024 THEN CAST(CAST(ROUND([VALUE]/1024/1024, 3) AS NUMERIC(16,3)) AS VARCHAR(16)) + ' MB'
					ELSE CAST(CAST(ROUND([VALUE]/1024/1024/1024, 3) AS NUMERIC(16,3)) AS VARCHAR(16)) + ' GB'
				END
			ELSE CAST(FLOOR([VALUE]) AS VARCHAR(16))
		END)
 
 
-- Leaf level data
SELECT 
	 @Num_Variable_Cols = COUNT(1)
	,@Max_Var_Size = ISNULL(SUM([DataLength]), 0)
FROM @TableColumn WHERE [IsVariableLength] = 1
 
SELECT 
	 @Num_Cols = COUNT(1) + @Num_Variable_Cols
	,@Fixed_Data_Size = CEILING(ISNULL(SUM([DataLength]), 0))
FROM @TableColumn WHERE [IsVariableLength] = 0
 
 
INSERT INTO @VAR VALUES ('@Fixed_Data_Size', @Fixed_Data_Size)
 
IF @ClusteredIndexIsUnique <> 0
BEGIN
	SET @Num_Cols += 1
	IF @Num_Variable_Cols > 0
		SET @Num_Variable_Cols += 1
	IF @Max_Var_Size > 0
		SET @Max_Var_Size += 4
END
 
DECLARE @Null_Bitmap INT = CEILING(2 + ((@Num_Cols + 7) / 8))
INSERT INTO @VAR VALUES ('@Null_Bitmap', @Null_Bitmap)
 
-- if there are variable columns
IF @Num_Variable_Cols > 0
	SET @Variable_Data_Size = 2 + (@Num_Variable_Cols * 2) + @Max_Var_Size 
INSERT INTO @VAR VALUES ('@Num_Cols', @Num_Cols)
INSERT INTO @VAR VALUES ('@Num_Variable_Cols', @Num_Variable_Cols)
INSERT INTO @VAR VALUES ('@Max_Var_Size', @Max_Var_Size)
 
DECLARE @Row_Size FLOAT = CEILING(@Fixed_Data_Size + @Variable_Data_Size + @Null_Bitmap + 4 )
INSERT INTO @VAR VALUES ('@Row_Size', @Row_Size)
 
DECLARE @Rows_Per_Page FLOAT = FLOOR(8096 / (@Row_Size + 2))
INSERT INTO @VAR VALUES ('@Rows_Per_Page', @Rows_Per_Page)
 
DECLARE @Free_Rows_Per_Page FLOAT = FLOOR(8096 * ((100 - @Fill_Factor) / 100) / (@Row_Size + 2) )
INSERT INTO @VAR VALUES ('@Free_Rows_Per_Page', @Free_Rows_Per_Page)
 
DECLARE @Num_Leaf_Pages FLOAT = CEILING(@Num_Rows / (@Rows_Per_Page - @Free_Rows_Per_Page) )
INSERT INTO @VAR VALUES ('@Num_Leaf_Pages', @Num_Leaf_Pages)
 
DECLARE @Leaf_space_used FLOAT = 8192 * @Num_Leaf_Pages
INSERT INTO @VAR VALUES ('@Leaf_space_used', @Leaf_space_used)
 
-- Index Information
DECLARE
	 @Num_Key_Cols FLOAT = 0			-- total number of key columns (fixed-length and variable-length)
	,@Fixed_Key_Size FLOAT = 0			-- total byte size of all fixed-length key columns
	,@Num_Variable_Key_Cols FLOAT = 0	-- number of variable-length key columns
	,@Max_Var_Key_Size FLOAT = 0		-- maximum byte size of all variable-length key columns
 
 
SELECT 
	 @Num_Variable_Key_Cols = COUNT(1)
	,@Max_Var_Key_Size = ISNULL(SUM([DataLength]), 0)
FROM @TableColumn 
WHERE 
	[IsVariableLength] = 1 
	AND [IsKey] = 1
 
SELECT 
	 @Num_Key_Cols = COUNT(1) + @Num_Variable_Key_Cols
	,@Fixed_Key_Size = ISNULL(SUM([DataLength]), 0)
FROM @TableColumn 
WHERE 
	[IsVariableLength] = 0
	AND [IsKey] = 1
 
IF @ClusteredIndexIsUnique <> 0
BEGIN
	SET @Num_Key_Cols += 1
	IF @Num_Variable_Key_Cols > 0
		SET @Num_Variable_Key_Cols += 1
	IF @Max_Var_Key_Size > 0
		SET @Max_Var_Key_Size += 4
END
INSERT INTO @VAR VALUES ('@Num_Variable_Key_Cols', @Num_Variable_Key_Cols)
INSERT INTO @VAR VALUES ('@Max_Var_Key_Size', @Max_Var_Key_Size)
INSERT INTO @VAR VALUES ('@Num_Key_Cols', @Num_Key_Cols)
 
-- Who would seriously have a nullable key column?  ridiculous.
DECLARE @Index_Null_Bitmap FLOAT = 2 + ((@Num_Key_Cols + 7) / 8) 
INSERT INTO @VAR VALUES ('@Index_Null_Bitmap', @Index_Null_Bitmap)
 
DECLARE @Variable_Key_Size FLOAT = 0
 
IF @Max_Var_Key_Size > 0
	SET @Variable_Key_Size = 2 + (@Num_Variable_Key_Cols * 2) + @Max_Var_Key_Size 
INSERT INTO @VAR VALUES ('@Variable_Key_Size', @Variable_Key_Size)
 
DECLARE @Index_Row_Size FLOAT = @Fixed_Key_Size + @Variable_Key_Size + @Index_Null_Bitmap + 1 + 6
INSERT INTO @VAR VALUES ('@Index_Row_Size',@Index_Row_Size )
 
DECLARE @Index_Rows_Per_Page  FLOAT = FLOOR(8096 / (@Index_Row_Size + 2))
INSERT INTO @VAR VALUES ('@Index_Rows_Per_Page', @Index_Rows_Per_Page)
 
DECLARE @Non_Leaf_Levels FLOAT = 1
 
-- Log base @Non_Leaf_Levels hack
WHILE (@Num_Leaf_Pages / @Index_Rows_Per_Page) > POWER(@Index_Rows_Per_Page,@Non_Leaf_Levels)
	SET @Non_Leaf_Levels += 1
 
SET @Non_Leaf_Levels += 1
INSERT INTO @VAR VALUES ('@Non_Leaf_Levels', @Non_Leaf_Levels)
 
-- Summation hack
DECLARE 
	 @Num_Index_Pages FLOAT = 0
	,@Summand INT = @Non_Leaf_Levels
 
WHILE @Summand > 0
BEGIN
	SET @Num_Index_Pages += CEILING(@Num_Leaf_Pages/POWER(@Index_Rows_Per_Page, @Summand))
	SET @Summand -= 1
END
INSERT INTO @VAR VALUES ('@Num_Index_Pages', @Num_Index_Pages)
 
DECLARE @Index_Space_Used FLOAT = 8192 * @Num_Index_Pages 
INSERT INTO @VAR VALUES ('@Index_Space_Used', @Index_Space_Used)
 
DECLARE @Clustered_Index_Size FLOAT = @Leaf_Space_Used + @Index_Space_used
INSERT INTO @VAR VALUES ('@Clustered_Index_Size', @Clustered_Index_Size)
 
SELECT * FROM @VAR

Now for an example of how to run this doodad. This particular estimate was for a staging table - I needed the UNIQUEIDENTIFER as the key and the CreatedTime field had to be part of the clustered index for the partition scheme.

DECLARE @TableColumn TableColumnDefinition
 
INSERT INTO @TableColumn
VALUES 
	 ('UNIQUEIDENTIFIER', NULL, 1)
	,('DATETIME', NULL, 1)
	,('INT', NULL, 0)
	,('INT', NULL, 0)
	,('INT', NULL, 0)
	,('INT', NULL, 0)
	,('INT', NULL, 0)
	,('INT', NULL, 0)
	,('INT', NULL, 0)
	,('INT', NULL, 0)
	,('INT', NULL, 0)
 
EXEC [dbo].[Utility_EstimateClusteredIndexSize]	
 @Num_Rows = 750000		-- number of rows in the table
,@ClusteredIndexIsUnique = 1 	-- is it?
,@Fill_Factor = 90			-- page fill factor
,@TableColumn = @TableColumn	-- definition of table columns

As for what the output looks like, I wanted it to output all the variables used so I could validate along the way with th emanual method. The results look something like this

Name                                                             Value                  DisplayValue
---------------------------------------------------------------- ---------------------- -------------------
@Fixed_Data_Size                                                 60                     60 B
@Null_Bitmap                                                     5                      5
@Num_Cols                                                        12                     12
@Num_Variable_Cols                                               0                      0
@Max_Var_Size                                                    0                      0 B
@Row_Size                                                        69                     69 B
@Rows_Per_Page                                                   114                    114
@Free_Rows_Per_Page                                              11                     11
@Num_Leaf_Pages                                                  7282                   7282
@Leaf_space_used                                                 59654144               56.891 MB
@Num_Variable_Key_Cols                                           0                      0
@Max_Var_Key_Size                                                0                      0 B
@Num_Key_Cols                                                    3                      3
@Index_Null_Bitmap                                               3.25                   3
@Variable_Key_Size                                               0                      0 B
@Index_Row_Size                                                  34.25                  34 B
@Index_Rows_Per_Page                                             223                    223
@Non_Leaf_Levels                                                 2                      2
@Num_Index_Pages                                                 34                     34
@Index_Space_Used                                                278528                 272.000 KB
@Clustered_Index_Size                                            59932672               57.156 MB

So now you may ask "but what about estimating Non-Clustered Indexes and Heaps?" Well, good question as this is the methodology for estimating the size of a Clustered Index. I ran this for a fact table with two non-clustered indexes in addition to the clustered index by basically defining the index (so pretending the index was a table for the purposes of this proc - included columns would just be non-key columns). All in all, it came out +3.31% on 600+ million rows, so I called that methodology good enough

tim laqua dot com » mssql 2008

Estimating the Size of a Table in SQL Server 2008