<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>A Developer&#039;s Blog &#187; ETL</title>
	<atom:link href="http://blog.hoegaerden.be/tag/etl/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.hoegaerden.be</link>
	<description>SQL Server, BI, .NET, IT and anything else I have been playing with.</description>
	<lastBuildDate>Wed, 01 Sep 2010 06:27:08 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.9.2</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>List All SSIS Packages Deployed On Your Integration Server</title>
		<link>http://blog.hoegaerden.be/2010/01/10/list-all-ssis-packages-deployed-on-your-integration-server/</link>
		<comments>http://blog.hoegaerden.be/2010/01/10/list-all-ssis-packages-deployed-on-your-integration-server/#comments</comments>
		<pubDate>Sun, 10 Jan 2010 21:23:40 +0000</pubDate>
		<dc:creator>Valentino Vranken</dc:creator>
				<category><![CDATA[Integration Services]]></category>
		<category><![CDATA[SQLServerPedia Syndication]]></category>
		<category><![CDATA[T-SQL]]></category>
		<category><![CDATA[data]]></category>
		<category><![CDATA[ETL]]></category>
		<category><![CDATA[Script]]></category>
		<category><![CDATA[SSIS]]></category>

		<guid isPermaLink="false">http://blog.hoegaerden.be/2010/01/10/list-all-ssis-packages-deployed-on-your-integration-server/</guid>
		<description><![CDATA[When deploying packages to SQL Server Integration Services, it&#8217;s advisable to set up a folder structure so that you can easily distinguish packages belonging to different projects.&#160; Furthermore it may be interesting to create subfolders under the main project folder to separate packages according to the different phases in your ETL (Extract, Transform, Load) process.&#160; [...]]]></description>
			<content:encoded><![CDATA[<p>When deploying packages to SQL Server Integration Services, it&#8217;s advisable to set up a folder structure so that you can easily distinguish packages belonging to different projects.&#160; Furthermore it may be interesting to create subfolders under the main project folder to separate packages according to the different phases in your <a title="Wikipedia: Extract, transform, load" href="http://en.wikipedia.org/wiki/Extract,_transform,_load" target="_blank">ETL (Extract, Transform, Load)</a> process.&#160; When loading a data warehouse, interesting folder names are Dimensions for your dimension ETLs and Facts for the packages that load the fact tables.</p>
<p>After a while you end up with lots of packages spread over lots of folders.&#160; To get a good view of what is deployed on your server, it may be interesting to find a way to list all the packages.&#160; And that&#8217;s exactly the reason why I&#8217;m writing this article.</p>
<p>The query further down generates <strong>a list of all packages deployed in the MSDB database</strong> on your SQL Server.&#160; What you get is the name of the packages, their location and version-related information.&#160; I&#8217;ve also created a&#160; RootFolder column so that it&#8217;s easy to filter on project.&#160; (See now why it&#8217;s interesting to create separate folders per project?)</p>
<p>It’s <strong>important to note</strong> that packages deployed to the <strong>File System will not be shown</strong> in the list.&#160; After all, they are not stored in the MSDB database but in a folder somewhere on the server&#8217;s hard drive, more precisely in a subfolder of where you&#8217;ve installed your SQL Server.&#160; In case you&#8217;ve forgotten where that was, here&#8217;s a small tip.&#160; On your server, open up the list of Windows Services (Start &gt; Run &gt; type &quot;services.msc&quot; &gt; enter) and locate the service called <em>SQL Server Integration Services 10.0</em>.&#160; Open the properties of that service and have a look at the <em>Path to executable</em> value in the General tab.&#160; Take the path, drop the \Binn part and add \Packages instead.&#160; That is where, by default, the packages are deployed.&#160; (If you’re running SQL Server 2005, apply the same procedure but look for a service called <em>SQL Server Integration Services</em>.)</p>
<p>On my system, this is where the packages are located: D:\Program Files\Microsoft SQL Server\100\DTS\Packages.&#160; I will also prove it with following screenshot:</p>
<p><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="Packages deployed to the file system on SSIS 2008" border="0" alt="Packages deployed to the file system on SSIS 2008" src="http://blog.hoegaerden.be/wp-content/uploads/image140.png" width="381" height="82" /></p>
<p>Okay, time for the real stuff, the query:</p>
<pre class="code"><span style="color: green">/*
    DESCRIPTION: Lists all SSIS packages deployed to the MSDB database.
    WRITTEN BY: Valentino Vranken
    VERSION: 1.1
    COPIED FROM: http://blog.hoegaerden.be

    Note: this query was written for SQL Server 2008. For SQL2005:
        o sysssispackagefolders =&gt; sysdtspackagefolders90
        o sysssispackages =&gt; sysdtspackages90
*/
</span><span style="color: blue">with </span>ChildFolders
<span style="color: blue">as
</span><span style="color: gray">(
    </span><span style="color: blue">select </span>PARENT<span style="color: gray">.</span>parentfolderid<span style="color: gray">, </span>PARENT<span style="color: gray">.</span>folderid<span style="color: gray">, </span>PARENT<span style="color: gray">.</span>foldername<span style="color: gray">,
        </span><span style="color: magenta">cast</span><span style="color: gray">(</span><span style="color: red">'' </span><span style="color: blue">as sysname</span><span style="color: gray">) </span><span style="color: blue">as </span>RootFolder<span style="color: gray">,
        </span><span style="color: magenta">cast</span><span style="color: gray">(</span>PARENT<span style="color: gray">.</span>foldername <span style="color: blue">as varchar</span><span style="color: gray">(</span><span style="color: magenta">max</span><span style="color: gray">)) </span><span style="color: blue">as </span>FullPath<span style="color: gray">,
        </span>0 <span style="color: blue">as </span>Lvl
    <span style="color: blue">from </span>msdb<span style="color: gray">.</span>dbo<span style="color: gray">.</span>sysssispackagefolders PARENT
    <span style="color: blue">where </span>PARENT<span style="color: gray">.</span>parentfolderid <span style="color: gray">is null
    </span><span style="color: blue">UNION </span><span style="color: gray">ALL
    </span><span style="color: blue">select </span>CHILD<span style="color: gray">.</span>parentfolderid<span style="color: gray">, </span>CHILD<span style="color: gray">.</span>folderid<span style="color: gray">, </span>CHILD<span style="color: gray">.</span>foldername<span style="color: gray">,
        </span><span style="color: blue">case </span>ChildFolders<span style="color: gray">.</span>Lvl
            <span style="color: blue">when </span>0 <span style="color: blue">then </span>CHILD<span style="color: gray">.</span>foldername
            <span style="color: blue">else </span>ChildFolders<span style="color: gray">.</span>RootFolder
        <span style="color: blue">end as </span>RootFolder<span style="color: gray">,
        </span><span style="color: magenta">cast</span><span style="color: gray">(</span>ChildFolders<span style="color: gray">.</span>FullPath <span style="color: gray">+ </span><span style="color: red">'/' </span><span style="color: gray">+ </span>CHILD<span style="color: gray">.</span>foldername <span style="color: blue">as varchar</span><span style="color: gray">(</span><span style="color: magenta">max</span><span style="color: gray">))
            </span><span style="color: blue">as </span>FullPath<span style="color: gray">,
        </span>ChildFolders<span style="color: gray">.</span>Lvl <span style="color: gray">+ </span>1 <span style="color: blue">as </span>Lvl
    <span style="color: blue">from </span>msdb<span style="color: gray">.</span>dbo<span style="color: gray">.</span>sysssispackagefolders CHILD
        <span style="color: gray">inner join </span>ChildFolders <span style="color: blue">on </span>ChildFolders<span style="color: gray">.</span>folderid <span style="color: gray">= </span>CHILD<span style="color: gray">.</span>parentfolderid
<span style="color: gray">)
</span><span style="color: blue">select </span>F<span style="color: gray">.</span>RootFolder<span style="color: gray">, </span>F<span style="color: gray">.</span>FullPath<span style="color: gray">, </span>P<span style="color: gray">.</span>name <span style="color: blue">as </span>PackageName<span style="color: gray">,
    </span>P<span style="color: gray">.</span><span style="color: blue">description as </span>PackageDescription<span style="color: gray">, </span>P<span style="color: gray">.</span>packageformat<span style="color: gray">, </span>P<span style="color: gray">.</span>packagetype<span style="color: gray">,
    </span>P<span style="color: gray">.</span>vermajor<span style="color: gray">, </span>P<span style="color: gray">.</span>verminor<span style="color: gray">, </span>P<span style="color: gray">.</span>verbuild<span style="color: gray">, </span>P<span style="color: gray">.</span>vercomments<span style="color: gray">,
    </span><span style="color: magenta">cast</span><span style="color: gray">(</span><span style="color: magenta">cast</span><span style="color: gray">(</span>P<span style="color: gray">.</span>packagedata <span style="color: blue">as varbinary</span><span style="color: gray">(</span><span style="color: magenta">max</span><span style="color: gray">)) </span><span style="color: blue">as xml</span><span style="color: gray">) </span><span style="color: blue">as </span>PackageData
<span style="color: blue">from </span>ChildFolders F
    <span style="color: gray">inner join </span>msdb<span style="color: gray">.</span>dbo<span style="color: gray">.</span>sysssispackages P <span style="color: blue">on </span>P<span style="color: gray">.</span>folderid <span style="color: gray">= </span>F<span style="color: gray">.</span>folderid
<span style="color: blue">order by </span>F<span style="color: gray">.</span>FullPath <span style="color: blue">asc</span><span style="color: gray">, </span>P<span style="color: gray">.</span>name <span style="color: blue">asc</span><span style="color: gray">;</span></pre>
<p>The query uses a recursive CTE (<a title="BOL 2008: Using Common Table Expressions" href="http://msdn.microsoft.com/en-us/library/ms190766.aspx">Common Table Expression</a>) to get data out of a system table called <a title="BOL 2008: sysssispackagefolders (Transact-SQL)" href="http://msdn.microsoft.com/en-us/library/ms189477.aspx">sysssispackagefolders</a>, located in the MSDB system database.&#160; The CTE gives us a list of all folders stored in the database and at the same time uses the hierarchical structure of the table to build the <em>FullPath</em> and the <em>Lvl</em> columns.</p>
<p><strong>Note:</strong> the <a title="BOL 2008: CAST and CONVERT (Transact-SQL)" href="http://msdn.microsoft.com/en-us/library/ms187928.aspx">CAST()</a> calls are needed because the data type of the <em>foldername</em> column is <a title="BOL 2008: Using Special Data Types" href="http://msdn.microsoft.com/en-us/library/ms191240.aspx">sysname</a>.&#160; And sysname does not implicitly convert to varchar, which is needed for the concatenation building the <em>FullPath</em> column.</p>
<p>The CTE is joined with another system table called <a title="BOL 2008: sysssispackages (Transact-SQL)" href="http://msdn.microsoft.com/en-us/library/ms181582.aspx">sysssispackages</a>, also located in MSDB.&#160; Not all columns are being retrieved from that table but I believe I&#8217;ve selected the most important ones.&#160; Have a look in the Books Online for more info on the columns available.</p>
<p>There’s one column however on which I’d like to add some additional info myself.&#160; That column is called <em>packagedata</em> and it contains the actual SSIS package.&#160; The data type of this column is <a title="BOL 2008: ntext, text, and image (Transact-SQL)" href="http://msdn.microsoft.com/en-us/library/ms187993.aspx">image</a>, not sure why because after all, an SSIS package (or .dtsx file for that matter) is pure XML.&#160; So why isn’t it stored as XML? <strike> If anyone knows the reason: post a comment!</strike></p>
<p><strong>Update:</strong> since I wrote the above paragraph I’ve come across the answer myself.&#160; The reason that the XML is not stored as xml datatype is because of the overhead that this would cause.&#160; So there you go, use image instead of xml if you’re not going to query the xml structure itself.</p>
<p>Anyway, as you can see in the query, to get it converted from image to XML you need to go through varbinary.&#160; The image datatype cannot convert directly to XML.&#160; See the Books Online here on what casts are allowed: <a title="http://msdn.microsoft.com/en-us/library/ms187928.aspx" href="http://msdn.microsoft.com/en-us/library/ms187928.aspx">http://msdn.microsoft.com/en-us/library/ms187928.aspx</a></p>
<p><strong>Note for SQL Server 2005 users:</strong> as mentioned in the query’s comments, these tables don’t exist in SQL Server 2005.&#160; Well, actually they do, they just have different names.&#160; See the comment in the code for their equivalent.</p>
<p>To finish off I&#8217;ll show you what the results look like when executing the query on my test system.&#160; But first, following screenshot shows all deployed packages as reported by the Management Studio.&#160; As you can see, two packages are deployed to the File System.&#160; These two packages were shown earlier in the first screenshot.&#160; Some other packages have been deployed to the MSDB database.</p>
<p><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="Object Explorer showing all deployed SSIS packages" border="0" alt="Object Explorer showing all deployed SSIS packages" src="http://blog.hoegaerden.be/wp-content/uploads/image141.png" width="329" height="306" /></p>
<p>And here are the results of the query:</p>
<p><a href="http://blog.hoegaerden.be/wp-content/uploads/image143.png"><img style="border-right-width: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px" title="A list of all SSIS packages deployed to my SQL Server 2008 MSDB database" border="0" alt="A list of all SSIS packages deployed to my SQL Server 2008 MSDB database" src="http://blog.hoegaerden.be/wp-content/uploads/image_thumb37.png" width="692" height="84" /></a></p>
<p>To be honest, I added a little filter to keep the results clean.&#160; The <a title="BOL 2008: Introducing the Data Collector" href="http://msdn.microsoft.com/en-us/library/bb677248.aspx">Data Collector</a>, a new feature of SQL Server 2008, also uses some packages so I’ve filtered those out by adding a WHERE clause to the SELECT statement at the bottom of the full query:</p>
<pre class="code"><span style="color: blue">select </span>F<span style="color: gray">.</span>RootFolder<span style="color: gray">, </span>F<span style="color: gray">.</span>FullPath<span style="color: gray">, </span>P<span style="color: gray">.</span>name <span style="color: blue">as </span>PackageName<span style="color: gray">,
    </span>P<span style="color: gray">.</span><span style="color: blue">description as </span>PackageDescription<span style="color: gray">, </span>P<span style="color: gray">.</span>packageformat<span style="color: gray">, </span>P<span style="color: gray">.</span>packagetype<span style="color: gray">,
    </span>P<span style="color: gray">.</span>vermajor<span style="color: gray">, </span>P<span style="color: gray">.</span>verminor<span style="color: gray">, </span>P<span style="color: gray">.</span>verbuild<span style="color: gray">, </span>P<span style="color: gray">.</span>vercomments<span style="color: gray">,
    </span><span style="color: magenta">cast</span><span style="color: gray">(</span><span style="color: magenta">cast</span><span style="color: gray">(</span>P<span style="color: gray">.</span>packagedata <span style="color: blue">as varbinary</span><span style="color: gray">(</span><span style="color: magenta">max</span><span style="color: gray">)) </span><span style="color: blue">as xml</span><span style="color: gray">) </span><span style="color: blue">as </span>PackageData
<span style="color: blue">from </span>ChildFolders F
    <span style="color: gray">inner join </span>msdb<span style="color: gray">.</span>dbo<span style="color: gray">.</span>sysssispackages P <span style="color: blue">on </span>P<span style="color: gray">.</span>folderid <span style="color: gray">= </span>F<span style="color: gray">.</span>folderid
<span style="color: blue">where </span>F<span style="color: gray">.</span>RootFolder <span style="color: gray">&lt;&gt; </span><span style="color: red">'Data Collector'
</span><span style="color: blue">order by </span>F<span style="color: gray">.</span>FullPath <span style="color: blue">asc</span><span style="color: gray">, </span>P<span style="color: gray">.</span>name <span style="color: blue">asc</span><span style="color: gray">;</span></pre>
<p><a href="http://11011.net/software/vspaste"></a></p>
<p><a href="http://11011.net/software/vspaste"></a></p>
<p>If you’ve been paying attention, you’ve noticed that the two packages deployed to the File System are not mentioned in the output of the query, as expected.</p>
<p>Now that you know how to list your packages, <a title="Recursively Delete SSIS Folder" href="http://blog.hoegaerden.be/2010/02/28/recursively-delete-ssis-folder/" target="_blank">check out my article on deleting them</a>.</p>
<p>That’s all for now folks, have fun!</p>
<p><strong>References</strong></p>
<p><a title="BOL 2008: Tutorial: Creating a Simple ETL Package" href="http://msdn.microsoft.com/en-us/library/ms169917.aspx">BOL 2008: Tutorial: Creating a Simple ETL Package</a></p>
<p><a title="BOL 2008: sysssispackagefolders (Transact-SQL)" href="http://msdn.microsoft.com/en-us/library/ms189477.aspx">BOL 2008: sysssispackagefolders (Transact-SQL)</a></p>
<p><a title="BOL 2008: sysssispackages (Transact-SQL)" href="http://msdn.microsoft.com/en-us/library/ms181582.aspx">BOL 2008: sysssispackages (Transact-SQL)</a></p>
<p><a title="BOL 2008: Recursive Queries Using Common Table Expressions" href="http://msdn.microsoft.com/en-us/library/ms186243.aspx">BOL 2008: Recursive Queries Using Common Table Expressions</a></p>
]]></content:encoded>
			<wfw:commentRss>http://blog.hoegaerden.be/2010/01/10/list-all-ssis-packages-deployed-on-your-integration-server/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
	</channel>
</rss>
