Integration Services

You are currently browsing the archive for the Integration Services category.

Quick post to ask a moment of your time.

Have you ever wanted a feature in the BIDS to quickly identify package variables which have become obsolete?  Well, I’ve been involved in the cleanup of existing packages and I can tell you, that feature would be very handy!

After a search on the internet, it turns out that one of the planned features for the BIDS Helper contains exactly that.  The request is a bit wider than what I need, but at least “highlight unused variables” is part of it.

All I’m asking now is just a minute of your time to vote for that feature request.

Have fun, and thank you!

Valentino.

Share

Tags: , , ,

When I opened an existing SSIS project in the new SQL Server 2012 RC0, I came to an interesting discovery: an empty Toolbox pane!  Even with an SSIS package open in the designer.  Hmm, that’s funny!  So where are my SSIS components?

Take a good look at the following screenshot:

The Toolbox is no longer the SSIS Toolbox but the new SSIS Toolbox is!

That’s right, they are not in the Toolbox anymore but in the SSIS Toolbox instead.  This new toolbox is a bit different from the old one.  Besides the grouping of components that has changed, the most important change is that it will automatically detect any custom components.  You no longer need to right-click, select Choose Items, go fetch a coffee, wait until it cools down a bit, drink it and finally … select your custom component.  No, you’ll have to find another reason to get that coffee shot.  Actually, that’s not entirely true: you still need to right-click and then click Refresh Toolbox and then the custom components will be shown.

Another difference is that it’s split in two parts.  The bottom half of the pane now contains a description of the selected item, including a link that should lead to samples and a link to the Books Online.

The new SSIS Toolbox shows a description of the selected=

Out of curiosity I tried the Find Samples link a couple of times, but for now it doesn’t seem to deliver much content:

Not many results through Find Samples link

Okay, so one thing remains: how do you open the new SSIS Toolbox pane?  According to the Books Online it should be opened automatically when you open an existing project.  Well, apparently not all the time!

The first place I’d look is in the View menu.  But alas, SSIS Toolbox is not one of the menu items.  Not even in the Other Windows submenu.  Why oh why?!

Long story short: do you see those two buttons in the below screenshot?  They’re new!

Package designer has gotten two new buttons

The first button leads to the Variables pane, the second button will open the SSIS Toolbox.  Good to know isn’t it?!

Further investigation led me to the following: according to the Books Online, the SSIS Toolbox item should actually be located in the View > Other Windows menu.  As that is not the case and I think it’s only logical to have that pane added to the View menu as well, I’ve filed a bug on Microsoft Connect.  Go ahead and vote!

Have fun!

Valentino.

References

SSIS Toolbox

Share

Tags: , ,

A couple of days ago I came across a funny and weird object in the Control Flow.  It looked like this:

Group Tasks

As you can see, this container looks a bit similar to the Sequence Container but the Sequence Container has a different icon.  Also, this container does not have any connectors sticking out.  I’m currently involved in a 2005 > 2008 migration project, which is how I came across this container in the first place, and thus I really needed to find out what this object actually is.  So I started to investigate.  Unfortunately the properties were not very helpful:

Group does not have any properties

No properties, nothing at all, not even an object name.  Then I started to scan the Toolbox pane for the icon.  Guess what?  Right, nothing either!  Is this some kind of custom control??

The logical next step is to perform some internet searches and I found the answer: this is standard SSIS functionality that exists since SQL Server 2005!  WTF! (is what I thought at that moment)

The Grouping Container

In the Control Flow, if you right-click on at least one task, you get the following options:

image

And when you select Group, you’ll get that group container around the selected objects, allowing you to collapse the items.  Well, okay, I’ll probably still never use it now that I know that it exists.  Which is possibly the reason that I don’t know about it in the first place.

How Not To Use It

Using that grouping functionality you can end up with some weird-looking flows:

How not to use the group tasks functionality

What have we learned?  That even with years of SSIS experience, it’s still possible to discover “new” functionality.

And speaking of new functionality, that brings us to SQL Server 2012 (formerly known as Denali).

Grouping In SQL Server 2012

As of SQL Server 2012, not only will you be able to group tasks in the Control Flow.  You can also group components in the Data Flow.  Now that may prove more interesting than its Control Flow counterpart.  Why?  Because in the Control Flow chances are that you’ve already grouped your tasks using some Sequence Containers while you don’t have any containers in the Data Flow.

Here’s what it looks like:

SQL Server 2012 allows grouping components in the Data Flow

As you can see, the icon is now gone.  But the properties pane is still totally empty with the group control selected.  Oh well…

Have fun!

Valentino.

References

How to: Group Tasks and Containers in a Control Flow

Share

Tags: ,

When you’ve used SSIS for a while, you may have run into the following situation already.  Or maybe today is your first time and that’s the reason that you’ve arrived here.

“Huh, what’s he talking about?”, I hear you thinking.  Read on then. :-)

The Scenario

You’ve got a stored procedure or another SQL statement that needs to get called from the Execute SQL Task in the Control Flow of your package.  So far so good.  One of the parameters that needs to get passed into the statement is of the DateTime type.

How would you do that?

Parameter Mapping – Take 1

Following the KISS principle, let’s say we’ve got the following really complex table in our database:

create table dt ( dtVal datetime );

And in our Execute SQL task we have this extremely complex INSERT statement:

insert into dt values (?)

The statement is expecting one parameter.  The parameter that I want to pass into it is System::StartTime which is of type DateTime as shown in the screenshot below.

Show the system variables by activating the Show System Variables button

“Hang on, how did you get the Variables window to display the system variables?”

Ah, good question, by clicking that Show System Variables button, indicated with the red rectangle.

So you set up the Parameter Mapping as follows, specifying DBTIMESTAMP as Data Type and zero as Parameter Name because it’s the first parameter in the statement:

Execute SQL Task: Parameter Mapping

Then you decide to give it a test run.  But alas, it throws you the following error:

Error: 0xC002F210 at Execute SQL Task 1, Execute SQL Task: Executing the query “insert into dt values (?)” failed with the following error: “Invalid time format”. Possible failure reasons: Problems with the query, “ResultSet” property not set correctly, parameters not set correctly, or connection not established correctly.

So we’ve got a datetime column in the table and we’ve got a DateTime package variable.  But alas, the Execute SQL Task is not happy with passing this value to the query.

Now what?

The SqlStatementSource Expression

Let’s try another method then.  Instead of passing the parameter’s value through the Parameter Mapping page, we’ll set up an expression that constructs the whole INSERT statement, including the parameter’s value.

Have a look at the following expression:

"insert into dt values ('" +

(DT_STR, 4, 1252) DATEPART("yyyy", @[System::StartTime]) + "-" +

(DT_STR, 2, 1252) DATEPART("mm", @[System::StartTime]) + "-" +

(DT_STR, 2, 1252) DATEPART("dd", @[System::StartTime]) + " " +

(DT_STR, 2, 1252) DATEPART("hh", @[System::StartTime]) + ":" +

(DT_STR, 2, 1252) DATEPART("mi", @[System::StartTime]) + ":" +

(DT_STR, 2, 1252) DATEPART("ss", @[System::StartTime]) + "." +

(DT_STR, 3, 1252) DATEPART("ms", @[System::StartTime]) + "')"

It uses the DATEPART function to fetch parts of the System::StartTime variable and feed it into the INSERT statement using a format that works all the time (YYYY-MM-DD HH:MM:SS.MIL).  Here’s what it generated when I clicked the Evaluate Expression button in the Expression Builder:

insert into dt values (’2011-5-31 17:59:37.0′)

So where exactly would you specify that expression?  In the Execute SQL Task editor, open up the Expressions page.  Then click the Expressions item in the Misc list so that the button with the ellipsis appears.  Now click that button, select SqlStatementSource as property and click the Ellipsis button in the Expression field to get to the Expression Builder.

Then you’ll end up with something like this:

The Property Expressions Editor with an expression specified for the SqlStatementSource property

Give the package another run.  If everything has been set up as expected, the Execute SQL Task should color green and a select on the table should give one record:

Our test table contains one timestamp!

Hang on, does it really have to be this complicated?

Well, maybe not…

Parameter Mapping – Take 2

So let’s give the Parameter Mapping another go.

Set up the Execute SQL Task just like in Take 1 above, with one small difference: select DATE instead of DBTIMESTAMP as Data Type for the parameter.

Choose DATE as Data Type when passing a DateTime package variable into the Execute SQL Task

Now give the package another run.  Look at that, it colors green and there’s an extra record in the table:

An extra timestamp was written to the table

DATE doesn’t seem like the most logical type to choose in this scenario, which is why most people won’t even consider it.  But it works!  Actually, “DATE” is not really the best name that could be given to this particular data type.  Here’s the description of DT_DATE (not to be confused with DT_DBDATE!) according to MSDN:

A date structure that consists of year, month, day, hour, minute, seconds, and fractional seconds. The fractional seconds have a fixed scale of 7 digits.

The DT_DATE data type is implemented using an 8-byte floating-point number. Days are represented by whole number increments, starting with 30 December 1899, and midnight as time zero. Hour values are expressed as the absolute value of the fractional part of the number. However, a floating point value cannot represent all real values; therefore, there are limits on the range of dates that can be presented in DT_DATE.

On the other hand, DT_DBTIMESTAMP is represented by a structure that internally has individual fields for year, month, day, hours, minutes, seconds, and milliseconds. This data type has larger limits on ranges of the dates it can present.

What this means is that you have to be careful when using this type.  Even though it works fine today, it may not run fine in a similar scenario that required different date ranges.  But obviously you’ve got that covered by your unit test scenarios!

Conclusion

In this article I have demonstrated how a DateTime package variable can be passed as parameter into the Execute SQL Task in more than one different way.  My method of preference is the one using the DATE type in the Parameter Mapping.

Have fun!

Valentino.

References

SSIS Execute SQL Task

SSIS DatePart function

KISS Principle

SSIS Junkie: Datetime variables don’t always do what you expect

Share

Tags: , ,

The following is based on recent experience when I needed to call a web service to send out emails from Integration Services 2008 R2.

You probably already know that there are two components in SSIS that allow you to write custom .NET code.  In the Control Flow we’ve got the Script Task, while the Data Flow offers us the Script Component.

In my case I wanted to encapsulate the code that uses the external web service into a custom .NET DLL, aka assembly.  The code to send out emails is needed in several SSIS packages, so to be able to handle changes easily and to promote reuse, this really needed to get encapsulated into one library.

Furthermore, the library is configuring the binding with the web service completely through code so that no app.config file is needed.  The assembly is developed using Visual Studio 2008 and needs the 3.5 .NET Framework.

Assuming that the assembly is already developed and fully tested, open up a package in the BIDS and add a Script Task to the Control Flow.  Open up the Visual Studio Tools for Applications 2.0 (aka VSTA) development environment by clicking the Edit Script… button in the Script Task Editor.

Right-click the References node in the Project Explorer and select Add Reference….

Adding a reference to the Script Task

In the Add Reference popup window, select the Browse tab and select your custom assembly.  After clicking OK, you’re getting the following warning:

Warning when adding a custom assembly to a Script Task in Integration Services

It says that your assembly, or one of its dependencies, requires a later version of the .NET Framework than the one specified in the project.  Click the No button so that we can first set the target framework to the expected version.

To do that, right-click the project node in the Project Explorer and select Properties.  In the Application page, you can see that Target Framework is set to .NET Framework 2.0.

By default, the Target Framework for the Script Task in SSIS 2008 is set to .NET Framework 2.0

Apparently by default the Script Task in Integration Services 2008 R2 still targets the 2.0 Framework.  Change that setting to .NET Framework 3.5.  You’ll get asked if it’s okay to close and reopen the current project, which is needed to change the Target Framework.  That’s okay, so click the Yes button.

Target Framework Change: requires a close/reopen of the current project

With this setting modified you can now add the reference to your custom assembly and start using it in the code.

Tip: in case that the reference to your assembly keeps disappearing every time you reopen the Script Task’s code, click the Save All button – that’s the one that looks like a bunch of blue floppies – in the toolbar after you’ve added the reference.  Or hit CTRL + SHIFT + S.

Besides the scenario described above, a second reason when you may want (or better, have) to change the target framework version for your Script Task or Component is when you actually need to use functionality that doesn’t exist yet in 2.0.  New functionality in .NET 3.5, such as Linq, can only be used if the target framework is switched to 3.5.

That’s it for now, have fun!

Valentino.

References

Where is my app.config for SSIS? by Darren Green

Share

Tags: ,

« Older entries § Newer entries »

© 2008-2013 A Developer's Blog All Rights Reserved