SSIS 2008 lookup Altering the SQL statement

Who is online?  0 guests and 0 members
Home  »  Articles  »  SSIS 2008 lookup Altering the SQL statement

SSIS 2008 lookup Altering the SQL statement

change text size: A A A
Published: 3/17/2010 by  MikeDavis  - Views:  [2346]  

When setting up a lookup in SSIS it is usually just a basic comparison between two key fields like ID fields. What if you want to compare an id and also find a specific date range. Let's say the id is in the lookup reference table multiple times and you need to find the one that has the date that surrounds a source date.

First let's take a look at the tables.

I have a Lookup Source table that will be the source in the data flow and a Lookup Reference table that will be used in the Lookup Transform. Notice there are three IDs in the source and each id shows up twice in the reference table. So if I want to control the ID the lookup returns I will need more criteria. I want the Source Date to be between the start date and end date on the reference table. This is easy to do in the Lookup transform.

SSIS 2008 lookup Altering the SQL statement

The source in the data flow is just a simple OLEDB source. I have duplicated the source date and gave both and alias. One called Source Begin Date and the other called Source End Date. This will give me the two dates I need in the lookup transform.

SSIS 2008 lookup Altering the SQL statement

You must set the cache to partial cache to allow the changes to the lookup reference query.

SSIS 2008 lookup Altering the SQL statement

 

In the column node I map over the ID. I also map the two source dates to the reference dates. If we left the lookup in this state we would get back no rows.

 

SSIS 2008 lookup Altering the SQL statement

 

Go into the advanced node in the lookup transform (Not the advanced editor). Check the option to modify the SQL statement and change the start date and end date from equals, to greater than or equal to, and less than or equal to, as seen in the image. This will find the row that has a date range that includes the date from the source.

SSIS 2008 lookup Altering the SQL statement

 

I placed a data viewer after the lookup in the data flow and the image shows the results. Notice I got all the rows with the term bad in the lookup info.

SSIS 2008 lookup Altering the SQL statement

Now I will change all the dates in the source table. This will make the lookup find different rows.

SSIS 2008 lookup Altering the SQL statement

 

Notice now I get the rows with good in the Lookup Info.

SSIS 2008 lookup Altering the SQL statement

A lookup tranform is a great tool in SSIS and has many possibilities. Take some time and understand these concepts and it will be a big help when creating data flows. Let me know if you have any questions.

 

 

 
0
/5
Avg: 0/5: (0 votes)

Comments (10)

MarkGStacey
MarkGStacey said:
Hey Mike I have a different pattern that I use for Lookup transforms that allows us to alter the start and end date and still use FULL CACHE. I'll post a fuller article in time, but for now the summary is this: Create a FilterDates table, with a start date and an End Date field , and also a "PhaseName" field. At the beginning of each phases ETL, have a SQL Statement that MERGES the MIN(xDate) and MAX(xDate) from the landing table into the FilterDates table, using the "PhaseName" Then, create a view : CREATE VIEW vwPhaseNameDates SELECT xxxx FROM tableXX INNER JOIN FilterDates ON tableXX.PhaseName = "PhaseName" WHERE tableXX.DateID BETWEEN FilterDate.StartDateID and FilterDate.EndDateID Then use this view in the lookup transform. I've found orders of magnitude difference between Partial and Full Cache - if you have the memory of course :-=)
3/28/2010
 · 
 
by
mikedavis
mikedavis said:
Great, I look forward to the article
3/28/2010
 · 
 
by
Ginni
Ginni said:
thanks
3/30/2010
 · 
 
by
nitya
nitya said:
Found It useful. Thanks
3/31/2010
 · 
 
by
Rob_Randall
Rob_Randall said:
Clear, concise and helpful. Good info, thanks Mike.
4/5/2010
 · 
 
by
sambireddy
sambireddy said:
Excelenc to write these type of explanation
4/11/2010
 · 
 
by
nashworth
nashworth said:
Realy useful article, thanks
4/13/2010
 · 
 
by
Daniel
Daniel said:

Good stuff.

5/10/2010
 · 
 
by
rajforyu
rajforyu said:

Hi mike

I have one question.

I have  one table(TABLE_1) in text file in which I have one column called MONTH (varhcar (50)) and has data like jan,apr,may.....

I have one more table(TABLE_2) in which I have two columns like MONTH AND MONTH_CODE it contains data like 

jan-1, feb-1,mar-3... 

My question is I want to convert MONTH COLUMN in TABLE_1 with appropriate month code , I mean for jan its 1, for feb its 2..

any suggetion? 

I really appreciate your help.

6/10/2010
 · 
 
by
mikedavis
mikedavis said:

Use a Derived Column with a nested condition. Month == 1? Jan : Month == 2 ? Feb  : Ect...

6/14/2010
 · 
 
by
  • Name:*
  • Email:*
  • Website:
Type the characters you see in the image: *

Most Recent Articles