flat file import - exclude specific rows

Who is online?  0 guests and 0 members
Home  »  Forums   »  microsoft business intelligence   »  integration services   » flat file import - exclude specific rows

flat file import - exclude specific rows

Topic RSS Feed

Posts under the topic: flat file import - exclude specific rows

Posted: 5/21/2010

Padawan 104  points  Padawan
  • Joined on: 12/21/2009
  • Posts: 37

I import many files a day into our database via my ssis packages.

There is one file that comes in every so often that we do not want to process the full file.

I need to check each row of the file and if it has certain data, that row should NOT be processed. Otherwise, it should be processed. (If 'column H' has 'TT' in it, don't process. else, process)

I'm wondering what my best option is:

a) use a conditional split to determine which rows have 'TT' in it. If they do, don't continue down the pipeline

b) use a streamreader to determine if a line has this 'TT' in it. If it does, do not process

What I really want to know is which option is more 'expensive'. A conditional split sounds much easier to me, but does it matter which one I do?

Maybe there's a better option that I'm not aware of.......(I try to stick away from script components - if that's the other optionSmile)

 

Any input would be appreciated:)


tags exclude

Posted: 5/21/2010

Jedi Master 5335  points  Jedi Master
  • Joined on: 10/27/2009
  • Posts: 296
Answered  Answered

The conditional split will be you best bet. That compoent is synchronous and therefore very fast.


Page 1 of 1 (2 items)