Hi,
I have a new requirement. we have folder contains more than 500 csv files with same structure . each file contains more than 2 million records . using SSIS ,time takesn for one file to import around 4 s. my requirement is i want to import all the files in the folder with in below 10 sec. Is there any way to acheive this . please advise ?
structure of file
Sdate ,startValue ,endValue, adjustedValue
Just reading the requirement, you are talking about reading around 250 files all at once (given 500 files and each takes around 4s). I'm not sure if that can be done.
DennisC
Yes . Is there any suggessions ?
SSIS will run dataflows in parallel if there are no dependencies between them. Don't think you can achieve 250 in parallel though! The best you can do is the number of cores * the number of threads per core. Even on very high-end gear you won't get 250 in parallel.