I want to import CSV file and convert columns into rows depending on Customer count(2nd record in each row of CSV file) and save to SQL table
--CSV file format
State, Customer_Count, Name_1, Total_1,Name_2, Total_2,Name_3, Total_3..can go upto 600
GA,2,'John Doe',14.00,'Roger Smith',15.00 
FL,3,'John Doe',14.00,'Roger Smith',15.00,'Sally Cox',16.00 
SC,5,'John Doe',14.00,'Roger Smith',15.00,'Sally Cox',16.00,'James Brown',17.00,'Rick Davis',18.00
Data in SQL table from csv file should look like this
State,Name,Total
GA,John Doe,14.00
GA,Roger Smith,15.00
FL,John Doe,14.00,
FL,Roger Smith,15.00
FL,Sally Cox,16.00 
I have multiple CSV files with millions of records. How can i achieve this using Integration Services or Bulk Data Import.
You can use something like this..
Create Table #DataIntoRows
(
 State Varchar(10),
 Name Varchar(100),
 Total Int
)
SELECT * INTO #Data FROM OPENROWSET('MSDASQL','Driver={Microsoft Text Driver (*.txt; *.csv)};DefaultDir={PATH}','SELECT * FROM Data.csv')
Declare @.I as Int;
Declare @.Count as Int;
Select 
 @.I = 1, 
 @.Count=Count(Colid) 
From 
 tempdb..Syscolumns 
Where 
 id = Object_Id('tempdb..#Data') and Colid > 2;
While @.I < @.Count
begin
 Declare @.SQL as Varchar(300);
 Select @.SQL = 'Insert Into #DataIntoRows Select State,'
 Select @.SQL = @.SQL + name From tempdb..Syscolumns Where id = Object_Id('tempdb..#Data') and colId= @.I + 2
 Select @.SQL = @.SQL + ',' + name + ' From #Data Where ' + name + ' Is NOT NULL' From tempdb..Syscolumns Where id = Object_Id('tempdb..#Data') and colId= @.I + 3
 Exec(@.SQL)
 Select @.I = @.I + 2;
End
 
No comments:
Post a Comment