SQL Performance - importing

greenspun.com : LUSENET : SQL Server Database Administration : One Thread

I am not a developer, but am managing the development and implementation of a small VB/SQL Server 6.5 project. We are reading in text files using VB, formatting them and inserting into a SQL table. The database in on a large server. The text input files are on another computer on the network. The application is running on a third computer. We are only getting 50 records in per second. I believe it should be much faster. Any thoughts?

-- Anonymous, November 24, 1998


Re: SQL Performance - importing


The fastest way to get data into SQL Server is by using the bcp (bulk copy) process. When used properly this can avoid the overhead of table indexes and transaction logging. There is an article in the SQL Server Books Online titled, "Copying Data from a File to SQL Server" that details how to transfer data into SQL Server quickly and safely.


-- Anonymous, November 30, 1998


I quickly learned about BCP. We are now reading a text file, formatting it into a new text file and then BCP'ing it into SQL. On the subject of BCP, when I use BCP to export, I have to specify my field terminator (the -t parameter). Is there any selection that will not place any character (not even a space) between the fields so that they are bumped right up against each other?

-- Anonymous, November 30, 1998



In the format file, just provide a pair of double quotes with nothing in them: ""

for the termination.


-- Anonymous, December 15, 1998

Moderation questions? read the FAQ