下面的C#代码可以利用SqlBulkCopy实现大数据量的上传:
public void SaveDataTable(string connectionString, String tableName, int batchSize, DataTable table)
{
using (SqlConnection sqlConnection = new SqlConnection(connectionString))
{
sqlConnection.Open();
var sqlTransaction = sqlConnection.BeginTransaction();
try
{
//通过SqlBulkCopyOptions和ExternalTrasaction这两个参数我们可以对事务有很好的控制选择。
using (SqlBulkCopy sqlBulkCopy = new SqlBulkCopy(sqlConnection, SqlBulkCopyOptions.TableLock, sqlTransaction))
{
sqlBulkCopy.BatchSize = batchSize;
sqlBulkCopy.BulkCopyTimeout = 900;
sqlBulkCopy.DestinationTableName = tableName;
sqlBulkCopy.WriteToServer(table);
sqlTransaction.Commit();
}
}
catch (Exception)
{
sqlTransaction.Rollback();
throw;
}
}
}
以下为此方法在实际使用中的统计数据(单位为毫秒):
每次10000条数据:
bulkcopy:4718.6594
bulkcopy:1921.8381
bulkcopy:2437.4532
bulkcopy:1906.2134
bulkcopy:2640.5743
bulkcopy:2578.0755
bulkcopy:2734.3225
bulkcopy:2406.2038
bulkcopy:2671.8237
bulkcopy:2578.0755
bulkcopy:2531.2014
bulkcopy:2046.8357
bulkcopy:2078.0851
bulkcopy:2421.8285
bulkcopy:2593.7002
bulkcopy:2656.199
bulkcopy:2640.5743
bulkcopy:2421.8285
bulkcopy:2578.0755
bulkcopy:2593.7002
bulkcopy:2890.5695
每次50000条数据:
bulkcopy:22030.827
bulkcopy:6640.4975
bulkcopy:7156.1126
bulkcopy:8171.7181
bulkcopy:7109.2385
bulkcopy:6718.621
bulkcopy:6734.2457
bulkcopy:6187.3812
bulkcopy:6734.2457
bulkcopy:6937.3668
bulkcopy:6796.7445
bulkcopy:6812.3692
bulkcopy:10327.9267
bulkcopy:7859.2241
bulkcopy:7624.8536
bulkcopy:7390.4831
每次100000条数据:
bulkcopy:6296.7541
bulkcopy:6843.6186
bulkcopy:32311.8796
bulkcopy:8062.3452
bulkcopy:7702.9771
bulkcopy:7577.9795
bulkcopy:8109.2193
bulkcopy:8577.9603
bulkcopy:7734.2265
bulkcopy:7499.856
bulkcopy:7593.6042
bulkcopy:7656.103
bulkcopy:7765.4759
每次200000条数据:
bulkcopy:16138.3021
bulkcopy:13833.7382
bulkcopy:14786.1739
bulkcopy:17549.7988
bulkcopy:15629.3137
bulkcopy:7135.4609
bulkcopy:7307.2116
bulkcopy:9368.22
bulkcopy:8931.0364
bulkcopy:10008.3817
bulkcopy:8571.9213
bulkcopy:9055.946
bulkcopy:37832.6585
每次500000条数据:
bulkcopy:31163.097
bulkcopy:26992.3968
bulkcopy:26211.3668