mongodb支持批量插入。
1.使用Java mongodb api
查看源码com.mongodb.MongoCollectionImpl,有两个方法
@Override public void insertMany(final List<? extends TDocument> documents) { insertMany(documents, new InsertManyOptions()); } @Override public void insertMany(final List<? extends TDocument> documents, final InsertManyOptions options) { notNull("documents", documents); List<InsertRequest> requests = new ArrayList<InsertRequest>(documents.size()); for (TDocument document : documents) { if (document == null) { throw new IllegalArgumentException("documents can not contain a null value"); } if (getCodec() instanceof CollectibleCodec) { document = ((CollectibleCodec<TDocument>) getCodec()).generateIdIfAbsentFromDocument(document); } requests.add(new InsertRequest(documentToBsonDocument(document))); } executor.execute(new MixedBulkWriteOperation(namespace, requests, options.isOrdered(), writeConcern) .bypassDocumentValidation(options.getBypassDocumentValidation())); }
insertMany(final List<? extends TDocument> documents) 默认使用 private boolean ordered = true;即有序插入。
insertMany(final List<? extends TDocument> documents, final InsertManyOptions options)调用者自己设置。
ordered属性有什么用?
/** * Gets whether the documents should be inserted in the order provided, stopping on the first failed insertion. The default is true. * If false, the server will attempt to insert all the documents regardless of an failures. * * @return whether the the documents should be inserted in order */ public boolean isOrdered() { return ordered; }
大概意思是:
true:按提供的顺序插入文档,并在首次插入失败时停止。 (按顺序插入文档,遇到失败后停止。之前已经插入成功的不返回,失败及失败之后的不插入。)
false: 服务器将尝试插入所有文档,而不考虑失败。(只要能插入成功的,都存储进数据库)
2.spring-data-mongodb
org.springframework.data.mongodb.core.MongoTemplate
以下是该方法的源码和使用案例:
/* * (non-Javadoc) * @see org.springframework.data.mongodb.core.ExecutableInsertOperation#bulkOps(org.springframework.data.mongodb.core.BulkMode, java.lang.String) */ public BulkOperations bulkOps(BulkMode bulkMode, String collectionName) { return bulkOps(bulkMode, null, collectionName); }
public int batchInsertStudents() {
BulkWriteResult result = null;
try { List<Student> documents = new ArrayList<>(); String collectionName = "myTestCol"; BulkOperations bulkOp = this.mongoTemplate.bulkOps(BulkMode.UNORDERED, collectionName); for(int i = 502610; i< 2000000; i++) { Student student = new Student(); student.setId(String.valueOf(i)); student.setAge(i); student.setGender("男"); student.setName("李三"+ i); documents.add(student); } bulkOp.insert(documents); result = bulkOp.execute(); }catch (DuplicateKeyException e) { System.out.println("**********" + e.getMessage()); } return result; }
有两种模式:
/** * Mode for bulk operation. **/ enum BulkMode { /** Perform bulk operations in sequence. The first error will cancel processing. */ ORDERED, /** Perform bulk operations in parallel. Processing will continue on errors. */ UNORDERED };
与之前的order(true|false)相对应。