一、背景
spring-data-mongo 实现了基于 MongoDB 的 ORM-Mapping 能力,
通过一些简单的注解、Query封装以及工具类,就可以通过对象操作来实现集合、文档的增删改查;
在 SpringBoot 体系中,spring-data-mongo 是 MongoDB Java 工具库的不二之选。
二、问题产生
在一次项目问题的追踪中,发现SpringBoot 应用启动失败,报错信息如下:
1 Error creating bean with name 'mongoTemplate' defined in class path resource [org/bootfoo/BootConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.mongodb.core.MongoTemplate]: Factory method 'mongoTemplate' threw exception; nested exception is org.springframework.dao.DataIntegrityViolationException: Cannot create index for 'deviceId' in collection 'T_MDevice' with keys '{ "deviceId" : 1}' and options '{ "name" : "deviceId"}'. Index already defined as '{ "v" : 1 , "unique" : true , "key" : { "deviceId" : 1} , "name" : "deviceId" , "ns" : "appdb.T_MDevice"}'.; nested exception is com.mongodb.MongoCommandException: Command failed with error 85: 'exception: Index with name: deviceId already exists with different options' on server 127.0.0.1:27017. The full response is { "createdCollectionAutomatically" : false, "numIndexesBefore" : 6, "errmsg" : "exception: Index with name: deviceId already exists with different options", "code" : 85, "ok" : 0.0 } 2 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:588) 3 at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88) 4 at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366) 5 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1264) 6 at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553) 7 8 ... 9 10 Caused by: org.springframework.dao.DataIntegrityViolationException: Cannot create index for 'deviceId' in collection 'T_MDevice' with keys '{ "deviceId" : 1}' and options '{ "name" : "deviceId"}'. Index already defined as '{ "v" : 1 , "unique" : true , "key" : { "deviceId" : 1} , "name" : "deviceId" , "ns" : "appdb.T_MDevice"}'.; nested exception is com.mongodb.MongoCommandException: Command failed with error 85: 'exception: Index with name: deviceId already exists with different options' on server 127.0.0.1:27017. The full response is { "createdCollectionAutomatically" : false, "numIndexesBefore" : 6, "errmsg" : "exception: Index with name: deviceId already exists with different options", "code" : 85, "ok" : 0.0 } 11 at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.createIndex(MongoPersistentEntityIndexCreator.java:157) 12 at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.checkForAndCreateIndexes(MongoPersistentEntityIndexCreator.java:133) 13 at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.checkForIndexes(MongoPersistentEntityIndexCreator.java:125) 14 at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.<init>(MongoPersistentEntityIndexCreator.java:91) 15 at org.springframework.data.mongodb.core.index.MongoPersistentEntityIndexCreator.<init>(MongoPersistentEntityIndexCreator.java:68) 16 at org.springframework.data.mongodb.core.MongoTemplate.<init>(MongoTemplate.java:229) 17 at org.bootfoo.BootConfiguration.mongoTemplate(BootConfiguration.java:121) 18 at org.bootfoo.BootConfiguration$$EnhancerBySpringCGLIB$$1963a75.CGLIB$mongoTemplate$2(<generated>) 19 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) 20 at java.lang.reflect.Method.invoke(Unknown Source) 21 at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:162) 22 ... 58 more 23 24 Caused by: com.mongodb.MongoCommandException: Command failed with error 85: 'exception: Index with name: deviceId already exists with different options' on server 127.0.0.1:27017. The full response is { "createdCollectionAutomatically" : false, "numIndexesBefore" : 6, "errmsg" : "exception: Index with name: deviceId already exists with different options", "code" : 85, "ok" : 0.0 } 25 at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115) 26 at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114) 27 at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:168)
关键信息:org.springframework.dao.DataIntegrityViolationException: Cannot create index
从异常信息上看,出现的是索引冲突(Command failed with error 85),spring-data-mongo 组件在程序启动时会实现根据注解创建索引的功能。
查看业务实体定义:
1 @Document(collection = "T_MDevice") 2 public class MDevice { 3 4 @Id 5 private String id; 6 7 @Indexed(unique=true) 8 private String deviceId;
deviceId 这个字段上定义了一个索引,unique=true表示这是一个唯一索引。
我们继续 查看 MongoDB中表的定义:
1 db.getCollection('T_MDevice').getIndexes() 2 3 >> 4 [ 5 { 6 "v" : 1, 7 "key" : { 8 "_id" : 1 9 }, 10 "name" : "_id_", 11 "ns" : "appdb.T_MDevice" 12 }, 13 { 14 "v" : 1, 15 "key" : { 16 "deviceId" : 1 17 }, 18 "name" : "deviceId", 19 "ns" : "appdb.T_MDevice" 20 } 21 ]
发现数据库表中同样存在一个名为 deviceId的索引,但是并非唯一索引!
三、详细分析
为了核实错误产生的原因,我们尝试通过 Mongo Shell去执行索引的创建,发现返回了同样的错误。
通过将数据库中的索引删除,或更正为 unique=true 之后可以解决当前的问题。
从严谨度上看,一个索引冲突导致 SpringBoot 服务启动不了,是可以接受的。
但从灵活性来看,是否有某些方式能禁用索引的自动创建,或者仅仅是打印日志呢?
尝试 google spring data mongodb disable index creation
发现 JIRA-DATAMONGO-1201在2015年就已经提出,至今未解决。
图
stackoverflow 找到许多同样问题,
但大多数的解答是不采用索引注解,选择其他方式对索引进行管理。
这些结果并不能令人满意。
尝试查看 spring-data-mongo 的机制,定位到 MongoPersistentEntityIndexCreator类:
- 初始化方法中,会根据 MappingContext(实体映射上下文)中已有的实体去创建索引
1 public MongoPersistentEntityIndexCreator(MongoMappingContext mappingContext, MongoDbFactory mongoDbFactory, 2 IndexResolver indexResolver) { 3 ... 4 //根据已有实体创建 5 for (MongoPersistentEntity<?> entity : mappingContext.getPersistentEntities()) { 6 checkForIndexes(entity); 7 } 8 }
2. 在接收到MappingContextEvent时,创建对应实体的索引
1 public void onApplicationEvent(MappingContextEvent<?, ?> event) { 2 3 if (!event.wasEmittedBy(mappingContext)) { 4 return; 5 } 6 7 PersistentEntity<?, ?> entity = event.getPersistentEntity(); 8 9 // Double check type as Spring infrastructure does not consider nested generics 10 if (entity instanceof MongoPersistentEntity) { 11 //创建单个实体索引 12 checkForIndexes((MongoPersistentEntity<?>) entity); 13 } 14 }
MongoPersistentEntityIndexCreator是通过MongoTemplate引入的,如下:
1 public MongoTemplate(MongoDbFactory mongoDbFactory, MongoConverter mongoConverter) { 2 3 Assert.notNull(mongoDbFactory); 4 5 this.mongoDbFactory = mongoDbFactory; 6 this.exceptionTranslator = mongoDbFactory.getExceptionTranslator(); 7 this.mongoConverter = mongoConverter == null ? getDefaultMongoConverter(mongoDbFactory) : mongoConverter; 8 ... 9 10 // We always have a mapping context in the converter, whether it's a simple one or not 11 mappingContext = this.mongoConverter.getMappingContext(); 12 // We create indexes based on mapping events 13 if (null != mappingContext && mappingContext instanceof MongoMappingContext) { 14 indexCreator = new MongoPersistentEntityIndexCreator((MongoMappingContext) mappingContext, mongoDbFactory); 15 eventPublisher = new MongoMappingEventPublisher(indexCreator); 16 if (mappingContext instanceof ApplicationEventPublisherAware) { 17 ((ApplicationEventPublisherAware) mappingContext).setApplicationEventPublisher(eventPublisher); 18 } 19 } 20 } 21 22 23 ... 24 //MongoTemplate实现了 ApplicationContextAware,当ApplicationContext被实例化时被感知 25 public void setApplicationContext(ApplicationContext applicationContext) throws BeansException { 26 27 prepareIndexCreator(applicationContext); 28 29 eventPublisher = applicationContext; 30 if (mappingContext instanceof ApplicationEventPublisherAware) { 31 //MappingContext作为事件来源,向ApplicationContext发布 32 ((ApplicationEventPublisherAware) mappingContext).setApplicationEventPublisher(eventPublisher); 33 } 34 resourceLoader = applicationContext; 35 } 36 37 ... 38 //注入事件监听 39 private void prepareIndexCreator(ApplicationContext context) { 40 41 String[] indexCreators = context.getBeanNamesForType(MongoPersistentEntityIndexCreator.class); 42 43 for (String creator : indexCreators) { 44 MongoPersistentEntityIndexCreator creatorBean = context.getBean(creator, MongoPersistentEntityIndexCreator.class); 45 if (creatorBean.isIndexCreatorFor(mappingContext)) { 46 return; 47 } 48 } 49 50 if (context instanceof ConfigurableApplicationContext) { 51 //使 IndexCreator 监听 ApplicationContext的事件 52 ((ConfigurableApplicationContext) context).addApplicationListener(indexCreator); 53 } 54 }
由此可见,MongoTemplate在初始化时,先通过MongoConverter 带入 MongoMappingContext,
随后完成一系列初始化,整个过程如下:
- 实例化 MongoTemplate;
- 实例化 MongoConverter;
- 实例化 MongoPersistentEntityIndexCreator;
- 初始化索引(通过MappingContext已有实体);
- Repository初始化 -> MappingContext 发布映射事件;
- ApplicationContext 将事件通知到 IndexCreator;
- IndexCreator 创建索引
在实例化过程中,没有任何配置可以阻止索引的创建。
四、解决问题
从前面的分析中,可以发现问题关键在 IndexCreator,能否提供一个自定义的实现呢,答案是可以的!
实现的要点如下
- 实现一个IndexCreator,可继承MongoPersistentEntityIndexCreator,去掉索引的创建功能;
- 实例化 MongoConverter和 MongoTemplate时,使用一个空的 MongoMappingContext对象避免初始化索引;
- 将自定义的IndexCreator作为Bean进行注册,这样在prepareIndexCreator方法执行时,
原来的 MongoPersistentEntityIndexCreator不会监听ApplicationContext的事件 - IndexCreator 实现了ApplicationContext监听,接管 MappingEvent事件处理。
实例化Bean
1 @Bean 2 public MongoMappingContext mappingContext() { 3 return new MongoMappingContext(); 4 } 5 6 // 使用 MappingContext 实例化 MongoTemplate 7 @Bean 8 public MongoTemplate mongoTemplate(MongoDbFactory mongoDbFactory, MongoMappingContext mappingContext) { 9 MappingMongoConverter converter = new MappingMongoConverter(new DefaultDbRefResolver(mongoDbFactory), 10 mappingContext); 11 converter.setTypeMapper(new DefaultMongoTypeMapper(null)); 12 13 MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory, converter); 14 15 return mongoTemplate; 16 }
自定义IndexCreator
1 // 自定义IndexCreator实现 2 @Component 3 public static class CustomIndexCreator extends MongoPersistentEntityIndexCreator { 4 5 // 构造器引用MappingContext 6 public CustomIndexCreator(MongoMappingContext mappingContext, MongoDbFactory mongoDbFactory) { 7 super(mappingContext, mongoDbFactory); 8 } 9 10 public void onApplicationEvent(MappingContextEvent<?, ?> event) { 11 PersistentEntity<?, ?> entity = event.getPersistentEntity(); 12 13 // 获得Mongo实体类 14 if (entity instanceof MongoPersistentEntity) { 15 System.out.println("Detected MongoEntity " + entity.getName()); 16 17 //可实现索引处理.. 18 } 19 } 20 }
在这里 CustomIndexCreator继承了MongoPersistentEntityIndexCreator,将自动接管MappingContextEvent事件的监听。
在业务实现上可以根据需要完成索引的处理!
小结
spring-data-mongo 提供了非常大的便利性,但在灵活性支持上仍然不足。上述的方法实际上有些隐晦,在官方文档中并未提及这样的方式。
ORM-Mapping 框架在实现Schema映射处理时需要考虑校验级别,比如 Hibernate便提供了 none/create/update/validation 多种选择,毕竟这对开发者来说更加友好。
期待 spring-data-mongo 在后续的演进中能尽快完善 Schema的管理功能!
作者:美码师
HDC.Cloud 华为开发者大会2020 即将于2020年2月11日-12日在深圳举办,是一线开发者学习实践鲲鹏通用计算、昇腾AI计算、数据库、区块链、云原生、5G等ICT开放能力的最佳舞台。