环境依赖:
Spring Boot:1.5.9
JDK:1.8.0
MySQL:5.7.17
Mybatis:3.3.0
本文主要就mybatis的多数据源切换和动态数据源加载的实现原理做分享;对于mybatis的基础可自行百度。由于在开始学习的时候,发现网上有很多人把多数据源切换和动态数据源加载混为一谈,导致在实现动态加载的时候,所很苦恼。所以有必要在这里做以简要说明:
多数据源切换:指项目所需要不止一个数据库的连接信息,eg:同一数据库地址下的不同库或者压根连地址都不同。
动态切换:指所需要的数据所在的数据库信息在项目启动前并不知道,只有在项目运行后根据业务逻辑获取到对应的数据库信息,并在代码的运行过程中,向Spring Boot中添加一个或多个mybatis实例。
单一数据源的连接
顾名思义,在项目中,在项目中只需要配置一个数据库的信息即可,业务所需要的所有数据均在这一个数据库下;这种场景通常能够适用于绝大部分的实际需求,因此这种实现的原理再次不做赘述,如有需求可自行百度。具体实现可参考源码spring-boot-mybatis-demo。
多数据源切换
业务场景:需要分别获取所有的用户信息和学生信息;但是已知用户信息在mybatis_demo数据库中,学生信息在mybatis_demo2 数据库中。如下图所示:
数据库mybatis_demo内有个用户表:user_info,表结构如下:
数据库mybatis_demo2内有一个学生表:student_info,表结构如下:
配置文件信息如下:
在这里介绍一种最为简单的实现方案:多数据源 - 多实例。
在熟悉了单实例数据源的实现后,不难看出,在Spring Boot中,通过为该数据源DataSource初始化一个与之对应的SessionFactory,从而实现连接。因此在面对多数据源的时候,可以分别为每个数据源写一个mybatis的config类,使其每个DataSource都拥有一个只属于自己的SessionFactory,这样就可以根据各自的mapper映射目录找到对应的mybaits实例;
这种实现方法要求不同的mybatis实例的mapper映射目录不能相同
把一个配置类作下的Bean命名统一,并注入相应的Bean,从而可以保证每一个SessionFactory所对应的配置信息唯一;具体配置如下:
第一个数据源的配置:
@Configuration @MapperScan(basePackages = "com.yhyr.mybatis.mapper.UserMapper", sqlSessionTemplateRef = "oneSqlSessionTemplate") public class MybatisConfig { @Bean(name = "oneDataSource") @ConfigurationProperties(prefix = "spring.datasource.one") @Primary public DataSource customDataSource() { return DataSourceBuilder.create().build(); } @Bean(name = "oneSqlSessionFactory") @Primary public SqlSessionFactory customSqlSessionFactory(@Qualifier("oneDataSource") DataSource dataSource) throws Exception { SqlSessionFactoryBean bean = new SqlSessionFactoryBean(); bean.setDataSource(dataSource); bean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath*:mapper/*.xml")); return bean.getObject(); } @Bean(name = "oneTransactionManager") @Primary public DataSourceTransactionManager customTransactionManager(@Qualifier("oneDataSource") DataSource dataSource) { return new DataSourceTransactionManager(dataSource); } @Bean(name = "oneSqlSessionTemplate") @Primary public SqlSessionTemplate customSqlSessionTemplate(@Qualifier("oneSqlSessionFactory") SqlSessionFactory sqlSessionFactory) throws Exception { return new SqlSessionTemplate(sqlSessionFactory); } }
第二个数据源的配置:
@Configuration @MapperScan(basePackages = "com.yhyr.mybatis.mapper.StudentMapper", sqlSessionTemplateRef = "anotherSqlSessionTemplate") public class MybatisConfig2 { @Bean(name = "anotherDataSource") @ConfigurationProperties(prefix = "spring.datasource.another") public DataSource customDataSource() { return DataSourceBuilder.create().build(); } @Bean(name = "anotherSqlSessionFactory") public SqlSessionFactory customSqlSessionFactory(@Qualifier("anotherDataSource") DataSource dataSource) throws Exception { SqlSessionFactoryBean bean = new SqlSessionFactoryBean(); bean.setDataSource(dataSource); bean.setMapperLocations(new PathMatchingResourcePatternResolver().getResources("classpath*:mapper/*.xml")); return bean.getObject(); } @Bean(name = "anotherTransactionManager") public DataSourceTransactionManager customTransactionManager(@Qualifier("anotherDataSource") DataSource dataSource) { return new DataSourceTransactionManager(dataSource); } @Bean(name = "anotherSqlSessionTemplate") public SqlSessionTemplate customSqlSessionTemplate(@Qualifier("anotherSqlSessionFactory") SqlSessionFactory sqlSessionFactory) throws Exception { return new SqlSessionTemplate(sqlSessionFactory); } }
完成配置文件的配置后,可在工程目录的mapper包下新建两个目录:UserMapper和StudentMapper,分别对应两个数据源。这两个目录只能同级,或者不同目录,不能互为子父目录。
通过mapper接口和注解方式实现对数据的获取,代码如下:
@Select("SELECT * FROM user_info") public interface UserInfoMapper { List<UserInfo> getUserInfo(); }
@Select("SELECT * FROM student_info") public interface StudentInfoMapper { List<StudentInfo> getStudentInfo(); }
Service层的逻辑:分别注入UserInfoMapper 和 StudentInfoMapper,获取用户和学生信息;
@SpringBootApplication public class MybatisApplication implements CommandLineRunner { @Autowired UserService userService; @Autowired StudentService studentService; public static void main(String[] args) { SpringApplication.run(MybatisApplication.class, args); } @Override public void run(String... strings) { List<UserInfo> userInfoList = userService.getUserInfo(); userInfoList.stream().forEach(userInfo -> System.out.println("name is : " + userInfo.getName() + "; sex is : " + userInfo.getSex() + "; age is : " + userInfo.getAge())); List<StudentInfo> studentInfoList = studentService.getStudentInfo(); studentInfoList.stream().forEach(studentInfo -> System.out.println("studentName is : " + studentInfo.getStudentName() + "; className is : " + studentInfo.getClassName())); } }
最后贴上入口函数的逻辑和运行结果:
动态数据源切换
业务场景:
现有已知的两个数据源:default和master;
default:用户常规的业务逻辑,(eg:单数据源的业务需求)
master:该数据源内只有一个db_info表,该表内维护这数据库的基本信息(dbName, dbIp, dbPort, dbUser, dbPasswd)
现在需要根据业务需求,获取master中相应的数据库基本信息,然后根据从获取到的数据库基本信息中获取所需要的业务数据。(可类比Hadoop中的NameNode和DataNode的关系)
在这种业务场景下,上述那种在程序执行前就一次性初始化所有mybatis实例的方法就行不通了。所以可以借助如下思路来思考:
基于这种方式,不仅可是实现真正意义上的多数据源的切换(第二种实现多数据源切换的思路),还可以实现在程序的运行过程中,实现动态添加一个或多个新的数据源。这里重点关注的是配置文件之间的关系,对象模型如下:
首先分析一下AbstractRoutingDataSource抽象类的源码:
public abstract class AbstractRoutingDataSource extends AbstractDataSource implements InitializingBean { private Map<Object, Object> targetDataSources; private Object defaultTargetDataSource; private boolean lenientFallback = true; private DataSourceLookup dataSourceLookup = new JndiDataSourceLookup(); private Map<Object, DataSource> resolvedDataSources; private DataSource resolvedDefaultDataSource; public AbstractRoutingDataSource() { } public void setTargetDataSources(Map<Object, Object> targetDataSources) { this.targetDataSources = targetDataSources; } public void setDefaultTargetDataSource(Object defaultTargetDataSource) { this.defaultTargetDataSource = defaultTargetDataSource; } public void setLenientFallback(boolean lenientFallback) { this.lenientFallback = lenientFallback; } public void setDataSourceLookup(DataSourceLookup dataSourceLookup) { this.dataSourceLookup = (DataSourceLookup)(dataSourceLookup != null ? dataSourceLookup : new JndiDataSourceLookup()); } public void afterPropertiesSet() { if (this.targetDataSources == null) { throw new IllegalArgumentException("Property 'targetDataSources' is required"); } else { this.resolvedDataSources = new HashMap(this.targetDataSources.size()); Iterator var1 = this.targetDataSources.entrySet().iterator(); while(var1.hasNext()) { Entry<Object, Object> entry = (Entry)var1.next(); Object lookupKey = this.resolveSpecifiedLookupKey(entry.getKey()); DataSource dataSource = this.resolveSpecifiedDataSource(entry.getValue()); this.resolvedDataSources.put(lookupKey, dataSource); } if (this.defaultTargetDataSource != null) { this.resolvedDefaultDataSource = this.resolveSpecifiedDataSource(this.defaultTargetDataSource); } } } protected Object resolveSpecifiedLookupKey(Object lookupKey) { return lookupKey; } protected DataSource resolveSpecifiedDataSource(Object dataSource) throws IllegalArgumentException { if (dataSource instanceof DataSource) { return (DataSource)dataSource; } else if (dataSource instanceof String) { return this.dataSourceLookup.getDataSource((String)dataSource); } else { throw new IllegalArgumentException("Illegal data source value - only [javax.sql.DataSource] and String supported: " + dataSource); } } public Connection getConnection() throws SQLException { return this.determineTargetDataSource().getConnection(); } public Connection getConnection(String username, String password) throws SQLException { return this.determineTargetDataSource().getConnection(username, password); } public <T> T unwrap(Class<T> iface) throws SQLException { return iface.isInstance(this) ? this : this.determineTargetDataSource().unwrap(iface); } public boolean isWrapperFor(Class<?> iface) throws SQLException { return iface.isInstance(this) || this.determineTargetDataSource().isWrapperFor(iface); } protected DataSource determineTargetDataSource() { Assert.notNull(this.resolvedDataSources, "DataSource router not initialized"); Object lookupKey = this.determineCurrentLookupKey(); DataSource dataSource = (DataSource)this.resolvedDataSources.get(lookupKey); if (dataSource == null && (this.lenientFallback || lookupKey == null)) { dataSource = this.resolvedDefaultDataSource; } if (dataSource == null) { throw new IllegalStateException("Cannot determine target DataSource for lookup key [" + lookupKey + "]"); } else { return dataSource; } } protected abstract Object determineCurrentLookupKey(); }
对于该抽象类,关注两组变量:Map<Object, Object> targetDataSources和Object defaultTargetDataSource、Map<Object, DataSource> resolvedDataSources和DataSource resolvedDefaultDataSource;这两组变量是相互对应的;在熟悉多实例数据源切换代码的不难发现,当有多个数据源的时候,一定要指定一个作为默认的数据源,在这里也同理,当同时初始化多个数据源的时候,需要显示的调用setDefaultTargetDataSource方法指定一个作为默认数据源;
我们需要关注的是Map<Object, Object> targetDataSources和Map<Object, DataSource> resolvedDataSources,targetDataSources是暴露给外部程序用来赋值的,而resolvedDataSources是程序内部执行时的依据,因此会有一个赋值的操作,如下图所示:
根据这段源码可以看出,每次执行时,都会遍历targetDataSources内的所有元素并赋值给resolvedDataSources;这样如果我们在外部程序新增一个新的数据源,都会添加到内部使用,从而实现数据源的动态加载。
继承该抽象类的时候,必须实现一个抽象方法:protected abstract Object determineCurrentLookupKey(),该方法用于指定到底需要使用哪一个数据源。
到此基本上清楚了该抽象类的使用方法,接下来贴下具体的实现代码:
自定义数据源DataSource类:
public class DynamicDataSource extends AbstractRoutingDataSource { private static DynamicDataSource instance; private static byte[] lock=new byte[0]; private static Map<Object,Object> dataSourceMap=new HashMap<Object, Object>(); @Override public void setTargetDataSources(Map<Object, Object> targetDataSources) { super.setTargetDataSources(targetDataSources); dataSourceMap.putAll(targetDataSources); super.afterPropertiesSet();// 必须添加该句,否则新添加数据源无法识别到 } public Map<Object, Object> getDataSourceMap() { return dataSourceMap; } public static synchronized DynamicDataSource getInstance(){ if(instance==null){ synchronized (lock){ if(instance==null){ instance=new DynamicDataSource(); } } } return instance; } //必须实现其方法 protected Object determineCurrentLookupKey() { return DataSourceContextHolder.getDBType(); } }
通过ThreadLocal维护一个全局唯一的map来实现数据源的动态切换
public class DataSourceContextHolder { private static final ThreadLocal<String> contextHolder = new ThreadLocal<String>(); public static synchronized void setDBType(String dbType){ contextHolder.set(dbType); } public static String getDBType(){ return contextHolder.get(); } public static void clearDBType(){ contextHolder.remove(); } }
Mybatis配置文件:
@Configuration public class DataSourceConfig { @Value("${spring.datasource.default.url}") private String defaultDBUrl; @Value("${spring.datasource.default.username}") private String defaultDBUser; @Value("${spring.datasource.default.password}") private String defaultDBPassword; @Value("${spring.datasource.default.driver-class-name}") private String defaultDBDreiverName; @Value("${spring.datasource.master.url}") private String masterDBUrl; @Value("${spring.datasource.master.username}") private String masterDBUser; @Value("${spring.datasource.master.password}") private String masterDBPassword; @Value("${spring.datasource.default.driver-class-name}") private String masterDBDreiverName; @Bean public DynamicDataSource dynamicDataSource() { DynamicDataSource dynamicDataSource = DynamicDataSource.getInstance(); DruidDataSource defaultDataSource = new DruidDataSource(); defaultDataSource.setUrl(defaultDBUrl); defaultDataSource.setUsername(defaultDBUser); defaultDataSource.setPassword(defaultDBPassword); defaultDataSource.setDriverClassName(defaultDBDreiverName); DruidDataSource masterDataSource = new DruidDataSource(); masterDataSource.setDriverClassName(masterDBDreiverName); masterDataSource.setUrl(masterDBUrl); masterDataSource.setUsername(masterDBUser); masterDataSource.setPassword(masterDBPassword); Map<Object,Object> map = new HashMap<>(); map.put("default", defaultDataSource); map.put("master", masterDataSource); dynamicDataSource.setTargetDataSources(map); dynamicDataSource.setDefaultTargetDataSource(defaultDataSource); return dynamicDataSource; } @Bean public SqlSessionFactory sqlSessionFactory( @Qualifier("dynamicDataSource") DataSource dynamicDataSource) throws Exception { SqlSessionFactoryBean bean = new SqlSessionFactoryBean(); bean.setDataSource(dynamicDataSource); bean.setMapperLocations(new PathMatchingResourcePatternResolver() .getResources("classpath*:mapper/*.xml")); return bean.getObject(); } @Bean(name = "sqlSessionTemplate") public SqlSessionTemplate sqlSessionTemplate( @Qualifier("sqlSessionFactory") SqlSessionFactory sqlSessionFactory) throws Exception { return new SqlSessionTemplate(sqlSessionFactory); } }
其他业务逻辑同多数据源切换,下面贴上如何切换数据源:
@SpringBootApplication public class DynamicApplication implements CommandLineRunner { @Autowired UserService userService; @Autowired DBService dbService; @Autowired StudentService studentService; public static void main(String[] args) { SpringApplication.run(DynamicApplication.class, args); } @Override public void run(String... strings) throws Exception { /** * 获取maste数据库信息 */ DataSourceContextHolder.setDBType("default"); List<UserInfo> userInfoList = userService.getUserInfo(); userInfoList.stream().forEach(userInfo -> System.out.println("name is : " + userInfo.getName() + "; sex is : " + userInfo.getSex() + "; age is : " + userInfo.getAge())); /** * 根据slave数据源获取目标数据库信息 */ DataSourceContextHolder.setDBType("master"); int primayrId = 1; DBInfo dbInfo = dbService.getDBInfoByprimayrId(primayrId); System.out.println("dbName is -> " + dbInfo.getDbName() + "; dbIP is -> " + dbInfo.getDbIp() + "; dbUser is -> " + dbInfo.getDbUser() + "; dbPasswd is -> " + dbInfo.getDbPasswd()); DruidDataSource dynamicDataSource = new DruidDataSource(); dynamicDataSource.setDriverClassName("com.mysql.jdbc.Driver"); dynamicDataSource.setUrl("jdbc:mysql://" + dbInfo.getDbIp() + ":" + dbInfo.getDbPort() + "/" + dbInfo.getDbName() + "?characterEncoding=utf-8&zeroDateTimeBehavior=convertToNull"); dynamicDataSource.setUsername(dbInfo.getDbUser()); dynamicDataSource.setPassword(dbInfo.getDbPasswd()); /** * 创建动态数据源 */ Map<Object, Object> dataSourceMap = DynamicDataSource.getInstance().getDataSourceMap(); dataSourceMap.put("dynamic-slave", dynamicDataSource); DynamicDataSource.getInstance().setTargetDataSources(dataSourceMap); /** * 切换为动态数据源实例,打印学生信息 */ DataSourceContextHolder.setDBType("dynamic-slave"); List<StudentInfo> studentInfoList = studentService.getStudentInfo(); studentInfoList.stream().forEach(studentInfo -> System.out.println("studentName is : " + studentInfo.getStudentName() + "; className is : " + studentInfo.getClassName() + "; gradeName is : " + studentInfo.getGradeName())); } }
这种是在业务中使用代码设置数据源的方式,也可以使用AOP+注解的方式实现控制,还可以前端头部设置后端通过拦截器统一设置!