• Mysql优化小记1


    在项目开发中,需要写个windows服务从sqlserver复制数据到mysql(5.6.13 Win64(x86_64)),然后对这些数据进行计算分析。
    每15分钟复制一次,每次复制大概200条数据,每隔1小时对新同步的数据进行计算。

    我们的mysql服务器是个破台式机,如下:(最初4G内存)

    随着时间推移,数据量不断增加,目前数据表有7295400条数据。
    眼看着同步数据耗时越来越长,从最初的10秒左右,到目前的7,8分钟,有时更甚达到十几分钟。
    计算服务耗时从最初的几分钟到目前的40多分钟。
    磁盘使用率一直在100%左右,mysql数据库的innodb buffer 使用率一直100%。

    因此赶快买了个8G内存条(原来只有4G),然后修改mysql变量:
    innodb_buffer_pool_size=2G(原innodb_buffer_pool_size=99MB);
    innodb_flush_log_at_trx_commit=0(原innodb_flush_log_at_trx_commit=1)

    效果十分明显:同步数据又回到了以前的10秒左右,计算服务现在耗时不到10分钟;

    innodb buffer usage大概在50%左右,磁盘使用率也只是在数据插入时比较高,其它时间也就0%。

    现在来看看innodb_buffer_pool_size,innodb_flush_log_at_trx_commit的注释说明:

    innodb_buffer_pool_size

    InnoDB, unlike MyISAM, uses a buffer pool to cache both indexes and
    row data. The bigger you set this the less disk I/O is needed to
    access data in tables. On a dedicated database server you may set this
    parameter up to 80% of the machine physical memory size. Do not set it
    too large, though, because competition of the physical memory may
    cause paging in the operating system. Note that on 32bit systems you
    might be limited to 2-3.5G of user level memory per process, so do not
    set it too high.

    Innodb,使用一个缓冲池来缓存索引和数据行,你设置的越大,磁盘I/O访问越小,设置值大概在数据库服务器物理内存的80%左右,

    当然也不能过高,我的设置在2G显然够用了,在32为系统中,我们一般只能用3G内存,所有不要设置太高。

    innodb_flush_log_at_trx_commit

    If set to 1, InnoDB will flush (fsync) the transaction logs to the
    disk at each commit, which offers full ACID behavior. If you are
    willing to compromise this safety, and you are running small
    transactions, you may set this to 0 or 2 to reduce disk I/O to the
    logs. Value 0 means that the log is only written to the log file and
    the log file flushed to disk approximately once per second. Value 2
    means the log is written to the log file at each commit, but the log
    file is only flushed to disk approximately once per second.

    默认值1的意思是每一次事务提交或事务外的指令都需要把日志写入(flush)硬盘,这是很费时的。特别是使用电 池供电缓存(Battery backed up cache)时。设成2对于很多运用,特别是从MyISAM表转过来的是可以的,它的意思是不写入硬盘而是写入系统缓存。日志仍然会每秒flush到硬 盘,所以你一般不会丢失超过1-2秒的更新。设成0会更快一点,但安全方面比较差,即使MySQL挂了也可能会丢失事务的数据。而值2只会在整个操作系统 挂了时才可能丢数据。

    附加:2014-10-17

    # The number of regions that the InnoDB buffer pool is divided into.
    # For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency,
    # by reducing contention as different threads read and write to cached pages.
    innodb_buffer_pool_instances=8

    当innodb_buffer_pool_instances=1时,innodb_buffer_pool_size>1G是无效的,mysql无法启动,因此当设置innodb_buffer_pool_size>1G时,

    应该设置innodb_buffer_pool_instances>1

    以上仅仅是工作中的一点点经验,也不是很具有说明性,但对于mysql查询优化还是有一点点帮助。

     

  • 相关阅读:
    Nginx02 Nginx的的目录结构、基本工作原理、基本配置文件介绍
    正向代理和反向代理
    Nginx 05 动静分离
    Nginx06 Rewrite
    Nginx07 keepalived
    Nginx01 简介和安装
    负载均衡
    Nginx04 反向代理和负载均衡
    VM安装Centos 经典安装
    linux CentOS7安装MySQL8.0.28
  • 原文地址:https://www.cnblogs.com/zhangzhi19861216/p/3732787.html
Copyright © 2020-2023  润新知