问题详情
[hadoop@master bin]$ ./azkaban-web-start.sh Using Hadoop from /home/hadoop/app/hadoop-2.6.0 Using Hive from /home/hadoop/app/hive ./.. /home/hadoop/app/jdk/lib:.:/home/hadoop/app/jdk/lib:/home/hadoop/app/jdk/jre/lib:./../lib/azkaban-2.5.0.jar:./../lib/commons-collections-3.2.1.jar:./../lib/commons-configuration-1.8.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-email-1.2.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.1.jar:./../lib/commons-pool-1.6.jar:./../lib/guava-13.0.1.jar:./../lib/h2-1.3.170.jar:./../lib/httpclient-4.2.1.jar:./../lib/httpcore-4.2.1.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/junit-4.8.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.6.1.jar:./../lib/slf4j-log4j12-1.6.4.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/mysql-connector-java-5.1.21.jar:./../plugins/*/*.jar:/home/hadoop/app/hadoop-2.6.0/conf:/home/hadoop/app/hadoop-2.6.0/*:/home/hadoop/app/hive/conf:/home/hadoop/app/hive/lib/* [hadoop@master bin]$ 2017/07/26 09:06:39.979 +0800 ERROR [AzkabanWebServer] [Azkaban] Starting Jetty Azkaban Executor... 2017/07/26 09:06:40.098 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from ./../conf 2017/07/26 09:06:40.100 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file 2017/07/26 09:06:40.190 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up Jetty Https Server with port:8443 and numThreads:25 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/hadoop/app/azkaban/azkaban-web-2.5.0/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/hadoop/app/hive-1.0.0/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2017/07/26 09:06:40.484 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2017/07/26 09:06:41.013 +0800 INFO [AzkabanWebServer] [Azkaban] Loading user manager class azkaban.user.XmlUserManager 2017/07/26 09:06:41.477 +0800 INFO [XmlUserManager] [Azkaban] Loading user azkaban 2017/07/26 09:06:41.484 +0800 INFO [XmlUserManager] [Azkaban] Loading user metrics 2017/07/26 09:06:41.485 +0800 INFO [XmlUserManager] [Azkaban] Loading user admin Exception in thread "main" azkaban.executor.ExecutorManagerException: Error fetching active flows at azkaban.executor.JdbcExecutorLoader.fetchActiveFlows(JdbcExecutorLoader.java:209) at azkaban.executor.ExecutorManager.loadRunningFlows(ExecutorManager.java:146) at azkaban.executor.ExecutorManager.<init>(ExecutorManager.java:84) at azkaban.webapp.AzkabanWebServer.loadExecutorManager(AzkabanWebServer.java:248) at azkaban.webapp.AzkabanWebServer.<init>(AzkabanWebServer.java:185) at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:726) Caused by: java.sql.SQLException: Table 'execution_flows' is marked as crashed and should be repaired Query: SELECT ex.exec_id exec_id, ex.enc_type enc_type, ex.flow_data flow_data, ax.host host, ax.port port, ax.update_time axUpdateTime FROM execution_flows ex INNER JOIN active_executing_flows ax ON ex.exec_id = ax.exec_id Parameters: [] at org.apache.commons.dbutils.AbstractQueryRunner.rethrow(AbstractQueryRunner.java:363) at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:350) at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:306) at azkaban.executor.JdbcExecutorLoader.fetchActiveFlows(JdbcExecutorLoader.java:202) ... 5 more
问题分析
错误产生原因，有网友说是频繁查询和更新dede_archives表造成的索引错误，因为我的页面没有静态生成，而是动态页面，因此比较同意这种说法。
还有说法为是MYSQL数据库因为某种原因而受到了损坏，如：数据库服务器突发性的断电、在提在数据库表提供服务时对表的原文件进行某种操作都有可能导致
MYSQL数据库表被损坏而无法读取数据。总之就是因为某些不可测的问题造成表的损坏。
我这里，是在跑azkaban任务的时候，间隔取的太小，同时自己电脑配置不太高，导致，速度很慢和卡顿。我中途关闭了导致。
解决办法
[hadoop@master ~]$ mysql -uroot -prootroot Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 24 Server version: 5.1.73 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | azkaban | | hive | | mysql | | test | +--------------------+ 5 rows in set (0.09 sec) mysql> use azkaban Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +------------------------+ | Tables_in_azkaban | +------------------------+ | active_executing_flows | | active_sla | | execution_flows | | execution_jobs | | execution_logs | | project_events | | project_files | | project_flows | | project_permissions | | project_properties | | project_versions | | projects | | properties | | schedules | | triggers | +------------------------+ 15 rows in set (0.00 sec) mysql>
然后，再，检测下这个表execution_flows表的状态情况，如下出现错误的状态是正确的，本来这个表就已经是不健康的了。
mysql> check table execution_flows; +-------------------------+-------+----------+----------------------------------------------------------+ | Table | Op | Msg_type | Msg_text | +-------------------------+-------+----------+----------------------------------------------------------+ | azkaban.execution_flows | check | warning | 6 clients are using or haven't closed the table properly | | azkaban.execution_flows | check | error | Wrong bytesec: 40-0-0 at linkstart: 53868 | | azkaban.execution_flows | check | error | Corrupt | +-------------------------+-------+----------+----------------------------------------------------------+ 3 rows in set (0.40 sec)
执行，修复
mysql> repair table execution_flows; +-------------------------+--------+----------+-------------------------------------------------------+ | Table | Op | Msg_type | Msg_text | +-------------------------+--------+----------+-------------------------------------------------------+ | azkaban.execution_flows | repair | info | Wrong bytesec: 40- 0- 0 at 53868; Skipped | | azkaban.execution_flows | repair | info | Found block that points outside data file at 49260 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 49472 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 49476 | | azkaban.execution_flows | repair | info | Found block with too small length at 49628; Skipped | | azkaban.execution_flows | repair | info | Found block that points outside data file at 49812 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 49824 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 49876 | | azkaban.execution_flows | repair | info | Wrong block with wrong total length starting at 49996 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 50024 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 50100 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 50120 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 50128 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 50304 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 50588 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 50772 | | azkaban.execution_flows | repair | info | Found block that points outside data file at 51380 | | azkaban.execution_flows | repair | warning | Number of rows changed from 74 to 71 | | azkaban.execution_flows | repair | status | OK | +-------------------------+--------+----------+-------------------------------------------------------+ 19 rows in set (0.36 sec)
mysql> check table execution_flows; +-------------------------+-------+----------+----------+ | Table | Op | Msg_type | Msg_text | +-------------------------+-------+----------+----------+ | azkaban.execution_flows | check | status | OK | +-------------------------+-------+----------+----------+ 1 row in set (0.00 sec)
然后，再
再
[hadoop@master bin]$ ./azkaban-web-start.sh Using Hadoop from /home/hadoop/app/hadoop-2.6.0 Using Hive from /home/hadoop/app/hive ./.. /home/hadoop/app/jdk/lib:.:/home/hadoop/app/jdk/lib:/home/hadoop/app/jdk/jre/lib:./../lib/azkaban-2.5.0.jar:./../lib/commons-collections-3.2.1.jar:./../lib/commons-configuration-1.8.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-email-1.2.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.1.jar:./../lib/commons-pool-1.6.jar:./../lib/guava-13.0.1.jar:./../lib/h2-1.3.170.jar:./../lib/httpclient-4.2.1.jar:./../lib/httpcore-4.2.1.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/junit-4.8.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.6.1.jar:./../lib/slf4j-log4j12-1.6.4.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/mysql-connector-java-5.1.21.jar:./../plugins/*/*.jar:/home/hadoop/app/hadoop-2.6.0/conf:/home/hadoop/app/hadoop-2.6.0/*:/home/hadoop/app/hive/conf:/home/hadoop/app/hive/lib/* [hadoop@master bin]$ 2017/07/26 09:38:33.582 +0800 ERROR [AzkabanWebServer] [Azkaban] Starting Jetty Azkaban Executor... 2017/07/26 09:38:33.726 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from ./../conf 2017/07/26 09:38:33.748 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file 2017/07/26 09:38:33.865 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up Jetty Https Server with port:8443 and numThreads:25 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/hadoop/app/azkaban/azkaban-web-2.5.0/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/hadoop/app/hive-1.0.0/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2017/07/26 09:38:34.222 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2017/07/26 09:38:34.876 +0800 INFO [AzkabanWebServer] [Azkaban] Loading user manager class azkaban.user.XmlUserManager 2017/07/26 09:38:35.260 +0800 INFO [XmlUserManager] [Azkaban] Loading user azkaban 2017/07/26 09:38:35.274 +0800 INFO [XmlUserManager] [Azkaban] Loading user metrics 2017/07/26 09:38:35.278 +0800 INFO [XmlUserManager] [Azkaban] Loading user admin 2017/07/26 09:38:38.564 +0800 INFO [AzkabanWebServer] [Azkaban] Loading JDBC for project management 2017/07/26 09:38:38.569 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old logs from execution_logs 2017/07/26 09:38:38.590 +0800 INFO [ProjectManager] [Azkaban] Project version retention is set to 3 2017/07/26 09:38:39.402 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old log files before 2017-05-03T09:38:38.659+08:00 2017/07/26 09:38:39.532 +0800 INFO [TriggerManager] [Azkaban] TriggerManager loaded. 2017/07/26 09:38:39.532 +0800 INFO [AzkabanWebServer] [Azkaban] Loading built-in checker and action types 2017/07/26 09:38:39.560 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker BasicTimeChecker 2017/07/26 09:38:39.560 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker SlaChecker 2017/07/26 09:38:39.561 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker ExecutionChecker
扩展
网上还有说，
Table '' is marked as crashed and should be repaired 解决方法
解决方法：
找到mysql的安装目录的bin/myisamchk工具，在命令行中输入：
myisamchk -c -r ../data/mysql/user.MYI
然后myisamchk 工具会帮助你恢复数据表的索引。重新启动mysql，问题解决。
这种方法，我没尝试过。
