has是高可用服务，用在restart环境，可以是单机环境也可以是RAC环境，不管单机环境还是RAC环境，都可以执行crsctl check has，在RAC环境下，可以简单理解成cluster的一部分
ohasd是has后台进程的概念
crs是集群就绪服务，用在RAC环境，单机环境不能执行crsctl check crs，crsctl start CRS不单单启动CRSD了，等于启动OHASD、CRSD、CSSD三者了
cluster就是集群的意思，用在RAC环境，单机环境不能执行crsctl check cluster
单机环境无法执行crsctl check crs、crsctl check cluster，但是可以执行crsctl stat res、crsctl check css、crsctl check has
Oracle high availability services(HAS)
Oracle Restart uses Oracle high availability services to start and stop automatically the components managed by Oracle Restart. For example, Oracle high availability services daemons automatically start databases, listeners, and Oracle ASM instances. When Oracle high availability services are disabled, none of the components managed by Oracle Restart are started when a node is rebooted.
Oracle Restart使用Oracle高可用性服务来自动启动和停止由Oracle Restart管理的组件。 例如，Oracle高可用性服务守护进程自动启动数据库，侦听器和Oracle ASM实例。 当Oracle高可用性服务被禁用时，重启节点时将不启动Oracle Restart管理的组件。
crsctl command has
where command is a verb such as start, stop, or enable. The has object indicates Oracle high availability services
check Displays the Oracle Restart status.
config Displays the Oracle Restart configuration.
disable Disables automatic restart of Oracle Restart.
enable Enables automatic restart of Oracle Restart.
start Starts Oracle Restart.
stop Stops Oracle Restart.
Oracle High Availability Services Daemon(OHASD)
This process anchors the lower part of the Oracle Clusterware stack, which consists of processes that facilitate cluster operations.
Oracle Restart improves the availability of your Oracle database. When you install OracleRestart, various Oracle components can be automatically restarted after ahardware or software failure or whenever your database host computer restarts.
此过程将固定Oracle Clusterware堆栈的较低部分，该堆栈由促进群集操作的进程组成。
Oracle Restart 能提高数据库的可用性，当安装了Oracle Restart 之后，在系统出现硬件或者软件问题，或者主机重启之后，OracleRestart管理的组件都能自动的进行启动。
OHASD是一个后台的守护进程，其可用来启动和监控OracleRestart进程。该进程由/etc/init.d/ohasd 脚本来初始化，并由root用户来执行ohasd.bin来启动，如下
[root@pdba ~]# ps -ef|grep has|grep -v grep
root 5281 1 0 Jun25 ? 14:28:09 /u01/app/11.2.0/grid/bin/ohasd.bin reboot
root 13261 1 0 Jun25 ? 00:00:00 /bin/sh /etc/init.d/init.ohasd run
OHASD无法kill，一旦kill，立马又会被拉起了（参考/etc/initab中的respawn含义）
[root@TEST ~]# cat /etc/inittab|grep init.ohasd
h1:35:respawn:/etc/init.d/init.ohasd run >/dev/null 2>&1
Note: Oracle Linux 6 (OL6) or Red Hat Linux 6 (RHEL6) has deprecated inittab, rather, init.ohasd will be configured via upstart in /etc/init/oracle-ohasd.conf, however, the process ""/etc/init.d/init.ohasd run" should still be up. Oracle Linux 7 (and Red Hat Linux 7) uses systemd to manage start/stop services (example: /etc/systemd/system/oracle-ohasd.service)
注意：Oracle Linux 6（OL6）或Red Hat Linux 6（RHEL6）已经弃用了inittab，相反，init.ohasd将通过/etc/init/oracle-ohasd.conf中的upstart配置，但是，进程“/etc/init.d/init.ohasd run“仍然应该启动，Oracle Linux 7（和Red Hat Linux 7）使用systemd来管理启动/停止服务（例如：/etc/systemd/system/oracle-ohasd.service）
[root@pdba ~]# cat /etc/issue
CentOS release 6.8 (Final)
Kernel \r on an \m
[root@pdba ~]# cat /etc/init/oracle-ohasd.conf
# Copyright (c) 2001, 2011, Oracle and/or its affiliates. All rights reserved.
#
# Oracle OHASD startup
start on runlevel [35]
stop on runlevel [!35]
respawn
exec /etc/init.d/init.ohasd run >/dev/null 2>&1
对于standalone server，使用OHASD来管理Oracle Restart，并且不需要CRSD进程的支持。可以使用OHASD管理的组件如下：
1.CSSD: This is used for Group Services as it was inprevious releases (when it was installed using “localconfig add“)
2.ASM Instance :if Automatic Storage Management is used.
3.ASM Disk Groups: if Automatic Storage Management is used.
4.Listeners
5.Database Instances
6.Database Services
7.ONS/EONS :Used for automatic failover of connections usingFast Application Notification (FAN) in a Data Guard environment
Cluster Ready Services (CRS)
Cluster Ready Services Daemon (CRSD)
The primary Oracle Clusterware process that performs high availability recovery and management operations, such as maintaining OCR. Also manages application resources and runs as root user (or by a user in the admin group on Mac OS X-based systems) and restarts automatically upon failure.
执行高可用性恢复和管理操作（如维护OCR）的主要Oracle Clusterware进程。 还管理应用程序资源并以root用户（或基于Mac OS X的系统上的管理员组中的用户）身份运行，并在失败时自动重新启动。
[root@pdb02 ~]# ps -ef|grep crs|grep -v grep
root 3672 1 0 13:15 ? 00:00:48 /u01/app/11.2.0/grid/bin/crsd.bin reboot
Cluster
Oracle Clusterware enables servers to communicate with each other, so that they appear to function as a collective unit. This combination of servers is commonly known as a cluster. Although the servers are standalone servers, each server has additional processes that communicate with other servers. In this way the separate servers appear as if they are one system to applications and end users.
Oracle集群件使服务器能够相互通信，以使它们看起来像是一个集体单元。 这种服务器组合通常被称为集群。 虽然服务器是独立的服务器，但每台服务器都有其他的进程与其他服务器通信。 通过这种方式，独立服务器看起来好像是应用程序和最终用户的一个系统。
CRS
crsctl start crs
Use the crsctl start crs command to start Oracle High Availability Services on the local server.
crsctl check crs
Use the crsctl check crs command to check the status of Oracle High Availability Services and the Oracle Clusterware stack on the local server.
crsctl stop crs
Use the crsctl stop crs command to stop Oracle High Availability Services on the local server.
cluster
crsctl start cluster
Use the crsctl start cluster command on any node in the cluster to start the Oracle Clusterware stack.
crsctl check cluster
Use the crsctl check cluster command on any node in the cluster to check the status of the Oracle Clusterware stack.
crsctl stop cluster
Use the crsctl stop cluster command on any node in the cluster to stop the Oracle Clusterware stack on all servers in the cluster or specific servers.
has
crsctl start has
Use the crsctl start has command to start Oracle High Availability Services on the local server
crsctl check has
Use the crsctl check has command to check the status of ohasd.
crsctl stop has
Use the crsctl stop has command to stop Oracle High Availability Services on the local server
RAC的一台节点执行的结果
[root@pdb02 ~]# /u01/app/11.2.0/grid/bin/crsctl config has
CRS-4622: Oracle High Availability Services autostart is enabled.
[root@pdb02 ~]# /u01/app/11.2.0/grid/bin/crsctl config crs
CRS-4622: Oracle High Availability Services autostart is enabled.
[root@pdb02 ~]# /u01/app/11.2.0/grid/bin/crsctl config cluster
Parse error:
'cluster' is an invalid argument
[root@node1 grid]# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
----crsctl start CRS不单单启动CRSD了，等于启动OHASD、CRSD、CSSD三者了
[root@node1 grid]# crsctl check cluster
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@pdba ~]# crsctl check has
CRS-4638: Oracle High Availability Services is online
crsctl check cssd被crsctl check css取代了
[root@node1 ~]# crsctl check cssd
CRS-272: This command remains for backward compatibility only
Cluster Synchronization Services is online
crsctl check crsd被crsctl check crs取代了
[root@node1 grid]# crsctl check crsd
CRS-272: This command remains for backward compatibility only
Cluster Ready Services is online
但是没有crsctl start cssd、crsctl start crsd这样的命令
RAC的一台节点执行的结果
[root@pdb02 ~]# /u01/app/11.2.0/grid/bin/crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[root@pdb02 ~]# /u01/app/11.2.0/grid/bin/crsctl start has
CRS-4640: Oracle High Availability Services is already active
CRS-4000: Command Start failed, or completed with errors.
[root@pdb02 ~]# /u01/app/11.2.0/grid/bin/crsctl check has
CRS-4638: Oracle High Availability Services is online
[root@pdb02 ~]# /u01/app/11.2.0/grid/bin/crsctl stop has
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'pdb02'
CRS-2673: Attempting to stop 'ora.crsd' on 'pdb02'
CRS-2677: Stop of 'ora.crsd' on 'pdb02' succeeded
CRS-2673: Attempting to stop 'ora.mdnsd' on 'pdb02'
CRS-2673: Attempting to stop 'ora.ctssd' on 'pdb02'
CRS-2673: Attempting to stop 'ora.evmd' on 'pdb02'
CRS-2673: Attempting to stop 'ora.asm' on 'pdb02'
CRS-2677: Stop of 'ora.ctssd' on 'pdb02' succeeded
CRS-2677: Stop of 'ora.evmd' on 'pdb02' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'pdb02' succeeded
CRS-2677: Stop of 'ora.asm' on 'pdb02' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'pdb02'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'pdb02' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'pdb02'
CRS-2677: Stop of 'ora.cssd' on 'pdb02' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'pdb02'
CRS-2677: Stop of 'ora.crf' on 'pdb02' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'pdb02'
CRS-2677: Stop of 'ora.gipcd' on 'pdb02' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'pdb02'
CRS-2677: Stop of 'ora.gpnpd' on 'pdb02' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'pdb02' has completed
CRS-4133: Oracle High Availability Services has been stopped
[root@pdb02 ~]# /u01/app/11.2.0/grid/bin/crsctl check has
CRS-4639: Could not contact Oracle High Availability Services
[root@pdb02 ~]# /u01/app/11.2.0/grid/bin/crsctl stop has
CRS-2796: The command may not proceed when Cluster Ready Services is not running
CRS-4687: Shutdown command has completed with errors.
CRS-4000: Command Stop failed, or completed with errors.
单机执行的结果
[root@GIOSONLY app]# /u01/app/oracle/product/11.2.0/grid/bin/crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[root@GIOSONLY app]# /u01/app/oracle/product/11.2.0/grid/bin/crsctl start has
CRS-4640: Oracle High Availability Services is already active
CRS-4000: Command Start failed, or completed with errors.
[root@GIOSONLY app]# /u01/app/oracle/product/11.2.0/grid/bin/crsctl check has
CRS-4638: Oracle High Availability Services is online
[root@GIOSONLY app]# /u01/app/oracle/product/11.2.0/grid/bin/crsctl stop has
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'giosonly'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'giosonly'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'giosonly'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'giosonly' succeeded
CRS-5022: Stop of resource "ora.DATA.dg" failed: current state is "UNKNOWN"
CRS-2675: Stop of 'ora.DATA.dg' on 'giosonly' failed
CRS-2679: Attempting to clean 'ora.DATA.dg' on 'giosonly'
CRS-2681: Clean of 'ora.DATA.dg' on 'giosonly' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'giosonly'
CRS-2677: Stop of 'ora.asm' on 'giosonly' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'giosonly'
CRS-2677: Stop of 'ora.cssd' on 'giosonly' succeeded
CRS-2673: Attempting to stop 'ora.evmd' on 'giosonly'
CRS-2677: Stop of 'ora.evmd' on 'giosonly' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'giosonly' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@GIOSONLY app]# /u01/app/oracle/product/11.2.0/grid/bin/crsctl check has
CRS-4639: Could not contact Oracle High Availability Services
[root@GIOSONLY app]# /u01/app/oracle/product/11.2.0/grid/bin/crsctl stop has
CRS-4544: Unable to connect to OHAS
CRS-4000: Command Stop failed, or completed with errors.