配置文件说明:
%% -*- mode: erlang -*- %% ---------------------------------------------------------------------------- %% RabbitMQ Sample Configuration File. %% %% See http://www.rabbitmq.com/configure.html for details. %% ---------------------------------------------------------------------------- [ {rabbit, [%% %% Network Connectivity %% ==================== %% %% By default, RabbitMQ will listen on all interfaces, using %% the standard (reserved) AMQP port. %% %% {tcp_listeners, [5672]}, %% To listen on a specific interface, provide a tuple of {IpAddress, Port}. %% For example, to listen only on localhost for both IPv4 and IPv6: %% %% {tcp_listeners, [{"127.0.0.1", 5672}, %% {"::1", 5672}]}, %% SSL listeners are configured in the same fashion as TCP listeners, %% including the option to control the choice of interface. %% %% {ssl_listeners, [5671]}, %% Number of Erlang processes that will accept connections for the TCP %% and SSL listeners. %% %% {num_tcp_acceptors, 10}, %% {num_ssl_acceptors, 1}, %% Maximum time for AMQP 0-8/0-9/0-9-1 handshake (after socket connection %% and SSL handshake), in milliseconds. %% %% {handshake_timeout, 10000}, %% Log levels (currently just used for connection logging). %% One of 'debug', 'info', 'warning', 'error' or 'none', in decreasing %% order of verbosity. Defaults to 'info'. %% %% {log_levels, [{connection, info}, {channel, info}]}, %% Set to 'true' to perform reverse DNS lookups when accepting a %% connection. Hostnames will then be shown instead of IP addresses %% in rabbitmqctl and the management plugin. %% %% {reverse_dns_lookups, true}, %% %% Security / AAA %% ============== %% %% The default "guest" user is only permitted to access the server %% via a loopback interface (e.g. localhost). %% {loopback_users, [<<"guest">>]}, %% %% Uncomment the following line if you want to allow access to the %% guest user from anywhere on the network. %% {loopback_users, []}, %% Configuring SSL. %% See http://www.rabbitmq.com/ssl.html for full documentation. %% %% {ssl_options, [{cacertfile, "/path/to/testca/cacert.pem"}, %% {certfile, "/path/to/server/cert.pem"}, %% {keyfile, "/path/to/server/key.pem"}, %% {verify, verify_peer}, %% {fail_if_no_peer_cert, false}]}, %% Choose the available SASL mechanism(s) to expose. %% The two default (built in) mechanisms are 'PLAIN' and %% 'AMQPLAIN'. Additional mechanisms can be added via %% plugins. %% %% See http://www.rabbitmq.com/authentication.html for more details. %% %% {auth_mechanisms, ['PLAIN', 'AMQPLAIN']}, %% Select an authentication database to use. RabbitMQ comes bundled %% with a built-in auth-database, based on mnesia. %% %% {auth_backends, [rabbit_auth_backend_internal]}, %% Configurations supporting the rabbitmq_auth_mechanism_ssl and %% rabbitmq_auth_backend_ldap plugins. %% %% NB: These options require that the relevant plugin is enabled. %% See http://www.rabbitmq.com/plugins.html for further details. %% The RabbitMQ-auth-mechanism-ssl plugin makes it possible to %% authenticate a user based on the client's SSL certificate. %% %% To use auth-mechanism-ssl, add to or replace the auth_mechanisms %% list with the entry 'EXTERNAL'. %% %% {auth_mechanisms, ['EXTERNAL']}, %% The rabbitmq_auth_backend_ldap plugin allows the broker to %% perform authentication and authorisation by deferring to an %% external LDAP server. %% %% For more information about configuring the LDAP backend, see %% http://www.rabbitmq.com/ldap.html. %% %% Enable the LDAP auth backend by adding to or replacing the %% auth_backends entry: %% %% {auth_backends, [rabbit_auth_backend_ldap]}, %% This pertains to both the rabbitmq_auth_mechanism_ssl plugin and %% STOMP ssl_cert_login configurations. See the rabbitmq_stomp %% configuration section later in this file and the README in %% https://github.com/rabbitmq/rabbitmq-auth-mechanism-ssl for further %% details. %% %% To use the SSL cert's CN instead of its DN as the username %% %% {ssl_cert_login_from, common_name}, %% SSL handshake timeout, in milliseconds. %% %% {ssl_handshake_timeout, 5000}, %% Password hashing implementation. Will only affect newly %% created users. To recalculate hash for an existing user %% it's necessary to update her password. %% %% {password_hashing_module, rabbit_password_hashing_sha256}, %% Configuration entry encryption. %% See http://www.rabbitmq.com/configure.html#configuration-encryption %% %% To specify the passphrase in the configuration file: %% %% {config_entry_decoder, [{passphrase, <<"mypassphrase">>}]} %% %% To specify the passphrase in an external file: %% %% {config_entry_decoder, [{passphrase, {file, "/path/to/passphrase/file"}}]} %% %% To make the broker request the passphrase when it starts: %% %% {config_entry_decoder, [{passphrase, prompt}]} %% %% To change encryption settings: %% %% {config_entry_decoder, [{cipher, aes_cbc256}, %% {hash, sha512}, %% {iterations, 1000}]} %% %% Default User / VHost %% ==================== %% %% On first start RabbitMQ will create a vhost and a user. These %% config items control what gets created. See %% http://www.rabbitmq.com/access-control.html for further %% information about vhosts and access control. %% %% {default_vhost, <<"/">>}, %% {default_user, <<"guest">>}, %% {default_pass, <<"guest">>}, %% {default_permissions, [<<".*">>, <<".*">>, <<".*">>]}, %% Tags for default user %% %% For more details about tags, see the documentation for the %% Management Plugin at http://www.rabbitmq.com/management.html. %% %% {default_user_tags, [administrator]}, %% %% Additional network and protocol related configuration %% ===================================================== %% %% Set the default AMQP heartbeat delay (in seconds). %% %% {heartbeat, 60}, %% Set the max permissible size of an AMQP frame (in bytes). %% %% {frame_max, 131072}, %% Set the max frame size the server will accept before connection %% tuning occurs %% %% {initial_frame_max, 4096}, %% Set the max permissible number of channels per connection. %% 0 means "no limit". %% %% {channel_max, 128}, %% Customising Socket Options. %% %% See (http://www.erlang.org/doc/man/inet.html#setopts-2) for %% further documentation. %% %% {tcp_listen_options, [{backlog, 128}, %% {nodelay, true}, %% {exit_on_close, false}]}, %% %% Resource Limits & Flow Control %% ============================== %% %% See http://www.rabbitmq.com/memory.html for full details. %% Memory-based Flow Control threshold. %% %% {vm_memory_high_watermark, 0.4}, %% Alternatively, we can set a limit (in bytes) of RAM used by the node. %% %% {vm_memory_high_watermark, {absolute, 1073741824}}, %% %% Or you can set absolute value using memory units. %% %% {vm_memory_high_watermark, {absolute, "1024M"}}, %% %% Supported units suffixes: %% %% k, kiB: kibibytes (2^10 bytes) %% M, MiB: mebibytes (2^20) %% G, GiB: gibibytes (2^30) %% kB: kilobytes (10^3) %% MB: megabytes (10^6) %% GB: gigabytes (10^9) %% Fraction of the high watermark limit at which queues start to %% page message out to disc in order to free up memory. %% %% Values greater than 0.9 can be dangerous and should be used carefully. %% %% {vm_memory_high_watermark_paging_ratio, 0.5}, %% Interval (in milliseconds) at which we perform the check of the memory %% levels against the watermarks. %% %% {memory_monitor_interval, 2500}, %% Set disk free limit (in bytes). Once free disk space reaches this %% lower bound, a disk alarm will be set - see the documentation %% listed above for more details. %% %% {disk_free_limit, 50000000}, %% %% Or you can set it using memory units (same as in vm_memory_high_watermark) %% {disk_free_limit, "50MB"}, %% {disk_free_limit, "50000kB"}, %% {disk_free_limit, "2GB"}, %% Alternatively, we can set a limit relative to total available RAM. %% %% Values lower than 1.0 can be dangerous and should be used carefully. %% {disk_free_limit, {mem_relative, 2.0}}, %% %% Misc/Advanced Options %% ===================== %% %% NB: Change these only if you understand what you are doing! %% %% To announce custom properties to clients on connection: %% %% {server_properties, []}, %% How to respond to cluster partitions. %% See http://www.rabbitmq.com/partitions.html for further details. %% %% {cluster_partition_handling, ignore}, %% Make clustering happen *automatically* at startup - only applied %% to nodes that have just been reset or started for the first time. %% See http://www.rabbitmq.com/clustering.html#auto-config for %% further details. %% %% {cluster_nodes, {['rabbit@my.host.com'], disc}}, %% Interval (in milliseconds) at which we send keepalive messages %% to other cluster members. Note that this is not the same thing %% as net_ticktime; missed keepalive messages will not cause nodes %% to be considered down. %% %% {cluster_keepalive_interval, 10000}, %% Set (internal) statistics collection granularity. %% %% {collect_statistics, none}, %% Statistics collection interval (in milliseconds). %% %% {collect_statistics_interval, 5000}, %% Explicitly enable/disable hipe compilation. %% %% {hipe_compile, true}, %% Number of times to retry while waiting for Mnesia tables in a cluster to %% become available. %% %% {mnesia_table_loading_retry_limit, 10}, %% Time to wait per retry for Mnesia tables in a cluster to become %% available. %% %% {mnesia_table_loading_retry_timeout, 30000}, %% Size in bytes below which to embed messages in the queue index. See %% http://www.rabbitmq.com/persistence-conf.html %% %% {queue_index_embed_msgs_below, 4096}, %% Whether or not to enable background GC. %% %% {background_gc_enabled, true}, %% %% Interval (in milliseconds) at which we run background GC. %% %% {background_gc_target_interval, 60000} ]}, %% ---------------------------------------------------------------------------- %% Advanced Erlang Networking/Clustering Options. %% %% See http://www.rabbitmq.com/clustering.html for details %% ---------------------------------------------------------------------------- {kernel, [%% Sets the net_kernel tick time. %% Please see http://erlang.org/doc/man/kernel_app.html and %% http://www.rabbitmq.com/nettick.html for further details. %% %% {net_ticktime, 60} ]}, %% ---------------------------------------------------------------------------- %% RabbitMQ Management Plugin %% %% See http://www.rabbitmq.com/management.html for details %% ---------------------------------------------------------------------------- {rabbitmq_management, [%% Pre-Load schema definitions from the following JSON file. See %% http://www.rabbitmq.com/management.html#load-definitions %% %% {load_definitions, "/path/to/schema.json"}, %% Log all requests to the management HTTP API to a file. %% %% {http_log_dir, "/path/to/access.log"}, %% Change the port on which the HTTP listener listens, %% specifying an interface for the web server to bind to. %% Also set the listener to use SSL and provide SSL options. %% %% {listener, [{port, 12345}, %% {ip, "127.0.0.1"}, %% {ssl, true}, %% {ssl_opts, [{cacertfile, "/path/to/cacert.pem"}, %% {certfile, "/path/to/cert.pem"}, %% {keyfile, "/path/to/key.pem"}]}]}, %% One of 'basic', 'detailed' or 'none'. See %% http://www.rabbitmq.com/management.html#fine-stats for more details. %% {rates_mode, basic}, %% Configure how long aggregated data (such as message rates and queue %% lengths) is retained. Please read the plugin's documentation in %% http://www.rabbitmq.com/management.html#configuration for more %% details. %% %% {sample_retention_policies, %% [{global, [{60, 5}, {3600, 60}, {86400, 1200}]}, %% {basic, [{60, 5}, {3600, 60}]}, %% {detailed, [{10, 5}]}]} ]}, %% ---------------------------------------------------------------------------- %% RabbitMQ Shovel Plugin %% %% See http://www.rabbitmq.com/shovel.html for details %% ---------------------------------------------------------------------------- {rabbitmq_shovel, [{shovels, [%% A named shovel worker. %% {my_first_shovel, %% [ %% List the source broker(s) from which to consume. %% %% {sources, %% [%% URI(s) and pre-declarations for all source broker(s). %% {brokers, ["amqp://user:password@host.domain/my_vhost"]}, %% {declarations, []} %% ]}, %% List the destination broker(s) to publish to. %% {destinations, %% [%% A singular version of the 'brokers' element. %% {broker, "amqp://"}, %% {declarations, []} %% ]}, %% Name of the queue to shovel messages from. %% %% {queue, <<"your-queue-name-goes-here">>}, %% Optional prefetch count. %% %% {prefetch_count, 10}, %% when to acknowledge messages: %% - no_ack: never (auto) %% - on_publish: after each message is republished %% - on_confirm: when the destination broker confirms receipt %% %% {ack_mode, on_confirm}, %% Overwrite fields of the outbound basic.publish. %% %% {publish_fields, [{exchange, <<"my_exchange">>}, %% {routing_key, <<"from_shovel">>}]}, %% Static list of basic.properties to set on re-publication. %% %% {publish_properties, [{delivery_mode, 2}]}, %% The number of seconds to wait before attempting to %% reconnect in the event of a connection failure. %% %% {reconnect_delay, 2.5} %% ]} %% End of my_first_shovel ]} %% Rather than specifying some values per-shovel, you can specify %% them for all shovels here. %% %% {defaults, [{prefetch_count, 0}, %% {ack_mode, on_confirm}, %% {publish_fields, []}, %% {publish_properties, [{delivery_mode, 2}]}, %% {reconnect_delay, 2.5}]} ]}, %% ---------------------------------------------------------------------------- %% RabbitMQ Stomp Adapter %% %% See http://www.rabbitmq.com/stomp.html for details %% ---------------------------------------------------------------------------- {rabbitmq_stomp, [%% Network Configuration - the format is generally the same as for the broker %% Listen only on localhost (ipv4 & ipv6) on a specific port. %% {tcp_listeners, [{"127.0.0.1", 61613}, %% {"::1", 61613}]}, %% Listen for SSL connections on a specific port. %% {ssl_listeners, [61614]}, %% Number of Erlang processes that will accept connections for the TCP %% and SSL listeners. %% %% {num_tcp_acceptors, 10}, %% {num_ssl_acceptors, 1}, %% Additional SSL options %% Extract a name from the client's certificate when using SSL. %% %% {ssl_cert_login, true}, %% Set a default user name and password. This is used as the default login %% whenever a CONNECT frame omits the login and passcode headers. %% %% Please note that setting this will allow clients to connect without %% authenticating! %% %% {default_user, [{login, "guest"}, %% {passcode, "guest"}]}, %% If a default user is configured, or you have configured use SSL client %% certificate based authentication, you can choose to allow clients to %% omit the CONNECT frame entirely. If set to true, the client is %% automatically connected as the default user or user supplied in the %% SSL certificate whenever the first frame sent on a session is not a %% CONNECT frame. %% %% {implicit_connect, true} ]}, %% ---------------------------------------------------------------------------- %% RabbitMQ MQTT Adapter %% %% See https://github.com/rabbitmq/rabbitmq-mqtt/blob/stable/README.md %% for details %% ---------------------------------------------------------------------------- {rabbitmq_mqtt, [%% Set the default user name and password. Will be used as the default login %% if a connecting client provides no other login details. %% %% Please note that setting this will allow clients to connect without %% authenticating! %% %% {default_user, <<"guest">>}, %% {default_pass, <<"guest">>}, %% Enable anonymous access. If this is set to false, clients MUST provide %% login information in order to connect. See the default_user/default_pass %% configuration elements for managing logins without authentication. %% %% {allow_anonymous, true}, %% If you have multiple chosts, specify the one to which the %% adapter connects. %% %% {vhost, <<"/">>}, %% Specify the exchange to which messages from MQTT clients are published. %% %% {exchange, <<"amq.topic">>}, %% Specify TTL (time to live) to control the lifetime of non-clean sessions. %% %% {subscription_ttl, 1800000}, %% Set the prefetch count (governing the maximum number of unacknowledged %% messages that will be delivered). %% %% {prefetch, 10}, %% TCP/SSL Configuration (as per the broker configuration). %% %% {tcp_listeners, [1883]}, %% {ssl_listeners, []}, %% Number of Erlang processes that will accept connections for the TCP %% and SSL listeners. %% %% {num_tcp_acceptors, 10}, %% {num_ssl_acceptors, 1}, %% TCP/Socket options (as per the broker configuration). %% %% {tcp_listen_options, [{backlog, 128}, %% {nodelay, true}]} ]}, %% ---------------------------------------------------------------------------- %% RabbitMQ AMQP 1.0 Support %% %% See https://github.com/rabbitmq/rabbitmq-amqp1.0/blob/stable/README.md %% for details %% ---------------------------------------------------------------------------- {rabbitmq_amqp1_0, [%% Connections that are not authenticated with SASL will connect as this %% account. See the README for more information. %% %% Please note that setting this will allow clients to connect without %% authenticating! %% %% {default_user, "guest"}, %% Enable protocol strict mode. See the README for more information. %% %% {protocol_strict_mode, false} ]}, %% ---------------------------------------------------------------------------- %% RabbitMQ LDAP Plugin %% %% See http://www.rabbitmq.com/ldap.html for details. %% %% ---------------------------------------------------------------------------- {rabbitmq_auth_backend_ldap, [%% %% Connecting to the LDAP server(s) %% ================================ %% %% Specify servers to bind to. You *must* set this in order for the plugin %% to work properly. %% %% {servers, ["your-server-name-goes-here"]}, %% Connect to the LDAP server using SSL %% %% {use_ssl, false}, %% Specify the LDAP port to connect to %% %% {port, 389}, %% LDAP connection timeout, in milliseconds or 'infinity' %% %% {timeout, infinity}, %% Enable logging of LDAP queries. %% One of %% - false (no logging is performed) %% - true (verbose logging of the logic used by the plugin) %% - network (as true, but additionally logs LDAP network traffic) %% %% Defaults to false. %% %% {log, false}, %% %% Authentication %% ============== %% %% Pattern to convert the username given through AMQP to a DN before %% binding %% %% {user_dn_pattern, "cn=${username},ou=People,dc=example,dc=com"}, %% Alternatively, you can convert a username to a Distinguished %% Name via an LDAP lookup after binding. See the documentation for %% full details. %% When converting a username to a dn via a lookup, set these to %% the name of the attribute that represents the user name, and the %% base DN for the lookup query. %% %% {dn_lookup_attribute, "userPrincipalName"}, %% {dn_lookup_base, "DC=gopivotal,DC=com"}, %% Controls how to bind for authorisation queries and also to %% retrieve the details of users logging in without presenting a %% password (e.g., SASL EXTERNAL). %% One of %% - as_user (to bind as the authenticated user - requires a password) %% - anon (to bind anonymously) %% - {UserDN, Password} (to bind with a specified user name and password) %% %% Defaults to 'as_user'. %% %% {other_bind, as_user}, %% %% Authorisation %% ============= %% %% The LDAP plugin can perform a variety of queries against your %% LDAP server to determine questions of authorisation. See %% http://www.rabbitmq.com/ldap.html#authorisation for more %% information. %% Set the query to use when determining vhost access %% %% {vhost_access_query, {in_group, %% "ou=${vhost}-users,ou=vhosts,dc=example,dc=com"}}, %% Set the query to use when determining resource (e.g., queue) access %% %% {resource_access_query, {constant, true}}, %% Set queries to determine which tags a user has %% %% {tag_queries, []} ]} ].
启动服务设置配置文件:
RABBITMQ_NODENAME
RABBITMQ_NAME_TYPE
RABBITMQ_BOOT_MODULE
RABBITMQ_CONFIG_FILE
ERL_MAX_ETS_TABLES
rabbitmqctl.bat 命令
C:eClass abbitmq_serversbin>rabbitmqctl.bat Error: could not recognise command Usage: rabbitmqctl [-n <node>] [-t <timeout>] [-q] <command> [<command options>] Options: -n node -q -t timeout Default node is "rabbit@server", where server is the local host. On a host named "server.example.com", the node name of the RabbitMQ Erlang node will usually be rabbit@server (unless RABBITMQ_NODENAME has been set to some non-default value at broker startup time). The output of hostname -s is usually the correct suffix to use after the "@" sign. See rabbitmq-server(1) for details of configuring the RabbitMQ broker. Quiet output mode is selected with the "-q" flag. Informational messages are suppressed when quiet mode is in effect. Operation timeout in seconds. Only applicable to "list" commands. Default is "infinity". Commands: stop [<pid_file>] shutdown stop_app start_app wait <pid_file> reset force_reset rotate_logs <suffix> hipe_compile <directory> join_cluster <clusternode> [--ram] cluster_status change_cluster_node_type disc | ram forget_cluster_node [--offline] rename_cluster_node oldnode1 newnode1 [oldnode2] [newnode2 ...] update_cluster_nodes clusternode force_boot sync_queue [-p <vhost>] queue cancel_sync_queue [-p <vhost>] queue purge_queue [-p <vhost>] queue set_cluster_name name add_user <username> <password> delete_user <username> change_password <username> <newpassword> clear_password <username> authenticate_user <username> <password> set_user_tags <username> <tag> ... list_users add_vhost <vhost> delete_vhost <vhost> list_vhosts [<vhostinfoitem> ...] set_permissions [-p <vhost>] <user> <conf> <write> <read> clear_permissions [-p <vhost>] <username> list_permissions [-p <vhost>] list_user_permissions <username> set_parameter [-p <vhost>] <component_name> <name> <value> clear_parameter [-p <vhost>] <component_name> <key> list_parameters [-p <vhost>] set_global_parameter <name> <value> clear_global_parameter <name> list_global_parameters set_policy [-p <vhost>] [--priority <priority>] [--apply-to <apply-to>] <name> <pattern> <definition> clear_policy [-p <vhost>] <name> list_policies [-p <vhost>] list_queues [-p <vhost>] [--offline|--online|--local] [<queueinfoitem> ...] list_exchanges [-p <vhost>] [<exchangeinfoitem> ...] list_bindings [-p <vhost>] [<bindinginfoitem> ...] list_connections [<connectioninfoitem> ...] list_channels [<channelinfoitem> ...] list_consumers [-p <vhost>] status node_health_check environment report eval <expr> close_connection <connectionpid> <explanation> trace_on [-p <vhost>] trace_off [-p <vhost>] set_vm_memory_high_watermark <fraction> set_vm_memory_high_watermark absolute <memory_limit> set_disk_free_limit <disk_limit> set_disk_free_limit mem_relative <fraction> encode [--decode] [<value>] [<passphrase>] [--list-ciphers] [--list-hashes] [--cipher <cipher>] [--hash <hash>] [--iterations <iterations>] <vhostinfoitem> must be a member of the list [name, tracing]. The list_queues, list_exchanges and list_bindings commands accept an optional virtual host parameter for which to display results. The default value is "/". <queueinfoitem> must be a member of the list [name, durable, auto_delete, arguments, policy, pid, owner_pid, exclusive, exclusive_consumer_pid, exclusive_consumer_tag, messages_ready, messages_unacknowledged, messages, messages_ready_ram, messages_unacknowledged_ram, messages_ram, messages_persistent, message_bytes, message_bytes_ready, message_bytes_unacknowledged, message_bytes_ram, message_bytes_persistent, head_message_timestamp, disk_reads, disk_writes, consumers, consumer_utilisation, memory, slave_pids, synchronised_slave_pids, state]. <exchangeinfoitem> must be a member of the list [name, type, durable, auto_delete, internal, arguments, policy]. <bindinginfoitem> must be a member of the list [source_name, source_kind, destination_name, destination_kind, routing_key, arguments]. <connectioninfoitem> must be a member of the list [pid, name, port, host, peer_port, peer_host, ssl, ssl_protocol, ssl_key_exchange, ssl_cipher, ssl_hash, peer_cert_subject, peer_cert_issuer, peer_cert_validity, state, channels, protocol, auth_mechanism, user, vhost, timeout, frame_max, channel_max, client_properties, recv_oct, recv_cnt, send_oct, send_cnt, send_pend, connected_at]. <channelinfoitem> must be a member of the list [pid, connection, name, number, user, vhost, transactional, confirm, consumer_count, messages_unacknowledged, messages_uncommitted, acks_uncommitted, messages_unconfirmed, prefetch_count, global_prefetch_count].
linux环境下
cookie文件:/var/lib/rabbitmq/.erlang.cookie
rabbitmq节点容量配置
http://www.rabbitmq.com/memory.html
rabbitmqctl set_vm_memory_high_watermark 0.5 // 将rabbitmq占用机器的内存上限改为0.5 可以通过 rabbitmqctl status 查看结果中的 {vm_memory_high_watermark,0.5} |
内存管理
RabbitMQ服务器在启动时会计算系统内存总大小。然后会根据vm_memory_high_watermark参数指定的百分比,进行控制.可通过命令 rabbitmqctl set_vm_memory_high_watermark fraction 动态设置。
默认下,vm_memory_high_watermark的值为0.4,当RabbitMQ使用的内存超过40%时,系统会阻塞所有连接。一旦警报解除(消息被消费者取走,或者消息被写到硬盘中),系统重新恢复工作。
32位系统的单进程内存使用最大为2G,即使系统有10G,那么内存警报线为2G*40%.
在日志中可以找到内存限制的相关信息
例如 :/home/data/rabbitmq/log
=INFO REPORT==== 28-Apr-2015::14:11:16 ===
Memory limit set to 3804MB of 7609MB total.
当vm_memory_high_watermark为0时,不再有内存警报。并且所有的消息发布都将停止。这个方法可用来,如果要禁止消息发布的情况。
rabbitmqctl set_vm_memory_high_watermark 0.5
磁盘管理
在日志中可以找到内存限制的相关信息
例如 :/home/data/rabbitmq/log
=INFO REPORT==== 28-Apr-2015::14:11:16 ===
Disk free limit set to 50MB
当RabbitMQ内存使用达到预设值,并且阻塞信息发布前,会尝试把内存中的信息输出到磁盘上。持久化信息和非持久化信息都将被写到磁盘。(持 久化信息一进入队列就会被写到磁盘)
如果磁盘的预设值为50%,内存预设值默认是0.4,那么就是当内存使用量达到20%时,队列信息会被写到磁盘上。
可以通过设置 vm_memory_high_watermark_paging_ratio (默认值为0.5)来改变写入磁盘的策略,例如:
[{rabbit, [{vm_memory_high_watermark_paging_ratio, 0.75}, {vm_memory_high_watermark, 0.4}]}].
The above configuration starts paging at 30% of memory used, and blocks publishers at 40%.
上面的配置将会在内存全用30%时将队列信息写入到磁盘,阻塞信息发布是在内存使用40%时发生。
建议vm_memory_high_watermark 不超过50%.
RabbitMQ队列超长导致QueueingConsumerJVM溢出
解决RabbitMQ队列超长QueueingConsumer导致JVM内存溢出的问题
我们的服务器使用RabbitMQ作为消息中转的容器。某天我怀疑RabbitMQ队列是否都能及时消化。于是用命令查询了下:rabbitmqctl list_vhosts | grep -P ".*.host" | xargs -i rabbitmqctl list_queues -p {} | grep "queue"。
不查不知道,一查吓一跳:大多数服务器的队列基本上都是空的,但是有些服务器的某个队列竟然有超过500W条的记录。一般RabbitMQ进程占用内存不过100M-200M,这些队列超长的RabbitMQ进程可以占用超过2G的内存。
显然消息队列的消费者出现了问题。开发查看日志发现作为该队列消费者的Java服务的日志也卡住了,重启服务后(这点做得不对,应该用jstat、jstack进行排查,而不是直接重启)又很快卡住。这时候他才想起来用jstat,
通过jstat发现JVM内存都耗尽了,之后进入无尽的Full GC,所以当然不会处理队列消息和输出日志信息了。jstat的输出如下:
--------------------------------------------------------------------------
[root@mail ~]# jstat -gcutil 29389
S0 S1 E O P YGC YGCT FGC FGCT GCT
100.00 0.00 100.00 100.00 59.59 1639 2.963 219078 99272.246 99275.209
--------------------------------------------------------------------------
使用jmap导出这时候的Java堆栈,命令:jmap -dump:format=b,file=29389.hprof 29389。将得到的dump文件放到MAT(Eclipse Memory Analyzer)里进行分析,发现很明显是QueueingConsumer持有大量对象导致JVM内存溢出,截图如下:
上网搜索了下,发现有人遇到了类似的问题:RabbitMQ QueueingConsumer possible memory leak 。解决办法是调用Channel的basicQos方法,设置临时内存中最多保存的消息数。这个数值的设置建议参考 《Some queuing theory: throughput, latency and bandwidth》 权衡决定。
拍脑袋将basicQos设置为16后重启服务,队列终于开始消化了。用jstat 观察JVM内存使用情况,也没有再出现剧增溢出的现象。
总结:使用RabbitMQ的时候,一定要合理地设置QoS参数。我感觉RabbitMQ的默认做法其实是很脆弱的,容易导致雪崩。“You have a queue in Rabbit. You have some clients consuming from that queue. If you don't set a QoS setting at all (basic.qos), then Rabbit will push all the queue's messages to the clients as fast as the network and the clients will allow.“。 这样如果由于某些原因,队列中堆积了比较多的消息,就可能导致Comsumer内存溢出卡死,于是发生恶性循环,队列消息不断堆积得不到消化,彻底地悲剧了。
Pika提供以下适配器
BlockingConnection - 启用对库进行阻塞,同步操作以进行简单的使用
LibevConnection - 用于libev事件循环http://libev.schmorp.de的适配器
SelectConnection - 快速异步适配器
TornadoConnection - 适用于Tornado IO Loop的适配器http://tornadoweb.org
TwistedConnection - 用于Twisted异步包的适配器http://twistedmatrix.com/
链接: http://blog.csdn.net/u014308482/article/details/52947590