原文地址：https://spotify.github.io/snakebite/hadoop_rpc.html
Snakebite currently implements the following protocol in snakebite.channel.SocketRpcChannel to communicate with the NameNode.
Connection
The Hadoop RPC protocol works as described below. On connection, headers are sent to setup a session. After that, multiple requests can be sent within the session.
| Function | Type | Default |
|---|---|---|
| Header | bytes | “hrpc” |
| Version | uint8 | 7 |
| Auth method | uint8 | 80 (Auth method SIMPLE) |
| Serialization type | uint8 | 0 (protobuf) |
| IpcConnectionContextProto length | uint32 | |
| IpcConnectionContextProto | bytes |
Sending messages
When sending a message, the following is sent to the sever:
| Function | Type |
|---|---|
| Length of the next two parts | uint32 |
| RpcPayloadHeaderProto length | varint |
| RpcPayloadHeaderProto | protobuf serialized message |
| HadoopRpcRequestProto length | varint |
| HadoopRpcRequestProto | protobuf serialized message |
varint is a Protocol Buffer variable int.
Note
The Java protobuf implementation uses writeToDelimited to prepend the message with their lenght, but the python implementation doesn’t implement such a method (yet).
Next to an rpcKind (snakebites default is RPC_PROTOCOL_BUFFER), an rpcOp (snakebites default isRPC_FINAL_PAYLOAD), the RpcPayloadHeaderProto message defines a callId that is added in the RPC response (described below).
The HadoopRpcRequestProto contains a methodName field that defines what server method is called and a has a property request that contains the serialized actual request message.
Receiving messages
After a message is sent, the response can be read in the following way:
| Function | Type |
|---|---|
| Length of the RpcResponseHeaderProto | varint |
| RpcResponseHeaderProto | bytes |
| Length of the RPC response | uint32 |
| Serialized RPC response | bytes |
The RpcResponseHeaderProto contains the callId of the request and a status field. The status can beSUCCESS, ERROR or FAILURE. In case SUCCESS the rest of response is a complete protobuf response.
In case of ERROR, the response looks like follows:
| Function | Type |
|---|---|
| Length of the RpcResponseHeaderProto | varint |
| RpcResponseHeaderProto | bytes |
| Length of the RPC response | uint32 |
| Length of the Exeption class name | uint32 |
| Exception class name | utf-8 string |
| Length of the stack trace | uint32 |
| Stack trace | utf-8 string |