- 1 Introduction
- 2 Data Format Details
- 2.1 Client Message
- 3.1 Message Header
- 5.1.1 Type
- 5.1.2 Correlation ID
- 5.1.3 Partition ID
- 5.2 Message Header Extension
- 5.3 Message Body
- 5.4 Message Framing
- 5.5 Message Types
- 5.5.1 Request Message
- 5.5.2 Response Message
- 5.5.3 Event Message
- 6 Client Protocol Data Types
- 8 Connection Guide
- 9 Requests and Responses
- 10.1 Proxies
- 10.1.1 Java Example
- 11.1 Python Example
- 14.1 List
- 14.2 Java Example
- 16.1 Python Example
- 18.1 Lock
- 18.2 Java Example
- 19.1 Python Example
- 21.1 Java Example
- 22.1 Python Example
- 25.1 Python Example
- 27.1 Java Example
- 28.1 Python Example
- 30.1 Multiple Responses to a Single Request
- 30.1.1 Get Updates On Cluster Member Changes
- 30.1.2 Listeners
- 30.2 Error Codes
- 30.3 Timeouts and Retry
- 31 Miscellaneous
- 31.1 Smart Client versus Dummy Client
- 31.2 Serialization
- 31.3 Security
- 32 Protocol Messages
- 33 Glossary
Introduction
This document explains the new binary protocol Hazelcast uses to communicate with clients. This document is not a guide to implement a client that will interact with Hazelcast; rather, it specifies the wire data format for messages exchanged between a client and a Hazelcast server member node (server). Any client that wants to communicate with the Hazelcast cluster should obey the data format and communication details explained in this document.
Hazelcast client protocol is a binary protocol specification. The communication between the client and Hazelcast cluster starts when a client connects to a cluster member. The protocol is designed to be strict enough to ensure a standardization in the communication, but still flexible enough that developers may expand upon the protocol to implement custom features.
General guidelines:
- This document uses the following terms as defined by IETF RFC 2119: MUST, MUST NOT, MAY, SHOULD, SHOULD NOT.
- “Client” refers to the entity which communicates with a Hazelcast member node.
- “Server” refers to the Hazelcast member to which the client connects.
Data Format Details
Hazelcast provides a communication interface to access distributed objects through client protocol. This interface is a TCP socket listening for request messages. Currently, TCP socket communication is the only way a client can connect to the server. The client MUST connect to the port that Hazelcast is listening to for new connections. Because of this, there is no specific fixed port to which the client must connect. Protocol communication is built on message sending and receiving. Client protocol defines a simple message called "client message" for communication. It is the only data format defined by this protocol.
Before we go into the details of the communication, we define a client message in the following section.
Client Message
A client message is a transmission data unit composed of a fixed header and a variable payload data. Its main purpose is to encapsulate a unit of data to be transferred from one entity to another. It may represent a request, a response or an event response. A client message can be fragmented into multiple client messages and sent in order one-by-one. Please see the Message Framing section for details.
A message is composed of a header part and a payload body as shown below.
MESSAGE HEADER | MESSAGE HEADER EXTENSION | MESSAGE | |
16 BYTES | data-offset field, 2 bytes | 0 to 2^16 -18 bytes | PAYLOAD |
<-------------DATA OFFSET DEFINES THIS SIZE---------> |
| ||
The Client Message header could be extended (using the "Message Header Extension" field) or even include custom data that you (client developers) could define. In order to achieve a variable size header, you can use the data offset field of the header to set the start of the payload.
The current version of the protocol defines the header as the first 18 bytes of the Client Message as defined in the following section.
Message Header
Message header fields uniquely identify and define the client message.
Name | Data Type | Description |
Frame Size | int32 | Frame data size is the total size in bytes. The minimum value of the frame size can be a fixed header size of 18 bytes: header message only, no payload. |
Version | uint8 | Protocol version. Current version is 1. |
Flags | uint8 | Flag bits. Please see the table below for an explanation of the flags. |
Type | uint16 | Type of the message corresponding to a unique operation of a distributed object: e.g. map.put, map.get, etc., a response, a general protocol message type, or an event. |
Correlation ID | uint32 | This ID correlates the requests to responses. It should be unique to identify one message in the communication. This ID is used to track the request response cycle of a client operation. Server side should send response messages with the same ID as the request message. The uniqueness is per connection. Each message generating entity must generate unique IDs. If the client receives the response to a request and the request is not a multi-response request (i.e. not a request for event transmission), then the correlation ID for the request can be reused by subsequent requests. |
Partition ID | int32 | Target partition where this message will be processed. Negative values will go into a global execution pool and do not have a partition. |
Data Offset | uint16 | A message may have a payload data. Data offset value must be set to the number of bytes from the beginning of the frame to the start of the payload including the "Data Offset" field. |
Please see Client Protocol Data Types for the description of the "Data Type" column values.
Flag bits are depicted in the table below. The first row shows the bit numbers in the order of transmission.
8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 |
BEGIN | END | N/A | N/A | N/A | N/A | N/A | EVENT |
Flag bits are one bit tags to define some features of the client message. In the current protocol version, there are three different flags as defined below:
- Begin Flag: If this bit is set, this is the first part of the message the client sends.
- End Flag: If this bit is set, this is the final part of the message the client sends.
- Event Flag: If this bit is set, this message is an event response from the server.
Please note that if both BEGIN and END bits are set, it means that this message is not fragmented, i.e. it starts and ends in the same message. If neither the BEGIN nor END bit is set, then the message is an intermediate part of the entire message.
Type
The message type refers to the kind of operation the client runs on the server. A full list of types can be found in Protocol Messages.
If the server receives a message with an unsupported message type, it will return the "No such message task" error to the client. The client is guaranteed to receive only the messages listed in the Protocol Messages and the error messages.
Correlation ID
The correlation ID must be a unique unsigned 32-bit integer. The server always sends a response message with the same correlation ID as the corresponding request. The correlation ID also implies how a client registers for event updates (i.e. all subsequent event updates are sent using the same correlation ID as the original registration request correlation ID). Note that once a correlation ID is used to register for an event, it SHOULD NOT be used again unless the client unregisters (stops listening) for the event.
Partition ID
The partition ID defines the partition against which the operation is executed. The client can get the partition list of the cluster. This information tells the client which member handles which partition. The client can use this information to send the related requests to the responsible member (for the request key if it exists) directly for processing. The client gets this information from the “Get Partitions” request (see the Protocol Messages).
To determine the partition ID of an operation, compute the Murmur Hash (version 3, 32-bit, see https://en.wikipedia.org/wiki/MurmurHash and https://code.google.com/p/smhasher/wiki/MurmurHash3) of a certain byte-array (which is identified for each operation) and take the modulus of the result over the total number of partitions. The seed for the Murmur Hash SHOULD be 0x01000193. Most operations with a key parameter use the key parameter byte-array as the data for the hash calculation.
Some operations are not required to be executed on a specific partition, but can be run on any partition. For these operations, the partition ID is set to a negative value. No hash calculation is required in this case.
Message Header Extension
This value should be set to 1 for this version of the protocol. It identifies the version of the prevailing protocol. Since this is the first version of the protocol, this field has no use for now.
Message Body
The payload (the body of the message) starts from byte number "data-offset" (18 for the current header definition). The total number of bytes in the payload can be calculated as "frame-size - data-offset". The message payload can carry one of the following contents:
- If the message is a fragmented message, i.e. if either or both of the BEGIN or END flags are not set, then this body is just a fragment of an actual message that should be merged to form the actual message before message handling.
- If the message is a non-fragmented message, i.e. both BEGIN and END flags are set, then the body is a set of binary encoded fields specific to a message type (structure is identified by the TYPE field of the header).
Please refer to Protocol Messages for details of the encoding of each message type.
Message Framing
This protocol supports messages being split into smaller sized messages (fragments) to overcome the problem of huge messages blocking the channel. Message splitting is optionally applied depending on the implementation internals and various optimization reasons. Although message splitting is optional, merging the received frames into a complete message (if fragmented) must be supported (reassembly).
A frame size may vary from a size of 18 bytes (minimum header of a client message) to 2^31-1 bytes.
If a client message is required to be split into smaller frames, then the header part must be the same in the subsequent messages for all fragments except for the flag byte. While the body part should be split and distributed into new frames consecutively, there is no set of rules on how to split the payload.
The BEGIN flag must be set in the first frame and the END flag must be set in the last frame. Intermediate messages should have no BEGIN or END flags set.
On receiving the message with the END flag set, the frames should then be merged into a single frame and then processed.
Message Types
There are three different kinds of messages used in this protocol. These are:
- Request Message
- Response Message
- Event Message
Request Message
Each distributed object defines various operations. Each operation corresponds to a well defined request message to be sent to the cluster. For each request message, the client will get a response message from the cluster. Request messages MUST be sent from the client to the server.
The request parameters are binary encoded entirely within the payload of the message.
Response Message
Once a request message is received and processed at the server side, the server may produce a response message and send it to the client. Each request message type defines possible response message types that can be sent in response. The correlation ID relates all instances of the response messages to their requests.
The payload of the message is defined by the message type. The payload includes binary encoded parameters of the response.
Event Message
An event message is a special kind of response message. A client can register to a specific listener by sending a request message with the message type of adding a listener (for the specific message types that add listeners, see the “Type” section of the header format). When an event is triggered that the client is listening for, the server will send a message to the client using the same correlation ID as the original request message. The payload of the message carries the specific event object. The possible event message types for a registration request are documented in the "Events" section of each request in the Protocol Messages document.
The server will continue to send the client event updates until the client unregisters from that event or the connection is broken.
Client Protocol Data Types
Type | Description | Size | Min Value | Max Value |
uint8 | unsigned 8 bit integer | 8 bit | 0 | 255 (2^8 - 1) |
uint16 | unsigned 16 bit integer | 16 bit | 0 | 65535 (2^16 - 1) |
uint32 | unsigned 32 bit integer | 32 bit | 0 | 2^32 - 1 |
uint64 | unsigned 64 bit integer | 64 bit | 0 | 2^64 - 1 |
int8 | signed 8 bit integer in 2's compliment | 8 bit | -128 | 127 |
int16 | signed 16 bit integer in 2's compliment | 16 bit | -32768 | 32767 |
int32 | signed 32 bit integer in 2's compliment | 32 bit | -2^31 | 2^31 - 1 |
int64 | signed 64 bit integer in 2's compliment | 64 bit | -2^63 | 2^63 - 1 |
float | single precision IEEE 754, see link for details | 32 bit |
|
|
double | double precision IEEE 754, see link for details | 64 bit |
|
|
boolean | same as uint8 with special meanings 0 is "false" any other value is "true" | 8 bit | N/A | N/A |
string | String encoded as a byte array with UTF-8 encoding. Initial 4 bytes (uint32 type) represents the size of the string in bytes Example: the string “bad” {0x62,0x61,0x64} will be encoded as 0x03,0x00,0x00,0x00,0x62,0x61,0x64 | variable |
|
|
byte-array | Variable size bytes with a uint32 size prefix. Example: { 0x11, 0x22 } will be encoded as ,0x02, 0x00, 0x00, 0x00,0x11, 0x22 | variable |
|
|
Data types are consistent with those defined in The Open Group Base Specifications Issue 7 IEEE Std 1003.1, 2013 Edition. Data types are in Little Endian format.
Connection Guide
Open Connection
As previously stated, TCP socket communication is used for client-server communication. Each server has a server socket listening for incoming connections. This TCP socket is used for both server-to-server and client-to-server communications.
As the first step of client-to-server communication, the client MUST open a TCP socket connection to the server.
A client needs to establish a single connection to each member node if it is a smart client. If it is a dummy client, a single connection is enough for the client. For more information, please see Dummy Client versus Smart Client.
Connection Initialization
After successfully connecting to the server TCP socket port, the client MUST send three bytes of initialization data to identify the connection type to the server.
For any client, the three byte initializer data is [0x43, 0x42, 0x32], which is the string "CB2" in utf8 encoding.
Authentication
The first message sent through an initialized connection must be an authentication message. Any other type of messages will fail with an authorization error unless the authentication is complete. Authentication is the process of sending an authentication message and receiving an authentication result message if successful or an exception result if not successful.
Upon successful authentication, the client will receive a response from the server with the server’s IP address, the connection UUID and the owner UUID. If the client’s username/password/credentials are incorrect, it will receive an error AuthenticationException stating “Invalid credentials!”
There are two types of authentications:
Username/Password authentication: "Authentication" (See Protocol Messages for message fields) message is used for this authentication. The response is AuthenticationResult message. The result contains the address, UUID and owner UUID information.
Custom credentials authentication: “Custom Authentication” message is used. This method sends the custom authentication credentials as a byte array. The response is the same as the username/password authentication.
One of the parameters in the Authentication request is the boolean flag “isOwnerConnection”. Each client is associated with an owner server. This owner server tracks the availability of the client and if it detects that the client does not exist, it cleans up any resource allocated for that client. The locks acquired by the client is one such resource. “isOwnerConnection” needs to be set to true if this connection will be the owner connection.
Communication via Messages
After a successful authentication, a client may send request messages to the server to access distributed objects or perform other operations on the cluster. This step is the actual communication step.
Once connected, a client can do the following:
- Send periodic updates.
- Retrieve partition list.
- Send operation messages and receive responses.
- Get updates on cluster member changes.
All request messages will be sent to the server and all response and event messages will be sent to the client.
See Requests and Responses for details.
Closing Connections
To end the communication, the network socket that was opened in the "Open Connection" step should be closed. This will result in releasing resources on the server side specific to this connection.
Requests and Responses
Distributed Objects
To access distributed object information, use the “Get Distributed Object” message type (See Protocol Messages).
To add a listener for adding distributed objects, use the “Add Distributed Object Listener” message type.
To remove a listener for adding distributed objects, use the “Remove Distributed Object Listener" message type.
See Listeners for more information.
Partition List
The client can get the partition list of the cluster. This information tells the client which member handles which partition key. The client can use this information to send the related requests to the responsible member (for the request key if it exists) directly for processing. The request message is the “Get Partitions” request message, which has no payload.
The response contains the full cluster member list and the member-partition ownership information (using the owner index array).
To create a listener for the case of a partition being lost, use the “Add Partition Lost” request message. See below for the request, response, and event formatting.
To remove the listener, use the “Remove Partition Lost Listener” message.
Operation Messages and Responses
Operational messages are the messages where a client can expect exactly one response for a given request. The client knows which request the response correlates to via the correlation ID. An example of one of these messages is a “map put” operation.
General operation messages are listed in the "General Protocol Operations" section of Protocol Messages. To execute a particular operation, set the message type ID to the corresponding operation type. If a parameter is followed by "isNullable=true", this means that the parameter may be null. Such a parameter is encoded first by the boolean value being true or false, followed by the parameter value if the parameter is not null.
Proxies
Before using a distributed object, you SHOULD first create a proxy for the object. Do this by using the “Create Proxy” request message (See Protocol Messages).
To destroy a proxy, use the “Destroy Proxy” request message.
Java Example
HazelcastInstance client = HazelcastClient.newHazelcastClient(); IMap map=client.getMap("map-name"); |
Python Example
client=HazelcastClient() map=client.getMap("map-name") |
Raw bytes:
Client request
0x34 0x00 0x00 0x00 //Size of message
0x01 //Version
0xc0 //Flag
0x05 0x00 //Type
0x03 0x00 0x00 0x00 //Correlation ID
0xff 0xff 0xff 0xff //Partition ID
0x12 0x00 //Data Offset
0x08 0x00 0x00 0x00 //Size of following string:
0x6d 0x61 0x70 0x2d 0x6e 0x61 0x6d 0x65 //name: "map-name"
0x12 0x00 0x00 0x00 //Size of following string:
0x68 0x7a 0x3a 0x69 0x6d 0x70 0x6c 0x3a 0x6d 0x61 0x70 0x53 0x65 0x72 0x76 0x69 0x63 0x65 //serviceName: "hz:impl:mapService"
Server response
0x12 0x00 0x00 0x00 //Size of message
0x00 //Version
0xc0 //Flag
0x64 0x00 //Type
0x03 0x00 0x00 0x00 //Correlation ID
0xff 0xff 0xff 0xff //Partition ID
0x12 0x00 //Data Offset
For a request with a key, the client SHOULD send the request to the cluster member that houses the data for the key. A client can do this by using the partition ID. For more specific information on computing a given partition ID, see its details in the header section.
The response to a request message is always one of the following:
- Regular response message: The response is the message as listed in the protocol specification for the specific request message type.
- An error message: See “Error Codes” section.
We give examples of operations on various data structures below.
List
Java Example
IList myList=client.getList("list"); //create proxy System.out.println(myList.get(3)); |
Python Example
mylist=client.getList("list") #create proxy print mylist.get(3) |
Raw bytes
Client request
0x1e 0x00 0x00 0x00 //Size of message
0x01 //Version
0xc0 //Flags
0x05 0x0f //Type
0x05 0x00 0x00 0x00 //Correlation ID
0x96 0x00 0x00 0x00 //Partition ID
0x12 0x00 //Data Offset
0x04 0x00 0x00 0x00 //Size of following string
0x6c 0x69 0x73 0x74 //name: "list"
0x03 0x00 0x00 0x00 //index: 3
Server response
0x20 0x00 0x00 0x00 //Size of message
0x00 //Version
0xc0 //Flags
0x69 0x00 //Type
0x05 0x00 0x00 0x00 //Correlation ID
0xff 0xff 0xff 0xff //Partition ID
0x12 0x00 //Data Offset
0x00 0x09 0x00 0x00 0x00 0xff 0xff 0xff 0xf9 0x00 0x00 0x00 0x00 0x00 0x03 //response: bytes
Lock
Java Example
ILock myLock=client.getLock("lock"); //create proxy myLock.lock(); |
Python Example
lock=client.getLock("lock") #create proxy lock.lock() |
Raw bytes:
Client request
0x2a 0x00 0x00 0x00 //Size of message
0x01 //Version
0xc0 //Flag
0x05 0x07 //Type
0x05 0x00 0x00 0x00 //Correlation Id
0xda 0x00 0x00 0x00 //Partition Id
0x12 0x00 //Data Offset
0x04 0x00 0x00 0x00 //Size of following string
0x6c 0x6f 0x63 0x6b //name: "lock"
0xff 0xff 0xff 0xff //leaseTime: -1
0xff 0xff 0xff 0xff //threadId: -1
Server response:
0x12 0x00 0x00 0x00 //Size of message
0x00 //Version
0xc0 //Flag
0x64 0x00 //Type
0x05 0x00 0x00 0x00 //Correlation Id
0xff 0xff 0xff 0xff //Partition Id
0x12 0x00 //Data Offset
Map
Java Example
String key = "key1"; int value=54
IMap myMap = client.getMap("map"); //create proxy myMap.put(key1,value); |
Python Example
key="key1" value=54 map=client.getMap("map") #create proxy map.put(key,value) |
Raw bytes:
Client request
0x4e 0x00 0x00 0x00 //Size of message
0x01 //Version
0xc0 //Flag
0x01 0x01 //Type
0x05 0x00 0x00 0x00 //Correlation ID
0x98 0x00 0x00 0x00 //Partition ID
0x12 0x00 //Data Offset
0x03 0x00 0x00 0x00 //size of following string
0x6d 0x61 0x70 //name: "map"
0x14 0x00 0x00 0x00 //size of following byte object
0xff 0xff 0xff 0xf5 0x00 0x00 0x00 0x00 0x00 0x04 0x00 0x00 0x00 0x04 0x00 0x04 0x6b 0x65 0x79 0x31 //entry: bytes
0x09 0x00 0x00 0x00 0xff 0xff 0xff 0xf9
0x00 0x00 0x00 0x00 0x36 0x01 0x00 0x00
0x00 0x00 0x00 0x00 0x00 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff //I don't know why these bytes are being sent over
Server response
0x13 0x00 0x00 0x00 //Size of message
0x00 //Version
0xc0 //Flag
0x69 0x00 //Type
0x05 0x00 0x00 0x00 //Correlation ID
0x98 0x00 0x00 0x00 //Partition ID
0x12 0x00 //Data Offset
0x01 //success: true
Queue
Java Example
IQueue myQueue=client.getQueue("queue"); //create proxy System.out.println(myQueue.size()); |
Python Example
myQueue=client.getQueue("queue") #create proxy print myQueue.size() |
Raw bytes:
Client request
0x1b 0x00 0x00 0x00 //Size
0x01 //Version
0xc0 //flag
0x03 0x03 //Type
0x05 0x00 0x00 0x00 //Correlation ID
0x90 0x00 0x00 0x00 //Partition ID
0x12 0x00 //Data Offset
0x5 0x00 0x00 0x00 //Size of the following string
0x71 0x71 0x65 0x75 0x65 //Name: ”queue”
Server response
0x16 0x00 0x00 0x00 //Size of message
0x00 //Version
0xc0 //flag
0x66 0x00 //Type
0x05 0x00 0x00 0x00 //Correlation ID
0xff 0xff 0xff 0xff //Partition ID
0x12 0x00 //Data Offset
0x0a 0x00 0x00 0x00 //size: 10
Set
Java Example
ISet set=client.getSet("set"); //create proxy set.clear(); |
Python Example
set=client.getSet("set") #create proxy set.clear() |
Raw Bytes
Client request:
0x19 0x00 0x00 0x00 //Size of message
0x01 //Version
0xc0 //Flags
0x09 0x06 //Type
0x05 0x00 0x00 0x00 //Correlation Id
0x96 0x00 0x00 0x00 //Partition Id
0x12 0x00 //Data Offset
0x03 0x00 0x00 0x00 //Size of following string
0x73 0x65 0x74 //"set"
Server response:
0x12 0x00 0x00 0x00 //Size of message
0x01 //Version
0xc0 //Flags
0x64 0x00 //Type
0x05 0x00 0x00 0x00 //Correlation Id
0xff 0xff 0xff 0xff //Partition Id
0x12 0x00 //Data Offset
Multiple Responses to a Single Request
The client can listen for updates on a member or when specific actions are taken on the cluster. This is managed by the Event Listener mechanism. The event messages have the EVENT flag set in the message header FLAGS field and they use the same correlation ID as used in the original registration request for all the subsequent event update messages. The registration message and possible event messages sent are described in the Events section of the message descriptions in Protocol Messages.
Get Updates On Cluster Member Changes
The client can register to listen for cluster member list updates using the “Cluster Membership Listener” request message, and thus the client can retrieve the list of member server addresses in the cluster. This request message has no payload.
The response to the request message are events. The response events can be one of the following: the full member list, a specific member add/removal to the cluster, or a member server attribute change. This information is needed if the client operates as a smart client.
Listeners
Listeners are a means to communicate multiple responses to a client. The client uses one of the listener registration messages to listen for updates at the cluster. Listeners are specific to a data structure. For example, there is a specific listener for map entries versus queue entries. To see how these listeners are explicitly encoded, see the relevant message in Protocol Messages document.
Because the same correlation ID is reused for every event response for a given request, the correlation ID MUST NOT be reused from event requests unless the client unregisters the listener.
To remove all of the listeners that the client has registered to, you can send the “Remove All Listeners” request message.
Error Codes
The server may return an error response. In this case, the payload of the server’s message will contain the error message. You may choose to provide the direct error codes to the API user (developers using the API of your client) or you may use some other technique, such as exceptions, to communicate the error to the API user. See the "Error Message" section of the Protocol Messages document for details of this message.
Timeouts and Retry
It is recommended that the client should be able to handle situations where the member may not be able to return the response in an expected time interval. Even if the response to a specific message is not received, the user may or may not retry the request. If the client retries the request, they SHOULD NOT use the same correlation ID.
If no message has been sent in the server’s heartbeat time (configured by the "hazelcast.client.max.no.heartbeat.seconds" system property), the server will automatically disconnect from the client. To prevent this from occurring, a client SHOULD submit a “ping” request to the server periodically. A ping message is only sent from the client to the server; the server does NOT perform any ping request. For more information, please see the Client Protocol Implementation Guide.
Miscellaneous
Smart Client versus Dummy Client
The client can work as a smart client or as a dummy client. In both cases, a client SHOULD calculate which partition ID is responsible for the request and put this information in the Partition ID field of the header.
- Smart client: A smart client sends the request directly to the cluster member that is responsible for the related key. In order to do this, the client determines the address of the cluster member that handles the calculated partition ID. The request message will be sent on this cluster member connection.
- Dummy client: The client sends the request to any cluster member that it is connected to, regardless of the key for the request. The cluster member will in turn redirect the request to the correct member in the cluster that handles the request for the provided key.
The biggest difference between the two types of clients is that a smart client must be connected to all of the members and must constantly update its partition tables so it knows which connection to use to submit a request. Both clients are compliant with the protocol.
Serialization
While mostly an implementation detail, serialization plays a crucial role in the protocol. In order for a client to execute an operation on the server that involves a variable data structure, such as putting some entry in a map or queue, the client must be aware of how objects are serialized and deserialized so that the client can process the bytes it receives accordingly. In general, you need to use serialization byte-array type fields in the messages as specified in the Protocol Messages document. The following are examples of such objects that must be serialized before being sent over the wire:
- Key
- Value
- Old Value
- New Value
- Callable (Executor Service)
- IFunction (Atomics)
- EntryProcessor (JCache)
- ExpiryPolicy (JCache)
- CacheConfig (JCache)
- ListenerConfig (JCache)
- Mapper (MapReduce)
- CombinerFactory (MapReduce)
- ReducerFactory (MapReduce)
- KeyValueSource (MapReduce)
- Interceptor (Map)
A client may follow Hazelcast’s native serialization or it may implement its own custom serialization solution. For more information on how Hazelcast serializes its objects, see the official Reference Manual serialization section.
For all the byte-array parameters, the API user should implement the following, depending on the operation type:
- If the operation is such that no server side de-serialization is needed for the parameter, then the user can just use any serialization and there is no need to do any implementation for the server side.
- If the operation processing at the member server requires de-serialization of the byte-array parameter, then the user should use a Java object implementing one of the Hazelcast serializations (including portable serialization) and also implementing an interface required by the server for the object during processing of the specific operation. This Java interface is specified in the protocol documentation. Furthermore, this serializer must be registered in the serialization configuration of the member server as described in the Serialization section of the Reference Manual.
If Hazelcast’s serialization changes between releases and a client does not update its serialization accordingly, the client and server will exchange malformed data.
Security
Most of the security is configured on the server-side in a Hazelcast cluster. A client must authenticate itself, which in turn lets the server establish an end-point for the client. The server can restrict what a client can and cannot access. Current protocol does not provide an explicit support for encryption. For more information, see the Security chapter of the Hazelcast Reference Manual.
Protocol Messages
You can see the detailed description of protocol messages at Protocol Messages.
Glossary
Terminology | Definition |
client | Any Hazelcast native client implementation |
server/member | A Hazelcast cluster member |
protocol | Hazelcast client-server communication protocol |
serialization | Hazelcast internal implementation of serialization used to encode an object into a byte array and decode a byte array into an object |
protocol-version | The version of the protocol starting at 1 |
| fragmentation | Splitting a large message into pieces for transmission |
| Reassembly | Combining the message parts (fragments) to form the actual large message on reception |
| Cluster | A virtual environment formed by Hazelcast members communicating with each other |