Welcome to the Hazelcast Reference Manual. This manual includes concepts, instructions, and samples to guide you on how to use Hazelcast and build Hazelcast applications.
As the reader of this manual, you must be familiar with the Java programming language and you should have installed your preferred Integrated Development Environment (IDE).
This Reference Manual covers all editions of Hazelcast. Throughout this manual:
You can see the features for all Hazelcast editions in the following architecture diagram.
NOTE You can see small "HD" boxes for some features in the above diagram. Those features can use High-Density (HD) Memory Store when it is available. It means if you have Hazelcast Enterprise HD, you can use those features with HD Memory Store.
For more information on Hazelcast's Architecture, please see the white paper An Architect’s View of Hazelcast.
You can extend Hazelcast's functionality by using its plugins. These plugins have their own lifecycles. Please see Plugins page to learn about Hazelcast plugins you can use. Hazelcast plugins are marked with label throughout this manual.
Hazelcast and Hazelcast Reference Manual are free and provided under the Apache License, Version 2.0. Hazelcast Enterprise is commercially licensed by Hazelcast, Inc.
For more detailed information on licensing, please see the License Questions appendix.
Hazelcast is a registered trademark of Hazelcast, Inc. All other trademarks in this manual are held by their respective owners.
Support for Hazelcast is provided via GitHub, Mail Group and StackOverflow
For information on the commercial support for Hazelcast and Hazelcast Enterprise, please see hazelcast.com.
Please refer to the Release Notes document for the new features, enhancements and fixes performed for each Hazelcast release.
You can contribute to the Hazelcast code, report a bug, or request an enhancement. Please see the following resources.
Hazelcast partners with leading hardware and software technologies, system integrators, resellers and OEMs including Amazon Web Services, Vert.x, Azul Systems, C2B2. Please see the Partners page for the full list of and information on our partners.
Hazelcast uses phone home data to learn about usage of Hazelcast.
Hazelcast member instances call our phone home server initially when they are started and then every 24 hours. This applies to all the instances joined to the cluster.
What is sent in?
The following information is sent in a phone home:
Phone Home Code
The phone home code itself is open source. Please see here.
Disabling Phone Homes
Set the hazelcast.phone.home.enabled
system property to false either in the config or on the Java command line. Please see the System Properties section for information on how to set a property.
Phone Home URLs
For versions 1.x and 2.x: http://www.hazelcast.com/version.jsp.
For versions 3.x up to 3.6: http://versioncheck.hazelcast.com/version.jsp.
For versions after 3.6: http://phonehome.hazelcast.com/ping.
Below table shows the conventions used in this manual.
Convention | Description |
---|---|
bold font | - Indicates part of a sentence that requires the reader's specific attention. - Also indicates property/parameter values. |
italic font | - When italicized words are enclosed with "<" and ">", it indicates a variable in the command or code syntax that you must replace (for example, hazelcast-< version>.jar ). - Note and Related Information texts are in italics. |
monospace |
Indicates files, folders, class and library names, code snippets, and inline code words in a sentence. |
RELATED INFORMATION | Indicates a resource that is relevant to the topic, usually with a link or cross-reference. |
NOTE | Indicates information that is of special interest or importance, for example an additional action required only in certain circumstances. |
element & attribute | Mostly used in the context of declarative configuration that you perform using Hazelcast XML file. Element refers to an XML tag used to configure a Hazelcast feature. Attribute is a parameter owned by an element, contributing into the declaration of that element's configuration. Please see the following example.<port port-count="100">5701</port> In this example, port-count is an attribute of the port element. |
This chapter lists the changes made to this document from the previous release.
NOTE: Please refer to the Release Notes for the new features, enhancements and fixes performed for each Hazelcast release.
Chapter | Section | Description |
---|---|---|
Chapter 1 - Preface | Updated the architecture diagram. | |
Chapter 3 - Getting Started | Deploying on Microsoft Azure and Deploying On Pivotal Cloud Foundry added as a new sections. | |
Chapter 5 - Understanding Configuration | Added as a new chapter to provide the fundamentals of Hazelcast configuration. | |
Chapter 6 - Setting Up Clusters | Added as a new chapter to provide all Hazelcast clusters related information. | |
Discovering Native Clients | Added as a new section to explain Hazelcast's multicast discovery plugin. | |
Discovering Members with jclouds | Section's content moved to its own repo since this feature has become a Hazelcast plugin. You can find its repo's link in this section. | |
Discovering Members within EC2 Cloud | Section's content moved to its own repo since this feature has become a Hazelcast plugin. You can find its repo's link in this section. | |
Discovering Members within Azure Cloud | Added as a new section. | |
Partition Group Configuration | Added explanations of the new member group types ZONE_AWARE and SPI. | |
Chapter 7 - Distributed Data Structures | Replicated Map | Replicating instead of Partitioning updated by adding a note related to replicated map usage in a lite member. The whole section enhanced. |
Lock | Added explanations related to the maximum lease time for locks. | |
Map | Added the new section Custom Eviction Policy to explain how a customized eviction policy can be plugged. Listening to Map Entries with Predicates section updated by adding the explanation of a new system property ( hazelcast.map.entry.filtering.natural.event.types ) that allows better continuous query implementations. |
|
Chapter 9 - Distributed Computing | Using Indexes | Added as a new section. |
Durable Executor Service | Added as a new section to describe Hazelcast's newly introduced data structure, Durable Executor Service. | |
Chapter 10 - Distributed Query | ValueExtractor with Portable Serialization | Added as a new section. |
Explanation for the __key attribute added under Querying with SQL section. |
||
Chapter 11 - Transactions | Integrating into J2EE | Added information related to class loaders. |
Chapter 12 - Hazelcast JCache | ICache Configuration | Added description of the new element disable-per-entry-invalidation-events . |
JCache - Hazelcast Instance Integration | Added as a new section. | |
JCache Eviction | Custom Eviction Policies added as a new section. | |
Chapter 13 - Integrated Clustering | Web Session Replication | Updated Tomcat and Jetty based web session replication sections since they have become Hazelcast plugins. These sections' content is moved to their own repos. You can find these repos' links in the related sections. |
Hibernate Second Level Cache | Section's content moved to its own repo since this feature has become a Hazelcast plugin. You can find its repo's link in this section. | |
Spring Integration | Configuring Hazelcast Transaction Manager added as a new section. | |
Chapter 14 - Storage | Hot Restart Persistence | Added the new section Hot Restart Performance Considerations to summarize the results of performance tests of Hot Restart Persistence. |
Configuring High-Density Memory Store | Enhanced the content for allocator-type configuration element. |
|
Chapter 15 - Hazelcast Java Client | Enhanced the definition for the property hazelcast.client.invocation.timeout.seconds . |
|
Feature Comparison | Updated to reflect the latest feature developments. | |
Client System Properties | Added description for the property hazelcast.client.max.concurrent.invocations . |
|
Chapter 16 - Other Client and Language Implementations | Content of C++ and .NET clients updated so that the reader is directed to the GitHub repositories of these clients. | |
Chapter 17 - Serialization | Removed java.lang.Enum from the default types since it is not among the default serializers. |
|
Chapter 18 - Management | Management Center | Added information explaining how to configure Hazelcast Management Center when it is deployed onto an SSL-enabled web container. Added information explaining the "Update License" button. |
Clustered JMX via Management Center | List of attributes updated by adding the Replicated Map attributes. | |
Hazelcast CLI | Added as a new section. | |
Safety Checking Cluster Members | Updated the content to reflect the improvements in graceful shutdown feature. | |
Chapter 19 - Security | SSL | Added information explaining the performance overhead for the clients when they use SSL. |
Encryption | Added a note about the encryption at the client side. | |
Chapter 21 - Hazelcast Simulator | Moved the content to Simulator's own GitHub repository at Hazelcast Simulator. | |
Chapter 22 - WAN | Updated to reflect the improvement which is the ability of generic WAN replication endpoint configurations. Cleared the content related to WanNoDelayReplication since this implementation has been removed, and added a note under the Defining WAN Replication section. |
|
Synchronizing WAN Target Cluster | Added as a new section. | |
Solace Integration | Added as a new section explaining how to integrate Hazelcast WAN replication with Solace messaging appliances. | |
Chapter 26 - System Properties | Added definitions for the new properties: - hazelcast.partition.migration.stale.read.disabled - hazelcast.map.entry.filtering.natural.event.types - hazelcast.internal.map.expiration.cleanup.operation.count - hazelcast.internal.map.expiration.cleanup.percentage - hazelcast.internal.map.expiration.task.period.seconds |
|
Chapter 29 - FAQ | Added new questions/answers. | |
Chapter 30 - Glossary | Added new glossary items. |
This chapter explains how to install Hazelcast and start a Hazelcast member and client. It describes the executable files in the download package and also provides the fundamentals for configuring Hazelcast and its deployment options.
The following sections explain the installation of Hazelcast and Hazelcast Enterprise. It also includes notes and changes to consider when upgrading Hazelcast.
You can find Hazelcast in standard Maven repositories. If your project uses Maven, you do not need to add
additional repositories to your pom.xml
or add hazelcast-<version>.jar
file into your
classpath (Maven does that for you). Just add the following lines to your pom.xml
:
<dependencies>
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast</artifactId>
<version>3.7</version>
</dependency>
</dependencies>
As an alternative, you can download and install Hazelcast yourself. You only need to:
Download the package hazelcast-<version>.zip
or hazelcast-<version>.tar.gz
from
hazelcast.org.
Extract the downloaded hazelcast-<version>.zip
or hazelcast-<version>.tar.gz
.
Add the file hazelcast-<version>.jar
to your classpath.
There are two Maven repositories defined for Hazelcast Enterprise:
<repository>
<id>Hazelcast Private Snapshot Repository</id>
<url>https://repository-hazelcast-l337.forge.cloudbees.com/snapshot/</url>
</repository>
<repository>
<id>Hazelcast Private Release Repository</id>
<url>https://repository-hazelcast-l337.forge.cloudbees.com/release/</url>
</repository>
Hazelcast Enterprise customers may also define dependencies, a sample of which is shown below.
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-enterprise-tomcat6</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-enterprise-tomcat7</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-enterprise</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-enterprise-all</artifactId>
<version>${project.version}</version>
</dependency>
Hazelcast Enterprise offers you two types of licenses: Enterprise and Enterprise HD. The supported features differ in your Hazelcast setup according to the license type you own.
To use Hazelcast Enterprise, you need to set the provided license key using one of the configuration methods shown below.
Declarative Configuration:
Add the below line to any place you like in the file hazelcast.xml
. This XML file offers you a declarative way to configure your Hazelcast. It is included in the Hazelcast download package. When you extract the downloaded package, you will see the file hazelcast.xml
under the /bin
directory.
<hazelcast>
...
<license-key>Your Enterprise License Key</license-key>
...
</hazelcast>
Client Declarative Configuration:
Native client distributions (Java, C++, .NET) of Hazelcast are open source. However, there are some Hazelcast Enterprise features which can be used with the Java Client such as SSL, Socket Interceptors, High-Density backed Near Cache, etc. In that case, you also need to have a Hazelcast Enterprise license and you should include this license in the file hazelcast-client-full.xml
which is located under the directory src/main/resources
of your hazelcast-client
package. Set the license key in the hazelcast-client-full.xml
as shown below.
<hazelcast-client>
...
<license-key>Your Enterprise License Key</license-key>
...
</hazelcast-client>
Programmatic Configuration:
Alternatively, you can set your license key programmatically as shown below.
Config config = new Config();
config.setLicenseKey( "Your Enterprise License Key" );
Spring XML Configuration:
If you are using Spring with Hazelcast, then you can set the license key using the Spring XML schema, as shown below.
<hz:config>
...
<hz:license-key>Your Enterprise License Key</hz:license-key>
...
</hz:config>
JVM System Property:
As another option, you can set your license key using the below command (the "-D" command line option).
-Dhazelcast.enterprise.license.key=Your Enterprise License Key
Upgrading from 3.6.x to 3.7.x when using JCache
:
Hazelcast 3.7 introduced changes in JCache
implementation which broke compatibility of 3.6.x clients to 3.7-3.7.2 cluster members and vice versa,
so 3.7-3.7.2 clients are also incompatible with 3.6.x cluster members. This issue only affects Java clients which use JCache
functionality.
Starting with Hazelcast version 3.7.3, a compatibility option is provided which can be used to ensure backwards compatibility with 3.6.x clients. In order to upgrade a 3.6.x cluster and clients to 3.7.3 (or later), you will need to use this compatibility option on either the member or the client side, depending on which one is upgraded first:
hazelcast.compatibility.3.6.client=true
to your configuration; when started with this
property, cluster members are compatible with 3.6.x and 3.7.3+ clients but not with 3.7-3.7.2 clients. Once your cluster is upgraded, you may
upgrade your applications to use client version 3.7.3+.upgrade your clients from 3.6.x to 3.7.3, adding property hazelcast.compatibility.3.6.server=true
to your Hazelcast client configuration. A
3.7.3 client started with this compatibility option is compatible with 3.6.x and 3.7.3+ cluster members but incompatible with 3.7-3.7.2 cluster
members. Once your clients are upgraded, you may then proceed to upgrade your cluster members to version 3.7.3 or later.
You may use any of the supported ways as described in System Properties section to configure the compatibility option. When done upgrading your cluster and clients, you may remove the compatibility property from your Hazelcast member configuration.
Introducing the spring-aware
element:
Before the release 3.5, Hazelcast uses SpringManagedContext
to scan SpringAware
annotations by default. This may cause some performance overhead for the users who do not use SpringAware
.
This behavior has been changed with the release of Hazelcast 3.5. SpringAware
annotations are disabled by default. By introducing the spring-aware
element, now it is possible to enable it by adding the <hz:spring-aware />
tag to the configuration. Please see the Spring Integration section.
Introducing new configuration options for WAN replication: Starting with the release 3.6, WAN replication related system properties, which are configured on a per member basis, can now be configured per target cluster. The 4 system properties below are no longer valid.
hazelcast.enterprise.wanrep.batch.size
, please see the WAN Replication Batch Size.
hazelcast.enterprise.wanrep.batchfrequency.seconds
, please see the WAN Replication Batch Maximum Delay.
hazelcast.enterprise.wanrep.optimeout.millis
, please see the WAN Replication Response Timeout.
hazelcast.enterprise.wanrep.queue.capacity
, please see the WAN Replication Queue Capacity.
Removal of deprecated getId() method:
The method getId()
in the interface DistributedObject
has been removed. Please use the method getName()
instead.
Change in the Custom Serialization in the C++ Client Distribution:
Before, the method getTypeId()
was used to retrieve the ID of the object to be serialized. Now, the method getHazelcastTypeId()
is used and you give your object as a parameter to this new method. Also, getTypeId()
was used in your custom serializer class, now it has been renamed to getHazelcastTypeId()
too. Note that, these changes also apply when you want to switch from Hazelcast 3.6.1 to 3.6.2 too.
Map<Integer, String> customers = Hazelcast.getMap( "customers" );
with
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
// or if you already started an instance named "instance1"
// HazelcastInstance hazelcastInstance = Hazelcast.getHazelcastInstanceByName( "instance1" );
Map<Integer, String> customers = hazelcastInstance.getMap( "customers" );
public static void main( String[] args ) throws InterruptedException {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IMap map = hazelcastInstance.getMap( "test" );
Collection<Instance> instances = hazelcastInstance.getInstances();
for ( Instance instance : instances ) {
if ( instance.getInstanceType() == Instance.InstanceType.MAP ) {
System.out.println( "There is a map with name: " + instance.getId() );
}
}
}
with
public static void main( String[] args ) throws InterruptedException {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IMap map = hz.getMap( "test" );
Collection<DistributedObject> objects = hazelcastInstance.getDistributedObjects();
for ( DistributedObject distributedObject : objects ) {
if ( distributedObject instanceof IMap ) {
System.out.println( "There is a map with name: " + distributedObject.getName() );
}
}
}
com.hazelcast.core
from com.hazelcast.partition
.removeListener
methods were taking the Listener object as a parameter. But this caused confusion because same listener object may be used as a parameter for different listener registrations. So we have changed the listener API. addListener
methods returns a unique ID and you can remove a listener by using this ID. So you should do the following replacement if needed:IMap map = hazelcastInstance.getMap( "map" );
map.addEntryListener( listener, true );
map.removeEntryListener( listener );
with
IMap map = hazelcastInstance.getMap( "map" );
String listenerId = map.addEntryListener( listener, true );
map.removeEntryListener( listenerId );
tryRemove(K key, long timeout, TimeUnit timeunit)
returns boolean indicating whether operation is successful.tryLockAndGet(K key, long time, TimeUnit timeunit)
is removed.putAndUnlock(K key, V value)
is removed.lockMap(long time, TimeUnit timeunit)
and unlockMap()
are removed.getMapEntry(K key)
is renamed as getEntryView(K key)
. The returned object's type, MapEntry class is renamed as EntryView.<merge-policy>com.hazelcast.map.merge.PassThroughMergePolicy</merge-policy>
Also MergePolicy interface has been renamed to MapMergePolicy and also returning null from the implemented merge()
method causes the existing entry to be removed.
IQueue
is configured. With Hazelcast 3.0 there will be no backing map configuration for queue. Settings like backup count will be directly configured on queue config. For queue configuration details, please see the Queue section.pause()
, resume()
, restart()
methods have been removed.AtomicNumber
class has been renamed to IAtomicLong
.await()
operation has been removed. We expect users to use await()
method with timeout parameters.ISemaphore
has been substantially changed. attach()
, detach()
methods have been removed.max-size
eviction policy was cluster_wide_map_size. In 3.x releases, default is PER_NODE. After upgrading, the max-size
should be set according to this new default, if it is not changed. Otherwise, it is likely that OutOfMemory exception may be thrown.Having installed Hazelcast, you can get started.
In this short tutorial, you perform the following activities.
Let's begin.
customers
map and queue.import com.hazelcast.core.*;
import com.hazelcast.config.*;
import java.util.Map;
import java.util.Queue;
public class GettingStarted {
public static void main(String[] args) {
Config cfg = new Config();
HazelcastInstance instance = Hazelcast.newHazelcastInstance(cfg);
Map<Integer, String> mapCustomers = instance.getMap("customers");
mapCustomers.put(1, "Joe");
mapCustomers.put(2, "Ali");
mapCustomers.put(3, "Avi");
System.out.println("Customer with key 1: "+ mapCustomers.get(1));
System.out.println("Map Size:" + mapCustomers.size());
Queue<String> queueCustomers = instance.getQueue("customers");
queueCustomers.offer("Tom");
queueCustomers.offer("Mary");
queueCustomers.offer("Jane");
System.out.println("First customer: " + queueCustomers.poll());
System.out.println("Second customer: "+ queueCustomers.peek());
System.out.println("Queue size: " + queueCustomers.size());
}
}
Now, add the hazelcast-client-
<version>
.jar
library to your classpath.
This is required to use a Hazelcast client.
The following code starts a Hazelcast Client, connects to our cluster,
and prints the size of the customers
map.
package com.hazelcast.test;
import com.hazelcast.client.config.ClientConfig;
import com.hazelcast.client.HazelcastClient;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.IMap;
public class GettingStartedClient {
public static void main( String[] args ) {
ClientConfig clientConfig = new ClientConfig();
HazelcastInstance client = HazelcastClient.newHazelcastClient( clientConfig );
IMap map = client.getMap( "customers" );
System.out.println( "Map Size:" + map.size() );
}
}
Hazelcast also offers a tool, Management Center, that enables you to monitor your cluster.
To use it, deploy the mancenter-
<version>
.war
included in the ZIP file to your web server.
You can use it to monitor your maps, queues, and other distributed data structures and members. Please
see the Management Center section for usage explanations.
By default, Hazelcast uses Multicast to discover other members that can form a cluster. If you are
working with other Hazelcast developers on the same network, you may find yourself joining their
clusters under the default settings. Hazelcast provides a way to segregate clusters within the same
network when using Multicast. Please see the Creating Cluster Groups
for more information. Alternatively, if you do not wish to use the default Multicast mechanism,
you can provide a fixed list of IP addresses that are allowed to join. Please see
the Join Configuration section for more information.
RELATED INFORMATION
You can also check the video tutorials here.
When you download and extract the Hazelcast ZIP or TAR.GZ package, you will see three scripts under the /bin
folder that provide basic functionalities for member and cluster management.
The following are the names and descriptions of each script:
start.sh
/ start.bat
: Starts a Hazelcast member with default configuration in the working directory*.stop.sh
/ stop.bat
: Stops the Hazelcast member that was started in the current working directory.cluster.sh
: Provides basic functionalities for cluster management, such as getting and changing the cluster state, shutting down the cluster or forcing the cluster to clean its persisted data and make a fresh start. NOTE: start.sh
/ start.bat
scripts lets you start one Hazelcast instance per folder. To start a new instance, please unzip Hazelcast ZIP or TAR.GZ package in a new folder.
Please refer to the Using the Script cluster.sh section to learn the usage of this script.
You can deploy your Hazelcast project onto an Amazon EC2 environment using Third Party tools such as Vagrant and Chef.
You can find a sample deployment project (amazon-ec2-vagrant-chef
) with step-by-step instructions in the hazelcast-integration
folder of the hazelcast-code-samples package, which you can download at hazelcast.org. Please refer to this sample project for more information.
You can deploy your Hazelcast cluster onto a Microsoft Azure environment. For this, your cluster should make use of Hazelcast Discovery Plugin for Microsoft Azure. You can find information about this plugin on its GitHub repository at Hazelcast Azure.
For information on how to automatically deploy your cluster onto Azure, please see the Deployment section of Hazelcast Azure plugin repository.
Starting with Hazelcast 3.7, you can deploy your Hazelcast cluster onto Cloud Foundry. This deployment enables Hazelcast to be used in multiple ways on Cloud Foundry platform. You can deploy Hazelcast in the following ways:
Integration between Hazelcast and Pivotal Cloud Foundry is provided as a Hazelcast plugin. Please see its own GitHub repo at Hazelcast Cloud Foundry for details on configurations and usages.
You can deploy your Hazelcast projects using the Docker containers. Hazelcast has the following images on Docker:
After you pull an image from the Docker registry, you can run your image to start the management center or a Hazelcast instance with Hazelcast's default configuration. All repositories provide the latest stable releases but you can pull a specific release too. You can also specify environment variables when running the image.
If you want to start a customized Hazelcast instance, you can extend the Hazelcast image by providing your own configuration file.
This feature is provided as a Hazelcast plugin. Please see its own GitHub repo at Hazelcast Docker for details on configurations and usages.
Hazelcast is an open source In-Memory Data Grid (IMDG). It provides elastically scalable distributed In-Memory computing, widely recognized as the fastest and most scalable approach to application performance. Hazelcast does this in open source. More importantly, Hazelcast makes distributed computing simple by offering distributed implementations of many developer-friendly interfaces from Java such as Map, Queue, ExecutorService, Lock, and JCache. For example, the Map interface provides an In-Memory Key Value store which confers many of the advantages of NoSQL in terms of developer friendliness and developer productivity.
In addition to distributing data In-Memory, Hazelcast provides a convenient set of APIs to access the CPUs in your cluster for maximum processing speed. Hazelcast is designed to be lightweight and easy to use. Since Hazelcast is delivered as a compact library (JAR) and since it has no external dependencies other than Java, it easily plugs into your software solution and provides distributed data structures and distributed computing utilities.
Hazelcast is highly scalable and available (100% operational, never failing). Distributed applications can use Hazelcast for distributed caching, synchronization, clustering, processing, pub/sub messaging, etc. Hazelcast is implemented in Java and has clients for Java, C/C++, .NET and REST. Hazelcast also speaks memcache protocol. It plugs into Hibernate and can easily be used with any existing database system.
If you are looking for In-Memory speed, elastic scalability, and the developer friendliness of NoSQL, Hazelcast is a great choice.
Hazelcast is Simple
Hazelcast is written in Java with no other dependencies. It exposes the same API from the familiar Java util package,
exposing the same interfaces. Just add hazelcast.jar
to your classpath and you can quickly enjoy JVMs clustering
and start building scalable applications.
Hazelcast is Peer-to-Peer
Unlike many NoSQL solutions, Hazelcast is peer-to-peer. There is no master and slave; there is no single point of failure. All members store equal amounts of data and do equal amounts of processing. You can embed Hazelcast in your existing application or use it in client and server mode where your application is a client to Hazelcast members.
Hazelcast is Scalable
Hazelcast is designed to scale up to hundreds and thousands of members. Simply add new members and they will automatically discover the cluster and will linearly increase both memory and processing capacity. The members maintain a TCP connection between each other and all communication is performed through this layer.
Hazelcast is Fast
Hazelcast stores everything in-memory. It is designed to perform very fast reads and updates.
Hazelcast is Redundant
Hazelcast keeps the backup of each data entry on multiple members. On a member failure, the data is restored from the backup and the cluster will continue to operate without downtime.
Hazelcast shards are called Partitions. By default, Hazelcast has 271 partitions. Given a key, we serialize, hash and mode it with the number of partitions to find the partition which the key belongs to. The partitions themselves are distributed equally among the members of the cluster. Hazelcast also creates the backups of partitions and distributes them among members for redundancy.
RELATED INFORMATION
Please refer to the Data Partitioning section for more information on how Hazelcast partitions your data.
You can deploy a Hazelcast cluster in two ways: Embedded or Client/Server.
If you have an application whose main focal point is asynchronous or high performance computing and lots of task executions, then Embedded deployment is useful. In Embedded deployment, members include both the application and Hazelcast data and services. The advantage of the Embedded deployment is having a low-latency data access.
See the below illustration.
In the Client/Server deployment, Hazelcast data and services are centralized in one or more server members and they are accessed by the application through clients. You can have a cluster of server members that can be independently created and scaled. Your clients communicate with these members to reach to Hazelcast data and services on them. Hazelcast provides native clients (Java, .NET and C++), Memcache clients and REST clients. See the illustration at the end of this section.
Client/Server deployment has advantages including more predictable and reliable Hazelcast performance, easier identification of problem causes, and most importantly, better scalability. When you need to scale in this deployment type, just add more Hazelcast server members. You can address client and server scalability concerns separately.
If you want low-latency data access, as in the Embedded deployment, and you also want the scalability advantages of the Client/Server deployment, you can consider defining near caches for your clients. This enables the frequently used data to be kept in the client's local memory. Please refer to Configuring Client Near Cache.
A Glance at Traditional Data Persistence
Data is at the core of software systems. In conventional architectures, a relational database persists and provides access to data. Applications are talking directly with a database which has its backup as another machine. To increase performance, tuning or a faster machine is required. This can cost a large amount of money or effort.
There is also the idea of keeping copies of data next to the database, which is performed using technologies like external key-value stores or second level caching that help offload the database. However, when the database is saturated or the applications perform mostly "put" operations (writes), this approach is of no use because it insulates the database only from the "get" loads (reads). Even if the applications are read-intensive there can be consistency problems--when data changes, what happens to the cache, and how are the changes handled? This is when concepts like time-to-live (TTL) or write-through come in.
In the case of TTL, if the access is less frequent than the TTL, the result will always be a cache miss. On the other hand, in the case of write-through caches, if there are more than one of these caches in a cluster, we again will have consistency issues. This can be avoided by having the nodes communicate with each other so that entry invalidations can be propagated.
We can conclude that an ideal cache would combine TTL and write-through features. There are several cache servers and in-memory database solutions in this field. However, these are stand-alone single instances with a distribution mechanism that is provided by other technologies to an extent. So, we are back to square one; we experience saturation or capacity issues if the product is a single instance or if consistency is not provided by the distribution.
And, there is Hazelcast
Hazelcast, a brand new approach to data, is designed around the concept of distribution. Hazelcast shares data around the cluster for flexibility and performance. It is an in-memory data grid for clustering and highly scalable data distribution.
One of the main features of Hazelcast is that it does not have a master member. Each cluster member is configured to be the same in terms of functionality. The oldest member (the first member created in the cluster) automatically performs the data assignment to cluster members. If the oldest member dies, the second oldest member takes over.
Another main feature of Hazelcast is that the data is held entirely in-memory. This is fast. In the case of a failure, such as a member crash, no data will be lost since Hazelcast distributes copies of the data across all the cluster members.
As shown in the feature list in the Hazelcast Overview, Hazelcast supports a number of distributed data structures and distributed computing utilities. These provide powerful ways of accessing distributed clustered memory and accessing CPUs for true distributed computing.
Hazelcast's Distinctive Strengths
Finally, Hazelcast has a vibrant open source community enabling it to be continuously developed.
Hazelcast is a fit when you need:
As you read in the Sharding in Hazelcast section, Hazelcast shards are called Partitions. Partitions are memory segments that can contain hundreds or thousands of data entries each, depending on the memory capacity of your system.
By default, Hazelcast offers 271 partitions. When you start a cluster member, it starts with these 271 partitions. The following illustration shows the partitions in a Hazelcast cluster with single member.
When you start a second member on that cluster (creating a Hazelcast cluster with two members), the partitions are distributed as shown in the illustration here.
In the illustration, the partitions with black text are primary partitions and the partitions with blue text are replica partitions (backups). The first member has 135 primary partitions (black), and each of these partitions are backed up in the second member (blue). At the same time, the first member also has the replica partitions of the second member's primary partitions.
As you add more members, Hazelcast moves some of the primary and replica partitions to the new members one by one, making all members equal and redundant. Only the minimum amount of partitions will be moved to scale out Hazelcast. The following is an illustration of the partition distributions in a Hazelcast cluster with four members.
Hazelcast distributes the partitions equally among the members of the cluster. Hazelcast creates the backups of partitions and distributes them among the members for redundancy.
Partition distributions in the above illustrations are for your convenience and descriptive purposes. Normally, the partitions are not distributed in an order (as they are shown in these illustrations), but are distributed randomly. The important point here is that Hazelcast equally distributes the partitions and their backups among the members.
Starting with Hazelcast 3.6, lite members are introduced. Lite members are a new type of members that do not own any partition. Lite members are intended for use in computationally-heavy task executions and listener registrations. Although they do not own any partitions, they can access partitions that are owned by other members in the cluster.
RELATED INFORMATION
Please refer to the Enabling Lite Members section.
Hazelcast distributes data entries into the partitions using a hashing algorithm. Given an object key (for example, for a map) or an object name (for example, for a topic or list):
The result of this modulo - MOD(hash result, partition count) - is the partition in which the data will be stored, that is the partition ID. For ALL members you have in your cluster, the partition ID for a given key will always be the same.
When you start a member, a partition table is created within it. This table stores the partition IDs and the cluster members to which they belong. The purpose of this table is to make all members (including lite members) in the cluster aware of this information, making sure that each member knows where the data is.
The oldest member in the cluster (the one that started first) periodically sends the partition table to all members. In this way each member in the cluster is informed about any changes to partition ownership. The ownerships may be changed when, for example, a new member joins the cluster, or when a member leaves the cluster.
NOTE: If the oldest member of the cluster goes down, the next oldest member sends the partition table information to the other ones.
You can configure the frequency (how often) that the member sends the partition table the information by using the hazelcast.partition.table.send.interval
system property. The property is set to every 15 seconds by default.
Repartitioning is the process of redistribution of partition ownerships. Hazelcast performs the repartitioning in the following cases:
In these cases, the partition table in the oldest member is updated with the new partition ownerships.
Note that if a lite member joins or leaves a cluster, repartitioning is not triggered since lite members do not own any partitions.
Hazelcast can be used:
This chapter describes the options to configure your Hazelcast applications and explains the utilities which you can make use of while configuring. You can configure Hazelcast using one or mix of the following options:
This is the configuration option where you use an XML configuration file. When you download and unzip hazelcast-<version>.zip
, you will see the following files present in /bin
folder, which are standard XML-formatted configuration files:
hazelcast.xml
: Default declarative configuration file for Hazelcast. The configuration in this XML file should be fine for most of the Hazelcast users. If not, you can tailor this XML file according to your needs by adding/removing/modifying properties.hazelcast-full-example.xml
: Configuration file which includes all Hazelcast configuration elements and attributes with their descriptions. It is the "superset" of hazelcast.xml
. You can use hazelcast-full-example.xml
as a reference document to learn about any element or attribute, or you can change its name to hazelcast.xml
and start to use it as your Hazelcast configuration file.A part of hazelcast.xml
is shown as an example below.
<group>
<name>dev</name>
<password>dev-pass</password>
</group>
<management-center enabled="false">http://localhost:8080/mancenter</management-center>
<network>
<port auto-increment="true" port-count="100">5701</port>
<outbound-ports>
<!--
Allowed port range when connecting to other members.
0 or * means the port provided by the system.
-->
<ports>0</ports>
</outbound-ports>
<join>
<multicast enabled="true">
<multicast-group>224.2.2.3</multicast-group>
<multicast-port>54327</multicast-port>
</multicast>
<tcp-ip enabled="false">
You can compose the declarative configuration of your Hazelcast member or Hazelcast client from multiple declarative configuration snippets. In order to compose a declarative configuration, you can use the <import/>
element to load different declarative configuration files.
Let's say you want to compose the declarative configuration for Hazelcast out of two configurations: development-group-config.xml
and development-network-config.xml
. These two configurations are shown below.
development-group-config.xml
:
<hazelcast>
<group>
<name>dev</name>
<password>dev-pass</password>
</group>
</hazelcast>
development-network-config.xml
:
<hazelcast>
<network>
<port auto-increment="true" port-count="100">5701</port>
<join>
<multicast enabled="true">
<multicast-group>224.2.2.3</multicast-group>
<multicast-port>54327</multicast-port>
</multicast>
</join>
</network>
</hazelcast>
To get your example Hazelcast declarative configuration out of the above two, use the <import/>
element as shown below.
<hazelcast>
<import resource="development-group-config.xml"/>
<import resource="development-network-config.xml"/>
</hazelcast>
This feature also applies to the declarative configuration of Hazelcast client. Please see the following examples.
client-group-config.xml
:
<hazelcast-client>
<group>
<name>dev</name>
<password>dev-pass</password>
</group>
</hazelcast-client>
client-network-config.xml
:
<hazelcast-client>
<network>
<cluster-members>
<address>127.0.0.1:7000</address>
</cluster-members>
</network>
</hazelcast-client>
To get a Hazelcast client declarative configuration from the above two examples, use the <import/>
element as shown below.
<hazelcast-client>
<import resource="client-group-config.xml"/>
<import resource="client-network-config.xml"/>
</hazelcast>
NOTE: Use <import/>
element on top level of the XML hierarchy.
Using the element <import>
, you can also load XML resources from classpath and file system:
<hazelcast>
<import resource="file:///etc/hazelcast/development-group-config.xml"/> <!-- loaded from filesystem -->
<import resource="classpath:development-network-config.xml"/> <!-- loaded from classpath -->
</hazelcast>
The element <import>
supports placeholders too. Please see the following example snippet:
<hazelcast>
<import resource="${environment}-group-config.xml"/>
<import resource="${environment}-network-config.xml"/>
</hazelcast>
Besides declarative configuration, you can configure your cluster programmatically. For this you can create a Config
object, set/change its properties and attributes, and use this Config
object to create a new Hazelcast member. Following is an example code which configures some network and Hazelcast Map properties.
Config config = new Config();
config.getNetworkConfig().setPort( 5900 )
.setPortAutoIncrement( false );
MapConfig mapConfig = new MapConfig();
mapConfig.setName( "testMap" )
.setBackupCount( 2 );
.setTimeToLiveSeconds( 300 );
config.addMapConfig( mapConfig );
To create a Hazelcast member with the above example configuration, pass the configuration object as shown below:
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance( config );
NOTE: The Config
must not be modified after the Hazelcast instance is started. In other words, all configuration must be completed before creating the HazelcastInstance
.
You can also create a named Hazelcast member. In this case, you should set instanceName
of Config
object as shown below:
Config config = new Config();
config.setInstanceName( "my-instance" );
Hazelcast.newHazelcastInstance( config );
To retrieve an existing Hazelcast member by its name, use the following:
Hazelcast.getHazelcastInstanceByName( "my-instance" );
To retrieve all existing Hazelcast members, use the following:
Hazelcast.getAllHazelcastInstances();
NOTE: Hazelcast performs schema validation through the file hazelcast-config-<version>.xsd
which comes with your Hazelcast libraries. Hazelcast throws a meaningful exception if there is an error in the declarative or programmatic configuration.
If you want to specify your own configuration file to create Config
, Hazelcast supports several ways including filesystem, classpath, InputStream, and URL:
Config cfg = new XmlConfigBuilder(xmlFileName).build();
Config cfg = new XmlConfigBuilder(inputStream).build();
Config cfg = new ClasspathXmlConfig(xmlFileName);
Config cfg = new FileSystemXmlConfig(configFilename);
Config cfg = new UrlXmlConfig(url);
Config cfg = new InMemoryXmlConfig(xml);
You can use system properties to configure some aspects of Hazelcast. You set these properties as name and value pairs through declarative configuration, programmatic configuration or JVM system property. Following are examples for each option.
Declaratively:
....
<properties>
<property name="hazelcast.property.foo">value</property>
....
</properties>
</hazelcast>
Programmatically:
Config config = new Config() ;
config.setProperty( "hazelcast.property.foo", "value" );
Using JVM's System
class or -D
argument:
System.setProperty( "hazelcast.property.foo", "value" );
or
java -Dhazelcast.property.foo=value
You will see Hazelcast system properties mentioned throughout this Reference Manual as required in some of the chapters and sections. All Hazelcast system properties are listed in the System Properties appendix with their descriptions, default values and property types as a reference for you.
If you use Hazelcast with Spring you can declare beans using the namespace hazelcast
. When you add the namespace declaration to the element beans
in the Spring context file, you can start to use the namespace shortcut hz
to be used as a bean declaration. Following is an example Hazelcast configuration when integrated with Spring:
<hz:hazelcast id="instance">
<hz:config>
<hz:group name="dev" password="password"/>
<hz:network port="5701" port-auto-increment="false">
<hz:join>
<hz:multicast enabled="false"/>
<hz:tcp-ip enabled="true">
<hz:members>10.10.1.2, 10.10.1.3</hz:members>
</hz:tcp-ip>
</hz:join>
</hz:network>
</hz:config>
</hz:hazelcast>
Please see the Spring Integration section for more information on Hazelcast-Spring integration.
When you start a Hazelcast member without passing a Config
object, as explained in the Configuring Programmatically section, Hazelcast checks the member's configuration as follows:
First, it looks for the hazelcast.config
system property. If it is set, its value is used as the path. This is useful if you want to be able to change your Hazelcast configuration; you can do this because it is not embedded within the application. You can set the config
option with the following command:
- Dhazelcast.config=
<path to the hazelcast.xml>
.
The path can be a regular one or a classpath reference with the prefix classpath:
.
hazelcast.xml
file in the working directory.hazelcast.xml
exists on the classpath.hazelcast.xml
) that comes with your Hazelcast package.Before configuring Hazelcast, please try to work with the default configuration to see if it works for you. This default configuration should be fine for most of the users. If not, you can consider to modify the configuration to be more suitable for your environment.
Hazelcast supports wildcard configuration for all distributed data structures that can be configured using Config
, that is, for all except IAtomicLong
, IAtomicReference
. Using an asterisk (*) character in the name, different instances of maps, queues, topics, semaphores, etc. can be configured by a single configuration.
A single asterisk (*) can be placed anywhere inside the configuration name.
For instance, a map named com.hazelcast.test.mymap
can be configured using one of the following configurations.
<map name="com.hazelcast.test.*">
...
</map>
<map name="com.hazel*">
...
</map>
<map name="*.test.mymap">
...
</map>
<map name="com.*test.mymap">
...
</map>
Or a queue 'com.hazelcast.test.myqueue
':
<queue name="*hazelcast.test.myqueue">
...
</queue>
<queue name="com.hazelcast.*.myqueue">
...
</queue>
In your Hazelcast and/or Hazelcast Client declarative configuration, you can use variables to set the values of the elements. This is valid when you set a system property programmatically or you use the command line interface. You can use a variable in the declarative configuration to access the values of the system properties you set.
For example, see the following command that sets two system properties.
-Dgroup.name=dev -Dgroup.password=somepassword
Let's get the values of these system properties in the declarative configuration of Hazelcast, as shown below.
<hazelcast>
<group>
<name>${group.name}</name>
<password>${group.password}</password>
</group>
</hazelcast>
This also applies to the declarative configuration of Hazelcast Client, as shown below.
<hazelcast-client>
<group>
<name>${group.name}</name>
<password>${group.password}</password>
</group>
</hazelcast-client>
If you do not want to rely on the system properties, you can use the XmlConfigBuilder
and explicitly set a Properties
instance, as shown below.
Properties properties = new Properties();
// fill the properties, e.g. from database/LDAP, etc.
XmlConfigBuilder builder = new XmlConfigBuilder();
builder.setProperties(properties)
Config config = builder.build();
HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);
This chapter describes Hazelcast clusters and the methods cluster members and native clients use to form a Hazelcast cluster.
A Hazelcast cluster is a network of cluster members that run Hazelcast. Cluster members (also called nodes) automatically join together to form a cluster. This automatic joining takes place with various discovery mechanisms that the cluster members use to find each other. Hazelcast uses the following discovery mechanisms:
Each discovery mechanism is explained in the following sections.
NOTE: After a cluster is formed, communication between cluster members is always via TCP/IP, regardless of the discovery mechanism used.
With the multicast auto-discovery mechanism, Hazelcast allows cluster members to find each other using multicast communication. The cluster members do not need to know the concrete addresses of the other members, as they just multicast to all the other members for listening. Whether multicast is possible or allowed depends on your environment.
To set your Hazelcast to multicast auto-discovery, set the following configuration elements. Please refer to the multicast element section for the full description of the multicast discovery configuration elements.
enabled
attribute of the multicast
element to "true".multicast-group
, multicast-port
, multicast-time-to-live
, etc. to your multicast values.enabled
attribute of both tcp-ip
and aws
elements to "false".The following is an example declarative configuration.
<hazelcast>
...
<network>
...
<join>
<multicast enabled="true">
<multicast-group>224.2.2.3</multicast-group>
<multicast-port>54327</multicast-port>
<multicast-time-to-live>32</multicast-time-to-live>
<multicast-timeout-seconds>2</multicast-timeout-seconds>
<trusted-interfaces>
<interface>192.168.1.102</interface>
</trusted-interfaces>
</multicast>
<tcp-ip enabled="false">
</tcp-ip>
<aws enabled="false">
</aws>
</join>
<network>
Pay attention to the multicast-timeout-seconds
element. multicast-timeout-seconds
specifies the time in seconds that a member should wait for a valid multicast response from another member running in the network before declaring itself the leader member (the first member joined to the cluster) and creating its own cluster. This only applies to the startup of members where no leader has been assigned yet. If you specify a high value to multicast-timeout-seconds
, such as 60 seconds, it means that until a leader is selected, each member will wait 60 seconds before moving on. Be careful when providing a high value. Also, be careful not to set the value too low, or the members might give up too early and create their own cluster.
NOTE: Multicast auto-discovery is not supported for Hazelcast native clients yet. However, we offer Multicast Discovery Plugin for this purpose. Please refer to the Discovering Native Clients section.
If multicast is not the preferred way of discovery for your environment, then you can configure Hazelcast to be a full TCP/IP cluster. When you configure Hazelcast to discover members by TCP/IP, you must list all or a subset of the members' hostnames and/or IP addresses as cluster members. You do not have to list all of these cluster members, but at least one of the listed members has to be active in the cluster when a new member joins.
To set your Hazelcast to be a full TCP/IP cluster, set the following configuration elements. Please refer to the tcp-ip element section for the full description of the TCP/IP discovery configuration elements.
enabled
attribute of the multicast
element to "false".enabled
attribute of the aws
element to "false".enabled
attribute of the tcp-ip
element to "true".member
elements within the tcp-ip
element.The following is an example declarative configuration.
<hazelcast>
...
<network>
...
<join>
<multicast enabled="false">
</multicast>
<tcp-ip enabled="true">
<member>machine1</member>
<member>machine2</member>
<member>machine3:5799</member>
<member>192.168.1.0-7</member>
<member>192.168.1.21</member>
</tcp-ip>
...
</join>
...
</network>
...
</hazelcast>
As shown above, you can provide IP addresses or hostnames for member
elements. You can also give a range of IP addresses, such as 192.168.1.0-7
.
Instead of providing members line-by-line as shown above, you also have the option to use the members
element and write comma-separated IP addresses, as shown below.
<members>192.168.1.0-7,192.168.1.21</members>
If you do not provide ports for the members, Hazelcast automatically tries the ports 5701, 5702, and so on.
By default, Hazelcast binds to all local network interfaces to accept incoming traffic. You can change this behavior using the system property hazelcast.socket.bind.any
. If you set this property to false
, Hazelcast uses the interfaces specified in the interfaces
element (please refer to the Interfaces Configuration section). If no interfaces are provided, then it will try to resolve one interface to bind from the member
elements.
Hazelcast supports EC2 auto-discovery. It is useful when you do not want to provide or you cannot provide the list of possible IP addresses. This discovery feature is provided as a Hazelcast plugin. Please see its own GitHub repo at Hazelcast AWS for information on configuring and using it.
NOTE: hazelcast-cloud module has been renamed as hazelcast-aws module (starting with Hazelcast 3.7.3). If you want to use AWS Discovery, you should add the library hazelcast-aws JAR to your environment. For more information please look see the README of Hazelcast AWS repo.
Hazelcast offers a discovery strategy for your Hazelcast applications running on Azure. This strategy provides all of your Hazelcast instances by returning the virtual machines within your Azure resource group that are tagged with a specified value. This discovery feature is provided as a Hazelcast plugin. Please see its own GitHub repo at Hazelcast Azure for information on configuring and using it.
Hazelcast members and native clients support jclouds® for discovery. It is useful when you do not want to provide or you cannot provide the list of possible IP addresses on various cloud providers. This discovery feature is provided as a Hazelcast plugin. Please see its own GitHub repo at Hazelcast JClouds for information on configuring and using it.
Hazelcast members and native clients can find each other with multicast discovery plugin. This plugin is implemented using Hazelcast Discovery SPI. You should configure the plugin both at Hazelcast members and clients in order to use multicast discovery.
To configure your cluster to have the multicast discovery plugin, follow these steps:
enabled
attributes of the multicast
and tcp-ip
elements to false
in your hazelcast.xml
configuration fileenabled
attribute of the hazelcast.discovery.enabled
property to true
.<discovery-strategies>
element.The following is an example declarative configuration.
...
<properties>
<property name="hazelcast.discovery.enabled">true</property>
</properties>
....
<join>
<multicast enabled="false">
</multicast>
<tcp-ip enabled="false">
</tcp-ip>
<discovery-strategies>
<discovery-strategy class="com.hazelcast.spi.discovery.multicast.MulticastDiscoveryStrategy" enabled="true">
<properties>
<property name="group">224.2.2.3</property>
<property name="port">54327</property>
</properties>
</discovery-strategy>
</discovery-strategies>
</join>
...
The table below lists the multicast discovery plugin configuration properties with their descriptions.
Property Name | Type | Description |
---|---|---|
group |
String | String value that is used to set the multicast group, so that you can isolate your clusters. |
port |
Integer | Integer value that is used to set the multicast port. |
You can create cluster groups. To do this, use the group
configuration element.
By specifying a group name and group password, you can separate your clusters in a simple way. Example groupings can be by development, production, test, app, etc. The following is an example declarative configuration.
<hazelcast>
<group>
<name>app1</name>
<password>app1-pass</password>
</group>
...
</hazelcast>
You can also define the cluster groups using the programmatic configuration. A JVM can host multiple Hazelcast instances. Each Hazelcast instance can only participate in one group. Each Hazelcast instance only joins to its own group and does not interact with other groups. The following code example creates three separate Hazelcast instances--h1
belongs to the app1
cluster, while h2
and h3
belong to the app2
cluster.
Config configApp1 = new Config();
configApp1.getGroupConfig().setName( "app1" ).setPassword( "app1-pass" );
Config configApp2 = new Config();
configApp2.getGroupConfig().setName( "app2" ).setPassword( "app2-pass" );
HazelcastInstance h1 = Hazelcast.newHazelcastInstance( configApp1 );
HazelcastInstance h2 = Hazelcast.newHazelcastInstance( configApp2 );
HazelcastInstance h3 = Hazelcast.newHazelcastInstance( configApp2 );
Hazelcast distributes key objects into partitions using a consistent hashing algorithm. Those partitions are assigned to members. An entry is stored in the member that owns the partition to which the entry's key is assigned. The total partition count is 271 by default; you can change it with the configuration property hazelcast.partition.count
. Please see the System Properties section.
Along with those partitions, there are also copies of the partitions as backups. Backup partitions can have multiple copies due to the backup count defined in configuration, such as first backup partition, second backup partition, etc. A member cannot hold more than one copy of a partition (ownership or backup). By default, Hazelcast distributes partitions and their backup copies randomly and equally among cluster members, assuming all members in the cluster are identical.
But what if some members share the same JVM or physical machine or chassis and you want backups of these members to be assigned to members in another machine or chassis? What if processing or memory capacities of some members are different and you do not want an equal number of partitions to be assigned to all members?
You can group members in the same JVM (or physical machine) or members located in the same chassis. Or you can group members to create identical capacity. We call these groups partition groups. Partitions are assigned to those partition groups instead of to single members. Backups of these partitions are located in another partition group.
When you enable partition grouping, Hazelcast presents the following choices for you to configure partition groups.
1. HOST_AWARE:
You can group members automatically using the IP addresses of members, so members sharing the same network interface will be grouped together. All members on the same host (IP address or domain name) will be a single partition group. This helps to avoid data loss when a physical server crashes, because multiple replicas of the same partition are not stored on the same host. But if there are multiple network interfaces or domain names per physical machine, that will make this assumption invalid.
Following are declarative and programmatic configuration snippets that show how to enable HOST_AWARE grouping.
<partition-group enabled="true" group-type="HOST_AWARE" />
Config config = ...;
PartitionGroupConfig partitionGroupConfig = config.getPartitionGroupConfig();
partitionGroupConfig.setEnabled( true )
.setGroupType( MemberGroupType.HOST_AWARE );
2. CUSTOM:
You can do custom grouping using Hazelcast's interface matching configuration. This way, you can add different and multiple interfaces to a group. You can also use wildcards in the interface addresses. For example, the users can create rack-aware or data warehouse partition groups using custom partition grouping.
Following are declarative and programmatic configuration examples that show how to enable and use CUSTOM grouping.
<partition-group enabled="true" group-type="CUSTOM">
<member-group>
<interface>10.10.0.*</interface>
<interface>10.10.3.*</interface>
<interface>10.10.5.*</interface>
</member-group>
<member-group>
<interface>10.10.10.10-100</interface>
<interface>10.10.1.*</interface>
<interface>10.10.2.*</interface>
</member-group
</partition-group>
Config config = ...;
PartitionGroupConfig partitionGroupConfig = config.getPartitionGroupConfig();
partitionGroupConfig.setEnabled( true )
.setGroupType( MemberGroupType.CUSTOM );
MemberGroupConfig memberGroupConfig = new MemberGroupConfig();
memberGroupConfig.addInterface( "10.10.0.*" )
.addInterface( "10.10.3.*" ).addInterface("10.10.5.*" );
MemberGroupConfig memberGroupConfig2 = new MemberGroupConfig();
memberGroupConfig2.addInterface( "10.10.10.10-100" )
.addInterface( "10.10.1.*").addInterface( "10.10.2.*" );
partitionGroupConfig.addMemberGroupConfig( memberGroupConfig );
partitionGroupConfig.addMemberGroupConfig( memberGroupConfig2 );
3. PER_MEMBER:
You can give every member its own group. Each member is a group of its own and primary and backup partitions are distributed randomly (not on the same physical member). This gives the least amount of protection and is the default configuration for a Hazelcast cluster. This grouping type provides good redundancy when Hazelcast members are on separate hosts. However, if multiple instances run on the same host, this type is not a good option.
Following are declarative and programmatic configuration snippets that show how to enable PER_MEMBER grouping.
<partition-group enabled="true" group-type="PER_MEMBER" />
Config config = ...;
PartitionGroupConfig partitionGroupConfig = config.getPartitionGroupConfig();
partitionGroupConfig.setEnabled( true )
.setGroupType( MemberGroupType.PER_MEMBER );
4. ZONE_AWARE:
You can use ZONE_AWARE configuration with Hazelcast jclouds or Hazelcast Azure Discovery Service plugins.
As discovery services, these plugins put zone, rack, and host information to the Hazelcast member attributes map during the discovery process. Hazelcast creates the partition groups with respect to member attributes map entries that include zone, rack, and host information.
When using ZONE_AWARE configuration, backups are created in the other zones. Each zone will be accepted as one partition group.
NOTE: Some cloud providers have rack information instead of zone information. In such cases, Hazelcast looks for zone, rack, and host information in the given order.
Following are declarative and programmatic configuration snippets that show how to enable ZONE_AWARE grouping.
<partition-group enabled="true" group-type="ZONE_AWARE" />
Config config = ...;
PartitionGroupConfig partitionGroupConfig = config.getPartitionGroupConfig();
partitionGroupConfig.setEnabled( true )
.setGroupType( MemberGroupType.ZONE_AWARE );
NOTE: Currently ZONE_AWARE configuration works only with Hazelcast jclouds and Hazelcast Azure Discovery Service plugins. Please refer to their GitHub repositories at Hazelcast jclouds and Hazelcast Azure for more information on these plugins.
5. SPI:
You can provide your own partition group implementation using the SPI configuration. To create your partition group implementation, you need to first extend the DiscoveryStrategy
class of the discovery service plugin, override the method public PartitionGroupStrategy getPartitionGroupStrategy()
, and return the PartitionGroupStrategy
configuration in that overridden method.
Following is a sample code covering the implementation steps mentioned in the above paragraph:
public class CustomDiscovery extends JCloudsDiscoveryStrategy {
public CustomDiscovery(Map<String, Comparable> properties) {
super(properties);
}
@Override
public PartitionGroupStrategy getPartitionGroupStrategy() {
return new CustomPartitionGroupStrategy();
}
private class CustomPartitionGroupStrategy implements PartitionGroupStrategy {
@Override
public Iterable<MemberGroup> getMemberGroups() {
...
...
}
}
}
Hazelcast has a flexible logging configuration and does not depend on any logging framework except JDK logging. It has built-in adapters for a number of logging frameworks and it also supports custom loggers by providing logging interfaces.
To use built-in adapters, set the hazelcast.logging.type
property to one of the predefined types below.
jdk: JDK logging (default)
log4j: Log4j
slf4j: Slf4j
none: disable logging
You can set hazelcast.logging.type
through declarative configuration, programmatic configuration, or JVM system property.
NOTE: If you choose to use log4j
or slf4j
, you should include the proper dependencies in the classpath.
Declarative Configuration
....
<properties>
<property name="hazelcast.logging.type">jdk</property>
....
</properties>
</hazelcast>
Programmatic Configuration
Config config = new Config() ;
config.setProperty( "hazelcast.logging.type", "log4j" );
System Property
java -Dhazelcast.logging.type=slf4j
System.setProperty( "hazelcast.logging.type", "none" );
If the provided logging mechanisms are not satisfactory, you can implement your own using the custom logging feature. To use it, implement the com.hazelcast.logging.LoggerFactory
and com.hazelcast.logging.ILogger
interfaces and set the system property hazelcast.logging.class
as your custom LoggerFactory
class name.
-Dhazelcast.logging.class=foo.bar.MyLoggingFactory
You can also listen to logging events generated by Hazelcast runtime by registering LogListener
s to LoggingService
.
LogListener listener = new LogListener() {
public void log( LogEvent logEvent ) {
// do something
}
}
HazelcastInstance instance = Hazelcast.newHazelcastInstance();
LoggingService loggingService = instance.getLoggingService();
loggingService.addLogListener( Level.INFO, listener );
Through the LoggingService
, you can get the currently used ILogger implementation and log your own messages too.
NOTE: If you are not using command line for configuring logging, you should be careful about Hazelcast classes. They may be defaulted to jdk
logging before newly configured logging is read. When logging mechanism is selected, it will not change.
All network related configurations are performed via the network
element in the Hazelcast XML configuration file or the class NetworkConfig
when using programmatic configuration. Following subsections describe the available configurations that you can perform under the network
element.
public-address
overrides the public address of a member. By default, a member selects its socket address as its public address. But behind a network address translation (NAT), two endpoints (members) may not be able to see/access each other. If both members set their public addresses to their defined addresses on NAT, then that way they can communicate with each other. In this case, their public addresses are not an address of a local network interface but a virtual address defined by NAT. It is optional to set and useful when you have a private cloud. Note that, the value for this element should be given in the format host IP address:port number
. See the following examples.
Declarative:
<network>
<public-address>11.22.33.44:5555</public-address>
</network>
Programmatic:
Config config = new Config();
config.getNetworkConfig()
.setPublicAddress( "11.22.33.44", "5555" );
You can specify the ports that Hazelcast will use to communicate between cluster members. Its default value is 5701
. The following are example configurations.
Declarative:
<network>
<port port-count="20" auto-increment="false">5701</port>
</network>
Programmatic:
Config config = new Config();
config.getNetworkConfig().setPort( "5701" );
.setPortCount( "20" ).setPortAutoIncrement( false );
port
has the following attributes.
port-count
: By default, Hazelcast will try 100 ports to bind. Meaning that, if you set the value of port as 5701, as members are joining to the cluster, Hazelcast tries to find ports between 5701 and 5801. You can choose to change the port count in the cases like having large instances on a single machine or willing to have only a few ports to be assigned. The parameter port-count
is used for this purpose, whose default value is 100.auto-increment
: According to the above example, Hazelcast will try to find free ports between 5701 and 5801. Normally, you will not need to change this value, but it will come very handy when needed. You may also want to choose to use only one port. In that case, you can disable the auto-increment feature of port
by setting auto-increment
to false
.The parameter port-count
is ignored when the above configuration is made.
By default, Hazelcast lets the system pick up an ephemeral port during socket bind operation. But security policies/firewalls may require you to restrict outbound ports to be used by Hazelcast-enabled applications. To fulfill this requirement, you can configure Hazelcast to use only defined outbound ports. The following are example configurations.
Declarative:
<network>
<outbound-ports>
<!-- ports between 33000 and 35000 -->
<ports>33000-35000</ports>
<!-- comma separated ports -->
<ports>37000,37001,37002,37003</ports>
<ports>38000,38500-38600</ports>
</outbound-ports>
</network>
Programmatic:
...
NetworkConfig networkConfig = config.getNetworkConfig();
// ports between 35000 and 35100
networkConfig.addOutboundPortDefinition("35000-35100");
// comma separated ports
networkConfig.addOutboundPortDefinition("36001, 36002, 36003");
networkConfig.addOutboundPort(37000);
networkConfig.addOutboundPort(37001);
...
Note: You can use port ranges and/or comma separated ports.
As shown in the programmatic configuration, you use the method addOutboundPort
to add only one port. If you need to add a group of ports, then use the method addOutboundPortDefinition
.
In the declarative configuration, the element ports
can be used for both single and multiple port definitions.
When you shutdown a cluster member, the server socket port will be in the TIME_WAIT
state for the next couple of minutes. If you start the member right after shutting it down, you may not be able to bind it to the same port because it is in the TIME_WAIT
state. If you set the reuse-address
element to true
, the TIME_WAIT
state is ignored and you can bind the member to the same port again.
The following are example configurations.
Declarative:
<network>
<reuse-address>true</reuse-address>
</network>
Programmatic:
...
NetworkConfig networkConfig = config.getNetworkConfig();
networkConfig.setReuseAddress( true );
...
The join
configuration element is used to discover Hazelcast members and enable them to form a cluster. Hazelcast provides multicast, TCP/IP, EC2, and jclouds® discovery mechanisms. These mechanisms are explained the Discovering Cluster Members section. This section describes all the sub-elements and attributes of join
element. The following are example configurations.
Declarative:
<network>
<join>
<multicast enabled="true">
<multicast-group>224.2.2.3</multicast-group>
<multicast-port>54327</multicast-port>
<multicast-time-to-live>32</multicast-time-to-live>
<multicast-timeout-seconds>2</multicast-timeout-seconds>
<trusted-interfaces>
<interface>192.168.1.102</interface>
</trusted-interfaces>
</multicast>
<tcp-ip enabled="false">
<required-member>192.168.1.104</required-member>
<member>192.168.1.104</member>
<members>192.168.1.105,192.168.1.106</members>
</tcp-ip>
<aws enabled="false">
<access-key>my-access-key</access-key>
<secret-key>my-secret-key</secret-key>
<region>us-west-1</region>
<host-header>ec2.amazonaws.com</host-header>
<security-group-name>hazelcast-sg</security-group-name>
<tag-key>type</tag-key>
<tag-value>hz-members</tag-value>
</aws>
<discovery-strategies>
<discovery-strategy ... />
</discovery-strategies>
</join>
<network>
Programmatic:
Config config = new Config();
NetworkConfig network = config.getNetworkConfig();
JoinConfig join = network.getJoin();
join.getMulticastConfig().setEnabled( false )
.addTrustedInterface( "192.168.1.102" );
join.getTcpIpConfig().addMember( "10.45.67.32" ).addMember( "10.45.67.100" )
.setRequiredMember( "192.168.10.100" ).setEnabled( true );
The join
element has the following sub-elements and attributes.
The multicast
element includes parameters to fine tune the multicast join mechanism.
enabled
: Specifies whether the multicast discovery is enabled or not, true
or false
.multicast-group
: The multicast group IP address. Specify it when you want to create clusters within the same network. Values can be between 224.0.0.0 and 239.255.255.255. Default value is 224.2.2.3.multicast-port
: The multicast socket port that the Hazelcast member listens to and sends discovery messages through. Default value is 54327.multicast-time-to-live
: Time-to-live value for multicast packets sent out to control the scope of multicasts. See more information here.multicast-timeout-seconds
: Only when the members are starting up, this timeout (in seconds) specifies the period during which a member waits for a multicast response from another member. For example, if you set it as 60 seconds, each member will wait for 60 seconds until a leader member is selected. Its default value is 2 seconds. trusted-interfaces
: Includes IP addresses of trusted members. When a member wants to join to the cluster, its join request will be rejected if it is not a trusted member. You can give an IP addresses range using the wildcard (*) on the last digit of IP address (e.g. 192.168.1.* or 192.168.1.100-110).The tcp-ip
element includes parameters to fine tune the TCP/IP join mechanism.
enabled
: Specifies whether the TCP/IP discovery is enabled or not. Values can be true
or false
.required-member
: IP address of the required member. Cluster will only formed if the member with this IP address is found.member
: IP address(es) of one or more well known members. Once members are connected to these well known ones, all member addresses will be communicated with each other. You can also give comma separated IP addresses using the members
element.NOTE: tcp-ip
element also accepts the interface
parameter. Please refer to the Interfaces element description.
connection-timeout-seconds
: Defines the connection timeout. This is the maximum amount of time Hazelcast is going to try to connect to a well known member before giving up. Setting it to a too low value could mean that a member is not able to connect to a cluster. Setting it to a too high value means that member startup could slow down because of longer timeouts (e.g. when a well known member is not up). Increasing this value is recommended if you have many IPs listed and the members cannot properly build up the cluster. Its default value is 5.The aws
element includes parameters to allow the members to form a cluster on the Amazon EC2 environment.
enabled
: Specifies whether the EC2 discovery is enabled or not, true
or false
.access-key
, secret-key
: Access and secret keys of your account on EC2.region
: The region where your members are running. Default value is us-east-1
. You need to specify this if the region is other than the default one.host-header
: The URL that is the entry point for a web service. It is optional.security-group-name
: Name of the security group you specified at the EC2 management console. It is used to narrow the Hazelcast members to be within this group. It is optional.tag-key
, tag-value
: To narrow the members in the cloud down to only Hazelcast members, you can set these parameters as the ones you specified in the EC2 console. They are optional.connection-timeout-seconds
: The maximum amount of time Hazelcast will try to connect to a well known member before giving up. Setting this value too low could mean that a member is not able to connect to a cluster. Setting the value too high means that member startup could slow down because of longer timeouts (for example, when a well known member is not up). Increasing this value is recommended if you have many IPs listed and the members cannot properly build up the cluster. Its default value is 5.NOTE: If you are using a cloud provider other than AWS, you can use the programmatic configuration to specify a TCP/IP cluster. The members will need to be retrieved from that provider (e.g. JClouds).
The discovery-strategies
element configures internal or external discovery strategies based on the Hazelcast Discovery SPI. For further information, please refer to the Discovery SPI section and the vendor documentation of the used discovery strategy.
To make sure EC2 instances are found correctly, you can use the AWSClient
class. It determines the private IP addresses of EC2 instances to be connected. Give the AWSClient
class the values for the parameters that you specified in the aws
element, as shown below. You will see whether your EC2 instances are found.
public static void main( String[] args )throws Exception{
AwsConfig config = new AwsConfig();
config.setSecretKey( ... ) ;
config.setSecretKey( ... );
config.setRegion( ... );
config.setSecurityGroupName( ... );
config.setTagKey( ... );
config.setTagValue( ... );
config.setEnabled( true );
AWSClient client = new AWSClient( config );
List<String> ipAddresses = client.getPrivateIpAddresses();
System.out.println( "addresses found:" + ipAddresses );
for ( String ip: ipAddresses ) {
System.out.println( ip );
}
}
You can specify which network interfaces that Hazelcast should use. Servers mostly have more than one network interface, so you may want to list the valid IPs. Range characters ('*' and '-') can be used for simplicity. For instance, 10.3.10.* refers to IPs between 10.3.10.0 and 10.3.10.255. Interface 10.3.10.4-18 refers to IPs between 10.3.10.4 and 10.3.10.18 (4 and 18 included). If network interface configuration is enabled (it is disabled by default) and if Hazelcast cannot find an matching interface, then it will print a message on the console and will not start on that member.
The following are example configurations.
Declarative:
<hazelcast>
...
<network>
...
<interfaces enabled="true">
<interface>10.3.16.*</interface>
<interface>10.3.10.4-18</interface>
<interface>192.168.1.3</interface>
</interfaces>
</network>
...
</hazelcast>
Programmatic:
Config config = new Config();
NetworkConfig network = config.getNetworkConfig();
InterfacesConfig interface = network.getInterfaces();
interface.setEnabled( true )
.addInterface( "192.168.1.3" );
Hazelcast supports IPv6 addresses seamlessly (This support is switched off by default, please see the note at the end of this section).
All you need is to define IPv6 addresses or interfaces in network configuration. The only current limitation is that you cannot define wildcard IPv6 addresses in the TCP/IP join configuration (tcp-ip
element). Interfaces configuration does not have this limitation, you can configure wildcard IPv6 interfaces in the same way as IPv4 interfaces.
<hazelcast>
...
<network>
<port auto-increment="true">5701</port>
<join>
<multicast enabled="false">
<multicast-group>FF02:0:0:0:0:0:0:1</multicast-group>
<multicast-port>54327</multicast-port>
</multicast>
<tcp-ip enabled="true">
<member>[fe80::223:6cff:fe93:7c7e]:5701</member>
<interface>192.168.1.0-7</interface>
<interface>192.168.1.*</interface>
<interface>fe80:0:0:0:45c5:47ee:fe15:493a</interface>
</tcp-ip>
</join>
<interfaces enabled="true">
<interface>10.3.16.*</interface>
<interface>10.3.10.4-18</interface>
<interface>fe80:0:0:0:45c5:47ee:fe15:*</interface>
<interface>fe80::223:6cff:fe93:0-5555</interface>
</interfaces>
...
</network>
...
</hazelcast>
JVM has two system properties for setting the preferred protocol stack (IPv4 or IPv6) as well as the preferred address family types (inet4 or inet6). On a dual stack machine, IPv6 stack is preferred by default, you can change this through the java.net.preferIPv4Stack=<true|false>
system property. When querying name services, JVM prefers IPv4 addresses over IPv6 addresses and will return an IPv4 address if possible. You can change this through java.net.preferIPv6Addresses=<true|false>
system property.
Also see additional details on IPv6 support in Java.
NOTE: IPv6 support has been switched off by default, since some platforms have issues using the IPv6 stack. Some other platforms such as Amazon AWS have no support at all. To enable IPv6 support, just set configuration property hazelcast.prefer.ipv4.stack
to false. Please refer to the System Properties section for details.
As mentioned in the Overview section, Hazelcast offers distributed implementations of Java interfaces. Below is the list of these implementations with links to the corresponding sections in this manual.
Standard utility collections
java.util.Map
. It lets you read from and write to a Hazelcast map with methods such as get
and put
.java.util.concurrent.BlockingQueue
. You can add an item in one member and remove it from another one.java.util.Set
. It does not allow duplicate elements and does not preserve their order.Topic is the distributed mechanism for publishing messages that are delivered to multiple subscribers. It is also known as the publish/subscribe (pub/sub) messaging model. Please see the Topic section for more information. Hazelcast also has a structure called Reliable Topic which uses the same interface of Hazelcast Topic. The difference is that it is backed up by the Ringbuffer data structure. Please see the Reliable Topic section.
Concurrency utilities
java.util.concurrent.locks.Lock
. When you use lock, the critical section that Hazelcast Lock guards is guaranteed to be executed by only one thread in the entire cluster.java.util.concurrent.Semaphore
. When performing concurrent activities, semaphores offer permits to control the thread counts.java.util.concurrent.atomic.AtomicLong
. Most of AtomicLong's operations are available. However, these operations involve remote calls and hence their performances differ from AtomicLong, due to being distributed.java.util.concurrent.atomic.AtomicReference
. When you need to deal with a reference in a distributed environment, you can use Hazelcast AtomicReference. AtomicLong.incrementAndGet()
.java.util.concurrent.CountDownLatch
. Hazelcast CountDownLatch is a gate keeper for concurrent activities. It enables the threads to wait for other threads to complete their operations.Common Features of all Hazelcast Data Structures
Here is an example of how you can retrieve existing data structure instances (map, queue, set, lock, topic, etc.) and how you can listen for instance events, such as an instance being created or destroyed.
import java.util.Collection;
import com.hazelcast.config.Config;
import com.hazelcast.core.*;
public class Sample implements DistributedObjectListener {
public static void main(String[] args) {
Sample sample = new Sample();
Config config = new Config();
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(config);
hazelcastInstance.addDistributedObjectListener(sample);
Collection<DistributedObject> distributedObjects = hazelcastInstance.getDistributedObjects();
for (DistributedObject distributedObject : distributedObjects) {
System.out.println(distributedObject.getName() + "," + distributedObject.getId());
}
}
@Override
public void distributedObjectCreated(DistributedObjectEvent event) {
DistributedObject instance = event.getDistributedObject();
System.out.println("Created " + instance.getName() + "," + instance.getId());
}
@Override
public void distributedObjectDestroyed(DistributedObjectEvent event) {
DistributedObject instance = event.getDistributedObject();
System.out.println("Destroyed " + instance.getName() + "," + instance.getId());
}
}
Hazelcast Map (IMap
) extends the interface java.util.concurrent.ConcurrentMap
and hence java.util.Map
. It is the distributed implementation of Java map. You can perform operations like reading and writing from/to a Hazelcast map with the well known get and put methods.
Hazelcast will partition your map entries and almost evenly distribute them onto all Hazelcast members. Each member carries approximately "(1/n *
total-data) + backups", n being the number of members in the cluster. For example, if you have a member with 1000 objects to be stored in the cluster, and then you start a second member, each member will both store 500 objects and back up the 500 objects in the other member.
Let's create a Hazelcast instance and fill a map named Capitals
with key-value pairs using the following code. Use the HazelcastInstance getMap
method to get the map, then use the map put
method to put an entry into the map.
public class FillMapMember {
public static void main( String[] args ) {
HazelcastInstance hzInstance = Hazelcast.newHazelcastInstance();
Map<String, String> capitalcities = hzInstance.getMap( "capitals" );
capitalcities.put( "1", "Tokyo" );
capitalcities.put( "2", "Paris” );
capitalcities.put( "3", "Washington" );
capitalcities.put( "4", "Ankara" );
capitalcities.put( "5", "Brussels" );
capitalcities.put( "6", "Amsterdam" );
capitalcities.put( "7", "New Delhi" );
capitalcities.put( "8", "London" );
capitalcities.put( "9", "Berlin" );
capitalcities.put( "10", "Oslo" );
capitalcities.put( "11", "Moscow" );
...
...
capitalcities.put( "120", "Stockholm" )
}
}
When you run this code, a cluster member is created with a map whose entries are distributed across the members' partitions. See the below illustration. For now, this is a single member cluster.
NOTE: Please note that some of the partitions will not contain any data entries since we only have 120 objects and the partition count is 271 by default. This count is configurable and can be changed using the system property hazelcast.partition.count
. Please see the System Properties section.
Now let's create a second member by running the above code again. This will create a cluster with two members. This is also where backups of entries are created--remember the backup partitions mentioned in the Hazelcast Overview section. The following illustration shows two members and how the data and its backup is distributed.
As you see, when a new member joins the cluster, it takes ownership and loads some of the data in the cluster. Eventually, it will carry almost "(1/n *
total-data) + backups" of the data, reducing the load on other members.
HazelcastInstance::getMap
returns an instance of com.hazelcast.core.IMap
which extends
the java.util.concurrent.ConcurrentMap
interface. Methods like
ConcurrentMap.putIfAbsent(key,value)
and ConcurrentMap.replace(key,value)
can be used
on the distributed map, as shown in the example below.
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import java.util.concurrent.ConcurrentMap;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Customer getCustomer( String id ) {
ConcurrentMap<String, Customer> customers = hazelcastInstance.getMap( "customers" );
Customer customer = customers.get( id );
if (customer == null) {
customer = new Customer( id );
customer = customers.putIfAbsent( id, customer );
}
return customer;
}
public boolean updateCustomer( Customer customer ) {
ConcurrentMap<String, Customer> customers = hazelcastInstance.getMap( "customers" );
return ( customers.replace( customer.getId(), customer ) != null );
}
public boolean removeCustomer( Customer customer ) {
ConcurrentMap<String, Customer> customers = hazelcastInstance.getMap( "customers" );
return customers.remove( customer.getId(), customer );
}
All ConcurrentMap
operations such as put
and remove
might wait if the key is locked by another thread in the local or remote JVM. But, they will eventually return with success. ConcurrentMap
operations never throw a java.util.ConcurrentModificationException
.
Also see:
Hazelcast distributes map entries onto multiple cluster members (JVMs). Each member holds some portion of the data.
Distributed maps have one backup by default. If a member goes down, you do not lose data. Backup operations are synchronous, so when a map.put(key, value)
returns, it is guaranteed that the map entry is replicated to one other member. For the reads, it is also guaranteed that map.get(key)
returns the latest value of the entry. Consistency is strictly enforced.
To provide data safety, Hazelcast allows you to specify the number of backup copies you want to have. That way, data on a cluster member will be copied onto other member(s).
To create synchronous backups, select the number of backup copies using the backup-count
property.
<hazelcast>
<map name="default">
<backup-count>1</backup-count>
</map>
</hazelcast>
When this count is 1, a map entry will have its backup on one other member in the cluster. If you set it to 2, then a map entry will have its backup on two other members. You can set it to 0 if you do not want your entries to be backed up, e.g., if performance is more important than backing up. The maximum value for the backup count is 6.
Hazelcast supports both synchronous and asynchronous backups. By default, backup operations are synchronous and configured with backup-count
. In this case, backup operations block operations until backups are successfully copied to backup members (or deleted from backup members in case of remove) and acknowledgements are received. Therefore, backups are updated before a put
operation is completed. Sync backup operations have a blocking cost which may lead to latency issues.
Asynchronous backups, on the other hand, do not block operations. They are fire & forget and do not require acknowledgements; the backup operations are performed at some point in time.
To create asynchronous backups, select the number of async backups with the async-backup-count
property. An example is shown below.
<hazelcast>
<map name="default">
<backup-count>0</backup-count>
<async-backup-count>1</async-backup-count>
</map>
</hazelcast>
NOTE: Backups increase memory usage since they are also kept in memory.
NOTE: A map can have both sync and aysnc backups at the same time.
By default, Hazelcast has one sync backup copy. If backup-count
is set to more than 1, then each member will carry both owned entries and backup copies of other members. So for the map.get(key)
call, it is possible that the calling member has a backup copy of that key. By default, map.get(key)
will always read the value from the actual owner of the key for consistency.
To enable backup reads (read local backup entries), set the value of the read-backup-data
property to true. Its default value is false for strong consistency. Enabling backup reads can improve performance.
<hazelcast>
<map name="default">
<backup-count>0</backup-count>
<async-backup-count>1</async-backup-count>
<read-backup-data>true</read-backup-data>
</map>
</hazelcast>
This feature is available when there is at least one sync or async backup.
Please note that if you are performing a read from a backup, you should take into account that your hits to the keys in the backups are not reflected as hits to the original keys on the primary members. This has an impact on IMap's maximum idle seconds or time-to-live seconds expiration. Therefore, even though there is a hit on a key in backups, your original key on the primary member may expire.
NOTE: Starting with Hazelcast 3.7, Hazelcast Map uses a new eviction mechanism which is based on the sampling of entries. Please see the Eviction Algorithm section for details.
Unless you delete the map entries manually or use an eviction policy, they will remain in the map. Hazelcast supports policy-based eviction for distributed maps. Currently supported policies are LRU (Least Recently Used) and LFU (Least Frequently Used).
Hazelcast Map performs eviction based on partitions. For example, when you specify a size using the PER_NODE
attribute for max-size
(please see Configuring Map Eviction), Hazelcast internally calculates the maximum size for every partition. Hazelcast uses the following equation to calculate the maximum size of a partition:
partition maximum size = max-size * member-count / partition-count
The eviction process starts according to this calculated partition maximum size when you try to put an entry. When entry count in that partition exceeds partition maximum size, eviction starts on that partition.
Assume that you have the following figures as examples:
max-size
(PER_NODE): 20000The total number of entries here is 20000 (partition count * entry count for each partition). This means you are at the eviction threshold since you set the max-size
to 20000. When you try to put an entry
max-size
);As a result of this eviction process, when you check the size of your map, it is 19999. After this eviction, subsequent put operations will not trigger the next eviction until the map size is again close to the max-size
.
NOTE: The above scenario is simply an example that describes how the eviction process works. Hazelcast finds the most optimum number of entries to be evicted according to your cluster size and selected policy.
The following is an example declarative configuration for map eviction.
<hazelcast>
<map name="default">
...
<time-to-live-seconds>0</time-to-live-seconds>
<max-idle-seconds>0</max-idle-seconds>
<eviction-policy>LRU</eviction-policy>
<max-size policy="PER_NODE">5000</max-size>
...
</map>
</hazelcast>
Let's describe each element:
time-to-live
. Maximum time in seconds for each entry to stay in the map. If it is not 0, entries that are older than this time and not updated for this time are evicted automatically. Valid values are integers between 0 and Integer.MAX VALUE
. Default value is 0, which means infinite. If it is not 0, entries are evicted regardless of the set eviction-policy
.max-idle-seconds
. Maximum time in seconds for each entry to stay idle in the map. Entries that are idle for more than this time are evicted automatically. An entry is idle if no get
, put
, EntryProcessor.process
or containsKey
is called. Valid values are integers between 0 and Integer.MAX VALUE
. Default value is 0, which means infinite.eviction-policy
. Valid values are described below.
max-size
will be ignored. You still can combine it with time-to-live-seconds
and max-idle-seconds
.max-size
. Maximum size of the map. When maximum size is reached, the map is evicted based on the policy defined. Valid values are integers between 0 and Integer.MAX VALUE
. Default value is 0. If you want max-size
to work, set the eviction-policy
property to a value other than NONE. Its attributes are described below.
PER_NODE
. Maximum number of map entries in each cluster member. This is the default policy. If you use this option, please note that you cannot set the max-size
to a value lower than the partition count (which is 271 by default).
<max-size policy="PER_NODE">5000</max-size>
PER_PARTITION
. Maximum number of map entries within each partition. Storage size depends on the partition count in a cluster member. This attribute should not be used often. For instance, avoid using this attribute with a small cluster. If the cluster is small, it will be hosting more partitions, and therefore map entries, than that of a larger cluster. Thus, for a small cluster, eviction of the entries will decrease performance (the number of entries is large).
<max-size policy="PER_PARTITION">27100</max-size>
USED_HEAP_SIZE
. Maximum used heap size in megabytes per map for each Hazelcast instance. Please note that this policy does not work when in-memory format is set to OBJECT
, since the memory footprint cannot be determined when data is put as OBJECT
.
<max-size policy="USED_HEAP_SIZE">4096</max-size>
USED_HEAP_PERCENTAGE
. Maximum used heap size percentage per map for each Hazelcast instance. If, for example, a JVM is configured to have 1000 MB and this value is 10, then the map entries will be evicted when used heap size exceeds 100 MB. Please note that this policy does not work when in-memory format is set to OBJECT
, since the memory footprint cannot be determined when data is put as OBJECT
.
<max-size policy="USED_HEAP_PERCENTAGE">10</max-size>
FREE_HEAP_SIZE
. Minimum free heap size in megabytes for each JVM.
<max-size policy="FREE_HEAP_SIZE">512</max-size>
FREE_HEAP_PERCENTAGE
. Minimum free heap size percentage for each JVM. If, for example, a JVM is configured to have 1000 MB and this value is 10, then the map entries will be evicted when free heap size is below 100 MB.
<max-size policy="FREE_HEAP_PERCENTAGE">10</max-size>
USED_NATIVE_MEMORY_SIZE
. (Hazelcast Enterprise HD) Maximum used native memory size in megabytes per map for each Hazelcast instance.
<max-size policy="USED_NATIVE_MEMORY_SIZE">1024</max-size>
USED_NATIVE_MEMORY_PERCENTAGE
. (Hazelcast Enterprise HD) Maximum used native memory size percentage per map for each Hazelcast instance.
<max-size policy="USED_NATIVE_MEMORY_PERCENTAGE">65</max-size>
FREE_NATIVE_MEMORY_SIZE
. (Hazelcast Enterprise HD) Minimum free native memory size in megabytes for each Hazelcast instance.
<max-size policy="FREE_NATIVE_MEMORY_SIZE">256</max-size>
FREE_NATIVE_MEMORY_PERCENTAGE
. (Hazelcast Enterprise HD) Minimum free native memory size percentage for each Hazelcast instance.
<max-size policy="FREE_NATIVE_MEMORY_PERCENTAGE">5</max-size>
NOTE: As of Hazelcast 3.7, the elements eviction-percentage
and min-eviction-check-millis
are deprecated. They will be ignored if configured since map eviction is based on the sampling of entries. Please see the Eviction Algorithm section for details.
<map name="documents">
<max-size policy="PER_NODE">10000</max-size>
<eviction-policy>LRU</eviction-policy>
<max-idle-seconds>60</max-idle-seconds>
</map>
In the above example, documents
map starts to evict its entries from a member when the map size exceeds 10000 in that member. Then the entries least recently used will be evicted. The entries not used for more than 60 seconds will be evicted as well.
And the following is an example eviction configuration for a map having NATIVE
as the in-memory format:
<map name="nativeMap*">
<in-memory-format>NATIVE</in-memory-format>
<eviction-policy>LFU</eviction-policy>
<max-size policy="USED_NATIVE_MEMORY_PERCENTAGE">99</max-size>
</map>
The eviction policies and configurations explained above apply to all the entries of a map. The entries that meet the specified eviction conditions are evicted.
You may also want to evict some specific map entries. To do this, you can use the ttl
and timeunit
parameters of the method map.put()
. An example code line is given below.
myMap.put( "1", "John", 50, TimeUnit.SECONDS )
The map entry with the key "1" will be evicted 50 seconds after it is put into myMap
.
To evict all keys from the map except the locked ones, use the method evictAll()
. If a MapStore is defined for the map, deleteAll
is not called by evictAll
. If you want to call the method deleteAll
, use clear()
.
An example is given below.
public class EvictAll {
public static void main(String[] args) {
final int numberOfKeysToLock = 4;
final int numberOfEntriesToAdd = 1000;
HazelcastInstance node1 = Hazelcast.newHazelcastInstance();
HazelcastInstance node2 = Hazelcast.newHazelcastInstance();
IMap<Integer, Integer> map = node1.getMap(EvictAll.class.getCanonicalName());
for (int i = 0; i < numberOfEntriesToAdd; i++) {
map.put(i, i);
}
for (int i = 0; i < numberOfKeysToLock; i++) {
map.lock(i);
}
// should keep locked keys and evict all others.
map.evictAll();
System.out.printf("# After calling evictAll...\n");
System.out.printf("# Expected map size\t: %d\n", numberOfKeysToLock);
System.out.printf("# Actual map size\t: %d\n", map.size());
}
}
NOTE: Only EVICT_ALL event is fired for any registered listeners.
NOTE: This section is valid for Hazelcast 3.7 and higher releases.
Apart from the policies such as LRU and LFU, which Hazelcast provides out of the box, you can develop and use your own eviction policy.
To achieve this, you need to provide an implementation of MapEvictionPolicy
as in the following OddEvictor
example:
public class MapCustomEvictionPolicy {
public static void main(String[] args) {
Config config = new Config();
config.getMapConfig("test")
.setMapEvictionPolicy(new OddEvictor())
.getMaxSizeConfig()
.setMaxSizePolicy(PER_NODE).setSize(10000);
HazelcastInstance instance = Hazelcast.newHazelcastInstance(config);
IMap<Integer, Integer> map = instance.getMap("test");
final Queue<Integer> oddKeys = new ConcurrentLinkedQueue<Integer>();
final Queue<Integer> evenKeys = new ConcurrentLinkedQueue<Integer>();
map.addEntryListener(new EntryEvictedListener<Integer, Integer>() {
@Override
public void entryEvicted(EntryEvent<Integer, Integer> event) {
Integer key = event.getKey();
if (key % 2 == 0) {
evenKeys.add(key);
} else {
oddKeys.add(key);
}
}
}, false);
// Wait some more time to receive evicted events.
parkNanos(SECONDS.toNanos(5));
for (int i = 0; i < 15000; i++) {
map.put(i, i);
}
String msg = "IMap uses sampling based eviction. After eviction is completed, we are expecting " +
"number of evicted-odd-keys should be greater than number of evicted-even-keys" +
"\nNumber of evicted-odd-keys = %d, number of evicted-even-keys = %d";
out.println(format(msg, oddKeys.size(), evenKeys.size()));
instance.shutdown();
}
/**
* Odd evictor tries to evict odd keys first.
*/
private static class OddEvictor extends MapEvictionPolicy {
@Override
public int compare(EntryView o1, EntryView o2) {
Integer key = (Integer) o1.getKey();
if (key % 2 != 0) {
return -1;
}
return 1;
}
}
}
Then you can enable your policy by setting it via the method MapConfig#setMapEvictionPolicy
programmatically or via XML declaratively. Following is the example declarative configuration for the eviction policy OddEvictor
implemented above:
<map name="test">
...
<map-eviction-policy-class-name>com.package.OddEvictor</map-eviction-policy-class-name>
....
</map>
If you Hazelcast with Spring, you can enable your policy as shown below.
<hz:map name="test">
<hz:map-eviction-policy class-name="com.package.OddEvictor"/>
</hz:map>
IMap (and a few other Hazelcast data structures, such as ICache) has an in-memory-format
configuration option. By default, Hazelcast stores data into memory in binary (serialized) format. Sometimes it can be efficient to store the entries in their object form, especially in cases of local processing, such as entry processor and queries.
To set how the data will be stored in memory, set in-memory-format
in the configuration. You have the following format options:
BINARY
(default). The data will be stored in serialized binary format. You can use this option if you mostly perform regular map operations, such as put
and get
.
OBJECT
. The data will be stored in deserialized form. This configuration is good for maps where entry processing and queries form the majority of all operations and the objects are complex, making the serialization cost comparatively high. By storing objects, entry processing will not contain the deserialization cost.
NATIVE
: (Hazelcast Enterprise HD) This option is used to enable the map to use Hazelcast's High-Density Memory Store. Please refer to the Using High-Density Memory Store with Map section.
Regular operations like get
rely on the object instance. When the OBJECT
format is used and a get
is performed, the map does not return the stored instance, but creates a clone. Therefore, this whole get
operation first includes a serialization on the member owning the instance, and then a deserialization on the member calling the instance. When the BINARY
format is used, only a deserialization is required; BINARY
is faster.
Similarly, a put
operation is faster when the BINARY
format is used. If the format was OBJECT
, the map would create a clone of the instance, and there would first be a serialization and then a deserialization. When BINARY is used, only a deserialization is needed.
NOTE: If a value is stored in OBJECT
format, a change on a returned value does not affect the stored instance. In this case, the returned instance is not the actual one but a clone. Therefore, changes made on an object after it is returned will not reflect on the actual stored data. Similarly, when a value is written to a map and the value is stored in OBJECT
format, it will be a copy of the put
value. Therefore, changes made on the object after it is stored will not reflect on the stored data.
Hazelcast Enterprise HD
Hazelcast instances are Java programs. In case of BINARY
and OBJECT
in-memory formats, Hazelcast stores your distributed data into the heap of its server instances. Java heap is subject to garbage collection (GC). In case of larger heaps, garbage collection might cause your application to pause for tens of seconds (even minutes for really large heaps), badly affecting your application performance and response times.
As the data gets bigger, you either run the application with larger heap, which would result in longer GC pauses or run multiple instances with smaller heap which can turn into an operational nightmare if the number of such instances becomes very high.
To overcome this challenge, Hazelcast offers High-Density Memory Store for your maps. You can configure your map to use High-Density Memory Store by setting the in-memory format to NATIVE
. The following snippet is the declarative configuration example.
<map name="nativeMap*">
<in-memory-format>NATIVE</in-memory-format>
</map>
Keep in mind that you should have already enabled the High-Density Memory Store usage for your cluster. Please see Configuring High-Density Memory Store section.
Note that the eviction mechanism is different for NATIVE
in-memory format.
The new eviction algorithm for map with High-Density Memory Store is similar to that of JCache with High-Density Memory Store and is described here.
Eviction percentage has no effect.
<map name="nativeMap*">
<in-memory-format>NATIVE</in-memory-format>
<eviction-percentage>25</eviction-percentage> <-- NO IMPACT with NATIVE
</map>
These IMap eviction policies for max-size
cannot be used: FREE_HEAP_PERCENTAGE
, FREE_HEAP_SIZE
, USED_HEAP_PERCENTAGE
, USED_HEAP_SIZE
.
Near cache eviction configuration is also different for NATIVE
in-memory format.
For a near cache configuration with in-memory format set to BINARY
:
<map name="nativeMap*">
<near-cache>
<in-memory-format>BINARY</in-memory-format>
<max-size>10000</max-size> <-- NO IMPACT with NATIVE
<eviction-policy>LFU</eviction-policy> <-- NO IMPACT with NATIVE
</near-cache>
</map>
the equivalent configuration for NATIVE
in-memory format would be similar to the following:
<map name="nativeMap*">
<near-cache>
<in-memory-format>NATIVE</in-memory-format>
<eviction size="10000" eviction-policy="LFU" max-size-policy="USED_NATIVE_MEMORY_SIZE"/> <-- Correct configuration with NATIVE
</near-cache>
</map>
Near cache eviction policy ENTRY_COUNT
cannot be used for max-size-policy
.
RELATED INFORMATION
Please refer to the High-Density Memory Store section for more information.
Hazelcast allows you to load and store the distributed map entries from/to a persistent data store such as a relational database. To do this, you can use Hazelcast's MapStore
and MapLoader
interfaces.
When you provide a MapLoader
implementation and request an entry (IMap.get()
) that does not exist in memory, MapLoader
's load
or loadAll
methods will load that entry from the data store. This loaded entry is placed into the map and will stay there until it is removed or evicted.
When a MapStore
implementation is provided, an entry is also put into a user defined data store.
NOTE: Data store needs to be a centralized system that is accessible from all Hazelcast members. Persistence to a local file system is not supported.
NOTE: Also note that the MapStore
interface extends the MapLoader
interface as you can see in the interface code.
Following is a MapStore
example.
public class PersonMapStore implements MapStore<Long, Person> {
private final Connection con;
public PersonMapStore() {
try {
con = DriverManager.getConnection("jdbc:hsqldb:mydatabase", "SA", "");
con.createStatement().executeUpdate(
"create table if not exists person (id bigint, name varchar(45))");
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public synchronized void delete(Long key) {
System.out.println("Delete:" + key);
try {
con.createStatement().executeUpdate(
format("delete from person where id = %s", key));
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public synchronized void store(Long key, Person value) {
try {
con.createStatement().executeUpdate(
format("insert into person values(%s,'%s')", key, value.name));
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public synchronized void storeAll(Map<Long, Person> map) {
for (Map.Entry<Long, Person> entry : map.entrySet())
store(entry.getKey(), entry.getValue());
}
public synchronized void deleteAll(Collection<Long> keys) {
for (Long key : keys) delete(key);
}
public synchronized Person load(Long key) {
try {
ResultSet resultSet = con.createStatement().executeQuery(
format("select name from person where id =%s", key));
try {
if (!resultSet.next()) return null;
String name = resultSet.getString(1);
return new Person(name);
} finally {
resultSet.close();
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
public synchronized Map<Long, Person> loadAll(Collection<Long> keys) {
Map<Long, Person> result = new HashMap<Long, Person>();
for (Long key : keys) result.put(key, load(key));
return result;
}
public Iterable<Long> loadAllKeys() {
return null;
}
}
NOTE: During the initial loading process, MapStore uses a thread different from the partition threads that are used by the ExecutorService. After the initialization is completed, the map.get
method looks up any nonexistent value from the database in a partition thread, or the map.put
method looks up the database to return the previously associated value for a key also in a partition thread.
RELATED INFORMATION
For more MapStore/MapLoader code samples, please see here.
Hazelcast supports read-through, write-through, and write-behind persistence modes, which are explained in the subsections below.
If an entry does not exist in memory when an application asks for it, Hazelcast asks the loader implementation to load that entry from the data store. If the entry exists there, the loader implementation gets it, hands it to Hazelcast, and Hazelcast puts it into memory. This is read-through persistence mode.
MapStore
can be configured to be write-through by setting the write-delay-seconds
property to 0. This means the entries will be put to the data store synchronously.
In this mode, when the map.put(key,value)
call returns:
MapStore.store(key,value)
is successfully called so the entry is persisted.backup-count
is greater than 0).The same behavior goes for a map.remove(key)
call. The only difference is that MapStore.delete(key)
is called when the entry will be deleted.
If MapStore
throws an exception, then the exception will be propagated back to the original put
or remove
call in the form of RuntimeException
.
You can configure MapStore
as write-behind by setting the write-delay-seconds
property to a value bigger than 0. This means the modified entries will be put to the data store asynchronously after a configured delay.
NOTE: In write-behind mode, Hazelcast coalesces updates on a specific key by default, which means it applies only the last update on that key. However, you can set MapStoreConfig#setWriteCoalescing
to FALSE
and you can store all updates performed on a key to the data store.
NOTE: When you set MapStoreConfig#setWriteCoalescing
to FALSE
, after you reached per-node maximum write-behind-queue capacity, subsequent put operations will fail with ReachedMaxSizeException
. This exception will be thrown to prevent uncontrolled grow of write-behind queues. You can set per-node maximum capacity using the system property hazelcast.map.write.behind.queue.capacity
. Please refer to the System Properties section for information on this property and how to set the system properties.
In write-behind mode, when the map.put(key,value)
call returns:
backup-count
is greater than 0).write-delay-seconds
, it can be persisted with MapStore.store(key,value)
call.The same behavior goes for the map.remove(key)
, the only difference is that MapStore.delete(key)
is called when the entry will be deleted.
If MapStore
throws an exception, then Hazelcast tries to store the entry again. If the entry still cannot be stored, a log message is printed and the entry is re-queued.
For batch write operations, which are only allowed in write-behind mode, Hazelcast will call MapStore.storeAll(map)
and MapStore.deleteAll(collection)
to do all writes in a single call.
NOTE: If a map entry is marked as dirty, meaning that it is waiting to be persisted to the MapStore
in a write-behind scenario, the eviction process forces the entry to be stored. This way you have control over the number of entries waiting to be stored, and thus you can prevent a possible OutOfMemory exception.
NOTE: MapStore
or MapLoader
implementations should not use Hazelcast Map/Queue/MultiMap/List/Set operations. Your implementation should only work with your data store. Otherwise, you may get into deadlock situations.
Here is a sample configuration:
<hazelcast>
...
<map name="default">
...
<map-store enabled="true" initial-mode="LAZY">
<class-name>com.hazelcast.examples.DummyStore</class-name>
<write-delay-seconds>60</write-delay-seconds>
<write-batch-size>1000</write-batch-size>
<write-coalescing>true</write-coalescing>
</map-store>
</map>
</hazelcast>
The following are the descriptions of MapStore configuration elements and attributes:
class-name
: Name of the class implementing MapLoader and/or MapStore.write-delay-seconds
: Number of seconds to delay to call the MapStore.store(key, value). If the value is zero then it is write-through so MapStore.store(key, value) will be called as soon as the entry is updated. Otherwise it is write-behind so updates will be stored after write-delay-seconds value by calling Hazelcast.storeAll(map). Default value is 0.write-batch-size
: Used to create batch chunks when writing map store. In default mode, all map entries will be tried to be written in one go. To create batch chunks, the minimum meaningful value for write-batch-size is 2. For values smaller than 2, it works as in default mode.write-coalescing
: In write-behind mode, Hazelcast coalesces updates on a specific key by default; it applies only the last update on it. You can set this element to false
to store all updates performed on a key to the data store.enabled
: True to enable this map-store, false to disable. Default value is true.initial-mode
: Sets the initial load mode. LAZY is the default load mode, where load is asynchronous. EAGER means load is blocked till all partitions are loaded.A configuration can be applied to more than one map using wildcards (see Using Wildcards), meaning that the configuration is shared among the maps. But MapStore
does not know which entries to store when there is one configuration applied to multiple maps.
To store entries when there is one configuration applied to multiple maps, use Hazelcast's MapStoreFactory
interface. Using the MapStoreFactory
interface, MapStore
s for each map can be created when a wildcard configuration is used. Example code is shown below.
Config config = new Config();
MapConfig mapConfig = config.getMapConfig( "*" );
MapStoreConfig mapStoreConfig = mapConfig.getMapStoreConfig();
mapStoreConfig.setFactoryImplementation( new MapStoreFactory<Object, Object>() {
@Override
public MapLoader<Object, Object> newMapStore( String mapName, Properties properties ) {
return null;
}
});
To initialize the MapLoader
implementation with the given map name, configuration properties, and the Hazelcast instance, implement the MapLoaderLifecycleSupport
interface. This interface has the methods init()
and destroy()
as shown below.
public interface MapLoaderLifecycleSupport {
void init( HazelcastInstance hazelcastInstance, Properties properties, String mapName );
void destroy();
}
The method init()
initializes the MapLoader
implementation. Hazelcast calls this method when the map is first used on the Hazelcast instance. The MapLoader
implementation can initialize the required resources for implementing MapLoader
such as reading a configuration file or creating a database connection.
Hazelcast calls the method destroy()
before shutting down. You can override this method to cleanup the resources held by this MapLoader
implementation, such as closing the database connections.
To pre-populate the in-memory map when the map is first touched/used, use the MapLoader.loadAllKeys
API.
If MapLoader.loadAllKeys
returns NULL, then nothing will be loaded. Your MapLoader.loadAllKeys
implementation can return all or some of the keys. For example, you may select and return only the hot
keys. MapLoader.loadAllKeys
is the fastest way of pre-populating the map since Hazelcast will optimize the loading process by having each cluster member load its owned portion of the entries.
The InitialLoadMode
configuration parameter in the class MapStoreConfig has two values: LAZY
and EAGER
. If InitialLoadMode
is set to LAZY
, data is not loaded during the map creation. If it is set to EAGER
, all the data is loaded while the map is created, and everything becomes ready to use. Also, if you add indices to your map with the MapIndexConfig class or the addIndex
method, then InitialLoadMode
is overridden and MapStoreConfig
behaves as if EAGER
mode is on.
Here is the MapLoader
initialization flow:
getMap()
is first called from any member, initialization will start depending on the value of InitialLoadMode
. If it is set to EAGER
, initialization starts. If it is set to LAZY
, initialization does not start but data is loaded each time a partition loading completes.MapLoader.loadAllKeys()
to get all your keys on one of the members.MapLoader.loadAll(keys)
.IMap.putTransient(key,value)
. NOTE: If the load mode is LAZY
and the clear()
method is called (which triggers MapStore.deleteAll()
), Hazelcast will remove ONLY the loaded entries from your map and datastore. Since all the data is not loaded in this case (LAZY
mode), please note that there may still be entries in your datastore.
NOTE: The return type of loadAllKeys()
is changed from Set
to Iterable
with the release of Hazelcast 3.5. MapLoader implementations from previous releases are also supported and do not need to be adapted.
If the number of keys to load is large, it is more efficient to load them incrementally rather than loading them all at once. To support incremental loading, the MapLoader.loadAllKeys()
method returns an Iterable
which can be lazily populated with the results of a database query.
Hazelcast iterates over the Iterable
and, while doing so, sends out the keys to their respective owner members. The Iterator
obtained from MapLoader.loadAllKeys()
may also implement the Closeable
interface, in which case Iterator
is closed once the iteration is over. This is intended for releasing resources such as closing a JDBC result set.
The method loadAll
loads some or all keys into a data store in order to optimize the multiple load operations. The method has two signatures; the same method can take two different parameter lists. One signature loads the given keys and the other loads all keys. Please see the example code below.
public class LoadAll {
public static void main(String[] args) {
final int numberOfEntriesToAdd = 1000;
final String mapName = LoadAll.class.getCanonicalName();
final Config config = createNewConfig(mapName);
final HazelcastInstance node = Hazelcast.newHazelcastInstance(config);
final IMap<Integer, Integer> map = node.getMap(mapName);
populateMap(map, numberOfEntriesToAdd);
System.out.printf("# Map store has %d elements\n", numberOfEntriesToAdd);
map.evictAll();
System.out.printf("# After evictAll map size\t: %d\n", map.size());
map.loadAll(true);
System.out.printf("# After loadAll map size\t: %d\n", map.size());
}
}
In some scenarios, you may need to modify the object after storing it into the map store. For example, you can get an ID or version auto-generated by your database and then need to modify your object stored in the distributed map, but not to break the synchronization between the database and the data grid.
To post-process an object in the map store, implement the PostProcessingMapStore
interface to put the modified object into the distributed map. This will trigger an extra step of Serialization
, so use it only when needed. (This is only valid when using the write-through
map store configuration.)
Here is an example of post processing map store:
class ProcessingStore implements MapStore<Integer, Employee>, PostProcessingMapStore {
@Override
public void store( Integer key, Employee employee ) {
EmployeeId id = saveEmployee();
employee.setId( id.getId() );
}
}
NOTE: Please note that if you are using a post processing map store in combination with entry processors, post-processed values will not be carried to backups.
Map entries in Hazelcast are partitioned across the cluster. Suppose you read the key k
a number of times and k
is owned by another member in your cluster. Each map.get(k)
will be a remote operation, meaning lots of network trips. If you have a map that is read-mostly, then you should consider creating a near cache for the map so that reads can be much faster and consume less network traffic. These benefits do not come free; when using near cache, you should consider the following issues:
Near cache is highly recommended for the maps that are read-mostly. The following is the configuration example for map's near cache in the Hazelcast configuration file.
<hazelcast>
...
<map name="my-read-mostly-map">
...
<near-cache name="default">
<in-memory-format>BINARY</in-memory-format>
<max-size>5000</max-size>
<time-to-live-seconds>0</time-to-live-seconds>
<max-idle-seconds>60</max-idle-seconds>
<eviction-policy>LRU</eviction-policy>
<invalidate-on-change>true</invalidate-on-change>
<cache-local-entries>false</cache-local-entries>
</near-cache>
</map>
</hazelcast>
The element <near-cache>
has an optional attribute "name" whose default value is default
. Following are the descriptions of all configuration elements:
<max-size>
: Maximum size of the near cache. When this is reached, near cache is evicted based on the policy defined. Any integer between 0 and Integer.MAX_VALUE. 0 means Integer.MAX_VALUE
. Its default value is 0.<time-to-live-seconds>
: Maximum number of seconds for each entry to stay in the near cache. Entries that are older than this period are automatically evicted from the near cache. Regardless of the eviction policy used, <time-to-live-seconds>
still applies. Any integer between 0 and Integer.MAX_VALUE
. 0 means infinite. Its default value is 0.<max-idle-seconds>
: Maximum number of seconds each entry can stay in the near cache as untouched (not read). Entries that are not read more than this period are removed from the near cache. Any integer between 0 and Integer.MAX_VALUE
. 0 means Integer.MAX_VALUE
. Its default value is 0.<eviction-policy>
: Eviction policy configuration. Its default values is NONE. Available values are as follows:<invalidate-on-change>
: Specifies whether the cached entries are evicted when the entries are updated or removed. Its default value is true.<in-memory-format>
: Specifies in which format data will be stored in your near cache. Note that a map's in-memory format can be different from that of its near cache. Available values are as follows:<cache-local-entries>
: Specifies whether the local entries will be cached. It can be useful when in-memory format for near cache is different from that of the map. By default, it is disabled. NOTE: If you use High-Density Memory Store for your near cache, the elements <max-size>
and <eviction-policy>
do not have any impact. In this case, you need to use the element <eviction>
to specify the eviction behavior. Please refer to the Using High-Density Memory Store with Near Cache section.
Programmatically, you configure near cache by using the class NearCacheConfig. This class is used both in the cluster members and clients. In a client/server system, you must enable the near cache separately on the client, without you needing to configure it on the member. For information on how to create a near cache on a client (native Java client), please see Configuring Client Near Cache. Please note that near cache configuration is specific to the member or client itself, a map in a member may not have near cache configured while the same map in a client may have near cache configured.
If you are using near cache, you should take into account that your hits to the keys in near cache are not reflected as hits to the original keys on the primary members; this has an impact on IMap's maximum idle seconds or time-to-live seconds expiration. Therefore, even though there is a hit on a key in near cache, your original key on the primary member may expire.
NOTE: Near cache works only when you access data via map.get(k)
methods. Data returned using a predicate is not stored in the near cache.
NOTE: Even though lite members do not store any data for Hazelcast data structures, you can enable near cache on lite members for faster reads.
When you enable invalidations on near cache, either programmatically via NearCacheConfig#setInvalidateOnChange
or declaratively via <invalidate-on-change>true</invalidate-on-change>
, when entires are updated or removed from an entry in the underlying IMap, corresponding entries are removed from near caches to prevent stale reads.
This is called near cache invalidation.
Invalidation can be sent from members to client near caches or to member near caches, either individually or in batches. Default behavior is sending in batches. If there are lots of mutating operations such as put/remove on IMap, it is advised that you make invalidations in batches. This reduces the network traffic and keeps the eventing system less busy.
You can use the following system properties to configure the near cache invalidation:
hazelcast.map.invalidation.batch.enabled
: Enable or disable batching. Default value is true. When it is set to false, all invalidations are sent immediately.hazelcast.map.invalidation.batch.size
: Maximum number of invalidations in a batch. Default value is 100.hazelcast.map.invalidation.batchfrequency.seconds
: If we cannot reach the configured batch size, a background process sends invalidations periodically. Default value is 10 seconds.If there are a lot of clients or many mutating operations, batching should remain enabled and the batch size should be configured with the hazelcast.map.invalidation.batch.size
system property to a suitable value.
Hazelcast Enterprise HD
Hazelcast offers High-Density Memory Store for the near caches in your maps. You can enable your near cache to use the High-Density Memory Store by setting the in-memory format to NATIVE
. The following snippet is the declarative configuration example.
<hazelcast>
...
<map name="my-read-mostly-map">
...
<near-cache>
...
<in-memory-format>NATIVE</in-memory-format>
<eviction size="1000" max-size-policy="ENTRY_COUNT" eviction-policy="LFU"/>
...
</near-cache>
...
</map>
</hazelcast>
The element <eviction>
is used to specify the eviction behavior when you use High-Density Memory Store for your near cache. It has the following attributes:
size
: Maximum size (entry count) of the near cache.max-size-policy
: Maximum size policy for eviction of the near cache. Available values are as follows:eviction-policy
: Eviction policy configuration. Its default values is NONE. Available values are as follows:Keep in mind that you should have already enabled the High-Density Memory Store usage for your cluster. Please see the Configuring High-Density Memory Store section.
Note that a map and its near cache can independently use High-Density Memory Store. For example, if your map does not use High-Density Memory Store, its near cache can still use it.
Hazelcast Distributed Map (IMap) is thread-safe to meet your thread safety requirements. When these requirements increase or you want to have more control on the concurrency, consider the Hazelcast solutions described here.
Let's work on a sample case as shown below.
public class RacyUpdateMember {
public static void main( String[] args ) throws Exception {
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
IMap<String, Value> map = hz.getMap( "map" );
String key = "1";
map.put( key, new Value() );
System.out.println( "Starting" );
for ( int k = 0; k < 1000; k++ ) {
if ( k % 100 == 0 ) System.out.println( "At: " + k );
Value value = map.get( key );
Thread.sleep( 10 );
value.amount++;
map.put( key, value );
}
System.out.println( "Finished! Result = " + map.get(key).amount );
}
static class Value implements Serializable {
public int amount;
}
}
If the above code is run by more than one cluster member simultaneously, a race condition is likely. You can solve this condition with Hazelcast using either pessimistic locking or optimistic locking.
One way to solve the race issue is by using pessimistic locking--lock the map entry until you are finished with it.
To perform pessimistic locking, use the lock mechanism provided by the Hazelcast distributed map, i.e., the map.lock
and map.unlock
methods. See the below example code.
public class PessimisticUpdateMember {
public static void main( String[] args ) throws Exception {
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
IMap<String, Value> map = hz.getMap( "map" );
String key = "1";
map.put( key, new Value() );
System.out.println( "Starting" );
for ( int k = 0; k < 1000; k++ ) {
map.lock( key );
try {
Value value = map.get( key );
Thread.sleep( 10 );
value.amount++;
map.put( key, value );
} finally {
map.unlock( key );
}
}
System.out.println( "Finished! Result = " + map.get( key ).amount );
}
static class Value implements Serializable {
public int amount;
}
}
The IMap lock will automatically be collected by the garbage collector when the lock is released and no other waiting conditions exist on the lock.
The IMap lock is reentrant, but it does not support fairness.
Another way to solve the race issue is by acquiring a predictable Lock
object from Hazelcast. This way, every value in the map can be given a lock, or you can create a stripe of locks.
In Hazelcast, you can apply the optimistic locking strategy with the map's replace
method. This method compares values in object or data forms depending on the in-memory format configuration. If the values are equal, it replaces the old value with the new one. If you want to use your defined equals
method, in-memory-format
should be OBJECT
. Otherwise, Hazelcast serializes objects to BINARY
forms and compares them.
See the below example code.
public class OptimisticMember {
public static void main( String[] args ) throws Exception {
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
IMap<String, Value> map = hz.getMap( "map" );
String key = "1";
map.put( key, new Value() );
System.out.println( "Starting" );
for ( int k = 0; k < 1000; k++ ) {
if ( k % 10 == 0 ) System.out.println( "At: " + k );
for (; ; ) {
Value oldValue = map.get( key );
Value newValue = new Value( oldValue );
Thread.sleep( 10 );
newValue.amount++;
if ( map.replace( key, oldValue, newValue ) )
break;
}
}
System.out.println( "Finished! Result = " + map.get( key ).amount );
}
static class Value implements Serializable {
public int amount;
public Value() {
}
public Value( Value that ) {
this.amount = that.amount;
}
public boolean equals( Object o ) {
if ( o == this ) return true;
if ( !( o instanceof Value ) ) return false;
Value that = ( Value ) o;
return that.amount == this.amount;
}
}
}
NOTE: The above example code is intentionally broken.
The locking strategy you choose will depend on your locking requirements.
Optimistic locking is better for mostly read-only systems. It has a performance boost over pessimistic locking.
Pessimistic locking is good if there are lots of updates on the same key. It is more robust than optimistic locking from the perspective of data consistency.
In Hazelcast, use IExecutorService
to submit a task to a key owner, or to a member or members. This is the recommended way to perform task executions, rather than using pessimistic or optimistic locking techniques. IExecutorService
will have fewer network hops and less data over wire, and tasks will be executed very near to the data. Please refer to the Data Affinity section.
The ABA problem occurs in environments when a shared resource is open to change by multiple threads. Even if one thread sees the same value for a particular key in consecutive reads, it does not mean that nothing has changed between the reads. Another thread may change the value, do work, and change the value back, while the first thread thinks that nothing has changed.
To prevent these kind of problems, you can assign a version number and check it before any write to be sure that nothing has changed between consecutive reads. Although all the other fields will be equal, the version field will prevent objects from being seen as equal. This is the optimistic locking strategy, and it is used in environments that do not expect intensive concurrent changes on a specific key.
In Hazelcast, you can apply the optimistic locking strategy with the map replace
method.
Hazelcast keeps statistics about each map entry, such as creation time, last update time, last access time, number of hits, and version. To access the map entry statistics, use an IMap.getEntryView(key)
call. Here is an example.
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.EntryView;
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
EntryView entry = hz.getMap( "quotes" ).getEntryView( "1" );
System.out.println ( "size in memory : " + entry.getCost() );
System.out.println ( "creationTime : " + entry.getCreationTime() );
System.out.println ( "expirationTime : " + entry.getExpirationTime() );
System.out.println ( "number of hits : " + entry.getHits() );
System.out.println ( "lastAccessedTime: " + entry.getLastAccessTime() );
System.out.println ( "lastUpdateTime : " + entry.getLastUpdateTime() );
System.out.println ( "version : " + entry.getVersion() );
System.out.println ( "key : " + entry.getKey() );
System.out.println ( "value : " + entry.getValue() );
Please refer to the Listening for Map Events section.
You can listen to the modifications performed on specific map entries. You can think of it as an entry listener with predicates. Please see the Listening for Map Events section for information on how to add entry listeners to a map.
IMPORTANT: The default backwards-compatible event publishing strategy only publishes
UPDATED
events when map entries are updated to a value that matches the predicate with which the listener was registered.
This implies that when using the default event publishing strategy, your listener will not be notified about an entry whose
value is updated from one that matches the predicate to a new value that does not match the predicate.
Since version 3.7, when you configure Hazelcast members with property hazelcast.map.entry.filtering.natural.event.types
set to true
,
handling of entry updates conceptually treats value transition as entry, update or exit with regards to the predicate value space.
The following table compares how a listener is notified about an update to a map entry value under the default
backwards-compatible Hazelcast behavior (when property hazelcast.map.entry.filtering.natural.event.types
is not set or is set
to false
) versus when set to true
:
Default | hazelcast.map.entry.filtering.natural.event.types = true |
|
---|---|---|
When old value matches predicate, new value does not match predicate |
No event is delivered to entry listener | REMOVED event is delivered to entry listener |
When old value matches predicate, new value matches predicate |
UPDATED event is delivered to entry listener |
UPDATED event is delivered to entry listener |
When old value does not match predicate, new value does not match predicate |
No event is delivered to entry listener | No event is delivered to entry listener |
When old value does not match predicate, new value matches predicate |
UPDATED event is delivered to entry listener |
ADDED event is delivered to entry listener |
As an example, let's listen to the changes made on an employee with the surname "Smith". First, let's create the Employee
class.
import java.io.Serializable;
public class Employee implements Serializable {
private final String surname;
public Employee(String surname) {
this.surname = surname;
}
@Override
public String toString() {
return "Employee{" +
"surname='" + surname + '\'' +
'}';
}
}
Then, let's create a listener with predicate by adding a listener that tracks ADDED
, UPDATED
and REMOVED
entry events with the surname
predicate.
import com.hazelcast.core.*;
import com.hazelcast.query.SqlPredicate;
public class ListenerWithPredicate {
public static void main(String[] args) {
Config config = new Config();
config.setProperty("hazelcast.map.entry.filtering.natural.event.types", "true");
HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);
IMap<String, String> map = hz.getMap("map");
map.addEntryListener(new MyEntryListener(),
new SqlPredicate("surname=smith"), true);
System.out.println("Entry Listener registered");
}
static class MyEntryListener
implements EntryAddedListener<String, String>,
EntryUpdatedListener<String, String>,
EntryRemovedListener<String, String> {
@Override
public void entryAdded(EntryEvent<String, String> event) {
System.out.println("Entry Added:" + event);
}
@Override
public void entryRemoved(EntryEvent<String, String> event); {
System.out.println("Entry Removed:" + event);
}
@Override
public void entryUpdated(EntryEvent<String, String> event) {
System.out.println("Entry Updated:" + event);
}
}
}
And now, let's play with the employee "smith" and see how that employee will be listened to.
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.IMap;
public class Modify {
public static void main(String[] args) {
Config config = new Config();
config.setProperty("hazelcast.map.entry.filtering.natural.event.types", "true");
HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);
IMap<String, Employee> map = hz.getMap("map");
map.put("1", new Employee("smith"));
map.put("2", new Employee("jordan"));
System.out.println("done");
System.exit(0);
}
}
When you first run the class ListenerWithPredicate
and then run Modify
, you will see output similar to the listing below.
entryAdded:EntryEvent {Address[192.168.178.10]:5702} key=1,oldValue=null,
value=Person{name= smith }, event=ADDED, by Member [192.168.178.10]:5702
RELATED INFORMATION
Please refer to Continuous Query Cache for more information.
You can add intercept operations and execute your own business logic synchronously blocking the operations. You can change the returned value from a get
operation, change the value in put
, or cancel
operations by throwing an exception.
Interceptors are different from listeners. With listeners, you take an action after the operation has been completed. Interceptor actions are synchronous and you can alter the behavior of operation, change its values, or totally cancel it.
Map interceptors are chained, so adding the same interceptor multiple times to the same map can result in duplicate effects. This can easily happen when the interceptor is added to the map at member initialization, so that each member adds the same interceptor. When you add the interceptor in this way, be sure to implement the hashCode()
method to return the same value for every instance of the interceptor. It is not strictly necessary, but it is a good idea to also implement equals()
as this will ensure that the map interceptor can be removed reliably.
The IMap API has two methods for adding and removing an interceptor to the map: addInterceptor
and removeInterceptor
.
/**
* Adds an interceptor for the map. Added interceptor intercepts operations
* and executes user defined methods and cancels operations if
* user defined methods throw exceptions.
*
* @param interceptor map interceptor.
* @return id of registered interceptor.
*/
String addInterceptor( MapInterceptor interceptor );
/**
* Removes the given interceptor for this map. So it does not
* intercept operations anymore.
*
* @param id registration ID of the map interceptor.
*/
void removeInterceptor( String id );
Here is the MapInterceptor
interface:
public interface MapInterceptor extends Serializable {
/**
* Intercept the get operation before it returns a value.
* Return another object to change the return value of get().
* Returning null causes the get() operation to return the original value,
* namely return null if you do not want to change anything.
*
*
* @param value the original value to be returned as the result of get() operation.
* @return the new value that is returned by get() operation.
*/
Object interceptGet( Object value );
/**
* Called after get() operation is completed.
*
*
* @param value the value returned as the result of get() operation.
*/
void afterGet( Object value );
/**
* Intercept put operation before modifying map data.
* Return the object to be put into the map.
* Returning null causes the put() operation to operate as expected,
* namely no interception. Throwing an exception cancels the put operation.
*
*
* @param oldValue the value currently existing in the map.
* @param newValue the new value to be put.
* @return new value after intercept operation.
*/
Object interceptPut( Object oldValue, Object newValue );
/**
* Called after put() operation is completed.
*
*
* @param value the value returned as the result of put() operation.
*/
void afterPut( Object value );
/**
* Intercept remove operation before removing the data.
* Return the object to be returned as the result of remove operation.
* Throwing an exception cancels the remove operation.
*
*
* @param removedValue the existing value to be removed.
* @return the value to be returned as the result of remove operation.
*/
Object interceptRemove( Object removedValue );
/**
* Called after remove() operation is completed.
*
*
* @param value the value returned as the result of remove(.) operation
*/
void afterRemove( Object value );
}
Example Usage:
public class InterceptorTest {
@Test
public void testMapInterceptor() throws InterruptedException {
HazelcastInstance hazelcastInstance1 = Hazelcast.newHazelcastInstance();
HazelcastInstance hazelcastInstance2 = Hazelcast.newHazelcastInstance();
IMap<Object, Object> map = hazelcastInstance1.getMap( "testMapInterceptor" );
SimpleInterceptor interceptor = new SimpleInterceptor();
map.addInterceptor( interceptor );
map.put( 1, "New York" );
map.put( 2, "Istanbul" );
map.put( 3, "Tokyo" );
map.put( 4, "London" );
map.put( 5, "Paris" );
map.put( 6, "Cairo" );
map.put( 7, "Hong Kong" );
try {
map.remove( 1 );
} catch ( Exception ignore ) {
}
try {
map.remove( 2 );
} catch ( Exception ignore ) {
}
assertEquals( map.size(), 6) ;
assertEquals( map.get( 1 ), null );
assertEquals( map.get( 2 ), "ISTANBUL:" );
assertEquals( map.get( 3 ), "TOKYO:" );
assertEquals( map.get( 4 ), "LONDON:" );
assertEquals( map.get( 5 ), "PARIS:" );
assertEquals( map.get( 6 ), "CAIRO:" );
assertEquals( map.get( 7 ), "HONG KONG:" );
map.removeInterceptor( interceptor );
map.put( 8, "Moscow" );
assertEquals( map.get( 8 ), "Moscow" );
assertEquals( map.get( 1 ), null );
assertEquals( map.get( 2 ), "ISTANBUL" );
assertEquals( map.get( 3 ), "TOKYO" );
assertEquals( map.get( 4 ), "LONDON" );
assertEquals( map.get( 5 ), "PARIS" );
assertEquals( map.get( 6 ), "CAIRO" );
assertEquals( map.get( 7 ), "HONG KONG" );
}
static class SimpleInterceptor implements MapInterceptor, Serializable {
@Override
public Object interceptGet( Object value ) {
if (value == null)
return null;
return value + ":";
}
@Override
public void afterGet( Object value ) {
}
@Override
public Object interceptPut( Object oldValue, Object newValue ) {
return newValue.toString().toUpperCase();
}
@Override
public void afterPut( Object value ) {
}
@Override
public Object interceptRemove( Object removedValue ) {
if(removedValue.equals( "ISTANBUL" ))
throw new RuntimeException( "you can not remove this" );
return removedValue;
}
@Override
public void afterRemove( Object value ) {
// do something
}
}
}
It is very easy to trigger an out of memory exception (OOME) with query-based map methods, especially with large clusters or heap sizes. For example, on a cluster with five members having 10 GB of data and 25 GB heap size per member, a single call of IMap.entrySet()
fetches 50 GB of data and crashes the calling instance.
A call of IMap.values()
may return too much data for a single member. This can also happen with a real query and an unlucky choice of predicates, especially when the parameters are chosen by a user of your application.
To prevent this, you can configure a maximum result size limit for query based operations. This is not a limit like SELECT * FROM map LIMIT 100
, which you can achieve by a Paging Predicate. A maximum result size limit for query based operations is meant to be a last line of defense to prevent your members from retrieving more data than they can handle.
The Hazelcast component which calculates this limit is the QueryResultSizeLimiter
.
If the QueryResultSizeLimiter
is activated, it calculates a result size limit per partition. Each QueryOperation
runs on all partitions of a member, so it collects result entries as long as the member limit is not exceeded. If that happens, a QueryResultSizeExceededException
is thrown and propagated to the calling instance.
This feature depends on an equal distribution of the data on the cluster members to calculate the result size limit per member. Therefore, there is a minimum value defined in QueryResultSizeLimiter.MINIMUM_MAX_RESULT_LIMIT
. Configured values below the minimum will be increased to the minimum.
In addition to the distributed result size check in the QueryOperations
, there is a local pre-check on the calling instance. If you call the method from a client, the pre-check is executed on the member that invokes the QueryOperations
.
Since the local pre-check can increase the latency of a QueryOperation
, you can configure how many local partitions should be considered for the pre-check, or you can deactivate the feature completely.
Besides the designated query operations, there are other operations that use predicates internally. Those method calls will throw the QueryResultSizeExceededException
as well. Please see the following matrix to see the methods that are covered by the query result size limit.
The query result size limit is configured via the following system properties.
hazelcast.query.result.size.limit
: Result size limit for query operations on maps. This value defines the maximum number of returned elements for a single query result. If a query exceeds this number of elements, a QueryResultSizeExceededException is thrown.hazelcast.query.max.local.partition.limit.for.precheck
: Maximum value of local partitions to trigger local pre-check for TruePredicate query operations on maps.Please refer to the System Properties section to see the full descriptions of these properties and how to set them.
Hazelcast distributed queue is an implementation of java.util.concurrent.BlockingQueue
. Being distributed, Hazelcast distributed queue enables all cluster members to interact with it. Using Hazelcast distributed queue, you can add an item in one cluster member and remove it from another one.
Use the Hazelcast instance's getQueue
method to get the queue, then use the queue's put
method to put items into the queue.
import com.hazelcast.core.Hazelcast;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.TimeUnit;
public class SampleQueue {
public static void main(String[] args) throws Exception {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
BlockingQueue<MyTask> queue = hazelcastInstance.getQueue( "tasks" );
queue.put( new MyTask() );
MyTask task = queue.take();
boolean offered = queue.offer( new MyTask(), 10, TimeUnit.SECONDS );
task = queue.poll( 5, TimeUnit.SECONDS );
if ( task != null ) {
//process task
}
}
}
FIFO ordering will apply to all queue operations across the cluster. The user objects (such as MyTask
in the example above) that are enqueued or dequeued have to be Serializable
.
Hazelcast distributed queue performs no batching while iterating over the queue. All items will be copied locally and iteration will occur locally.
Hazelcast distributed queue uses ItemListener
to listen to the events that occur when items are added to and removed from the queue. Please refer to the Listening for Item Events section for information on how to create an item listener class and register it.
The following example code illustrates a distributed queue that connects a producer and consumer.
Let's put
one integer on the queue every second, 100 integers total.
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.IQueue;
public class ProducerMember {
public static void main( String[] args ) throws Exception {
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
IQueue<Integer> queue = hz.getQueue( "queue" );
for ( int k = 1; k < 100; k++ ) {
queue.put( k );
System.out.println( "Producing: " + k );
Thread.sleep(1000);
}
queue.put( -1 );
System.out.println( "Producer Finished!" );
}
}
Producer
puts a -1 on the queue to show that the put
s are finished.
Now, let's create a Consumer
class to take
a message from this queue, as shown below.
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.IQueue;
public class ConsumerMember {
public static void main( String[] args ) throws Exception {
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
IQueue<Integer> queue = hz.getQueue( "queue" );
while ( true ) {
int item = queue.take();
System.out.println( "Consumed: " + item );
if ( item == -1 ) {
queue.put( -1 );
break;
}
Thread.sleep( 5000 );
}
System.out.println( "Consumer Finished!" );
}
}
As seen in the above example code, Consumer
waits five seconds before it consumes the next message. It stops once it receives -1. Also note that Consumer
puts -1 back on the queue before the loop is ended.
When you first start Producer
and then start Consumer
, items produced on the queue will be consumed from the same queue.
From the above example code, you can see that an item is produced every second and consumed every five seconds. Therefore, the consumer keeps growing. To balance the produce/consume operation, let's start another consumer. This way, consumption is distributed to these two consumers, as seen in the sample outputs below.
The second consumer is started. After a while, here is the first consumer output:
...
Consumed 13
Consumed 15
Consumer 17
...
Here is the second consumer output:
...
Consumed 14
Consumed 16
Consumer 18
...
In the case of a lot of producers and consumers for the queue, using a list of queues may solve the queue bottlenecks. In this case, be aware that the order of the messages sent to different queues is not guaranteed. Since in most cases strict ordering is not important, a list of queues is a good solution.
NOTE: The items are taken from the queue in the same order they were put on the queue. However, if there is more than one consumer, this order is not guaranteed.
Hazelcast gives an itemId
for each item you offer, which is an incrementing sequence identification for the queue items. You should consider the following to understand the itemId
assignment behavior:
itemId
assignment resumes from the last known highest itemId
before the restart; itemId
assignment does not start from the beginning for the new items.QueueStore
). If the queue has QueueStore
, the itemId
for the new items are given, starting from the highest itemId
found in the IDs returned by the method loadAllKeys
. If the method loadAllKeys
does not return anything, the itemId
s will started from the beginning after a cluster restart.itemId
s in the memory or in the persistent data store.A bounded queue is a queue with a limited capacity. When the bounded queue is full, no more items can be put into the queue until some items are taken out.
To turn a Hazelcast distributed queue into a bounded queue, set the capacity limit with the max-size
property. You can set the max-size
property in the configuration, as shown below. max-size
specifies the maximum size of the queue. Once the queue size reaches this value, put
operations will be blocked until the queue size goes below max-size
, which happens when a consumer removes items from the queue.
Let's set 10 as the maximum size of our example queue in Creating an Example Queue.
<hazelcast>
...
<queue name="queue">
<max-size>10</max-size>
</queue>
...
</hazelcast>
When the producer is started, ten items are put into the queue and then the queue will not allow more put
operations. When the consumer is started, it will remove items from the queue. This means that the producer can put
more items into the queue until there are ten items in the queue again, at which point the put
operation again becomes blocked.
In this example code, the producer is five times faster than the consumer. It will effectively always be waiting for the consumer to remove items before it can put more on the queue. For this example code, if maximum throughput is the goal, it would be a good option to start multiple consumers to prevent the queue from filling up.
Hazelcast allows you to load and store the distributed queue items from/to a persistent datastore using the interface QueueStore
. If queue store is enabled, each item added to the queue will also be stored at the configured queue store. When the number of items in the queue exceeds the memory limit, the subsequent items are persisted in the queue store, they are not stored in the queue memory.
The QueueStore
interface enables you to store, load, and delete queue items with methods like store
, storeAll
, load
and delete
. The following example class includes all of the QueueStore
methods.
public class TheQueueStore implements QueueStore<Item> {
@Override
public void delete(Long key) {
System.out.println("delete");
}
@Override
public void store(Long key, Item value) {
System.out.println("store");
}
@Override
public void storeAll(Map<Long, Item> map) {
System.out.println("store all");
}
@Override
public void deleteAll(Collection<Long> keys) {
System.out.println("deleteAll");
}
@Override
public Item load(Long key) {
System.out.println("load");
return null;
}
@Override
public Map<Long, Item> loadAll(Collection<Long> keys) {
System.out.println("loadAll");
return null;
}
@Override
public Set<Long> loadAllKeys() {
System.out.println("loadAllKeys");
return null;
}
Item
must be serializable. The following is an example queue store configuration.
<queue-store>
<class-name>com.hazelcast.QueueStoreImpl</class-name>
<properties>
<property name="binary">false</property>
<property name="memory-limit">1000</property>
<property name="bulk-load">500</property>
</properties>
</queue-store>
Let's explain the queue store properties.
Binary: By default, Hazelcast stores the queue items in serialized form, and before it inserts the queue items into the , it deserializes them. If you are not reaching the queue store from an external application, you might prefer that the items be inserted in binary form. Do this by setting the binary
property to true: then you can get rid of the deserialization step, which is a performance optimization. The binary
property is false by default.
Memory Limit: This is the number of items after which Hazelcast will store items only to the datastore. For example, if the memory limit is 1000, then the 1001st item will be put only to the datastore. This feature is useful when you want to avoid out-of-memory conditions. If you want to always use memory, you can set it to Integer.MAX_VALUE
. The default number for memory-limit
is 1000.
Bulk Load: When the queue is initialized, items are loaded from QueueStore
in bulks. Bulk load is the size of these bulks. The default value of bulk-load
is 250.
The following are examples of queue configurations. It includes the QueueStore
configuration, which is explained in the Queueing with Persistent Datastore section.
Declarative:
<queue name="default">
<max-size>0</max-size>
<backup-count>1</backup-count>
<async-backup-count>0</async-backup-count>
<empty-queue-ttl>-1</empty-queue-ttl>
<item-listeners>
<item-listener>
com.hazelcast.examples.ItemListener
</item-listener>
<item-listeners>
</queue>
<queue-store>
<class-name>com.hazelcast.QueueStoreImpl</class-name>
<properties>
<property name="binary">false</property>
<property name="memory-limit">10000</property>
<property name="bulk-load">500</property>
</properties>
</queue-store>
Programmatic:
Config config = new Config();
QueueConfig queueConfig = config.getQueueConfig();
queueConfig.setName( "MyQueue" ).setBackupCount( "1" )
.setMaxSize( "0" ).setStatisticsEnabled( "true" );
queueConfig.getQueueStoreConfig()
.setEnabled ( "true" )
.setClassName( "com.hazelcast.QueueStoreImpl" )
.setProperty( "binary", "false" );
Hazelcast distributed queue has one synchronous backup by default. By having this backup, when a cluster member with a queue goes down, another member having the backup of that queue will continue. Therefore, no items are lost. You can define the number of synchronous backups for a queue using the backup-count
element in the declarative configuration. A queue can also have asynchronous backups: you can define the number of asynchronous backups using the async-backup-count
element.
To set the maximum size of the queue, use the max-size
element. To purge unused or empty queues after a period of time, use the empty-queue-ttl
element. If you define a value (time in seconds) for the empty-queue-ttl
element, then your queue will be destroyed if it stays empty or unused for the time in seconds that you give.
The following is the full list of queue configuration elements with their descriptions.
max-size
: Maximum number of items in the queue. It is used to set an upper bound for the queue. You will not be able to put more items when the queue reaches to this maximum size whether you have a queue store configured or not.backup-count
: Number of synchronous backups. Queue is a non-partitioned data structure, so all entries of a queue reside in one partition. When this parameter is '1', it means there will be one backup of that queue in another member in the cluster. When it is '2', two members will have the backup.async-backup-count
: Number of asynchronous backups.empty-queue-ttl
: Used to purge unused or empty queues. If you define a value (time in seconds) for this element, then your queue will be destroyed if it stays empty or unused for that time.item-listeners
: Adds listeners (listener classes) for the queue items. You can also set the attribute include-value
to true
if you want the item event to contain the item values, and you can set local
to true
if you want to listen to the items on the local member.queue-store
: Includes the queue store factory class name and the properties binary, memory limit and bulk load. Please refer to Queueing with Persistent Datastore.statistics-enabled
: If set to true
, you can retrieve statistics for this queue using the method getLocalQueueStats()
.Hazelcast MultiMap
is a specialized map where you can store multiple values under a single key. Just like any other distributed data structure implementation in Hazelcast, MultiMap
is distributed and thread-safe.
Hazelcast MultiMap
is not an implementation of java.util.Map
due to the difference in method signatures. It supports most features of Hazelcast Map except for indexing, predicates and MapLoader/MapStore. Yet, like Hazelcast Map, entries are almost evenly distributed onto all cluster members. When a new member joins the cluster, the same ownership logic used in the distributed map applies.
The following example creates a MultiMap and puts items into it. Use the HazelcastInstance getMultiMap
method to get the MultiMap, then use the MultiMap put
method to put an entry into the MultiMap.
public class PutMember {
public static void main( String[] args ) {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
MultiMap <String , String > map = hazelcastInstance.getMultiMap( "map" );
map.put( "a", "1" );
map.put( "a", "2" );
map.put( "b", "3" );
System.out.println( "PutMember:Done" );
}
}
Now let's print the entries in this MultiMap.
public class PrintMember {
public static void main( String[] args ) {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
MultiMap <String, String > map = hazelcastInstance.getMultiMap( "map" );
for ( String key : map.keySet() ){
Collection <String > values = map.get( key );
System.out.println( "%s -> %s\n",key, values );
}
}
}
After you run the first code sample, run the PrintMember
sample. You will see the key a
has two values, as shown below.
b -> [3]
a -> [2, 1]
Hazelcast MultiMap uses EntryListener
to listen to events which occur when entries are added to, updated in or removed from the MultiMap. Please refer to the Listening for MultiMap Events section for information on how to create an entry listener class and register it.
When using MultiMap, the collection type of the values can be either Set or List. Configure the collection type with the valueCollectionType
parameter. If you choose Set
, duplicate and null values are not allowed in your collection and ordering is irrelevant. If you choose List
, ordering is relevant and your collection can include duplicate and null values.
You can also enable statistics for your MultiMap with the statisticsEnabled
parameter. If you enable statisticsEnabled
, statistics can be retrieved with getLocalMultiMapStats()
method.
NOTE: Currently, eviction is not supported for the MultiMap data structure.
The following are the example MultiMap configurations.
Declarative:
<hazelcast>
<multimap name="default">
<backup-count>0</backup-count>
<async-backup-count>1</async-backup-count>
<value-collection-type>SET</value-collection-type>
<entry-listeners>
<entry-listener include-value="false" local="false">
com.hazelcast.examples.EntryListener
</entry-listener>
</entry-listeners>
</map>
</hazelcast>
Programmatic:
MultiMapConfig mmConfig = new MultiMapConfig();
mmConfig.setName( "default" );
mmConfig.setBackupCount( "0" ).setAsyncBackupCount( "1" );
mmConfig.setValueCollectionType( "SET" );
The following are the configuration elements and their descriptions:
backup-count
: Defines the number of asynchronous backups. For example, if it is set to 1, backup of a partition will be
placed on one other member. If it is 2, it will be placed on two other members.async-backup-count
: The number of synchronous backups. Behavior is the same as that of the backup-count
element.statistics-enabled
: You can retrieve some statistics such as owned entry count, backup entry count, last update time, and locked entry count by setting this parameter's value as "true". The method for retrieving the statistics is getLocalMultiMapStats()
.value-collection-type
: Type of the value collection. It can be Set
or List
.entry-listeners
: Lets you add listeners (listener classes) for the map entries. You can also set the attribute
include-value to true if you want the item event to contain the entry values, and you can set
local to true if you want to listen to the entries on the local member.Hazelcast Set is a distributed and concurrent implementation of java.util.Set
.
java.util.HashSet
.Use the HazelcastInstance getSet
method to get the Set, then use the set put
method to put items into the Set.
import com.hazelcast.core.Hazelcast;
import java.util.Set;
import java.util.Iterator;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Set<Price> set = hazelcastInstance.getSet( "IBM-Quote-History" );
set.add( new Price( 10, time1 ) );
set.add( new Price( 11, time2 ) );
set.add( new Price( 12, time3 ) );
set.add( new Price( 11, time4 ) );
//....
Iterator<Price> iterator = set.iterator();
while ( iterator.hasNext() ) {
Price price = iterator.next();
//analyze
}
Hazelcast Set uses ItemListener
to listen to events that occur when items are added to and removed from the Set. Please refer to the Listening for Item Events section for information on how to create an item listener class and register it.
The following are the example set configurations.
Declarative:
<set name="default">
<backup-count>1</backup-count>
<async-backup-count>0</async-backup-count>
<max-size>10</max-size>
<item-listeners>
<item-listener>
com.hazelcast.examples.ItemListener
</item-listener>
<item-listeners>
</set>
Programmatic:
Config config = new Config();
CollectionConfig collectionSet = config.getCollectionConfig();
collectionSet.setName( "MySet" ).setBackupCount( "1" )
.setMaxSize( "10" );
Set configuration has the following elements.
statistics-enabled
: True (default) if statistics gathering is enabled on the Set, false otherwise.backup-count
: Count of synchronous backups. Set is a non-partitioned data structure, so all entries of a Set reside in one partition. When this parameter is '1', it means there will be one backup of that Set in another member in the cluster. When it is '2', two members will have the backup.async-backup-count
: Count of asynchronous backups.max-size
: The maximum number of entries for this Set.item-listeners
: Lets you add listeners (listener classes) for the list items. You can also set the attributes include-value
to true
if you want the item event to contain the item values, and you can set local
to true
if you want to listen to the items on the local member.Hazelcast List is similar to Hazelcast Set, but Hazelcast List also allows duplicate elements.
Use the HazelcastInstance getList
method to get the list, then use the list put
method to put items into the List.
import com.hazelcast.core.Hazelcast;
import java.util.List;
import java.util.Iterator;
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
List<Price> list = hz.getList( "IBM-Quote-Frequency" );
list.add( new Price( 10 ) );
list.add( new Price( 11 ) );
list.add( new Price( 12 ) );
list.add( new Price( 11 ) );
list.add( new Price( 12 ) );
//....
Iterator<Price> iterator = list.iterator();
while ( iterator.hasNext() ) {
Price price = iterator.next();
//analyze
}
Hazelcast List uses ItemListener
to listen to events that occur when items are added to and removed from the List. Please refer to the Listening for Item Events section for information on how to create an item listener class and register it.
The following are example list configurations.
Declarative:
<list name="default">
<backup-count>1</backup-count>
<async-backup-count>0</async-backup-count>
<max-size>10</max-size>
<item-listeners>
<item-listener>
com.hazelcast.examples.ItemListener
</item-listener>
</item-listeners>
</list>
Programmatic:
Config config = new Config();
CollectionConfig collectionList = config.getCollectionConfig();
collectionList.setName( "MyList" ).setBackupCount( "1" )
.setMaxSize( "10" );
List configuration has the following elements.
statistics-enabled
: True (default) if statistics gathering is enabled on the list, false otherwise.backup-count
: Number of synchronous backups. List is a non-partitioned data structure, so all entries of a List reside in one partition. When this parameter is '1', there will be one backup of that List in another member in the cluster. When it is '2', two members will have the backup.async-backup-count
: Number of asynchronous backups.max-size
: The maximum number of entries for this List.item-listeners
: Lets you add listeners (listener classes) for the list items. You can also set the attribute include-value
to true
if you want the item event to contain the item values, and you can set the attribute local
to true
if you want to listen the items on the local member.Hazelcast Ringbuffer is a distributed data structure that stores its data in a ring-like structure. You can think of it as a circular array with a given capacity. Each Ringbuffer has a tail and a head. The tail is where the items are added and the head is where the items are overwritten or expired. You can reach each element in a Ringbuffer using a sequence ID, which is mapped to the elements between the head and tail (inclusive) of the Ringbuffer.
Reading from Ringbuffer is simple: get the Ringbuffer with the HazelcastInstance getRingbuffer
method, get its current head with
the headSequence
method, and start reading. Use the method readOne
to return the item at the
given sequence; readOne
blocks if no item is available. To read the next item, increment the sequence by one.
Ringbuffer<String> ringbuffer = hz.getRingbuffer("rb");
long sequence = ringbuffer.headSequence();
while(true){
String item = ringbuffer.readOne(sequence);
sequence++;
... process item
}
By exposing the sequence, you can now move the item from the Ringbuffer as long as the item is still available. If the item is not available
any longer, StaleSequenceException
is thrown.
Adding an item to a Ringbuffer is also easy with the Ringbuffer add
method:
Ringbuffer<String> ringbuffer = hz.getRingbuffer("rb");
ringbuffer.add("someitem")
Use the method add
to return the sequence of the inserted item; the sequence value will always be unique. You can use this as a
very cheap way of generating unique IDs if you are already using Ringbuffer.
Hazelcast Ringbuffer can sometimes be a better alternative than an Hazelcast IQueue. Unlike IQueue, Ringbuffer does not remove the items, it only reads items using a certain position. There are many advantages to this approach:
queue.take()
is more expensive than a ringBuffer.read(...)
.By default, a Ringbuffer is configured with a capacity
of 10000 items. This creates an array with a size of 10000. If
a time-to-live
is configured, then an array of longs is also created that stores the expiration time for every item.
In a lot of cases you may want to change this capacity
number to something that better fits your needs.
Below is a declarative configuration example of a Ringbuffer with a capacity
of 2000 items.
<ringbuffer name="rb">
<capacity>2000</capacity>
</ringbuffer>
Currently, Hazelcast Ringbuffer is not a partitioned data structure; its data is stored in a single partition and the replicas are stored in another partition. Therefore, create a Ringbuffer that can safely fit in a single cluster member.
Hazelcast Ringbuffer has a single synchronous backup by default. You can control the Ringbuffer backup just like most of the other Hazelcast
distributed data structures by setting the synchronous and asynchronous backups: backup-count
and async-backup-count
. In the example below, a Ringbuffer is configured with no
synchronous backups and one asynchronous backup:
<ringbuffer name="rb">
<backup-count>0</backup-count>
<async-backup-count>1</async-backup-count>
</ringbuffer>
An asynchronous backup will probably give you better performance. However, there is a chance that the item added will be lost when the member owning the primary crashes before the backup could complete. You may want to consider batching methods if you need high performance but do not want to give up on consistency.
You can configure Hazelcast Ringbuffer with a time to live in seconds. Using this setting, you can control how long the items remain in the Ringbuffer before they are expired. By default, the time to live is set to 0, meaning that unless the item is overwritten, it will remain in the Ringbuffer indefinitely. If you set a time to live and an item is added, then, depending on the Overflow Policy, either the oldest item is overwritten, or the call is rejected.
In the example below, a Ringbuffer is configured with a time to live of 180 seconds.
<ringbuffer name="rb">
<time-to-live-seconds>180</time-to-live-seconds>
</ringbuffer>
Using the overflow policy, you can determine what to do if the oldest item in the Ringbuffer is not old enough to expire when more items than the configured Ringbuffer capacity are being added. The below options are currently available.
OverflowPolicy.OVERWRITE
: The oldest item is overwritten. OverflowPolicy.FAIL
: The call is aborted. The methods that make use of the OverflowPolicy return -1
to indicate that adding
the item has failed. Overflow policy gives you fine control on what to do if the Ringbuffer is full. You can also use the overflow policy to apply a back pressure mechanism. The following example code shows the usage of an exponential backoff.
long sleepMs = 100;
for (; ; ) {
long result = ringbuffer.addAsync(item, OverflowPolicy.FAIL).get();
if (result != -1) {
break;
}
TimeUnit.MILLISECONDS.sleep(sleepMs);
sleepMs = min(5000, sleepMs * 2);
}
You can configure Hazelcast Ringbuffer with an in-memory format that controls the format of the Ringbuffer's stored items. By default, BINARY
in-memory format is used,
meaning that the object is stored in a serialized form. You can select the OBJECT
in-memory format, which is useful when filtering is
applied or when the OBJECT
in-memory format has a smaller memory footprint than BINARY
.
In the declarative configuration example below, a Ringbuffer is configured with the OBJECT
in-memory format:
<ringbuffer name="rb">
<in-memory-format>BINARY</in-memory-format>
</ringbuffer>
In the previous examples, the method ringBuffer.add()
is used to add an item to the Ringbuffer. The problems with this method
are that it always overwrites and that it does not support batching. Batching can have a huge
impact on the performance. You can use the method addAllAsync
to support batching.
Please see the following example code.
List<String> items = Arrays.asList("1","2","3");
ICompletableFuture<Long> f = rb.addAllAsync(items, OverflowPolicy.OVERWRITE);
f.get()
In the above case, three strings are added to the Ringbuffer using the policy OverflowPolicy.OVERWRITE
. Please see the Overflow Policy section
for more information.
In the previous example, the readOne
method read items from the Ringbuffer. readOne
is simple but not very efficient for the following reasons:
readOne
does not use batching.readOne
cannot filter items at the source; the items need to be retrieved before being filtered.The method readManyAsync
can read a batch of items and can filter items at the source.
Please see the following example code.
ICompletableFuture<ReadResultSet<E>> readManyAsync(
long startSequence,
int minCount,
int maxCount,
IFunction<E, Boolean> filter);
The meanings of the readManyAsync
arguments are given below.
startSequence
: Sequence of the first item to read.minCount
: Minimum number of items to read. If you do not want to block, set it to 0. If you want to block for at least one item,
set it to 1.maxCount
: Maximum number of the items to retrieve. Its value cannot exceed 1000.filter
: A function that accepts an item and checks if it should be returned. If no filtering should be applied, set it to null.A full example is given below.
long sequence = rb.headSequence();
for(;;) {
ICompletableFuture<ReadResultSet<String>> f = rb.readManyAsync(sequence, 1, 10, null);
ReadResultSet<String> rs = f.get();
for (String s : rs) {
System.out.println(s);
}
sequence+=rs.readCount();
}
Please take a careful look at how your sequence is being incremented. You cannot always rely on the number of items being returned if the items are filtered out.
Hazelcast Ringbuffer provides asynchronous methods for more powerful operations like batched writing or batched reading with filtering.
To make these methods synchronous, just call the method get()
on the returned future.
Please see the following example code.
ICompletableFuture f = ringbuffer.addAsync(item, OverflowPolicy.FAIL);
f.get();
However, you can also use ICompletableFuture
to get notified when the operation has completed. The advantage of ICompletableFuture
is that the thread used for the call is not blocked till the response is returned.
Please see the below code as an example of when you want to get notified when a batch of reads has completed.
ICompletableFuture<ReadResultSet<String>> f = rb.readManyAsync(sequence, min, max, someFilter);
f.andThen(new ExecutionCallback<ReadResultSet<String>>() {
@Override
public void onResponse(ReadResultSet<String> response) {
for (String s : response) {
System.out.println("Received:" + s);
}
}
@Override
public void onFailure(Throwable t) {
t.printStackTrace();
}
});
The following shows the declarative configuration of a Ringbuffer called rb
. The configuration is modeled after the Ringbuffer defaults.
<ringbuffer name="rb">
<capacity>10000</capacity>
<backup-count>1</backup-count>
<async-backup-count>0</async-backup-count>
<time-to-live-seconds>0</time-to-live-seconds>
<in-memory-format>BINARY</in-memory-format>
</ringbuffer>
You can also configure a Ringbuffer programmatically. The following is a programmatic version of the above declarative configuration.
RingbufferConfig rbConfig = new RingbufferConfig("rb")
.setCapacity(10000)
.setBackupCount(1)
.setAsyncBackupCount(0)
.setTimeToLiveSeconds(0)
.setInMemoryFormat(InMemoryFormat.BINARY);
Config config = new Config();
config.addRingbufferConfig(rbConfig);
Hazelcast provides a distribution mechanism for publishing messages that are delivered to multiple subscribers. This is also known as a publish/subscribe (pub/sub) messaging model. Publishing and subscribing operations are cluster wide. When a member subscribes to a topic, it is actually registering for messages published by any member in the cluster, including the new members that joined after you add the listener.
NOTE: Publish operation is async. It does not wait for operations to run in remote members; it works as fire and forget.
Use the HazelcastInstance getTopic
method to get the Topic, then use the topic publish
method to publish your messages (messageObject
).
import com.hazelcast.core.Topic;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.MessageListener;
public class Sample implements MessageListener<MyEvent> {
public static void main( String[] args ) {
Sample sample = new Sample();
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ITopic topic = hazelcastInstance.getTopic( "default" );
topic.addMessageListener( sample );
topic.publish( new MyEvent() );
}
public void onMessage( Message<MyEvent> message ) {
MyEvent myEvent = message.getMessageObject();
System.out.println( "Message received = " + myEvent.toString() );
if ( myEvent.isHeavyweight() ) {
messageExecutor.execute( new Runnable() {
public void run() {
doHeavyweightStuff( myEvent );
}
} );
}
}
// ...
private final Executor messageExecutor = Executors.newSingleThreadExecutor();
}
Hazelcast Topic uses the MessageListener
interface to listen for events that occur when a message is received. Please refer to the Listening for Topic Messages section for information on how to create a message listener class and register it.
Topic has two statistic variables that you can query. These values are incremental and local to the member.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ITopic<Object> myTopic = hazelcastInstance.getTopic( "myTopicName" );
myTopic.getLocalTopicStats().getPublishOperationCount();
myTopic.getLocalTopicStats().getReceiveOperationCount();
getPublishOperationCount
and getReceiveOperationCount
returns the total number of published and received messages since the start of this member, respectively. Please note that these values are not backed up, so if the member goes down, these values will be lost.
You can disable this feature with topic configuration. Please see the Configuring Topic section.
NOTE: These statistics values can be also viewed in Management Center. Please see Monitoring Topics.
Each cluster member has a list of all registrations in the cluster. When a new member is registered for a topic, it sends a registration message to all members in the cluster. Also, when a new member joins the cluster, it will receive all registrations made so far in the cluster.
The behavior of a topic varies depending on the value of the configuration parameter globalOrderEnabled
.
If globalOrderEnabled
is disabled, messages are not ordered and listeners (subscribers) process the messages in the order that the messages are published. If cluster member M publishes messages m1, m2, m3, ..., mn to a topic T, then Hazelcast makes sure that all of the subscribers of topic T will receive and process m1, m2, m3, ..., mn in the given order.
Here is how it works. Let's say that we have three members (member1, member2 and member3) and that member1 and member2 are registered to a topic named news
. Note that all three members know that member1 and member2 are registered to news
.
In this example, member1 publishes two messages, a1
and a2
, and member3 publishes two messages, c1
and c2
. When member1 and member3 publish a message, they will check their local list for registered members, they will discover that member1 and member2 are in their lists, and then they will fire messages to those members. One possible order of the messages received could be the following.
member1 -> c1
, a1
, a2
, c2
member2 -> c1
, c2
, a1
, a2
If globalOrderEnabled
is enabled, all members listening to the same topic will get its messages in the same order.
Here is how it works. Let's say that we have three members (member1, member2 and member3) and that member1 and member2 are registered to a topic named news
. Note that all three members know that member1 and member2 are registered to news
.
In this example, member1 publishes two messages: a1
and a2
, and member3 publishes two messages: c1
and c2
. When a member publishes messages over the topic news
, it first calculates which partition the news
ID corresponds to. Then it sends an operation to the owner of the partition for that member to publish messages. Let's assume that news
corresponds to a partition that member2 owns. member1 and member3 first sends all messages to member2. Assume that the messages are published in the following order:
member1 -> a1
, c1
, a2
, c2
member2 then publishes these messages by looking at registrations in its local list. It sends these messages to member1 and member2 (it makes a local dispatch for itself).
member1 -> a1
, c1
, a2
, c2
member2 -> a1
, c1
, a2
, c2
This way we guarantee that all members will see the events in the same order.
In both cases, there is a StripedExecutor
in EventService that is responsible for dispatching the received message. For all events in Hazelcast, the order that events are generated and the order they are published to the user are guaranteed to be the same via this StripedExecutor
.
In StripedExecutor
, there are as many threads as are specified in the property hazelcast.event.thread.count
(default is five). For a specific event source (for a particular topic name), hash of that source's name % 5 gives the ID of the responsible thread. Note that there can be another event source (entry listener of a map, item listener of a collection, etc.) corresponding to the same thread. In order not to make other messages to block, heavy processing should not be done in this thread. If there is time-consuming work that needs to be done, the work should be handed over to another thread. Please see the Getting a Topic and Publishing Messages section.
To configure a topic, set the topic name, decide on statistics and global ordering, and set message listeners. Default values are:
global-ordering
is false, meaning that by default, there is no guarantee of global order.statistics
is true, meaning that by default, statistics are calculated.You can see the example configuration snippets below.
Declarative:
<hazelcast>
...
<topic name="yourTopicName">
<global-ordering-enabled>true</global-ordering-enabled>
<statistics-enabled>true</statistics-enabled>
<message-listeners>
<message-listener>MessageListenerImpl</message-listener>
</message-listeners>
</topic>
...
</hazelcast>
Programmatic:
TopicConfig topicConfig = new TopicConfig();
topicConfig.setGlobalOrderingEnabled( true );
topicConfig.setStatisticsEnabled( true );
topicConfig.setName( "yourTopicName" );
MessageListener<String> implementation = new MessageListener<String>() {
@Override
public void onMessage( Message<String> message ) {
// process the message
}
};
topicConfig.addMessageListenerConfig( new ListenerConfig( implementation ) );
HazelcastInstance instance = Hazelcast.newHazelcastInstance()
Topic configuration has the following elements.
statistics-enabled
: Default is true
, meaning statistics are calculated.global-ordering-enabled
: Default is false
, meaning there is no global order guarantee.message-listeners
: Lets you add listeners (listener classes) for the topic messages.Besides the above elements, there are the following system properties that are topic related but not topic specific:
hazelcast.event.queue.capacity
with a default value of 1,000,000hazelcast.event.queue.timeout.millis
with a default value of 250hazelcast.event.thread.count
with a default value of 5For a description of these parameters, please see the Global Event Configuration section.
The Reliable Topic data structure was introduced in Hazelcast 3.5. The Reliable Topic uses the same ITopic
interface
as a regular topic. The main difference is that Reliable Topic is backed up by the Ringbuffer (also introduced with Hazelcast
3.5) data structure. The following are the advantages of this approach:
ITopic
gets its own Ringbuffer; if a topic has a very fast producer, it will not lead to problems at topics that run at a slower pace.ITopic
is shared with other data structures (e.g., collection listeners),
you can run into isolation problems. This does not happen with the Reliable ITopic
.import com.hazelcast.core.Topic;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.MessageListener;
public class Sample implements MessageListener<MyEvent> {
public static void main( String[] args ) {
Sample sample = new Sample();
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ITopic topic = hazelcastInstance.getReliableTopic( "default" );
topic.addMessageListener( sample );
topic.publish( new MyEvent() );
}
public void onMessage( Message<MyEvent> message ) {
MyEvent myEvent = message.getMessageObject();
System.out.println( "Message received = " + myEvent.toString() );
}
}
You can configure the Reliable ITopic
using its Ringbuffer. If a Reliable Topic has the name Foo
, then you can configure this topic
by adding a ReliableTopicConfig
for a Ringbuffer with the name Foo
. By default, a Ringbuffer does not have any TTL (time to live) and
it has a limited capacity; you may want to change that configuration.
By default, the Reliable ITopic
uses a shared thread pool. If you need better isolation, you can configure a custom executor on the
ReliableTopicConfig
.
Because the reads on a Ringbuffer are not destructive, batching is easy to apply. ITopic
uses read batching and reads
ten items at a time (if available) by default.
The Reliable ITopic
provides control and a way to deal with slow consumers. It is unwise to keep events for a slow consumer in memory
indefinitely since you do not know when the slow consumer is going to catch up. You can control the size of the Ringbuffer by using its capacity. For the cases when a Ringbuffer runs out of its capacity, you can specify the following policies for the TopicOverloadPolicy
configuration:
DISCARD_OLDEST
: Overwrite the oldest item, even if a TTL is set. In this case the fast producer supersedes a slow consumer.DISCARD_NEWEST
: Discard the newest item.BLOCK
: Wait until the items are expired in the Ringbuffer.ERROR
: Immediately throw TopicOverloadException
if there is no space in the Ringbuffer.The following are example Reliable Topic configurations.
Declarative:
<reliable-topic name="default">
<statistics-enabled>true</statistics-enabled>
<message-listeners>
<message-listener>
...
</message-listener>
</message-listeners>
<read-batch-size>10</read-batch-size>
<topic-overload-policy>BLOCK</topic-overload-policy>
</reliable-topic>
Programmatic:
Config config = new Config();
ReliableTopicConfig rtConfig = config.getReliableTopicConfig();
rtConfig.setTopicOverloadPolicy( TopicOverloadPolicy.BLOCK )
.setReadBatchSize( 10 )
.setStatisticsEnabled( true );
Reliable Topic configuration has the following elements.
statistics-enabled
: Enables or disables the statistics collection for the Reliable Topic. The default value is true
.message-listener
: Message listener class that listens to the messages when they are added or removed.read-batch-size
: Minimum number of messages that Reliable Topic will try to read in batches. The default value is 10.topic-overload-policy
: Policy to handle an overloaded topic. Available values are DISCARD_OLDEST
, DISCARD_NEWEST
, BLOCK
and ERROR
. The default value is `BLOCK. See Slow Consumers for definitions of these policies.ILock is the distributed implementation of java.util.concurrent.locks.Lock
, meaning that if you lock using an ILock, the critical
section that it guards is guaranteed to be executed by only one thread in the entire cluster. Even though locks are great for synchronization, they can lead to problems if not used properly. Also note that Hazelcast Lock does not support fairness.
Always use locks with try-catch blocks. This will ensure that locks are released if an exception is thrown from
the code in a critical section. Also note that the lock
method is outside the try-catch block because we do not want to unlock
if the lock operation itself fails.
import com.hazelcast.core.Hazelcast;
import java.util.concurrent.locks.Lock;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Lock lock = hazelcastInstance.getLock( "myLock" );
lock.lock();
try {
// do something here
} finally {
lock.unlock();
}
If a lock is not released in the cluster, another thread that is trying to get the
lock can wait forever. To avoid this, use tryLock
with a timeout value. You can
set a high value (normally it should not take that long) for tryLock
. You can check the return value of tryLock
as follows:
if ( lock.tryLock ( 10, TimeUnit.SECONDS ) ) {
try {
// do some stuff here..
} finally {
lock.unlock();
}
} else {
// warning
}
You can also avoid indefinitely waiting threads by using lock with lease time--the lock will be released in the given lease time. The lock can be safely unlocked before the lease time expires. Note that the unlock operation can
throw an IllegalMonitorStateException
if the lock is released because the lease time expires. If that is the case, critical section guarantee is broken.
Please see the below example.
lock.lock( 5, TimeUnit.SECONDS )
try {
// do some stuff here..
} finally {
try {
lock.unlock();
} catch ( IllegalMonitorStateException ex ){
// WARNING Critical section guarantee can be broken
}
}
You can also specify a lease time when trying to acquire a lock: tryLock(time, unit, leaseTime, leaseUnit)
. In that case, it tries to acquire the lock within the specified lease time. If the lock is not available, the current thread becomes disabled for thread scheduling purposes until either it acquires the lock or the specified waiting time elapses. Note that this lease time cannot be longer than the time you specify with the system property hazelcast.lock.max.lease.time.seconds
. Please see the System Properties section to see the description of this property and to learn how to set a system property.
Locks are fail-safe. If a member holds a lock and some other members go down, the cluster will keep your locks safe and available. Moreover, when a member leaves the cluster, all the locks acquired by that dead member will be removed so that those locks are immediately available for live members.
Locks are re-entrant. The same thread can lock multiple times on the same lock. Note that for other threads to be
able to require this lock, the owner of the lock must call unlock
as many times as the owner called lock
.
In the split-brain scenario, the cluster behaves as if it were two different clusters. Since two separate clusters are not aware of each other, two members from different clusters can acquire the same lock. For more information on places where split brain syndrome can be handled, please see split brain syndrome.
Locks are not automatically removed. If a lock is not used anymore, Hazelcast will not automatically garbage collect the lock.
This can lead to an OutOfMemoryError
. If you create locks on the fly, make sure they are destroyed.
Hazelcast IMap also provides locking support on the entry level with the method IMap.lock(key)
. Although the same infrastructure
is used, IMap.lock(key)
is not an ILock and it is not possible to expose it directly.
ICondition
is the distributed implementation of the notify
, notifyAll
and wait
operations on the Java object. You can use it to synchronize
threads across the cluster. More specifically, you use ICondition
when a thread's work depends on another thread's output. A good example
is producer/consumer methodology.
Please see the below code examples for a producer/consumer implementation.
Producer thread:
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Lock lock = hazelcastInstance.getLock( "myLockId" );
ICondition condition = lock.newCondition( "myConditionId" );
lock.lock();
try {
while ( !shouldProduce() ) {
condition.await(); // frees the lock and waits for signal
// when it wakes up it re-acquires the lock
// if available or waits for it to become
// available
}
produce();
condition.signalAll();
} finally {
lock.unlock();
}
NOTE: The method await()
takes time value and time unit as arguments. If you specify a negative value for the time, it is interpreted as infinite.
Consumer thread:
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Lock lock = hazelcastInstance.getLock( "myLockId" );
ICondition condition = lock.newCondition( "myConditionId" );
lock.lock();
try {
while ( !canConsume() ) {
condition.await(); // frees the lock and waits for signal
// when it wakes up it re-acquires the lock if
// available or waits for it to become
// available
}
consume();
condition.signalAll();
} finally {
lock.unlock();
}
Hazelcast IAtomicLong
is the distributed implementation of java.util.concurrent.atomic.AtomicLong
. It offers most of AtomicLong's operations such as get
, set
, getAndSet
, compareAndSet
and incrementAndGet
. Since IAtomicLong is a distributed implementation, these operations involve remote calls and thus their performances differ from AtomicLong.
The following example code creates an instance, increments it by a million, and prints the count.
public class Member {
public static void main( String[] args ) {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IAtomicLong counter = hazelcastInstance.getAtomicLong( "counter" );
for ( int k = 0; k < 1000 * 1000; k++ ) {
if ( k % 500000 == 0 ) {
System.out.println( "At: " + k );
}
counter.incrementAndGet();
}
System.out.printf( "Count is %s\n", counter.get() );
}
}
When you start other instances with the code above, you will see the count as member count times a million.
You can send functions to an IAtomicLong. IFunction
is a Hazelcast owned, single method interface. The following sample IFunction
implementation adds two to the original value.
private static class Add2Function implements IFunction <Long, Long> {
@Override
public Long apply( Long input ) {
return input + 2;
}
}
You can use the following methods to execute functions on IAtomicLong.
apply
: Applies the function to the value in IAtomicLong without changing the actual value and returning the result.alter
: Alters the value stored in the IAtomicLong by applying the function. It will not send back a result.alterAndGet
: Alters the value stored in the IAtomicLong by applying the function, storing the result in the IAtomicLong and returning the result.getAndAlter
: Alters the value stored in the IAtomicLong by applying the function and returning the original value.The following sample code includes these methods.
public class Member {
public static void main( String[] args ) {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IAtomicLong atomicLong = hazelcastInstance.getAtomicLong( "counter" );
atomicLong.set( 1 );
long result = atomicLong.apply( new Add2Function() );
System.out.println( "apply.result: " + result);
System.out.println( "apply.value: " + atomicLong.get() );
atomicLong.set( 1 );
atomicLong.alter( new Add2Function() );
System.out.println( "alter.value: " + atomicLong.get() );
atomicLong.set( 1 );
result = atomicLong.alterAndGet( new Add2Function() );
System.out.println( "alterAndGet.result: " + result );
System.out.println( "alterAndGet.value: " + atomicLong.get() );
atomicLong.set( 1 );
result = atomicLong.getAndAlter( new Add2Function() );
System.out.println( "getAndAlter.result: " + result );
System.out.println( "getAndAlter.value: " + atomicLong.get() );
}
}
The reason for using a function instead of a simple code line like atomicLong.set(atomicLong.get() + 2));
is that the IAtomicLong read and write operations are not atomic. Since IAtomicLong
is a distributed implementation, those operations can be remote ones, which may lead to race problems. By using functions, the data is not pulled into the code, but the code is sent to the data. This makes it more scalable.
NOTE: IAtomicLong has one synchronous backup and no asynchronous backups. Its backup count is not configurable.
Hazelcast ISemaphore is the distributed implementation of java.util.concurrent.Semaphore
.
Semaphores offer permits to control the thread counts when performing concurrent activities. To execute a concurrent activity, a thread grants a permit or waits until a permit becomes available. When the execution is completed, the permit is released.
NOTE: Semaphore with a single permit may be considered a lock. Unlike the locks, however, when semaphores are used, any thread can release the permit, and semaphores can have multiple permits.
NOTE: Hazelcast ISemaphore does not support fairness at all times. There are some edge cases where the fairness is not honored, e.g., when the permit becomes available at the time when an internal timeout occurs.
When a permit is acquired on ISemaphore:
InstanceDestroyedException
is thrown.The following example code uses an IAtomicLong
resource 1000 times, increments the resource when a thread starts to use it, and decrements it when the thread completes.
public class SemaphoreMember {
public static void main( String[] args ) throws Exception{
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ISemaphore semaphore = hazelcastInstance.getSemaphore( "semaphore" );
IAtomicLong resource = hazelcastInstance.getAtomicLong( "resource" );
for ( int k = 0 ; k < 1000 ; k++ ) {
System.out.println( "At iteration: " + k + ", Active Threads: " + resource.get() );
semaphore.acquire();
try {
resource.incrementAndGet();
Thread.sleep( 1000 );
resource.decrementAndGet();
} finally {
semaphore.release();
}
}
System.out.println("Finished");
}
}
Let's limit the concurrent access to this resource by allowing at most three threads. You can configure it declaratively by setting the initial-permits
property, as shown below.
<semaphore name="semaphore">
<initial-permits>3</initial-permits>
</semaphore>
NOTE: If there is a shortage of permits while the semaphore is being created, value of this property can be set to a negative number.
If you execute the above SemaphoreMember
class 5 times, the output will be similar to the following:
At iteration: 0, Active Threads: 1
At iteration: 1, Active Threads: 2
At iteration: 2, Active Threads: 3
At iteration: 3, Active Threads: 3
At iteration: 4, Active Threads: 3
As you can see, the maximum count of concurrent threads is equal or smaller than three. If you remove the semaphore acquire/release statements in SemaphoreMember
, you will see that there is no limitation on the number of concurrent usages.
Hazelcast also provides backup support for ISemaphore
. When a member goes down, you can have another member take over the semaphore with the permit information (permits are automatically released when a member goes down). To enable this, configure synchronous or asynchronous backup with the properties backup-count
and async-backup-count
(by default, synchronous backup is already enabled).
The following are example semaphore configurations.
Declarative:
<semaphore name="semaphore">
<backup-count>1</backup-count>
<async-backup-count>0</async-backup-count>
<initial-permits>3</initial-permits>
</semaphore>
Programmatic:
Config config = new Config();
SemaphoreConfig semaphoreConfig = config.getSemaphoreConfig();
semaphoreConfig.setName( "semaphore" ).setBackupCount( "1" )
.setInitialPermits( "3" );
Semaphore configuration has the below elements.
initial-permits
: the thread count to which the concurrent access is limited. For example, if you set it to "3", concurrent access to the object is limited to 3 threads.backup-count
: Number of synchronous backups.async-backup-count
: Number of asynchronous backups. NOTE: If high performance is more important than not losing the permit information, you can disable the backups by setting backup-count
to 0.
The IAtomicLong
is very useful if you need to deal with a long, but in some cases you need to deal with a reference. That is why Hazelcast also supports the IAtomicReference
which is the distributed version of the java.util.concurrent.atomic.AtomicReference
.
Here is an IAtomicReference example.
public class Member {
public static void main(String[] args) {
Config config = new Config();
HazelcastInstance hz = Hazelcast.newHazelcastInstance(config);
IAtomicReference<String> ref = hz.getAtomicReference("reference");
ref.set("foo");
System.out.println(ref.get());
System.exit(0);
}
}
When you execute the above example, you will see the following output.
foo
Just like IAtomicLong
, IAtomicReference
has methods that accept a 'function' as an argument, such as alter
, alterAndGet
, getAndAlter
and apply
. There are two big advantages of using these methods:
Below are some issues you need to know when you use IAtomicReference.
IAtomicReference
works based on the byte-content and not on the object-reference. If you use the compareAndSet
method, do not change to original value because its serialized content will then be different.
It is also important to know that if you rely on Java serialization, sometimes (especially with hashmaps) the same object can result in different binary content.IAtomicReference
will always have one synchronous backup.IAtomicReference
; but be careful about introducing a data-race. IAtomicReference
is binary
. The receiving side does not need to have the class definition available unless it needs to be deserialized on the other side (e.g., because a method like 'alter' is executed). This deserialization is done for every call that needs to have the object instead of the binary content, so be careful with expensive object graphs that need to be deserialized.apply
method. With the apply
method, the whole object does not need to be sent over the line; only the information that is relevant is sent.Hazelcast ICountDownLatch
is the distributed implementation of java.util.concurrent.CountDownLatch
.
CountDownLatch
is considered to be a gate keeper for concurrent activities. It enables the threads to wait for other threads to complete their operations.
The following code samples describe the mechanism of ICountDownLatch
. Assume that there is a leader process and there are follower processes that will wait until the leader completes. Here is the leader:
public class Leader {
public static void main( String[] args ) throws Exception {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ICountDownLatch latch = hazelcastInstance.getCountDownLatch( "countDownLatch" );
System.out.println( "Starting" );
latch.trySetCount( 1 );
Thread.sleep( 30000 );
latch.countDown();
System.out.println( "Leader finished" );
latch.destroy();
}
}
Since only a single step is needed to be completed as a sample, the above code initializes the latch with 1. Then, the code sleeps for a while to simulate a process and starts the countdown. Finally, it clears up the latch. Let's write a follower:
public class Follower {
public static void main( String[] args ) throws Exception {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ICountDownLatch latch = hazelcastInstance.getCountDownLatch( "countDownLatch" );
System.out.println( "Waiting" );
boolean success = latch.await( 10, TimeUnit.SECONDS );
System.out.println( "Complete: " + success );
}
}
The follower class above first retrieves ICountDownLatch
and then calls the await
method to enable the thread to listen for the latch. The method await
has a timeout value as a parameter. This is useful when the countDown
method fails. To see ICountDownLatch
in action, start the leader first and then start one or more followers. You will see that the followers will wait until the leader completes.
In a distributed environment, the counting down cluster member may go down. In this case, all listeners are notified immediately and automatically by Hazelcast. The state of the current process just before the failure should be verified and 'how to continue now' should be decided (e.g. restart all process operations, continue with the first failed process operation, throw an exception, etc.).
Although the ICountDownLatch
is a very useful synchronization aid, you will probably not use it on a daily basis. Unlike Java’s implementation, Hazelcast’s ICountDownLatch
count can be reset after a countdown has finished, but not during an active count.
NOTE: ICountDownLatch has 1 synchronous backup and no asynchronous backups. Its backup count is not configurable. Also, the count cannot be re-set during an active count, it should be re-set after the countdown is finished.
Hazelcast IdGenerator is used to generate cluster-wide unique identifiers. Generated identifiers are long type primitive values between 0 and Long.MAX_VALUE
.
ID generation occurs almost at the speed of AtomicLong.incrementAndGet()
. A group of 10,000 identifiers is allocated for each cluster member. In the background, this allocation takes place with an IAtomicLong
incremented by 10,000. Once a cluster member generates IDs (allocation is done), IdGenerator
increments a local counter. If a cluster member uses all IDs in the group, it will get another 10,000 IDs. This way, only one time of network traffic is needed, meaning that 9,999 identifiers are generated in memory instead of over the network. This is fast.
Let's write a sample identifier generator.
public class IdGeneratorExample {
public static void main( String[] args ) throws Exception {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IdGenerator idGen = hazelcastInstance.getIdGenerator( "newId" );
while (true) {
Long id = idGen.newId();
System.err.println( "Id: " + id );
Thread.sleep( 1000 );
}
}
}
Let's run the above code two times. The output will be similar to the following.
Members [1] {
Member [127.0.0.1]:5701 this
}
Id: 1
Id: 2
Id: 3
Members [2] {
Member [127.0.0.1]:5701
Member [127.0.0.1]:5702 this
}
Id: 10001
Id: 10002
Id: 10003
You can see that the generated IDs are unique and counting upwards. If you see duplicated identifiers, it means your instances could not form a cluster.
NOTE: Generated IDs are unique during the life cycle of the cluster. If the entire cluster is restarted, IDs start from 0, again or you can initialize to a value using the init()
method of IdGenerator.
NOTE: IdGenerator has one synchronous backup and no asynchronous backups. Its backup count is not configurable.
A Replicated Map is a distributed key-value data structure where the data is replicated to all members in the cluster. It provides full replication of entries to all members for high speed access. The following are its features:
java.util.Map
.A Replicated Map does not partition data (it does not spread data to different cluster members); instead, it replicates the data to all members. All other data structures are partitioned in design.
Replication leads to higher memory consumption. However, a Replicated Map has faster read and write access since the data is available on all members.
Writes could take place on local/remote members in order to provide write-order, eventually being replicated to all other members.
Replicated Map is suitable for objects, catalog data, or idempotent calculable data (such as HTML pages). It fully implements the java.util.Map
interface, but it lacks the methods from java.util.concurrent.ConcurrentMap
since
there are no atomic guarantees to writes or reads.
NOTE: If Replicated Map is used from a dummy client and this dummy client is connected to a lite member, the entry listeners cannot be registered/de-registered.
NOTE: You cannot use Replicated Map from a lite member. A com.hazelcast.replicatedmap.ReplicatedMapCantBeCreatedOnLiteMemberException
is thrown if com.hazelcast.core.HazelcastInstance#getReplicatedMap(name)
is invoked on a lite member.
Here is an example of Replicated Map code. The HazelcastInstance's getReplicatedMap
method gets the Replicated Map, and the Replicated Map's put
method creates map entries.
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import java.util.Collection;
import java.util.Map;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Map<String, Customer> customers = hazelcastInstance.getReplicatedMap("customers");
customers.put( "1", new Customer( "Joe", "Smith" ) );
customers.put( "2", new Customer( "Ali", "Selam" ) );
customers.put( "3", new Customer( "Avi", "Noyan" ) );
Collection<Customer> colCustomers = customers.values();
for ( Customer customer : colCustomers ) {
// process customer
}
HazelcastInstance::getReplicatedMap
returns com.hazelcast.core.ReplicatedMap
which, as stated above, extends the
java.util.Map
interface.
The com.hazelcast.core.ReplicatedMap
interface has some additional methods for registering entry listeners or retrieving values in an expected order.
If you have a large cluster or very high occurrences of updates, the Replicated Map may not scale linearly as expected since it has to replicate update operations to all members in the cluster.
Since the replication of updates is performed in an asynchronous manner, we recommend you enable back pressure in case your system has high occurrences of updates. Please refer to the Back Pressure section to learn how to enable it.
Replicated Map has an anti-entropy system that will converge values to a common one if some of the members are missing replication updates.
Replicated Map does not guarantee eventual consistency because there are some edge cases that fail to provide consistency.
Replicated Map uses the internal partition system of Hazelcast in order to serialize updates happening on the same key at the same time. This happens by sending updates of the same key to the same Hazelcast member in the cluster.
Due to the asynchronous nature of replication, a Hazelcast member could die before successfully replicating a "write" operation to other members after sending the "write completed" response to its caller during the write process. In this scenario, Hazelcast's internal partition system will promote one of the replicas of the partition as the primary one. The new primary partition will not have the latest "write" since the dead member could not successfully replicate the update. (This will leave the system in a state that the caller is the only one that has the update and the rest of the cluster have not.) In this case even the anti-entropy system simply could not converge the value since the source of true information is lost for the update. This leads to a break in the eventual consistency because different values can be read from the system for the same key.
Other than the aforementioned scenario, the Replicated Map will behave like an eventually consistent system with read-your-writes consistency.
There are several technical design decisions you should consider when you configure a Replicated Map.
Initial Provisioning
If a new member joins the cluster, there are two ways you can handle the initial provisioning that is executed to replicate all existing values to the new member. Each involves how you configure the async fill up.
First, you can configure async fill up to true, which does not block reads while the fill up operation is underway. That way, you have immediate access on the new member, but it will take time until all the values are eventually accessible. Not yet replicated values are returned as non-existing (null).
Second, you can configure for a synchronous initial fill up (by configuring the async fill up to false), which blocks every read or write access to the map until the fill up operation is finished. Use this with caution since it might block your application from operating.
Replicated Map can be configured programmatically or declaratively.
You can declare your Replicated Map configuration in the Hazelcast configuration file hazelcast.xml
. Please see the following example.
<replicatedmap name="default">
<in-memory-format>BINARY</in-memory-format>
<async-fillup>true</async-fillup>
<statistics-enabled>true</statistics-enabled>
<entry-listeners>
<entry-listener include-value="true">
com.hazelcast.examples.EntryListener
</entry-listener>
</entry-listeners>
</replicatedmap>
in-memory-format
: Internal storage format. Please see the In-Memory Format section. The default value is OBJECT
.async-fillup
: Specifies whether the Replicated Map is available for reads before the initial replication is completed. The default value is true
. If set to false
(i.e., synchronous initial fill up), no exception will be thrown when the Replicated Map is not yet ready, but null
values can be seen until the initial replication is completed.statistics-enabled
: If set to true
, the statistics such as cache hits and misses are collected. The default value is true
.entry-listener
: Full canonical classname of the EntryListener
implementation.entry-listener#include-value
: Specifies whether the event includes the value or not. Sometimes the key is enough to react on an event. In those situations, setting this value to false
will save a deserialization cycle. The default value is true
.entry-listener#local
: Not used for Replicated Map since listeners are always local.You can configure a Replicated Map programmatically, as you can do for all other data structures in Hazelcast. You must create the configuration upfront, when you instantiate the HazelcastInstance
.
A basic example of how to configure the Replicated Map using the programmatic approach is shown in the following snippet.
Config config = new Config();
ReplicatedMapConfig replicatedMapConfig =
config.getReplicatedMapConfig( "default" );
replicatedMapConfig.setInMemoryFormat( InMemoryFormat.BINARY );
All properties that can be configured using the declarative configuration are also available using programmatic configuration by transforming the tag names into getter or setter names.
Currently, two in-memory-format
values are usable with the Replicated Map.
OBJECT
(default): The data will be stored in deserialized form. This configuration is the default choice since
the data replication is mostly used for high speed access. Please be aware that changing the values without a Map::put
is
not reflected on the other members but is visible on the changing members for later value accesses.
BINARY
: The data is stored in serialized binary format and has to be deserialized on every request. This
option offers higher encapsulation since changes to values are always discarded as long as the newly changed object is
not explicitly Map::put
into the map again.
A com.hazelcast.core.EntryListener
used on a Replicated Map serves the same purpose as it would on other
data structures in Hazelcast. You can use it to react on add, update, and remove operations. Replicated Maps do not yet support eviction.
The fundamental difference in Replicated Map behavior, compared to the other data structures, is that an EntryListener only reflects changes on local data. Since replication is asynchronous, all listener events are fired only when an operation is finished on a local member. Events can fire at different times on different members.
Here is a code example for using EntryListener on a Replicated Map.
The HazelcastInstance
's getReplicatedMap
method gets a Replicated Map (customers), and the ReplicatedMap
's addEntryListener
method adds an entry listener to the Replicated Map. Then, the ReplicatedMap
's put
method adds a Replicated Map
entry and updates it. The method remove
removes the entry.
import com.hazelcast.core.EntryEvent;
import com.hazelcast.core.EntryListener;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.ReplicatedMap;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ReplicatedMap<String, Customer> customers =
hazelcastInstance.getReplicatedMap( "customers" );
customers.addEntryListener( new EntryListener<String, Customer>() {
@Override
public void entryAdded( EntryEvent<String, Customer> event ) {
log( "Entry added: " + event );
}
@Override
public void entryUpdated( EntryEvent<String, Customer> event ) {
log( "Entry updated: " + event );
}
@Override
public void entryRemoved( EntryEvent<String, Customer> event ) {
log( "Entry removed: " + event );
}
@Override
public void entryEvicted( EntryEvent<String, Customer> event ) {
// Currently not supported, will never fire
}
});
customers.put( "1", new Customer( "Joe", "Smith" ) ); // add event
customers.put( "1", new Customer( "Ali", "Selam" ) ); // update event
customers.remove( "1" ); // remove event
You can register for Hazelcast entry events so you will be notified when those events occur. Event Listeners are cluster-wide--when a listener is registered in one member of cluster, it is actually registered for events that originated at any member in the cluster. When a new member joins, events originated at the new member will also be delivered.
An Event is created only if you registered an event listener. If no listener is registered, then no event will be created. If you provided a predicate when you registered the event listener, pass the predicate before sending the event to the listener (member/client).
As a rule of thumb, your event listener should not implement heavy processes in its event methods that block the thread for a long time. If needed, you can use ExecutorService
to transfer long running processes to another thread and thus offload the current listener thread.
NOTE: In a failover scenario, events are not highly available and may get lost. Eventing mechanism is being improved for failover scenarios.
Hazelcast offers the following event listeners:
HazelcastInstance
lifecycle events.IMap
and MultiMap
entry events.IQueue
, ISet
and IList
item events.ITopic
message events.The Membership Listener interface has methods that are invoked for the following events.
memberAdded
: A new member is added to the cluster.memberRemoved
: An existing member leaves the cluster.memberAttributeChanged
: An attribute of a member is changed. Please refer to Defining Member Attributes to learn about member attributes.To write a Membership Listener class, you implement the MembershipListener interface and its methods.
The following is an example Membership Listener class.
public class ClusterMembershipListener
implements MembershipListener {
public void memberAdded(MembershipEvent membershipEvent) {
System.err.println("Added: " + membershipEvent);
}
public void memberRemoved(MembershipEvent membershipEvent) {
System.err.println("Removed: " + membershipEvent);
}
public void memberAttributeChanged(MemberAttributeEvent memberAttributeEvent) {
System.err.println("Member attribute changed: " + memberAttributeEvent);
}
}
When a respective event is fired, the membership listener outputs the addresses of the members that joined and left, and also which attribute changed on which member.
After you create your class, you can configure your cluster to include the membership listener. Below is an example using the method addMembershipListener
.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
hazelcastInstance.getCluster().addMembershipListener( new ClusterMembershipListener() );
With the above approach, there is the possibility of missing events between the creation of the instance and registering the listener. To overcome this race condition, Hazelcast allows you to register listeners in the configuration. You can register listeners using declarative, programmatic, or Spring configuration, as shown below.
The following is an example programmatic configuration.
Config config = new Config();
config.addListenerConfig(
new ListenerConfig( "com.your-package.ClusterMembershipListener" ) );
The following is an example of the equivalent declarative configuration.
<hazelcast>
...
<listeners>
<listener type="membership-listener">
com.your-package.ClusterMembershipListener
</listener>
</listeners>
...
</hazelcast>
The following is an example of the equivalent Spring configuration.
<hz:listeners>
<hz:listener class-name="com.your-package.ClusterMembershipListener"/>
<hz:listener implementation="MembershipListener"/>
</hz:listeners>
The Distributed Object Listener methods distributedObjectCreated
and distributedObjectDestroyed
are invoked when a distributed object is created and destroyed throughout the cluster. To write a Distributed Object Listener class, you implement the DistributedObjectListener interface and its methods.
The following is an example Distributed Object Listener class.
public class SampleDistObjListener implements DistributedObjectListener {
public static void main(String[] args) {
SampleDistObjListener sample = new SampleDistObjListener();
Config config = new Config();
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(config);
hazelcastInstance.addDistributedObjectListener(sample);
Collection<DistributedObject> distributedObjects = hazelcastInstance.getDistributedObjects();
for (DistributedObject distributedObject : distributedObjects) {
System.out.println(distributedObject.getName() + "," + distributedObject.getId());
}
}
@Override
public void distributedObjectCreated(DistributedObjectEvent event) {
DistributedObject instance = event.getDistributedObject();
System.out.println("Created " + instance.getName() + "," + instance.getId());
}
@Override
public void distributedObjectDestroyed(DistributedObjectEvent event) {
DistributedObject instance = event.getDistributedObject();
System.out.println("Destroyed " + instance.getName() + "," + instance.getId());
}
}
When a respective event is fired, the distributed object listener outputs the event type, and the name, service (for example, if a Map service provides the distributed object, than it is a Map object), and ID of the object.
After you create your class, you can configure your cluster to include distributed object listeners. Below is an example using the method addDistributedObjectListener
. You can also see this portion in the above class creation.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
SampleDistObjListener sample = new SampleDistObjListener();
hazelcastInstance.addDistributedObjectListener( sample );
With the above approach, there is the possibility of missing events between the creation of the instance and registering the listener. To overcome this race condition, Hazelcast allows you to register the listeners in the configuration. You can register listeners using declarative, programmatic, or Spring configuration, as shown below.
The following is an example programmatic configuration.
config.addListenerConfig(
new ListenerConfig( "com.your-package.SampleDistObjListener" ) );
The following is an example of the equivalent declarative configuration.
<hazelcast>
...
<listeners>
<listener>
com.your-package.SampleDistObjListener
</listener>
</listeners>
...
</hazelcast>
The following is an example of the equivalent Spring configuration.
<hz:listeners>
<hz:listener class-name="com.your-package.SampleDistObjListener"/>
<hz:listener implementation="DistributedObjectListener"/>
</hz:listeners>
The Migration Listener interface has methods that are invoked for the following events:
migrationStarted
: A partition migration is started.migrationCompleted
: A partition migration is completed.migrationFailed
: A partition migration failed.To write a Migration Listener class, you implement the DistributedObjectListener interface and its methods.
The following is an example Migration Listener class.
public class ClusterMigrationListener implements MigrationListener {
@Override
public void migrationStarted(MigrationEvent migrationEvent) {
System.err.println("Started: " + migrationEvent);
}
@Override
public void migrationCompleted(MigrationEvent migrationEvent) {
System.err.println("Completed: " + migrationEvent);
}
@Override
public void migrationFailed(MigrationEvent migrationEvent) {
System.err.println("Failed: " + migrationEvent);
}
}
When a respective event is fired, the migration listener outputs the partition ID, status of the migration, the old member and the new member. The following is an example output.
Started: MigrationEvent{partitionId=98, oldOwner=Member [127.0.0.1]:5701,
newOwner=Member [127.0.0.1]:5702 this}
After you create your class, you can configure your cluster to include migration listeners. Below is an example using the method addMigrationListener
.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
PartitionService partitionService = hazelcastInstance.getPartitionService();
partitionService.addMigrationListener( new ClusterMigrationListener );
With the above approach, there is the possibility of missing events between the creation of the instance and registering the listener. To overcome this race condition, Hazelcast allows you to register the listeners in the configuration. You can register listeners using declarative, programmatic, or Spring configuration, as shown below.
The following is an example programmatic configuration.
config.addListenerConfig(
new ListenerConfig( "com.your-package.ClusterMigrationListener" ) );
The following is an example of the equivalent declarative configuration.
<hazelcast>
...
<listeners>
<listener>
com.your-package.ClusterMigrationListener
</listener>
</listeners>
...
</hazelcast>
The following is an example of the equivalent Spring configuration.
<hz:listeners>
<hz:listener class-name="com.your-package.ClusterMigrationListener"/>
<hz:listener implementation="MigrationListener"/>
</hz:listeners>
Hazelcast provides fault-tolerance by keeping multiple copies of your data. For each partition, one of your cluster members becomes the owner and some of the other members become replica members, based on your configuration. Nevertheless, data loss may occur if a few members crash simultaneously.
Let`s consider the following example with three members: N1, N2, N3 for a given partition-0. N1 is owner of partition-0, and N2 and N3 are the first and second replicas respectively. If N1 and N2 crash simultaneously, partition-0 loses its data that is configured with less than two backups. For instance, if we configure a map with one backup, that map loses its data in partition-0 since both owner and first replica of partition-0 have crashed. However, if we configure our map with two backups, it does not lose any data since a copy of partition-0's data for the given map also resides in N3.
The Partition Lost Listener notifies for possible data loss occurrences with the information of how many replicas are lost for a partition. It listens to PartitionLostEvent
instances. Partition lost events are dispatched per partition.
Partition loss detection is done after a member crash is detected by the other members and the crashed member is removed from the cluster. Please note that false-positive PartitionLostEvent
instances may be fired on the network split errors.
To write a Partition Lost Listener, you implement the PartitionLostListener interface and its partitionLost
method, which is invoked when a partition loses its owner and all backups.
The following is an example Partition Lost Listener class.
public class ConsoleLoggingPartitionLostListener implements PartitionLostListener {
@Override
public void partitionLost(PartitionLostEvent event) {
System.out.println(event);
}
}
When a PartitionLostEvent
is fired, the partition lost listener given above outputs the partition ID, the replica index that is lost, and the member that has detected the partition loss. The following is an example output.
com.hazelcast.partition.PartitionLostEvent{partitionId=242, lostBackupCount=0,
eventSource=Address[192.168.2.49]:5701}
After you create your class, you can configure your cluster programmatically or declaratively to include the partition lost listener. Below is an example of its programmatic configuration.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
hazelcastInstance.getPartitionService().addPartitionLostListener( new ConsoleLoggingPartitionLostListener() );
The following is an example of the equivalent declarative configuration.
<hazelcast>
...
<partition-lost-listeners>
<partition-lost-listener>
com.your-package.ConsoleLoggingPartitionLostListener
</partition-lost-listener>
</partition-lost-listeners>
...
</hazelcast>
The Lifecycle Listener notifies for the following events:
STARTING
: A member is starting.STARTED
: A member started.SHUTTING_DOWN
: A member is shutting down.SHUTDOWN
: A member's shutdown has completed.MERGING
: A member is merging with the cluster.MERGED
: A member's merge operation has completed.CLIENT_CONNECTED
: A Hazelcast Client connected to the cluster.CLINET_DISCONNECTED
: A Hazelcast Client disconnected from the cluster.The following is an example Lifecycle Listener class.
public class NodeLifecycleListener implements LifecycleListener {
@Override
public void stateChanged(LifecycleEvent event) {
System.err.println(event);
}
}
This listener is local to an individual member. It notifies the application that uses Hazelcast about the events mentioned above for a particular member.
After you create your class, you can configure your cluster to include lifecycle listeners. Below is an example using the method addLifecycleListener
.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
hazelcastInstance.getLifecycleService().addLifecycleListener( new NodeLifecycleListener() );
With the above approach, there is the possibility of missing events between the creation of the instance and registering the listener. To overcome this race condition, Hazelcast allows you to register the listeners in the configuration. You can register listeners using declarative, programmatic, or Spring configuration, as shown below.
The following is an example programmatic configuration.
config.addListenerConfig(
new ListenerConfig( "com.your-package.NodeLifecycleListener" ) );
The following is an example of the equivalent declarative configuration.
<hazelcast>
...
<listeners>
<listener>
com.your-package.NodeLifecycleListener
</listener>
</listeners>
...
</hazelcast>
The following is an example of the equivalent Spring configuration.
<hz:listeners>
<hz:listener class-name="com.your-package.NodeLifecycleListener"/>
<hz:listener implementation="LifecycleListener"/>
</hz:listeners>
You can listen to map-wide or entry-based events using the listeners provided by the Hazelcast's eventing framework. To listen to these events, implement a MapListener
sub-interface.
A map-wide event is fired as a result of a map-wide operation. For
example, IMap#clear
or IMap#evictAll
.
An entry-based event is fired after the operations that affect a
specific entry. For example, IMap#remove
or IMap#evict
.
To catch an event, you should explicitly
implement a corresponding sub-interface of a MapListener
,
such as EntryAddedListener
or MapClearedListener
.
NOTE: The EntryListener
interface still can be implemented (we kept
it for backward compatibility reasons). However, if you need to listen to a
different event, one that is not available in the EntryListener
interface, you should also
implement a relevant MapListener
sub-interface.
Let's take a look at the following class example.
public class Listen {
public static void main( String[] args ) {
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
IMap<String, String> map = hz.getMap( "somemap" );
map.addEntryListener( new MyEntryListener(), true );
System.out.println( "EntryListener registered" );
}
static class MyEntryListener implements EntryAddedListener<String, String>,
EntryRemovedListener<String, String>,
EntryUpdatedListener<String, String>,
EntryEvictedListener<String, String> ,
MapEvictedListener,
MapClearedListener {
@Override
public void entryAdded( EntryEvent<String, String> event ) {
System.out.println( "Entry Added:" + event );
}
@Override
public void entryRemoved( EntryEvent<String, String> event ) {
System.out.println( "Entry Removed:" + event );
}
@Override
public void entryUpdated( EntryEvent<String, String> event ) {
System.out.println( "Entry Updated:" + event );
}
@Override
public void entryEvicted( EntryEvent<String, String> event ) {
System.out.println( "Entry Evicted:" + event );
}
@Override
public void mapEvicted( MapEvent event ) {
System.out.println( "Map Evicted:" + event );
}
@Override
public void mapCleared( MapEvent event ) {
System.out.println( "Map Cleared:" + event );
}
}
}
Now, let's perform some modifications on the map entries using the following example code.
public class Modify {
public static void main( String[] args ) {
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
IMap<String, String> map = hz.getMap( "somemap");
String key = "" + System.nanoTime();
String value = "1";
map.put( key, value );
map.put( key, "2" );
map.delete( key );
}
}
If you execute the Listen
class and then the Modify
class, you get the following output
produced by the Listen
class.
entryAdded:EntryEvent {Address[192.168.1.100]:5702} key=251359212222282,
oldValue=null, value=1, event=ADDED, by Member [192.168.1.100]:5702
entryUpdated:EntryEvent {Address[192.168.1.100]:5702} key=251359212222282,
oldValue=1, value=2, event=UPDATED, by Member [192.168.1.100]:5702
entryRemoved:EntryEvent {Address[192.168.1.100]:5702} key=251359212222282,
oldValue=2, value=2, event=REMOVED, by Member [192.168.1.100]:5702
public class MyEntryListener implements EntryListener{
private Executor executor = Executors.newFixedThreadPool(5);
@Override
public void entryAdded(EntryEvent event) {
executor.execute(new DoSomethingWithEvent(event));
}
...
NOTE: Please note that the method IMap.clear()
does not fire an "EntryRemoved" event, but fires a "MapCleared" event.
A map listener runs on the event threads that are also used by the other listeners. For example, the collection listeners and pub/sub message listeners. This means that the entry listeners can access other partitions. Consider this when you run long tasks, since listening to those tasks may cause the other map/event listeners to starve.
You can listen to MapPartitionLostEvent
instances by registering an implementation
of MapPartitionLostListener
, which is also a sub-interface of MapListener
.
Let`s consider the following example code:
public static void main(String[] args) {
Config config = new Config();
config.getMapConfig("map").setBackupCount(1); // might lose data if any member crashes
HazelcastInstance instance = HazelcastInstanceFactory.newHazelcastInstance(config);
IMap<Object, Object> map = instance1.getMap("map");
map.put(0, 0);
map.addPartitionLostListener(new MapPartitionLostListener() {
@Override
public void partitionLost(MapPartitionLostEvent event) {
System.out.println(event);
}
});
}
Within this example code, a MapPartitionLostListener
implementation is registered to a map
that is configured with one backup. For this particular map and any of the partitions in the
system, if the partition owner member and its first backup member crash simultaneously, the
given MapPartitionLostListener
receives a
corresponding MapPartitionLostEvent
. If only a single member crashes in the cluster,
there will be no MapPartitionLostEvent
fired for this map since backups for the partitions
owned by the crashed member are kept on other members.
Please refer to Listening for Partition Lost Events for more information about partition lost detection and partition lost events.
After you create your listener class, you can configure your cluster to include map listeners using the method addEntryListener
(as you can see in the example Listen
class above). Below is the related portion from this code, showing how to register a map listener.
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
IMap<String, String> map = hz.getMap( "somemap" );
map.addEntryListener( new MyEntryListener(), true );
With the above approach, there is the possibility of missing events between the creation of the instance and registering the listener. To overcome this race condition, Hazelcast allows you to register listeners in configuration. You can register listeners using declarative, programmatic, or Spring configuration, as shown below.
The following is an example programmatic configuration.
mapConfig.addEntryListenerConfig(
new EntryListenerConfig( "com.yourpackage.MyEntryListener",
false, false ) );
The following is an example of the equivalent declarative configuration.
<hazelcast>
...
<map name="somemap">
...
<entry-listeners>
<entry-listener include-value="false" local="false">
com.your-package.MyEntryListener
</entry-listener>
</entry-listeners>
</map>
...
</hazelcast>
The following is an example of the equivalent Spring configuration.
<hz:map name="somemap">
<hz:entry-listeners>
<hz:entry-listener include-value="true"
class-name="com.hazelcast.spring.DummyEntryListener"/>
<hz:entry-listener implementation="dummyEntryListener" local="true"/>
</hz:entry-listeners>
</hz:map>
As you see, there are attributes of the map listeners in the above examples: include-value
and local
. The attribute include-value
is a boolean attribute that is optional, and if you set it to true
, the map event will contain the map value. Its default value is true
.
The attribute local
is also a boolean attribute that is optional, and if you set it to true
, you can listen to the map on the local member. Its default value is false
.
You can listen to entry-based events in the MultiMap using EntryListener
. The following is an example listener class for MultiMap.
public class Listen {
public static void main( String[] args ) {
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
MultiMap<String, String> map = hz.getMultiMap( "somemap" );
map.addEntryListener( new MyEntryListener(), true );
System.out.println( "EntryListener registered" );
}
static class SampleEntryListener implements EntryListener<String, String>{
@Override
public void entryAdded( EntryEvent<String, String> event ) {
System.out.println( "Entry Added:" + event );
}
@Override
public void entryRemoved( EntryEvent<String, String> event ) {
System.out.println( "Entry Removed:" + event );
}
}
}
After you create your listener class, you can configure your cluster to include MultiMap listeners using the method addEntryListener
(as you can see in the example Listen
class above). Below is the related portion from this code, showing how to register a map listener.
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
MultiMap<String, String> map = hz.getMultiMap( "somemap" );
map.addEntryListener( new MyEntryListener(), true );
With the above approach, there is the possibility of missing events between the creation of the instance and registering the listener. To overcome this race condition, Hazelcast allows you to register listeners in the configuration. You can register listeners using declarative, programmatic, or Spring configuration, as shown below.
The following is an example programmatic configuration.
multiMapConfig.addEntryListenerConfig(
new EntryListenerConfig( "com.your-package.SampleEntryListener",
false, false ) );
The following is an example of the equivalent declarative configuration.
<hazelcast>
...
<multimap name="somemap">
<value-collection-type>SET</value-collection-type>
<entry-listeners>
<entry-listener include-value="false" local="false">
com.your-package.SampleEntryListener
</entry-listener>
</entry-listeners>
</multimap>
...
</hazelcast>
The following is an example of the equivalent Spring configuration.
<hz:multimap name="default" value-collection-type="LIST">
<hz:entry-listeners>
<hz:entry-listener include-value="false"
class-name="com.your-package.SampleEntryListener"/>
<hz:entry-listener implementation="EntryListener" local="false"/>
</hz:entry-listeners>
</hz:multimap>
As you see, there are attributes of the MultiMap listeners in the above examples: include-value
and local
. The attribute include-value
is a boolean attribute that is optional, and if you set it to true
, the MultiMap event will contain the map value. Its default value is true
.
The attribute local
is also a boolean attribute that is optional, and if you set it to true
, you can listen to the MultiMap on the local member. Its default value is false
.
The Item Listener is used by the Hazelcast IQueue
, ISet
and IList
interfaces.
To write an Item Listener class, you implement the ItemListener interface and its methods itemAdded
and itemRemoved
. These methods
are invoked when an item is added or removed.
The following is an example Item Listener class for an ISet
structure.
public class SampleItemListener implements ItemListener {
public static void main( String[] args ) {
SampleItemListener sampleItemListener = new SampleItemListener();
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ICollection<Price> set = hazelcastInstance.getSet( "default" );
set.addItemListener( sampleItemListener, true );
Price price = new Price( 10, time1 )
set.add( price );
set.remove( price );
}
public void itemAdded( Object item ) {
System.out.println( "Item added = " + item );
}
public void itemRemoved( Object item ) {
System.out.println( "Item removed = " + item );
}
}
NOTE: You can use ICollection
when creating any of the collection (queue, set and list) data structures, as shown above. You can also use IQueue
, ISet
or IList
instead of ICollection
.
After you create your class, you can configure your cluster to include item listeners. Below is an example using the method addItemListener
for ISet
(it applies also to IQueue
and IList
). You can also see this portion in the above class creation.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ICollection<Price> set = hazelcastInstance.getSet( "default" );
// or ISet<Prices> set = hazelcastInstance.getSet( "default" );
default.addItemListener( sampleItemListener, true );
With the above approach, there is the possibility of missing events between the creation of the instance and registering the listener. To overcome this race condition, Hazelcast allows you to register listeners in the configuration. You can register listeners using declarative, programmatic, or Spring configuration, as shown below.
The following is an example programmatic configuration.
setConfig.addItemListenerConfig(
new ItemListenerConfig( "com.your-package.SampleItemListener", true ) );
The following is an example of the equivalent declarative configuration.
<hazelcast>
...
<item-listeners>
<item-listener include-value="true">
com.your-package.SampleItemListener
</item-listener>
</item-listeners>
...
</hazelcast>
The following is an example of the equivalent Spring configuration.
<hz:set name="default" >
<hz:item-listeners>
<hz:item-listener include-value="true"
class-name="com.your-package.SampleItemListener"/>
</hz:item-listeners>
</hz:set>
As you see, there is an attribute in the above examples: include-value
. It is a boolean attribute that is optional, and if you set it to true
, the item event will contain the item value. Its default value is true
.
There is also another attribute called local
, which is not shown in the above examples. It is also a boolean attribute that is optional, and if you set it to true
, you can listen to the items on the local member. Its default value is false
.
The Message Listener is used by the ITopic
interface. It notifies when a message is received for the registered topic.
To write a Message Listener class, you implement the MessageListener interface and its method onMessage
, which is invoked
when a message is received for the registered topic.
The following is an example Message Listener class.
public class SampleMessageListener implements MessageListener<MyEvent> {
public static void main( String[] args ) {
SampleMessageListener sample = new SampleMessageListener();
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ITopic topic = hazelcastInstance.getTopic( "default" );
topic.addMessageListener( sample );
topic.publish( new MyEvent() );
}
public void onMessage( Message<MyEvent> message ) {
MyEvent myEvent = message.getMessageObject();
System.out.println( "Message received = " + myEvent.toString() );
if ( myEvent.isHeavyweight() ) {
messageExecutor.execute( new Runnable() {
public void run() {
doHeavyweightStuff( myEvent );
}
} );
}
}
After you create your class, you can configure your cluster to include message listeners. Below is an example using the method addMessageListener
. You can also see this portion in the above class creation.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
ITopic topic = hazelcastInstance.getTopic( "default" );
topic.addMessageListener( sample );
With the above approach, there is the possibility of missing messaging events between the creation of the instance and registering the listener. To overcome this race condition, Hazelcast allows you to register this listener in the configuration. You can register it using declarative, programmatic, or Spring configuration, as shown below.
The following is an example programmatic configuration.
topicConfig.addMessageListenerConfig(
new ListenerConfig( "com.your-package.SampleMessageListener" ) );
The following is an example of the equivalent declarative configuration.
<hazelcast>
...
<topic name="default">
<message-listeners>
<message-listener>
com.your-package.SampleMessageListener
</message-listener>
</message-listeners>
</topic>
...
</hazelcast>
The following is an example of the equivalent Spring configuration.
<hz:topic name="default">
<hz:message-listeners>
<hz:message-listener
class-name="com.your-package.SampleMessageListener"/>
</hz:message-listeners>
</hz:topic>
The Client Listener is used by the Hazelcast cluster members. It notifies the cluster members when a client is connected to or disconnected from the cluster.
To write a client listener class, you implement the ClientListener
interface and its methods clientConnected
and clientDisconnected
,
which are invoked when a client is connected to or disconnected from the cluster. You can add your client listener as shown below.
hazelcast.getClientService().addClientListener(SampleClientListener);
The following is the equivalent declarative configuration.
<listeners>
<listener>
com.your-package.SampleClientListener
</listener>
</listeners>
The following is the equivalent configuration in the Spring context.
<hz:listeners>
<hz:listener class-name="com.your-package.SampleClientListener"/>
<hz:listener implementation="com.your-package.SampleClientListener"/>
</hz:listeners>
NOTE: You can also add event listeners to a Hazelcast client. Please refer to Client Listenerconfig for the related information.
You can add event listeners to a Hazelcast Java client. You can configure the following listeners to listen to the events on the client side. Please see the respective sections under the Event Listeners for Hazelcast Members section for example code.
RELATED INFORMATION
Please refer to the Client Listenerconfig section for more information.
hazelcast.event.queue.capacity
: default value is 1000000hazelcast.event.queue.timeout.millis
: default value is 250hazelcast.event.thread.count
: default value is 5A striped executor in each cluster member controls and dispatches the received events. This striped executor also guarantees the event order. For all events in Hazelcast, the order in which events are generated and the order in which they are published are guaranteed for given keys. For map and multimap, the order is preserved for the operations on the same key of the entry. For list, set, topic and queue, the order is preserved for events on that instance of the distributed data structure.
To achieve the order guarantee, you make only one thread responsible for a particular set of events (entry events of a key in a map, item events of a collection, etc.) in StripedExecutor
(within com.hazelcast.util.executor
).
If the event queue reaches its capacity (hazelcast.event.queue.capacity
) and the last item cannot be put into the event queue for the period specified in hazelcast.event.queue.timeout.millis
, these events will be dropped with a warning message, such as "EventQueue overloaded".
If event listeners perform a computation that takes a long time, the event queue can reach its maximum capacity and lose events. For map and multimap, you can configure hazelcast.event.thread.count
to a higher value so that fewer collisions occur for keys, and therefore worker threads will not block each other in StripedExecutor
. For list, set, topic and queue, you should offload heavy work to another thread. To preserve order guarantee, you should implement similar logic with StripedExecutor
in the offloaded thread pool.
This chapter explains Hazelcast's executor service, durable executor service, and entry processor implementations.
One of the coolest features of Java 1.5 is the Executor framework, which allows you to asynchronously execute your tasks (logical units of work), such as database queries, complex calculations, and image rendering.
The default implementation of this framework (ThreadPoolExecutor
) is designed to run within a single JVM (cluster member). In distributed systems, this implementation is not desired since you may want a task submitted in one JVM and processed in another one. Hazelcast offers IExecutorService
for you to use in distributed environments. It implements java.util.concurrent.ExecutorService
to serve the applications requiring computational and data processing power.
With IExecutorService
, you can execute tasks asynchronously and perform other useful tasks. If your task execution takes longer than expected, you can cancel the task execution. Tasks should be Serializable
since they will be distributed.
In the Java Executor framework, you implement tasks two ways: Callable or Runnable.
java.util.concurrent.Callable
.java.util.concurrent.Runnable
.In Hazelcast, when you implement a task as java.util.concurrent.Callable
(a task that returns a value), you implement Callable and Serializable.
Below is an example of a Callable task. SumTask prints out map keys and returns the summed map values.
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.HazelcastInstanceAware;
import com.hazelcast.core.IMap;
import java.io.Serializable;
import java.util.concurrent.Callable;
public class SumTask
implements Callable<Integer>, Serializable, HazelcastInstanceAware {
private transient HazelcastInstance hazelcastInstance;
public void setHazelcastInstance( HazelcastInstance hazelcastInstance ) {
this.hazelcastInstance = hazelcastInstance;
}
public Integer call() throws Exception {
IMap<String, Integer> map = hazelcastInstance.getMap( "map" );
int result = 0;
for ( String key : map.localKeySet() ) {
System.out.println( "Calculating for key: " + key );
result += map.get( key );
}
System.out.println( "Local Result: " + result );
return result;
}
}
Another example is the Echo callable below. In its call() method, it returns the local member and the input passed in. Remember that instance.getCluster().getLocalMember()
returns the local member and toString()
returns the member's address (IP + port) in String form, just to see which member actually executed the code for our example. Of course, the call()
method can do and return anything you like.
import java.util.concurrent.Callable;
import java.io.Serializable;
public class Echo implements Callable<String>, Serializable {
String input = null;
public Echo() {
}
public Echo(String input) {
this.input = input;
}
public String call() {
Config cfg = new Config();
HazelcastInstance instance = Hazelcast.newHazelcastInstance(cfg);
return instance.getCluster().getLocalMember().toString() + ":" + input;
}
}
To execute a callable task with the executor framework:
ExecutorService
instance (generally via Executors
).Future
. Future
object to retrieve the result as shown in the code example below.Below, the Echo task is executed.
ExecutorService executorService = Executors.newSingleThreadExecutor();
Future<String> future = executorService.submit( new Echo( "myinput") );
//while it is executing, do some useful stuff
//when ready, get the result of your execution
String result = future.get();
Please note that the Echo callable in the above code sample also implements a Serializable interface, since it may be sent to another member to be processed.
NOTE: When a task is deserialized, HazelcastInstance needs to be accessed. To do this, the task should implement HazelcastInstanceAware
interface. Please see the HazelcastInstanceAware Interface section for more information.
In Hazelcast, when you implement a task as java.util.concurrent.runnable
(a task that does not return a value), you implement Runnable and Serializable.
Below is Runnable example code. It is a task that waits for some time and echoes a message.
public class EchoTask implements Runnable, Serializable {
private final String msg;
public EchoTask( String msg ) {
this.msg = msg;
}
@Override
public void run() {
try {
Thread.sleep( 5000 );
} catch ( InterruptedException e ) {
}
System.out.println( "echo:" + msg );
}
}
To execute the runnable task:
HazelcastInstance
.Now let's write a class that submits and executes these echo messages. Executor is retrieved from HazelcastInstance
and 1000 echo tasks are submitted.
public class MasterMember {
public static void main( String[] args ) throws Exception {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IExecutorService executor = hazelcastInstance.getExecutorService( "exec" );
for ( int k = 1; k <= 1000; k++ ) {
Thread.sleep( 1000 );
System.out.println( "Producing echo task: " + k );
executor.execute( new EchoTask( String.valueOf( k ) ) );
}
System.out.println( "EchoTaskMain finished!" );
}
}
You can scale the Executor service both vertically (scale up) and horizontally (scale out).
To scale up, you should improve the processing capacity of the cluster member (JVM). You can do this by increasing the pool-size
property mentioned in Configuring Executor Service (i.e., increasing the thread count). However, please be aware of your member's capacity. If you think it cannot handle such an additional load caused by increasing the thread count, you may want to consider improving the member's resources (CPU, memory, etc.). As an example, set the pool-size
to 5 and run the above MasterMember
. You will see that EchoTask
is run as soon as it is produced.
To scale out, add more members instead of increasing only one member's capacity. In reality, you may want to expand your cluster by adding more physical or virtual machines. For example, in the EchoTask example in the Runnable section, you can create another Hazelcast instance. That instance will automatically get involved in the executions started in MasterMember
and start processing.
The distributed executor service is a distributed implementation of java.util.concurrent.ExecutorService
. It allows you to execute your code in the cluster. In this section, the code examples are based on the Echo class above (please note that the Echo class is Serializable
). The code examples show how Hazelcast can execute your code (Runnable, Callable
):
echoOnTheMember
: On a specific cluster member you choose with the IExecutorService
submitToMember
method.echoOnTheMemberOwningTheKey
: On the member owning the key you choose with the IExecutorService
submitToKeyOwner
method.echoOnSomewhere
: On the member Hazelcast picks with the IExecutorService
submit
method.echoOnMembers
: On all or a subset of the cluster members with the IExecutorService
submitToMembers
method.import com.hazelcast.core.Member;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.IExecutorService;
import java.util.concurrent.Callable;
import java.util.concurrent.Future;
import java.util.Set;
public void echoOnTheMember( String input, Member member ) throws Exception {
Callable<String> task = new Echo( input );
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IExecutorService executorService =
hazelcastInstance.getExecutorService( "default" );
Future<String> future = executorService.submitToMember( task, member );
String echoResult = future.get();
}
public void echoOnTheMemberOwningTheKey( String input, Object key ) throws Exception {
Callable<String> task = new Echo( input );
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IExecutorService executorService =
hazelcastInstance.getExecutorService( "default" );
Future<String> future = executorService.submitToKeyOwner( task, key );
String echoResult = future.get();
}
public void echoOnSomewhere( String input ) throws Exception {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IExecutorService executorService =
hazelcastInstance.getExecutorService( "default" );
Future<String> future = executorService.submit( new Echo( input ) );
String echoResult = future.get();
}
public void echoOnMembers( String input, Set<Member> members ) throws Exception {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IExecutorService executorService =
hazelcastInstance.getExecutorService( "default" );
Map<Member, Future<String>> futures = executorService
.submitToMembers( new Echo( input ), members );
for ( Future<String> future : futures.values() ) {
String echoResult = future.get();
// ...
}
}
NOTE: You can obtain the set of cluster members via HazelcastInstance#getCluster().getMembers()
call.
A task in the code that you execute in a cluster might take longer than expected. If you cannot stop/cancel that task, it will keep eating your resources.
To cancel a task, you can use the standard Java executor framework's cancel()
API. This framework encourages us to code and design for cancellations, a highly ignored part of software development.
The Fibonacci callable class below calculates the Fibonacci number for a given number. In the calculate
method, we check if the current thread is interrupted so that the code can respond to cancellations once the execution is started.
public class Fibonacci<Long> implements Callable<Long>, Serializable {
int input = 0;
public Fibonacci() {
}
public Fibonacci( int input ) {
this.input = input;
}
public Long call() {
return calculate( input );
}
private long calculate( int n ) {
if ( Thread.currentThread().isInterrupted() ) {
return 0;
}
if ( n <= 1 ) {
return n;
} else {
return calculate( n - 1 ) + calculate( n - 2 );
}
}
}
The fib()
method below submits the Fibonacci calculation task above for number 'n' and waits a maximum of 3 seconds for the result. If the execution does not completed in three seconds, future.get()
will throw a TimeoutException
and upon catching it, we cancel the execution, saving some CPU cycles.
long fib( int n ) throws Exception {
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IExecutorService es = hazelcastInstance.getExecutorService();
Future future = es.submit( new Fibonacci( n ) );
try {
return future.get( 3, TimeUnit.SECONDS );
} catch ( TimeoutException e ) {
future.cancel( true );
}
return -1;
}
fib(20)
will probably take less than 3 seconds. However, fib(50)
will take much longer. (This is not an example for writing better Fibonacci calculation code, but for showing how to cancel a running execution that takes too long.) The method future.cancel(false)
can only cancel execution before it is running (executing), but future.cancel(true)
can interrupt running executions provided that your code is able to handle the interruption. If you are willing to cancel an already running task, then your task should be designed to handle interruptions. If the calculate (int n)
method did not have the (Thread.currentThread().isInterrupted())
line, then you would not be able to cancel the execution after it is started.
You can use the ExecutionCallback
offered by Hazelcast to asynchronously be notified when the execution is done.
onResponse
method.onFailure
method.Let's use the Fibonacci series to explain this. The example code below is the calculation that will be executed. Note that it is Callable and Serializable.
public class Fibonacci<Long> implements Callable<Long>, Serializable {
int input = 0;
public Fibonacci() {
}
public Fibonacci( int input ) {
this.input = input;
}
public Long call() {
return calculate( input );
}
private long calculate( int n ) {
if (n <= 1) {
return n;
} else {
return calculate( n - 1 ) + calculate( n - 2 );
}
}
}
The example code below submits the Fibonacci calculation to ExecutionCallback
and prints the result asynchronously. ExecutionCallback
has the methods onResponse
and onFailure
. In this example code, onResponse
is called upon a valid response and prints the calculation result, whereas onFailure
is called upon a failure and prints the stacktrace.
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.ExecutionCallback;
import com.hazelcast.core.IExecutorService;
import java.util.concurrent.Future;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IExecutorService es = hazelcastInstance.getExecutorService();
Callable<Long> task = new Fibonacci( 10 );
es.submit(task, new ExecutionCallback<Long> () {
@Override
public void onResponse( Long response ) {
System.out.println( "Fibonacci calculation result = " + response );
}
@Override
public void onFailure( Throwable t ) {
t.printStackTrace();
}
};
As previously mentioned, it is possible to indicate where in the Hazelcast cluster the Runnable
or Callable
is executed. Usually you execute these in the cluster based on the location of a key or a set of keys, or you allow Hazelcast to select a member.
If you want more control over where your code runs, use the MemberSelector
interface. For example, you may want certain tasks to run only on certain members, or you may wish to implement some form of custom load balancing regime. The MemberSelector
is an interface that you can implement and then provide to the IExecutorService
when you submit or execute.
The select(Member)
method is called for every available member in the cluster. Implement this method to decide if the member is going to be used or not.
In a simple example shown below, we select the cluster members based on the presence of an attribute.
public class MyMemberSelector implements MemberSelector {
public boolean select(Member member) {
return Boolean.TRUE.equals(member.getAttribute("my.special.executor"));
}
}
You can use MemberSelector
instances provided by the com.hazelcast.cluster.memberselector.MemberSelectors
class. For example, you can select a lite member for running a task using com.hazelcast.cluster.memberselector.MemberSelectors#LITE_MEMBER_SELECTOR
.
The following are example configurations for executor service.
Declarative:
<executor-service name="exec">
<pool-size>1</pool-size>
<queue-capacity>10</queue-capacity>
<statistics-enabled>true</statistics-enabled>
</executor-service>
Programmatic:
Config config = new Config();
ExecutorConfig executorConfig = config.getExecutorConfig("exec");
executorConfig.setPoolSize( "1" ).setQueueCapacity( "10" )
.setStatisticsEnabled( true );
Executor service configuration has the following elements.
pool-size
: The number of executor threads per Member for the Executor. By default, Executor is configured to have 16 threads in the pool. You can change that with this element.queue-capacity
: Executor's task queue capacity; the number of tasks this queue can hold.statistics-enabled
: You can retrieve some statistics (such as pending operations count, started operations count, completed operations count, and cancelled operations count) by setting this parameter's value to true
. The method for retrieving the statistics is getLocalExecutorStats()
.Hazelcast's durable executor service is a data structure which is able to store an execution task both on the executing Hazelcast member and its backup member(s), if configured. By this way, you do not lose any tasks if a member goes down or any results if the submitter (member or client) goes down while executing the task. When using the durable executor service you can either submit or execute a task randomly or on the owner of a provided key. Note that in executor service, you can submit or execute tasks to/on the selected member(s).
Processing of the tasks when using durable executor service involves two invocations:
As you may already know, Hazelcast's executor service returns a future
representing the task to the user. With the above two-invocations approach, it is guaranteed that the task is executed before the future
returns and you can track the response of a submitted task with a unique ID. Hazelcast stores the task on both primary and backup members, and starts the execution also.
With the first invocation, a Ringbuffer stores the task and a generated sequence for the task is returned to the caller as a result. In addition to the storing, the task is executed on the local execution service for the primary member. By this way, the task is now resilient to member failures and you are able to track the task with its ID.
After the first invocation has completed and the sequence of task is returned, second invocation starts to retrieve the result of task with that sequence. This retrieval waits in the waiting operations queue until notified, or it runs immediately if the result is already available.
When task execution is completed, Ringbuffer replaces the task with the result for the given task sequence. This replacement notifies the waiting operations queue.
This section presents example configurations for durable executor service along with the descriptions of its configuration elements and attributes.
Declarative:
<durable-executor-service name="myDurableExecSvc">
<pool-size>8</pool-size>
<durability>1</durability>
<capacity>1</capacity>
</durable-executor-service>
Programmatic:
HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
DurableExecutorService durableExecSvc = hazelcast.getDurableExecutorService("myDurableExecSvc");
Config config = new Config();
config.getDurableExecutorConfig( "myDurableExecSvc" ).
.setPoolSize ( "8" )
.setDurability( "1" )
.setCapacity( "1" );
Following are the descriptions of each configuration element and attribute:
name
: Name of the executor task.pool-size
: Number of executor threads per member for the executor.durability
: Durability of the executor.capacity
: Capacity of the executor task. 0 means Integer.MAX_VALUE.Hazelcast supports entry processing. An entry processor is a function that executes your code on a map entry in an atomic way.
An entry processor is a good option if you perform bulk processing on an IMap
. Usually you perform a loop of keys-- executing IMap.get(key)
, mutating the value, and finally putting the entry back in the map using IMap.put(key,value)
. If you perform this process from a client or from a member where the keys do not exist, you effectively perform two network hops for each update: the first to retrieve the data and the second to update the mutated value.
If you are doing the process described above, you should consider using entry processors. An entry processor executes a read and updates upon the member where the data resides. This eliminates the costly network hops described above.
NOTE: Entry processor is meant to process a single entry per call. Processing multiple entries and data structures in an entry processor is not supported as it may result in deadlocks.
An entry processor enables fast in-memory operations on your map without you having to worry about locks or concurrency issues. You can apply it to a single map entry or to all map entries. Entry processors support choosing target entries using predicates. You do not need any explicit lock on entry thanks to the isolated threading model: Hazelcast runs the EntryProcessor for all entries on a partitionThread
so there will NOT be any interleaving of the EntryProcessor and other mutations.
Hazelcast sends the entry processor to each cluster member and these members apply it to map entries. Therefore, if you add more members, your processing completes faster.
Entry processors can be used with predicates. Predicates help to process a subset of data by selecting eligible entries. This selection can happen either by doing a full-table scan or by using indexes. To accelerate entry selection step, you can consider to add indexes. If indexes are there, entry processor will automatically use them.
If entry processing is the major operation for a map and if the map consists of complex objects, you should use OBJECT
as the in-memory-format
to minimize serialization cost. By default, the entry value is stored as a byte array (BINARY
format). When it is stored as an object (OBJECT
format), then the entry processor is applied directly on the object. In that case, no serialization or deserialization is performed. However, if there is a defined event listener, a new entry value will be serialized when passing to the event publisher service.
NOTE: When in-memory-format
is OBJECT
, the old value of the updated entry will be null.
The methods below are in the IMap interface for entry processing.
executeOnKey
processes an entry mapped by a key.executeOnKeys
processes entries mapped by a collection of keys.submitToKey
processes an entry mapped by a key while listening to event status.executeOnEntries
processes all entries in a map.executeOnEntries
can also process all entries in a map with a defined predicate./**
* Applies the user defined EntryProcessor to the entry mapped by the key.
* Returns the object which is the result of the process() method of EntryProcessor.
*/
Object executeOnKey( K key, EntryProcessor entryProcessor );
/**
* Applies the user defined EntryProcessor to the entries mapped by the collection of keys.
* Returns the results mapped by each key in the collection.
*/
Map<K, Object> executeOnKeys( Set<K> keys, EntryProcessor entryProcessor );
/**
* Applies the user defined EntryProcessor to the entry mapped by the key with
* specified ExecutionCallback to listen to event status and return immediately.
*/
void submitToKey( K key, EntryProcessor entryProcessor, ExecutionCallback callback );
/**
* Applies the user defined EntryProcessor to all entries in the map.
* Returns the results mapped by each key in the map.
*/
Map<K, Object> executeOnEntries( EntryProcessor entryProcessor );
/**
* Applies the user defined EntryProcessor to the entries in the map which satisfies
provided predicate.
* Returns the results mapped by each key in the map.
*/
Map<K, Object> executeOnEntries( EntryProcessor entryProcessor, Predicate predicate );
NOTE: Entry Processors run via Operation Threads that are dedicated to specific partitions. Therefore, with long running Entry Processor executions, other partition operations such as map.put(key)
cannot be processed. With this in mind, it is good practice to make your Entry Processor executions as quick as possible.
EntryProcessor
InterfaceThe following is the EntryProcessor
interface:
public interface EntryProcessor<K, V> extends Serializable {
Object process( Map.Entry<K, V> entry );
EntryBackupProcessor<K, V> getBackupProcessor();
}
NOTE: If you want to execute a task on a single key, you can also use executeOnKeyOwner
provided by Executor Service. However, in this case you need to perform a lock and serialization.
When using the executeOnEntries
method, if the number of entries is high and you need the results, then returning null with the process()
method is a good practice. By returning null, results of the processing is not stored in the map and thus out of memory errors are eliminated.
If your code modifies the data, then you should also provide a processor for backup entries. This is required to prevent the primary map entries from having different values than the backups because it causes the entry processor to be applied both on the primary and backup entries.
public interface EntryBackupProcessor<K, V> extends Serializable {
void processBackup( Map.Entry<K, V> entry );
}
NOTE: It is possible that an Entry Processor could see that a key exists though its backup processor may not find it at the run time due to an unsent backup of a previous operation (e.g., a previous put operation). In those situations, Hazelcast internally/eventually will synchronize those owner and backup partitions so you will not lose any data. When coding an EntryBackupProcessor
, you should take that case into account, otherwise NullPointerException
can be seen since Map.Entry.getValue()
may return null
.
The EntryProcessorTest class has the following methods.
testMapEntryProcessor
puts one map entry and calls executeOnKey
to process that map entry.testMapEntryProcessor
puts all the entries in a map and calls executeOnEntries
to process
all the entries.The static class IncrementingEntryProcessor
creates an entry processor to process the map
entries in the EntryProcessorTest class. It creates the entry processor class by:
EntryProcessor
and EntryBackupProcessor
.java.io.Serializable
interface.EntryProcessor
methods process
and getBackupProcessor
.EntryBackupProcessor
method processBackup
.public class EntryProcessorTest {
@Test
public void testMapEntryProcessor() throws InterruptedException {
Config config = new Config().getMapConfig( "default" )
.setInMemoryFormat( MapConfig.InMemoryFormat.OBJECT );
HazelcastInstance hazelcastInstance1 = Hazelcast.newHazelcastInstance( config );
HazelcastInstance hazelcastInstance2 = Hazelcast.newHazelcastInstance( config );
IMap<Integer, Integer> map = hazelcastInstance1.getMap( "mapEntryProcessor" );
map.put( 1, 1 );
EntryProcessor entryProcessor = new IncrementingEntryProcessor();
map.executeOnKey( 1, entryProcessor );
assertEquals( map.get( 1 ), (Object) 2 );
hazelcastInstance1.getLifecycleService().shutdown();
hazelcastInstance2.getLifecycleService().shutdown();
}
@Test
public void testMapEntryProcessorAllKeys() throws InterruptedException {
StaticNodeFactory factory = new StaticNodeFactory( 2 );
Config config = new Config().getMapConfig( "default" )
.setInMemoryFormat( MapConfig.InMemoryFormat.OBJECT );
HazelcastInstance hazelcastInstance1 = factory.newHazelcastInstance( config );
HazelcastInstance hazelcastInstance2 = factory.newHazelcastInstance( config );
IMap<Integer, Integer> map = hazelcastInstance1
.getMap( "mapEntryProcessorAllKeys" );
int size = 100;
for ( int i = 0; i < size; i++ ) {
map.put( i, i );
}
EntryProcessor entryProcessor = new IncrementingEntryProcessor();
Map<Integer, Object> res = map.executeOnEntries( entryProcessor );
for ( int i = 0; i < size; i++ ) {
assertEquals( map.get( i ), (Object) (i + 1) );
}
for ( int i = 0; i < size; i++ ) {
assertEquals( map.get( i ) + 1, res.get( i ) );
}
hazelcastInstance1.getLifecycleService().shutdown();
hazelcastInstance2.getLifecycleService().shutdown();
}
static class IncrementingEntryProcessor
implements EntryProcessor, EntryBackupProcessor, Serializable {
public Object process( Map.Entry entry ) {
Integer value = (Integer) entry.getValue();
entry.setValue( value + 1 );
return value + 1;
}
public EntryBackupProcessor getBackupProcessor() {
return IncrementingEntryProcessor.this;
}
public void processBackup( Map.Entry entry ) {
entry.setValue( (Integer) entry.getValue() + 1 );
}
}
}
NOTE: You should explicitly call the setValue
method of Map.Entry
when modifying data in Entry Processor. Otherwise, Entry Processor will be accepted as read-only.
NOTE: An Entry Processor instance is not thread safe. If you are storing a partition specific state between invocations, be sure to register this in a thread-local. An Entry Processor instance can be used by multiple partition threads.
You can use the AbstractEntryProcessor
class when the same processing will be performed both on the primary and backup map entries (i.e. the same logic applies to them). If you use Entry Processor, you need to apply the same logic to the backup entries separately. The AbstractEntryProcessor
class makes this primary/backup processing easier.
The code below shows the Hazelcast AbstractEntryProcessor
class. You can use it to create your own Abstract Entry Processor.
public abstract class AbstractEntryProcessor <K, V>
implements EntryProcessor <K, V> {
private final EntryBackupProcessor <K,V> entryBackupProcessor;
public AbstractEntryProcessor() {
this(true);
}
public AbstractEntryProcessor(boolean applyOnBackup) {
if ( applyOnBackup ) {
entryBackupProcessor = new EntryBackupProcessorImpl();
} else {
entryBackupProcessor = null;
}
}
@Override
public abstract Object process(Map.Entry<K, V> entry);
@Override
public final EntryBackupProcessor <K, V> getBackupProcessor() {
return entryBackupProcessor;
}
private class EntryBackupProcessorImpl implements EntryBackupProcessor <K,V>{
@Override
public void processBackup(Map.Entry<K, V> entry) {
process(entry);
}
}
}
In the above code, the method getBackupProcessor
returns an EntryBackupProcessor
instance. This means the same processing will be applied to both the primary and backup entries. If you want to apply the processing only upon the primary entries, make the getBackupProcessor
method return null.
NOTE: Beware of the null issue described in the note in the Processing Backup Entries section. Due to a yet unsent backup from a previous operation, an EntryBackupProcessor
may temporarily receive null
from Map.Entry.getValue()
even though the value actually exists in the map. If you decide to use AbstractEntryProcessor
, make sure your code logic is not sensitive to null values, or you may encounter NullPointerException
during runtime.
Distributed queries access data from multiple data sources stored on either the same or different members.
Hazelcast partitions your data and spreads it across cluster of members. You can iterate over the map entries and look for certain entries (specified by predicates) you are interested in. However, this is not very efficient because you will have to bring the entire entry set and iterate locally. Instead, Hazelcast allows you to run distributed queries on your distributed map.
Distributed query is highly scalable. If you add new members to the cluster, the partition count for each member is reduced and thus the time spent by each member on iterating its entries is reduced. In addition, the pool of partition threads evaluates the entries concurrently in each member, and the network traffic is also reduced since only filtered data is sent to the requester.
Hazelcast offers the following APIs for distributed query purposes:
Assume that you have an "employee" map containing values of Employee
objects, as coded below.
import java.io.Serializable;
public class Employee implements Serializable {
private String name;
private int age;
private boolean active;
private double salary;
public Employee(String name, int age, boolean live, double price) {
this.name = name;
this.age = age;
this.active = live;
this.salary = price;
}
public Employee() {
}
public String getName() {
return name;
}
public int getAge() {
return age;
}
public double getSalary() {
return salary;
}
public boolean isActive() {
return active;
}
}
Now let's look for the employees who are active and have an age less than 30 using the aforementioned APIs (Criteria API and Distributed SQL Query). The following subsections describe each query mechanism for this example.
NOTE: When using Portable objects, if one field of an object exists on one member but does not exist on another one, Hazelcast does not throw an unknown field exception. Instead, Hazelcast treats that predicate, which tries to perform a query on an unknown field, as an always false predicate.
Criteria API is a programming interface offered by Hazelcast that is similar to the Java Persistence Query Language (JPQL). Below is the code for the above example query.
import com.hazelcast.core.IMap;
import com.hazelcast.query.Predicate;
import com.hazelcast.query.PredicateBuilder;
import com.hazelcast.query.EntryObject;
import com.hazelcast.config.Config;
IMap<String, Employee> map = hazelcastInstance.getMap( "employee" );
EntryObject e = new PredicateBuilder().getEntryObject();
Predicate predicate = e.is( "active" ).and( e.get( "age" ).lessThan( 30 ) );
Set<Employee> employees = map.values( predicate );
In the above example code, predicate
verifies whether the entry is active and its age
value is less than 30. This predicate
is
applied to the employee
map using the map.values(predicate)
method. This method sends the predicate to all cluster members
and merges the results coming from them. Since the predicate is communicated between the members, it needs to
be serializable.
NOTE: Predicates can also be applied to keySet
, entrySet
and localKeySet
of the Hazelcast distributed
map.
The Predicates
class offered by Hazelcast includes many operators for your query requirements. Some of them are
explained below.
equal
: Checks if the result of an expression is equal to a given value.notEqual
: Checks if the result of an expression is not equal to a given value.instanceOf
: Checks if the result of an expression has a certain type.like
: Checks if the result of an expression matches some string pattern. % (percentage sign) is the placeholder for many
characters, (underscore) is placeholder for only one character.greaterThan
: Checks if the result of an expression is greater than a certain value.greaterEqual
: Checks if the result of an expression is greater than or equal to a certain value.lessThan
: Checks if the result of an expression is less than a certain value.lessEqual
: Checks if the result of an expression is less than or equal to a certain value.between
: Checks if the result of an expression is between two values (this is inclusive).in
: Checks if the result of an expression is an element of a certain collection.isNot
: Checks if the result of an expression is false.regex
: Checks if the result of an expression matches some regular expression.
RELATED INFORMATION
Please see the Predicates class for all predicates provided.
You can combine predicates using the and
, or
, and not
operators, as shown in the below examples.
public Set<Person> getWithNameAndAge( String name, int age ) {
Predicate namePredicate = Predicates.equal( "name", name );
Predicate agePredicate = Predicates.equal( "age", age );
Predicate predicate = Predicates.and( namePredicate, agePredicate );
return personMap.values( predicate );
}
public Set<Person> getWithNameOrAge( String name, int age ) {
Predicate namePredicate = Predicates.equal( "name", name );
Predicate agePredicate = Predicates.equal( "age", age );
Predicate predicate = Predicates.or( namePredicate, agePredicate );
return personMap.values( predicate );
}
public Set<Person> getNotWithName( String name ) {
Predicate namePredicate = Predicates.equal( "name", name );
Predicate predicate = Predicates.not( namePredicate );
return personMap.values( predicate );
}
You can simplify predicate usage with the PredicateBuilder
class, which offers simpler predicate building. Please see the
below example code which selects all people with a certain name and age.
public Set<Person> getWithNameAndAgeSimplified( String name, int age ) {
EntryObject e = new PredicateBuilder().getEntryObject();
Predicate agePredicate = e.get( "age" ).equal( age );
Predicate predicate = e.get( "name" ).equal( name ).and( agePredicate );
return personMap.values( predicate );
}
com.hazelcast.query.SqlPredicate
takes the regular SQL where
clause. Here is an example:
IMap<Employee> map = hazelcastInstance.getMap( "employee" );
Set<Employee> employees = map.values( new SqlPredicate( "active AND age < 30" ) );
AND/OR: <expression> AND <expression> AND <expression>...
active AND age>30
active=false OR age = 45 OR name = ‘Joe‘
active AND ( age > 20 OR salary < 60000 )
Equality: =, !=, <, <=, >, >=
<expression> = value
age <= 30
name = ‘Joe‘
salary != 50000
BETWEEN: <attribute> [NOT] BETWEEN <value1> AND <value2>
age BETWEEN 20 AND 33 ( same as age >= 20 AND age <= 33 )
age NOT BETWEEN 30 AND 40 ( same as age < 30 OR age > 40 )
IN: <attribute> [NOT] IN (val1, val2,...)
age IN ( 20, 30, 40 )
age NOT IN ( 60, 70 )
active AND ( salary >= 50000 OR ( age NOT BETWEEN 20 AND 30 ) )
age IN ( 20, 30, 40 ) AND salary BETWEEN ( 50000, 80000 )
LIKE: <attribute> [NOT] LIKE "expression"
The %
(percentage sign) is placeholder for multiple characters, an _
(underscore) is placeholder for only one character.
name LIKE ‘Jo%‘
(true for 'Joe', 'Josh', 'Joseph' etc.)name LIKE ‘Jo_‘
(true for 'Joe'; false for 'Josh')name NOT LIKE ‘Jo_‘
(true for 'Josh'; false for 'Joe')name LIKE ‘J_s%‘
(true for 'Josh', 'Joseph'; false 'John', 'Joe')
ILIKE: <attribute> [NOT] ILIKE ‘expression’
Similar to LIKE predicate but in a case-insensitive manner.
name ILIKE ‘Jo%‘
(true for 'Joe', 'joe', 'jOe','Josh','joSH', etc.)name ILIKE ‘Jo_‘
(true for 'Joe' or 'jOE'; false for 'Josh')
REGEX: <attribute> [NOT] REGEX ‘expression’
name REGEX ‘abc-.*‘
(true for 'abc-123'; false for 'abx-123')You can use __key
attribute to perform a predicated search for entry keys. Please see the following example:
IMap<String, Person> personMap = hazelcastInstance.getMap(persons);
personMap.put("Alice", new Person("Alice", 35, Gender.FEMALE));
personMap.put("Andy", new Person("Andy", 37, Gender.MALE));
personMap.put("Bob", new Person("Bob", 22, Gender.MALE));
[...]
Predicate predicate = new SqlPredicate("__key like A%");
Collection<Person> startingWithA = personMap.values(predicate);
In this example, the code creates a collection with the entries whose keys start with the letter "A".
Hazelcast provides paging for defined predicates. With its PagingPredicate
class, you can
get a collection of keys, values, or entries page by page by filtering them with predicates and giving the size of the pages. Also, you
can sort the entries by specifying comparators.
In the example code below:
greaterEqual
predicate gets values from the "students" map. This predicate has a filter
to retrieve the objects with an "age" greater than or equal to 18. PagingPredicate
is constructed in which the page size is 5, so there will be five objects in each page.
The first time the values are called creates the first page. nextPage()
method of PagingPredicate
and querying the map again with the updated PagingPredicate
.IMap<Integer, Student> map = hazelcastInstance.getMap( "students" );
Predicate greaterEqual = Predicates.greaterEqual( "age", 18 );
PagingPredicate pagingPredicate = new PagingPredicate( greaterEqual, 5 );
// Retrieve the first page
Collection<Student> values = map.values( pagingPredicate );
...
// Set up next page
pagingPredicate.nextPage();
// Retrieve next page
values = map.values( pagingPredicate );
...
If a comparator is not specified for PagingPredicate
, but you want to get a collection of keys or values page by page, this collection must be an instance of Comparable
(i.e., it must implement java.lang.Comparable
). Otherwise, the java.lang.IllegalArgument
exception is thrown.
Starting with Hazelcast 3.6, you can also access a specific page more easily with the help of the method setPage()
. This way, if you make a query for the hundredth page, for example, it will get all 100 pages at once instead of reaching the hundredth page one by one using the method nextPage()
. Please note that this feature tires the memory and refer to the PagingPredicate class.
Paging Predicate, also known as Order & Limit, is not supported in Transactional Context.
RELATED INFORMATION
Please see the Predicates class for all predicates provided.
Hazelcast distributed queries will run on each member in parallel and will return only the results to the caller. Then, on the caller side, the results will be merged.
When a query runs on a
member, Hazelcast will iterate through all the owned entries and find the matching ones. This can be made faster by indexing
the mostly queried fields, just like you would do for your database. Indexing will add overhead for each write
operation but queries will be a lot faster. If you query your map a lot, make sure to add indexes for the most frequently
queried fields. For example, if you do an active and age < 30
query, make sure you add an index for the active
and
age
fields. The following example code does that by:
addIndex
method.IMap map = hazelcastInstance.getMap( "employees" );
// ordered, since we have ranged queries for this field
map.addIndex( "age", true );
// not ordered, because boolean field cannot have range
map.addIndex( "active", false );
IMap.addIndex(fieldName, ordered)
is used for adding index. For each indexed field, if you have ranged queries such as age>30
,
age BETWEEN 40 AND 60
, then you should set the ordered
parameter to true
. Otherwise, set it to false
.
Also, you can define IMap
indexes in configuration. An example is shown below.
<map name="default">
...
<indexes>
<index ordered="false">name</index>
<index ordered="true">age</index>
</indexes>
</map>
You can also define IMap
indexes using programmatic configuration, as in the example below.
mapConfig.addMapIndexConfig( new MapIndexConfig( "name", false ) );
mapConfig.addMapIndexConfig( new MapIndexConfig( "age", true ) );
The following is the Spring declarative configuration for the same sample.
<hz:map name="default">
<hz:indexes>
<hz:index attribute="name"/>
<hz:index attribute="age" ordered="true"/>
</hz:indexes>
</hz:map>
NOTE: Non-primitive types to be indexed should implement Comparable
.
You can change the size of thread pool dedicated to query operations using the pool-size
property. Each query consumes a single thread from a Generic Operations ThreadPool on each Hazelcast member - let's call it the query-orchestrating thread. That thread is blocked throughout the whole execution-span of a query on the member.
The query-orchestrating thread will use the threads from the query-thread pool in two cases:
PagingPredicate
- since each page is run as a separate task,hazelcast.query.predicate.parallel.evaluation
to true - since the predicates are evaluated in parallel.Please see Filtering with Paging Predicates and System Properties sections for information on paging predicates and for description of the above system property.
Below is an example of that declarative configuration.
<executor-service name="hz:query">
<pool-size>100</pool-size>
</executor-service>
Below is the equivalent programmatic configuration.
Config cfg = new Config();
cfg.getExecutorConfig("hz:query").setPoolSize(100);
Hazelcast allows querying in collections and arrays. Querying in collections and arrays is compatible with all Hazelcast serialization methods, including the Portable serialization.
Let's have a look at the following data structure expressed in pseudo-code:
class Motorbike {
Wheel wheels[2];
}
class Wheel {
String name;
}
In order to query a single element of a collection/array, you can execute the following query:
// it matches all motorbikes where the zero wheel's name is 'front-wheel'
Predicate p = Predicates.equals('wheels[0].name', 'front-wheel');
Collection<Motorbike> result = map.values(p);
It is also possible to query a collection/array using the any
semantic as shown below:
// it matches all motorbikes where any wheel's name is 'front-wheel'
Predicate p = Predicates.equals('wheels[any].name', 'front');
Collection<Motorbike> result = map.values(p);
The exact same query may be executed using the SQLPredicate
as shown below:
Predicate p = new SQLPredicate('wheels[any].name', 'front');
Collection<Motorbike> result = map.values(p);
[]
notation applies to both collections and arrays.
You can also create an index using a query in collections and arrays.
Please note that in order to leverage the index, the attribute name used in the query has to be the same as the one used in the index definition.
Let's assume you have the following index definition:
<indexes>
<index ordered="false">wheels[any].name</index>
</indexes>
The following query will use the index:
Predicate p = Predicates.equals('wheels[any].name', 'front-wheel');
The following query, however, will NOT leverage the index, since it does not use exactly the same attribute name that was used in the index:
Predicates.equals('wheels[0].name', 'front-wheel')
In order to use the index in the case mentioned above, you have to create another index, as shown below:
<indexes>
<index ordered="false">wheels[0].name</index>
</indexes>
Handling of corner cases may be a bit different than in a programming language like Java
.
Let's have a look at the following examples in order to understand the differences.
To make the analysis simpler, let's assume that there is only one Motorbike
object stored in a Hazelcast Map.
Id | Query | Data state | Extraction Result | Match |
---|---|---|---|---|
1 | Predicates.equals('wheels[7].name', 'front-wheel') | wheels.size() == 1 | null | No |
2 | Predicates.equals('wheels[7].name', null) | wheels.size() == 1 | null | Yes |
3 | Predicates.equals('wheels[0].name', 'front-wheel') | wheels[0].name == null | null | No |
4 | Predicates.equals('wheels[0].name', null) | wheels[0].name == null | null | Yes |
5 | Predicates.equals('wheels[0].name', 'front-wheel') | wheels[0] == null | null | No |
6 | Predicates.equals('wheels[0].name', null) | wheels[0] == null | null | Yes |
7 | Predicates.equals('wheels[0].name', 'front-wheel') | wheels == null | null | No |
8 | Predicates.equals('wheels[0].name', null) | wheels == null | null | Yes |
As you can see, no NullPointerException
s or IndexOutOfBoundException
s are thrown in the extraction process, even
though parts of the expression are null
.
Looking at examples 4, 6 and 8, we can also easily notice that it is impossible to distinguish which part of the
expression was null.
If we execute the following query wheels[1].name = null
, it may be evaluated to true because:
wheels
collection/array is null.index == 1
is out of bound.name
attribute of the wheels[1] object is null
.In order to make the query unambiguous, extra conditions would have to be added, e.g.,
wheels != null AND wheels[1].name = null
.
It is possible to define a custom attribute that may be referenced in predicates, queries and indexes.
A custom attribute is a "synthetic" attribute that does not exist as a field
or a getter
in the object that it is extracted from.
Thus, it is necessary to define the policy on how the attribute is supposed to be extracted.
Currently the only way to extract a custom attribute is to implement a com.hazelcast.query.extractor.ValueExtractor
that encompasses the extraction logic.
Custom Attributes are compatible with all Hazelcast serialization methods, including the Portable serialization.
In order to implement a ValueExtractor
, extend the abstract com.hazelcast.query.extractor.ValueExtractor
class
and implement the extract()
method.
The ValueExtractor
interface looks as follows:
/***
* Common superclass for all extractors.
*
* @param <T> type of the target object to extract the value from
* @param <A> type of the extraction argument object passed to the extract() method
*
*/
public abstract class ValueExtractor<T, A> {
/**
* Extracts custom attribute's value from the given target object.
*
* @param target object to extract the value from
* @param argument extraction argument
* @param collector collector of the extracted value(s)
*
*/
public abstract void extract(T target, A argument, ValueCollector collector);
}
The extract()
method does not return any value since the extracted value is collected by the ValueCollector
.
In order to return multiple results from a single extraction, invoke the ValueCollector.collect()
method
multiple times, so that the collector collects all results.
Here is the ValueCollector
contract:
/**
* Enables collecting values extracted by a {@see com.hazelcast.query.extractor.ValueExtractor}
*/
public abstract class ValueCollector {
/**
* Collects a value extracted by a ValueExtractor.
* <p/>
* More than one value may be collected in a single extraction
*
* @param value value to be collected
*/
public abstract void addObject(Object value);
}
Portable serialization is a special kind of serialization where there is no need to have the class of the serialized object on the
classpath in order to read its attributes. That is the reason why the target object passed to the ValueExtractor.extract()
method will not be of the exact type that has been stored. Instead, an instance of a com.hazelcast.query.extractor.ValueReader
will be passed.
ValueReader
enables reading the attributes of a Portable object in a generic and type-agnostic way.
It contains two methods:
read(String path, ValueCollector<T> collector)
- enables passing all results directly to the ValueCollector
.read(String path, ValueCallback<T> callback)
- enables filtering, transforming and grouping the result of the read operation and manually passing it to the ValueCollector
.Here is the ValueReader
contract:
/**
* Enables reading the value of the attribute specified by the path
* <p>
* The path may be:
* - simple -> it includes a single attribute only, like "name"
* - nested -> it includes more then a single attribute separated with a dot (.), e.g. person.address.city
* <p>
* The path may also includes array cells:
* - specific quantifier, like person.leg[1] -> returns the leg with index 1
* - wildcard quantifier, like person.leg[any] -> returns all legs
* <p>
* The wildcard quantifier may be used a couple of times, like person.leg[any].finger[any] which returns all fingers
* from all legs.
*/
public abstract class ValueReader {
/**
* Read the value of the attribute specified by the path and returns the result via the callback.
*
*/
public abstract <T> void read(String path, ValueCallback<T> callback) throws ValueReadingException;
/**
* Read the value of the attribute specified by the path and returns the result directly to the collector.
*
*/
public abstract <T> void read(String path, ValueCollector<T> collector) throws ValueReadingException;
}
It sounds counter-intuitive, but a single extraction may return multiple values when arrays or collections are involved. Let's have a look at the following data structure in pseudo-code:
class Motorbike {
Wheel wheel[2];
}
class Wheel {
String name;
}
Let's assume that we want to extract the names of all wheels from a single motorbike object. Each motorbike has two
wheels so there are two names for each bike. In order to return both values from the extraction operation, collect them
separately using the ValueCollector
. Collecting multiple values in this way allows you to operate on these multiple
values as if they were single values during the evaluation of the predicates.
Let's assume that we registered a custom extractor with the name wheelName
and executed the following query:
wheelName = front-wheel
.
The extraction may return up to two wheel names for each Motorbike
since each Motorbike
has up to two wheels.
In such a case, it is enough if a single value evaluates the predicate's condition to true to return a match, so
it will return a Motorbike
if "any" of the wheels matches the expression.
A ValueExtractor
may use a custom argument if it is specified in the query.
The custom argument may be passed within the square brackets located after the name of the custom attribute,
e.g., customAttribute[argument]
.
Let's have a look at the following query: currency[incoming] == EUR
The currency
is a custom attribute that uses a com.test.CurrencyExtractor
for extraction.
The string incoming
is an argument that will be passed to the ArgumentParser
during the extraction.
The parser will parse the string according to the parser's custom logic and it will return a parsed object.
The parsed object may be a single object, array, collection, or any arbitrary object.
It is up to the ValueExtractor
's implementor to understand the semantics of the parsed argument object.
For now it is not possible to register a custom ArgumentParser
, thus a default parser is used.
It follows a pass-through
semantic, which means that the string located in the square brackets is passed "as is" to
the ValueExtractor.extract()
method.
Please note that using square brackets within the argument string is not allowed.
The following snippet demonstrates how to define a custom attribute using a ValueExtractor
.
MapAttributeConfig attributeConfig = new MapAttributeConfig();
attributeConfig.setName("currency");
attributeConfig.setExtractor("com.bank.CurrencyExtractor");
MapConfig mapConfig = new MapConfig();
mapConfig.addMapAttributeConfig(attributeConfig);
currency
is the name of the custom attribute that will be extracted using the CurrencyExtractor
class.
Keep in mind that an extractor may not be added after the map has been instantiated. All extractors have to be defined upfront in the map's initial configuration.
The following snippet demonstrates how to define a custom attribute in the Hazelcast XML Configuration.
<map name="trades">
<attributes>
<attribute extractor="com.bank.CurrencyExtractor">currency</attribute>
</attributes>
</map>
Analogous to the example above, currency
is the name of the custom attribute that will be extracted using the
CurrencyExtractor
class.
Please note that an attribute name may begin with an ASCII letter [A-Za-z] or digit [0-9] and may contain ASCII letters [A-Za-z], digits [0-9] or underscores later on.
You can create an index using a custom attribute.
The name of the attribute used in the index definition has to match the one used in the attributes configuration.
Defining indexes with extraction arguments is allowed, as shown in the example below:
<indexes>
<!-- custom attribute without an extraction argument -->
<index ordered="true">currency</index>
<!-- custom attribute using an extraction argument -->
<index ordered="true">currency[EUR]</index>
</indexes>
You have likely heard about MapReduce ever since Google released its research white paper on this concept. With Hadoop as the most common and well known implementation, MapReduce gained a broad audience and made it into all kinds of business applications dominated by data warehouses.
MapReduce is a software framework for processing large amounts of data in a distributed way. Therefore, the processing is normally spread over several machines. The basic idea behind MapReduce is that source data is mapped into a collection of key-value pairs and reducing those pairs, grouped by key, in a second step towards the final result.
The main idea can be summarized with the following steps.
Use Cases
The best known examples for MapReduce algorithms are text processing tools, such as counting the word frequency in large texts or websites. Apart from that, there are more interesting examples of use cases listed below.
This section will give deeper insight into the MapReduce pattern and will help you understand the semantics behind the different MapReduce phases and how they are implemented in Hazelcast.
In addition to this, the following sections compare Hadoop and Hazelcast MapReduce implementations to help adopters with Hadoop backgrounds quickly get familiar with Hazelcast MapReduce.
The flowchart below demonstrates the basic workflow of the word count example (distributed occurrences analysis) mentioned in the MapReduce section introduction. From left to right, it iterates over all the entries of a data structure (in this case an IMap). In the mapping phase, it splits the sentence into single words and emits a key-value pair per word: the word is the key, 1 is the value. In the next phase, values are collected (grouped) and transported to their corresponding reducers, where they are eventually reduced to a single key-value pair, the value being the number of occurrences of the word. At the last step, the different reducer results are grouped up to the final result and returned to the requester.
In pseudo code, the corresponding map and reduce function would look like the following. A Hazelcast code example will be shown in the next section.
map( key:String, document:String ):Void ->
for each w:word in document:
emit( w, 1 )
reduce( word:String, counts:List[Int] ):Int ->
return sum( counts )
As seen in the workflow example, a MapReduce process consists of multiple phases. The original MapReduce pattern describes two phases (map, reduce) and one optional phase (combine). In Hazelcast, these phases either only exist virtually to explain the data flow, or are executed in parallel during the real operation while the general idea is still persisting.
(K x V)* -> (L x W)*
[(k1, v1), ..., (kn, vn)] -> [(l1, w1), ..., (lm, wm)]
Mapping Phase
The mapping phase iterates all key-value pairs of any kind of legal input source. The mapper then analyzes the input pairs and emits zero or more new key-value pairs.
K x V -> (L x W)*
(k, v) -> [(l1, w1), ..., (ln, wn)]
Combine Phase
In the combine phase, multiple key-value pairs with the same key are collected and combined to an intermediate result before being sent to the reducers. Combine phase is also optional in Hazelcast, but is highly recommended to lower the traffic.
In terms of the word count example, this can be explained using the sentences "Saturn is a planet but the Earth is a planet, too". As shown above, we would send two key-value pairs (planet, 1). The registered combiner now collects those two pairs and combines them into an intermediate result of (planet, 2). Instead of two key-value pairs sent through the wire, there is now only one for the key "planet".
The pseudo code for a combiner is similar to the reducer.
combine( word:String, counts:List[Int] ):Void ->
emit( word, sum( counts ) )
Grouping / Shuffling Phase
The grouping or shuffling phase only exists virtually in Hazelcast since it is not a real phase; emitted key-value pairs with the same key are always transferred to the same reducer in the same job. They are grouped together, which is equivalent to the shuffling phase.
Reducing Phase
In the reducing phase, the collected intermediate key-value pairs are reduced by their keys to build the final by-key result. This value can be a sum of all the emitted values of the same key, an average value, or something completely different, depending on the use case.
Here is a reduced representation of this phase.
L x W* -> X*
(l, [w1, ..., wn]) -> [x1, ..., xn]
Producing the Final Result
This is not a real MapReduce phase, but it is the final step in Hazelcast after all reducers are notified that reducing has finished. The original job initiator then requests all reduced results and builds the final result.
The Internet is full of useful resources for finding deeper information on MapReduce. Below is a short collection of more introduction material. In addition, there are books written about all kinds of MapReduce patterns and how to write a MapReduce function for your use case. To name them all is out of the scope of this documentation, but here are some resources:
This section explains the basics of the Hazelcast MapReduce framework. While walking through the different API classes, we will build the word count example that was discussed earlier and create it step by step.
The Hazelcast API for MapReduce operations consists of a fluent DSL-like configuration syntax to build and submit jobs. JobTracker
is the basic entry point to all MapReduce operations and is retrieved from com.hazelcast.core.HazelcastInstance
by calling getJobTracker
and supplying the name of the required JobTracker
configuration. The configuration for JobTracker
s will be discussed later; for now we focus on the API itself.
In addition, the complete submission part of the API is built to support a fully reactive way of programming.
To give an easy introduction to people used to Hadoop, we created the class names to be as familiar as possible to their counterparts on Hadoop. That means while most users will recognize a lot of similar sounding classes, the way to configure the jobs is more fluent due to the DSL-like styled API.
While building the example, we will go through as many options as possible, e.g., we will create a specialized JobTracker
configuration (at the end). Special JobTracker
configuration is not required, because for all other Hazelcast features you can use "default" as the configuration name. However, special configurations offer better options to predict behavior of the framework execution.
The full example is available here as a ready to run Maven project.
JobTracker
creates Job instances, whereas every instance of com.hazelcast.mapreduce.Job
defines a single MapReduce configuration. The same Job can be submitted multiple times regardless of whether it is executed in parallel or after the previous execution is finished.
NOTE: After retrieving the JobTracker
, be aware that it should only be used with data structures derived from the same HazelcastInstance. Otherwise, you can get unexpected behavior.
To retrieve a JobTracker
from Hazelcast, we will start by using the "default" configuration for convenience reasons to show the basic way.
import com.hazelcast.mapreduce.*;
JobTracker jobTracker = hazelcastInstance.getJobTracker( "default" );
JobTracker
is retrieved using the same kind of entry point as most other Hazelcast features. After building the cluster connection, you use the created HazelcastInstance to request the configured (or default) JobTracker
from Hazelcast.
The next step will be to create a new Job
and configure it to execute our first MapReduce request against cluster data.
As mentioned in Retrieving a JobTracker Instance, you create a Job using the retrieved JobTracker
instance. A Job defines exactly one configuration of a MapReduce task. Mapper, combiner and reducers will be defined per job. However, since the Job instance is only a configuration, it can be submitted multiple times, regardless of whether executions happen in parallel or one after the other.
A submitted job is always identified using a unique combination of the JobTracker
's name and a jobId generated on submit-time. The way to retrieve the jobId will be shown in one of the later sections.
To create a Job, a second class com.hazelcast.mapreduce.KeyValueSource
is necessary. We will have a deeper look at the KeyValueSource
class in the next section. KeyValueSource
is used to wrap any kind of data or data structure into a well defined set of key-value pairs.
The example code below is a direct follow up to the example in Retrieving a JobTracker Instance, and it reuses the already created HazelcastInstance and JobTracker
instances.
The example starts by retrieving an instance of our data map, and then it creates the Job instance. Implementations used to configure the Job will be discussed while walking further through the API documentation.
NOTE: Since the Job class is highly dependent upon generics to support type safety, the generics change over time and may not be assignment compatible to old variable types. To make use of the full potential of the fluent API, we recommend you use fluent method chaining as shown in this example to prevent the need for too many variables.
IMap<String, String> map = hazelcastInstance.getMap( "articles" );
KeyValueSource<String, String> source = KeyValueSource.fromMap( map );
Job<String, String> job = jobTracker.newJob( source );
ICompletableFuture<Map<String, Long>> future = job
.mapper( new TokenizerMapper() )
.combiner( new WordCountCombinerFactory() )
.reducer( new WordCountReducerFactory() )
.submit();
// Attach a callback listener
future.andThen( buildCallback() );
// Wait and retrieve the result
Map<String, Long> result = future.get();
As seen above, we create the Job instance and define a mapper, combiner, and reducer. Then we submit the request to the cluster. The submit
method returns an ICompletableFuture that can be used to attach our callbacks or to wait for the result to be processed in a blocking fashion.
There are more options available for job configurations, such as defining a general chunk size or on what keys the operation will operate. For more information, please refer to the Hazelcast source code for Job.java.
KeyValueSource
can either wrap Hazelcast data structures (like IMap, MultiMap, IList, ISet) into key-value pair input sources, or build your own custom key-value input source. The latter option makes it possible to feed Hazelcast MapReduce with all kinds of data, such as just-in-time downloaded web page contents or data files. People familiar with Hadoop will recognize similarities with the Input class.
You can imagine a KeyValueSource
as a bigger java.util.Iterator
implementation. Whereas most methods must be implemented, implementing the getAllKeys
method is optional. If implementation is able to gather all keys upfront, it should be implemented and isAllKeysSupported
must return true
. That way, Job configured KeyPredicates are able to evaluate keys upfront before sending them to the cluster. Otherwise they are serialized and transferred as well, to be evaluated at execution time.
As shown in the example above, the abstract KeyValueSource
class provides a number of static methods to easily wrap Hazelcast data structures into KeyValueSource
implementations already provided by Hazelcast. The data structures' generics are inherited by the resulting KeyValueSource
instance. For data structures like IList or ISet, the key type is always String. While mapping, the key is the data structure's name, whereas
the value type and value itself are inherited from the IList or ISet itself.
// KeyValueSource from com.hazelcast.core.IMap
IMap<String, String> map = hazelcastInstance.getMap( "my-map" );
KeyValueSource<String, String> source = KeyValueSource.fromMap( map );
// KeyValueSource from com.hazelcast.core.MultiMap
MultiMap<String, String> multiMap = hazelcastInstance.getMultiMap( "my-multimap" );
KeyValueSource<String, String> source = KeyValueSource.fromMultiMap( multiMap );
// KeyValueSource from com.hazelcast.core.IList
IList<String> list = hazelcastInstance.getList( "my-list" );
KeyValueSource<String, String> source = KeyValueSource.fromList( list );
// KeyValueSource from com.hazelcast.core.ISet
ISet<String> set = hazelcastInstance.getSet( "my-set" );
KeyValueSource<String, String> source = KeyValueSource.fromSet( set );
PartitionIdAware
The com.hazelcast.mapreduce.PartitionIdAware
interface can be implemented by the KeyValueSource
implementation if the underlying data set is aware of the Hazelcast partitioning schema (as it is for all internal data structures). If this interface is implemented, the same KeyValueSource
instance is reused multiple times for all partitions on the cluster member. As a consequence, the close
and open
methods are also executed
multiple times but once per partitionId.
Using the Mapper
interface, you will implement the mapping logic. Mappers can transform, split, calculate, and aggregate data from data sources. In Hazelcast you can also integrate data from more than the KeyValueSource data source by implementing com.hazelcast.core.HazelcastInstanceAware
and requesting additional maps, multimaps, list, and/or sets.
The mappers map
function is called once per available entry in the data structure. If you work on distributed data structures that operate in a partition-based fashion, multiple mappers work in parallel on the different cluster members on the members' assigned partitions. Mappers then prepare and maybe transform the input key-value pair and emit zero or more key-value pairs for the reducing phase.
For our word count example, we retrieve an input document (a text document) and we transform it by splitting the text into the available words. After that, as discussed in the pseudo code, we emit every single word with a key-value pair with the word as the key and 1 as the value.
A common implementation of that Mapper
might look like the following example:
public class TokenizerMapper implements Mapper<String, String, String, Long> {
private static final Long ONE = Long.valueOf( 1L );
@Override
public void map(String key, String document, Context<String, Long> context) {
StringTokenizer tokenizer = new StringTokenizer( document.toLowerCase() );
while ( tokenizer.hasMoreTokens() ) {
context.emit( tokenizer.nextToken(), ONE );
}
}
}
This code splits the mapped texts into their tokens, iterates over the tokenizer as long as there are more tokens, and emits a pair per word. Note that we're not yet collecting multiple occurrences of the same word, we just fire every word on its own.
LifecycleMapper / LifecycleMapperAdapter
The LifecycleMapper interface or its adapter class LifecycleMapperAdapter can be used to make the Mapper implementation lifecycle aware. That means it will be notified when mapping of a partition or set of data begins and when the last entry was mapped.
Only special algorithms might need those additional lifecycle events to prepare, clean up, or emit additional values.
As stated in the introduction, a Combiner is used to minimize traffic between the different cluster members when transmitting mapped values from mappers to the reducers. It does this by aggregating multiple values for the same emitted key. This is a fully optional operation, but using it is highly recommended.
Combiners can be seen as an intermediate reducer. The calculated value is always assigned back to the key for which the combiner initially was created. Since combiners are created per emitted key, the Combiner implementation itself is not defined in the jobs configuration; instead, a CombinerFactory that is able to create the expected Combiner instance is created.
Because Hazelcast MapReduce is executing the mapping and reducing phases in parallel, the Combiner implementation must be able to deal with chunked data. Therefore, you must reset its internal state whenever you call finalizeChunk
. Calling the finalizeChunk
method creates a chunk of intermediate data to be grouped (shuffled) and sent to the reducers.
Combiners can override beginCombine
and finalizeCombine
to perform preparation or cleanup work.
For our word count example, we are going to have a simple CombinerFactory and Combiner implementation similar to the following example.
public class WordCountCombinerFactory
implements CombinerFactory<String, Long, Long> {
@Override
public Combiner<Long, Long> newCombiner( String key ) {
return new WordCountCombiner();
}
private class WordCountCombiner extends Combiner<Long, Long> {
private long sum = 0;
@Override
public void combine( Long value ) {
sum++;
}
@Override
public Long finalizeChunk() {
return sum;
}
@Override
public void reset() {
sum = 0;
}
}
}
The Combiner must be able to return its current value as a chunk and reset the internal state by setting sum
back to 0. Since combiners are always called from a single thread, no synchronization or volatility of the variables is necessary.
Reducers do the last bit of algorithm work. This can be aggregating values, calculating averages, or any other work that is expected from the algorithm.
Since values arrive in chunks, the reduce
method is called multiple times for every emitted value of the creation key. This also can happen multiple times per chunk if no Combiner implementation was configured for a job configuration.
Unlike combiners, a reducer's finalizeReduce
method is only called once per reducer (which means once per key). Therefore, a reducer does not need to reset its internal state at any time.
Reducers can override beginReduce
to perform preparation work.
For our word count example, the implementation will look similar to the following code example.
public class WordCountReducerFactory implements ReducerFactory<String, Long, Long> {
@Override
public Reducer<Long, Long> newReducer( String key ) {
return new WordCountReducer();
}
private class WordCountReducer extends Reducer<Long, Long> {
private volatile long sum = 0;
@Override
public void reduce( Long value ) {
sum += value.longValue();
}
@Override
public Long finalizeReduce() {
return sum;
}
}
}
Unlike combiners, reducers tend to switch threads if running out of data to prevent blocking threads from the JobTracker
configuration. They are rescheduled at a later point when new data to be processed arrives, but are unlikely to be executed on the same thread as before. As of Hazelcast version 3.3.3 the guarantee for memory visibility on the new thread is ensured by the framework. This means the previous requirement for making fields volatile is dropped.
A Collator is an optional operation that is executed on the job emitting member and is able to modify the finally reduced result before returned to the user's codebase. Only special use cases are likely to use collators.
For an imaginary use case, we might want to know how many words were all over in the documents we analyzed. For this case, a Collator implementation can be given to the submit
method of the Job instance.
A collator would look like the following snippet:
public class WordCountCollator implements Collator<Map.Entry<String, Long>, Long> {
@Override
public Long collate( Iterable<Map.Entry<String, Long>> values ) {
long sum = 0;
for ( Map.Entry<String, Long> entry : values ) {
sum += entry.getValue().longValue();
}
return sum;
}
}
The definition of the input type is a bit strange, but because Combiner and Reducer implementations are optional, the input type heavily depends on the state of the data. As stated above, collators are non-typical use cases and the generics of the framework always help in finding the correct signature.
You can use KeyPredicate
to pre-select whether or not a key should be selected for mapping in the mapping phase. If the KeyValueSource
implementation is able to know all keys prior to execution, the keys are filtered before the operations are divided among the different cluster members.
A KeyPredicate
can also be used to select only a special range of data (e.g., a time frame), or in similar use cases.
A basic KeyPredicate
implementation that only maps keys containing the word "hazelcast" might look like the following code example:
public class WordCountKeyPredicate implements KeyPredicate<String> {
@Override
public boolean evaluate( String s ) {
return s != null && s.toLowerCase().contains( "hazelcast" );
}
}
You can retrieve a TrackableJob
instance after submitting a job. It is requested from the JobTracker
using the unique jobId (per JobTracker
). You can use it get runtime statistics of the job. The information available is limited to the number of processed (mapped) records and the processing state of the different partitions or members (if KeyValueSource
is not PartitionIdAware).
To retrieve the jobId after submission of the job, use com.hazelcast.mapreduce.JobCompletableFuture
instead of the com.hazelcast.core.ICompletableFuture
as the variable type for the returned future.
The example code below gives a quick introduction on how to retrieve the instance and the runtime data. For more information, please have a look at the Javadoc corresponding your running Hazelcast version.
The example performs the following steps to get the job instance.
getMap
method.fromMap
method.newJob
method.IMap<String, String> map = hazelcastInstance.getMap( "articles" );
KeyValueSource<String, String> source = KeyValueSource.fromMap( map );
Job<String, String> job = jobTracker.newJob( source );
JobCompletableFuture<Map<String, Long>> future = job
.mapper( new TokenizerMapper() )
.combiner( new WordCountCombinerFactory() )
.reducer( new WordCountReducerFactory() )
.submit();
String jobId = future.getJobId();
TrackableJob trackableJob = jobTracker.getTrackableJob(jobId);
JobProcessInformation stats = trackableJob.getJobProcessInformation();
int processedRecords = stats.getProcessedRecords();
log( "ProcessedRecords: " + processedRecords );
JobPartitionState[] partitionStates = stats.getPartitionStates();
for ( JobPartitionState partitionState : partitionStates ) {
log( "PartitionOwner: " + partitionState.getOwner()
+ ", Processing state: " + partitionState.getState().name() );
}
NOTE: Caching of the JobProcessInformation does not work on Java native clients since current values are retrieved while retrieving the instance to minimize traffic between executing member and client.
You configure JobTracker
configuration to set up behavior of the Hazelcast MapReduce framework.
Every JobTracker
is capable of running multiple MapReduce jobs at once; one configuration is meant as a shared resource for all jobs created by the same JobTracker
. The configuration gives full control over the expected load behavior and thread counts to be used.
The following snippet shows a typical JobTracker
configuration. The configuration properties are discussed below the example.
<jobtracker name="default">
<max-thread-size>0</max-thread-size>
<!-- Queue size 0 means number of partitions * 2 -->
<queue-size>0</queue-size>
<retry-count>0</retry-count>
<chunk-size>1000</chunk-size>
<communicate-stats>true</communicate-stats>
<topology-changed-strategy>CANCEL_RUNNING_OPERATION</topology-changed-strategy>
</jobtracker>
com.hazelcast.mapreduce.TopologyChangedException
).This section explains some of the internals of the MapReduce framework. This is more advanced information. If you're not interested in how it works internally, you might want to skip this section.
To understand the following technical internals, we first have a short look at what happens in terms of an example workflow.
As a simple example, think of an IMap<String, Integer>
and emitted keys having the same types. Imagine you have a cluster with three members, and you initiate the MapReduce job on the first member. After you requested the JobTracker from your running/connected Hazelcast, we submit the task and retrieve the ICompletableFuture, which gives us a chance to wait for the result to be calculated or to add a callback (and being more reactive).
The example expects that the chunk size is 0 or 1, so an emitted value is directly sent to the reducers. Internally, the job is prepared, started, and executed on all members as shown below. The first member acts as the job owner (job emitter).
Member1 starts MapReduce job
Member1 emits key=Foo, value=1
Member1 does PartitionService::getKeyOwner(Foo) => results in Member3
Member2 emits key=Foo, value=14
Member2 asks jobOwner (Member1) for keyOwner of Foo => results in Member3
Member1 sends chunk for key=Foo to Member3
Member3 receives chunk for key=Foo and looks if there is already a Reducer,
if not creates one for key=Foo
Member3 processes chunk for key=Foo
Member2 sends chunk for key=Foo to Member3
Member3 receives chunk for key=Foo and looks if there is already a Reducer and uses
the previous one
Member3 processes chunk for key=Foo
Member1 send LastChunk information to Member3 because processing local values finished
Member2 emits key=Foo, value=27
Member2 has cached keyOwner of Foo => results in Member3
Member2 sends chunk for key=Foo to Member3
Member3 receives chunk for key=Foo and looks if there is already a Reducer and uses
the previous one
Member3 processes chunk for key=Foo
Member2 send LastChunk information to Member3 because processing local values finished
Member3 finishes reducing for key=Foo
Member1 registers its local partitions are processed
Member2 registers its local partitions are processed
Member1 sees all partitions processed and requests reducing from all members
Member1 merges all reduced results together in a final structure and returns it
The flow is quite complex but extremely powerful since everything is executed in parallel. Reducers do not wait until all values are emitted, but they immediately begin to reduce (when the first chunk for an emitted key arrives).
Beginning with the package level, there is one basic package: com.hazelcast.mapreduce
. This includes the external API and the impl package, which itself contains the internal implementation.
KeyValueSource
implementations and abstract base and support classes for the exposed API.Now to the technical walk-through: A MapReduce Job is always retrieved from a named JobTracker
, which is implemented in NodeJobTracker
(extends AbstractJobTracker
) and is configured using the configuration DSL. All of the internal implementation is completely ICompletableFuture-driven and mostly non-blocking in design.
On submit, the Job creates a unique UUID which afterwards acts as a jobId and is combined with the JobTracker's name to be uniquely identifiable inside the cluster. Then, the preparation is sent around the cluster and every member prepares its execution by creating a JobSupervisor, MapCombineTask, and ReducerTask. The job-emitting JobSupervisor gains special capabilities to synchronize and control JobSupervisors on other members for the same job.
If preparation is finished on all members, the job itself is started by executing a StartProcessingJobOperation on every member. This initiates a MappingPhase implementation (defaults to KeyValueSourceMappingPhase) and starts the actual mapping on the members.
The mapping process is currently a single threaded operation per member, but will be extended to run in parallel on multiple partitions (configurable per Job) in future versions. The Mapper is now called on every available value on the partition and eventually emits values. For every emitted value, either a configured CombinerFactory is called to create a Combiner or a cached one is used (or the default CollectingCombinerFactory is used to create Combiners). When the chunk limit is reached on a member, a IntermediateChunkNotification is prepared by collecting emitted keys to their corresponding members. This is either done by asking the job owner to assign members or by an already cached assignment. In later versions, a PartitionStrategy might also be configurable.
The IntermediateChunkNotification is then sent to the reducers (containing only values for this member) and is offered to the ReducerTask. On every offer, the ReducerTask checks if it is already running and if not, it submits itself to the configured ExecutorService (from the JobTracker configuration).
If reducer queue runs out of work, the ReducerTask is removed from the ExecutorService to not block threads but eventually will be resubmitted on next chunk of work.
On every phase, the partition state is changed to keep track of the currently running operations. A JobPartitionState can be in one of the following states with self-explanatory titles: [WAITING, MAPPING, REDUCING, PROCESSED, CANCELLED]
. If you have a deeper interest of these states, look at the Javadoc.
Eventually, all JobPartitionStates reach the state of PROCESSED. Then, the job emitter's JobSupervisor asks all members for their reduced results and executes a potentially offered Collator. With this Collator, the overall result is calculated before it removes itself from the JobTracker, doing some final cleanup and returning the result to the requester (using the internal TrackableJobFuture).
If a job is cancelled while execution, all partitions are immediately set to the CANCELLED state and a CancelJobSupervisorOperation is executed on all members to kill the running processes.
While the operation is running in addition to the default operations, some more operations like ProcessStatsUpdateOperation (updates processed records statistics) or NotifyRemoteExceptionOperation (notifies the members that the sending member encountered an unrecoverable situation and the Job needs to be cancelled - e.g. NullPointerException inside of a Mapper) are executed against the job owner to keep track of the process.
Based on the Hazelcast MapReduce framework, Aggregators are ready-to-use data aggregations. These are typical operations like sum up values, finding minimum or maximum values, calculating averages, and other operations that you would expect in the relational database world.
Aggregation operations are implemented, as mentioned above, on top of the MapReduce framework, and all operations can be achieved using pure MapReduce calls. However, using the Aggregation feature is more convenient for a big set of standard operations.
This section will quickly guide you through the basics of the Aggregations framework and some of its available classes. We also will implement a first base example.
Aggregations are available on both types of map interfaces, com.hazelcast.core.IMap
and com.hazelcast
.core.MultiMap
, using
the aggregate
methods. Two overloaded methods are available that customize resource management of the
underlying MapReduce framework by supplying a custom configured
com.hazelcast.mapreduce.JobTracker
instance. To find out how to
configure the MapReduce framework, please see Configuring JobTracker. We will
later see another way to configure the automatically used MapReduce framework if no special JobTracker
is supplied.
To make Aggregations more convenient to use and future proof, the API is heavily optimized for Java 8 and future versions. The API is still fully compatible with any Java version Hazelcast supports (Java 6 and Java 7). The biggest difference is how you work with the Java generics: on Java 6 and 7, the process to resolve generics is not as strong as on Java 8 and future Java versions. In addition, the whole Aggregations API has full Java 8 Project Lambda (or Closure, JSR 335) support.
For illustration of the differences in Java 6 and 7 in comparison to Java 8, we will have a quick look at code examples for both. After that, we will focus on using Java 8 syntax to keep examples short and easy to understand, and we will see some hints about what the code looks like in Java 6 or 7.
The first example will produce the sum of some int
values stored in a Hazelcast IMap. This example does not use much of the functionality of the Aggregations framework, but it will show the main difference.
IMap<String, Integer> personAgeMapping = hazelcastInstance.getMap( "person-age" );
for ( int i = 0; i < 1000; i++ ) {
String lastName = RandomUtil.randomLastName();
int age = RandomUtil.randomAgeBetween( 20, 80 );
personAgeMapping.put( lastName, Integer.valueOf( age ) );
}
With our demo data prepared, we can see how to produce the sums in different Java versions.
Since Java 6 and 7 are not as strong on resolving generics as Java 8, you need to be a bit more verbose with the code you write. You might also consider using raw types but breaking the type safety to ease this process.
For a short introduction on what the following code example means, look at the source code comments. We will later dig deeper into the different options.
// No filter applied, select all entries
Supplier<String, Integer, Integer> supplier = Supplier.all();
// Choose the sum aggregation
Aggregation<String, Integer, Integer> aggregation = Aggregations.integerSum();
// Execute the aggregation
int sum = personAgeMapping.aggregate( supplier, aggregation );
With Java 8, the Aggregations API looks simpler because Java 8 can resolve the generic parameters for us. That means the above lines of Java 6/7 example code will end up in just one easy line on Java 8.
int sum = personAgeMapping.aggregate( Supplier.all(), Aggregations.integerSum() );
As mentioned before, the Aggregations implementation is based on the Hazelcast MapReduce framework and therefore you might find
overlaps in their APIs. One overload of the aggregate
method can be supplied with
a JobTracker
, which is part of the MapReduce framework.
If you implement your own aggregations, you will use a mixture of the Aggregations and the MapReduce API. If you do so, e.g., to make the life of colleagues easier, please read the Implementing Aggregations section.
For the full MapReduce documentation please see the MapReduce section.
We now look into what can be achieved using the Aggregations API. To work on some deeper examples, let's quickly have a look at the available classes and interfaces and discuss their usage.
The com.hazelcast.mapreduce.aggregation.Supplier
provides filtering and data extraction to the aggregation operation.
This class already provides a few different static methods to achieve the most common cases. Supplier.all()
accepts all incoming values and does not apply any data extraction or transformation upon them before supplying them to
the aggregation function itself.
For filtering data sets, you have two different options by default:
com.hazelcast.query.Predicate
if you want to filter on values and/or keys, orcom.hazelcast.mapreduce.KeyPredicate
if you can decide directly on the data
key without the need to deserialize the value.As mentioned above, all APIs are fully Java 8 and Lambda compatible. Let's have a look on how we can do basic filtering using those two options.
First, we have a look at a KeyPredicate
and we only accept people whose last name is "Jones".
Supplier<...> supplier = Supplier.fromKeyPredicate(
lastName -> "Jones".equalsIgnoreCase( lastName )
);
class JonesKeyPredicate implements KeyPredicate<String> {
public boolean evaluate( String key ) {
return "Jones".equalsIgnoreCase( key );
}
}
Using the standard Hazelcast Predicate
interface, we can also filter based on the value of a data entry. In the following example, you can
only select values that are divisible by 4 without a remainder.
Supplier<...> supplier = Supplier.fromPredicate(
entry -> entry.getValue() % 4 == 0
);
class DivisiblePredicate implements Predicate<String, Integer> {
public boolean apply( Map.Entry<String, Integer> entry ) {
return entry.getValue() % 4 == 0;
}
}
As well as filtering, Supplier
can also extract or transform data before providing it
to the aggregation operation itself. The following example shows how to transform an input value to a string.
Supplier<String, Integer, String> supplier = Supplier.all(
value -> Integer.toString(value)
);
You can see a Java 6/7 example in the Aggregations Examples section.
Apart from the fact we transformed the input value of type int
(or Integer) to a string, we can see that the generic information
of the resulting Supplier
has changed as well. This indicates that we now have an aggregation working on string values.
Another feature of Supplier
is its ability to chain multiple filtering rules. Let's combine all of the
above examples into one rule set:
Supplier<String, Integer, String> supplier =
Supplier.fromKeyPredicate(
lastName -> "Jones".equalsIgnoreCase( lastName ),
Supplier.fromPredicate(
entry -> entry.getValue() % 4 == 0,
Supplier.all( value -> Integer.toString(value) )
)
);
You might prefer or need to implement your Supplier
based on special
requirements. This is a very basic task. The Supplier
abstract class has just one method: the apply
method.
NOTE: Due to a limitation of the Java Lambda API, you cannot implement abstract classes using Lambdas. Instead it is recommended that you create a standard named class.
class MyCustomSupplier extends Supplier<String, Integer, String> {
public String apply( Map.Entry<String, Integer> entry ) {
Integer value = entry.getValue();
if (value == null) {
return null;
}
return value % 4 == 0 ? String.valueOf( value ) : null;
}
}
The Supplier
apply
methods are expected to return null whenever the input value should not be mapped to the aggregation
process. This can be used, as in the example above, to implement filter rules directly. Implementing filters using the
KeyPredicate
and Predicate
interfaces might be more convenient.
To use your own Supplier
, just pass it to the aggregate method or use it in combination with other Supplier
s.
int sum = personAgeMapping.aggregate( new MyCustomSupplier(), Aggregations.count() );
Supplier<String, Integer, String> supplier =
Supplier.fromKeyPredicate(
lastName -> "Jones".equalsIgnoreCase( lastName ),
new MyCustomSupplier()
);
int sum = personAgeMapping.aggregate( supplier, Aggregations.count() );
The com.hazelcast.mapreduce.aggregation.Aggregation
interface defines the aggregation operation itself. It contains a set of
MapReduce API implementations like Mapper
, Combiner
, Reducer
, and Collator
. These implementations are normally unique to
the chosen Aggregation
. This interface can also be implemented with your aggregation operations based on MapReduce calls. For
more information, refer to Implementing Aggregations section.
The com.hazelcast.mapreduce.aggregation.Aggregations
class provides a common predefined set of aggregations. This class
contains type safe aggregations of the following types:
Those aggregations are similar to their counterparts on relational databases and can be equated to SQL statements as set out below.
Calculates an average value based on all selected values.
map.aggregate( Supplier.all( person -> person.getAge() ),
Aggregations.integerAvg() );
SELECT AVG(person.age) FROM person;
Calculates a sum based on all selected values.
map.aggregate( Supplier.all( person -> person.getAge() ),
Aggregations.integerSum() );
SELECT SUM(person.age) FROM person;
Finds the minimal value over all selected values.
map.aggregate( Supplier.all( person -> person.getAge() ),
Aggregations.integerMin() );
SELECT MIN(person.age) FROM person;
Finds the maximal value over all selected values.
map.aggregate( Supplier.all( person -> person.getAge() ),
Aggregations.integerMax() );
SELECT MAX(person.age) FROM person;
Returns a collection of distinct values over the selected values
map.aggregate( Supplier.all( person -> person.getAge() ),
Aggregations.distinctValues() );
SELECT DISTINCT person.age FROM person;
Returns the element count over all selected values
map.aggregate( Supplier.all(), Aggregations.count() );
SELECT COUNT(*) FROM person;
We used the com.hazelcast.mapreduce.aggregation.PropertyExtractor
interface before when we had a look at the example
on how to use a Supplier
to transform a value to another type. It can also be used to extract attributes from values.
class Person {
private String firstName;
private String lastName;
private int age;
// getters and setters
}
PropertyExtractor<Person, Integer> propertyExtractor = (person) -> person.getAge();
class AgeExtractor implements PropertyExtractor<Person, Integer> {
public Integer extract( Person value ) {
return value.getAge();
}
}
In this example, we extract the value from the person's age attribute. The value type changes from Person to Integer
which is reflected in the generics information to stay type safe.
You can use PropertyExtractor
s for any kind of transformation of data. You might even want to have multiple
transformation steps chained one after another.
As stated before, the easiest way to configure the resources used by the underlying MapReduce framework is to supply a JobTracker
to the aggregation call itself by passing it to either IMap::aggregate
or MultiMap::aggregate
.
There is another way to implicitly configure the underlying used JobTracker
. If no specific JobTracker
was
passed for the aggregation call, internally one will be created using the following naming specifications:
For IMap
aggregation calls the naming specification is created as:
hz::aggregation-map-
and the concatenated name of the map.For MultiMap
it is very similar:
hz::aggregation-multimap-
and the concatenated name of the MultiMap.Knowing the specification of the name, we can configure the JobTracker
as expected
(as described in Retrieving a JobTracker Instance) using the naming spec we just learned. For more information on configuration of the
JobTracker
, please see Configuring Jobtracker.
To finish this section, let's have a quick example for the above naming specs:
IMap<String, Integer> map = hazelcastInstance.getMap( "mymap" );
// The internal JobTracker name resolves to 'hz::aggregation-map-mymap'
map.aggregate( ... );
MultiMap<String, Integer> multimap = hazelcastInstance.getMultiMap( "mymultimap" );
// The internal JobTracker name resolves to 'hz::aggregation-multimap-mymultimap'
multimap.aggregate( ... );
For the final example, imagine you are working for an international company and you have an employee database stored in Hazelcast
IMap
with all employees worldwide and a MultiMap
for assigning employees to their certain locations or offices. In addition,
there is another IMap
that holds the salary per employee.
Let's have a look at our data model.
class Employee implements Serializable {
private String firstName;
private String lastName;
private String companyName;
private String address;
private String city;
private String county;
private String state;
private int zip;
private String phone1;
private String phone2;
private String email;
private String web;
// getters and setters
}
class SalaryMonth implements Serializable {
private Month month;
private int salary;
// getters and setters
}
class SalaryYear implements Serializable {
private String email;
private int year;
private List<SalaryMonth> months;
// getters and setters
public int getAnnualSalary() {
int sum = 0;
for ( SalaryMonth salaryMonth : getMonths() ) {
sum += salaryMonth.getSalary();
}
return sum;
}
}
The two IMap
s and the MultiMap
are keyed by the string of email. They are defined as follows:
IMap<String, Employee> employees = hz.getMap( "employees" );
IMap<String, SalaryYear> salaries = hz.getMap( "salaries" );
MultiMap<String, String> officeAssignment = hz.getMultiMap( "office-employee" );
So far, we know all the important information to work out some example aggregations. We will look into some deeper implementation details and how we can work around some current limitations that will be eliminated in future versions of the API.
Let's start with a very basic example. We want to know the average salary of all of our employees. To do this,
we need a PropertyExtractor
and the average aggregation for type Integer
.
IMap<String, SalaryYear> salaries = hazelcastInstance.getMap( "salaries" );
PropertyExtractor<SalaryYear, Integer> extractor =
(salaryYear) -> salaryYear.getAnnualSalary();
int avgSalary = salaries.aggregate( Supplier.all( extractor ),
Aggregations.integerAvg() );
That's it. Internally, we created a MapReduce task based on the predefined aggregation and fired it up immediately. Currently all
aggregation calls are blocking operations, so it is not yet possible to execute the aggregation in a reactive way (using
com.hazelcast.core.ICompletableFuture
), but this will be part of an upcoming version.
The following example is a little more complex. We only want to have our US-based employees selected into the average salary calculation, so we need to execute a join operation between the employees and salaries maps.
class USEmployeeFilter implements KeyPredicate<String>, HazelcastInstanceAware {
private transient HazelcastInstance hazelcastInstance;
public void setHazelcastInstance( HazelcastInstance hazelcastInstance ) {
this.hazelcastInstance = hazelcastInstance;
}
public boolean evaluate( String email ) {
IMap<String, Employee> employees = hazelcastInstance.getMap( "employees" );
Employee employee = employees.get( email );
return "US".equals( employee.getCountry() );
}
}
Using the HazelcastInstanceAware
interface, we get the current instance of Hazelcast injected into our filter and we can perform data
joins on other data structures of the cluster. We now only select employees that work as part of our US offices into the
aggregation.
IMap<String, SalaryYear> salaries = hazelcastInstance.getMap( "salaries" );
PropertyExtractor<SalaryYear, Integer> extractor =
(salaryYear) -> salaryYear.getAnnualSalary();
int avgSalary = salaries.aggregate( Supplier.fromKeyPredicate(
new USEmployeeFilter(), extractor
), Aggregations.integerAvg() );
For our next example, we will do some grouping based on the different worldwide offices. Currently, a group aggregator is not yet available, so we need a small workaround to achieve this goal. (In later versions of the Aggregations API this will not be required because it will be available out of the box in a much more convenient way.)
Again, let's start with our filter. This time, we want to filter based on an office name and we need to do some data joins to achieve this kind of filtering.
A short tip: to minimize the data transmission on the aggregation we can use Data Affinity rules to influence the partitioning of data. Be aware that this is an expert feature of Hazelcast.
class OfficeEmployeeFilter implements KeyPredicate<String>, HazelcastInstanceAware {
private transient HazelcastInstance hazelcastInstance;
private String office;
// Deserialization Constructor
public OfficeEmployeeFilter() {
}
public OfficeEmployeeFilter( String office ) {
this.office = office;
}
public void setHazelcastInstance( HazelcastInstance hazelcastInstance ) {
this.hazelcastInstance = hazelcastInstance;
}
public boolean evaluate( String email ) {
MultiMap<String, String> officeAssignment = hazelcastInstance
.getMultiMap( "office-employee" );
return officeAssignment.containsEntry( office, email );
}
}
Now we can execute our aggregations. As mentioned before, we currently need to do the grouping on our own by executing multiple aggregations in a row.
Map<String, Integer> avgSalariesPerOffice = new HashMap<String, Integer>();
IMap<String, SalaryYear> salaries = hazelcastInstance.getMap( "salaries" );
MultiMap<String, String> officeAssignment =
hazelcastInstance.getMultiMap( "office-employee" );
PropertyExtractor<SalaryYear, Integer> extractor =
(salaryYear) -> salaryYear.getAnnualSalary();
for ( String office : officeAssignment.keySet() ) {
OfficeEmployeeFilter filter = new OfficeEmployeeFilter( office );
int avgSalary = salaries.aggregate( Supplier.fromKeyPredicate( filter, extractor ),
Aggregations.integerAvg() );
avgSalariesPerOffice.put( office, avgSalary );
}
We want to end this section by executing one final and easy aggregation. We want to know how many employees we currently have on a worldwide basis. Before reading the next lines of example code, you can try to do it on your own to see if you understood how to execute aggregations.
IMap<String, Employee> employees = hazelcastInstance.getMap( "employees" );
int count = employees.size();
Ok, after the quick joke of the previous two code lines, we look at the real two code lines:
IMap<String, Employee> employees = hazelcastInstance.getMap( "employees" );
int count = employees.aggregate( Supplier.all(), Aggregations.count() );
We now have an overview of how to use aggregations in real life situations. If you want to do your colleagues a favor, you might want to write your own additional set of aggregations. If so, then read the next section, Implementing Aggregations.
This section explains how to implement your own aggregations in your own application. It is an advanced section, so if you do not intend to implement your own aggregation, you might want to stop reading here and come back later when you need to know how to implement your own aggregation.
An Aggregation
implementation is defining a MapReduce task, but with a small difference: the Mapper
is always expected to work on a Supplier
that filters and/or transforms the mapped input value to some output value.
The main interface for making your own aggregation is com.hazelcast.mapreduce.aggregation.Aggregation
. It consists of four
methods.
interface Aggregation<Key, Supplied, Result> {
Mapper getMapper(Supplier<Key, ?, Supplied> supplier);
CombinerFactory getCombinerFactory();
ReducerFactory getReducerFactory();
Collator<Map.Entry, Result> getCollator();
}
The getMapper
and getReducerFactory
methods should return non-null values. getCombinerFactory
and getCollator
are
optional operations and you do not need to implement them. You can decide to implement them depending on the use case you want
to achieve.
For more information on how you implement mappers, combiners, reducers, and collators, refer to the MapReduce section.
For best speed and traffic usage, as mentioned in the MapReduce section, you should add a Combiner
to your aggregation
whenever it is possible to do some kind of pre-reduction step.
Your implementation also should use DataSerializable
or IdentifiedDataSerializable
for best compatibility and speed/stream-size
reasons.
A continuous query cache is used to cache the result of a continuous query. After the construction of a continuous query cache, all changes on the underlying IMap
are immediately reflected to this cache as a stream of events. Therefore, this cache will be an always up-to-date view of the IMap
. You can create a continuous query cache either on the client or member.
A continuous query cache is beneficial when you need to query the distributed IMap
data in a very frequent and fast way. By using a continuous query cache, the result of the query will always be ready and local to the application.
The following code snippet shows how you can access a continuous query cache from a member.
QueryCacheConfig queryCacheConfig = new QueryCacheConfig("cache-name");
queryCacheConfig.getPredicateConfig().setImplementation(new OddKeysPredicate());
MapConfig mapConfig = new MapConfig("map-name");
mapConfig.addQueryCacheConfig(queryCacheConfig);
Config config = new Config();
config.addMapConfig(mapConfig);
HazelcastInstance node = Hazelcast.newHazelcastInstance(config);
IEnterpriseMap<Integer, String> map = (IEnterpriseMap) node.getMap("map-name");
QueryCache<Integer, String> cache = map.getQueryCache("cache-name");
The following code snippet shows how you can access a continuous query cache from the client side. The difference in this code from the member side code above is that you configure and instantiate a client instance instead of a member instance.
QueryCacheConfig queryCacheConfig = new QueryCacheConfig("cache-name");
queryCacheConfig.getPredicateConfig().setImplementation(new OddKeysPredicate());
ClientConfig clientConfig = new ClientConfig();
clientConfig.addQueryCacheConfig("map-name", queryCacheConfig);
HazelcastInstance client = HazelcastClient.newHazelcastClient(clientConfig);
IEnterpriseMap<Integer, Integer> clientMap = (IEnterpriseMap) client.getMap("map-name");
QueryCache<Integer, Integer> cache = clientMap.getQueryCache("cache-name");
The following features of continuous query cache are valid for both the member and client.
IMap
data during the continuous query cache construction can be enabled/disabled according to the supplied predicate via QueryCacheConfig#setPopulate
.QueryCacheConfig#setEvictionConfig
.QueryCache#addEntryListener
.IMap
events are reflected in continuous query cache in the same order as they were generated on map entries. Since events are created on entries stored in partitions, ordering of events is maintained based on the ordering within the partition. You can add listeners to capture lost events using EventLostListener
and you can recover lost events with the method QueryCache#tryRecover
.
Recovery of lost events largely depends on the size of the buffer on Hazelcast members. Default buffer size is 16 per partition; i.e. 16 events per partition can be maintained in the buffer. If the event generation is high, setting the buffer size to a higher number will provide better chances of recovering lost events. You can set buffer size with QueryCacheConfig#setBufferSize
.
You can use the following example code for a recovery case.
QueryCache queryCache = map.getQueryCache("cache-name", new SqlPredicate("this > 20"), true);
queryCache.addEntryListener(new EventLostListener() {
@Override
public void eventLost(EventLostEvent event) {
queryCache.tryRecover();
}
}, false);
You can configure continuous query cache declaratively or programmatically.
QueryCache#get
from the underlying IMap
. This helps to decrease the initial population time when the values are very large.
This chapter explains the usage of Hazelcast in a transactional context. It describes the Hazelcast transaction types and how they work, how to provide XA (eXtended Architeture) transactions, and how to integrate Hazelcast with J2EE containers.
You create a TransactionContext
object to begin, commit, and rollback a transaction. You can obtain transaction-aware instances of queues, maps, sets, lists, multimaps via TransactionContext
, work with them, and commit/rollback in one shot. You can see the TransactionContext source code here.
Hazelcast supports two types of transactions: ONE_PHASE and TWO_PHASE. The type of transaction controls what happens when a member crashes while a transaction is committing. The default behavior is TWO_PHASE.
NOTE: Starting with Hazelcast 3.6, the transaction type LOCAL
has been deprecated. Please use ONE_PHASE
for the Hazelcast releases 3.6 and higher.
ONE_PHASE: By selecting this transaction type, you execute the transactions with a single phase that is committing the changes. Since a preparing phase does not exist, the conflicts are not detected. When a conflict happens while committing the changes (e.g., due to a member crash), not all the changes are written and this leaves the system in an inconsistent state.
TWO_PHASE: When you select this transaction type, Hazelcast first tries to execute the prepare phase. This phase fails if there are any conflicts. Once the prepare phase is successful, Hazelcast executes the commit phase (writing the changes). Before TWO_PHASE commits, Hazelcast copies the commit log to other members, so in case of a member failure, another member can complete the commit.
import java.util.Queue;
import java.util.Map;
import java.util.Set;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.Transaction;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
TransactionOptions options = new TransactionOptions()
.setTransactionType( TransactionType.ONE_PHASE );
TransactionContext context = hazelcastInstance.newTransactionContext( options );
context.beginTransaction();
TransactionalQueue queue = context.getQueue( "myqueue" );
TransactionalMap map = context.getMap( "mymap" );
TransactionalSet set = context.getSet( "myset" );
try {
Object obj = queue.poll();
//process obj
map.put( "1", "value1" );
set.add( "value" );
//do other things..
context.commitTransaction();
} catch ( Throwable t ) {
context.rollbackTransaction();
}
In a transaction, operations will not be executed immediately. Their changes will be local to the TransactionContext
until committed. However, they will ensure the changes via locks.
For the above example, when map.put
is executed, no data will be put in the map but the key will be locked against changes. While committing, operations will be executed, the value will be put to the map, and the key will be unlocked.
The isolation level in Hazelcast Transactions is READ_COMMITTED
. If you are in a transaction, you can read the data in your transaction and the data that is already committed. If you are not in a transaction, you can only read the committed data.
NOTE: The REPEATABLE_READ isolation level can also be exercised using the method getForUpdate()
of TransactionalMap
.
Hazelcast implements queue/set/list operations differently than map/multimap operations. For queue operations (offer, poll), offered and/or polled objects are copied to the owner member in order to safely commit/rollback. For map/multimap, Hazelcast first acquires the locks for the write operations (put, remove) and holds the differences (what is added/removed/updated) locally for each transaction. When the transaction is set to commit, Hazelcast will release the locks and apply the differences. When rolling back, Hazelcast will release the locks and discard the differences.
MapStore
and QueueStore
do not participate in transactions. Hazelcast will suppress exceptions thrown by the store in a transaction. Please refer to the XA Transactions section for further information.
As discussed in Creating a Transaction Interface, when you choose ONE_PHASE as the transaction type, Hazelcast tracks all changes you make locally in a commit log, i.e., a list of changes. In this case, all the other members are asked to agree that the commit can succeed and once they agree, Hazelcast starts to write the changes. However, if the member that initiates the commit crashes after it has written to at least one member (but has not completed writing to all other members), your system may be left in an inconsistent state.
On the other hand, if you choose TWO_PHASE as the transaction type, the commit log is again tracked locally but it is copied to another cluster member. Therefore, when a failure happens (e.g. the member initiating the commit crashes), you still have the commit log in another member and that member can complete the commit. However, copying the commit log to another member makes the TWO_PHASE approach slow.
Consequently, it is recommended that you choose ONE_PHASE as the transaction type if you want better performance, and that you choose TWO_PHASE if reliability of your system is more important than the performance.
XA describes the interface between the global transaction manager and the local resource manager. XA allows multiple resources (such as databases, application servers, message queues, transactional caches, etc.) to be accessed within the same transaction, thereby preserving the ACID properties across applications. XA uses a two-phase commit to ensure that all resources either commit or rollback any particular transaction consistently (all do the same).
When you implement the XAResource
interface, Hazelcast provides XA transactions. You can obtain the HazelcastXAResource
instance via the HazelcastInstance getXAResource
method. You can see the
HazelcastXAResource source code here.
Below is example code that uses Atomikos for transaction management.
UserTransactionManager tm = new UserTransactionManager();
tm.setTransactionTimeout(60);
tm.begin();
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
HazelcastXAResource xaResource = hazelcastInstance.getXAResource();
Transaction transaction = tm.getTransaction();
transaction.enlistResource(xaResource);
// other resources (database, app server etc...) can be enlisted
try {
TransactionContext context = xaResource.getTransactionContext();
TransactionalMap map = context.getMap("m");
map.put("key", "value");
// other resource operations
transaction.delistResource(xaResource, XAResource.TMSUCCESS);
tm.commit();
} catch (Exception e) {
tm.rollback();
}
You can integrate Hazelcast into J2EE containers. This integration is offered as a Hazelcast plugin. Please see its own GitHub repository at Hazelcast Resource Adapter for information on configuring the resource adapter, glassfish applications, and JBoss web applications.
This chapter describes the basics of JCache, the standardized Java caching layer API. The JCache caching API is specified by the Java Community Process (JCP) as Java Specification Request (JSR) 107.
Caching keeps data in memory that either are slow to calculate/process or originate from another underlying backend system. Caching is used to prevent additional request round trips for frequently used data. In both cases, caching can be used to gain performance or decrease application latencies.
Starting with Hazelcast release 3.3.1, Hazelcast offers a specification-compliant JCache implementation. To show our commitment to this important specification that the Java world was waiting for over a decade, we did not just provide a simple wrapper around our existing APIs; we implemented a caching structure from the ground up to optimize the behavior to the needs of JCache. The Hazelcast JCache implementation is 100% TCK (Technology Compatibility Kit) compliant and therefore passes all specification requirements.
In addition to the given specification, we added some features like asynchronous versions of almost all operations to give the user extra power.
This chapter gives a basic understanding of how to configure your application and how to setup Hazelcast to be your JCache provider. It also shows examples of basic JCache usage as well as the additionally offered features that are not part of JSR-107. To gain a full understanding of the JCache functionality and provided guarantees of different operations, read the specification document (which is also the main documentation for functionality) at the specification page of JSR-107.
This section shows what is necessary to provide the JCache API and the Hazelcast JCache implementation for your application. In addition, it demonstrates the different configuration options and describes the configuration properties.
To provide your application with this JCache functionality, your application needs the JCache API inside its classpath. This API is the bridge between the specified JCache standard and the implementation provided by Hazelcast.
The method of integrating the JCache API JAR into the application classpath depends on the build system used. For Maven, Gradle, SBT, Ivy, and many other build systems, all using Maven-based dependency repositories, perform the integration by adding the Maven coordinates to the build descriptor.
As already mentioned, you have to add JCache coordinates next to the default Hazelcast coordinates that might be already part of the application.
For Maven users, the coordinates look like the following code:
<dependency>
<groupId>javax.cache</groupId>
<artifactId>cache-api</artifactId>
<version>1.0.0</version>
</dependency>
With other build systems, you might need to describe the coordinates in a different way.
To activate Hazelcast as the JCache provider implementation, add either hazelcast-all.jar
or
hazelcast.jar
to the classpath (if not already available) by either one of the following Maven snippets.
If you use hazelcast-all.jar
:
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-all</artifactId>
<version>"your Hazelcast version, e.g. 3.7"</version>
</dependency>
If you use hazelcast.jar
:
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast</artifactId>
<version>"your Hazelcast version, e.g. 3.7"</version>
</dependency>
The users of other build systems have to adjust the definition of the dependency to their needs.
When the users want to use Hazelcast clients to connect to a remote cluster, the hazelcast-client.jar
dependency is also required
on the client side applications. This JAR is already included in hazelcast-all.jar
. Or, you can add it to the classpath using the following
Maven snippet:
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-client</artifactId>
<version>"your Hazelcast version, e.g. 3.7"</version>
</dependency>
For other build systems, for instance, ANT, the users have to download these dependencies from either the JSR-107 specification and Hazelcast community website (http://www.hazelcast.org) or from the Maven repository search page (http://search.maven.org).
Before moving on to configuration, let's have a look at a basic introductory example. The following code shows how to use the Hazelcast JCache integration inside an application in an easy but typesafe way.
// Retrieve the CachingProvider which is automatically backed by
// the chosen Hazelcast member or client provider
CachingProvider cachingProvider = Caching.getCachingProvider();
// Create a CacheManager
CacheManager cacheManager = cachingProvider.getCacheManager();
// Create a simple but typesafe configuration for the cache
CompleteConfiguration<String, String> config =
new MutableConfiguration<String, String>()
.setTypes( String.class, String.class );
// Create and get the cache
Cache<String, String> cache = cacheManager.createCache( "example", config );
// Alternatively to request an already existing cache
// Cache<String, String> cache = cacheManager
// .getCache( name, String.class, String.class );
// Put a value into the cache
cache.put( "world", "Hello World" );
// Retrieve the value again from the cache
String value = cache.get( "world" );
// Print the value 'Hello World'
System.out.println( value );
Although the example is simple, let's go through the code lines one by one.
First of all, we retrieve the javax.cache.spi.CachingProvider
using the static method from
javax.cache.Caching::
getCachingManager
, which automatically picks up Hazelcast as the underlying JCache implementation, if
available in the classpath. This way, the Hazelcast implementation of a CachingProvider
will automatically start a new Hazelcast
member or client (depending on the chosen provider type) and pick up the configuration from either the command line parameter
or from the classpath. We will show how to use an existing HazelcastInstance
later in this chapter; for now, we keep it simple.
In the next line, we ask the CachingProvider
to return a javax.cache.CacheManager
. This is the general application's entry
point into JCache. The CachingProvider
creates and manages named caches.
The next few lines create a simple javax.cache.configuration.MutableConfiguration
to configure the cache before actually
creating it. In this case, we only configure the key and value types to make the cache typesafe which is highly recommended and
checked on retrieval of the cache.
To create the cache, we call javax.cache.CacheManager::createCache
with a name for the cache and the previously created
configuration; the call returns the created cache. If you need to retrieve a previously created cache, you can use the corresponding method overload javax.cache.CacheManager::getCache
. If the cache was created using type parameters, you must retrieve the cache afterward using the type checking version of getCache
.
The following lines are simple put
and get
calls from the java.util.Map
interface. The
javax.cache.Cache::put
has a void
return type and does not return the previously assigned value of the key. To imitate the
java.util.Map::put
method, the JCache cache has a method called getAndPut
.
Hazelcast JCache provides for two different methods of cache configuration:
hazelcast.xml
or hazelcast-client.xml
.You can declare your JCache cache configuration using the hazelcast.xml
or hazelcast-client.xml
configuration files. Using this declarative configuration makes creating the javax.cache.Cache
fully transparent and automatically ensures internal thread safety. You do not need a call to javax.cache.Cache::createCache
in this case: you can retrieve the cache using
javax.cache.Cache::getCache
overloads and by passing in the name defined in the configuration for the cache.
To retrieve the cache that you defined in the declaration files, you need only perform a simple call (example below) because the cache is created automatically by the implementation.
CachingProvider cachingProvider = Caching.getCachingProvider();
CacheManager cacheManager = cachingProvider.getCacheManager();
Cache<Object, Object> cache = cacheManager
.getCache( "default", Object.class, Object.class );
Note that this section only describes the JCache provided standard properties. For the Hazelcast specific properties, please see the ICache Configuration section.
<cache name="default">
<key-type class-name="java.lang.Object" />
<value-type class-name="java.lang.Object" />
<statistics-enabled>false</statistics-enabled>
<management-enabled>false</management-enabled>
<read-through>true</read-through>
<write-through>true</write-through>
<cache-loader-factory
class-name="com.example.cache.MyCacheLoaderFactory" />
<cache-writer-factory
class-name="com.example.cache.MyCacheWriterFactory" />
<expiry-policy-factory
class-name="com.example.cache.MyExpirePolicyFactory" />
<cache-entry-listeners>
<cache-entry-listener old-value-required="false" synchronous="false">
<cache-entry-listener-factory
class-name="com.example.cache.MyEntryListenerFactory" />
<cache-entry-event-filter-factory
class-name="com.example.cache.MyEntryEventFilterFactory" />
</cache-entry-listener>
...
</cache-entry-listeners>
</cache>
key-type#class-name
: Fully qualified class name of the cache key type. Its default value is java.lang.Object
.value-type#class-name
: Fully qualified class name of the cache value type. Its default value is java.lang.Object
.statistics-enabled
: If set to true, statistics like cache hits and misses are collected. Its default value is false.management-enabled
: If set to true, JMX beans are enabled and collected statistics are provided. It doesn't automatically enable statistics collection. Defaults to false.read-through
: If set to true, enables read-through behavior of the cache to an underlying configured javax.cache.integration.CacheLoader
which is also known as lazy-loading. Its default value is false.write-through
: If set to true, enables write-through behavior of the cache to an underlying configured javax.cache.integration.CacheWriter
which passes any changed value to the external backend resource. Its default value is false.cache-loader-factory#class-name
: Fully qualified class name of the javax.cache.configuration.Factory
implementation providing a javax.cache.integration.CacheLoader
instance to the cache.cache-writer-factory#class-name
: Fully qualified class name of the javax.cache.configuration.Factory
implementation providing a javax.cache.integration.CacheWriter
instance to the cache.expiry-policy-factory#-class-name
: Fully qualified class name of the javax.cache.configuration.Factory
implementation providing a javax.cache.expiry.ExpiryPolicy
instance to the cache.cache-entry-listener
: A set of attributes and elements, explained below, to describe a javax.cache.event.
CacheEntryListener
.cache-entry-listener#old-value-required
: If set to true, previously assigned values for the affected keys will be sent to the javax.cache.event.CacheEntryListener
implementation. Setting this attribute to true creates additional traffic. Its default value is false.cache-entry-listener#synchronous
: If set to true, the javax.cache.event.CacheEntryListener
implementation will be called in a synchronous manner. Its default value is false.cache-entry-listener/entry-listener-factory#class-name
: Fully qualified class name of the javax.cache.configuration.Factory
implementation providing a javax.cache.event.CacheEntryListener
instance.cache-entry-listener/entry-event-filter-factory#class-name
: Fully qualified class name of the javax.cache.configuration.Factory
implementation providing a javax.cache.event.
CacheEntryEventFilter
instance.
NOTE: The JMX MBeans provided by Hazelcast JCache show statistics of the local member only.
To show the cluster-wide statistics, the user should collect statistic information from all members and accumulate them to
the overall statistics.
To configure the JCache programmatically:
javax.cache.configuration.MutableConfiguration
if you will use
only the JCache standard configuration,com.hazelcast.config.CacheConfig
for a deeper Hazelcast integration.com.hazelcast.config.CacheConfig
offers additional options that are specific to Hazelcast, such as asynchronous and synchronous backup counts.
Both classes share the same supertype interface javax.cache.configuration.
CompleteConfiguration
which is part of the JCache
standard.
NOTE: To stay vendor independent, try to keep your code as near as possible to the standard JCache API. We recommend that you use declarative configuration
and that you use the javax.cache.configuration.Configuration
or javax.cache.configuration.CompleteConfiguration
interfaces in
your code only when you need to pass the configuration instance throughout your code.
If you don't need to configure Hazelcast specific properties, we recommend that you instantiate
javax.cache.configuration.MutableConfiguration
and that you use the setters to configure Hazelcast as shown in the example in the
Example JCache Application section. Since the configurable properties are the same as the ones explained in the
JCache Declarative Configuration section, they are not mentioned here. For Hazelcast specific
properties, please read the ICache Configuration section section.
Use JCache providers to create caches for a specification compliant implementation. Those providers abstract the platform specific behavior and bindings, and provide the different JCache required features.
Hazelcast has two types of providers. Depending on your application setup and the cluster topology, you can use the Client Provider (used by Hazelcast clients) or the Server Provider (used by cluster members).
Configure the JCache javax.cache.spi.CachingProvider
by either specifying the provider at the command line or by declaring the provider inside the Hazelcast configuration XML file. For more information on setting properties in this XML
configuration file, please see the JCache Declarative Configuration section.
Hazelcast implements a delegating CachingProvider
that can automatically be configured for either client or member mode and that
delegates to the real underlying implementation based on the user's choice. Hazelcast recommends that you use this CachingProvider
implementation.
The delegating CachingProvider
s fully qualified class name is
com.hazelcast.cache.HazelcastCachingProvider
To configure the delegating provider at the command line, add the following parameter to the Java startup call, depending on the chosen provider:
-Dhazelcast.jcache.provider.type=[client|server]
By default, the delegating CachingProvider
is automatically picked up by the JCache SPI and provided as shown above. In cases where multiple javax.cache.spi.CachingProvider
implementations reside on the classpath (like in some Application
Server scenarios), you can select a CachingProvider
by explicitly calling Caching::getCachingProvider
overloads and providing them using the canonical class name of the provider to be used. The class names of member and client providers
provided by Hazelcast are mentioned in the following two subsections.
NOTE: Hazelcast advises that you use the Caching::getCachingProvider
overloads to select a
CachingProvider
explicitly. This ensures that uploading to later environments or Application Server versions doesn't result in
unexpected behavior like choosing a wrong CachingProvider
.
For more information on cluster topologies and Hazelcast clients, please see the Hazelcast Topology section.
For cluster topologies where Hazelcast light clients are used to connect to a remote Hazelcast cluster, use the Client Provider to configure JCache.
The Client Provider provides the same features as the Server Provider. However, it does not hold data on its own but instead delegates requests and calls to the remotely connected cluster.
The Client Provider can connect to multiple clusters at the same time. This can be achieved by scoping the client side
CacheManager
with different Hazelcast configuration files. For more information, please see
Scoping to Join Clusters.
To request this CachingProvider
using Caching#getCachingProvider( String )
or
Caching#getCachingProvider( String, ClassLoader )
, use the following fully qualified class name:
com.hazelcast.client.cache.impl.HazelcastClientCachingProvider
If a Hazelcast member is embedded into an application directly and the Hazelcast client is not used, the Server Provider is required. In this case, the member itself becomes a part of the distributed cache and requests and operations are distributed directly across the cluster by its given key.
The Server Provider provides the same features as the Client provider, but it keeps data in the local Hazelcast member and also distributes non-owned keys to other direct cluster members.
Like the Client Provider, the Server Provider can connect to multiple clusters at the same time. This can be achieved by scoping the client side CacheManager
with different Hazelcast configuration files. For more
information please see Scoping to Join Clusters.
To request this CachingProvider
using Caching#getCachingProvider( String )
or
Caching#getCachingProvider( String, ClassLoader )
, use the following fully qualified class name:
com.hazelcast.cache.impl.HazelcastServerCachingProvider
This section explains the JCache API by providing simple examples and use cases. While walking through the examples, we will have a look at a couple of the standard API classes and see how these classes are used.
The code in this subsection creates a small account application by providing a caching layer over an imagined database abstraction. The database layer will be simulated using single demo data in a simple DAO interface. To show the difference between the "database" access and retrieving values from the cache, a small waiting time is used in the DAO implementation to simulate network and database latency.
Before we implement the JCache caching layer, let's have a quick look at some basic classes we need for this example.
The User
class is the representation of a user table in the database. To keep it simple, it has just two properties:
userId
and username
.
public class User {
private int userId;
private String username;
// Getters and setters
}
The DAO interface is also kept easy in this example. It provides a simple method to retrieve (find) a user by its userId
.
public interface UserDAO {
User findUserById( int userId );
boolean storeUser( int userId, User user );
boolean removeUser( int userId );
Collection<Integer> allUserIds();
}
To show most of the standard features, the configuration example is a little more complex.
// Create javax.cache.configuration.CompleteConfiguration subclass
CompleteConfiguration<Integer, User> config =
new MutableConfiguration<Integer, User>()
// Configure the cache to be typesafe
.setTypes( Integer.class, User.class )
// Configure to expire entries 30 secs after creation in the cache
.setExpiryPolicyFactory( FactoryBuilder.factoryOf(
new AccessedExpiryPolicy( new Duration( TimeUnit.SECONDS, 30 ) )
) )
// Configure read-through of the underlying store
.setReadThrough( true )
// Configure write-through to the underlying store
.setWriteThrough( true )
// Configure the javax.cache.integration.CacheLoader
.setCacheLoaderFactory( FactoryBuilder.factoryOf(
new UserCacheLoader( userDao )
) )
// Configure the javax.cache.integration.CacheWriter
.setCacheWriterFactory( FactoryBuilder.factoryOf(
new UserCacheWriter( userDao )
) )
// Configure the javax.cache.event.CacheEntryListener with no
// javax.cache.event.CacheEntryEventFilter, to include old value
// and to be executed synchronously
.addCacheEntryListenerConfiguration(
new MutableCacheEntryListenerConfiguration<Integer, User>(
new UserCacheEntryListenerFactory(),
null, true, true
)
);
Let's go through this configuration line by line.
First, we set the expected types for the cache, which is already known from the previous example. On the next line, a
javax.cache.expiry.ExpirePolicy
is configured. Almost all integration ExpirePolicy
implementations are configured using
javax.cache.configuration.Factory
instances. Factory
and FactoryBuilder
are explained later in this chapter.
The next two lines configure the thread that will be read-through and write-through to the underlying backend resource that is configured
over the next few lines. The JCache API offers javax.cache.integration.CacheLoader
and javax.cache.integration.CacheWriter
to
implement adapter classes to any kind of backend resource, e.g. JPA, JDBC, or any other backend technology implementable in Java.
The interface provides the typical CRUD operations like create
, get
, update
, delete
, and some bulk operation versions of those
common operations. We will look into the implementation of those implementations later.
The last configuration setting defines entry listeners based on sub-interfaces of javax.cache.event.CacheEntryListener
. This
config does not use a javax.cache.event.CacheEntryEventFilter
since the listener is meant to be fired on every change that
happens on the cache. Again we will look in the implementation of the listener in later in this chapter.
A full running example that is presented in this
subsection is available in the
code samples repository.
The application is built to be a command line app. It offers a small shell to accept different commands. After startup, you can
enter help
to see all available commands and their descriptions.
In the Example JCache Application section, we have already seen a couple of the base classes and explained how those work. The following are quick descriptions of them:
javax.cache.Caching
:
The access point into the JCache API. It retrieves the general CachingProvider
backed by any compliant JCache
implementation, such as Hazelcast JCache.
javax.cache.spi.CachingProvider
:
The SPI that is implemented to bridge between the JCache API and the implementation itself. Hazelcast members and clients use different providers chosen as seen in the Configuring JCache Provider section which enable the JCache API to interact with Hazelcast clusters.
When a javax.cache.spi.CachingProvider::getCacheManager
overload is used that takes a java.lang.ClassLoader
argument, this
classloader will be part of the scope of the created java.cache.Cache
and it is not possible to retrieve it on other members.
We advise not to use those overloads, as they are not meant to be used in distributed environments!
javax.cache.CacheManager
:
The CacheManager
provides the capability to create new and manage existing JCache caches.
NOTE: A javax.cache.Cache
instance created with key and value types in the configuration
provides a type checking of those types at retrieval of the cache. For that reason, all non-types retrieval methods like
getCache
throw an exception because types cannot be checked.
javax.cache.configuration.Configuration
, javax.cache.configuration.MutableConfiguration
:
These two classes are used to configure a cache prior to retrieving it from a CacheManager
. The Configuration
interface,
therefore, acts as a common super type for all compatible configuration classes such as MutableConfiguration
.
Hazelcast itself offers a special implementation (com.hazelcast.config.CacheConfig
) of the Configuration
interface which
offers more options on the specific Hazelcast properties that can be set to configure features like synchronous and asynchronous
backups counts or selecting the underlying In Memory Format of the cache. For more information on this
configuration class, please see the reference in JCache Programmatic Configuration section.
javax.cache.Cache
:
This interface represents the cache instance itself. It is comparable to java.util.Map
but offers special operations dedicated
to the caching use case. Therefore, for example javax.cache.Cache::put
, unlike java.util.Map::put
, does not return the old
value previously assigned to the given key.
NOTE: Bulk operations on the Cache
interface guarantee atomicity per entry but not over
all given keys in the same bulk operations since no transactional behavior is applied over the whole batch process.
The javax.cache.configuration.Factory
implementations configure features like
CacheEntryListener
, ExpirePolicy
, and CacheLoader
s or CacheWriter
s. These factory implementations are required to distribute the
different features to members in a cluster environment like Hazelcast. Therefore, these factory implementations have to be serializable.
Factory
implementations are easy to do, as they follow the default Provider- or Factory-Pattern. The sample class
UserCacheEntryListenerFactory
shown below implements a custom JCache Factory
.
public class UserCacheEntryListenerFactory
implements Factory<CacheEntryListener<Integer, User>> {
@Override
public CacheEntryListener<Integer, User> create() {
// Just create a new listener instance
return new UserCacheEntryListener();
}
}
To simplify the process for the users, JCache API offers a set of helper methods collected in
javax.cache.
configuration.FactoryBuilder
. In the above configuration example, FactoryBuilder::factoryOf
creates a
singleton factory for the given instance.
javax.cache.integration.CacheLoader
loads cache entries from any external backend resource.
If the cache is
configured to be read-through
, then CacheLoader::load
is called transparently from the cache when the key or the value is not
yet found in the cache. If no value is found for a given key, it returns null.
If the cache is not configured to be read-through
, nothing is loaded automatically. The user code must call javax.cache.Cache::loadAll
to load data for the given set of keys into the cache.
For the bulk load operation (loadAll()
), some keys may not be found in the returned result set. In this case, a javax.cache.integration.CompletionListener
parameter can be used as an asynchronous callback after all the key-value pairs are loaded because loading many key-value pairs can take lots of time.
Let's look at the UserCacheLoader
implementation. This implementation is quite straight forward.
CacheLoader
.load
method to compute or retrieve the value corresponding to key
.loadAll
method to compute or retrieve the values corresponding to keys
.An important note is that
any kind of exception has to be wrapped into javax.cache.integration.CacheLoaderException
.
public class UserCacheLoader
implements CacheLoader<Integer, User>, Serializable {
private final UserDao userDao;
public UserCacheLoader( UserDao userDao ) {
// Store the dao instance created externally
this.userDao = userDao;
}
@Override
public User load( Integer key ) throws CacheLoaderException {
// Just call through into the dao
return userDao.findUserById( key );
}
@Override
public Map<Integer, User> loadAll( Iterable<? extends Integer> keys )
throws CacheLoaderException {
// Create the resulting map
Map<Integer, User> loaded = new HashMap<Integer, User>();
// For every key in the given set of keys
for ( Integer key : keys ) {
// Try to retrieve the user
User user = userDao.findUserById( key );
// If user is not found do not add the key to the result set
if ( user != null ) {
loaded.put( key, user );
}
}
return loaded;
}
}
You use a javax.cache.integration.CacheWriter
to update an external backend resource. If the cache is configured to be
write-through
, this process is executed transparently to the user's code. Otherwise, there is currently no way to trigger
writing changed entries to the external resource to a user-defined point in time.
If bulk operations throw an exception, java.util.Collection
has to be cleaned of all successfully written keys so
the cache implementation can determine what keys are written and can be applied to the cache state.
The following example performs the following tasks:
CacheWriter
.write
method to write the specified entry to the underlying store.writeAll
method to write the specified entires to the underlying store.delete
method to delete the key entry from the store.deleteAll
method to delete the data and keys from the underlying store for the given collection of keys, if present.public class UserCacheWriter
implements CacheWriter<Integer, User>, Serializable {
private final UserDao userDao;
public UserCacheWriter( UserDao userDao ) {
// Store the dao instance created externally
this.userDao = userDao;
}
@Override
public void write( Cache.Entry<? extends Integer, ? extends User> entry )
throws CacheWriterException {
// Store the user using the dao
userDao.storeUser( entry.getKey(), entry.getValue() );
}
@Override
public void writeAll( Collection<Cache.Entry<...>> entries )
throws CacheWriterException {
// Retrieve the iterator to clean up the collection from
// written keys in case of an exception
Iterator<Cache.Entry<...>> iterator = entries.iterator();
while ( iterator.hasNext() ) {
// Write entry using dao
write( iterator.next() );
// Remove from collection of keys
iterator.remove();
}
}
@Override
public void delete( Object key ) throws CacheWriterException {
// Test for key type
if ( !( key instanceof Integer ) ) {
throw new CacheWriterException( "Illegal key type" );
}
// Remove user using dao
userDao.removeUser( ( Integer ) key );
}
@Override
public void deleteAll( Collection<?> keys ) throws CacheWriterException {
// Retrieve the iterator to clean up the collection from
// written keys in case of an exception
Iterator<?> iterator = keys.iterator();
while ( iterator.hasNext() ) {
// Write entry using dao
delete( iterator.next() );
// Remove from collection of keys
iterator.remove();
}
}
}
Again, the implementation is pretty straightforward and also as above all exceptions thrown by the external resource, like
java.sql.SQLException
has to be wrapped into a javax.cache.integration.CacheWriterException
. Note this is a different
exception from the one thrown by CacheLoader
.
With javax.cache.processor.EntryProcessor
, you can apply an atomic function to a cache entry. In a distributed
environment like Hazelcast, you can move the mutating function to the member that owns the key. If the value
object is big, it might prevent traffic by sending the object to the mutator and sending it back to the owner to update it.
By default, Hazelcast JCache sends the complete changed value to the backup partition. Again, this can cause a lot of traffic if the object is big. The Hazelcast ICache extension can also prevent this. Further information is available at Implementing BackupAwareEntryProcessor.
An arbitrary number of arguments can be passed to the Cache::invoke
and Cache::invokeAll
methods. All of those arguments need
to be fully serializable because in a distributed environment like Hazelcast, it is very likely that these arguments have to be passed around the cluster.
The following example performs the following tasks.
EntryProcessor
.process
method to process an entry.public class UserUpdateEntryProcessor
implements EntryProcessor<Integer, User, User> {
@Override
public User process( MutableEntry<Integer, User> entry, Object... arguments )
throws EntryProcessorException {
// Test arguments length
if ( arguments.length < 1 ) {
throw new EntryProcessorException( "One argument needed: username" );
}
// Get first argument and test for String type
Object argument = arguments[0];
if ( !( argument instanceof String ) ) {
throw new EntryProcessorException(
"First argument has wrong type, required java.lang.String" );
}
// Retrieve the value from the MutableEntry
User user = entry.getValue();
// Retrieve the new username from the first argument
String newUsername = ( String ) arguments[0];
// Set the new username
user.setUsername( newUsername );
// Set the changed user to mark the entry as dirty
entry.setValue( user );
// Return the changed user to return it to the caller
return user;
}
}
NOTE: By executing the bulk Cache::invokeAll
operation, atomicity is only guaranteed for a
single cache entry. No transactional rules are applied to the bulk operation.
NOTE: JCache EntryProcessor
implementations are not allowed to call
javax.cache.Cache
methods. This prevents operations from deadlocking between different calls.
In addition, when using a Cache::invokeAll
method, a java.util.Map
is returned that maps the key to its
javax.cache.processor.EntryProcessorResult
, which itself wraps the actual result or a thrown
javax.cache.processor.EntryProcessorException
.
The javax.cache.event.CacheEntryListener
implementation is straight forward. CacheEntryListener
is a super-interface that is used as a marker for listener classes in JCache. The specification brings a set of sub-interfaces.
CacheEntryCreatedListener
: Fires after a cache entry is added (even on read-through by a CacheLoader
) to the cache.CacheEntryUpdatedListener
: Fires after an already existing cache entry updates.CacheEntryRemovedListener
: Fires after a cache entry was removed (not expired) from the cache.CacheEntryExpiredListener
: Fires after a cache entry has been expired. Expiry does not have to be a parallel process-- it is only required to be executed on the keys that are requested by Cache::get
and some other operations. For a full table of expiry please see the https://www.jcp.org/en/jsr/detail?id=107 point 6. To configure CacheEntryListener
, add a javax.cache.configuration.CacheEntryListenerConfiguration
instance to
the JCache configuration class, as seen in the above example configuration. In addition, listeners can be configured to be
executed synchronously (blocking the calling thread) or asynchronously (fully running in parallel).
In this example application, the listener is implemented to print event information on the console. That visualizes what is going on in the cache. This application performs the following tasks:
onCreated
method to call after an entry is created.onUpdated
method to call after an entry is updated.onRemoved
method to call after an entry is removed.onExpired
method to call after an entry expires.printEvents
to print event information on the console.public class UserCacheEntryListener
implements CacheEntryCreatedListener<Integer, User>,
CacheEntryUpdatedListener<Integer, User>,
CacheEntryRemovedListener<Integer, User>,
CacheEntryExpiredListener<Integer, User> {
@Override
public void onCreated( Iterable<CacheEntryEvent<...>> cacheEntryEvents )
throws CacheEntryListenerException {
printEvents( cacheEntryEvents );
}
@Override
public void onUpdated( Iterable<CacheEntryEvent<...>> cacheEntryEvents )
throws CacheEntryListenerException {
printEvents( cacheEntryEvents );
}
@Override
public void onRemoved( Iterable<CacheEntryEvent<...>> cacheEntryEvents )
throws CacheEntryListenerException {
printEvents( cacheEntryEvents );
}
@Override
public void onExpired( Iterable<CacheEntryEvent<...>> cacheEntryEvents )
throws CacheEntryListenerException {
printEvents( cacheEntryEvents );
}
private void printEvents( Iterable<CacheEntryEvent<...>> cacheEntryEvents ) {
Iterator<CacheEntryEvent<...>> iterator = cacheEntryEvents.iterator();
while ( iterator.hasNext() ) {
CacheEntryEvent<...> event = iterator.next();
System.out.println( event.getEventType() );
}
}
}
In JCache, javax.cache.expiry.ExpirePolicy
implementations are used to automatically expire cache entries based on different rules.
Expiry timeouts are defined using javax.cache.expiry.Duration
, which is a pair of java.util.concurrent.TimeUnit
, that
describes a time unit and a long, defining the timeout value. The minimum allowed TimeUnit
is TimeUnit.MILLISECONDS
.
The long value durationAmount
must be equal or greater than zero. A value of zero (or Duration.ZERO
) indicates that the
cache entry expires immediately.
By default, JCache delivers a set of predefined expiry strategies in the standard API.
AccessedExpiryPolicy
: Expires after a given set of time measured from creation of the cache entry. The expiry timeout is updated on accessing the key.CreatedExpiryPolicy
: Expires after a given set of time measured from creation of the cache entry. The expiry timeout is never updated.EternalExpiryPolicy
: Never expires. This is the default behavior, similar to ExpiryPolicy
being set to null.ModifiedExpiryPolicy
: Expires after a given set of time measured from creation of the cache entry. The expiry timeout is updated on updating the key.TouchedExpiryPolicy
: Expires after a given set of time measured from creation of the cache entry. The expiry timeout is updated on accessing or updating the key.Because EternalExpirePolicy
does not expire cache entries, it is still possible to evict values from memory if an underlying
CacheLoader
is defined.
You can retrieve javax.cache.Cache
instances directly through HazelcastInstance::getCache(String name)
method.
The parameter name
in HazelcastInstance::getCache(String name)
is the full cache name except the Hazelcast prefix, i.e., /hz/
.
If you create a cache through a CacheManager
which has its own specified URI scope (and/or specified classloader),
it must be prepended to the pure cache name as a prefix while retrieving the cache through HazelcastInstance::getCache(String name)
.
Prefix generation for full cache name (except the Hazelcast prefix, which is /hz/
) is exposed through
com.hazelcast.cache.CacheUtil#getPrefixedCacheName(String name, java.net.URI uri, ClassLoader classloader)
.
If the URI scope and classloader is not specified, the pure cache name can be used directly while retrieving cache over HazelcastInstance
.
If you have a cache which is not created, but is defined/exists (cache is specified in Hazelcast configuration but not created yet), you can retrieve this cache by its name. This also triggers cache creation before retrieving it. This retrieval is supported through HazelcastInstance
. However, HazelcastInstance
does not support creating a cache by specifying configuration; this is supported by Hazelcast's CacheManager
as it is.
NOTE: If a valid (rather than 1.0.0-PFD or 0.x versions) JCache library does not exist on the classpath, IllegalStateException
is thrown.
HazelcastInstance
is injected into the following cache API interfaces (provided by javax.cache.Cache
and com.hazelcast.cache.ICache
) if they implement HazelcastInstanceAware
interface:
ExpiryPolicyFactory
and ExpiryPolicy
[provided by javax.cache.Cache
]CacheLoaderFactory
and CacheLoader
[provided by javax.cache.Cache
]CacheWriteFactory
and CacheWriter
[provided by javax.cache.Cache
]EntryProcessor
[provided by javax.cache.Cache
]CacheEntryListener
(CacheEntryCreatedListener
, CacheEntryUpdatedListener
, CacheEntryRemovedListener
, CacheEntryExpiredListener
) [provided by javax.cache.Cache
]CacheEntryEventFilter
[provided by javax.cache.Cache
]CompletionListener
[provided by javax.cache.Cache
]CachePartitionLostListener
[provided by com.hazelcast.cache.ICache
]Hazelcast provides extension methods to Cache API through the interface com.hazelcast.cache.ICache
.
It has two sets of extensions:
ExpiryPolicy
parameter to apply on that specific operation. See Custom ExpiryPolicy.As mentioned before, you can scope a CacheManager
in the case of a client to connect to multiple clusters. In the case of an embedded member, you can scope a CacheManager
to join different clusters at the same time. This process is called scoping. To apply scoping, request
a CacheManager
by passing a java.net.URI
instance to CachingProvider::getCacheManager
. The java.net.URI
instance must point to either a Hazelcast configuration or to the name of a named
com.hazelcast.core.HazelcastInstance
instance.
NOTE: Multiple requests for the same java.net.URI
result in returning a CacheManager
instance that shares the same HazelcastInstance
as the CacheManager
returned by the previous call.
To connect or join different clusters, apply a configuration scope to the CacheManager
. If the same URI
is
used to request a CacheManager
that was created previously, those CacheManager
s share the same underlying HazelcastInstance
.
To apply a configuration scope, pass in the path of the configuration file using the location property
HazelcastCachingProvider#HAZELCAST_CONFIG_LOCATION
(which resolves to hazelcast.config.location
) as a mapping inside a
java.util.Properties
instance to the CachingProvider#getCacheManager(uri, classLoader, properties)
call.
Here is an example of using Configuration Scope.
CachingProvider cachingProvider = Caching.getCachingProvider();
// Create Properties instance pointing to a Hazelcast config file
Properties properties = new Properties();
properties.setProperty( HazelcastCachingProvider.HAZELCAST_CONFIG_LOCATION,
"classpath://my-configs/scoped-hazelcast.xml" );
URI cacheManagerName = new URI( "my-cache-manager" );
CacheManager cacheManager = cachingProvider
.getCacheManager( cacheManagerName, null, properties );
Here is an example using HazelcastCachingProvider::propertiesByLocation
helper method.
CachingProvider cachingProvider = Caching.getCachingProvider();
// Create Properties instance pointing to a Hazelcast config file
String configFile = "classpath://my-configs/scoped-hazelcast.xml";
Properties properties = HazelcastCachingProvider
.propertiesByLocation( configFile );
URI cacheManagerName = new URI( "my-cache-manager" );
CacheManager cacheManager = cachingProvider
.getCacheManager( cacheManagerName, null, properties );
The retrieved CacheManager
is scoped to use the HazelcastInstance
that was just created and was configured using the given XML
configuration file.
Available protocols for config file URL include classpath://
to point to a classpath location, file://
to point to a filesystem
location, http://
an https://
for remote web locations. In addition, everything that does not specify a protocol is recognized
as a placeholder that can be configured using a system property.
String configFile = "my-placeholder";
Properties properties = HazelcastCachingProvider
.propertiesByLocation( configFile );
You can set this on the command line.
-Dmy-placeholder=classpath://my-configs/scoped-hazelcast.xml
You should consider the following rules about the Hazelcast instance name when you specify the configuration file location using HazelcastCachingProvider#HAZELCAST_CONFIG_LOCATION
(which resolves to hazelcast.config.location
):
HazelcastCachingProvider#HAZELCAST_INSTANCE_NAME
(which resolves to hazelcast.instance.name
) property, this property is used as the instance name even though you configured the instance name in the configuration file.HazelcastCachingProvider#HAZELCAST_INSTANCE_NAME
but you configure the instance name in the configuration file using the element <instance-name>
, this element's value will be used as the instance name.
NOTE: No check is performed to prevent creating multiple CacheManager
s with the same cluster
configuration on different configuration files. If the same cluster is referred from different configuration files, multiple
cluster members or clients are created.
NOTE: The configuration file location will not be a part of the resulting identity of the
CacheManager
. An attempt to create a CacheManager
with a different set of properties but an already used name will result in
undefined behavior.
You can bind CacheManager
to an existing and named HazelcastInstance
instance. If the instanceName
is specified in com.hazelcast.config.Config
, it can be used directly by passing it to CachingProvider
implementation. Otherwise (instanceName
not set or instance is a client instance) you must get the instance name from the HazelcastInstance
instance via the String getName()
method to pass the CachingProvider
implementation. Please note that instanceName
is not configurable for the client side HazelcastInstance
instance and is auto-generated by using group name (if it is specified). In general, String getName()
method over HazelcastInstance
is safer and the preferable way to get the name of the instance. Multiple CacheManager
s created using an equal java.net.URI
will share the same HazelcastInstance
.
A named scope is applied nearly the same way as the configuration scope: pass in the instance name using the HazelcastCachingProvider#HAZELCAST_INSTANCE_NAME
(which resolves to hazelcast.instance.name
) property as a mapping inside a java.util.Properties
instance to the CachingProvider#getCacheManager(uri, classLoader, properties)
call.
Here is an example of Named Instance Scope with specified name.
Config config = new Config();
config.setInstanceName( "my-named-hazelcast-instance" );
// Create a named HazelcastInstance
Hazelcast.newHazelcastInstance( config );
CachingProvider cachingProvider = Caching.getCachingProvider();
// Create Properties instance pointing to a named HazelcastInstance
Properties properties = new Properties();
properties.setProperty( HazelcastCachingProvider.HAZELCAST_INSTANCE_NAME,
"my-named-hazelcast-instance" );
URI cacheManagerName = new URI( "my-cache-manager" );
CacheManager cacheManager = cachingProvider
.getCacheManager( cacheManagerName, null, properties );
Here is an example of Named Instance Scope with auto-generated name.
Config config = new Config();
// Create a auto-generated named HazelcastInstance
HazelcastInstance instance = Hazelcast.newHazelcastInstance( config );
String instanceName = instance.getName();
CachingProvider cachingProvider = Caching.getCachingProvider();
// Create Properties instance pointing to a named HazelcastInstance
Properties properties = new Properties();
properties.setProperty( HazelcastCachingProvider.HAZELCAST_INSTANCE_NAME,
instanceName );
URI cacheManagerName = new URI( "my-cache-manager" );
CacheManager cacheManager = cachingProvider
.getCacheManager( cacheManagerName, null, properties );
Here is an example of Named Instance Scope with auto-generated name on client instance.
ClientConfig clientConfig = new ClientConfig();
ClientNetworkConfig networkConfig = clientConfig.getNetworkConfig();
networkConfig.addAddress("127.0.0.1", "127.0.0.2");
// Create a client side HazelcastInstance
HazelcastInstance instance = HazelcastClient.newHazelcastClient( clientConfig );
String instanceName = instance.getName();
CachingProvider cachingProvider = Caching.getCachingProvider();
// Create Properties instance pointing to a named HazelcastInstance
Properties properties = new Properties();
properties.setProperty( HazelcastCachingProvider.HAZELCAST_INSTANCE_NAME,
instanceName );
URI cacheManagerName = new URI( "my-cache-manager" );
CacheManager cacheManager = cachingProvider
.getCacheManager( cacheManagerName, null, properties );
Here is an example using HazelcastCachingProvider::propertiesByInstanceName
method.
Config config = new Config();
config.setInstanceName( "my-named-hazelcast-instance" );
// Create a named HazelcastInstance
Hazelcast.newHazelcastInstance( config );
CachingProvider cachingProvider = Caching.getCachingProvider();
// Create Properties instance pointing to a named HazelcastInstance
Properties properties = HazelcastCachingProvider
.propertiesByInstanceName( "my-named-hazelcast-instance" );
URI cacheManagerName = new URI( "my-cache-manager" );
CacheManager cacheManager = cachingProvider
.getCacheManager( cacheManagerName, null, properties );
NOTE: The instanceName
will not be a part of the resulting identity of the CacheManager
.
An attempt to create a CacheManager
with a different set of properties but an already used name will result in undefined behavior.
The java.net.URI
s that don't use the above-mentioned Hazelcast-specific schemes are recognized as namespacing. Those
CacheManager
s share the same underlying default HazelcastInstance
created (or set) by the CachingProvider
, but they cache with the
same names and different namespaces on the CacheManager
level, and therefore they won't share the same data. This is useful where multiple
applications might share the same Hazelcast JCache implementation (e.g., on application or OSGi servers) but are developed by
independent teams. To prevent interfering on caches using the same name, every application can use its own namespace when
retrieving the CacheManager
.
Here is an example of using namespacing.
CachingProvider cachingProvider = Caching.getCachingProvider();
URI nsApp1 = new URI( "application-1" );
CacheManager cacheManagerApp1 = cachingProvider.getCacheManager( nsApp1, null );
URI nsApp2 = new URI( "application-2" );
CacheManager cacheManagerApp2 = cachingProvider.getCacheManager( nsApp2, null );
That way both applications share the same HazelcastInstance
instance but not the same caches.
Besides Scoping to Join Clusters and Namespacing, which are implemented using the URI feature of the
specification, all other extended operations are required to retrieve the com.hazelcast.cache.ICache
interface instance from
the JCache javax.cache.Cache
instance. For Hazelcast, both interfaces are implemented on the same object instance. It
is recommended that you stay with the specification method to retrieve the ICache
version, since ICache
might be subject to change without notification.
To retrieve or unwrap the ICache
instance, you can execute the following code example:
CachingProvider cachingProvider = Caching.getCachingProvider();
CacheManager cacheManager = cachingProvider.getCacheManager();
Cache<Object, Object> cache = cacheManager.getCache( ... );
ICache<Object, Object> unwrappedCache = cache.unwrap( ICache.class );
After unwrapping the Cache
instance into an ICache
instance, you have access to all of the following operations, e.g.,
ICache Async Methods and ICache Convenience Methods.
As mentioned in the JCache Declarative Configuration section, the Hazelcast ICache extension offers additional configuration properties over the default JCache configuration. These additional properties include internal storage format, backup counts, eviction policy and quorum reference.
The declarative configuration for ICache is a superset of the previously discussed JCache configuration:
<cache>
<!-- ... default cache configuration goes here ... -->
<backup-count>1</backup-count>
<async-backup-count>1</async-backup-count>
<in-memory-format>BINARY</in-memory-format>
<eviction size="10000" max-size-policy="ENTRY_COUNT" eviction-policy="LRU" />
<partition-lost-listeners>
<partition-lost-listener>CachePartitionLostListenerImpl</partition-lost-listener>
</partition-lost-listeners>
<quorum-ref>quorum-name</quorum-ref>
<disable-per-entry-invalidation-events>true</disable-per-entry-invalidation-events>
</cache>
backup-count
: Number of synchronous backups. Those backups are executed before the mutating cache operation is finished. The mutating operation is blocked. backup-count
default value is 1.async-backup-count
: Number of asynchronous backups. Those backups are executed asynchronously so the mutating operation is not blocked and it will be done immediately. async-backup-count
default value is 0. in-memory-format
: Internal storage format. For more information, please see the In Memory Format section. Default is BINARY
.eviction
: Defines the used eviction strategies and sizes for the cache. For more information on eviction, please see the JCache Eviction.size
: Maximum number of records or maximum size in bytes depending on the max-size-policy
property. Size can be any integer between 0
and Integer.MAX_VALUE
. Default max-size-policy is ENTRY_COUNT
and default size is 10.000
.max-size-policy
: Maximum size. If maximum size is reached, the cache is evicted based on the eviction policy. Default max-size-policy is ENTRY_COUNT
and default size is 10.000
. The following eviction policies are available:ENTRY_COUNT
: Maximum number of cache entries in the cache. Available on heap based cache record store only.USED_NATIVE_MEMORY_SIZE
: Maximum used native memory size in megabytes per cache for each Hazelcast instance. Available on High-Density Memory cache record store only.USED_NATIVE_MEMORY_PERCENTAGE
: Maximum used native memory size percentage per cache for each Hazelcast instance. Available on High-Density Memory cache record store only.FREE_NATIVE_MEMORY_SIZE
: Minimum free native memory size in megabytes for each Hazelcast instance. Available on High-Density Memory cache record store only.FREE_NATIVE_MEMORY_PERCENTAGE
: Minimum free native memory size percentage for each Hazelcast instance. Available on High-Density Memory cache record store only.eviction-policy
: Eviction policy that compares values to find the best matching eviction candidate. Default is LRU
.LRU
: Less Recently Used - finds the best eviction candidate based on the lastAccessTime.LFU
: Less Frequently Used - finds the best eviction candidate based on the number of hits.partition-lost-listeners
: Defines listeners for dispatching partition lost events for the cache. For more information, please see the ICache Partition Lost Listener section.quorum-ref
: Name of quorum configuration that you want this cache to use.disable-per-entry-invalidation-events
: Disables invalidation events for each entry; but full-flush invalidation events are still enabled. Full-flush invalidation means the invalidation of events for all entries when clear
is called. The default value is false
.Since javax.cache.configuration.MutableConfiguration
misses the above additional configuration properties, Hazelcast ICache extension
provides an extended configuration class called com.hazelcast.config.CacheConfig
. This class is an implementation of javax.cache.configuration.CompleteConfiguration
and all the properties shown above can be configured
using its corresponding setter methods.
NOTE: At the client side, ICache can be configured only programmatically.
As another addition of Hazelcast ICache over the normal JCache specification, Hazelcast provides asynchronous versions of almost
all methods, returning a com.hazelcast.core.ICompletableFuture
. By using these methods and the returned future objects, you can use JCache in a reactive way by registering zero or more callbacks on the future to prevent blocking the current thread.
The asynchronous versions of the methods append the phrase Async
to the method name. The example code below uses the method putAsync()
.
ICache<Integer, String> unwrappedCache = cache.unwrap( ICache.class );
ICompletableFuture<String> future = unwrappedCache.putAsync( 1, "value" );
future.andThen( new ExecutionCallback<String>() {
public void onResponse( String response ) {
System.out.println( "Previous value: " + response );
}
public void onFailure( Throwable t ) {
t.printStackTrace();
}
} );
Following methods are available in asynchronous versions:
get(key)
:getAsync(key)
getAsync(key, expiryPolicy)
put(key, value)
:putAsync(key, value)
putAsync(key, value, expiryPolicy)
putIfAbsent(key, value)
:putIfAbsentAsync(key, value)
putIfAbsentAsync(key, value, expiryPolicy)
getAndPut(key, value)
:getAndPutAsync(key, value)
getAndPutAsync(key, value, expiryPolicy)
remove(key)
:removeAsync(key)
remove(key, value)
:removeAsync(key, value)
getAndRemove(key)
:getAndRemoveAsync(key)
replace(key, value)
:replaceAsync(key, value)
replaceAsync(key, value, expiryPolicy)
replace(key, oldValue, newValue)
:replaceAsync(key, oldValue, newValue)
replaceAsync(key, oldValue, newValue, expiryPolicy)
getAndReplace(key, value)
:getAndReplaceAsync(key, value)
getAndReplaceAsync(key, value, expiryPolicy)
The methods with a given javax.cache.expiry.ExpiryPolicy
are further discussed in the
Defining a Custom ExpiryPolicy.
NOTE: Asynchronous versions of the methods are not compatible with synchronous events.
The JCache specification has an option to configure a single ExpiryPolicy
per cache. Hazelcast ICache extension
offers the possibility to define a custom ExpiryPolicy
per key by providing a set of method overloads with an expirePolicy
parameter, as in the list of asynchronous methods in the Async Methods section. This means that you can pass custom expiry policies to a cache operation.
Here is how an ExpirePolicy
is set on JCache configuration:
CompleteConfiguration<String, String> config =
new MutableConfiguration<String, String>()
setExpiryPolicyFactory(
AccessedExpiryPolicy.factoryOf( Duration.ONE_MINUTE )
);
To pass a custom ExpirePolicy
, a set of overloads is provided. You can use them as shown in the following code example.
ICache<Integer, String> unwrappedCache = cache.unwrap( ICache.class );
unwrappedCache.put( 1, "value", new AccessedExpiryPolicy( Duration.ONE_DAY ) );
The ExpirePolicy
instance can be pre-created, cached, and re-used, but only for each cache instance. This is because ExpirePolicy
implementations can be marked as java.io.Closeable
. The following list shows the provided method overloads over javax.cache.Cache
by com.hazelcast.cache.ICache
featuring the ExpiryPolicy
parameter:
get(key)
:get(key, expiryPolicy)
getAll(keys)
:getAll(keys, expirePolicy)
put(key, value)
:put(key, value, expirePolicy)
getAndPut(key, value)
:getAndPut(key, value, expirePolicy)
putAll(map)
:putAll(map, expirePolicy)
putIfAbsent(key, value)
:putIfAbsent(key, value, expirePolicy)
replace(key, value)
:replace(key, value, expirePolicy)
replace(key, oldValue, newValue)
:replace(key, oldValue, newValue, expirePolicy)
getAndReplace(key, value)
:getAndReplace(key, value, expirePolicy)
Asynchronous method overloads are not listed here. Please see ICache Async Methods for the list of asynchronous method overloads.
Caches are generally not expected to grow to an infinite size. Implementing an expiry policy is one way you can prevent infinite growth, but sometimes it is hard to define a meaningful expiration timeout. Therefore, Hazelcast JCache provides the eviction feature. Eviction offers the possibility of removing entries based on the cache size or amount of used memory (Hazelcast Enterprise Only) and not based on timeouts.
Since a cache is designed for high throughput and fast reads, Hazelcast put a lot of effort into designing the eviction system to be as predictable as possible. All built-in implementations provide an amortized O(1) runtime. The default operation runtime is rendered as O(1), but it can be faster than the normal runtime cost if the algorithm finds an expired entry while sampling.
Most importantly, typical production systems have two common types of caches:
Hazelcast JCache eviction supports both types of caches using a slightly different approach based on the configured maximum size of the cache. For detailed information, please see the Eviction Algorithm section.
Hazelcast JCache provides two commonly known eviction policies, LRU and LFU, but loosens the rules for predictable runtime
behavior. LRU, normally recognized as Least Recently Used
, is implemented as Less Recently Used
, and LFU known as Least Frequently Used
is implemented as
Less Frequently Used
. The details about this difference are explained in the
Eviction Algorithm section.
Eviction Policies are configured by providing the corresponding abbreviation to the configuration as shown in the ICache Configuration section. As already mentioned, two built-in policies are available:
To configure the use of the LRU (Less Recently Used) policy:
<eviction size="10000" max-size-policy="ENTRY_COUNT" eviction-policy="LRU" />
And to configure the use of the LFU (Less Frequently Used) policy:
<eviction size="10000" max-size-policy="ENTRY_COUNT" eviction-policy="LFU" />
The default eviction policy is LRU. Therefore, Hazelcast JCache does not offer the possibility of performing no eviction.
Besides out of the box eviction policies LFU and LRU, you can also specify your custom eviction policies through the eviction configuration either programmatically or declaratively.
You can provide your com.hazelcast.cache.CacheEvictionPolicyComparator
implementation to compare com.hazelcast.cache.CacheEntryView
s. Supplied CacheEvictionPolicyComparator
is used to compare cache entry views to select the one with higher priority to evict.
Here is an example for custom eviction policy comparator implementation for JCache:
public class MyCacheEvictionPolicyComparator
extends CacheEvictionPolicyComparator<Long, String> {
@Override
public int compare(CacheEntryView<Long, String> e1, CacheEntryView<Long, String> e2) {
long id1 = e1.getKey();
long id2 = e2.getKey();
if (id1 > id2) {
return FIRST_ENTRY_HAS_HIGHER_PRIORITY_TO_BE_EVICTED; // -1
} else if (id1 < id2) {
return SECOND_ENTRY_HAS_HIGHER_PRIORITY_TO_BE_EVICTED; // +1
} else {
return BOTH_OF_ENTRIES_HAVE_SAME_PRIORITY_TO_BE_EVICTED; // 0
}
}
}
Custom eviction policy comparator can be specified through the eviction configuration
by giving the full class name of the EvictionPolicyComparator
(CacheEvictionPolicyComparator
for JCache and its near cache)
implementation or by specifying its instance itself.
Programmatic:
You can specify the full class name of custom EvictionPolicyComparator
(CacheEvictionPolicyComparator
for JCache and its near cache) implementation
through EvictionConfig
. This approach is useful when eviction configuration is specified at the client side
and custom EvictionPolicyComparator
implementation class itself does not exist at the client but at server side.
CacheConfig cacheConfig = new CacheConfig();
...
EvictionConfig evictionConfig =
new EvictionConfig(50000,
MaxSizePolicy.ENTRY_COUNT,
"com.mycompany.MyEvictionPolicyComparator");
cacheConfig.setEvictionConfig(evictionConfig);
You can specify the custom EvictionPolicyComparator
(CacheEvictionPolicyComparator
for JCache and its near cache) instance itself directly through EvictionConfig
.
CacheConfig cacheConfig = new CacheConfig();
...
EvictionConfig evictionConfig =
new EvictionConfig(50000,
MaxSizePolicy.ENTRY_COUNT,
new MyEvictionPolicyComparator());
cacheConfig.setEvictionConfig(evictionConfig);
Declarative:
You can specify the full class name of custom EvictionPolicyComparator
(CacheEvictionPolicyComparator
for JCache and its near cache) implementation
in the <eviction>
tag through comparator-class-name
attribute in Hazelcast configuration XML file.
<cache name="cacheWithCustomEvictionPolicyComparator">
<eviction size="50000" max-size-policy="ENTRY_COUNT" comparator-class-name="com.mycompany.MyEvictionPolicyComparator"/>
</cache>
Declarative for Spring:
You can specify the full class name of custom EvictionPolicyComparator
(CacheEvictionPolicyComparator
for JCache and its near cache) implementation
in the <eviction>
tag through comparator-class-name
attribute in Hazelcast Spring configuration XML file.
<hz:cache name="cacheWithCustomEvictionPolicyComparator">
<hz:eviction size="50000" max-size-policy="ENTRY_COUNT" comparator-class-name="com.mycompany.MyEvictionPolicyComparator"/>
</hz:cache>
You can specify the custom EvictionPolicyComparator
(CacheEvictionPolicyComparator
for JCache and its near cache) bean in the <eviction>
tag
by referencing through comparator-bean
attribute in Hazelcast Spring configuration XML file
<hz:cache name="cacheWithCustomEvictionPolicyComparator">
<hz:eviction size="50000" max-size-policy="ENTRY_COUNT" comparator-bean="myEvictionPolicyComparatorBean"/>
</hz:cache>
Eviction strategies implement the logic of selecting one or more eviction candidates from the underlying storage implementation and passing them to the eviction policies. Hazelcast JCache provides an amortized O(1) cost implementation for this strategy to select a fixed number of samples from the current partition that it is executed against.
The default implementation is com.hazelcast.cache.impl.eviction.impl.strategy.sampling.SamplingBasedEvictionStrategy
which, as
mentioned, samples 15 random elements. A detailed description of the algorithm will be explained in the next section.
The Hazelcast JCache eviction algorithm is specially designed for the use case of high performance caches and with predictability in mind. The built-in implementations provide an amortized O(1) runtime and therefore provide a highly predictable runtime behavior which does not rely on any kind of background threads to handle the eviction. Therefore, the algorithm takes some assumptions into account to prevent network operations and concurrent accesses.
As an explanation of how the algorithm works, let's examine the following flowchart step by step.
As seen in the flowchart, the general eviction operation is easy. As long as the cache does not reach its maximum capacity, or you execute updates (put/replace), no eviction is executed.
To prevent network operations and concurrent access, as mentioned earlier, the cache size is estimated based on the size of the currently handled partition. Due to the imbalanced partitions, the single partitions might start to evict earlier than the other partitions.
As mentioned in the Cache Types section, typically two types of caches are found in the production systems. For small caches, referred to as Reference Caches, the eviction algorithm has a special set of rules depending on the maximum configured cache size. Please see the Reference Caches section for details. The other type of cache is referred to as an Active DataSet Cache, which in most cases makes heavy use of the eviction to keep the most active data set in the memory. Those kinds of caches use a very simple but efficient way to estimate the cluster-wide cache size.
All of the following calculations have a well known set of fixed variables:
GlobalCapacity
: User defined maximum cache size (cluster-wide).PartitionCount
: Number of partitions in the cluster (defaults to 271).BalancedPartitionSize
: Number of elements in a balanced partition state, BalancedPartitionSize := GlobalCapacity / PartitionCount
.Deviation
: An approximated standard deviation (tests proofed it to be pretty near), Deviation := sqrt(BalancedPartitionSize)
.A Reference Cache is typically small and the number of elements to store in the reference caches is normally known prior to creating the cache. Typical examples of reference caches are lookup tables for abbreviations or the states of a country. They tend to have a fixed but small element number and the eviction is an unlikely event, and rather undesirable behavior.
Since an imbalanced partition is a worse problem in small and mid-sized caches than in caches with millions of entries, the normal estimation rule (as discussed in a bit) is not applied to these kinds of caches. To prevent unwanted eviction on the small and mid-sized caches, Hazelcast implements a special set of rules to estimate the cluster size.
To adjust the imbalance of partitions as found in the typical runtime, the actual calculated maximum cache size (known as the eviction threshold) is slightly higher than the user defined size. That means more elements can be stored into the cache than expected by the user. This needs to be taken into account especially for large objects, since those can easily exceed the expected memory consumption!
Small caches:
If a cache is configured with no more than 4.000
elements, this cache is considered to be a small cache. The actual partition
size is derived from the number of elements (GlobalCapacity
) and the deviation using the following formula:
MaxPartitionSize := Deviation * 5 + BalancedPartitionSize
This formula ends up with big partition sizes which, summed up, exceed the expected maximum cache size (set by the user). Since the small caches typically have a well known maximum number of elements, this is not a big issue. Only if the small caches are used for a use case other than as a reference cache, this needs to be taken into account.
Mid-sized caches
A mid-sized cache is defined as a cache with a maximum number of elements that is bigger than 4.000
but not bigger than
1.000.000
elements. The calculation of mid-sized caches is similar to that of the small caches but with a different
multiplier. To calculate the maximum number of elements per partition, the following formula is used:
MaxPartitionSize := Deviation * 3 + BalancedPartitionSize
For large caches, where the maximum cache size is bigger than 1.000.000
elements, there is no additional calculation needed. The maximum
partition size is considered to be equal to BalancedPartitionSize
since statistically big partitions are expected to almost
balance themselves. Therefore, the formula is as easy as the following:
MaxPartitionSize := BalancedPartitionSize
As mentioned earlier, Hazelcast JCache provides an estimation algorithm to prevent cluster-wide network operations, concurrent access to other partitions and background tasks. It also offers a highly predictable operation runtime when the eviction is necessary.
The estimation algorithm is based on the previously calculated maximum partition size (please see the Reference Caches section and Active DataSet Caches section) and is calculated against the current partition only.
The algorithm to reckon the number of stored entries in the cache (cluster-wide) and decide if the eviction is necessary is shown in the following pseudo-code example:
RequiresEviction[Boolean] := CurrentPartitionSize >= MaxPartitionSize
Cache entries in Hazelcast are stored as partitioned across the cluster.
When you try to read a record with the key k
, if the current member is not the owner of that key (i.e. not the owner of partition that the key belongs to),
Hazelcast sends a remote operation to the owner member. Each remote operation means lots of network trips.
If your cache is used for mostly read operations, it is advised to use a near cache storage in front of the cache itself to read cache records faster and consume less network traffic.
NOTE: Near cache for JCache is only available for clients, NOT members.
However, using near cache comes with trade-offs in some cases:
Invalidation is the process of removing an entry from the near cache since the entry is not valid anymore (its value is updated or it is removed from actual cache). Near cache invalidation happens asynchronously at the cluster level, but synchronously in real-time at the current member. This means when an entry is updated (explicitly or via entry processor) or removed (deleted explicitly or via entry processor, evicted, expired), it is invalidated from all near caches asynchronously within the whole cluster but updated/removed at/from the current member synchronously. Generally, whenever the state of an entry changes in the record store by updating its value or removing it, the invalidation event is sent for that entry.
Invalidation events can be sent either individually or in batches. If there are lots of mutating operations such as put/remove on the cache, sending the events in batches is advised. This reduces the network traffic and keeps the eventing system less busy.
You can use the following system properties to configure the sending of invalidation events in batches:
hazelcast.cache.invalidation.batch.enabled
: Specifies whether the cache invalidation event batch sending is enabled or not. The default value is true
.hazelcast.cache.invalidation.batch.size
: Maximum number of cache invalidation events to be drained and sent to the event listeners in a batch. The default value is 100
.hazelcast.cache.invalidation.batchfrequency.seconds
: Cache invalidation event batch sending frequency in seconds. When event size does not reach to hazelcast.cache.invalidation.batch.size
in the given time period, those events are gathered into a batch and sent to the target. The default value is 10
seconds.So if there are many clients or many mutating operations, batching should remain enabled and the batch size should be configured with the hazelcast.cache.invalidation.batch.size
system property to a suitable value.
Expiration means the eviction of expired records. A record is expired:
<max-idle-seconds>
,<time-to-live-seconds>
passed since it is put to near-cache.Expiration is performed in two cases:
null
to caller.In the scope of near cache, eviction means evicting (clearing) the entries selected according to the given eviction-policy
when the specified max-size-policy
has been reached. Eviction is handled with max-size policy
and eviction-policy
elements. Please see Configuring JCache Near Cache.
max-size-policy
This element defines the state when the near cache is full and determines whether the eviction should be triggered. The following policies for maximum cache size are supported by the near cache eviction:
BINARY
and OBJECT
in-memory formats.NATIVE
in-memory format. This is supported only by Hazelcast Enterprise.NATIVE
in-memory format. This is supported only by Hazelcast Enterprise.NATIVE
in-memory format. This is supported only by Hazelcast Enterprise.NATIVE
in-memory format. This is supported only by Hazelcast Enterprise.eviction-policy
Once a near cache is full (i.e., has reached its maximum size as specified by the max-size-policy
element), an eviction policy determines which, if any, entries must be evicted. Currently, the following eviction policies are supported by near cache eviction:
The following are example configurations for JCache near cache.
Declarative:
<hazelcast-client>
...
<near-cache name="myCache">
<in-memory-format>BINARY</in-memory-format>
<invalidate-on-change>true</invalidate-on-change>
<cache-local-entries>false</cache-local-entries>
<time-to-live-seconds>3600000</time-to-live-seconds>
<max-idle-seconds>600000</max-idle-seconds>
<eviction size="1000" max-size-policy="ENTRY_COUNT" eviction-policy="LFU"/>
</near-cache>
...
</hazelcast-client>
Programmatic:
EvictionConfig evictionConfig = new EvictionConfig();
evictionConfig.setMaxSizePolicy(MaxSizePolicy.ENTRY_COUNT);
evictionConfig.setEvictionPolicy(EvictionPolicy.LFU);
evictionConfig.setSize(10000);
NearCacheConfig nearCacheConfig =
new NearCacheConfig()
.setName("myCache")
.setInMemoryFormat(InMemoryFormat.BINARY)
.setInvalidateOnChange(true)
.setCacheLocalEntries(false)
.setTimeToLiveSeconds(60 * 60 * 1000) // 1 hour TTL
.setMaxIdleSeconds(10 * 60 * 1000) // 10 minutes max idle seconds
.setEvictionConfig(evictionConfig);
...
clientConfig.addNearCacheConfig(nearCacheConfig);
The following are the definitions of the configuration elements and attributes.
in-memory-format
: Storage type of near cache entries. Available values are BINARY
, OBJECT
and NATIVE_MEMORY
. NATIVE_MEMORY
is available only for Hazelcast Enterprise. Default value is BINARY
.invalidate-on-change
: Specifies whether the cached entries are evicted when the entries are changed (updated or removed) on the local and global. Available values are true
and false
. Default value is true
.cache-local-entries
: Specifies whether the local cache entries are stored eagerly (immediately) to near cache when a put operation from the local is performed on the cache. Available values are true
and false
. Default value is false
.time-to-live-seconds
: Maximum number of seconds for each entry to stay in the near cache. Entries that are older than <time-to-live-seconds>
will be automatically evicted from the near cache. It can be any integer between 0
and Integer.MAX_VALUE
. 0
means infinite. Default value is 0
.max-idle-seconds
: Maximum number of seconds each entry can stay in the near cache as untouched (not-read). Entries that are not read (touched) more than <max-idle-seconds>
value will be removed from the near cache. It can be any integer between 0
and Integer.MAX_VALUE
. 0
means Integer.MAX_VALUE
. Default is 0
.eviction
: Specifies when the eviction is triggered (max-size policy
) and which eviction policy (LRU
or LFU
) is used for the entries to be evicted. The default value for max-size-policy
is ENTRY_COUNT
, default size
is 10000
and default eviction-policy
is LRU
. For High-Density Memory Store near cache, since ENTRY_COUNT
eviction policy is not supported yet, you must explicitly configure eviction with one of the supported policies:USED_NATIVE_MEMORY_SIZE
USED_NATIVE_MEMORY_PERCENTAGE
FREE_NATIVE_MEMORY_SIZE
FREE_NATIVE_MEMORY_PERCENTAGE
.Near cache can be configured only at the client side.
NOTE: Specifying a time-to-live-seconds
value is recommended in order to guarantee the eventual eviction of invalidated near cache records.
Near cache configuration can be defined at the client side (using hazelcast-client.xml
or ClientConfig
) as independent configuration (independent from the CacheConfig
). Near cache configuration lookup is handled as described below:
In addition to the operations explained in ICache Async Methods and Defining a Custom ExpiryPolicy, Hazelcast ICache also provides a set of convenience methods. These methods are not part of the JCache specification.
size()
: Returns the estimated size of the distributed cache.destroy()
: Destroys the cache and removes the data from memory. This is different from the method javax.cache.Cache::close
.getLocalCacheStatistics()
: Returns a com.hazelcast.cache.CacheStatistics
instance providing the same statistics data as the JMX beans. This method is not available yet on Hazelcast clients--the exception java.lang.UnsupportedOperationException
is thrown when you use this method on a Hazelcast client.Another feature, especially interesting for distributed environments like Hazelcast, is the JCache specified
javax.cache.processor.EntryProcessor
. For more general information, please see the Implementing EntryProcessor section.
Since Hazelcast provides backups of cached entries on other members, the default way to backup an object changed by an
EntryProcessor
is to serialize the complete object and send it to the backup partition. This can be a huge network overhead for big objects.
Hazelcast offers a sub-interface for EntryProcessor
called com.hazelcast.cache.BackupAwareEntryProcessor
. This allows you to create or pass another EntryProcessor
to run on backup
partitions and apply delta changes to the backup entries.
The backup partition EntryProcessor
can either be the currently running processor (by returning this
) or it can be
a specialized EntryProcessor
implementation (different from the currently running one) that does different operations or leaves
out operations, e.g. sending emails.
If we again take the EntryProcessor
example from the demonstration application provided in the Implementing EntryProcessor section,
the changed code will look like the following snippet.
public class UserUpdateEntryProcessor
implements BackupAwareEntryProcessor<Integer, User, User> {
@Override
public User process( MutableEntry<Integer, User> entry, Object... arguments )
throws EntryProcessorException {
// Test arguments length
if ( arguments.length < 1 ) {
throw new EntryProcessorException( "One argument needed: username" );
}
// Get first argument and test for String type
Object argument = arguments[0];
if ( !( argument instanceof String ) ) {
throw new EntryProcessorException(
"First argument has wrong type, required java.lang.String" );
}
// Retrieve the value from the MutableEntry
User user = entry.getValue();
// Retrieve the new username from the first argument
String newUsername = ( String ) arguments[0];
// Set the new username
user.setUsername( newUsername );
// Set the changed user to mark the entry as dirty
entry.setValue( user );
// Return the changed user to return it to the caller
return user;
}
public EntryProcessor<K, V, T> createBackupEntryProcessor() {
return this;
}
}
You can use the additional method BackupAwareEntryProcessor::createBackupEntryProcessor
to create or return the EntryProcessor
implementation to run on the backup partition (in the example above, the same processor again).
NOTE: For the backup runs, the returned value from the backup processor is ignored and not
returned to the user.
You can listen to CachePartitionLostEvent
instances by registering an implementation
of CachePartitionLostListener
, which is also a sub-interface of java.util.EventListener
from ICache
.
Let's consider the following example code:
public static void main(String[] args) {
CachingProvider cachingProvider = Caching.getCachingProvider();
CacheManager cacheManager = cachingProvider.getCacheManager();
Cache<Object, Object> cache = cacheManager.getCache( ... );
ICache<Object, Object> unwrappedCache = cache.unwrap( ICache.class );
unwrappedCache.addPartitionLostListener(new CachePartitionLostListener() {
@Override
public void partitionLost(CachePartitionLostEvent event) {
System.out.println(event);
}
});
}
Within this example code, a CachePartitionLostListener
implementation is registered to a cache and assumes that this cache is configured with one backup. For this particular cache and any of the partitions in the
system, if the partition owner member and its first backup member crash simultaneously, the
given CachePartitionLostListener
receives a
corresponding CachePartitionLostEvent
. If only a single member crashes in the cluster,
a CachePartitionLostEvent
is not fired for this cache since backups for the partitions
owned by the crashed member are kept on other members.
Please refer to the Partition Lost Listener section for more information about partition lost detection and partition lost events.
Split-Brain handling is internally supported as a service inside Hazelcast (see Network Partitioning for more details) and JCache
uses same infrastructure with IMap
to support Split-Brain. You can specify cache merge policy to determine which entry is used while merging. You can also provide your own cache merge policy implementations through CacheMergePolicyInterface
.
NOTE: Split-Brain is only supported for heap-based JCache but not for HD-JCache, since merging a high volume of data in consistent way may cause significant performance loss on the system.
CacheMergePolicy
InterfaceAfter split clusters are joined again, they merge their entries with each other. This merge process is handled over the CacheMergePolicy
interface.
The CacheMergePolicy
instance takes two entries: the owned entry, and the merging entry which comes from the joined cluster. The CacheEntryView
instance wraps the key, value, and some metadata about the entry (such as creation time, expiration time, and access hit). Then the CacheMergePolicy
instance selects one of the entries and returns it.
The returned entry is used as the stored cache entry.
CacheEntryView
Wraps key, value and some metadata (such as expiration time, last access time, and access hit of cache entry) and exposes them to outside as read only.
CacheMergePolicy
Policy for merging cache entries. Entries from joined clusters are merged by using this policy to select one of them from source and target.
Passed CacheEntryView
instances wrap the key and value as their original types, with conversion to object from their storage types.
If the user doesn't need the original types of key and value, you should use StorageTypeAwareCacheMergePolicy
, which is a sub-type of this interface.
StorageTypeAwareCacheMergePolicy
Marker interface indicating that the key and value wrapped by CacheEntryView
will be not converted to their original types.
The motivation of this interface is that while merging cache entries, actual key and value are not usually not checked. Therefore, there is no need to convert them to their original types.
At worst case, value is returned from the merge method as selected, meaning that in all cases, value is accessed. So even if the the conversion is done as lazy, it will be processed at this point. By default, key and value are converted to their original types unless this StorageTypeAwareCacheMergePolicy
is used.
Another motivation for using this interface is that at the member side, there is no need to locate classes of stored entries. Entries can be put from the client with BINARY
in-memory format and the classpath of the client can be different from the member. So in this case, if entries try to convert to their original types while merging, ClassNotFoundException
is thrown here.
As a result, both for performance and for the ClassNotFoundException
mentioned above, it is strongly recommended that you use this interface if the original values of key and values are not needed.
There are four built-in cache merge policies:
com.hazelcast.cache.merge.PassThroughCacheMergePolicy
or with its constant name as PASS_THROUGH
.com.hazelcast.cache.merge.PutIfAbsentCacheMergePolicy
or with its constant name as PUT_IF_ABSENT
.com.hazelcast.cache.merge.HigherHitsCacheMergePolicy
or with its constant name as HIGHER_HITS
.com.hazelcast.cache.merge.LatestAccessCacheMergePolicy
or with its constant name as LATEST_ACCESS
.You can access full class names or constant names of all built-in cache merge policies over com.hazelcast.cache.BuiltInCacheMergePolicies
enum. You can specify merge policy configuration for cache declaratively or programmatically.
The following are example configurations for JCache Split-Brain.
Declarative:
<cache name="cacheWithBuiltInMergePolicyAsConstantName">
...
<merge-policy>HIGHER_HITS</merge-policy>
...
</cache><cache name="cacheWithBuiltInMergePolicyAsFullClassName">
...
<merge-policy>com.hazelcast.cache.merge.LatestAccessCacheMergePolicy</merge-policy>
...
</cache>
<cache name="cacheWithBuiltInMergePolicyAsCustomImpl">
...
<merge-policy>com.mycompany.cache.merge.MyCacheMergePolicy</merge-policy>
...
</cache>
Programmatic:
CacheConfig cacheConfigWithBuiltInMergePolicyAsConstantName = new CacheConfig();
cacheConfig.setMergePolicy(BuiltInCacheMergePolicies.HIGGER_HITS.name());
CacheConfig cacheConfigWithBuiltInMergePolicyAsFullClassName = new CacheConfig();
cacheConfig.setMergePolicy(BuiltInCacheMergePolicies.LATEST_ACCESS.getImplementationClassName());
CacheConfig cacheConfigWithBuiltInMergePolicyAsCustomImpl = new CacheConfig();
cacheConfig.setMergePolicy("com.mycompany.cache.merge.MyCacheMergePolicy");
Hazelcast JCache is fully compliant with the JSR 107 TCK (Technology Compatibility Kit),and therefore is officially a JCache implementation.
You can test Hazelcast JCache for compliance by executing the TCK. Just perform the instructions below:
tck-parent/pom.xml
as shown below.mvn clean install
.<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<CacheInvocationContextImpl>
javax.cache.annotation.impl.cdi.CdiCacheKeyInvocationContextImpl
</CacheInvocationContextImpl>
<domain-lib-dir>${project.build.directory}/domainlib</domain-lib-dir>
<domain-jar>domain.jar</domain-jar>
<!-- ################################################################# -->
<!-- Change the following properties on the command line
to override with the coordinates for your implementation-->
<implementation-groupId>com.hazelcast</implementation-groupId>
<implementation-artifactId>hazelcast</implementation-artifactId>
<implementation-version>3.4</implementation-version>
<!-- Change the following properties to your CacheManager and
Cache implementation. Used by the unwrap tests. -->
<CacheManagerImpl>
com.hazelcast.client.cache.impl.HazelcastClientCacheManager
</CacheManagerImpl>
<CacheImpl>com.hazelcast.cache.ICache</CacheImpl>
<CacheEntryImpl>
com.hazelcast.cache.impl.CacheEntry
</CacheEntryImpl>
<!-- Change the following to point to your MBeanServer, so that
the TCK can resolve it. -->
<javax.management.builder.initial>
com.hazelcast.cache.impl.TCKMBeanServerBuilder
</javax.management.builder.initial>
<org.jsr107.tck.management.agentId>
TCKMbeanServer
</org.jsr107.tck.management.agentId>
<jsr107.api.version>1.0.0</jsr107.api.version>
<!-- ################################################################# -->
</properties>
This will run the tests using an embedded Hazelcast Member.
In this chapter, we show you how Hazelcast is integrated with Hibernate 2nd level cache and Spring, and how Hazelcast helps with your Filter, Tomcat and Jetty based web session replications.
The Hibernate Second Level Cache section tells how you should configure both Hazelcast and Hibernate to integrate them. It explains the modes of Hazelcast that can be used by Hibernate and also provides how to perform advanced settings like accessing the underlying Hazelcast instance used by Hibernate.
The Web Session Replication section tells how to cluster user HTTP sessions automatically. You will learn how to enable session replication using filter based solution. In addition, Tomcat and Jetty specific modules will be explained.
The Spring Integration section tells how you can integrate Hazelcast into a Spring project by explaining the Hazelcast instance and client configurations with the hazelcast namespace. It also lists the supported Spring bean attributes.
Hazelcast provides distributed second level cache for your Hibernate entities, collections and queries. This feature is offered as Hazelcast plugins. We have two plugin repositories for Hibernate 2nd Level Cache. Please see their own GitHub repositories at Hazelcast Hibernate (for Hibernate 3.x and 4.x) and Hazelcast Hibernate 5 (for Hibernate 5.x) for information on configuring and using them.
This section explains how you can cluster your web sessions using Servlet Filter, Tomcat, and Jetty based solutions. Each web session clustering is explained in the following subsections.
Filter based web session replication has the option to use a map with High-Density Memory Store to keep your session objects. Note that High-Density Memory Store is available in Hazelcast Enterprise HD. Please refer to the High-Density Memory Store section for details on this feature.
Sample Code: Please see our sample application for Filter Based Web Session Replication.
Assume that you have more than one web server (A, B, C) with a load balancer in front of it. If server A goes down, your users on that server will be directed to one of the live servers (B or C), but their sessions will be lost.
We need to have all these sessions backed up somewhere if we do not want to lose the sessions upon server crashes. Hazelcast Web Manager (WM) allows you to cluster user HTTP sessions automatically.
Filter Based Web Session Replication is provided as a Hazelcast plugin. Please see its own GitHub repo at Hazelcast Filter Based Web Session Replication for information on configuring and using it.
Tomcat based web session replication is offered through Hazelcast Tomcat Session Manager. It is a container specific module that enables session replication for JEE Web Applications without requiring changes to the application. Tomcat Session Manager is provided as a Hazelcast plugin. Please see its own GitHub repo at Hazelcast Tomcat Session Manager for information on configuring and using it.
Jetty based web session replication is offered through Hazelcast Jetty Session Manager. It is a container specific module that enables session replication for JEE Web Applications without requiring changes to the application. Jetty Session Manager is provided as a Hazelcast plugin. Please see its own GitHub repo at Hazelcast Jetty Session Manager for information on configuring and using it.
You can integrate Hazelcast with Spring and this chapter explains the configuration of Hazelcast within Spring context.
Sample Code: Please see our sample application for Spring Configuration.
Classpath Configuration
This configuration requires the following jar file in the classpath:
hazelcast-
<version>.jar
Bean Declaration
You can declare Hazelcast Objects using the default Spring beans namespace. Example code for a Hazelcast Instance declaration is listed below.
<bean id="instance" class="com.hazelcast.core.Hazelcast" factory-method="newHazelcastInstance">
<constructor-arg>
<bean class="com.hazelcast.config.Config">
<property name="groupConfig">
<bean class="com.hazelcast.config.GroupConfig">
<property name="name" value="dev"/>
<property name="password" value="pwd"/>
</bean>
</property>
<!-- and so on ... -->
</bean>
</constructor-arg>
</bean>
<bean id="map" factory-bean="instance" factory-method="getMap">
<constructor-arg value="map"/>
</bean>
Configuring Classpath
Hazelcast-Spring integration requires the following JAR files in the classpath:
hazelcast-spring-
<version>.jar
hazelcast-
<version>.jar
or
hazelcast-all-
<version>.jar
Declaring Beans
Hazelcast has its own namespace hazelcast for bean definitions. You can easily add the namespace declaration xmlns:hz="http://www.hazelcast.com/schema/spring" to the beans
element in the context file so that hz namespace shortcut can be used as a bean declaration.
Here is an example schema definition for Hazelcast 3.3.x:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:hz="http://www.hazelcast.com/schema/spring"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.hazelcast.com/schema/spring
http://www.hazelcast.com/schema/spring/hazelcast-spring.xsd">
<hz:hazelcast id="instance">
<hz:config>
<hz:group name="dev" password="password"/>
<hz:network port="5701" port-auto-increment="false">
<hz:join>
<hz:multicast enabled="false"
multicast-group="224.2.2.3"
multicast-port="54327"/>
<hz:tcp-ip enabled="true">
<hz:members>10.10.1.2, 10.10.1.3</hz:members>
</hz:tcp-ip>
</hz:join>
</hz:network>
<hz:map name="map"
backup-count="2"
max-size="0"
eviction-percentage="30"
read-backup-data="true"
eviction-policy="NONE"
merge-policy="com.hazelcast.map.merge.PassThroughMergePolicy"/>
</hz:config>
</hz:hazelcast>
<hz:client id="client">
<hz:group name="${cluster.group.name}" password="${cluster.group.password}" />
<hz:network connection-attempt-limit="3"
connection-attempt-period="3000"
connection-timeout="1000"
redo-operation="true"
smart-routing="true">
<hz:member>10.10.1.2:5701</hz:member>
<hz:member>10.10.1.3:5701</hz:member>
</hz:network>
</hz:client>
Hazelcast Supported Type Configurations and Examples
map
multiMap
replicatedmap
queue
topic
set
list
executorService
idGenerator
atomicLong
atomicReference
semaphore
countDownLatch
lock
<hz:map id="map" instance-ref="client" name="map" lazy-init="true" />
<hz:multiMap id="multiMap" instance-ref="instance" name="multiMap"
lazy-init="false" />
<hz:replicatedmap id="replicatedmap" instance-ref="instance"
name="replicatedmap" lazy-init="false" />
<hz:queue id="queue" instance-ref="client" name="queue"
lazy-init="true" depends-on="instance"/>
<hz:topic id="topic" instance-ref="instance" name="topic"
depends-on="instance, client"/>
<hz:set id="set" instance-ref="instance" name="set" />
<hz:list id="list" instance-ref="instance" name="list"/>
<hz:executorService id="executorService" instance-ref="client"
name="executorService"/>
<hz:idGenerator id="idGenerator" instance-ref="instance"
name="idGenerator"/>
<hz:atomicLong id="atomicLong" instance-ref="instance" name="atomicLong"/>
<hz:atomicReference id="atomicReference" instance-ref="instance"
name="atomicReference"/>
<hz:semaphore id="semaphore" instance-ref="instance" name="semaphore"/>
<hz:countDownLatch id="countDownLatch" instance-ref="instance"
name="countDownLatch"/>
<hz:lock id="lock" instance-ref="instance" name="lock"/>
Hazelcast also supports lazy-init
, scope
and depends-on
bean attributes.
<hz:hazelcast id="instance" lazy-init="true" scope="singleton">
...
</hz:hazelcast>
<hz:client id="client" scope="prototype" depends-on="instance">
...
</hz:client>
For map-store, you should set either the class-name or the implementation attribute.
<hz:config>
<hz:map name="map1">
<hz:near-cache time-to-live-seconds="0" max-idle-seconds="60"
eviction-policy="LRU" max-size="5000" invalidate-on-change="true"/>
<hz:map-store enabled="true" class-name="com.foo.DummyStore"
write-delay-seconds="0"/>
</hz:map>
<hz:map name="map2">
<hz:map-store enabled="true" implementation="dummyMapStore"
write-delay-seconds="0"/>
</hz:map>
<bean id="dummyMapStore" class="com.foo.DummyStore" />
</hz:config>
You can mark Hazelcast Distributed Objects with @SpringAware if the object wants:
ApplicationContextAware
, BeanNameAware
,InitializingBean
, @PostConstruct
.Hazelcast Distributed ExecutorService
, or more generally any Hazelcast managed object, can benefit from these features. To enable SpringAware objects, you must first configure HazelcastInstance
using hazelcast namespace as explained in Configuring Spring and add <hz:spring-aware />
tag.
<hz:spring-aware />
to Hazelcast configuration to enable @SpringAware.<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:hz="http://www.hazelcast.com/schema/spring"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.hazelcast.com/schema/spring
http://www.hazelcast.com/schema/spring/hazelcast-spring.xsd">
<context:annotation-config />
<hz:hazelcast id="instance">
<hz:config>
<hz:spring-aware />
<hz:group name="dev" password="password"/>
<hz:network port="5701" port-auto-increment="false">
<hz:join>
<hz:multicast enabled="false" />
<hz:tcp-ip enabled="true">
<hz:members>10.10.1.2, 10.10.1.3</hz:members>
</hz:tcp-ip>
</hz:join>
</hz:network>
...
</hz:config>
</hz:hazelcast>
<bean id="someBean" class="com.hazelcast.examples.spring.SomeBean"
scope="singleton" />
...
</beans>
Distributed Map SpringAware Example:
SomeValue
which contains Spring Bean definitions like ApplicationContext
and SomeBean
.@SpringAware
@Component("someValue")
@Scope("prototype")
public class SomeValue implements Serializable, ApplicationContextAware {
private transient ApplicationContext context;
private transient SomeBean someBean;
private transient boolean init = false;
public void setApplicationContext( ApplicationContext applicationContext )
throws BeansException {
context = applicationContext;
}
@Autowired
public void setSomeBean( SomeBean someBean) {
this.someBean = someBean;
}
@PostConstruct
public void init() {
someBean.doSomethingUseful();
init = true;
}
...
}
SomeValue
Object from Context and put it into Hazelcast Distributed Map on the first member.HazelcastInstance hazelcastInstance =
(HazelcastInstance) context.getBean( "hazelcast" );
SomeValue value = (SomeValue) context.getBean( "someValue" )
IMap<String, SomeValue> map = hazelcastInstance.getMap( "values" );
map.put( "key", value );
SomeValue
Object from Hazelcast Distributed Map and assert that init
method is called since it is annotated with @PostConstruct
.HazelcastInstance hazelcastInstance =
(HazelcastInstance) context.getBean( "hazelcast" );
IMap<String, SomeValue> map = hazelcastInstance.getMap( "values" );
SomeValue value = map.get( "key" );
Assert.assertTrue( value.init );
ExecutorService SpringAware Example:
ApplicationContext
, SomeBean
.@SpringAware
public class SomeTask
implements Callable<Long>, ApplicationContextAware, Serializable {
private transient ApplicationContext context;
private transient SomeBean someBean;
public Long call() throws Exception {
return someBean.value;
}
public void setApplicationContext( ApplicationContext applicationContext )
throws BeansException {
context = applicationContext;
}
@Autowired
public void setSomeBean( SomeBean someBean ) {
this.someBean = someBean;
}
}
SomeTask
to two Hazelcast Members and assert that someBean
is autowired.HazelcastInstance hazelcastInstance =
(HazelcastInstance) context.getBean( "hazelcast" );
SomeBean bean = (SomeBean) context.getBean( "someBean" );
Future<Long> f = hazelcastInstance.getExecutorService().submit(new SomeTask());
Assert.assertEquals(bean.value, f.get().longValue());
// choose a member
Member member = hazelcastInstance.getCluster().getMembers().iterator().next();
Future<Long> f2 = (Future<Long>) hazelcast.getExecutorService()
.submitToMember(new SomeTask(), member);
Assert.assertEquals(bean.value, f2.get().longValue());
NOTE: Spring managed properties/fields are marked as transient
.
Sample Code: Please see our sample application for Spring Cache.
As of version 3.1, Spring Framework provides support for adding caching into an existing Spring application. Spring 3.2 and later versions support JCache compliant caching providers. You can also use JCache caching backed by Hazelcast if your Spring version supports JCache.
<cache:annotation-driven cache-manager="cacheManager" />
<hz:hazelcast id="hazelcast">
...
</hz:hazelcast>
<bean id="cacheManager" class="com.hazelcast.spring.cache.HazelcastCacheManager">
<constructor-arg ref="instance"/>
</bean>
Hazelcast uses its Map implementation for underlying cache. You can configure a map with your cache's name if you want to set additional configuration such as ttl
.
<cache:annotation-driven cache-manager="cacheManager" />
<hz:hazelcast id="hazelcast">
<hz:config>
...
<hz:map name="city" time-to-live-seconds="0" in-memory-format="BINARY" />
</hz:hazelcast>
<bean id="cacheManager" class="com.hazelcast.spring.cache.HazelcastCacheManager">
<constructor-arg ref="instance"/>
</bean>
public interface IDummyBean {
@Cacheable("city")
String getCity();
}
<cache:annotation-driven cache-manager="cacheManager" />
<hz:hazelcast id="hazelcast">
...
</hz:hazelcast>
<hz:cache-manager id="hazelcastJCacheCacheManager" instance-ref="instance" name="hazelcastJCacheCacheManager"/>
<bean id="cacheManager" class="org.springframework.cache.jcache.JCacheCacheManager">
<constructor-arg ref="hazelcastJCacheCacheManager" />
</bean>
You can use JCache implementation in both member and client mode. A cache manager should be bound to an instance. Instance can be referenced by instance-ref
attribute or provided by hazelcast.instance.name
property which is passed to CacheManager. Instance should be specified using one of these methods.
NOTE: Instance name provided in properties overrides instance-ref
attribute.
You can specify an URI for each cache manager with uri
attribute.
<hz:cache-manager id="cacheManager2" name="cacheManager2" uri="testURI">
<hz:properties>
<hz:property name="hazelcast.instance.name">named-spring-hz-instance</hz:property>
<hz:property name="testProperty">testValue</hz:property>
</hz:properties>
</hz:cache-manager>
Annotation-Based Configuration does not require any XML definition. To perform Annotation-Based Configuration:
CachingConfiguration
class with related Annotations.@Configuration
@EnableCaching
public class CachingConfiguration implements CachingConfigurer{
@Bean
public CacheManager cacheManager() {
ClientConfig config = new ClientConfig();
HazelcastInstance client = HazelcastClient.newHazelcastClient(config);
return new HazelcastCacheManager(client);
}
@Bean
public KeyGenerator keyGenerator() {
return null;
}
CachingConfiguration
.AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
context.register(CachingConfiguration.class);
context.refresh();
For more information about Spring Cache, please see Spring Cache Abstraction.
Sample Code: Please see our sample application for Hibernate 2nd Level Cache Config.
If you are using Hibernate with Hazelcast as a second level cache provider, you can easily create RegionFactory
instances within Spring configuration (by Spring version 3.1). That way, you can use the same HazelcastInstance
as Hibernate L2 cache instance.
<hz:hibernate-region-factory id="regionFactory" instance-ref="instance"
mode="LOCAL" />
...
<bean id="sessionFactory"
class="org.springframework.orm.hibernate3.LocalSessionFactoryBean"
scope="singleton">
<property name="dataSource" ref="dataSource"/>
<property name="cacheRegionFactory" ref="regionFactory" />
...
</bean>
Hibernate RegionFactory Modes
Please refer to Hibernate Configuring RegionFactory for more information.
Sample Code: Please see our sample application for Hazelcast Transaction Manager in our code samples repository.
Starting with Hazelcast 3.7, you can get rid of the boilerplate code to begin, commit or rollback transactions by using HazelcastTransactionManager
which is a PlatformTransactionManager
implementation to be used with Spring Transaction API.
You need to register HazelcastTransactionManager
as your transaction manager implementation and also you need to
register ManagedTransactionalTaskContext
to access transactional data structures within your service class.
...
<hz:hazelcast id="instance">
...
</hz:hazelcast>
...
<tx:annotation-driven transaction-manager="transactionManager"/>
<bean id="transactionManager" class="com.hazelcast.spring.transaction.HazelcastTransactionManager">
<constructor-arg ref="instance"/>
</bean>
<bean id="transactionalContext" class="com.hazelcast.spring.transaction.ManagedTransactionalTaskContext">
<constructor-arg ref="transactionManager"/>
</bean>
<bean id="YOUR_SERVICE" class="YOUR_SERVICE_CLASS">
<property name="transactionalTaskContext" ref="transactionalContext"/>
</bean>
...
public class ServiceWithTransactionalMethod {
private TransactionalTaskContext transactionalTaskContext;
@Transactional
public void transactionalPut(String key, String value) {
transactionalTaskContext.getMap("testMap").put(key, value);
}
...
}
After marking your method as Transactional
either declaratively or by annotation and accessing the data structure
through the TransactionalTaskContext
, HazelcastTransactionManager
will begin, commit or rollback the transaction for you.
Spring tries to create a new Map
/Collection
instance and fill the new instance by iterating and converting values of the original Map
/Collection
(IMap
, IQueue
, etc.) to required types when generic type parameters of the original Map
/Collection
and the target property/attribute do not match.
Since Hazelcast Map
s/Collection
s are designed to hold very large data which a single machine cannot carry, iterating through whole values can cause out of memory errors.
To avoid this issue, the target property/attribute can be declared as un-typed Map
/Collection
as shown below.
public class SomeBean {
@Autowired
IMap map; // instead of IMap<K, V> map
@Autowired
IQueue queue; // instead of IQueue<E> queue
...
}
Or, parameters of injection methods (constructor, setter) can be un-typed as shown below.
public class SomeBean {
IMap<K, V> map;
IQueue<E> queue;
// Instead of IMap<K, V> map
public SomeBean(IMap map) {
this.map = map;
}
...
// Instead of IQueue<E> queue
public void setQueue(IQueue queue) {
this.queue = queue;
}
...
}
RELATED INFORMATION
For more information please see Spring issue-3407.
This chapter describes Hazelcast's High-Density Memory Store and Hot Restart Persistence features along with their configurations, and gives recommendations on the storage sizing.
Hazelcast Enterprise HD
By default, data structures in Hazelcast store data on heap in serialized form for highest data compaction; yet, these data structures are still subject to Java Garbage Collection (GC). Modern hardware has much more available memory. If you want to make use of that hardware and scale up by specifying higher heap sizes, GC becomes an increasing problem: the application faces long GC pauses that make the application unresponsive. Also, you may get out of memory errors if you fill your whole heap. Garbage collection, which is the automatic process that manages application’s runtime memory, often forces you into configurations where multiple JVMs with small heaps (sizes of 2-4GB per member) run on a single physical hardware device to avoid garbage collection pauses. This results in oversized clusters to hold the data and leads to performance level requirements.
In Hazelcast Enterprise HD, the High-Density Memory Store is Hazelcast’s new enterprise in-memory storage solution. It solves garbage collection limitations so that applications can exploit hardware memory more efficiently without the need of oversized clusters. High-Density Memory Store is designed as a pluggable memory manager which enables multiple memory stores for different data structures. These memory stores are all accessible by a common access layer that scales up to terabytes of the main memory on a single JVM by minimizing the GC pressure. High-Density Memory Store enables predictable application scaling and boosts performance and latency while minimizing garbage collection pauses.
This foundation includes, but is not limited to, storing keys and values next to the heap in a native memory region.
High-Density Memory Store is currently provided for the following Hazelcast features and implementations:
To use the High-Density memory storage, the native memory usage must be enabled using the programmatic or declarative configuration. Also, you can configure its size, memory allocator type, minimum block size, page size and metadata space percentage.
allocator type: Type of the memory allocator. Available values are as follows:
STANDARD: This option is used internally by Hazelcast's POOLED allocator type or for debugging/testing purposes.
malloc()
and free()
methods which are subject to contention on multithreaded/multicore systems.jemalloc()
. Note that a large memory fragmentation can trigger the Linux Out of Memory Killer if there is no swap space enabled in your system. Even if the swap space is enabled, the killer can be again triggered if there is not enough swap space left. POOLED: This is the default option, Hazelcast's own pooling memory allocator.
minimum block size: Minimum size of the blocks in bytes to split and fragment a page block to assign to an allocation request. It is used only by the POOLED memory allocator. Default value is 16.
1 << 22
= 4194304 Bytes, about 4 MB.The following is the programmatic configuration example.
MemorySize memorySize = new MemorySize(512, MemoryUnit.MEGABYTES);
NativeMemoryConfig nativeMemoryConfig =
new NativeMemoryConfig()
.setAllocatorType(NativeMemoryConfig.MemoryAllocatorType.POOLED)
.setSize(memorySize)
.setEnabled(true)
.setMinBlockSize(16)
.setPageSize(1 << 20);
The following is the declarative configuration example.
<native-memory enabled="true" allocator-type="POOLED">
<size value="512" unit="MEGABYTES"/>
</native-memory>
Data in Hazelcast is both active data and backup data for high availability, so the total memory footprint is the size of active data plus the size of backup data. If you use a single backup, it means the total memory footprint is two times the active data (active data + backup data). If you use, for example, two backups, then the total memory footprint is three times the active data (active data + backup data + backup data).
If you use only heap memory, each Hazelcast member with a 4 GB heap should accommodate a maximum of 3.5 GB of total data (active and backup). If you use the High-Density Memory Store, up to 75% of the configured physical memory footprint may be used for active and backup data, with headroom of 25% for normal memory fragmentation. In both cases, however, you should also keep some memory headroom available to handle any member failure or explicit member shutdown. When a member leaves the cluster, the data previously owned by the newly offline member will be distributed among the remaining members. For this reason, we recommend that you plan to use only 60% of available memory, with 40% headroom to handle member failure or shutdown.
Hazelcast Enterprise HD
This chapter explains the Hazelcast's Hot Restart Persistence feature, introduced with Hazelcast 3.6. Hot Restart Persistence provides fast cluster restarts by storing the states of the cluster members on the disk. This feature is currently provided for the Hazelcast map data structure and the Hazelcast JCache implementation.
Hot Restart Persistence enables you to get your cluster up and running swiftly after a cluster restart. A restart can be caused by a planned shutdown (including rolling upgrades) or a sudden cluster-wide crash (e.g. power outage). For Hot Restart Persistence, required states for Hazelcast clusters and members are introduced. Please refer to the Managing Cluster and Member States section for information on the cluster and member states.
The Hot Restart feature is supported for the following restart types:
Restart after a planned shutdown:
The cluster is shutdown completely and restarted with the exact same previous setup and data.
You can shutdown the cluster completely using the method HazelcastInstance.getCluster().shutdown()
or you can manually change the cluster state to PASSIVE
and then shut down each member one by one. When you send the command to shut the cluster down, i.e. HazelcastInstance.getCluster().shutdown()
, the members that are not in the PASSIVE
state change their states to PASSIVE
. Then, each member shuts itself down by calling the method HazelcastInstance.shutdown()
.
Rolling upgrade: The cluster is restarted intentionally member by member. For example, this could be done to install an operating system patch or new hardware.
To be able to shutdown the cluster member by member as part of a planned restart, each member in the cluster should be in the FROZEN
or PASSIVE
state. After the cluster state is changed to FROZEN
or PASSIVE
, you can manually shutdown each member by calling the method HazelcastInstance.shutdown()
. When that member is restarted, it will rejoin the running cluster. After all members are restarted, the cluster state can be changed back to ACTIVE
.
Restart after a cluster crash: The cluster is restarted after all its members crashed at the same time due to a power outage, networking interruptions, etc.
During the restart process, each member waits to load data until all the members in the partition table are started. During this process, no operations are allowed. Once all cluster members are started, Hazelcast changes the cluster state to PASSIVE
and starts to load data. When all data is loaded, Hazelcast changes the cluster state to its previous known state before shutdown and starts to accept the operations which are allowed by the restored cluster state.
If a member fails to either start, join the cluster in time (within the timeout), or load its data, then that member will be terminated immediately. After the problems causing the failure are fixed, that member can be restarted. If the cluster start cannot be completed in time, then all members will fail to start. Please refer to the Configuring Hot Restart section for defining timeouts.
In the case of a restart after a cluster crash, the Hot Restart feature realizes that it was not a clean shutdown and Hazelcast tries to restart the cluster with the last saved data following the process explained above. In some cases, specifically when the cluster crashes while it has an ongoing partition migration process, currently it is not possible to restore the last saved state.
A member can crash permanently and then be unable to recover from the failure. In that case, restart process cannot be completed since some of the members do not start or fail to load their own data. In that case, you can force the cluster to clean its persisted data and make a fresh start. This process is called force start.
You can trigger the force start process using the Management Center, REST API and cluster management scripts. Force start process is managed by the master member. Therefore, you should trigger the force start on master member.
Please refer to the Hot Restart functionality of the Management Center section to learn how you can perform a force start using the Management Center.
You can configure Hot Restart programmatically or declaratively. The configuration includes elements to enable/disable the feature, to specify the directory where the Hot Restart data will be stored, and to define timeout values.
The following are the descriptions of the Hot Restart configuration elements.
hot-restart-persistence
: The configuration that enables the Hot Restart feature. It includes the element base-dir
that is used to specify the directory where the Hot Restart data will be stored. The default value for base-dir
is hot-restart
. You can use the default value, or you can specify the value of another folder containing the Hot Restart configuration, but it is mandatory that this hot-restart
element has a value. This directory will be created automatically if it does not exist.validation-timeout-seconds
: Validation timeout for the Hot Restart process when validating the cluster members expected to join and the partition table on the whole cluster.data-load-timeout-seconds
: Data load timeout for the Hot Restart process. All members in the cluster should finish restoring their local data before this timeout.hot-restart
: The configuration that enables or disables the Hot Restart feature per data structure. This element is used for the supported data structures (in the above examples, you can see that it is included in map
and cache
). Turning on fsync
guarantees that data is persisted to the disk device when a write operation returns successful response to the caller. By default, fsync
is turned off. That means data will be persisted to the disk device eventually, instead of on every disk write. This generally provides better performance.The following are example configurations for a Hazelcast map and JCache implementation.
Declarative Configuration:
An example configuration is shown below.
<hazelcast>
...
<hot-restart-persistence enabled="true">
<base-dir>/mnt/hot-restart</base-dir>
<validation-timeout-seconds>120</validation-timeout-seconds>
<data-load-timeout-seconds>900</data-load-timeout-seconds>
</hot-restart-persistence>
...
<map>
<hot-restart enabled="true">
<fsync>false</fsync>
</hot-restart>
</map>
...
<cache>
<hot-restart enabled="true">
<fsync>false</fsync>
</hot-restart>
</cache>
...
</hazelcast>
Programmatic Configuration:
The programmatic equivalent of the above declarative configuration is shown below.
HotRestartPersistenceConfig hotRestartPersistenceConfig = new HotRestartPersistenceConfig();
hotRestartPersistenceConfig.setEnabled(true);
hotRestartPersistenceConfig.setBaseDir(new File("/mnt/hot-restart"));
hotRestartPersistenceConfig.setValidationTimeoutSeconds(120);
hotRestartPersistenceConfig.setDataLoadTimeoutSeconds(900);
config.setHotRestartPersistenceConfig(hotRestartPersistenceConfig);
...
MapConfig mapConfig = new MapConfig();
mapConfig.getHotRestartConfig().setEnabled(true);
...
CacheConfig cacheConfig = new CacheConfig();
cacheConfig.getHotRestartConfig().setEnabled(true);
Hazelcast relies on the IP address-port pair as a unique identifier for a cluster member. The member must restart with these address-port settings the same as before shutdown. Otherwise, Hot Restart fails.
Hazelcast's Hot Restart Persistence uses the log-structured storage approach. The following is a top-level design description:
This kind of design focuses almost all of the system's complexity into the garbage collection (GC) process, stripping down the client's operation to the bare necessity of guaranteeing persistent behavior: a simple file append operation. Consequently, the latency of operations is close to the theoretical minimum in almost all cases. Complications arise only during prolonged periods of maximum load; this is where the details of the GC process begin to matter.
In order to maintain the lowest possible footprint in the update operation latency, the following properties are built into the garbage collection process:
The success of this scheme is subject to a bet on the Weak Generational Garbage Hypothesis, which states that a new record entering the system is likely to become garbage soon. In other words, a key updated now is more likely than average to be updated again soon.
The scheme was taken from the seminal Sprite LFS paper, Rosenblum, Ousterhout, The Design and Implementation of a Log-Structured File System. The following is an outline of the paper:
The Cost-Benefit factor of a chunk consists of two components multiplied together:
The essence is in the second component: given equal amount of garbage in all chunks, it will make the young ones less attractive to the Collector. Assuming the generational garbage hypothesis, this will allow the young chunks to quickly accumulate more garbage. On the flip side, it will also ensure that even files with little garbage are eventually garbage collected. This removes garbage which would otherwise linger on, thinly spread across many chunk files.
Sorting records by age will group young records together in a single chunk and will do the same for older records. Therefore the chunks will either tend to keep their data live for a longer time, or quickly become full of garbage.
In this section you can find performance test summaries which are results of benchmark tests performed with a single Hazelcast member running on a physical server and on AWS R3.
The member has the following:
The tests investigate the write and read performance of Hot Restart Persistence and are performed on HP ProLiant servers with RHEL 7 operating system using Hazelcast Simulator.
The following are the specifications of the server hardware used for the test:
The following are the storage media used for the test:
The below table shows the test results.
The member has the following:
fsync
turned off. The tests investigate the write and read performance of Hot Restart Persistence and are performed on R3.2xlarge and R3.4xlarge EC2 instances using Hazelcast Simulator.
The following are the AWS storage types used for the test:
The below table shows the test results.
There are currently three ways to connect to a running Hazelcast cluster:
Native Clients enable you to perform almost all Hazelcast operations without being a member of the cluster. It connects to one of the cluster members and delegates all cluster wide operations to it (dummy client), or it connects to all of them and delegates operations smartly (smart client). When the relied cluster member dies, the client will transparently switch to another live member.
Hundreds or even thousands of clients can be connected to the cluster. By default, there are core count 10* threads on the server side that will handle all the requests (e.g. if the server has 4 cores, there will be 40 threads).
Imagine a trading application where all the trading data are stored and managed in a Hazelcast cluster with tens of members. Swing/Web applications at the traders' desktops can use Native Clients to access and modify the data in the Hazelcast cluster.
Currently, Hazelcast has Native Java, C++ and .NET Clients available. This chapter describes the Java Client.
IMPORTANT: Starting with the Hazelcast 3.6, a new Java Native Client Library is introduced in the release package: hazelcast-client-new-<version>.jar
. This library contains clients which use the new Hazelcast Binary Client Protocol. This library does not exist for the releases before 3.5. For 3.5, it can be used experimentally.
Before detailing the Java Client, this section provides the below comparison matrix to show which features are supported by the Hazelcast native clients.
Feature | Java Client | .NET Client | C++ Client |
---|---|---|---|
Map | Yes | Yes | Yes |
Queue | Yes | Yes | Yes |
Set | Yes | Yes | Yes |
List | Yes | Yes | Yes |
MultiMap | Yes | Yes | Yes |
Replicated Map | Yes | No | No |
Topic | Yes | Yes | Yes |
MapReduce | Yes | No | No |
Lock | Yes | Yes | Yes |
Semaphore | Yes | Yes | Yes |
AtomicLong | Yes | Yes | Yes |
AtomicReference | Yes | Yes | Yes |
IdGenerator | Yes | Yes | Yes |
CountDownLatch | Yes | Yes | Yes |
Transactional Map | Yes | Yes | Yes |
Transactional MultiMap | Yes | Yes | Yes |
Transactional Queue | Yes | Yes | Yes |
Transactional List | Yes | Yes | Yes |
Transactional Set | Yes | Yes | Yes |
JCache | Yes | No | No |
Ringbuffer | Yes | Yes | No |
Reliable Topic | Yes | No | No |
Hot Restart | Yes (with a near cache) | No | No |
Client Configuration Import | Yes | No | No |
Hazelcast Client Protocol | Yes | Yes | Yes |
Fail Fast on Invalid Conviguration | Yes | No | No |
Sub-Listener Interfaces for Map ListenerMap | Yes | No | No |
Continuous Query | Yes | No | No |
Listener with Predicate | Yes | Yes | Yes |
Distributed Executor Service | Yes | No | No |
Query | Yes | Yes | Yes |
Near Cache | Yes | Yes | No |
Heartbeat | Yes | Yes | Yes |
Declarative Configuration | Yes | Yes | No |
Programmatic Configuration | Yes | Yes | Yes |
SSL Support | Yes | Yes | No |
XA Transactions | Yes | No | No |
Smart Client | Yes | Yes | Yes |
Dummy Client | Yes | Yes | Yes |
Lifecycle Service | Yes | Yes | Yes |
Event Listeners | Yes | Yes | Yes |
DataSerializable | Yes | Yes | Yes |
IdentifiedDataSerializable | Yes | Yes | Yes |
Portable | Yes | Yes | Yes |
The Java client is the most full featured Hazelcast native client. It is offered both with Hazelcast and Hazelcast Enterprise. The main idea behind the Java client is to provide the same Hazelcast functionality by proxying each operation through a Hazelcast member. It can access and change distributed data, and it can listen to distributed events of an already established Hazelcast cluster from another Java application.
You should include two dependencies in your classpath to start using the Hazelcast client: hazelcast.jar
and hazelcast-client.jar
.
After adding these dependencies, you can start using the Hazelcast client as if you are using the Hazelcast API. The differences are discussed in the below sections.
If you prefer to use maven, add the following lines to your pom.xml
.
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-client</artifactId>
<version>$LATEST_VERSION$</version>
</dependency>
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast</artifactId>
<version>$LATEST_VERSION$</version>
</dependency>
The first step is configuration. You can configure the Java client declaratively or programmatically. We will use the programmatic approach throughout this tutorial. Please refer to the Java Client Declarative Configuration section for details.
ClientConfig clientConfig = new ClientConfig();
clientConfig.getGroupConfig().setName("dev").setPassword("dev-pass");
clientConfig.getNetworkConfig().addAddress("10.90.0.1", "10.90.0.2:5702");
The second step is to initialize the HazelcastInstance to be connected to the cluster.
HazelcastInstance client = HazelcastClient.newHazelcastClient(clientConfig);
This client interface is your gateway to access all Hazelcast distributed objects.
Let's create a map and populate it with some data.
IMap<String, Customer> mapCustomers = client.getMap("customers"); //creates the map proxy
mapCustomers.put("1", new Customer("Joe", "Smith"));
mapCustomers.put("2", new Customer("Ali", "Selam"));
mapCustomers.put("3", new Customer("Avi", "Noyan"));
As a final step, if you are done with your client, you can shut it down as shown below. This will release all the used resources and will close connections to the cluster.
client.shutdown();
The client has two operation modes because of the distributed nature of the data and cluster.
Smart Client: In smart mode, clients connect to each cluster member. Since each data partition uses the well known and consistent hashing algorithm, each client can send an operation to the relevant cluster member, which increases the overall throughput and efficiency. Smart mode is the default mode.
Dummy Client: For some cases, the clients can be required to connect to a single member instead of to each member in the cluster. Firewalls, security, or some custom networking issues can be the reason for these cases.
In dummy client mode, the client will only connect to one of the configured addresses. This single member will behave as a gateway to the other members. For any operation requested from the client, it will redirect the request to the relevant member and return the response back to the client returned from this member.
There are two main failure cases you should be aware of, and configurations you can perform to achieve proper behavior.
While the client is trying to connect initially to one of the members in the ClientNetworkConfig.addressList
, all the members might be not available. Instead of giving up, throwing an exception and stopping the client, the client will retry as many as connectionAttemptLimit
times.
You can configure connectionAttemptLimit
for the number of times you want the client to retry connecting. Please see Setting Connection Attempt Limit.
The client executes each operation through the already established connection to the cluster. If this connection(s) disconnects or drops, the client will try to reconnect as configured.
While sending the requests to related members, operations can fail due to various reasons. Read-only operations are retried by default. If you want to enable retry for the other operations, set the redoOperation
to true
. Please see Enabling Redo Operation.
You can set a timeout for retrying the operations sent to a member. This can be provided by using the property hazelcast.client.invocation.timeout.seconds
in ClientProperties
. The client will retry an operation within this given period, of course, if it is a read-only operation or you enabled the redoOperation
as stated in the above paragraph. This timeout value is important when there is a failure resulted by either of the following causes:
Please see the Client System Properties section.
Most of the Distributed Data Structures are supported by the Java client. When you use clients in other languages, you should check for the exceptions.
As a general rule, you configure these data structures on the server side and access them through a proxy on the client side.
You can use any Distributed Map object with the client, as shown below.
Imap<Integer, String> map = client.getMap(“myMap”);
map.put(1, “Ali”);
String value= map.get(1);
map.remove(1);
Locality is ambiguous for the client, so addLocalEntryListener
and localKeySet
are not supported. Please see the Distributed Map section for more information.
A MultiMap usage example is shown below.
MultiMap<Integer, String> multiMap = client.getMultiMap("myMultiMap");
multiMap.put(1,”ali”);
multiMap.put(1,”veli”);
Collection<String> values = multiMap.get(1);
addLocalEntryListener
, localKeySet
and getLocalMultiMapStats
are not supported because locality is ambiguous for the client. Please see the Distributed MultiMap section for more information.
A sample usage is shown below.
IQueue<String> myQueue = client.getQueue(“theQueue”);
myQueue.offer(“ali”)
getLocalQueueStats
is not supported because locality is ambiguous for the client. Please see the Distributed Queue section for more information.
getLocalTopicStats
is not supported because locality is ambiguous for the client.
The distributed data structures listed below are also supported by the client. Since their logic is the same in both the member side and client side, you can refer to their sections as listed below.
Hazelcast provides the services discussed below for some common functionalities on the client side.
The distributed executor service is for distributed computing. It can be used to execute tasks on the cluster on a designated partition or on all the partitions. It can also be used to process entries. Please see the Distributed Executor Service section for more information.
IExecutorService executorService = client.getExecutorService("default");
After getting an instance of IExecutorService
, you can use the instance as the interface with the one provided on the server side. Please see the Distributed Computing chapter for detailed usage.
NOTE: This service is only supported by the Java client.
If you need to track clients and you want to listen to their connection events, you can use the clientConnected
and clientDisconnected
methods of the ClientService
class. This class must be run on the member side. The following is an example code.
final ClientService clientService = hazelcastInstance.getClientService();
final Collection<Client> connectedClients = clientService.getConnectedClients();
clientService.addClientListener(new ClientListener() {
@Override
public void clientConnected(Client client) {
//Handle client connected event
}
@Override
public void clientDisconnected(Client client) {
//Handle client disconnected event
}
});
You use partition service to find the partition of a key. It will return all partitions. See the example code below.
PartitionService partitionService = client.getPartitionService();
//partition of a key
Partition partition = partitionService.getPartition(key);
//all partitions
Set<Partition> partitions = partitionService.getPartitions();
Lifecycle handling performs the following:
LifecycleService lifecycleService = client.getLifecycleService();
if(lifecycleService.isRunning()){
//it is running
}
//shutdown client gracefully
lifecycleService.shutdown();
You can configure listeners to listen to various event types on the client side. You can configure global events not relating to any distributed object through Client ListenerConfig. You should configure distributed object listeners like map entry listeners or list item listeners through their proxies. You can refer to the related sections under each distributed data structure in this reference manual.
Transactional distributed objects are supported on the client side. Please see the Transactions chapter on how to use them.
You can configure Hazelcast Java Client declaratively (XML) or programmatically (API).
For declarative configuration, the Hazelcast client looks at the following places for the client configuration file.
System property: The client first checks if hazelcast.client.config
system property is set to a file path, e.g. -Dhazelcast.client.config=C:/myhazelcast.xml
.
Classpath: If config file is not set as a system property, the client checks the classpath for hazelcast-client.xml
file.
If the client does not find any configuration file, it starts with the default configuration (hazelcast-client-default.xml
) located in the hazelcast-client.jar
library. Before configuring the client, please try to work with the default configuration to see if it works for you. The default should be just fine for most users. If not, then consider custom configuration for your environment.
If you want to specify your own configuration file to create a Config
object, the Hazelcast client supports the following.
Config cfg = new XmlClientConfigBuilder(xmlFileName).build();
Config cfg = new XmlClientConfigBuilder(inputStream).build();
For programmatic configuration of the Hazelcast Java Client, just instantiate a ClientConfig
object and configure the desired aspects. An example is shown below.
ClientConfig clientConfig = new ClientConfig();
clientConfig.setGroupConfig(new GroupConfig("dev","dev-pass”);
clientConfig.setLoadBalancer(yourLoadBalancer);
...
...
All network related configuration of Hazelcast Java Client is performed via the network
element in the declarative configuration file, or in the class ClientNetworkConfig
when using programmatic configuration. Let's first give the examples for these two approaches. Then we will look at its sub-elements and attributes.
Here is an example of configuring network for Java Client declaratively.
<hazelcast-client xsi:schemaLocation=
"http://www.hazelcast.com/schema/client-config hazelcast-client-config-<version>.xsd"
xmlns="http://www.hazelcast.com/schema/client-config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
...
<network>
<cluster-members>
<address>127.0.0.1</address>
<address>127.0.0.2</address>
</cluster-members>
<smart-routing>true</smart-routing>
<redo-operation>true</redo-operation>
<socket-interceptor enabled="true">
<class-name>com.hazelcast.XYZ</class-name>
<properties>
<property name="kerberos-host">kerb-host-name</property>
<property name="kerberos-config-file">kerb.conf</property>
</properties>
</socket-interceptor>
<aws enabled="true" connection-timeout-seconds="11">
<inside-aws>false</inside-aws>
<access-key>my-access-key</access-key>
<secret-key>my-secret-key</secret-key>
<iam-role>s3access</iam-role>
<region>us-west-1</region>
<host-header>ec2.amazonaws.com</host-header>
<security-group-name>hazelcast-sg</security-group-name>
<tag-key>type</tag-key>
<tag-value>hz-members</tag-value>
</aws>
</network>
Here is an example of configuring network for Java Client programmatically.
ClientConfig clientConfig = new ClientConfig();
ClientNetworkConfig networkConfig = clientConfig.getNetworkConfig();
Address List is the initial list of cluster addresses to which the client will connect. The client uses this list to find an alive member. Although it may be enough to give only one address of a member in the cluster (since all members communicate with each other), it is recommended that you give the addresses for all the members.
Declarative:
<hazelcast-client>
...
<network>
<cluster-members>
<address>10.1.1.21</address>
<address>10.1.1.22:5703</address>
</cluster-members>
...
</network>
...
</hazelcast-client>
Programmatic:
ClientConfig clientConfig = new ClientConfig();
ClientNetworkConfig networkConfig = clientConfig.getNetworkConfig();
networkConfig.addAddress("10.1.1.21", "10.1.1.22:5703");
If the port part is omitted, then 5701, 5702, and 5703 will be tried in random order.
You can provide multiple addresses with ports provided or not, as seen above. The provided list is shuffled and tried in random order. Default value is localhost.
Smart routing defines whether the client mode is smart or dummy. The following are example configurations.
Declarative:
...
<network>
...
<smart-routing>true</smart-routing>
...
</network>
...
Programmatic:
ClientConfig clientConfig = new ClientConfig();
ClientNetworkConfig networkConfig = clientConfig.getNetworkConfig();
networkConfig().setSmartRouting(true);
The default is smart client mode.
It enables/disables redo-able operations as described in Handling Retry-able Operation Failure. The following are the example configurations.
Declarative:
...
<network>
...
<redo-operation>true</redo-operation>
...
</network>
Programmatic:
ClientConfig clientConfig = new ClientConfig();
ClientNetworkConfig networkConfig = clientConfig.getNetworkConfig();
networkConfig().setRedoOperation(true);
Default is disabled.
Connection timeout is the timeout value in milliseconds for members to accept client connection requests. The following are the example configurations.
Declarative:
...
<network>
...
<connection-timeout>5000</connection-timeout>
...
</network>
Programmatic:
ClientConfig clientConfig = new ClientConfig();
clientConfig.getNetworkConfig().setConnectionTimeout(5000);
The default value is 5000 milliseconds.
While the client is trying to connect initially to one of the members in the ClientNetworkConfig.addressList
, all members might be not available. Instead of giving up, throwing an exception and stopping the client, the client will retry as many as ClientNetworkConfig.connectionAttemptLimit
times. This is also the case when an existing client-member connection goes down. The following are example configurations.
Declarative:
...
<network>
...
<connection-attempt-limit>5</connection-attempt-limit>
...
</network>
Programmatic:
ClientConfig clientConfig = new ClientConfig();
clientConfig.getNetworkConfig().setConnectionAttemptLimit(5);
Default value is 2.
Connection timeout period is the duration in milliseconds between the connection attempts defined by ClientNetworkConfig.connectionAttemptLimit
. The following are example configurations.
Declarative:
...
<network>
...
<connection-attempt-period>5000</connection-attempt-period>
...
</network>
Programmatic:
ClientConfig clientConfig = new ClientConfig();
clientConfig.getNetworkConfig().setConnectionAttemptPeriod(5000);
Default value is 3000.
Hazelcast Enterprise
Following is a client configuration to set a socket intercepter. Any class implementing com.hazelcast.nio.SocketInterceptor
is a socket interceptor.
public interface SocketInterceptor {
void init(Properties properties);
void onConnect(Socket connectedSocket) throws IOException;
}
SocketInterceptor
has two steps. First, it will be initialized by the configured properties. Second, it will be informed just after the socket is connected using onConnect
.
SocketInterceptorConfig socketInterceptorConfig = clientConfig
.getNetworkConfig().getSocketInterceptorConfig();
MyClientSocketInterceptor myClientSocketInterceptor = new MyClientSocketInterceptor();
socketInterceptorConfig.setEnabled(true);
socketInterceptorConfig.setImplementation(myClientSocketInterceptor);
If you want to configure the socket connector with a class name instead of an instance, see the example below.
SocketInterceptorConfig socketInterceptorConfig = clientConfig
.getNetworkConfig().getSocketInterceptorConfig();
MyClientSocketInterceptor myClientSocketInterceptor = new MyClientSocketInterceptor();
socketInterceptorConfig.setEnabled(true);
//These properties are provided to interceptor during init
socketInterceptorConfig.setProperty("kerberos-host","kerb-host-name");
socketInterceptorConfig.setProperty("kerberos-config-file","kerb.conf");
socketInterceptorConfig.setClassName(myClientSocketInterceptor);
RELATED INFORMATION
Please see the Socket Interceptor section for more information.
You can configure the network socket options using SocketOptions
. It has the following methods.
socketOptions.setKeepAlive(x)
: Enables/disables the SO_KEEPALIVE socket option. The default value is true
.
socketOptions.setTcpNoDelay(x)
: Enables/disables the TCP_NODELAY socket option. The default value is true
.
socketOptions.setReuseAddress(x)
: Enables/disables the SO_REUSEADDR socket option. The default value is true
.
socketOptions.setLingerSeconds(x)
: Enables/disables SO_LINGER with the specified linger time in seconds. The default value is 3
.
socketOptions.setBufferSize(x)
: Sets the SO_SNDBUF and SO_RCVBUF options to the specified value in KB for this Socket. The default value is 32
.
SocketOptions socketOptions = clientConfig.getNetworkConfig().getSocketOptions();
socketOptions.setBufferSize(32);
socketOptions.setKeepAlive(true);
socketOptions.setTcpNoDelay(true);
socketOptions.setReuseAddress(true);
socketOptions.setLingerSeconds(3);
Hazelcast Enterprise
You can use SSL to secure the connection between the client and the members. If you want SSL enabled for the client-cluster connection, you should set SSLConfig
. Once set, the connection (socket) is established out of an SSL factory defined either by a factory class name or factory implementation. Please see the SSLConfig
class in the com.hazelcast.config
package at the JavaDocs page of the Hazelcast Documentation web site.
The example declarative and programmatic configurations below show how to configure a Java client for connecting to a Hazelcast cluster in AWS.
Declarative:
...
<network>
<aws enabled="true">
<inside-aws>false</inside-aws>
<access-key>my-access-key</access-key>
<secret-key>my-secret-key</secret-key>
<iam-role>s3access</iam-role>
<region>us-west-1</region>
<host-header>ec2.amazonaws.com</host-header>
<security-group-name>hazelcast-sg</security-group-name>
<tag-key>type</tag-key>
<tag-value>hz-members</tag-value>
</aws>
...
</network>
Programmatic:
ClientConfig clientConfig = new ClientConfig();
ClientAwsConfig clientAwsConfig = new ClientAwsConfig();
clientAwsConfig.setInsideAws( false )
.setAccessKey( "my-access-key" )
.setSecretKey( "my-secret-key" )
.setRegion( "us-west-1" )
.setHostHeader( "ec2.amazonaws.com" )
.setSecurityGroupName( ">hazelcast-sg" )
.setTagKey( "type" )
.setTagValue( "hz-members" )
.setIamRole( "s3access" )
.setEnabled( true );
clientConfig.getNetworkConfig().setAwsConfig( clientAwsConfig );
HazelcastInstance client = HazelcastClient.newHazelcastClient( clientConfig );
You can refer to the aws element section for the descriptions of above AWS configuration elements except inside-aws
and iam-role
, which are explained below.
If the inside-aws
element is not set, the private addresses of cluster members will always be converted to public addresses. Also, the client will use public addresses to connect to the members. In order to use private addresses, set the inside-aws
parameter to true
. Also note that, when connecting outside from AWS, setting the inside-aws
parameter to true
will cause the client to not be able to reach the members.
IAM roles are used to make secure requests from your clients. You can provide the name of your IAM role that you created previously on your AWS console using the iam-role
or setIamRole()
method.
LoadBalancer
allows you to send operations to one of a number of endpoints (Members). Its main purpose is to determine the next Member
if queried. It is up to your implementation to use different load balancing policies. You should implement the interface com.hazelcast.client.LoadBalancer
for that purpose.
If the client is configured in smart mode, only the operations that are not key-based will be routed to the endpoint that is returned by the LoadBalancer
. If the client is not a smart client, LoadBalancer
will be ignored.
The following are example configurations.
Declarative:
<hazelcast-client>
...
<load-balancer type=“random”>
yourLoadBalancer
</load-balancer>
...
</hazelcast-client>
Programmatic:
ClientConfig clientConfig = new ClientConfig();
clientConfig.setLoadBalancer(yourLoadBalancer);
Hazelcast distributed map has a Near Cache feature to reduce network latencies. Since the client always requests data from the cluster members, it can be helpful for some of your use cases to configure a near cache on the client side. The client supports the same Near Cache that is used in Hazelcast distributed map.
You can create Near Cache on the client side by providing a configuration per map name, as shown below.
ClientConfig clientConfig = new ClientConfig();
NearCacheConfig nearCacheConfig = new NearCacheConfig();
nearCacheConfig.setName("mapName");
clientConfig.addNearCacheConfig(nearCacheConfig);
You can use wildcards for the map name, as shown below.
nearCacheConfig.setName("map*");
nearCacheConfig.setName("*map");
The following is an example declarative configuration for Near Cache.
</hazelcast-client>
...
...
<near-cache name="MENU">
<max-size>2000</max-size>
<time-to-live-seconds>0</time-to-live-seconds>
<max-idle-seconds>0</max-idle-seconds>
<eviction-policy>LFU</eviction-policy>
<invalidate-on-change>true</invalidate-on-change>
<in-memory-format>OBJECT</in-memory-format>
</near-cache>
...
</hazelcast-client>
Name of Near Cache on the client side must be the same as the name of IMap on the server for which this Near Cache is being created.
Near Cache can have its own in-memory-format
which is independent of the in-memory-format
of the servers.
Clients should provide a group name and password in order to connect to the cluster.
You can configure them using GroupConfig
, as shown below.
clientConfig.setGroupConfig(new GroupConfig("dev","dev-pass"));
In the cases where the security established with GroupConfig
is not enough and you want your clients connecting securely to the cluster, you can use ClientSecurityConfig
. This configuration has a credentials
parameter to set the IP address and UID. Please see ClientSecurityConfig.java
in our code.
For the client side serialization, use Hazelcast configuration. Please refer to the Serialization chapter.
You can configure global event listeners using ListenerConfig
as shown below.
ClientConfig clientConfig = new ClientConfig();
ListenerConfig listenerConfig = new ListenerConfig(LifecycleListenerImpl);
clientConfig.addListenerConfig(listenerConfig);
ClientConfig clientConfig = new ClientConfig();
ListenerConfig listenerConfig = new ListenerConfig("com.hazelcast.example.MembershipListenerImpl");
clientConfig.addListenerConfig(listenerConfig);
You can add three types of event listeners.
RELATED INFORMATION
Please refer to LifecycleListener, MembershipListener and DistributedObjectListener.
Hazelcast has an internal executor service (different from the data structure Executor Service) that has threads and queues to perform internal operations such as handling responses. This parameter specifies the size of the pool of threads which perform these operations laying in the executor's queue. If not configured, this parameter has the value as 5 * core size of the client (i.e. it is 20 for a machine that has 4 cores).
You can configure a custom classLoader
. It will be used by the serialization service and to load any class configured in configuration, such as event listeners or ProxyFactories.
There are some advanced client configuration properties to tune some aspects of Hazelcast Client. You can set them as property name and value pairs through declarative configuration, programmatic configuration, or JVM system property. Please see the System Properties section to learn how to set these properties.
The table below lists the client configuration properties with their descriptions.
Property Name | Default Value | Type | Description |
---|---|---|---|
hazelcast.client.event.queue.capacity |
1000000 | string | Default value of the capacity of executor that handles incoming event packets. |
hazelcast.client.event.thread.count |
5 | string | Thread count for handling incoming event packets. |
hazelcast.client.heartbeat.interval |
10000 | string | Frequency of heartbeat messages sent by the clients to members. |
hazelcast.client.heartbeat.timeout |
60000 | string | Timeout for the heartbeat messages sent by the client to members. If no messages pass between client and member within the given time via this property in milliseconds, the connection will be closed. |
hazelcast.client.max.concurrent.invocations |
Integer.MAX_VALUE | string | Maximum allowed number of concurrent invocations. You can apply a constraint on the number of concurrent invocations in order to prevent the system from overloading. If the maximum number of concurrent invocations is exceeded and a new invocation comes in, Hazelcast throws HazelcastOverloadException . |
hazelcast.client.invocation.timeout.seconds |
120 | string | Time to give up the invocation when a member in the member list is not reachable. |
hazelcast.client.shuffle.member.list |
true | string | The client shuffles the given member list to prevent all clients to connect to the same member when this property is false . When it is set to true , the client tries to connect to the members in the given order. |
Please refer to Client Code Samples.
Hazelcast Enterprise HD
If you have Hazelcast Enterprise HD, your Hazelcast Java client's near cache can benefit from the High-Density Memory Store.
Let's recall the Java client's near cache configuration (please see the Configuring Client Near Cache section) without High-Density Memory Store:
</hazelcast-client>
...
...
<near-cache name="MENU">
<max-size>2000</max-size>
<time-to-live-seconds>0</time-to-live-seconds>
<max-idle-seconds>0</max-idle-seconds>
<eviction-policy>LFU</eviction-policy>
<invalidate-on-change>true</invalidate-on-change>
<in-memory-format>OBJECT</in-memory-format>
</near-cache>
...
</hazelcast-client>
You can configure this near cache to use Hazelcast's High-Density Memory Store by setting the in-memory format to NATIVE. Please see the following configuration example:
</hazelcast-client>
...
...
<near-cache>
...
<time-to-live-seconds>0</time-to-live-seconds>
<max-idle-seconds>0</max-idle-seconds>
<invalidate-on-change>true</invalidate-on-change>
<in-memory-format>NATIVE</in-memory-format>
<eviction size="1000" max-size-policy="ENTRY_COUNT" eviction-policy="LFU"/>
...
</near-cache>
</hazelcast-client>
Please notice that when the in-memory format is NATIVE, i.e. High-Density Memory Store is enabled, the configuration element <eviction>
is used to specify the eviction behavior of your client's near cache. In this case, the elements <max-size>
and <eviction-policy>
used in the configuration of a near cache without High-Density Memory Store do not have any impact.
The element <eviction>
has the following attributes:
size
: Maximum size (entry count) of the near cache.max-size-policy
: Maximum size policy for eviction of the near cache. Available values are as follows:eviction-policy
: Eviction policy configuration. Its default values is NONE. Available values are as follows:Keep in mind that you should have already enabled the High-Density Memory Store usage for your client, using the <native-memory>
element in the client's configuration.
Please see the High-Density Memory Store section for more information on Hazelcast's High-Density Memory Store feature.
Besides Java client, Hazelcast offers the following clients and language APIs to extend the benefits of operational in-memory computing to applications:
Please refer to Feature Comparison Matrix to see the features implemented across the clients and language APIs.
Following sections describe each client and language API. Most of them will direct you to their own GitHub repositories.
You can use Native C++ Client to connect to Hazelcast cluster members and perform almost all operations that a member can perform. Clients differ from members in that clients do not hold data. The C++ Client is by default a smart client, i.e., it knows where the data is and asks directly for the correct member. You can disable this feature (using the ClientConfig::setSmart
method) if you do not want the clients to connect to every member.
The features of C++ Clients are listed below:
Please refer to C++ client's own GitHub repo at Hazelcast C++ Client for information on setting the client up, installing and compiling it, its serialization support, and APIs such as raw pointer and query.
You can use the native .NET client to connect to Hazelcast client members. You need to add HazelcastClient3x.dll
into your .NET project references. The API is very similar to the Java native client.
Please refer to .NET client's own GitHub repo at Hazelcast .NET Client for information on configuring and starting the client.
Hazelcast provides a REST interface, i.e. it provides an HTTP service in each cluster member so that you can access your map
and queue
using HTTP protocol. Assuming mapName
and queueName
are already configured in your Hazelcast, its structure is shown below.
http://member IP address:port/hazelcast/rest/maps/mapName/key
http://member IP address:port/hazelcast/rest/queues/queueName
For the operations to be performed, standard REST conventions for HTTP calls are used.
NOTE: REST client request listener service is not enabled by default. You should enable it on your cluster members to use REST client. It can be enabled using the system property hazelcast.rest.enabled
. Please refer to the System Properties section for the definition of this property and how to set a system property.
In the following GET, POST, and DELETE examples, assume that your cluster members are as shown below.
Members [5] {
Member [10.20.17.1:5701]
Member [10.20.17.2:5701]
Member [10.20.17.4:5701]
Member [10.20.17.3:5701]
Member [10.20.17.5:5701]
}
NOTE: All of the requests below can return one of the following responses in case of a failure.
HTTP/1.1 400 Bad Request
Content-Length: 0
< HTTP/1.1 500 Internal Server Error
< Content-Length: 0
You can put a new key1/value1
entry into a map by using POST call to
http://10.20.17.1:5701/hazelcast/
rest/maps/mapName/key1
URL. This call's content body should contain the value of the key. Also, if the call contains the MIME type, Hazelcast stores this information, too.
A sample POST call is shown below.
$ curl -v -X POST -H "Content-Type: text/plain" -d "bar"
http://10.20.17.1:5701/hazelcast/rest/maps/mapName/foo
It will return the following response if successful:
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 0
If you want to retrieve an entry, you can use a GET call to http://10.20.17.1:5701/hazelcast/rest/maps/mapName/key1
. You can also retrieve this entry from another member of your cluster, such as
http://10.20.17.3:5701/hazelcast/rest/
maps/mapName/key1
.
An example of a GET call is shown below.
$ curl -X GET http://10.20.17.3:5701/hazelcast/rest/maps/mapName/foo
It will return the following response if there is a corresponding value:
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 3
bar
This GET call returned a value, its length, and also the MIME type (text/plain
) since the POST call example shown above included the MIME type.
It will return the following if there is no mapping for the given key:
< HTTP/1.1 204 No Content
< Content-Length: 0
You can use a DELETE call to remove an entry. A sample DELETE call is shown below with its response.
$ curl -v -X DELETE http://10.20.17.1:5701/hazelcast/rest/maps/mapName/foo
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 0
If you leave the key empty as follows, DELETE will delete all entries from the map.
$ curl -v -X DELETE http://10.20.17.1:5701/hazelcast/rest/maps/mapName
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 0
You can use a POST call to create an item on the queue. A sample is shown below.
$ curl -v -X POST -H "Content-Type: text/plain" -d "foo"
http://10.20.17.1:5701/hazelcast/rest/queues/myEvents
The above call is equivalent to HazelcastInstance#getQueue("myEvents").offer("foo");
.
It will return the following if successful:
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 0
It will return the following if the queue is full and the item is not able to be offered to the queue:
< HTTP/1.1 503 Service Unavailable
< Content-Length: 0
You can use a DELETE call for retrieving items from a queue. Note that you should state the poll timeout while polling for queue events by an extra path parameter.
An example is shown below (10 being the timeout value).
$ curl -v -X DELETE \http://10.20.17.1:5701/hazelcast/rest/queues/myEvents/10
The above call is equivalent to HazelcastInstance#getQueue("myEvents").poll(10, SECONDS);
. Below is the response.
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 3
foo
When the timeout is reached, the response will be No Content
success, i.e. there is no item on the queue to be returned.
< HTTP/1.1 204 No Content
< Content-Length: 0
$ curl -v -X GET \http://10.20.17.1:5701/hazelcast/rest/queues/myEvents/size
The above call is equivalent to HazelcastInstance#getQueue("myEvents").size();
. Below is a sample response.
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 1
5
Besides the above operations, you can check the status of your cluster, a sample of which is shown below.
$ curl -v http://127.0.0.1:5701/hazelcast/rest/cluster
The return will be similar to the following:
< HTTP/1.1 200 OK
< Content-Length: 119
Members [5] {
Member [10.20.17.1:5701] this
Member [10.20.17.2:5701]
Member [10.20.17.4:5701]
Member [10.20.17.3:5701]
Member [10.20.17.5:5701]
}
ConnectionCount: 5
AllConnectionCount: 20
RESTful access is provided through any member of your cluster. You can even put an HTTP load-balancer in front of your cluster members for load balancing and fault tolerance.
NOTE: You need to handle the failures on REST polls as there is no transactional guarantee.
NOTE: Hazelcast Memcache Client only supports ASCII protocol. Binary Protocol is not supported.
A Memcache client written in any language can talk directly to a Hazelcast cluster. No additional configuration is required.
NOTE: Memcache client request listener service is not enabled by default. You should enable it on your cluster members to use Memcache client. It can be enabled using the system property hazelcast.memcache.enabled
. Please refer to the System Properties section for the definition of this property and how to set a system property.
Assume that your cluster members are as shown below.
Members [5] {
Member [10.20.17.1:5701]
Member [10.20.17.2:5701]
Member [10.20.17.4:5701]
Member [10.20.17.3:5701]
Member [10.20.17.5:5701]
}
Assume that you have a PHP application that uses PHP Memcache client to cache things in Hazelcast. All you need to do is have your PHP Memcache client connect to one of these members. It does not matter which member the client connects to because the Hazelcast cluster looks like one giant machine (Single System Image). Here is a PHP client code example.
<?php
$memcache = new Memcache;
$memcache->connect( '10.20.17.1', 5701 ) or die ( "Could not connect" );
$memcache->set( 'key1', 'value1', 0, 3600 );
$get_result = $memcache->get( 'key1' ); // retrieve your data
var_dump( $get_result ); // show it
?>
Notice that Memcache client connects to 10.20.17.1
and uses port 5701
. Here is a Java client code example with SpyMemcached client:
MemcachedClient client = new MemcachedClient(
AddrUtil.getAddresses( "10.20.17.1:5701 10.20.17.2:5701" ) );
client.set( "key1", 3600, "value1" );
System.out.println( client.get( "key1" ) );
If you want your data to be stored in different maps (e.g. to utilize per map configuration), you can do that with a map name prefix as in the following example code.
MemcachedClient client = new MemcachedClient(
AddrUtil.getAddresses( "10.20.17.1:5701 10.20.17.2:5701" ) );
client.set( "map1:key1", 3600, "value1" ); // store to *hz_memcache_map1
client.set( "map2:key1", 3600, "value1" ); // store to hz_memcache_map2
System.out.println( client.get( "key1" ) ); // get from hz_memcache_map1
System.out.println( client.get( "key2" ) ); // get from hz_memcache_map2
hz_memcache prefix_ separates Memcache maps from Hazelcast maps. If no map name is given, it will be stored in a default map named hz_memcache_default.
An entry written with a Memcache client can be read by another Memcache client written in another language.
CAS operations are not supported. In operations that get CAS parameters, such as append, CAS values are ignored.
Only a subset of statistics are supported. Below is the list of supported statistic values.
Hazelcast needs to serialize the Java objects that you put into Hazelcast because Hazelcast is a distributed system. The data and its replicas are stored in different partitions on multiple cluster members. The data you need may not be present on the local member, and in that case, Hazelcast retrieves that data from another member. This requires serialization.
Hazelcast serializes all your objects into an instance of com.hazelcast.nio.serialization.Data
. Data
is the binary representation of an object.
Serialization is used when:
Hazelcast optimizes the serialization for the basic types and their array types. You cannot override this behavior.
Default Types;
java.util.Date
, java.math.BigInteger
, java.math.BigDecimal
, java.lang.Class
Hazelcast optimizes all of the above object types. You do not need to worry about their (de)serializations.
For complex objects, use the following interfaces for serialization and deserialization.
java.io.Serializable
: Please see the Implementing Java Serializable and Externalizable section.
java.io.Externalizable
: Please see the Implementing Java Externalizable section.
com.hazelcast.nio.serialization.DataSerializable
: Please see the Implementing DataSerializable section.
com.hazelcast.nio.serialization.IdentifiedDataSerializable
: Please see the IdentifiedDataSerializable section.
com.hazelcast.nio.serialization.Portable
: Please see the Implementing Portable Serialization section.
Custom Serialization (using StreamSerializer and ByteArraySerializer).
Global Serializer: Please see the Global Serializer section for details.
When Hazelcast serializes an object into Data
:
(1) It first checks whether the object is null
.
(2) If the above check fails, then Hazelcast checks if it is an instance of com.hazelcast.nio.serialization.DataSerializable
or com.hazelcast.nio.serialization.IdentifiedDataSerializable
.
(3) If the above check fails, then Hazelcast checks if it is an instance of com.hazelcast.nio.serialization.Portable
.
(4) If the above check fails, then Hazelcast checks if it is an instance of one of the default types (see the Serialization chapter introduction for default types).
(5) If the above check fails, then Hazelcast looks for a user-specified Custom Serializer, i.e. an implementation of ByteArraySerializer
or StreamSerializer
. Custom serializer is searched using the input Object's Class and its parent class up to Object. If parent class search fails, all interfaces implemented by the class are also checked (excluding java.io.Serializable
and java.io.Externalizable
).
(6) If the above check fails, then Hazelcast checks if it is an instance of java.io.Serializable
or java.io.Externalizable
and a Global Serializer is not registered with Java Serialization Override feature.
(7) If the above check fails, Hazelcast will use the registered Global Serializer if one exists.
If all of the above checks fail, then serialization will fail. When a class implements multiple interfaces, the above steps are important to determine the serialization mechanism that Hazelcast will use. When a class definition is required for any of these serializations, you need to have all the classes needed by the application on your classpath because Hazelcast does not download them automatically.
The table below provides a comparison between the interfaces listed in the previous section to help you in deciding which interface to use in your applications.
Serialization Interface | Advantages | Drawbacks |
---|---|---|
Serializable | - A standard and basic Java interface - Requires no implementation |
- More time and CPU usage - More space occupancy - Not supported by Native clients |
Externalizable | - A standard Java interface - More CPU and memory usage efficient than Serializable |
- Serialization interface must be implemented - Not supported by Native clients |
DataSerializable | - More CPU and memory usage efficient than Serializable | - Specific to Hazelcast - Not supported by Native clients |
IdentifiedDataSerializable | - More CPU and memory usage efficient than Serializable - Reflection is not used during deserialization - Supported by all Native Clients |
- Specific to Hazelcast - Serialization interface must be implemented - A Factory and configuration must be implemented |
Portable | - More CPU and memory usage efficient than Serializable - Reflection is not used during deserialization - Versioning is supported Partial deserialization is supported during Queries - Supported by all Native Clients |
- Specific to Hazelcast - Serialization interface must be implemented - A Factory and configuration must be implemented - Class definition is also sent with data but stored only once per class |
Custom Serialization | - Does not require class to implement an interface - Convenient and flexible - Can be based on StreamSerializer ByteArraySerializer |
- Serialization interface must be implemented - Plug in and configuration is required |
Let's dig into the details of the above serialization mechanisms in the following sections.
A class often needs to implement the java.io.Serializable
interface; native Java serialization is the easiest way to do serialization.
Let's take a look at the example code below for Java Serializable.
public class Employee implements Serializable {
private static final long serialVersionUID = 1L;
private String surname;
public Employee( String surname ) {
this.surname = surname;
}
}
Here, the fields that are non-static and non-transient are automatically serialized. To eliminate class compatibility issues, it is recommended that you add a serialVersionUID
, as shown above. Also, when you are using methods that perform byte-content comparisons (e.g. IMap.replace()
) and if byte-content of equal objects is different, you may face unexpected behaviors. For example, if the class relies on a hash map, the replace
method may fail. The reason for this is the hash map is a serialized data structure with unreliable byte-content.
Hazelcast also supports java.io.Externalizable
. This interface offers more control on the way fields are serialized or deserialized. Compared to native Java serialization, it also can have a positive effect on performance. With java.io.Externalizable
, there is no need to add serialVersionUID
.
Let's take a look at the example code below.
public class Employee implements Externalizable {
private String surname;
public Employee(String surname) {
this.surname = surname;
}
@Override
public void readExternal( ObjectInput in )
throws IOException, ClassNotFoundException {
this.surname = in.readUTF();
}
@Override
public void writeExternal( ObjectOutput out )
throws IOException {
out.writeUTF(surname);
}
}
You explicitly perform writing and reading of fields. Perform reading in the same order as writing.
As mentioned in Implementing Java Serializable & Externalizable, Java serialization is an easy mechanism. However, it does not control how fields are serialized or deserialized. Moreover, Java serialization can lead to excessive CPU loads since it keeps track of objects to handle the cycles and streams class descriptors. These are performance decreasing factors; thus, serialized data may not have an optimal size.
The DataSerializable
interface of Hazelcast overcomes these issues. Here is an example of a class implementing the com.hazelcast.nio.serialization.DataSerializable
interface.
public class Address implements DataSerializable {
private String street;
private int zipCode;
private String city;
private String state;
public Address() {}
//getters setters..
public void writeData( ObjectDataOutput out ) throws IOException {
out.writeUTF(street);
out.writeInt(zipCode);
out.writeUTF(city);
out.writeUTF(state);
}
public void readData( ObjectDataInput in ) throws IOException {
street = in.readUTF();
zipCode = in.readInt();
city = in.readUTF();
state = in.readUTF();
}
}
Let's take a look at another example which encapsulates a DataSerializable
field.
Since the address
field itself is DataSerializable
, it calls address.writeData(out)
when writing and address.readData(in)
when reading. Also note that you should have writing and reading of the fields occur
in the same order. When Hazelcast serializes a DataSerializable
, it writes the className
first. When Hazelcast deserializes it, className
is used to instantiate the object using reflection.
public class Employee implements DataSerializable {
private String firstName;
private String lastName;
private int age;
private double salary;
private Address address; //address itself is DataSerializable
public Employee() {}
//getters setters..
public void writeData( ObjectDataOutput out ) throws IOException {
out.writeUTF(firstName);
out.writeUTF(lastName);
out.writeInt(age);
out.writeDouble (salary);
address.writeData (out);
}
public void readData( ObjectDataInput in ) throws IOException {
firstName = in.readUTF();
lastName = in.readUTF();
age = in.readInt();
salary = in.readDouble();
address = new Address();
// since Address is DataSerializable let it read its own internal state
address.readData(in);
}
}
As you can see, since the address
field itself is DataSerializable
, it calls address.writeData(out)
when writing and address.readData(in)
when reading. Also note that you should have writing and reading of the fields occur in the same order. While Hazelcast serializes a DataSerializable
, it writes the className
first. When Hazelcast deserializes it, className
is used to instantiate the object using reflection.
NOTE: Since Hazelcast needs to create an instance during deserialization,DataSerializable
class has a no-arg constructor.
NOTE: DataSerializable
is a good option if serialization is only needed for in-cluster communication.
NOTE: DataSerializable
is not supported by non-Java clients as it uses Java reflection. If you need non-Java clients, please use IdentifiedDataSerializable
or Portable
.
For a faster serialization of objects, avoiding reflection and long class names, Hazelcast recommends you implement com.hazelcast.nio.serialization.IdentifiedDataSerializable
which is a slightly better version of DataSerializable
.
DataSerializable
uses reflection to create a class instance, as mentioned in Implementing DataSerializable. But IdentifiedDataSerializable
uses a factory for this purpose and it is faster during deserialization, which requires new instance creations.
IdentifiedDataSerializable
extends DataSerializable
and introduces two new methods.
int getId();
int getFactoryId();
IdentifiedDataSerializable
uses getId()
instead of class name, and it uses getFactoryId()
to load the class when given the Id. To complete the implementation, you should also implement com.hazelcast.nio.serialization.DataSerializableFactory
and register it into SerializationConfig
, which can be accessed from Config.getSerializationConfig()
. Factory's responsibility is to return an instance of the right IdentifiedDataSerializable
object, given the Id. This is currently the most efficient way of Serialization that Hazelcast supports off the shelf.
Let's take a look at the following example code and configuration to see IdentifiedDataSerializable
in action.
public class Employee
implements IdentifiedDataSerializable {
private String surname;
public Employee() {}
public Employee( String surname ) {
this.surname = surname;
}
@Override
public void readData( ObjectDataInput in )
throws IOException {
this.surname = in.readUTF();
}
@Override
public void writeData( ObjectDataOutput out )
throws IOException {
out.writeUTF( surname );
}
@Override
public int getFactoryId() {
return EmployeeDataSerializableFactory.FACTORY_ID;
}
@Override
public int getId() {
return EmployeeDataSerializableFactory.EMPLOYEE_TYPE;
}
@Override
public String toString() {
return String.format( "Employee(surname=%s)", surname );
}
}
The methods getId
and getFactoryId
return a unique positive number within the EmployeeDataSerializableFactory
. Now, let's create an instance of this EmployeeDataSerializableFactory
.
public class EmployeeDataSerializableFactory
implements DataSerializableFactory{
public static final int FACTORY_ID = 1;
public static final int EMPLOYEE_TYPE = 1;
@Override
public IdentifiedDataSerializable create(int typeId) {
if ( typeId == EMPLOYEE_TYPE ) {
return new Employee();
} else {
return null;
}
}
}
The only method you should implement is create
, as seen in the above example. It is recommended that you use a switch-case
statement instead of multiple if-else
blocks if you have a lot of subclasses. Hazelcast throws an exception if null is returned for typeId
.
As the last step, you need to register EmployeeDataSerializableFactory
declaratively (declare in the configuration file hazelcast.xml
) as shown below. Note that factory-id
has the same value of FACTORY_ID
in the above code. This is crucial to enable Hazelcast to find the correct factory.
<hazelcast>
...
<serialization>
<data-serializable-factories>
<data-serializable-factory factory-id="1">
EmployeeDataSerializableFactory
</data-serializable-factory>
</data-serializable-factories>
</serialization>
...
</hazelcast>
RELATED INFORMATION
Please refer to the Serialization Configuration Wrap-Up section for a full description of Hazelcast Serialization configuration.
As an alternative to the existing serialization methods, Hazelcast offers a language/platform independent Portable serialization that has the following advantages:
In order to support these features, a serialized Portable object contains meta information like the version and the concrete location of the each field in the binary data. This way, Hazelcast navigates in the byte[]
and deserializes only the required field without actually deserializing the whole object. This improves the Query performance.
With multi-version support, you can have two cluster members where each has different versions of the same object. Hazelcast will store both meta information and use the correct one to serialize and deserialize Portable objects depending on the member. This is very helpful when you are doing a rolling upgrade without shutting down the cluster.
Portable serialization is totally language independent and is used as the binary protocol between Hazelcast server and clients.
Here is example code for Portable implementation of a Foo class.
public class Foo implements Portable{
final static int ID = 5;
private String foo;
public String getFoo() {
return foo;
}
public void setFoo( String foo ) {
this.foo = foo;
}
@Override
public int getFactoryId() {
return 1;
}
@Override
public int getClassId() {
return ID;
}
@Override
public void writePortable( PortableWriter writer ) throws IOException {
writer.writeUTF( "foo", foo );
}
@Override
public void readPortable( PortableReader reader ) throws IOException {
foo = reader.readUTF( "foo" );
}
}
Similar to IdentifiedDataSerializable
, a Portable Class must provide classId
and factoryId
. The Factory object creates the Portable object given the classId
.
An example Factory
could be implemented as follows:
public class MyPortableFactory implements PortableFactory {
@Override
public Portable create( int classId ) {
if ( Foo.ID == classId )
return new Foo();
else
return null;
}
}
The last step is to register the Factory
to the SerializationConfig
. Below are the programmatic and declarative configurations for this step.
Config config = new Config();
config.getSerializationConfig().addPortableFactory( 1, new MyPortableFactory() );
<hazelcast>
<serialization>
<portable-version>0</portable-version>
<portable-factories>
<portable-factory factory-id="1">
com.hazelcast.nio.serialization.MyPortableFactory
</portable-factory>
</portable-factories>
</serialization>
</hazelcast>
Note that the id
that is passed to the SerializationConfig
is the same as the factoryId
that the Foo
class returns.
More than one version of the same class may need to be serialized and deserialized. For example, a client may have an older version of a class, and the member to which it is connected may have a newer version of the same class.
Portable serialization supports versioning. It is a global versioning, meaning that all portable classes that are serialized through a member get the globally configured portable version.
You can declare Version in the configuration file hazelcast.xml
using the portable-version
element, as shown below.
<serialization>
<portable-version>1</portable-version>
<portable-factories>
<portable-factory factory-id="1">
PortableFactoryImpl
</portable-factory>
</portable-factories>
</serialization>
You can also use the interface VersionedPortable which enables to upgrade the version per class, instead of global versioning. If you need to update only one class, you can use this interface. In this case, your class should implement VersionedPortable
instead of Portable
, and you can give the desired version using the method VersionedPortable.getClassVersion()
.
You should consider the following when you perform versioning.
int
to float
).Assume that a new member joins to the cluster with a class that has been modified and class' version has been upgraded due to this modification.
put
operations will include that new field. If this new member tries to get an object that was put from the older members, it will get null
for the newly added field.null
for the objects that are put by the new member.IncompatibleClassChangeError
is generated unless the change was made on a built-in type or the byte size of the new type is less than or equal to the old one. The following are example allowed type conversions:long
-> int
, byte
, char
, short
int
-> byte
, char
, short
If you have not modify a class at all, it will work as usual.
Be careful with serializing null portables. Hazelcast lazily creates a class definition of portable internally
when the user first serializes. This class definition is stored and used later for deserializing that portable class. When
the user tries to serialize a null portable when there is no class definition at the moment, Hazelcast throws an
exception saying that com.hazelcast.nio.serialization.HazelcastSerializationException: Cannot write null portable
without explicitly registering class definition!
.
There are two solutions to get rid of this exception. Either put a non-null portable class of the same type before any other operation, or manually register a class definition in serialization configuration as shown below.
Config config = new Config();
final ClassDefinition classDefinition = new ClassDefinitionBuilder(Foo.factoryId, Foo.getClassId)
.addUTFField("foo").build();
config.getSerializationConfig().addClassDefinition(classDefinition);
Hazelcast.newHazelcastInstance(config);
Putting a DistributedObject
(Hazelcast Semaphore, Queue, etc.) in a cluster member and getting it from another one is not a straightforward operation. Passing the ID and type of the DistributedObject
can be a solution. For deserialization, you can get the object from HazelcastInstance. For instance, if your object is an instance of IQueue
, you can either use HazelcastInstance.getQueue(id)
or Hazelcast.getDistributedObject
.
You can use the HazelcastInstanceAware
interface in the case of a deserialization of a Portable DistributedObject
if it gets an ID to be looked up. HazelcastInstance is set after deserialization, so you first need to store the ID and then retrieve the DistributedObject
using the setHazelcastInstance
method.
RELATED INFORMATION
Please refer to the Serialization Configuration Wrap-Up section for a full description of Hazelcast Serialization configuration.
Hazelcast lets you plug in a custom serializer for serializing your objects. You can use StreamSerializer and ByteArraySerializer interfaces for this purpose.
You can use a stream to serialize and deserialize data by using StreamSerializer
. This is a good option for your own implementations. It can also be adapted to external serialization libraries like Kryo, JSON, and protocol buffers.
First, let's create a simple object.
public class Employee {
private String surname;
public Employee( String surname ) {
this.surname = surname;
}
}
Now, let's implement StreamSerializer for Employee
class.
public class EmployeeStreamSerializer
implements StreamSerializer<Employee> {
@Override
public int getTypeId () {
return 1;
}
@Override
public void write( ObjectDataOutput out, Employee employee )
throws IOException {
out.writeUTF(employee.getSurname());
}
@Override
public Employee read( ObjectDataInput in )
throws IOException {
String surname = in.readUTF();
return new Employee(surname);
}
@Override
public void destroy () {
}
}
In practice, classes may have many fields. Just make sure the fields are read in the same order as they are written. The type ID must be unique and greater than or equal to 1. Uniqueness of the type ID enables Hazelcast to determine which serializer will be used during deserialization.
As the last step, let's register the EmployeeStreamSerializer
in the configuration file hazelcast.xml
, as shown below.
<serialization>
<serializers>
<serializer type-class="Employee" class-name="EmployeeStreamSerializer" />
</serializers>
</serialization>
NOTE: StreamSerializer
cannot be created for well-known types (e.g. Long, String) and primitive arrays. Hazelcast already registers these types.
Let's take a look at another example implementing StreamSerializer
.
public class Foo {
private String foo;
public String getFoo() {
return foo;
}
public void setFoo( String foo ) {
this.foo = foo;
}
}
Assume that our custom serialization will serialize
Foo into XML. First you need to implement a
com.hazelcast.nio.serialization.StreamSerializer
. A very simple one that uses XMLEncoder and XMLDecoder could look like the following:
public static class FooXmlSerializer implements StreamSerializer<Foo> {
@Override
public int getTypeId() {
return 10;
}
@Override
public void write( ObjectDataOutput out, Foo object ) throws IOException {
ByteArrayOutputStream bos = new ByteArrayOutputStream();
XMLEncoder encoder = new XMLEncoder( bos );
encoder.writeObject( object );
encoder.close();
out.write( bos.toByteArray() );
}
@Override
public Foo read( ObjectDataInput in ) throws IOException {
InputStream inputStream = (InputStream) in;
XMLDecoder decoder = new XMLDecoder( inputStream );
return (Foo) decoder.readObject();
}
@Override
public void destroy() {
}
}
Note that typeId
must be unique because Hazelcast will use it to look up the StreamSerializer
while it deserializes the object. The last required step is to register the StreamSerializer
in your Hazelcast configuration. Below are the programmatic and declarative configurations for this step.
SerializerConfig sc = new SerializerConfig()
.setImplementation(new FooXmlSerializer())
.setTypeClass(Foo.class);
Config config = new Config();
config.getSerializationConfig().addSerializerConfig(sc);
<hazelcast>
<serialization>
<serializers>
<serializer type-class="com.www.Foo" class-name="com.www.FooXmlSerializer" />
</serializers>
</serialization>
</hazelcast>
From now on, this Hazelcast example will use FooXmlSerializer
to serialize Foo objects. In this way, you can write an adapter (StreamSerializer) for any Serialization framework and plug it into Hazelcast.
RELATED INFORMATION
Please refer to the Serialization Configuration Wrap-Up section for a full description of Hazelcast Serialization configuration.
ByteArraySerializer
exposes the raw ByteArray used internally by Hazelcast. It is a good option if the serialization library you are using deals with ByteArrays instead of streams.
Let's implement ByteArraySerializer
for the Employee
class mentioned in Implementing StreamSerializer.
public class EmployeeByteArraySerializer
implements ByteArraySerializer<Employee> {
@Override
public void destroy () {
}
@Override
public int getTypeId () {
return 1;
}
@Override
public byte[] write( Employee object )
throws IOException {
return object.getName().getBytes();
}
@Override
public Employee read( byte[] buffer )
throws IOException {
String surname = new String( buffer );
return new Employee( surname );
}
}
As usual, let's register the EmployeeByteArraySerializer
in the configuration file hazelcast.xml
, as shown below.
<serialization>
<serializers>
<serializer type-class="Employee">EmployeeByteArraySerializer</serializer>
</serializers>
</serialization>
RELATED INFORMATION
Please refer to the Serialization Configuration Wrap-Up section for a full description of Hazelcast Serialization configuration.
The global serializer is identical to custom serializers from the implementation perspective. The global serializer is registered as a fallback serializer to handle all other objects if a serializer cannot be located for them.
By default, the global serializer does not handle java.io.Serializable
and java.io.Externalizable
instances. However, you can configure it to be responsible for those instances.
A custom serializer should be registered for a specific class type. The global serializer will handle all class types if all the steps in searching for a serializer fail as described in Serialization Interface Types.
Use cases
Third party serialization frameworks can be integrated using the global serializer.
For your custom objects, you can implement a single serializer to handle all of them.
You can replace the internal Java serialization by enabling the overrideJavaSerialization
option of the global serializer configuration.
Any custom serializer can be used as the global serializer. Please refer to the Custom Serialization section for implementation details.
NOTE: To function properly, Hazelcast needs the Java serializable objects to be handled correctly. If the global serializer is configured to handle the Java serialization, the global serializer must properly serialize/deserialize the java.io.Serializable
instances. Otherwise, it causes Hazelcast to malfunction.
A sample global serializer that integrates with a third party serializer is shown below.
public class GlobalStreamSerializer
implements StreamSerializer<Object> {
private SomeThirdPartySerializer someThirdPartySerializer;
private init() {
//someThirdPartySerializer = ...
}
@Override
public int getTypeId () {
return 123;
}
@Override
public void write( ObjectDataOutput out, Object object ) throws IOException {
byte[] bytes = someThirdPartySerializer.encode(object);
out.writeByteArray(bytes);
}
@Override
public Object read( ObjectDataInput in ) throws IOException {
byte[] bytes = in.readByteArray();
return someThirdPartySerializer.decode(bytes);
}
@Override
public void destroy () {
someThirdPartySerializer.destroy();
}
}
Now, we can register the global serializer in the configuration file hazelcast.xml
, as shown below.
<serialization>
<serializers>
<global-serializer override-java-serialization="true">GlobalStreamSerializer</global-serializer>
</serializers>
</serialization>
You can implement the HazelcastInstanceAware
interface to access distributed objects for cases where an object is deserialized and needs access to HazelcastInstance.
Let's implement it for the Employee
class mentioned in the Custom Serialization section.
public class Employee
implements Serializable, HazelcastInstanceAware {
private static final long serialVersionUID = 1L;
private String surname;
private transient HazelcastInstance hazelcastInstance;
public Person( String surname ) {
this.surname = surname;
}
@Override
public void setHazelcastInstance( HazelcastInstance hazelcastInstance ) {
this.hazelcastInstance = hazelcastInstance;
System.out.println( "HazelcastInstance set" );
}
@Override
public String toString() {
return String.format( "Person(surname=%s)", surname );
}
}
After deserialization, the object is checked to see if it implements HazelcastInstanceAware
and the method setHazelcastInstance
is called. Notice the hazelcastInstance
is transient
. This is because this field should not be serialized.
It may be a good practice to inject a HazelcastInstance into a domain object (e.g. Employee
in the above sample) when used together with Runnable
/Callable
implementations. These runnables/callables are executed by IExecutorService
which sends them to another machine. And after a task is deserialized, run/call method implementations need to access HazelcastInstance.
We recommend you only set the HazelcastInstance field while using setHazelcastInstance
method and you not execute operations on the HazelcastInstance. The reason is that when HazelcastInstance is injected for a HazelcastInstanceAware
implementation, it may not be up and running at the injection time.
This section summarizes the configuration of serialization options, explained in the above sections, into all-in-one examples. The following are example serialization configurations.
Declarative:
<serialization>
<portable-version>2</portable-version>
<use-native-byte-order>true</use-native-byte-order>
<byte-order>BIG_ENDIAN</byte-order>
<enable-compression>true</enable-compression>
<enable-shared-object>false</enable-shared-object>
<allow-unsafe>true</allow-unsafe>
<data-serializable-factories>
<data-serializable-factory factory-id="1001">
abc.xyz.Class
</data-serializable-factory>
</data-serializable-factories>
<portable-factories>
<portable-factory factory-id="9001">
xyz.abc.Class
</portable-factory>
</portable-factories>
<serializers>
<global-serializer>abc.Class</global-serializer>
<serializer type-class="Employee" class-name="com.EmployeeSerializer">
</serializer>
</serializers>
<check-class-def-errors>true</check-class-def-errors>
</serialization>
Programmatic:
Config config = new Config();
SerializationConfig srzConfig = config.getSerializationConfig();
srzConfig.setPortableVersion( "2" ).setUseNativeByteOrder( true );
srzConfig.setAllowUnsafe( true ).setEnableCompression( true );
srzConfig.setCheckClassDefErrors( true );
GlobalSerializerConfig globSrzConfig = srzConfig.getGlobalSerializerConfig();
globSrzConfig.setClassName( "abc.Class" );
SerializerConfig serializerConfig = srzConfig.getSerializerConfig();
serializerConfig.setTypeClass( "Employee" )
.setClassName( "com.EmployeeSerializer" );
Serialization configuration has the following elements.
portable-version
: Defines versioning of the portable serialization. Portable version differentiates two of the same classes that have changes, such as adding/removing field or changing a type of a field.use-native-byte-order
: Set to true
to use native byte order for the underlying platform. byte-order
: Defines the byte order that the serialization will use: BIG_ENDIAN
or LITTLE_ENDIAN
. The default value is BIG_ENDIAN
.enable-compression
: Enables compression if default Java serialization is used. enable-shared-object
: Enables shared object if default Java serialization is used. allow-unsafe
: Set to true
to allow unsafe
to be used. data-serializable-factory
: The DataSerializableFactory class to be registered.portable-factory
: The PortableFactory class to be registered.global-serializer
: The global serializer class to be registered if no other serializer is applicable.serializer
: The class name of the serializer implementation.check-class-def-errors
: When set to true
, the serialization system will check for class definitions error at start and will throw a Serialization Exception with an error definition.This chapter provides information on managing and monitoring your Hazelcast cluster. It gives detailed instructions related to gathering statistics, monitoring via JMX protocol, and managing the cluster with useful utilities. It also explains how to use Hazelcast Management Center.
You can get various statistics from your distributed data structures via the Statistics API. Since the data structures are distributed in the cluster, the Statistics API provides statistics for the local portion (1/Number of Members in the Cluster) of data on each member.
To get local map statistics, use the getLocalMapStats()
method from the IMap
interface. This method returns a
LocalMapStats
object that holds local map statistics.
Below is example code where the getLocalMapStats()
method and the getOwnedEntryCount()
method get the number of entries owned by this member.
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
IMap<String, Customer> customers = hazelcastInstance.getMap( "customers" );
LocalMapStats mapStatistics = customers.getLocalMapStats;
System.out.println( "number of entries owned on this member = "
+ mapStatistics.getOwnedEntryCount() );
Below is the list of metrics that you can access via the LocalMapStats
object.
/**
* Returns the number of entries owned by this member.
*/
long getOwnedEntryCount();
/**
* Returns the number of backup entries hold by this member.
*/
long getBackupEntryCount();
/**
* Returns the number of backups per entry.
*/
int getBackupCount();
/**
* Returns memory cost (number of bytes) of owned entries in this member.
*/
long getOwnedEntryMemoryCost();
/**
* Returns memory cost (number of bytes) of backup entries in this member.
*/
long getBackupEntryMemoryCost();
/**
* Returns the creation time of this map on this member.
*/
long getCreationTime();
/**
* Returns the last access (read) time of the locally owned entries.
*/
long getLastAccessTime();
/**
* Returns the last update time of the locally owned entries.
*/
long getLastUpdateTime();
/**
* Returns the number of hits (reads) of the locally owned entries.
*/
long getHits();
/**
* Returns the number of currently locked locally owned keys.
*/
long getLockedEntryCount();
/**
* Returns the number of entries that the member owns and are dirty (updated
* but not persisted yet).
* dirty entry count is meaningful when there is a persistence defined.
*/
long getDirtyEntryCount();
/**
* Returns the number of put operations.
*/
long getPutOperationCount();
/**
* Returns the number of get operations.
*/
long getGetOperationCount();
/**
* Returns the number of Remove operations.
*/
long getRemoveOperationCount();
/**
* Returns the total latency of put operations. To get the average latency,
* divide by number of puts
*/
long getTotalPutLatency();
/**
* Returns the total latency of get operations. To get the average latency,
* divide by the number of gets.
*/
long getTotalGetLatency();
/**
* Returns the total latency of remove operations. To get the average latency,
* divide by the number of gets.
*/
long getTotalRemoveLatency();
/**
* Returns the maximum latency of put operations. To get the average latency,
* divide by the number of puts.
*/
long getMaxPutLatency();
/**
* Returns the maximum latency of get operations. To get the average latency,
* divide by the number of gets.
*/
long getMaxGetLatency();
/**
* Returns the maximum latency of remove operations. To get the average latency,
* divide by the number of gets.
*/
long getMaxRemoveLatency();
/**
* Returns the number of Events Received.
*/
long getEventOperationCount();
/**
* Returns the total number of Other Operations.
*/
long getOtherOperationCount();
/**
* Returns the total number of total operations.
*/
long total();
/**
* Cost of map & near cache & backup in bytes.
* todo: in object mode, object size is zero.
*/
long getHeapCost();
/**
* Returns statistics related to the Near Cache.
*/
NearCacheStats getNearCacheStats();
To get Near Cache statistics, use the getNearCacheStats()
method from the LocalMapStats
object.
This method returns a NearCacheStats
object that holds Near Cache statistics.
Below is example code where the getNearCacheStats()
method and the getRatio
method from NearCacheStats
get a Near Cache hit/miss ratio.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
IMap<String, Customer> customers = node.getMap( "customers" );
LocalMapStats mapStatistics = customers.getLocalMapStats();
NearCacheStats nearCacheStatistics = mapStatistics.getNearCacheStats();
System.out.println( "near cache hit/miss ratio= "
+ nearCacheStatistics.getRatio() );
Below is the list of metrics that you can access via the NearCacheStats
object.
This behavior applies to both client and member near caches.
/**
* Returns the creation time of this NearCache on this member
*/
long getCreationTime();
/**
* Returns the number of entries owned by this member.
*/
long getOwnedEntryCount();
/**
* Returns memory cost (number of bytes) of entries in this cache.
*/
long getOwnedEntryMemoryCost();
/**
* Returns the number of hits (reads) of the locally owned entries.
*/
long getHits();
/**
* Returns the number of misses of the locally owned entries.
*/
long getMisses();
/**
* Returns the hit/miss ratio of the locally owned entries.
*/
double getRatio();
To get MultiMap statistics, use the getLocalMultiMapStats()
method from the MultiMap
interface.
This method returns a LocalMultiMapStats
object that holds local MultiMap statistics.
Below is example code where the getLocalMultiMapStats()
method and the getLastUpdateTime
method from LocalMultiMapStats
get the last update time.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
MultiMap<String, Customer> customers = node.getMultiMap( "customers" );
LocalMultiMapStats multiMapStatistics = customers.getLocalMultiMapStats();
System.out.println( "last update time = "
+ multiMapStatistics.getLastUpdateTime() );
Below is the list of metrics that you can access via the LocalMultiMapStats
object.
/**
* Returns the number of entries owned by this member.
*/
long getOwnedEntryCount();
/**
* Returns the number of backup entries hold by this member.
*/
long getBackupEntryCount();
/**
* Returns the number of backups per entry.
*/
int getBackupCount();
/**
* Returns memory cost (number of bytes) of owned entries in this member.
*/
long getOwnedEntryMemoryCost();
/**
* Returns memory cost (number of bytes) of backup entries in this member.
*/
long getBackupEntryMemoryCost();
/**
* Returns the creation time of this map on this member.
*/
long getCreationTime();
/**
* Returns the last access (read) time of the locally owned entries.
*/
long getLastAccessTime();
/**
* Returns the last update time of the locally owned entries.
*/
long getLastUpdateTime();
/**
* Returns the number of hits (reads) of the locally owned entries.
*/
long getHits();
/**
* Returns the number of currently locked locally owned keys.
*/
long getLockedEntryCount();
/**
* Returns the number of entries that the member owns and are dirty (updated
* but not persisted yet).
* Dirty entry count is meaningful when a persistence is defined.
*/
long getDirtyEntryCount();
/**
* Returns the number of put operations.
*/
long getPutOperationCount();
/**
* Returns the number of get operations.
*/
long getGetOperationCount();
/**
* Returns the number of Remove operations.
*/
long getRemoveOperationCount();
/**
* Returns the total latency of put operations. To get the average latency,
* divide by the number of puts.
*/
long getTotalPutLatency();
/**
* Returns the total latency of get operations. To get the average latency,
* divide by the number of gets.
*/
long getTotalGetLatency();
/**
* Returns the total latency of remove operations. To get the average latency,
* divide by the number of gets.
*/
long getTotalRemoveLatency();
/**
* Returns the maximum latency of put operations. To get the average latency,
* divide by the number of puts.
*/
long getMaxPutLatency();
/**
* Returns the maximum latency of get operations. To get the average latency,
* divide by the number of gets.
*/
long getMaxGetLatency();
/**
* Returns the maximum latency of remove operations. To get the average latency,
* divide by the number of gets.
*/
long getMaxRemoveLatency();
/**
* Returns the number of Events Received.
*/
long getEventOperationCount();
/**
* Returns the total number of Other Operations.
*/
long getOtherOperationCount();
/**
* Returns the total number of total operations.
*/
long total();
/**
* Cost of map & near cache & backup in bytes.
* todo: in object mode, object size is zero.
*/
long getHeapCost();
To get local queue statistics, use the getLocalQueueStats()
method from the IQueue
interface.
This method returns a LocalQueueStats
object that holds local queue statistics.
Below is example code where the getLocalQueueStats()
method and the getAvgAge
method from LocalQueueStats
get the average age of items.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
IQueue<Order> orders = node.getQueue( "orders" );
LocalQueueStats queueStatistics = orders.getLocalQueueStats();
System.out.println( "average age of items = "
+ queueStatistics.getAvgAge() );
Below is the list of metrics that you can access via the LocalQueueStats
object.
/**
* Returns the number of owned items in this member.
*/
long getOwnedItemCount();
/**
* Returns the number of backup items in this member.
*/
long getBackupItemCount();
/**
* Returns the min age of the items in this member.
*/
long getMinAge();
/**
* Returns the max age of the items in this member.
*/
long getMaxAge();
/**
* Returns the average age of the items in this member.
*/
long getAvgAge();
/**
* Returns the number of offer/put/add operations.
* Offers returning false will be included.
* #getRejectedOfferOperationCount can be used
* to get the rejected offers.
*/
long getOfferOperationCount();
/**
* Returns the number of rejected offers. Offer
* can be rejected because of max-size limit
* on the queue.
*/
long getRejectedOfferOperationCount();
/**
* Returns the number of poll/take/remove operations.
* Polls returning null (empty) will be included.
* #getEmptyPollOperationCount can be used to get the
* number of polls returned null.
*/
long getPollOperationCount();
/**
* Returns the number of null returning poll operations.
* Poll operation might return null if the queue is empty.
*/
long getEmptyPollOperationCount();
/**
* Returns the number of other operations.
*/
long getOtherOperationsCount();
/**
* Returns the number of event operations.
*/
long getEventOperationCount();
To get local topic statistics, use the getLocalTopicStats()
method from the ITopic
interface.
This method returns a LocalTopicStats
object that holds local topic statistics.
Below is example code where the getLocalTopicStats()
method and the getPublishOperationCount
method from LocalTopicStats
get the number of publish operations.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
ITopic<Object> news = node.getTopic( "news" );
LocalTopicStats topicStatistics = news.getLocalTopicStats();
System.out.println( "number of publish operations = "
+ topicStatistics.getPublishOperationCount() );
Below is the list of metrics that you can access via the LocalTopicStats
object.
/**
* Returns the creation time of this topic on this member.
*/
long getCreationTime();
/**
* Returns the total number of published messages of this topic on this member.
*/
long getPublishOperationCount();
/**
* Returns the total number of received messages of this topic on this member.
*/
long getReceiveOperationCount();
To get local executor statistics, use the getLocalExecutorStats()
method from the IExecutorService
interface.
This method returns a LocalExecutorStats
object that holds local executor statistics.
Below is example code where the getLocalExecutorStats()
method and the getCompletedTaskCount
method from LocalExecutorStats
get the number of completed operations of the executor service.
HazelcastInstance node = Hazelcast.newHazelcastInstance();
IExecutorService orderProcessor = node.getExecutorService( "orderProcessor" );
LocalExecutorStats executorStatistics = orderProcessor.getLocalExecutorStats();
System.out.println( "completed task count = "
+ executorStatistics.getCompletedTaskCount() );
Below is the list of metrics that you can access via the LocalExecutorStats
object.
/**
* Returns the number of pending operations of the executor service.
*/
long getPendingTaskCount();
/**
* Returns the number of started operations of the executor service.
*/
long getStartedTaskCount();
/**
* Returns the number of completed operations of the executor service.
*/
long getCompletedTaskCount();
/**
* Returns the number of cancelled operations of the executor service.
*/
long getCancelledTaskCount();
/**
* Returns the total start latency of operations started.
*/
long getTotalStartLatency();
/**
* Returns the total execution time of operations finished.
*/
long getTotalExecutionLatency();
Hazelcast members expose various management beans which include statistics about distributed data structures and the states of Hazelcast member internals.
The metrics are local to the members, i.e. they do not reflect cluster wide values.
You can find the JMX API definition below with descriptions and the API methods in parenthesis.
Atomic Long (IAtomicLong
)
name
)currentValue
)set(v)
)addAndGet(v)
)compareAndSet(e,v)
)decrementAndGet()
)getAndAdd(v)
)getAndIncrement()
)getAndSet(v)
)incrementAndGet()
)partitionKey
)Atomic Reference ( IAtomicReference
)
name
)partitionKey
)Countdown Latch ( ICountDownLatch
)
name
)count
)countDown()
)partitionKey
)Executor Service ( IExecutorService
)
localPendingTaskCount
)localStartedTaskCount
)localCompletedTaskCount
)localCancelledTaskCount
)localTotalStartLatency
)localTotalExecutionLatency
)List ( IList
)
name
)clear
)Lock ( ILock
)
name
)lockObject
)partitionKey
)Map ( IMap
)
name
)size
)config
)localOwnedEntryCount
)localOwnedEntryMemoryCost
)localBackupEntryCount
)localBackupEntryMemoryCost
)localBackupCount
)localCreationTime
)localLastAccessTime
)localLastUpdateTime
)localHits
)localLockedEntryCount
)localDirtyEntryCount
)localPutOperationCount
)localGetOperationCount
)localRemoveOperationCount
)localTotalPutLatency
)localTotalGetLatency
)localTotalRemoveLatency
)localMaxPutLatency
)localMaxGetLatency
)localMaxRemoveLatency
)localEventOperationCount
)localOtherOperationCount
)localTotal
)localHeapCost
)clear()
)values(p)
)entrySet(p)
)MultiMap ( MultiMap
)
name
)size
)localOwnedEntryCount
)localOwnedEntryMemoryCost
)localBackupEntryCount
)localBackupEntryMemoryCost
)localBackupCount
)localCreationTime
)localLastAccessTime
)localLastUpdateTime
)localHits
)localLockedEntryCount
)localPutOperationCount
)localGetOperationCount
)localRemoveOperationCount
)localTotalPutLatency
)localTotalGetLatency
)localTotalRemoveLatency
)localMaxPutLatency
)localMaxGetLatency
)localMaxRemoveLatency
)localEventOperationCount
)localOtherOperationCount
)localTotal
)clear()
)Replicated Map ( ReplicatedMap
)
name
)size
)config
)localOwnedEntryCount
)localCreationTime
)localLastAccessTime
)localLastUpdateTime
)localHits
)localPutOperationCount
)localGetOperationCount
)localRemoveOperationCount
)localTotalPutLatency
)localTotalGetLatency
)localTotalRemoveLatency
)localMaxPutLatency
)localMaxGetLatency
)localMaxRemoveLatency
)localEventOperationCount
)localReplicationEventCount
)localOtherOperationCount
)localTotal
)clear()
)values()
)entrySet()
)Queue ( IQueue
)
name
)QueueConfig
)partitionKey
)localOwnedItemCount
)localBackupItemCount
)localMinAge
)localMaxAge
)localAveAge
)localOfferOperationCount
)localRejectedOfferOperationCount
)localPollOperationCount
)localEmptyPollOperationCount
)localOtherOperationsCount
)localEventOperationCount
)clear()
)Semaphore ( ISemaphore
)
name
)available
)partitionKey
)drain()
)reduce(v)
)release(v)
)Set ( ISet
)
name
)partitionKey
)clear()
)Topic ( ITopic
)
name
)config
)localCreationTime
)localPublishOperationCount
)localReceiveOperationCount
)Hazelcast Instance ( HazelcastInstance
)
name
)version
)build
)config
)configSource
)groupName
)port
)clusterTime
)memberCount
)Members
)running
)shutdown()
)HazelcastInstance.Node
)address
)masterAddress
)HazelcastInstance.EventService
)eventThreadCount
)eventQueueSize
)eventQueueCapacity
)Operation Service ( HazelcastInstance.OperationService
)
responseQueueSize
)operationExecutorQueueSize
)runningOperationsCount
)remoteOperationCount
)executedOperationCount
)operationThreadCount
)Proxy Service ( HazelcastInstance.ProxyService
)
proxyCount
)Partition Service ( HazelcastInstance.PartitionService
)
partitionCount
)activePartitionCount
)isClusterSafe
)isLocalMemberSafe
)Connection Manager ( HazelcastInstance.ConnectionManager
)
clientConnectionCount
)activeConnectionCount
)connectionCount
)Client Engine ( HazelcastInstance.ClientEngine
)
clientEndpointCount
)System Executor ( HazelcastInstance.ManagedExecutorService
)
name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
) Operation Executor ( HazelcastInstance.ManagedExecutorService
)
name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)Async Executor (HazelcastInstance.ManagedExecutorService
)
name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)Scheduled Executor ( HazelcastInstance.ManagedExecutorService
)
name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)Client Executor ( HazelcastInstance.ManagedExecutorService
)
name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)Query Executor ( HazelcastInstance.ManagedExecutorService
)
name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)IO Executor ( HazelcastInstance.ManagedExecutorService
)
name
)queueSize
)poolSize
)maximumPoolSize
)remainingQueueCapacity
)isShutdown
)isTerminated
)completedTaskCount
)You can monitor your Hazelcast members via the JMX protocol.
To achieve this, first add the following system properties to enable JMX agent:
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=\_portNo\_
(to specify JMX port, the default is 1099
) (optional)-Dcom.sun.management.jmxremote.authenticate=false
(to disable JMX auth) (optional)Then enable the Hazelcast property hazelcast.jmx
(please refer to the System Properties section) using one of the following ways:
<properties>
<property name="hazelcast.jmx">true</property>
</properties>
config.setProperty("hazelcast.jmx", "true");
<hz:properties>
<hz: property name="hazelcast.jmx">true</hz:property>
</hz:properties>
-Dhazelcast.jmx=true
Hazelcast set the naming convention for MBeans as follows:
final ObjectName mapMBeanName = new ObjectName("com.hazelcast:instance=_hzInstance_1_dev,type=IMap,name=trial");
The MBeans name consists of the Hazelcast instance name, the type of the data structure, and that data structure's name. In the above sample, _hzInstance_1_dev
is the instance name, we connect to an IMap with the name trial
.
One of the ways you can connect to JMX agent is using jconsole, jvisualvm (with MBean plugin) or another JMX compliant monitoring tool.
The other way to connect is to use a custom JMX client.
First, you need to specify the URL where the Hazelcast JMX service is running. Please see the following sample code snippet. The port
in this sample should be the one that you define while setting the JMX remote port number (if different than the default port 1099
).
// Parameters for connecting to the JMX Service
int port = 1099;
String hostname = InetAddress.getLocalHost().getHostName();
JMXServiceURL url = new JMXServiceURL("service:jmx:rmi://" + hostname + ":" + port + "/jndi/rmi://" + hostname + ":" + port + "/jmxrmi");
Then use the URL you acquired to connect to the JMX service and get the JMXConnector
object. Using this object, get the MBeanServerConnection
object. The MBeanServerConnection
object will enable you to use the MBean methods. Please see the example code below.
// Connect to the JMX Service
JMXConnector jmxc = JMXConnectorFactory.connect(url, null);
MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
Once you get the MBeanServerConnection
object, you can call the getter methods of MBeans as follows:
System.out.println("\nTotal entries on map " + mbsc.getAttribute(mapMBeanName, "name") + " : "
+ mbsc.getAttribute(mapMBeanName, "localOwnedEntryCount"));
This section provides information on programmatic utilities you can use to listen to the cluster events, to change the state of your cluster, to check whether the cluster and/or members are safe before shutting down a member, and to define the minimum number of cluster members required for the cluster to remain up and running. It also gives information on the Hazelcast Lite Member.
Hazelcast allows you to register for membership events so you will be notified when members are added or removed. You can also get the set of cluster members.
The following example code does the above: registers for member events, notified when members are added or removed, and gets the set of cluster members.
import com.hazelcast.core.*;
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance();
Cluster cluster = hazelcastInstance.getCluster();
cluster.addMembershipListener( new MembershipListener() {
public void memberAdded( MembershipEvent membershipEvent ) {
System.out.println( "MemberAdded " + membershipEvent );
}
public void memberRemoved( MembershipEvent membershipEvent ) {
System.out.println( "MemberRemoved " + membershipEvent );
}
} );
Member localMember = cluster.getLocalMember();
System.out.println ( "my inetAddress= " + localMember.getInetAddress() );
Set setMembers = cluster.getMembers();
for ( Member member : setMembers ) {
System.out.println( "isLocalMember " + member.localMember() );
System.out.println( "member.inetaddress " + member.getInetAddress() );
System.out.println( "member.port " + member.getPort() );
}
RELATED INFORMATION
Please refer to the Membership Listener section for more information on membership events.
Starting with the release of 3.6, Hazelcast introduces cluster and member states in addition to the default ACTIVE
state. This section explains these states of Hazelcast clusters and members which you can use to allow or restrict the designated cluster/member operations.
By changing the state of your cluster, you can allow/restrict several cluster operations or change the behavior of those operations. You can use the methods changeClusterState()
and shutdown()
which are in the Cluster interface to change your cluster's state.
Hazelcast clusters have the following states:
ACTIVE
: This is the default cluster state. Cluster continues to operate without restrictions.
FROZEN
: ACTIVE
. When it joins back to the cluster, it will own all previous partition assignments as it was. On the other hand, when the cluster state changes to ACTIVE
, re-partitioning starts and unassigned partitions are assigned to the active members.FROZEN
when migration/replication tasks are being performed.
PASSIVE
:ACTIVE
. map.get()
and cache.get()
, replication and cluster heartbeat tasks. PASSIVE
when migration/replication tasks are being performed.
IN_TRANSITION
: IN_TRANSITION
explicitly. NOTE: All in-cluster methods are fail-fast, i.e. when a method fails in the cluster, it throws an exception immediately (it will not be retried).
The following snippet is from the Cluster
interface showing the new methods used to manage your cluster's states.
public interface Cluster {
...
...
ClusterState getClusterState();
void changeClusterState(ClusterState newState);
void changeClusterState(ClusterState newState, TransactionOptions transactionOptions);
void shutdown();
void shutdown(TransactionOptions transactionOptions);
Please refer to the Cluster interface for information on these methods.
Hazelcast cluster members have the following states:
ACTIVE
: This is the initial member state. The member can execute and process all operations. When the state of the cluster is ACTIVE
or FROZEN
, the members are in the ACTIVE
state.
PASSIVE
: In this state, member rejects all operations EXCEPT the read-only ones, replication and migration operations, heartbeat operations, and the join operations as explained in the Cluster States section above. A member can go into this state when either of the following happens:Node.shutdown(boolean)
is called. Note that, when the shutdown process is completed, member's state changes to SHUT_DOWN
. PASSIVE
using the method changeClusterState()
.
SHUT_DOWN
: A member goes into this state when the member's shutdown process is completed. The member in this state rejects all operations and invocations. A member in this state cannot be restarted.
You can use the script cluster.sh
, which comes with the Hazelcast package, to get/change the state of your cluster, to shutdown your cluster and to force your cluster to clean its persisted data and make a fresh start. The latter is the Force Start operation of Hazelcast's Hot Restart Persistence feature. Please refer to the Force Start section.
NOTE: The script cluster.sh
uses curl
command and curl
must be installed to be able to use the script.
The script cluster.sh
needs the following parameters to operate according to your needs. If these parameters are not provided, the default values are used.
Parameter | Default Value | Description |
---|---|---|
-o or --operation |
get-state |
Executes a cluster-wide operation. Operation can be get-state , change-state , shutdown and force-start . |
-s or --state |
None | Updates the state of the cluster to a new state. New state can be active , frozen , passive . This is used with the operation change-state . This parameter has no default value; when you use this, you should provide a valid state. |
-a or --address |
127.0.0.1 |
Defines the IP address of a cluster member. If you want to manage your cluster remotely, you should use this parameter to provide the IP address of a member to this script. |
-p or --port |
5701 |
Defines on which port Hazelcast is running on the local or remote machine. The default value is 5701 . |
-g or --groupname |
dev |
Defines the name of a cluster group which is used for a simple authentication. Please see the Creating Cluster Groups section. |
-P or --password |
dev-pass |
Defines the password of a cluster group. Please see the Creating Cluster Groups section. |
The script cluster.sh
is self-documented; you can see the parameter descriptions using the command sh cluster.sh -h
or sh cluster.sh --help
.
NOTE: You can perform the above operations using the Hot Restart tab of Hazelcast Management Center or using the REST API. Please see the Hot Restart section and Using REST API for Cluster Management section.
Let's say you have a cluster running on remote machines and one Hazelcast member is running on the IP 172.16.254.1
and on the port
5702
. The group name and password of the cluster are test
and test
.
Getting the cluster state:
To get the state of the cluster, use the following command:
sh cluster.sh -o get-state -a 172.16.254.1 -p 5702 -g test -P test
The following also gets the cluster state, using the alternative parameter names (e.g. --port
instead of -p
):
sh cluster.sh --operation get-state --address 172.16.254.1 --port 5702 --groupname test --password test
Changing the cluster state:
To change the state of the cluster to frozen
, use the following command:
sh cluster.sh -o change-state -s frozen -a 172.16.254.1 -p 5702 -g test -P test
Similarly, you can use the following command for the same purpose:
sh cluster.sh --operation change-state --state frozen --address 172.16.254.1 --port 5702 --groupname test --password test
Shutting down the cluster:
To shutdown the cluster, use the following command:
sh cluster.sh -o shutdown -a 172.16.254.1 -p 5702 -g test -P test
Similarly, you can use the following command for the same purpose:
sh cluster.sh --operation shutdown --address 172.16.254.1 --port 5702 --groupname test --password test
Force starting the cluster:
To force start the cluster, use the following command:
sh cluster.sh -o force-start -a 172.16.254.1 -p 5702 -g test -P test
Similarly, you can use the following command for the same purpose:
sh cluster.sh --operation force-start --address 172.16.254.1 --port 5702 --groupname test --password test
NOTE: Currently, this script is not supported on the Windows platforms.
Besides the Management Center's Hot Restart tab and the script cluster.sh
, you can also use REST API to manage your cluster's state. The following are the commands you can use.
Getting the cluster state:
To get the state of the cluster, use the following command:
curl --data "${GROUPNAME}&${PASSWORD}" http://127.0.0.1:5701/hazelcast/rest/management/cluster/state
Changing the cluster state:
To change the state of the cluster to frozen
, use the following command:
curl --data "${GROUPNAME}&${PASSWORD}&${STATE}" http://127.0.0.1:${PORT}/hazelcast/rest/management/cluster/changeState
Shutting down the cluster:
To shutdown the cluster, use the following command:
curl --data "${GROUPNAME}&${PASSWORD}" http://127.0.0.1:${PORT}/hazelcast/rest/management/cluster/shutdown
Force starting the cluster:
To force start the cluster, use the following command:
curl --data "${GROUPNAME}&${PASSWORD}" http://127.0.0.1:${PORT}/hazelcast/rest/management/cluster/forceStart/
NOTE: You can also perform the above operations using the Hot Restart tab of Hazelcast Management Center or using the script cluster.sh
. Please see the Hot Restart section and Using the Script cluster.sh section.
Lite members are the Hazelcast cluster members that do not store data. These members are used mainly to execute tasks and register listeners, and they do not have partitions.
You can form your cluster to include the regular Hazelcast members to store data and Hazelcast lite members to run heavy computations. The presence of the lite members do not affect the operations performed on the other members in the cluster. You can directly submit your tasks to the lite members, register listeners on them and invoke operations for the Hazelcast data structures on them (e.g. map.put()
and map.get()
).
You can enable a cluster member to be a lite member using declarative or programmatic configuration.
<hazelcast>
<lite-member enabled="true">
</hazelcast>
Config config = new Config();
config.setLiteMember(true);
NOTE: Note that you cannot change a member's role at runtime.
You can define various member attributes on your Hazelcast members. You can use these member attributes to tag your members as your business logic requirements.
To define member attribute on a member, you can either:
provide MemberAttributeConfig
to your Config
object,
or provide member attributes at runtime via attribute setter methods on the Member
interface.
For example, you can tag your members with their CPU characteristics and you can route CPU intensive tasks to those CPU-rich members.
MemberAttributeConfig fourCore = new MemberAttributeConfig();
memberAttributeConfig.setIntAttribute( "CPU_CORE_COUNT", 4 );
MemberAttributeConfig twelveCore = new MemberAttributeConfig();
memberAttributeConfig.setIntAttribute( "CPU_CORE_COUNT", 12 );
MemberAttributeConfig twentyFourCore = new MemberAttributeConfig();
memberAttributeConfig.setIntAttribute( "CPU_CORE_COUNT", 24 );
Config member1Config = new Config();
config.setMemberAttributeConfig( fourCore );
Config member2Config = new Config();
config.setMemberAttributeConfig( twelveCore );
Config member3Config = new Config();
config.setMemberAttributeConfig( twentyFourCore );
HazelcastInstance member1 = Hazelcast.newHazelcastInstance( member1Config );
HazelcastInstance member2 = Hazelcast.newHazelcastInstance( member2Config );
HazelcastInstance member3 = Hazelcast.newHazelcastInstance( member3Config );
IExecutorService executorService = member1.getExecutorService( "processor" );
executorService.execute( new CPUIntensiveTask(), new MemberSelector() {
@Override
public boolean select(Member member) {
int coreCount = (int) member.getIntAttribute( "CPU_CORE_COUNT" );
// Task will be executed at either member2 or member3
if ( coreCount > 8 ) {
return true;
}
return false;
}
} );
HazelcastInstance member4 = Hazelcast.newHazelcastInstance();
// We can also set member attributes at runtime.
member4.setIntAttribute( "CPU_CORE_COUNT", 2 );
To prevent data loss when shutting down a cluster member, Hazelcast provides a graceful shutdown feature. You perform this shutdown by calling the method HazelcastInstance.shutdown()
.
Starting with Hazelcast 3.7, the master member migrates all of the replicas owned by the shutdown-requesting member to the other running (not initiated shutdown) cluster members. After these migrations are completed, the shutting down member will not be the owner or a backup of any partition anymore. It means that you can shutdown any number of Hazelcast members in a cluster concurrently with no data loss.
Please note that the process of shutting down members waits for a predefined amount of time for the master to migrate their partition replicas. You can specify this graceful shutdown timeout duration using the property hazelcast.graceful.shutdown.max.wait
. Its default value is 10 minutes. If migrations are not completed within this duration, shutdown may continue non-gracefully and lead to data loss. Therefore, you should choose your own timeout duration considering the size of data in your cluster.
With the improvements in graceful shutdown procedure in Hazelcast 3.7, the following methods are not needed to perform graceful shutdown. Nevertheless, you can use them to check the current safety status of the partitions in your cluster.
public interface PartitionService {
...
...
boolean isClusterSafe();
boolean isMemberSafe(Member member);
boolean isLocalMemberSafe();
boolean forceLocalMemberToBeSafe(long timeout, TimeUnit unit);
}
The method isClusterSafe
checks whether the cluster is in a safe state. It returns true
if there are no active partition migrations and all backups are in sync for each partition.
The method isMemberSafe
checks whether a specific member is in a safe state. It checks if all backups of partitions of the given member are in sync with the primary ones. Once it returns true
, the given member is safe and it can be shut down without data loss.
Similarly, the method isLocalMemberSafe
does the same check for the local member. The method forceLocalMemberToBeSafe
forces the owned and backup partitions to be synchronized, making the local member safe.
NOTE: If you want to use the above methods, please note that they are available starting with Hazelcast 3.3.
RELATED INFORMATION
For more code samples please refer to PartitionService Code Samples.
Hazelcast Cluster Quorum enables you to define the minimum number of machines required in a cluster for the cluster to remain in an operational state. If the number of machines is below the defined minimum at any time, the operations are rejected and the rejected operations return a QuorumException
to their callers.
When a network partitioning happens, by default Hazelcast chooses to be available. With Cluster Quorum, you can tune your Hazelcast instance towards achieving better consistency by rejecting updates that do not pass a minimum threshold. This reduces the chance of concurrent updates to an entry from two partitioned clusters. Note that the consistency defined here is the best effort, it is not full or strong consistency. To prevent mutative operations in case of a split brain syndrome, you can define a minimum quorum that must be present in the cluster.
Hazelcast initiates a quorum when a change happens on the member list.
NOTE: Currently, cluster quorum only applies to the Map, Transactional Map and Cache; support for other data structures will be added soon. Also, lock methods in the IMap interface do not participate in a quorum.
You can set up Cluster Quorum using either declarative or programmatic configuration.
Assume that you have a 5-member Hazelcast Cluster and you want to set the minimum number of 3 members for the cluster to continue operating. The following examples are configurations for this scenario.
Declarative:
<hazelcast>
....
<quorum name="quorumRuleWithThreeMembers" enabled="true">
<quorum-size>3</quorum-size>
</quorum>
<map name="default">
<quorum-ref>quorumRuleWithThreeMembers</quorum-ref>
</map>
....
</hazelcast>
Programmatic:
QuorumConfig quorumConfig = new QuorumConfig();
quorumConfig.setName("quorumRuleWithThreeMembers")
quorumConfig.setEnabled(true);
quorumConfig.setSize(3);
MapConfig mapConfig = new MapConfig();
mapConfig.setQuorumName("quorumRuleWithThreeMembers");
Config config = new Config();
config.addQuorumConfig(quorumConfig);
config.addMapConfig(mapConfig);
Quorum configuration has the following elements.
quorum-size
: Minimum number of members required in a cluster for the cluster to remain in an operational state. If the number of members is below the defined minimum at any time, the operations are rejected and the rejected operations return a QuorumException to their callers.quorum-type
: Type of the cluster quorum. Available values are READ, WRITE and READ_WRITE.You can register quorum listeners to be notified about quorum results. Quorum listeners are local to the member where they are registered, so they receive only events that occurred on that local member.
Quorum listeners can be configured via declarative or programmatic configuration. The following examples are such configurations.
Declarative:
<hazelcast>
....
<quorum name="quorumRuleWithThreeMembers" enabled="true">
<quorum-size>3</quorum-size>
<quorum-listeners>
<quorum-listener>com.company.quorum.ThreeMemberQuorumListener</quorum-listener>
</quorum-listeners>
</quorum>
<map name="default">
<quorum-ref>quorumRuleWithThreeMembers</quorum-ref>
</map>
....
</hazelcast>
Programmatic:
QuorumListenerConfig listenerConfig = new QuorumListenerConfig();
// You can either directly set quorum listener implementation of your own
listenerConfig.setImplementation(new QuorumListener() {
@Override
public void onChange(QuorumEvent quorumEvent) {
if (QuorumResult.PRESENT.equals(quorumEvent.getType())) {
// handle quorum presence
} else if (QuorumResult.ABSENT.equals(quorumEvent.getType())) {
// handle quorum absence
}
}
});
// Or you can give the name of the class that implements QuorumListener interface.
listenerConfig.setClassName("com.company.quorum.ThreeMemberQuorumListener");
QuorumConfig quorumConfig = new QuorumConfig();
quorumConfig.setName("quorumRuleWithThreeMembers")
quorumConfig.setEnabled(true);
quorumConfig.setSize(3);
quorumConfig.addListenerConfig(listenerConfig);
MapConfig mapConfig = new MapConfig();
mapConfig.setQuorumName("quorumRuleWithThreeMembers");
Config config = new Config();
config.addQuorumConfig(quorumConfig);
config.addMapConfig(mapConfig);
Quorum service gives you the ability to query quorum results over the Quorum
instances. Quorum instances let you query the quorum result of a particular quorum.
Here is a Quorum interface that you can interact with.
/**
* {@link Quorum} provides access to the current status of a quorum.
*/
public interface Quorum {
/**
* Returns true if quorum is present, false if absent.
*
* @return boolean presence of the quorum
*/
boolean isPresent();
}
You can retrieve the quorum instance for a particular quorum over the quorum service, as in the following example.
String quorumName = "at-least-one-storage-member";
QuorumConfig quorumConfig = new QuorumConfig();
quorumConfig.setName(quorumName)
quorumConfig.setEnabled(true);
MapConfig mapConfig = new MapConfig();
mapConfig.setQuorumName(quorumName);
Config config = new Config();
config.addQuorumConfig(quorumConfig);
config.addMapConfig(mapConfig);
HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(config);
QuorumService quorumService = hazelcastInstance.getQuorumService();
Quorum quorum = quorumService.getQuorum(quorumName);
boolean quorumPresence = quorum.isPresent();
Hazelcast CLI is a command line management tool where you can manage your cluster and lifecycle of Hazelcast members. Hazelcast CLI is provided as a Hazelcast plugin. Please see its own GitHub repo at Hazelcast CLI for information on installing and using it.
Hazelcast Management Center enables you to monitor and manage your cluster members running Hazelcast. In addition to monitoring the overall state of your clusters, you can also analyze and browse your data structures in detail, update map configurations and take thread dumps from members. Uou can run scripts (JavaScript, Groovy, etc.) and commands on your members with its scripting and console modules.
You have two options for installing Hazelcast Management Center:
mancenter
-version.war
on your Java application server/container.mancenter
application before they start.Here are the steps.
mancenter
-version.war
file under the directory mancenter
.mancenter
-version.war
file from the command line. The following command will start Hazelcast Management Center on port 8080 with context root 'mancenter' (http://localhost:8080/mancenter
).java -jar mancenter-*version*.war 8080 mancenter
startManCenter.bat
or startManCenter.sh
located in the directory mancenter
.http://localhost:8080/mancenter
.http://localhost:8080/mancenter
is up.hazelcast.xml
. Hazelcast members will send their states to this URL.<management-center enabled="true">
http://localhost:8080/mancenter
</management-center>
mancenter-*version*.war
in your already-SSL-enabled web container, configure hazelcast.xml
as follows.<management-center enabled="true">
https://localhost:sslPortNumber/mancenter
</management-center>
If you are using an untrusted certificate for your container, which you created yourself, you need to add that certificate to your JVM first. Download the certificate from the browser, after this you can add it to JVM as follows.
keytool -import -noprompt -trustcacerts -alias <AliasName> -file <certificateFile> -keystore $JAVA_HOME/jre/lib/security/cacerts -storepass <Password>
update-interval
as shown below. update-interval
is optional and its default value is 3 seconds.<management-center enabled="true" update-interval="3">http://localhost:8080/
mancenter</management-center>
http://localhost:8080/mancenter
and setup your administrator account explained in the next section.If you have the open source edition of Hazelcast, Management Center can be used for at most 2 members in the cluster. To use it for more members, you need to have either a Management Center license, Hazelcast Enterprise license or Hazelcast Enterprise HD license. This license should be entered within the Management Center as described in the following paragraphs.
NOTE: Even if you have a Hazelcast Enterprise or Enterprise HD license key and you set it as explained in the Setting the License Key section, you still need to enter this same license within the Management Center. Please see the following paragraphs to learn how you can enter your license.
Once you browse to http://localhost:8080/mancenter
and since you are going to use Management Center for the first time, the following dialog box appears.
NOTE: If you already created an administrator account before, a login dialog box appears instead.
It asks you to create a username and password and give a valid e-mail address of yours. Once you press the Sign Up button, your administrator account credentials are created and the following dialog box appears.
"Select Cluster to Connect" dialog box lists the clusters that send statistics to Management Center. You can either select a cluster to connect using the Connect button or enter your Management Center license key using the Enter License button. Management Center can be used without a license if the cluster that you want to monitor has at most 2 members.
If you have a Management Center license or Hazelcast Enterprise license, you can enter it in the dialog box that appears once you press the Enter License button, as shown below.
When you try to connect to a cluster that has more than 2 members without entering a license key or if your license key is expired, the following dialog box appears.
Here, you can either choose to connect to a cluster without providing a license key or to enter your license key. If you choose to continue without a license, Management Center still continues to function but you will only be able to monitor up to 2 members of your cluster.
Management Center creates a folder with the name mancenter
under your user/home
folder to save data files and above settings/license information. You can change the data folder by setting the hazelcast.mancenter.home
system property. Please see the System Properties section to see the description of this property and to learn how to set a system property.
RELATED INFORMATION
Please refer to the Management Center Configuration section for a full description of Hazelcast Management Center configuration.
Once the page is loaded after selecting a cluster, the tool's home page appears as shown below.
This page provides the fundamental properties of the selected cluster which are explained in the Home Page section. The page has a toolbar on the top and a menu on the left.
The toolbar has the following buttons:
Cluster Selector: Switches between clusters. When the mouse is moved onto this item, a drop down list of clusters appears.
The user can select any cluster and once selected, the page immediately loads with the selected cluster's information.
NOTE: Some of the above listed toolbar items are not visible to users who are not admin or who have read-only permission. Also, some of the operations explained in the later sections cannot be performed by users with read-only permission. Please see the Administering Management Center section for details.
The Home Page includes a menu on the left which lists the distributed data structures in the cluster and all the cluster members, as shown below.
NOTE: Distributed data structures will be shown there when the proxies are created for them.
NOTE: WAN Replication tab is only visible with Hazelcast Enterprise license.
You can expand and collapse menu items by clicking on them. Below is the list of menu items with links to their explanations.
Each time you select an item from the toolbar or menu, the item is added to the main view as a tab, as shown below.
In the above example, Home, Scripting, Console, queue1 and map1 windows can be seen as tabs. Windows can be closed using the icon on each tab (except the Home Page; it cannot be closed).
This is the first page appearing after logging in. It gives an overview of the connected cluster. The following subsections describe each portion of the page.
This part of the page provides load and utilization information for the CPUs for each cluster member, as shown below.
The first column lists the members with their IPs and ports. The next columns list the system load averages on each member for the last 1, 5 and 15 minutes. These average values are calculated as the sum of the count of runnable entities running on and queued to the available CPUs ave