Hazelcast IMDG

Hazelcast IMDG Release Notes

Welcome to the Hazelcast IMDG Release Notes. This document includes the new features, enhancements and fixed issues for Hazelcast IMDG releases. The linked numbers refer to the issue number in the Hazelcast IMDG GitHub repository.

Please select a release from the Table of Contents on the left panel to see its release notes.

New feature and enhancement listings are grouped by Hazelcast IMDG editions, i.e., open source and enterprise. The purpose of this is to separate the ones that belong to the enterprise edition. The open source ones are already included in the enterprise edition.

4.2.8

Enhancements

  • Updated the Jackson core dependency version to 2.15.1. 24638

4.2.7

Fixes

  • Fixed an issue where the JSON Query results were inconsistent, using the data stored as JSON via HazelcastJsonValue. #23454

  • Fixed an issue where the WAN replicated entries were expiring on the passive cluster when the entries are accessed on the active cluster. #22294

4.2.6

Enhancements

  • Upgraded jackson-databind to 2.14.0. #22391

Fixes

  • Fixed an issue where replication over WAN was failing on the source cluster members, when there are multiple batch publishers configured in a single WAN replication. #22496

  • Fixed a memory leak due to incomplete clean-up of backup replica sync operations. #22406

  • Fixed the cluster failure occurred after requesting healthcheck of a member using REST API while the hazelcast.socket.buffer.direct property is enabled. #21702

  • Improved connection handling. #21643

  • Fixed an issue where a cluster could not be formed when security is enabled, various client permissions are set, and multiple members are started simultaneously. #21508

4.2.5

If you’re using eviction or expiration for maps, you must set the per-entry-stats-enabled property to true to fix an issue in version 4.2.x where some map entries are not always evicted or expired according to their time-to-live or maximum idle duration configurations.

Enhancements

  • Introduced a system property for allowing you to audit that all the Hazelcast instances running in your environment have the instance tracking file name set correctly in the configuration. See the note in Instance Tracking. #19929

  • Enabled XXE (XML External Entity Reference) protection for XMLInputFactory. The issue was reported through https://huntr.dev/bounties/d63972a2-b910-480a-a86b-d1f75d24d563/. #20942

  • The probe level for most of the network related statistics has been changed to "DEBUG" to decrease the pressure on Management Center; now they are not sent to Management Center by default. If you want to see these statistics, you need to set the "hazelcast.metrics.debug.enabled" property to true. #21275

Fixes

  • Fixed an issue where the statistics like puts and removals were not increasing when these operations are executed through Transactional interface. #21105

  • Fixed an issue where Hazelcast clients, which have only the IP address of a member to connect (but the member also has a hostname), were not able to connect to the cluster. #20631

  • Hazelcast’s memcached implementation was interpreting the number values and parameters for incr and decr wrongly (numbers were being converted into byte arrays instead of decimals). This has been fixed by making these commands' implementations strictly follow the memcached protocol specification. #19676

  • Fixed an issue where the totalPublishes statistics for the Reliable Topic data structure were always generated as 0. #19656

  • Fixed an issue where the map.clear() and cache.clear() methods were evicting all entries in all near caches of all the maps in a cluster, not only the map on which these methods are called. #19501

  • Fixed an issue that caused some map entries to remain in Hazelcast even when you configured eviction or expiration. #19483

Removed/Deprecated Features

The following system properties have been deprecated:

  • hazelcast.client.statistics.enabled

  • hazelcast.client.statistics.period.seconds

4.2.4

For the distributions packages of IMDG, we updated the vulnerable version of log4j2 in Management Center to 2.17.0. No changes were made to the IMDG code.

The 4.2.4 release notes for the Hazelcast distribution on Maven wrongly states that log4j2 version has been updated to 2.15.0; it should be 2.17.0.

4.2.3

For the distributions packages of IMDG, we updated the vulnerable version of log4j2 in Management Center to 2.15.0. No changes were made to the IMDG code.

4.2.2

Fixes

  • Fixed an issue where the client state listener was not properly working with failover clients (in blue-green deployments); it was failing with invalid configuration exception. #19115

4.2.1

Enhancements

  • Enhanced the getPartitionGroupStrategy() method to have cluster members as arguments so that useful partitioning strategies can be implemented by accessing the members using this method. #18904

  • The log message for infinite cluster connection timeout is clearer now. Previously, it was represented as the value of Long.MAX_VALUE. #18648

  • Introduced a new mechanism in the background expiration tasks; now a thread local array controls the allocations for these tasks otherwise which may cause increased garbage collection pressure and CPU usage spikes when you use aggressive expiration configurations, e.g., low time-to-live values. #18507

Fixes

  • Fixed an issue where the health monitor was incorrectly showing the value for free metadata memory. #18953

  • Fixed an issue where the updates made to a persistent map store might be lost when the write coalescing is enabled. #18929

  • Some merge policies like LatestUpdateMergePolicy for the map and WAN replication configurations require the per-entry statistics to be enabled. Previously, this configuration inconsistency was causing the related member to fail at runtime. Now, the Hazelcast member fails to start, i.e., fast fails, in such a case. #18875

  • Fixed an issue where the maximum size policy for a map was being ignored when the policy is PER_NODE and the cluster is scaled down (due to losing or killing a member). #18873

  • Fixed an issue where the map’s Near Cache was setting its maximum size as 10.000 even if the configured eviction policy is NONE. #18812

  • The LRU eviction policy now takes last access time value into account to prevent premature removal of the lately added but not yet accessed map entries. #18624

Contributors

We would like to thank the contributors from our open source community who worked on this release:

4.2

New Features

Hazelcast IMDG Open Source New Features:

  • Sorting Support for Hazelcast SQL: Added the support of SQL sorting feature. By using the newly implemented ORDER BY, LIMIT, and OFFSET constructions, you can see the query results being ordered in ascending or descending fashion, limit the count of results, or skip some results. See the documentation for examples.

  • Node Aware Partition Grouping: Added the support of partition grouping mechanism in the Hazelcast discovery plugin for Kubernetes. You can create partition groups according to the name of the node which is provided by this plugin and on which the containers/pods run. See the NODE_AWARE section of the IMDG Reference Manual for more information.

  • Placement Aware Partition Grouping: Added the support of partition grouping mechanism in the Hazelcast discovery plugin for AWS. You can group members according to their placement metadata provided by the cloud providers, such as rack, fault domain, power sources, network, and resources of a virtual machine in a zone. See the PLACEMENT_AWARE section of the IMDG Reference Manual for more information.

  • Dynamic Log Level Support: Added the support of changing Hazelcast log levels without restarting cluster members. See the Dynamically Changing Log Levels section.

Breaking Changes

BETA API Changes
  • Being in its BETA stage, we improved the GenericRecord API, and performed various related changes in the serialization API due to these improvements:

    • GenericRecord#read() and GenericRecord.Builder#write() methods have been renamed as get() and set(), respectively.

    • GenericRecord.Builder has been moved to its own class as GenericRecordBuilder.

    • UTF has been renamed as String for the following methods, which are now deprecated:

      • ClassDefinitionBuilder.addUTFField()

      • ClassDefinitionBuilder.addUTFArrayField()

      • PortableWriter.writeUTF()

      • PortableWriter.writeUTFArray()

      • PortableReader.readUTF()

      • PortableReader.readUTFArray()

      • ObjectDataOutput.writeUTF()

      • ObjectDataOutput.writeUTFArray()

      • ObjectDataInput.readUTF()

      • ObjectDataInput.readUTFArray()

    • UTF has been renamed as String for the following without deprecation:

      • GenericRecord.readUTF()

      • GenericRecord.readUTFArray()

      • GenericRecordBuilder.writeUTF()

      • GenericRecordBuilder.writeUTFArray()

Default Settings Change
  • Seeing that getting entry level statistics in a map has not been a common use case and it has been creating unnecessary memory overhead that can be otherwise used for storing user data, per-entry statistics of maps now are disabled by default; they were enabled in the previous releases. See the Enhancements section below.

  • The default value for cluster connection timeout for clients has been set as -1 (infinite timeout): This is the timeout value for the client to give up to connect to the current cluster. For this new default value, the client will not stop trying to connect to the target cluster. This change has been made to prevent the clients from shutting down when a cluster is unavailable; otherwise (previous default value was 2 minutes), it often meant restarting the whole application. The reason for keeping the previous default value has been not to break the backward compatibility; with this release we introduced this breaking change which would be better for users. See here for more information. #18094

Enhancements

Hazelcast IMDG Open Source Enhancements:

  • Per-Entry Statistics of Maps: Introduced a configuration option (per-entry-stats-enabled) for retrieving per-entry statistics of a map, which is disabled by default. See the Accessing Map and Entry statistics section.

  • Default Serializer Overriding: Added the ability to override Hazelcast’s built-in serializers. Also added the serialization support of java.util.Optional. See the Serialization chapter.

The following are the improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Added support of named parameters in SQL’s HazelcastFunction. #18328

  • Added the HTTPS/TLS support for phone home pings. Also enhanced the phone home data by adding the following information:

    • Java classpath on which the member runs

    • Detailed client information such as total connection durations and client versions

    • Total number of created proxies for each distributed object service

  • Improved the LIKE query predicate so that it now supports indexes. #18289

  • Added management-permission to the client permissions that defines which Management Center client principals/endpoints are allowed to perform management tasks. #18264

  • Added the temporal types support for indexes by introducing converters for the missing types (LocalTime, LocalDate, LocalDateTime and OffsetDateTime) #18244

  • Added the support of bitmap index statistics to be propagated to local map statistics. #18220

  • Improved the map store configuration such that it is enabled unless you explicitly disable it; this improvement has been introduced to eliminate the inconsistencies between map store’s declarative and programmatic configurations. #18217

  • Implemented the High-Density Metadata Store to store all the metadata for on-heap and off-heap storages. This new store is based on BinaryElasticHashMap. A new type of record called HDMetadataRecord has been introduced for this purpose that references the key and value part of the metadata. #18186

  • Added the ability to register the class definition of the parent generic record and check the class definition compatibility for nested portable fields. #18180

  • Improved the SQL engine so that it attempts to re-use the already deserialized values stored in an index; this has been improved the map scan operations performance when there is at least one index on an IMap. #18172

  • Added the iterator() and iterable() methods with different parameter combinations to the map. #18126

  • Added the support of non-nullable columns for the SQL service. #18114

  • Implemented the fetch and offset SQL clauses without pushing the operator on the individual member. #18091

  • Introduced a new serialization mechanism for SQL row data. Previously, every value was being serialized as Data which was inefficient and the non-Java clients could not read some types, e.g., decimal. With this new mechanism, a new custom codec SqlPageCodec has been introduced that serializes values using a custom built-in list serializer depending on the type of the column. #18089

  • Added Nonnull annotations to StreamSerializer. #18071

  • Added the ability to cancel queries on the client side when there are active requests sent to the member; this enhancement facilitates the future implementation of a non-blocking SQL client handler. #18047

  • Added the support of OBJECT type in the comparison operators in Hazelcast’s SQL service. #18016

  • Implemented the ORDER BY clause for the SQL service; it supports the ASC and DESC sortings with limitations. #18013

  • Added the support of remainder operation for Hazelcast’s SQL service. #17997

  • Added the support of NOT LIKE expression to Hazelcast’s SQL service. #17996

  • Introduced a new SQL threading model to improve its performance. #17985

  • Introduced the support of schemas for the SQL public API. #17953

  • Implemented the IMap.entrySet() method for the partition ID set. #17937

  • Improved the discovery by external smart clients in the cloud environments: the clients now only need to know the address of any member (or that of a load balancer if members are exposed via load balancer). #17895

  • Introduced a configuration property to ignore errors during enabling the XXE protection. This protection works with JAXP 1.5 (Java 7 Update 40) and newer. When an older JAXP implementation is added to the classpath, e.g., Xerces and Xalan, an exception is thrown. The newly introduced property, namely hazelcast.ignoreXxeProtectionFailures, allows you to ignore those exceptions. #17839

  • Added the missing replicatedmap-permission support to the XML and YAML configuration handlers. #17810

  • Replaced Charset with StandardCharsets to be used in JSON querying. #17741

  • Implemented migration listener for the Java client. #17713

  • Introduced tenant control when creating JCache caches. #17673

  • Added the BigDecimal,LocalTime,LocalDate,LocalDateTime,OffsetDateTime types to the Portable Serialization. #17257

  • Introduced the unparkAll() method for event journal read operations. #14081

Fixes

  • Fixed a regression issue where the locked and expired entry keys could not be reached over indexes. #18386

  • Improved the deserialization for indexes: when having multiple indexes on a map, each time an entry is put in the map and thus to the index, the entry is deserialized for each index instead just once. This was causing performance issues. #18343

  • Fixed an issue where SQL’s ORDER BY statement was failing if the field is indexed and there are other clauses such as WHERE in the query. #18341

  • Fixed an issue where GenericRecord could not be queried when the in-memory format of the map is OBJECT. #18336

  • Fixed a discrepancy between the behaviors of index-scan and full-scan in maps; the eviction of the idle map entries was not stable in IMDG 4.x series due to this discrepancy. #18334

  • Fixed an issue where the health monitor was logging no values for garbage collection metrics. #18317

  • Fixed an issue that occurred when there is Near Cache configured for both the member and client sides for the same map and serialize-keys option is false. #18312

  • Fixed an issue where the transactions was not throwing TransactionTimeOutException in case of a timeout. #18305

  • Fixed an issue where Hazelcast IMDG was not picking the hazelcast.xml configuration file from the download package but from the current working directory where IMDG has been started. #18304

  • Fixed the race condition occurring during the serialization of writes for the copy-on-write data structures. #18285

  • Fixed an issue where the clients, in a blue/green deployment, were hanging while reconnecting to the alternative cluster due to mishandling of member list. #18276

  • Fixed an issue where CachedQueryEntry could not be serialized since it didn’t have a default serializer. #18238

  • Fixed an issue where the Java client was not receiving membership events in its membership listener when a member with Hot Restart Persistence enabled is restarted. #18234

  • Fixed a regression issue: when overlapping wildcard configurations are defined declaratively, the most specific one was inheriting attributes from the more generic one during parsing. #18187

  • Fixed an issue where the non-UTF characters in a JSON value was causing query failures. #18183

  • Fixed an issue where tasks were not running in parallel when they are submitted to all the cluster members in some scenarios. #18107

  • Fixed an issue that prohibited Hazelcast from being used as Tomcat session manager when it is also deployed in a web application context: When a client application that uses Hazelcast tries to connect to the Hazelcast cluster and this application’s web sessions are persisted using Hazelcast’s Tomcat session manager, the session manager could not connect to the cluster. This has been fixed by improving Hazelcast’s service loader mechanism. #18103

  • When the in-memory format of a map is NATIVE and the uploaded user code has missing classes (in case the user code deployment feature is used), the resulting exception could not be seen on the client side when a map query is run. This was causing the client to hang indefinitely and fixed by improving the failure handling for this case. #18081

  • Fixed an issue where the queue items were being delivered more than once when they are reproduced after a member leaves the cluster. #18057

  • Fixed the syntax for inMemoryFormat variable in the MapConfig.toString() method. #17976

  • Fixed a failure which happened when a client is recreated with the same client failover configuration after creating a map that has the default near cache eviction configuration. #17952

  • Fixed several issues when handling the SQL expressions. The fixes include not relying on Apache Calcite for inference and coercion anymore and introducing custom operand checker implementations provided by every operator. #17947

  • Fixed an issue where the metrics for map hits statistics in Management Center were decreasing as the map entries are being expired. #17930

  • Fixed an issue where the clients were opening two connections to the same member when the member is behind a private network. #17844

  • Fixed an issue where the failures in SessionAwareSemaphore was preventing the acquired permits from their releases. #17826

  • Fixed an issue where the Javadoc of release methods for session-aware semaphore structure was incorrectly addressing "threads" instead of "Hazelcast instances". #17823

  • Fixed an issue where the remove() and delete() operations of maps were not updating the local map statistics. #17771

  • Fixed the metrics unit for cache statistics to be declared in microseconds. #17742

  • Fixed a failure when retrieving the member state before the member becomes ACTIVE on Kubernetes using Helm charts. #17773

  • Fixed an issue where the parsing of configurations for some Hazelcast features, such as Hot Restart and user code deployment, was overriding the existing configuration values. #18036, #18034, #17675, #17681, #17885, #17917, #17923, #17924, #17940, #17945, #17946, #17875, #17878, #17969, #17971, #17972

  • Removed InetSocketAddressCache from the Java client code so that the client can continue to work, while preserving the behaviors in a Blue/Green Deployment scenario; the Java client was not able to resolve the new address of a restarted member, e.g., for a setup in Docker environment. #17239

Removed/Deprecated Features

  • The Symmetric Encryption feature has been deprecated. You can use the TLS/SSL protocol to establish an encrypted communication across your Hazelcast cluster.

  • The following system properties have been deprecated:

    • hazelcast.partition.group.rack

    • hazelcast.partition.group.host

    • hazelcast.hotrestart.free.native.memory.percentage

Contributors

We would like to thank the contributors from our open source community who worked on this release:

4.1.10

Enhancements

  • Improved connection handling. #21644

  • To decrease the load on the Management Center for large clusters, the level of network related metrics has been changed to DEBUG. When you need these metrics, you can use the hazelcast.metrics.debug.enabled property. #21260

Fixes

  • Fixed an issue where replication over WAN was failing on the source cluster members, when there are multiple batch publishers configured in a single WAN replication. #22498

  • Fixed an issue where the cluster was failing to operate since the indexes on backup partitions were removed by map.ClearBackupOperation. #22305

  • Fixed an issue where a cluster could not be formed when security is enabled, various client permissions are set, and multiple members are started simultaneously. #21507 #21507

4.1.9

Enhancements

Fixes

  • Fixed an issue where a cluster was unresponsive when you perform a health check to see the members are in the safe state; cluster members were hanging in the REPLICA_NOT_SYNC state during such health checks. #21205

  • Fixed an issue where the list of members in the cluster was reset to an empty list when member IDs change after a cluster restart: this was causing startup failures since Hazelcast could not manage the events due to the empty member list after a restart. #21181

  • Fixed an issue where the statistics like puts and removals were not increasing when these operations are executed through Transactional interface. #21100

  • Fixed an issue where Hazelcast clients, which have only the IP address of a member to connect (but the member also has a hostname), were not able to connect to the cluster. #20632

4.1.8

For the distributions packages of IMDG, we updated the vulnerable version of log4j2 in Management Center to 2.17.0. No changes were made to the IMDG code.

4.1.7

For the distributions packages of IMDG, we updated the vulnerable version of log4j2 in Management Center to 2.15.0. No changes were made to the IMDG code.

4.1.6

Enhancements

  • Improved the log message that is displayed when using the hazelcast.io.selectorMode property to decrease the high CPU usage. The previous misleading message read as "Recreated Selector because of possible java/network stack bug". It has been changed to "Selector was rebuilt, consider updating Java and/or your network stack drivers". #19761

  • Introduced the hazelcast.internal.map.expired.key.scan.timeout.nanos property to limit the execution time of the cleanup operations, i.e., entry expiration tasks of a map; this is to prevent the increased CPU usage as the partition size grows. #19663

Fixes

  • Hazelcast was executing cluster wide operations when you query the state of a member using the health check endpoint - it was causing to kill all the members in a cluster; this issue has been fixed. #19837

  • The increment command (incr) in the Memcache client was returning the ASCII code of the expected result; this interpretation issue has been fixed. #19681

4.1.5

Fixes

  • Fixed an issue where a high amount of garbage collection pressure was occurring during repartitioning especially when having a high partition count. #19295

  • Fixed an issue where the client state listener was not properly working with failover clients (in blue-green deployments); it was failing with invalid configuration exception. #19116

Deprecated/Removed Features

  • Deprecated the following properties:

    • hazelcast.client.statistics.period.seconds

    • hazelcast.client.statistics.enabled

4.1.4

Fixes

  • Fixed an issue where the updates made to a persistent map store might be lost when the write coalescing is enabled. #18924

  • Fixed an issue where the maximum size policy for a map was being ignored when the policy is PER_NODE and the cluster is scaled down (due to losing or killing a member). #18871

  • Fixed an issue where the map’s Near Cache was setting its maximum size as 10.000 even if the configured eviction policy is NONE. #18834

  • The LRU eviction policy now takes last access time value into account to prevent premature removal of the lately added but not yet accessed map entries. #18632

4.1.3

Enhancements

  • Introduced a new mechanism in the background expiration tasks; now a thread local array controls the allocations for these tasks otherwise which may cause increased garbage collection pressure and CPU usage spikes when you use aggressive expiration configurations, e.g., low time-to-live values. #18480

  • Updated the Hazelcast Kubernetes plugin dependency version to 2.2.2. #18331

  • Added the ability to register the class definition of the parent generic record and check the class definition compatibility for nested portable fields. #18284

Fixes

  • The difference between the Near Cache configurations before and after a client is recreated with the same client failover configuration is fixed now. #18361

  • The eviction of the idle entries is now stable in IMDG 4.x series by aligning the behaviors of index-scan and full-scan in maps. #18346

  • The IllegalArgumentException is not thrown anymore when there is Near Cache configured for both the member and client sides for the same map and serialize-keys option is false. #18313

4.1.2

Fixes

  • Fixed an issue where the Java client was not receiving membership events when a member with Hot Restart Persistence is restarted. #18268

  • Fixed an issue where the Near Cache configuration of a map in client was not the same after the client is recreated with the same client failover configuration. #18356

  • Fixed an issue that prohibited Hazelcast from being used as Tomcat session manager when it is also deployed in a web application context: When a client application that uses Hazelcast tries to connect to the Hazelcast cluster and this application’s web sessions are persisted using Hazelcast’s Tomcat session manager, the session manager could not connect to the cluster. This has been fixed by improving Hazelcast’s service loader mechanism. #18116

  • Fixed an issue where the metrics for map hits statistics in Management Center were decreasing as the map entries are being expired. #18112

  • When the in-memory format of a map is NATIVE and the uploaded user code has missing classes (in case the user code deployment feature is used), the resulting exception could not be seen on the client side when a map query is run. This was causing the client to hang indefinitely and fixed by improving the failure handling for this case. #18084

  • Fixed an issue where the queue items were being delivered more than once when they are reproduced after a member leaves the cluster. #18057

  • When the user code deployment is used and the classes to be deployed have com.hazelcast prefix, it was causing failures in other Hazelcast products, e.g., Jet. This has been fixed by making use of the context classloader when loading such classes. #17915

  • The -c parameter of the cp-subsystem.sh script was used for both cluster name and group variables. This has been fixed by introducing the -g parameter for groups. #17907

  • Fixed an issue where the remove() and delete() operations of maps were not updating the local map statistics. #17750

4.1.1

New Features

  • Node Aware Partition Grouping: Added the support of partition grouping mechanism in the Hazelcast discovery plugin for Kubernetes. You can create partition groups according to the name of the node which is provided by this plugin and on which the containers/pods run. See the NODE_AWARE section of the IMDG Reference Manual for more information.

Enhancements

  • Introduced a configuration property to ignore errors during enabling the XXE protection. This protection works with JAXP 1.5 (Java 7 Update 40) and newer. When an older JAXP implementation is added to the classpath, e.g., Xerces and Xalan, an exception is thrown. The newly introduced property, namely hazelcast.ignoreXxeProtectionFailures, allows you to ignore those exceptions. See the System Properties appendix of the IMDG Reference Manual for more information.

  • Updated the Hazelcast Kubernetes plugin dependency to 2.2.1. #17928

  • Separated the hazelcast-sql Maven module into hazelcast-sql-core and hazelcast-sql. #17922

  • Implemented the IMap.entrySet() method for the partition ID set. #17886

Fixes

  • Fixed an issue where the clients were opening two connections to the same member when the member is behind a private network. #17888

  • Fixed an issue where the Javadoc of release methods for session-aware semaphore structure was incorrectly addressing "threads" instead of "Hazelcast instances". #17827

  • Fixed an issue where the Hazelcast Java client was not able to resolve the new address of a restarted member, e.g., for a setup in Docker environment. #17062

4.1

New Features

Hazelcast IMDG Open Source New Features:

  • SQL Implementation: Implemented the SQL engine that supports map scans, index scans, projections and filters. It also introduces the hazelcast-sql module to handle the SQL engine’s dependencies on Apache Calcite. See the SQL section.

  • Automatic Discovery Strategy Detection: Hazelcast now automatically detects the cloud environment it’s running on (Kubernetes, AWS, GCP or Azure) and applies the necessary discovery strategy for the environment. This allows automatically forming the Hazelcast clusters in cloud environments without any configuration. See the Auto Detection section.

  • Overriding Configuration: You can now override the configuration entries of your cluster without changing the declarative configuration files (XML/YAML). See the Overriding Configuration section.

  • Instance Tracking: Hazelcast now writes a file on a Hazelcast member startup at the configured location. This file contains metadata about the member such as version, product name and process ID, and it is not deleted on the member shutdown. See the Instance Tracking section.

Hazelcast IMDG Enterprise New Features:

  • Security Audit Logging: This feature allows observing some important cluster events. Auditable events have a unique type ID, and contain a timestamp and importance level. See the Logging Auditable Events section.

Breaking Changes

  • Starting a standalone Hazelcast instance requires disabling Auto Detection joiner (before it required disabling Multicast joiner). #17112

  • In a CP subsystem session, a generic IllegalStateException was being thrown when Hazelcast is shutdown. Now the same situation throws HazelcastInstanceNotActiveException. #17120

  • Implemented and/or overridden the default methods in Java 8 collections, such as computeIfAbsent(), forEach() compute() and replaceAll() for maps. #14913

Enhancements

Hazelcast IMDG Open Source Enhancements:

  • Improvements in Partial Network Disconnections: Introduced properties to be configured to detect and resolve the partial network issues among the Hazelcast IMDG members. See #16680 and the Partial Network Partitions section.

  • Parallel Independent Migrations: Parallelized the partition replica migrations so that the time needed for rebalancing the partitions after adding or removing a member to/from a cluster is reduced. See the hazelcast.partition.max.parallel.migrations property explanation in the System Properties appendix.

  • CP Subsystem Listeners and Metrics: Added membership and group availability listeners for CP Subsystem. Also, added support for publishing CP Subsystem and CP data structure statistics via Metrics. See the CP Subsystem Listeners section.

  • GenericRecord: Added support for accessing domain objects without domain classes. See the GenericRecord section.

  • Thread Affinity: Introduced the CPU thread affinity; threads can have affinity for particular CPUs and using this you have a better control on the latency and a better throughput. See the CPU Thread Affinity section.

  • Trusted Interfaces: It is now possible to restrict the source IP addresses from which the Management Center operations are allowed. See the Limiting Source Addresses section.

  • Priority Queue: Added support for the queues with comparators. Using Priority Queue, you can define items with higher priority to be polled first. See the Priority Queue section.

  • Statistics for Durable and Scheduled Executor Services: Added the statistics for scheduled and durable executor services, and for the executor of offloaded entry processors to be monitored by Management Center.

Hazelcast IMDG Enterprise Enhancements:

  • Out-of-the-Box Kerberos Support: Introduced support for Kerberos authentication protocol which is one of the standard solutions for single-sign-on. It also adds the GSSAPI authentication support in LDAP configuration. See the Kerberos Authentication Type section.

  • Multiple Persistent Directories: Added support for multiple mounting directories in the persistent memory configuration and also thread affinity for NVMs (Non-Volatile Main Memories). See the Using Persistent Memory section.

  • WAN Throttling Mechanism: This prevents the WAN consumers from getting overloaded by the WAN producers if transferring the WAN events takes less time than processing them in the target cluster. This can be the case with WAN synchronization if the network latency is low enough. It can be configured using the following properties:

    • hazelcast.wan.consumer.invocation.threshold

    • hazelcast.wan.consumer.ack.delay.backoff.init

    • hazelcast.wan.consumer.ack.delay.backoff.max

    • hazelcast.wan.consumer.ack.delay.backoff.multiplier

  • Implemented a new concurrent High-Density Memory Store index for the queries, based on B+ tree that is enabled by default; brings better performance for lookup queries and enables SQL queries.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Added the missing replicatedmap-permission support to the XML and YAML configuration handlers. #17812

  • Replaced the word "should" with "must" in the exception messages to improve the user focus and indicate obligation rather than advice. #17710

  • Added the getAndDecrement() operation support for the IAtomicLong data structure. #17699

  • Introduced the mode configuration attribute for the persistent memory. It defines operational mode of it which can be either MOUNTED or SYSTEM_MEMORY. #17696

  • Added the support for checking duplicate fields when building class definitions. #17682

  • Added the support for IList with nullable items to the client protocol. #17690

  • Improved the robustness of the diagnostics log writer and its plugins. #17665

  • Added the client name tag to the client related metrics. #17654

  • Added a check for IMap READ permissions for the maps involved in the SQL query executions. #17612

  • Added the support for accessing the Hazelcast instance in the login modules and credential factories. #17605

  • Introduced a simplified configuration of Kerberos authentication for simple scenarios. #17573

  • Added statistics support for IBM, OpenJ9, ZGC and Shenandoah. #17561

  • Introduced a state-tracking mechanism for the external configuration parser; all mismatched/unapplied configuration entries coming from environment variables or system properties are now logged. #17559

  • Updated the jackson.version to 2.11.2 and SnakeYAML to 2.1. #17484, #17446

  • Added support for the merge operation in IMap. #17419

  • Added public classes to expose the member- and client-side caching provider implementations (in the JCache implementation) without referring to internal classes. #17320

  • Improved the entry processing mechanism so that the read-only processors on backups are not executed anymore. #17318

  • Improved CacheSimpleConfig so that it now accepts cache name in its constructors. #17287

  • Cleaned up the noisy initial log messages on Hazelcast startups. #17243

  • Introduced auto-disposable tasks for the scheduled executor service. #17215

  • Added client labels to ClientEndpointImpl.toString() to simplify the client identification (in addition to client uuid) on the Hazelcast member side. #17178

  • Added exception handling when creating caches with tenant control. #17122

  • Added support for IMap.compute(). #17030

  • Added support to allow submitting and executing operations while a member is shutting down to increase the availability of a cluster more that keeps large data. #17028

  • Added ExtendedMapEntry in order to create a mechanism for setting TTLs in entry processors. #16991

  • Added hazelcast-azure discovery plugin to the hazelcast-all module. #16982

  • Introduced the ConfigRecognition API that determines if a provided declarative configuration is recognized by the rules defined in a given implementation. Along with the API the following three implementations are added:

    • MemberConfigRecognizer for recognizing member XML and YAML configurations

    • ClientConfigRecognizer for recognizing client XML and YAML configurations

    • ClientFailoverConfigRecognizer for recognizing failover client XML and YAML configurations

  • Added the publishAll(), publishAllAsync() and publishAsync() methods to ITopic. #16946

  • Made the diagnostics logs cloud-friendly so that they can also be sent to stdout in addition to sending to local files. #16941

  • Improved the mechanism of partition table updates to eliminate the latencies when these updates are sent to the clients by a member. #16939

  • Improved the client configuration such that when the client failover configuration is provided, the reconnect mode cannot be set as off; it now fails fast in this case. #16886

  • Introduced the forEach() loop support for IMap. #16877

  • Added the load() method to Config, ClientConfig and ClientFailoverConfig classes. This method loads the configuration with the known locations. If not found, the default configuration is returned. #16864

  • Improved the deleteAll() (MapStore) and ITopic Javadocs. #16862, #16861,

  • Added support for IMap.computeIfAbsent(). #16808

  • Added the setAll() and setAllAsync() methods for IMap. #16787

  • Added the creation time metric for the executor service. #16775

  • Improved an unclear exception message for credentials. #16756

  • Updated the related aspects of Hazelcast IMDG after the changes done on the client protocol side to add the ability to add new parameters, methods, services, events and custom types to codecs. #16718

  • Introduced the putAllAsync() method for MultiMap. #16698

  • Defined metrics for ISet and IList collections. #16665

  • Upgraded log4j2 version to 2.13.0. #16654

  • Improved the computeIfPresent() implementation so that now it keeps a clone of the old/existing value and uses that for replace/delete operations. #16636

  • Introduced the hazelcast.logging.details.enabled property to make the logging of cluster version, name and IP optional to decrease the noise in the logs when, for example, you have a single-member cluster. #16622

  • Upgraded the Hazelcast Kubernetes plugin version to 2.0.1. #16590

  • Added the support for automatically detecting the classloader of a type for the user code deployment feature. #16585

  • Made IMap.putAllAsync() and IMap.submitToKeys() methods public API. #16449

  • Clarified the exception message when you connect to a cluster with an invalid cluster name. #15099

  • Refactored the check and retry initialization logic of ReplicatedMapProxy so that they are performed in parallel for different partitions. #14331

  • Improved the behavior of ConcurrentMap.computeIfPresent: combined single client-server round trips instead of two (for get and replace methods). #11958

Fixes

  • Fixed an exception in the /node-state REST calls when the member is not fully activated. #17798

  • Fixed an issue where Hazelcast was not releasing the acquired lock sessions that fail for the reasons other than session expiration and wait key cancellation. #17697

  • Fixed an issue where Hazelcast was returning false if a client is successfully deregistered from any member, but events are still delivered for the non-deregistered ones. #17646

  • Fixed a data loss issue that was happening while promoting a lite member to a data member. #17621

  • Fixed an issue where the configuration was not updating entries' time-to-live values if the entry processor implements the Offloadable interface. #17606

  • Fixed an issue where the caller stacktrace was missing on the rethrown async runtime exceptions. #17546

  • Fixed the rendering of diagnostics in case there is an exception inside a diagnostics plugin. #17501

  • Fixed an exception that is thrown when using the entry store API and issuing a put into the IMap for an item which is not in the map but exists in the backing store. #17441

  • Fixed an issue where the custom load balancers could not be configured declaratively. #17415

  • Fixed a race issue when creating a cache (JCache) using multiple methods with the same cache name but having different keys. #17286

  • Fixed an issue where the repeated calls of executeOnKeys() in Hazelcast clients for NATIVE maps was causing a continuous increase in the used memory and exhaustion of the memory pool after a while. #17276

  • Fixed an issue where ReliableTopicMessageListener was firing a warning when the client is shutting down. #17153

  • Fixed an issue where the client was stuck with an outdated member list after a split-brain scenario. #17147

  • Fixed the member side user code deployment; it was throwing an exception when inner classes are used to be loaded. #17044

  • Fixed the broken interoperability between the CompletableFuture methods. #17020

  • Fixed an issue where touching a map entry having an entry processor working on it was modifying its time-to-live. #16987

  • Fixed an issue in the cache service where its pre-join operation was considering CacheConfig as resolved: it was assuming that key/value types, user customizations and other cache configurations have been loaded. This was an issue when the cache is not touched yet. #16917

  • Fixed an issue where Management Center was not working as expected when the cluster is set up using advanced network configuration. #16910

  • Fixed an issue where ServiceLoader was round-tripping between URL and URI, and consequently loses the associated URLStreamHandler when trying to load Hazelcast from a custom class loader. #16846

  • Fixed an issue where the class definitions, that are registered explicitly in the serialization configuration and have the same class ID in different factories, were not handled properly. #16831

  • Fixed the NullPointerException in IndexCopyBehavior.NEVER mode. #16784

  • Fixed an issue where the client XML configuration could not properly handle the empty Near Cache name. #16768

  • Fixed an issue where the client permissions for Reliable Topic and Ringbuffer were missing. #16755

  • Fixed an issue where the type information was missing the Metrics MBeans. #16747

  • Fixed an issue where the REST API was always requiring the call URLs to end with a slash character. #16688

  • Fixed an issue where the service URL for Eureka could not be set using the declarative configuration. #16679

  • Fixed an issue where the wait key of a blocking call within a Raft invocation was still being reported as a live operation, when the key times out. #16614

  • Fixed an issue where the upload of classes using the client user code deployment were not successful when they are retrieved not in their created order. #16612

  • Fixed an issue where the size() method was returning a negative value when map, cache and multimap contain more than Integer.MAX_VALUE entries. #16594

  • Fixed an invalidation issue when using a transactional map from a cache with a Near Cache: the cache invalidation event occurs when the transactionalMap.put method is called. As a result, the entry was getting invalidated before the change is committed to the map. #16579

  • Fixed an issue where InPredicate was not invoking value comparison when the read attribute is null. #15100

  • Fixed an issue where Map, Cache, MultiMap data structures were returning negative values (size()) when the size is more than Integer.MAX_VALUE. #14935

  • Fixed an issue when a Hazelcast client calls the distributed executor service and the callable throws an exception with a custom type, then the exception was not being transported to the client. #9753

4.0.6

Enhancements

  • Updated log4j2 dependency version to 2.17.0 in Hazelcast’s root pom.xml. #20161

  • Updated the Hazelcast Kubernetes plugin dependency version to 2.2.2. #18330

  • Added the support of partition grouping mechanism in the Hazelcast discovery plugin for Kubernetes. You can create partition groups according to the name of the node which is provided by this plugin and on which the containers/pods run. See the NODE_AWARE section of Partition Grouping. #17913

  • Added protection against XML External Entity attacks (XXE). #17753

  • Introduced a configuration property to ignore errors during enabling the XXE protection. This protection works with JAXP 1.5 (Java 7 Update 40) and newer. When an older JAXP implementation is added to the classpath, e.g., Xerces and Xalan, an exception is thrown. The newly introduced property, namely hazelcast.ignoreXxeProtectionFailures, allows you to ignore those exceptions. See System Properties for more information. #17869

Fixes

  • Hazelcast’s memcached implementation was interpreting the number values and parameters for incr and decr wrongly (numbers were being converted into byte arrays instead of decimals). This has been fixed by making these commands' implementations strictly follow the memcached protocol specification. #19680

  • Fixed an issue where the client state listener was not properly working with failover clients (in blue-green deployments); it was failing with invalid configuration exception. #19117

  • There was a discrepancy between index-scan and full-scan. The definition of "touching an entry" was different between these two scans, while index-scan shifts expiry time, full-scan doesn’t. This behavior of index-scan and full-scan from the perspective of expiry-time shifting has been fixed:

    • Expired entries for both index-scan and full-scan are not shown now.

    • These scans now are not counted as "touching an entry" so no expiry time shifting happens.

  • Fixed an issue where the Java client was not receiving membership events when a member with Hot Restart Persistence enabled is restarted. #18269

  • The -c parameter of the cp-subsystem.sh script was used for both cluster name and group variables. This has been fixed by introducing the -g parameter for groups. #18149

  • Fixed the DelegatingCompletableFuture and the ClientDelegatingFuture subclass behavior when HazelcastSerializationException is thrown:

    • Serialization exception is now properly wrapped in (Execution|Completion)Exception when thrown from get, getNow or join methods.

    • whenComplete, handle and exceptionally can act on HazelcastSerializationException.

  • Fixed an issue where data was being lost from maps in the event of a member failure in a 3-member cluster with backup count set to 1. This was occurring when lite members are promoted to be data members. #17757

  • Fixed an issue where the service URL for Eureka could not be set using the declarative configuration. #17744

  • When the user code deployment is used and the classes to be deployed have com.hazelcast prefix, it was causing failures in other Hazelcast products, e.g., Jet. This has been fixed by making use of the context classloader when loading such classes. #17584

  • Fixed an issue where the user code deployment feature was failing to work for inner-classes when not all the members have the user provided class on their classpaths. #17576

  • Fixed an issue where Hazelcast was failing to load the configuration file (hazelcast.xml or user-provided files) from the classpath when in development mode for frameworks such as Quarkus. #17538

Removed/Deprecated Features

  • The following properties have been deprecated:

    • hazelcast.client.statistics.enabled

    • hazelcast.client.statistics.period.seconds

4.0.5

For the distributions packages of IMDG, we updated the vulnerable version of log4j2 in Management Center to 2.17.0. No changes were made to the IMDG code.

4.0.4

For the distributions packages of IMDG, we updated the vulnerable version of log4j2 in Management Center to 2.15.0. No changes were made to the IMDG code.

4.0.3

Enhancements

  • Added the declarative configuration support for the client load balancer. #17482

  • Added public classes to expose the member and client side caching provider implementations without referring to internal classes. #17421

  • Updated the Hazelcast Kubernetes dependency to 2.0.2. #17255

  • Added the ability to set the expiration times for entries when using entry processors. #17175

Fixes

  • Fixed NullPointerException when using IMap with HASH indexes. #17527

  • Fixed the rendering of diagnostics in case there is an exception inside a diagnostics plugin. #17502

  • Increased the dependency versions for Jackson and SnakeYAML to mitigate the security vulnerabilities. #17510

  • Fixed the propagation of exception from a function registered on a not-yet completed CompletionStage. #17520

  • Fixed a race condition when overwriting CacheManager during JCache creation. #17292

  • Fixed the NATIVE memory leak when using IMap.executeOnKeys() from an IMDG client. #17274

  • Resolved a performance issue when using PagingPredicate in combination with JDK8. #17211

  • Fixed the client behavior when cluster encounters a split-brain. In some cases, the client was unable to reconnect to the cluster, even after the cluster is healed. #17148

  • Improved disposing of off-heap memory when metrics are being used. #17145

4.0.2

Enhancements

  • Introduced the ConfigRecognition API that determines if a provided declarative configuration is recognized by the rules defined in a given implementation. Along with the API the following three implementations are added:

    • MemberConfigRecognizer for recognizing member XML and YAML configurations

    • ClientConfigRecognizer for recognizing client XML and YAML configurations

    • ClientFailoverConfigRecognizer for recognizing failover client XML and YAML configurations

  • Introduced WAN throttling mechanism: this prevents the WAN consumers from getting overloaded by the WAN producers if transferring the WAN events takes less time than processing them in the target cluster. This can be the case with WAN synchronization if the network latency is low enough. It can be configured using the following properties:

    • hazelcast.wan.consumer.invocation.threshold

    • hazelcast.wan.consumer.ack.delay.backoff.init

    • hazelcast.wan.consumer.ack.delay.backoff.max

    • hazelcast.wan.consumer.ack.delay.backoff.multiplier

      See here for their explanations.

  • Removed a constraint which was rejecting all the operations except the ones marked with AllowedDuringPassiveState while a member is shutting down. With this enhancement, operations are allowed during shutdown. This keeps the same safety guarantees while increasing availability, especially for the members holding large amounts of data. #17029

  • Introduced a packet flag to distinguish between the connections of members having the same Hazelcast version and different versions. #16993

  • Updated pom.xml to have the latest version of the Hazelcast AWS plugin, which introduced the support for AWS Fargate. See here for details. #16990

  • Improved the client partition table update push mechanism to prevent latency spikes when there is a partition table change and many clients need to be notified. #16938

  • Added the load() method to Config, ClientConfig and ClientFailoverConfig classes. This method loads the configuration with the known locations. If not found, the default configuration is returned. #16903

  • Added the missing partition/active partition count descriptor constants to the metrics. #16858

Fixes

  • Fixed an issue where the split-brain protection events were triggered during the startup of members, but before they join the cluster. With this fix, these listeners will not be fired until the minimum cluster size (quorum) is met after the member startups. #17090

  • Fixed an issue where the CP client message tasks were deserializing the responses. #17055

  • Fixed the broken interoperability between the CompletableFuture methods. #17048

  • Fixed map store initialization behavior when instance is running under a Java Security Manager. #17033

  • Fixed an issue where the Hazelcast cluster having advanced network configuration was not sending the proper connection information to Management Center. #17003

  • Fixed an issue in the cache service where its pre-join operation was considering CacheConfig as resolved: it was assuming that key/value types, user customizations and other cache configurations have been loaded. This was an issue when the cache is not touched yet. #16918

  • Fixed an issue where the Hazelcast instances were failing to start due to a missing flag for the cluster status. #16852

  • Reverted the change where Hazelcast was not retrying an invocation if it is sent to a specific member and returns the TargetNotMemberException exception. #16843

  • Fixed an issue where the class definitions, that are registered explicitly in the serialization configuration and have the same class ID in different factories, were not handled properly. #16833

  • Fixed the behavior of WAN sync when the cluster contains lite members for dynamically added WAN configurations. WAN sync can now be invoked on the lite members for dynamically added WAN configurations only if the configuration was added using that lite member. Other lite members can still fail if WAN sync is invoked on them for the dynamically added configuration. In addition to this, the member coordinating the WAN sync will no longer broadcast the event to lite members, as they don’t own any data which needs to be synchronized.

4.0.1

Enhancements

  • Introduced trusted interfaces concept for the Management Center connections. It is now possible to restrict the source IP addresses from which the Management Center operations are allowed. See the Limiting Source Addresses section.

  • Added the creationTime field to the Executor JMX bean. #16774

  • Promoted the IMap.putAllAsync() and IMap.submitToKeys() methods to IMDG’s public API. #16771

  • Upgraded the Spring, Hibernate 5 and Session Clusterin (hazelcast-wm) dependency versions for Hazelcast IMDG 4.x. #16731

  • Updated the related aspects of Hazelcast IMDG after the changes done on the client protocol side to add the ability to add new parameters, methods, services, events and custom types to codecs. #16719

  • Upgraded Log4J2 version to 2.13.0. #16658

  • Introduced the hazelcast.logging.details.enabled property to make the logging of cluster version, name and IP optional to decrease the noise in the logs when, for example, you have a single-member cluster. #16630

  • Updated the Hazelcast Kubernetes plugin to 2.0.1. #16591

  • Added the support for automatically detecting the classloader of a type for the user code deployment feature. #16584

Fixes

  • Fixed an issue where the Metrics beans were missing the standard Type information when monitoring with JMX. #16729

  • Fixed an issue when a Raft leader stops receiving heartbeats from the majority and it switches to the follower role in the same term: during this switch, it was also deleting its own vote in the term. #16645

  • Fixed an invalidation issue when using a transactional map from a cache with a Near Cache: the cache invalidation event occurs when the transactionalMap.put method is called. As a result, the entry was getting invalidated before the change is committed to the map. #16638

  • Fixed an issue where the wait key of a blocking call within a Raft invocation was still being reported as a live operation, when the key times out. #16615

  • Fixed an issue where the upload of classes using the client user code deployment were not successful when they are retrieved not in their created order. #16611

  • Fixed an issue where the size() method was returning a negative value when map, cache and multimap contain more than Integer.MAX_VALUE entries. #14935

4.0

See the Hazelcast IMDG 4.0 Migration Guide for the detailed guidelines to be followed when upgrading Hazelcast IMDG to 4.0.

New Features

Hazelcast IMDG Enterprise New Features:

  • Persistent Memory Integration: Added support for persistent memory, such as Intel® Optane™ DC, to be used by Hazelcast’s High-Density Memory Store feature. See the Using Persistent Memory section.

  • Encrypting Data in Hot Restart Store: Provided a framework and implementation to encrypt data in Hot Restart stores ("at rest") for data held in distributed structures. See the Encryption at Rest section.

  • X.509 Certificate Authentication: Added support for credential retrieval by the members from the X.509 certificates configured on the clients. See the Security chapter.

  • Out of the Box LDAP Configuration: Introduced a new LDAP login module where you do not need to provide your own JAAS login module for this purpose. See the Security chapter.

  • CP Subsystem Persistence: Implemented persistency for Hazelcast’s CP (Consistency & Partition Tolerance) subsystem. See the CP Subsystem Persistence section.

Hazelcast IMDG Open Source New Features:

  • CP Subsystem Development Mode: Introduced UNSAFE mode for the CP data structures. This way, you do not need to have at least three IMDG members to use CP subsystem: you can benefit from its functionalities using only one or two members. See the CP Subsystem Unsafe Mode section.

  • Bitmap Indexes: Introduced this feature to significantly lower index memory usage for low-cardinality columns and also to speed up the queries and lower memory requirements for them when the queries have multiple predicates acting on the same bitmap index. Note that bitmap indexes usage, currently, is a good fit for array/collection attribute indexing, rather than regular single-valued attributes. See the Bitmap Indexes section.

Breaking Changes

Please see the Upgrading to Hazelcast IMDG 4.0 for the details of following breaking changes to be considered while upgrading to IMDG 4.0.
Distributed Data Structures
  • Externalized the hardcoded Flake ID Generator properties. So the following constants are now in FlakeIdGeneratorConfig:

    • EPOCH_START

    • BITS_TIMESTAMP

    • BITS_SEQUENCE

    • BITS_NODE_ID

    • ALLOWED_FUTURE_MILLIS

  • Removed the values() and entrySet() from map and replicated map’s MBeans since these are potentially dangerous operations that can cause an OutOfMemoryException on the member since by default there is no limit on how many entries can be returned as a response for a query. #16238

  • Changed the behavior of the getAll() method: when either of the loaded key or value returned by the MapLoader is null, this method now fails fast. #16204

  • Removed the MapEvictionPolicy class and its related configurations. This has brought the following changes:

    • EvictionConfig is used instead of MapEvictionPolicy for custom eviction policies.

    • MapEvictionPolicy has been removed and MapEvictionPolicyComparator has been addd for the same tasks.

    • EvictionPolicyComparator, MapEvictionPolicyComparator and CacheEvictionPolicyComparator have become interfaces.

    • Moved EvictionPolicyComparator and EvictableEntryView to the com.hazelcast.spi.eviction package.

  • Removed deprecated IMap methods accepting EntryListener. #15781

  • Removed deprecated DistributedObjectEvent.getObjectId. The replacement is DistributedObjectEvent.getObjectName. #15773

  • Removed the deprecated getReplicationEventCount() method of local replicated map statistics. #15676

  • Removed the legacy AtomicLong and deprecated IdGenerator implementations. #15601

  • Removed the legacy ILock implementation and the HazelcastInstance.getLock() method. Instead we provide the unsafe mode of CP Subsystem’s FencedLock. The ICondition is not supported anymore. #15625

  • Removed the legacy AtomicReference implementation and the HazelcastInstance.getAtomicReference() method. Instead we provide the unsafe mode for all CP data structures. #15593

  • Removed the legacy Semaphore implementation and the HazelcastInstance.getSemaphore() method. Instead we provide the unsafe mode for all CP data structures. #15539

  • Removed the legacy CountdownLatch implementation and the HazelcastInstance.getCountdownLatch() method. Instead we provide the unsafe mode for all CP data structures. #15538

  • Added Nullable and Nonnull annotations to IQueue, IMap, HazelcastInstance, Cardinality Estimator, IExecutor, Durable Executor, QuorumService, CP subsystem, logging service, lifecycle service, partition service and client service. #15156, #15003, #15442, #15842

  • Added null checks and annotations to Cluster, Ringbuffer, Replicated Map, IList, ISet, ITopic and MultiMap interfaces. #15351, #15220

  • Made the collection clones of IMap immutable so that UnsupportedOperationException is thrown consistently upon the attempts to update a collection returned by the keySet, entrySet, localKeySet, values and getAll methods. #15013

Distributed Events
  • Removed the unused entry listener configuration code since the return type of getImplementation() has been changed from EntryListener to MapListener. #16051

  • Fixed MemberAttributeEvents getMembers() method to return the correct members list for the client. #15231

  • Refactored the MigrationListener API. With this change, an event is published when a new migration process starts and another one when migration is completed. Additionally, on each replica migration, both for primary and backup replica migrations, a migration event is published. #15071

  • Removed the backward compatible ADDED event from the loadAll method. #14964

  • Refactored and cleaned up the internal partition/migration listeners:

    • Renamed PartitionListener to PartitionReplicaInterceptor and removed registering child listeners, which is not used.

    • Renamed InternalMigrationListener to MigrationInterceptor and converted to interface with default methods.

  • Added EntryExpiredListener to the EntryListener interface. #14959

Configuration
  • CachingProvider no longer resolves an URI as the instance name since it was used both as the namespace for the cache manager and as a means to locate a running Hazelcast instance. #15995

  • Removed the configuration for user defined services SPI. #15951

  • The group name in the client configuration renamed to cluster name. #15772

  • Unified InvalidConfigurationException and ConfigurationException. #15132

  • Removed the deprecated AwsConfig getter/setter methods, e.g., getAccessKey(). They have been replaced with the getProperty() methods, e.g., getProperty("access-key"). #15758

  • Moved the following client statistics properties to the public ClientProperty class.

    • hazelcast.client.statistics.enabled

    • hazelcast.client.statistics.period.seconds

  • Undeprecated the following group properties:

    • hazelcast.memcache.enabled

    • hazelcast.rest.enabled

    • hazelcast.http.healthcheck.enabled

  • Removed the deprecated get/setImplementation() methods of login module configuration. They have been replaced with get/setClassName(). #15729

  • Removed the deprecated get/setPartitionStrategy() methods of PartitioningStrategyConfig configuration. They have been replaced with get/setPartitioningStrategy(). #15730

  • Removed the deprecated get/setSyncBackupCount() methods of MultiMap configuration. They have been replaced with get/setBackupCount(). #15720

  • Removed the deprecated get/setServiceImpl() methods of service configuration. They have been replaced with get/setImplementation(). #15680

  • Removed the connection-attempt-period and connection-attempt-limit configuration elements. Instead, the elements of connection-retry are now used. #15675

  • Renamed MapAttributeConfig as AttributeConfig. Also, its extractor field is renamed as extractorClassName. #15548

  • Improved the index configuration API so that now you can specify the name of the index. Also, instead of boolean type, you can use index type enumeration. #15537

  • Renamed the group-name configuration element as cluster-name and removed the GroupConfig class. #15540

  • Removed the deprecated configuration parameters from Replicated Map, i.e., concurrency-level and replication-delay-millis. #15404

  • Removed the deprecated configuration parameters from the Near Cache configuration. #15313

  • Moved the Event Journal configuration inside the map/cache configuration. Before, it was configured as a parent-level element. #15185

  • Moved the Merkle tree configuration under map configuration. #15180

  • Removed the XSDs for Hazelcast IMDG 3.x versions. #15177

  • Removed deprecated client configuration methods such as isInsideAws() and newAliasedDiscoveryConfig(). #15012

  • Removed the hazelcast.executionservice.taskscheduler.remove.oncancel system property and related methods. #14998

  • Changed the non-space-string XSD type to collapse all whitespaces, so they are handled correctly in the declarative Hazelcast IMDG configuration files. #14919

Management Center
  • Scripting is now disabled by default for both Hazelcast IMDG Open Source and Enterprise editions. Previously, it was disabled only for the Enterprise edition. #16526

  • Removed all the codes providing HTTP based communications between Hazelcast Management Center and Hazelcast IMDG. Therefore:

    • Removed the MCMutualAuthConfig class.

    • Removed the enabled, url, mutualAuthConfig, and updateInterval fields from the ManagementCenterConfig class.

    • Declarative XML configuration simply looks like the following:

      <management-center scripting-enabled="true|false"/>
    • Declarative YAML configuration simply looks like the following:

      management-center
        scripting-enabled: true|false
    • Related REST API changes are as follows:

      • Removed the /hazelcast/rest/mancenter/changeurl endpoint.

      • Renamed /hazelcast/rest/mancenter/security/permissions as /hazelcast/rest/management/security/permissions.

      • Renamed the /hazelcast/rest/mancenter/wan/* endpoints as /hazelcast/rest/wan/*.

      • Removed the legacy /hazelcast/rest/mancenter/clearWanQueues alternative URL in favor of /hazelcast/rest/wan/clearWanQueues.

WAN Replication
  • Aligned the naming of WAN classes, interfaces and getters/setters. Some examples are listed below:

    • WanReplicationPublisherWanPublisher

    • WanReplicationConsumerWanConsumer

    • WanReplicationEventWanEvent

    • WanBatchPublisherConfigWanBatchPublisherConfig

    • WanCustomPublisherConfigWanCustomPublisherConfig

      See #16174 for all the changes.

  • Cleaned up the WAN publisher SPI to make it easier to implement integration between map/cache entry mutation and an external system.

  • Replaced the WAN prefix of classes with Wan for the sake of naming consistencies. #15571

  • Separated WAN private and public classes into different packages. #15195

Split-Brain Protection and Split-Brain Merge
  • Removed the dependencies on Data from the SplitBrainMergePolicy API:

    • The newly introduced getRawValue/Key methods (which supersede the old getValue/Key) in MergingValue/MergingEntry classes return the in-memory representation as OBJECT. The deserialized value can be obtained using getDeserializedValue/Key.

    • The merge types in SplitBrainMergeTypes no longer depend on Data. Also, the value type has been removed from the various "view" interfaces such as MergingHits, MergingCreationTime, etc.

    • The new marker super-interface MergingView has been introduced that all the "view" interfaces (including MergingValue) now extend.

    • The generic type signature of SplitBrainMergePolicy has been changed to specify the (deserialized) type of the merging value.

  • Introduced "split brain protection" concept to replace "quorum" to make it more explicit and unambiguous. Classes and configuration elements including the term "quorum" has been replaced by "splitbrainprotection". #15444

  • Renamed the isMinimumClusterSizeSatisfied() method as hasMinimumSize(). #15554

  • Removed the legacy merge policies specific to a data structure in favour of generic merge policies.

    • PASS_THROUGH

    • PUT_IF_ABSENT

    • HIGHER_HITS

    • LATEST_ACCESS

Serialization
  • Now, Data and SerializationService are not exposed in ObjectDataOutput/Input and ObjectDataInput, respectively. #16064

  • Since SerializationService is now an internal API, the implementations of ObjectDataOutput make use of SerializationServiceSupport where serialization service is needed in the user customizations. #16046

  • Added support for the following default Java serializers for collections:

    • ArrayDeque

    • HashSet

    • TreeSet

    • TreeMap

    • LinkedHashSet

    • LinkedHashMap

    • LinkedBlockingQueue

    • ArrayBlockingQueue

    • PriorityBlockingQueue

    • DelayQueue

    • SynchronousQueue

    • LinkedBlockingDeque

    • LinkedTransferQueue

    • CopyOnWriteArrayList

    • CopyOnWriteArraySet

    • ConcurrentSkipListSet

    • ConcurrentHashMap

    • ConcurrentSkipListMap

    • Map.Entry

    • PriorityQueue

REST
  • Performed the following cleanups:

    • Made all the HTTP status codes (including 200) to return a response body.

    • The exception handling now always returns a HTTP 500 for an error.

    • HTTP 400 is NOT returned now if any handler throws an IndexOutOfBoundsException.

  • Aligned the output format of the REST API to return JSON:

    • Changed the output format of the healthcheck and cluster URIs to return JSON since the other URIs already return JSON.

    • Now all the POST handlers use the checkCredentials() method since it handles the case when there is no data sent.

    • Now all the handlers use the common prepareResponse() method which prepares the response for different response types appropriately.

    • Expanded the return value of the cluster URI to return an array with JSON objects for each cluster member so you do not need to parse the member list but keep the list as a separate value.

    • Added credentials checks to the WAN URIs.

  • Changed the application/javascript "Content-Type" header used by REST API to respond to the JSON documents. Now, it uses application/json. #14972

Distribution Package Changes
  • Merged the client module into the core module: All the classes in the hazelcast-client module have been moved to hazelcast. hazelcast-client.jar will not be created anymore. #15366

Query Engine API
  • The Predicate API has been cleaned up to eliminate exposing internal interfaces and classes. The end result is that the public Predicate API provides only interfaces (Predicate, PagingPredicate, and PartitionPredicate) with no dependencies on internal APIs. #15142

  • Converted Projection to a functional interface so that it has become lambda friendly. #15204

  • Converted the Aggregator abstract class to an interface. #15764

  • Converted the following custom query attribute abstract classes to functional interfaces so that they have become lambda friendly.

    • ArgumentParser

    • ValueCallback

    • ValueCollector

    • ValueReader

    • ValueExtractor

API Package/Interface Changes
  • Relocated the following classes:

    • com.hazelcast.monitor.LocalQueueStatscom.hazelcast.collection.LocalQueueStats

    • com.hazelcast.monitor.LocalExecutorStatscom.hazelcast.executor.LocalExecutorStats

    • com.hazelcast.monitor.LocalInstanceStatscom.hazelcast.instance.LocalInstanceStats

    • com.hazelcast.internal.management.JsonSerializablecom.hazelcast.json.internal.JsonSerializable

    • com.hazelcast.monitor.LocalMapStatscom.hazelcast.map.LocalMapStats

    • com.hazelcast.monitor.LocalMultiMapStatscom.hazelcast.multimap.LocalMultiMapStats

    • com.hazelcast.monitor.NearCacheStatscom.hazelcast.nearcache.NearCacheStats

    • com.hazelcast.monitor.LocalReplicatedMapStatscom.hazelcast.replicatedmap.LocalReplicatedMapStats

    • com.hazelcast.monitor.LocalTopicStatscom.hazelcast.topic.LocalTopicStats

  • Moved the getXaResource() method from the TransactionContext class to HazelcastInstance. #15728

  • Moved various private classes to internal packages. #15569, #15570, #15588, #15599, #15603, #15616, #15171, #15151, #15146, #15145, #15129, #15124, #15123, #15122, #15121, #15888, #15887, #15888

  • The APIs that returned UUID string now returns UUID. These include Endpoint.getUUID, listener registrations/deregistrations, keys of replica timestamps of VectorClock, UUIDs in the executor service, UUID in the MigrationInfo, cluster ID and transaction ID. #15473

  • Removed ICompletableFuture which was a replacement for the missing JDK 8’s CompletableFuture. Now, it has been replaced by CompletionStage. See #15441 for more details.

  • Removed the usage of com.hazelcast.core.IBifunction, replaced it with java.util.function.Bifunction. #15201

  • Renamed the getId method of IdentifiedDataSerializable to getClassId. #15127

  • Made the EntryProcessor interface lambda friendly. #14995

  • Removed the LegacyAsyncMap interface. #14994

  • Removed the support for primitives for setAttribute and getAttribute member attributes. All member attributes support only String attributes now. #14974

  • Removed the java.util.function back ports. #14912

  • CacheService now implements StatisticsAwareService #14904

  • Renamed the class to start a Hazelcast member from com.hazelcast.core.server.StartServer to com.hazelcast.core.server.HazelcastMemberStarter. #12791

  • The packages of the following classes have been changed:

    Classes

    Package Before IMDG 4.0

    Package After IMDG 4.0

    Details

    LdapLoginModule, BasicLdapLoginModule

    com.hazelcast.security.impl

    com.hazelcast.security.loginimpl

    EventJournalMapEvent, EventJournalCacheEvent

    • com.hazelcast.map.impl.journal

    • com.hazelcast.cache.impl.journal

    • com.hazelcast.map

    • com.hazelcast.cache

    All private classes

    • com.hazelcast.client.config

    • com.hazelcast.config

    • com.hazelcast.spi.partition

    • com.hazelcast.map.journal

    • com.hazelcast.query.extractor

    • com.hazelcast.client.config.impl

    • com.hazelcast.internal.config

    • com.hazelcast.internal.partition

    • com.hazelcast.map.impl.journal

    • com.hazelcast.query.impl

    All classes

    com.hazelcast.internal.util.function

    com.hazelcast.function

    WanPublisherState

    com.hazelcast.config

    com.hazelcast.wan

    All private classes

    com.hazelcast.spi.hotrestart

    com.hazelcast.internal.hotrestart

    All private NIO and serialization classes

    com.hazelcast.nio

    com.hazelcast.internal.nio

    All private YAML, CRDT and memory classes

    • com.hazelcast.config.yaml

    • com.hazelcast.crdt

    • com.hazelcast.memory

    • com.hazelcast.elastic

    • com.hazelcast.internal.config.yaml

    • com.hazelcast.internal.crdt

    • com.hazelcast.internal.memory

    • com.hazelcast.internal.elastic

    All

    com.hazelcast.util

    com.hazelcast.internal.util

    SerializationService

    com.hazelcast.spi.serialization

    com.hazelcast.internal.serialization

    Private client classes

    • client.connection

    • client.proxy

    • client.spi.properties

    • client.spi

    • client.util.ClientDelegatingFuture

    • client.api

    • client.impl.connection

    • client.impl.proxy

    • client.properties

    • client.impl.spi

    • client.impl.ClientDelegatingFuture

    • client

    Joiner and TcpIpJoiner

    com.hazelcast.cluster and com.hazelcast.cluster.impl

    com.hazelcast.internal.cluster and com.hazelcast.internal.cluster.impl

    All IExecutor classes

    com.hazelcast.core

    com.hazelcast.executor

    Address

    com.hazelcast.nio

    com.hazelcast.cluster

    ClassNameFilter, SerializationClassNameFilter

    com.hazelcast.nio

    com.hazelcast.nio.serialization

    All IMap classes

    com.hazelcast.core

    com.hazelcast.map

    ReplicatedMap

    com.hazelcast.core

    com.hazelcast.replicatedmap

    IAtomicLong, IAtomicReference, ILock, ICondition, ISemaphore, ICountDownLatch

    com.hazelcast.core

    com.hazelcast.cp

    IndexAwarePredicate, VisitablePredicate, SqlPredicate/Parser, TruePredicate

    com.hazelcast.query

    com.hazelcast.query.impl.predicates

    Transaction collection classes (TransactionalMap, TransactionalList, etc.)

    com.hazelcast.core

    com.hazelcast.transaction

    IQueue, QueueStore, IList, ISet, ItemEvent, ItemListener

    com.hazelcast.core

    com.hazelcast.collection

    MultiMap

    com.hazelcast.core

    com.hazelcast.multimap

    ITopic, Message, MessageListener

    com.hazelcast.core

    com.hazelcast.topic

    RingbufferStoreFactory, RingbufferStore

    com.hazelcast.core

    com.hazelcast.ringbuffer

    Operation classes

    com.hazelcast.spi

    com.hazelcast.spi.impl.operationservice

    Partition SPI classes

    com.hazelcast.spi

    com.hazelcast.spi.partition

    Member and membership classes (Cluster, Member, etc.)

    com.hazelcast.core

    com.hazelcast.cluster

    Client classes (Client, ClientService, etc.)

    com.hazelcast.core

    com.hazelcast.client.api

    Partition classes

    com.hazelcast.core

    com.hazelcast.partition

Enhancements

Hazelcast IMDG Enterprise Enhancements:

  • Separating WAN Publisher Configuration: The previously known wan-publisher (or WanPublisherConfig) has been separated into two configuration elements to be used for built-in and custom WAN publishers:

  • Tracking WAN Synchronization Events: Management Center, logs and diagnostics now report the progress of a WAN synchronization. #15221

  • Improvements in the JAAS Authentication Mechanism:

    • Aligned with JAAS best practices.

    • Added support for the standard JAAS callbacks, i.e., NameCallback and PasswordCallback

    • Avoided automatic deserialization of custom credentials in the client protocol.

    • Introduce the concept of security roles to distinguish between (a single) connecting side identity and its privileges.

    • Used Credentials object only for authentication to prevent secrets leaks.

    • Cleaned up the Credentials interface.

  • Improvements in the Security Configuration:

    • Replaced the <group> configuration by simple <cluster-name>.

    • Removed the group-password configuration.

    • Introduced the concept of security realms on the members. #15651

    • Added typed authentication and identity configuration (e.g. <ldap> authentication, <token> identity)

    • Used similar identity configuration in client config

  • Javadoc for the Hazelcast IMDG Enterprise Edition: Added Javadoc JAR file for the Hazelcast IMDG Enterprise Edition. See the Getting Started section

Hazelcast IMDG Open Source Enhancements:

  • HTTPS for Scripts: Enabled the REST endpoints for the cluster.sh and healthcheck.sh scripts to use HTTPS. Before, they were using HTTP. See the cluster.sh and healthcheck.sh sections.

  • MapLoader with Custom time-to-live: Introduced the EntryStore and EntryLoader interfaces (adding expiration support for MapStore). See the Loading and Storing Persistent Data section.

  • Moby Naming: Introduced friendly names for the Hazelcast instances to be shown in the Hazelcast Management Center. See the Checking the Instances' Name section. Also, a new system property, hazelcast.member.naming.moby.enabled, is introduced for this purpose.

  • Improved Client Performance: Introduced a way to eliminate the sync backup wait from the client and send the backup ACK to the client: the smart clients have been made backup aware and the backups now are redirected to the client directly from the backup members. See the Configuring Backup Acknowledgment section.

  • Ownerless Client: Previously, the clients had an owner member responsible for cleaning up their resources after they leave the cluster. Also, the ownership information was needed to be replicated to the whole cluster when a client joins the cluster. This mechanism have been made simpler by introducing the following system properties to control the lifecycle of the clients on the member side:

  • Single Thread Hazelcast Clients Performance: Hazelcast clients have been designed to be used by multiple threads; the more threads you throw at it, the better the performance (until it is saturated). Now, it has also been optimized for a single thread doing requests: The default values for the hazelcast.client.io.write.through and hazelcast.client.response.thread.dynamic have been changed from false to true.

  • JSON Support for REST: Added support for HazelcastJsonValue over REST API: When a HazelcastJsonValue is requested, now a UTF8 encoded JSON value is returned. The response has the JSON string in the payload and "Content-Type" header is set to application/json. #15017

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Removed the following duplicated MBeans since the metrics MBean have been exposed:

    • ClientEngineMBean

    • EventServiceMBean

    • NetworkingServiceMBean

    • OperationServiceMBean

    • ProxyServiceMBean

  • Renamed the WanReplicationRef.setMergePolicy() method as setMergePolicyClassName() and made the PassThroughMergePolicy the default merge policy for WAN replication if none is specified. #16403

  • Added convenience constructor for SpringManagedContext to easily create it in the programmatic way. #16401

  • Added support for AWS PrivateLink. Now, Hazelcast IMDG Java client can work with Hazelcast Cloud when it uses AWS PrivateLink. #16371

  • Added the getOrCreate() method to the client configuration to fix the issue with setInstanceName() when using Spring Boot and Hazelcast client. #16362

  • Improved the Ringbuffer data structure so that it does not throw StaleSequenceException when using ReadManyOperation. #16303

  • Removed the shortened mancenter phrase from the source code. #16282

  • Removed the client side user executor and related configuration, i.e., executor-pool-size. #16215

  • Added the following client operations related to CP subsystem, Hot Restart and WAN replication to be used by Management Center:

    • getCPMembers

    • promoteToCPMember

    • removeCPMember

    • resetCPSubsystem

    • triggerPartialStart

    • triggerForceStart

    • triggerHotRestartBackup

    • interruptHotRestartBackup

    • changeWanReplicationState

    • clearWanQueues

    • addWanReplicationConfig

    • wanSyncMap

    • checkWanConsistency

  • Added the support for yml extension, in addition to yaml, for the Hazelcast configuration locator. #16205

  • Improved the IMap.putAll() and IMap.put() behaviors so that they match when they trigger listener events. #16144

  • Added option to disable retrieving the OSMBean.getFreePhysicalMemorySize() method. #16039

  • The recreateCachesOnCluster invocation is not being checked for the maximum invocations count anymore during cluster restarts. #16026

  • Introduced a special Java client type to be used by Management Center. #16006

  • Removed the PID management from the IMDG start and stop scripts. You can now start multiple Hazelcast instances, when using the start script, without the need to create another copy of the /bin directory, i.e., it now allows running multiple processes. #15934

  • Added the cache statistics to the dynamically collected metrics #15926

  • Removed fail-on-maxbackoff element in the connection retry configuration and added cluster-connect-timeout-millis instead to allow retrying with fixed amount of time and shutdown after some time. #15923

  • Introduced cluster fail-fast when there are missing security realms. #15872

  • Added ConnectionRetryConfig to ClientConfigXmlGenerator. #15821

  • Renamed the restart() method of CPSubsystemManagementService to reset(). #15798

  • Unified the IMap and ICache eviction configurations to decrease the configuration complexity. #15592

  • Introduced dynamic metric collection. Previously, Hazelcast metrics were reported programmatically to the Hazelcast Management Center, one by one. Introducing new metrics required changes both in IMDG and in MC, which limited the number of metrics sent to MC. In 4.0 this has been changed to collecting and reporting all available metrics dynamically just by declaring them in IMDG. Besides reporting the metrics dynamically to MC exposing them on JMX is done dynamically as well. Both reporting to MC and exposing on JMX are toggleable by using the metric configuration element introduced in 4.0.

  • Set the log level to FINEST for PartitionMigratingException. #15577

  • Added the support for nested JSON objects in arrays. #15425

  • To be shown on Management Center, the clients now send both its IP address and canonical hostname. Before, only the hostname of the client was shown. #15421

  • Added a new implementation of SecondsBasedEntryTaskScheduler for the FOR_EACH mode to improve the performance of TransactionContext.commit(). #15414

  • Added a level of memory protection to the Hazelcast client protocol: untrusted connections (the ones which haven’t finished authentication yet) do not accept fragmented messages; they check the frame size against a configurable limit. #15396

  • Made the Hazelcast specific root nodes in the YAML configurations optional. #15389

  • Updated the JavaVersion class to support JDK 13 and 14 builds. #15372

  • Added support for updating the licenses of all the running members of a Hazelcast IMDG cluster using the REST API. #15370

  • Introduced configuration of initial permits for CP subsystem semaphore. #15208

  • Added support for null keys for the client side implementations of IMap.addEntryListener(). #15155

  • Improved the generics for the API with Projection, Predicate and EntryListener by adding lower bounded wildcards to accept a wider range of parameters. #15153

  • Improved the performance of TransactionLog.add() by avoiding the LinkedList.remove() call. #15111

  • Made ClientConfig to override toString as it is the situation with Config to make it easier to troubleshoot. #15061

  • Added the full example configuration files (XML and YAML) for the Hazelcast Java client. #15056

  • Introduced functional and serializable interfaces having a single abstract method which declares a checked exception. The interfaces are also serializable and can be readily used in the IMDG API when providing a lambda which is then serialized. #14993

  • Enhanced the queries (read-only operations) in the CP Subsystem so that they are executed with linearizability but they are not appended to the Raft log. By this way, the grow of Raft logs and snapshots of read-only operations are prevented, leading to throughput improvement #14986

  • Improved the WAN feature so that now lazy deserialization is used when merging entries received via WAN. Otherwise, the unconditional deserialization was causing overhead. #14982

  • Updated the following packages to Java 8 and removed the 3.x rolling upgrade compatibility paths: cache, MultiMap, cluster, partition, WAN replication, CP subsystem, Hot Restart. #14896

  • Added the support for Java 8 Optionals in the queries. #14827

  • Fixed the Javadoc markup issues. #14971

  • Updated the Hazelcast Kubernetes dependency to version 1.5. #14898

  • Cleaned up the Maven repositories in Hazelcast’s pom.xml to simplify the usage of Maven proxies. #14850

  • Updated the web session manager dependency to its latest version. #14822

  • Separated the statistics for IMap.set() and IMap.put() methods. #14811

  • Introduced a warning log for illegal reflective access operation when using Java 9 and higher, and OpenJ9. #14798

  • Added a method to easily identify when all replicas of a partition have been lost: allReplicasInPartitionLost() #11983

  • Changed the Scheduled Executor’s capacity value from "per partition" to "per member". #11629

  • Improved the fluent interface of configuration classes by adding the return this statements to the setter methods. #11107

  • Aligned the put mechanism for IMap and ICache: As in ICache, now the put operations without time-to-live (TTL) in IMap makes an entry either to live forever or use the TTL in the map’s configuration (if configured). #10965

  • Added support for falling back to a "default" configuration for the cache data structure. #10695

Fixes

  • Fixed an issue where disabling the quorum had not an effect and was still checking the presence of split-brain protection. #16510

  • Fixed an issue where the Imap.containsKey() method was not able to find Near Cached entries when it is called from a client. #16462

  • Fixed an issue where the serializable singleton comparators for natural and reverse order was creating new instances on deserialization. #16439

  • Fixed the missing client XML/YAML configurations in mvn assembly. #16331

  • Fixed an issue where the Near Cache was not being updated with the new value as soon as putAsync future is completed, when local update policy is CACHE_ON_UPDATE. #16314

  • Fixed an issue where the destruction of a proxy that is not yet initialized was blocking on its construction, leading to the risk of deadlock. #16297

  • Fixed an issue where the MembershipEvent.getMembers() was not returning the cluster member list in the proper order at the time of event (when a new member joins). #16243

  • Fixed an issue where the CP group IDs were not unique for different CP subsystem initializations. #16240

  • Fixed an issue where a REST URI not matching any pattern was returning a response belonging in CLUSTER_WRITE endpoint group. Now, it throws an exception. #16237

  • Fixed the joining mechanism so that when the discovery strategy is enabled, multiple join configurations are prevented. #16177

  • Fixed an issue where the client-side HazelcastInstance was not throwing a configuration exception when there is a conflict between the dynamic and static configurations. #16165

  • Fixed an issue where the configuration objects, that have both implementation/class and name as the configuration, were not equal after (de)serializations. #16156

  • Eliminated the unnecessary iterations and object creations on the bulk client responses. #16138

  • Fixed an issue where repetitive calls of IMap.loadAll() may cause memory leaks. #16096

  • Fixed an issue where Address.equals() and hashCode was using hostname instead of IP addresses. #16075

  • Fixed an issue where a client in CLIENT_DISCONNECTED state was not aware of possible attribute changes in the cluster after its state becomes CLIENT_CONNECTED: Hazelcast now does not allow changing member attributes after restarts. #16168

  • Fixed the inconsistency in Near Cache when using CacheOnUpdate: Normally, a Near Cache is updated with get operations; another option is CacheOnUpdate and when it is enabled, put operations also update the Near Cache. To never miss any invalidation and never read any stale data indefinitely, get based updates use reservations. With this fix, this reservation based solution also applies to the put operations when CacheOnUpdate is configured. #12548

  • Fixed an issue where ProxyManager was not removing Proxy even after the original distributed object is destroyed. #12470

4.0-Beta-2
  • Fixed an issue where an HTTP request via hazelcast/rest/cluster was failing when using the advanced network configuration and a client endpoint is not specified. In this case, this request’s response has been improved to report 0 as the client connection count. #16152

  • Fixed the following Hot Restart issues:

    • Introduced an additional stage to the Hot Restart procedure, i.e., HotRestartIntegrationService.startup(), which waits until all members transition from the PASSIVE state. This guarantees all members to have the same state after Hot Restart operation is finished.

    • IMap proxies created during Hot Restart are not initialized and published to other cluster members. So the operation has been improved to force initialize any uninitialized proxies and publish them. This fixed the issue where the getDistributedObjects() method was not reporting the persisted objects after a Hot Restart.

  • Forced eviction was evicting all the entries regardless of the eviction configuration. This has been fixed: forced eviction now runs only if a map has eviction configured. Otherwise, it does not run and throws native OutOfMemoryException. #16085

  • Renamed the GroupProperty class as ClusterProperty due to the groupcluster term change. #16076

  • Fixed possible statistics miscalculations by checking an entry’s expiration in a Near Cache only if its state is READ_PERMITTED #16067

  • Fixed an issue where a StreamSerializer is added using setTypeClass instead of setTypeClassName; this was not correctly handled. #16047

  • Fixed an issue where some functions may not be working when a client provides a new client type: removed ClientType and ConnectionType enums and introduced free strings for them instead. #16030

  • Fixed a race condition between the new cluster member join and post-join operations executed as part of the concurrent member join. #16020

  • Fixed an issue where an enabled redoOperation() was not throwing an exception when an empty list is tried to be retrieved on the client. #16015

  • Aligned the exception mechanism of CacheManager.createCache() with the getCache() and getCacheManager().getCache() methods of the same class. #16007

  • Fixed an issue where a Raft node may leak and stay in the ACTIVE state after a CP member terminates, because of a race between the Hazelcast member shutdown and Raft node termination logic. #16022

  • Fixed an issue where gathering MultiMap statistics was breaking the split-brain healing with LatestAccessMergePolicy. #16001

  • Fixed an issue where enabling REST API configuration (getRestApiConfig().setEnabled(true)) was throwing NullPointerException since its default value was null. #15981

  • Fixed an issue where the configuration validator was not checking if the maximum size policy is appropriate for the selected in-memory format. #15964

  • Fixed an issue where ManagementCenterService was shutting down itself when it encounters an exception during the creation of TimedMemberState. This was causing the cluster to disappear from Management Center. #15946

  • Fixed an issue in the query operation for offloaded cases. #15944

  • Fixed the cache statistics handling: Previously used Config.findCacheConfig() could only lookup cache configurations added statically or dynamically via Config.addCacheConfig(), but was missing configurations of dynamically created caches via CacheManager.createCache(). Now, CacheService.getCacheConfigs() is used to fix this. #15937

  • Fixed an issue where an exception thrown from a dynamic metric provider was stopping the dynamic metric collector task. #15932

  • Fixed an issue where the map and Replicated Map in a client share the same near cache when they have identical names. #15912

  • Fixed the extensive Overwriting existing probe logs when starting a Hazelcast member. #15910

  • Fixed the InvocationTargetException thrown by the metrics service on JDK 11. #15884

  • Fixed an issue where tcp.connection.clientCount was collected and published twice. #15883

  • Fixed an issue where the client connection count was retrieved using an incorrect method. #15861

  • Fixed an issue where calling the IMap.removeAll() method without index was updating the last access and expiry time for all records. #15850

  • Fixed the consistency issue between the configuration replacers and XML configuration imports. #15810

  • Fixed a configuration failure with YAML for composite key indexes. #15806

  • Fixed an issue where Predicates.ilike() was not working for Cyrillic strings. #15748

  • Fixed an issue where the gauges could not be created from the dynamic metrics. #15718

  • Fixed an issue where the client’s Near Cache was not being invalidated after the IMap.executeOnKeys() method is called. #15468

  • Fixed the inconsistent behavior for sending a null message via Topic.publish() on the members and clients. Now, the client side also is not allowed to send it. #15338

  • Made the public createCachingProvider() method private since its class, HazelcastServerCachingProvider, is a private one. #15144

  • Fixed an issue where the client.getDistributedObjects() method may cause recreation of the destroyed objects. #14571

  • Fixed an issue where the query cache was missing key and value information for entries. #13423

4.0-Beta-1
  • Fixed an issue where a new CP member could create the Raft nodes before its local CP member field is not initialized yet, when it is being promoted. This could create non-determinism issues for CP groups relying on the local CP member information. #15803

  • Fixed an issue where the CompletableFuture#defaultExecutor() method caused compilation failure on JDK 9 due to the "protected" access. #15702

  • Fixed a race issue by initializing the local CP members before initializing the metadata group. #15684

  • Fixed an issue where the CP subsystems' restart operations were not being canceled and waiting the running/scheduled discovery tasks. #15567

  • Fixed an issue where the executor service message task was blocking the partition thread. #15522

  • Fixed an issue where the used memory in metrics was becoming a negative value. #15485

  • Moved the checkWanReplicationQueues operation from the caller side to the callee. #15412

  • Fixed an issue where the map configuration options readBackupData and statisticsEnabled were not being respected when a new MapConfig is dynamically added from a client to a running Hazelcast cluster. #15382

  • Fixed an issue where the comparators were not able to act on both keys and values. A custom paging predicate comparator may act on keys and values at the same time even if only the keys are requested, e.g., using the IMap.keySet() method. Before this fix only the keys were fetched for this method, making comparators unable to act on values. #15380

  • Optimized the shutdown for on-heap indexes: These indexes are cleaned on shutdown and the index entries are removed one by one. For large indexes, e.g., for array/collection attribute indexes, this was taking a considerable amount of time. #15340

  • Fixed the deserialization filtering for Externalizables and Deadlock in the map index. The deserialization filter was not properly protecting against the vulnerable Externalizable classes. The filtering has been extended. #15358

  • Fixed an issue where the named scheduled tasks was not respecting the HazelcastInstanceAware marker. #15352

  • Fixed a possible NullPointerException for the remove-if-same map operation. #15344

  • Fixed an issue where storing MapStore instances in MapStoreConfig could cause member failures when the configuration is added dynamically. #15224

  • Fixed a NullPointerException in the query caches by setting the publisher-listener-id if a query cache has already one. #15215

  • Fixed an issue where SimpleTokenCredentials could not be deserialized due to the missing handling in SpiPortableHook. #15196

  • Fixed an issue where the commit phase of transactional maps was not checking the member-wide upper limit for the entries in write behind queues. #15186

  • Fixed an issue where the queries like labels[any] = 0 and labels[any] = 1 were optimized only to false since labels[any] was interpreted as a regular attribute name having a single value. #15163

  • For on-heap indexes, fixed an issue where a record’s lastAccessTime was not updated when it is being accessed through an index. Now, this way, the expiration maxIdle mechanism takes this into account. #15136

  • Fixed an issue where ExecutorServiceProxy was unnecessarily serializing the same task multiple times before submitting it to multiple members. #15069

  • Added the missing user code deployment section to the configuration which is sent to Management Center. #15044

  • Fixed an issue where two client listeners are not registered since they listen on a single connection (not cluster wide listeners) by adding cleanups for them. #15041

  • Fixed the authentication mechanism between the clients and members by adding a check to prevent re-verification while the client is changing its owner member. #15030

  • Added support for the missing aliased discovery strategies, e.g., gcp and kubernetes, to ClientConfigXmlGenerator. #15010

  • Fixed an issue where the client user code deployment was becoming non-operational when assertions are enabled. #15006

  • Some operations such as heartbeat checks and partition migrations share common threads with the client login module. In case of the long running client login module implementations, some symptoms such as split brain syndrome can be seen. This has been fixed by introducing a blocking executor which is used only for the client authentications. #14956

  • Fixed an issue where the IMap.removeInterceptor() method was returning void. #14955

  • Removed the entryEvicted event from the event firing mechanism in the case of eviction. Before, both entryEvicted and entryExpired events were being fired. #14954

  • Fixed an issue where the Hazelcast IMDG configuration files, that have an extension other than .xml, .yaml or .yml or do not have an extension, were ignored silently. This was happening when the configuration file is set by using the hazelcast.config system property. #14953

  • Fixed an issue where the client was not considering the new address of a restarted member, which has the same UUID but could have a different IP address after it is restarted. #14842

  • Fixed an issue where the migration operations were running before the previous finalization is completed. #14832, #16189

  • Fixed an issue where the outbound pipeline was not waking up properly after various optimizations for write-through persistence is made. #14831

  • Fixed an issue caused by the cache being not ready to be used immediately after the cache proxy was created. #14821

  • Fixed an issue where the performance of IMap.values() was low when using PartitionPredicate. Also, PartitionPredicate was not respecting indexes. So, now global indexes are used for partition queries. #14814

  • Fixed a performance issue where there were unneeded iterations and object creations while converting the client messages to user objects. #13784

  • Fixed an issue where the locked entries with a time-to-live were not evicted. With this fix, the lock operation checks if an entry has already expired. #13272

  • Fixed an issue where there were excessive amount of logs on the target cluster when cache config is missing for the WAN replication. #12826

  • Fixed an issue where there was an inconsistent removeIf behavior among the collection views of IMap. #12198

  • Fixed a leak in the query cache due to ListenerRegistrationHelper, which has been removed with this fix. #11914

  • Fixed an issue where the IMap.replace() method was not loading entries from the MapLoader when the keys could not be found in the memory. #11300

Removed Features

4.0-Beta-2
  • Deprecated the following properties:

    • hazelcast.client.statistics.period.seconds

    • hazelcast.client.statistics.enabled

  • Removed the deprecated SimpleEntryView.evictionCriteriaNumber() method. #15846

  • Removed the deprecated IMap methods accepting EntryListener. This has been replaced with MapListener. #15781

4.0-Beta-1
  • Removed the User Defined Services (Hazelcast’s SPI) feature. #15403, #15401

  • Removed the setLicenseKey() method of ClientConfig.

  • Removed the methods in the Member and AddressPicker classes.

  • Removed the deprecated diagnostics property names.

  • Removed the deprecated EvictionPolicyType class. Instead, use the enhanced EvictionPolicy class.

  • Removed the legacy IdGenerator interface. Instead, FlakeIdGenerator has been used.

  • Removed the deprecated AsyncAtomicLong and AsyncAtomicReference classes.

  • Removed the deprecated cache eviction configurations.

  • Removed the MapReduce feature.

  • Removed the deprecated LOCAL transaction type (TransactionType.LOCAL)

  • Removed the deprecated optimizeQueries map configuration parameter.

  • Removed the following deprecated system properties:

    • hazelcast.version.check.enabled

    • hazelcast.icmp.enabled

    • hazelcast.icmp.parallel.mode

    • hazelcast.icmp.echo.fail.fast.on.startup

    • hazelcast.icmp.timeout

    • hazelcast.icmp.interval

    • hazelcast.icmp.max.attempts

    • hazelcast.icmp.ttl

    • hazelcast.mc.url.change.enabled

    • hazelcast.slow.invocation.detector.threshold.millis

  • Removed the legacy AtomicLong and deprecated IdGenerator implementations. #15601

  • Removed the legacy ILock implementation and the HazelcastInstance.getLock() method. Instead we provide the unsafe mode of CP Subsystem’s FencedLock. The ICondition is not supported anymore. #15625

  • Removed the legacy AtomicReference implementation and the HazelcastInstance.getAtomicReference() method. Instead we provide the unsafe mode for all CP data structures. #15593

  • Removed the legacy Semaphore implementation and the HazelcastInstance.getSemaphore() method. Instead we provide the unsafe mode for all CP data structures. #15539

  • Removed the legacy CountdownLatch implementation and the HazelcastInstance.getCountdownLatch() method. Instead we provide the unsafe mode for all CP data structures. #15538

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.12.13

Enhancements

  • Improved connection handling. #21885

Fixes

  • Fixed a split-brain recovery issue where the split clusters could not merge due to missing master member address. #21860

  • Fixed an issue that occurred when there is Near Cache configured for both the member and client sides for the same map and serialize-keys option is false. #18322

  • Fixed an issue where the Java client was not receiving membership events in its membership listener when a member with Hot Restart Persistence enabled is restarted. #18271

  • The difference between the Near Cache configurations before and after a client is recreated with the same client failover configuration is fixed now. #18168

  • Fixed an issue that prohibited Hazelcast from being used as Tomcat session manager when it is also deployed in a web application context: When a client application that uses Hazelcast tries to connect to the Hazelcast cluster and this application’s web sessions are persisted using Hazelcast’s Tomcat session manager, the session manager could not connect to the cluster. This has been fixed by improving Hazelcast’s service loader mechanism. #18119

3.12.12

Starting with Hazelcast IMDG 3.12, the Hazelcast IMDG download package does not include the documentation; please see https://docs.hazelcast.com/home/index.html.

Enhancements

  • Updated the version of Hazelcast Kubernetes plugin to 1.5.5. #17927

Fixes

  • When the in-memory format of a map is NATIVE and the uploaded user code has missing classes (in case the user code deployment feature is used), the resulting exception could not be seen on the client side when a map query is run. This was causing the client to hang indefinitely and fixed by improving the failure handling for this case. #18052

3.12.11

New Features

  • Node Aware Partition Grouping: Added the support of partition grouping mechanism in the Hazelcast discovery plugin for Kubernetes. You can create partition groups according to the name of the node which is provided by this plugin and on which the containers/pods run. See the NODE_AWARE section of the IMDG Reference Manual for more information.

Enhancements

  • Added protection against XML External Entity attacks (XXE). #17754

  • Introduced a configuration property to ignore errors during enabling the XXE protection. This protection works with JAXP 1.5 (Java 7 Update 40) and newer. When an older JAXP implementation is added to the classpath, e.g., Xerces and Xalan, an exception is thrown. The newly introduced property, namely hazelcast.ignoreXxeProtectionFailures, allows you to ignore those exceptions. See the System Properties appendix of the IMDG Reference Manual for more information.

Fixes

  • Fixed an issue where the Javadoc of release methods for session-aware semaphore structure was incorrectly addressing "threads" instead of "Hazelcast instances". #17828

  • Fixed a data loss issue that was happening while promoting a lite member to a data member. #17758

3.12.10

Enhancements

  • Added the support for checking duplicate fields when building class definitions. #17683

  • Enhanced the tracking of missing CP subsystem members after a split-brain merge situation. #17634

  • Added the missing validation checks to the client’s connection retry configuration. #17570

  • Upgraded the versions of Hazelcast’s snakeyaml-engine and Jackson library dependencies to eliminate the denial of service vulnerabilities. #17425, #17444

  • Added timeout values for connect and read operations to the Management Center service’s task connection method. #17387

Fixes

  • Fixed an issue where Hazelcast was not releasing the acquired lock sessions that fail for the reasons other than session expiration and wait key cancellation. #17701

  • Fixed an issue where the lock requests in clients' maps with infinite time lease were returning Long.MAX_VALUE; now it returns -1. #17680

  • Fixed an issue where Hazelcast was returning false if a client is successfully deregistered from any member, but events are still delivered for the non-deregistered ones. #17648

  • Fixed the member side user code deployment; it was throwing an exception when inner classes are used to be loaded. #17579

  • Fixed an issue where the diagnostic plugins were not being rendered properly when they fail. #17500

  • Fixed an issue where the load balancer for clients could not be configured declaratively. #17228

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.12.9

Enhancements

  • Updated the hazelcast-kubernetes dependency version to 1.5.4. #17256

  • Added support for tenant control when creating caches. #17121

Fixes

  • Fixed a race issue when creating a cache (JCache) using multiple methods with the same cache name but having different keys. #17284

  • Fixed an issue where the repeated calls of executeOnKeys() in Hazelcast clients for NATIVE maps was causing a continuous increase in the used memory and exhaustion of the memory pool after a while. #17271

  • Fixed a performance issue when using paging predicates with JDK 8. #17213

  • Fixed an issue where ReliableTopicMessageListener was firing a warning when the client is shutting down. #17070

  • Fixed an issue where the client-side HazelcastInstance was not throwing a configuration exception when there is a conflict between the dynamic and static configurations. #16927

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.12.8

Enhancements

  • Introduced WAN throttling mechanism: this prevents the WAN consumers from getting overloaded by the WAN producers if transferring the WAN events takes less time than processing them in the target cluster. This can be the case with WAN synchronization if the network latency is low enough. It can be configured using the following properties:

    • hazelcast.wan.consumer.invocation.threshold

    • hazelcast.wan.consumer.ack.delay.backoff.init

    • hazelcast.wan.consumer.ack.delay.backoff.max

    • hazelcast.wan.consumer.ack.delay.backoff.multiplier

      See here for their explanations.

Fixes

  • Fixed an issue where the CP client message tasks were deserializing the responses. #17050

  • Fixed an issue where the split-brain protection events were triggered during the startup of members, but before they join the cluster. With this fix, these events will not be fired until the minimum cluster size (quorum) is met after the member startups. #17038

  • Fixed an issue where notifying the partition table updates to the clients by a member was causing latencies. #16937

  • Fixed an issue in the cache service where its pre-join operation was considering CacheConfig as resolved: it was assuming that key/value types, user customizations and other cache configurations have been loaded. This was an issue when the cache is not touched yet. #16919

3.12.7

Enhancements

  • Added a public API to load configuration files: the new load() method in Config, ClientConfig, and ClientFailoverConfig loads the configurations with given locations. If not found, the default configuration is returned. #16864

  • Updated the version of Hazelcast Kubernetes discovery plugin to 1.5.3. #16589

  • Added the support for automatically detecting the classloader of a type for the user code deployment feature. #16421

Fixes

  • Fixed an issue where Map, Cache, MultiMap data structures were returning negative values (size()) when the size is more than Integer.MAX_VALUE. #16897

  • Fixed an issue where explicitly registering portable class definitions was failing with multiple portable factories and overlapping class IDs. #16832

  • Fixed an issue where Reliable Topic was not working properly due to missing permissions. #16773

  • Fixed an issue where the wait key of a blocking call within a Raft invocation was still being reported as a live operation, when the key times out. #16616

  • Fixed an issue where the upload of classes using the client user code deployment were not successful when they are retrieved not in their created order. #16610

  • Fixed an issue where InPredicate was not invoking value comparison when the read attribute is null. #16578

  • Fixed an invalidation issue when using a transactional map from a cache with Near Cache: the cache invalidation event occurs when the transactionalMap.put method is called. As a result, the entry was getting invalidated before the change is committed to the map. #16577

3.12.6

New Features

  • Introduced Bitmap Indexes to significantly lower index memory usage for low-cardinality columns and also to speed up the queries and lower memory requirements for them when the queries have multiple predicates acting on the same bitmap index. Note that bitmap indexes usage, currently, is a good fit for array/collection attribute indexing, rather than regular single-valued attributes. See the Bitmap Indexes section.

Enhancements

  • Added support for AWS PrivateLink: Now, Hazelcast IMDG Java client can work with Hazelcast Cloud when it uses AWS PrivateLink.

Fixes

  • Fixed the map configuration so that now the value of optimize-queries element is not serialized anymore. #16524

  • Fixed an issue where disabling the quorum had not an effect and was still checking the presence of split-brain protection. #16507

  • Fixed an issue where the Imap.containsKey() method was not able to find Near Cached entries when it is called from a client. #16472

  • Fixed a race condition caused by using on-heap indexes while indexing is in progress. #16461

  • Fixed the issue with setInstanceName() when using Spring Boot and Hazelcast client: the getOrCreate() method has been added to the client configuration. #16363

  • Fixed an issue where wildcard usage at CLASSPATH was breaking the stop.bat and start.bat scripts. #16355

  • Fixed an issue where the migration operations were running before the previous finalization is completed. #16190

  • Fixed an issue where repetitive calls of IMap.loadAll() may cause memory leaks. #16172

  • Fixed the joining mechanism so that when the discovery strategy is enabled, multiple join configurations are prevented. #16143

  • Fixed an issue where a client in the CLIENT_DISCONNECTED status was never receiving the changes in the cluster member attributes. #15623

  • Fixed an issue where the MembershipListener.memberAdded() method after a split-brain merge was showing a different member order. #15353

3.12.5

Enhancements

  • Added option to disable retrieving the OSMBean.getFreePhysicalMemorySize() method. #16065

  • The recreateCachesOnCluster invocation is not being checked for the maximum invocations count anymore during cluster restarts. #16014

Fixes

  • Fixed an issue where repetitive calls of IMap.loadAll() may cause memory leaks. #16171

  • Fixed an issue where an HTTP request via hazelcast/rest/cluster was failing when using the advanced network configuration and a client endpoint is not specified. In this case, this request’s response has been improved to report 0 as the client connection count. #16150

  • Fixed an issue where an enabled redoOperation() was not throwing an exception when an empty list is tried to be retrieved on the client. #16019

  • Fixed an issue where gathering MultiMap statistics was breaking the split-brain healing with LatestAccessMergePolicy. #16003

  • Fixed the multiplier element of client’s connection retry configuration to be set as a double in the XML configuration (hazelcast-client.xml). #15990

  • Fixed the following Hot Restart issues:

    • Introduced an additional stage to the Hot Restart procedure, i.e., HotRestartIntegrationService.startup(), which waits until all members transition from the PASSIVE state. This guarantees all members to have the same state after Hot Restart operation is finished.

    • IMap proxies created during Hot Restart are not initialized and published to other cluster members. So the operation has been improved to force initialize any uninitialized proxies and publish them. This fixed the issue where the getDistributedObjects() method was not reporting the persisted objects after a Hot Restart.

  • Fixed an issue where, after forming a cluster, certain members occasionally do not contain EventService registrations for some other members. #15950

  • Fixed an issue where calling the IMap.removeAll() method without index was updating the last access and expiry time for all records. #15853

  • Fixed an issue where the client connection count was retrieved using an incorrect method. #15681

3.12.4

Enhancements

  • Added support for the PartitionLostListener configuration on the client side. #15762

Fixes

  • Fixed an issue where the query was ignoring the index but was processing successfully. #15808

  • Fixed an issue where the predicate queries to retrieve multi-values were failing on the nested JSON objects in arrays. #15789

  • Fixed a possible livelock when many threads were trying to use the incrementMod() method causing an infinite loop. #15777

  • Fixed shutting down Management Center service caused by an NPE when MultiMap statistics are gathered after MultiMapService.shutdown(). #15656

3.12.3

Enhancements

  • Added a new implementation of SecondsBasedEntryTaskScheduler for the FOR_EACH mode to improve the performance of TransactionContext.commit(). #15586

  • Improved the client graceful shutdown mechanism so that its tasks are executed before marking a client as inactive. #15565

  • Added sent/received bytes logs for each ProtocolType into the diagnostics. #15541

  • Improved the performance of TransactionLog.add() by avoiding the LinkedList.remove() call. #15435

  • To be shown on Management Center, the clients now send both its IP address and canonical hostname. Before, only the hostname of the client was shown. #15421

  • Added the support for nested JSON objects in the arrays. #15368

Fixes

  • Fixed an issue where the node.isMaster() method could cause inconsistent behaviors when it is executed during a partition service lock. #15618

  • Fixed an issue where Management Center was not able to see the 3.12.2 members (members with CP subsystem enabled) after performing a rolling upgrade from Hazelcast IMDG 3.11.4 to 3.12.2. 'ManagementCenterService' was shutting down and throwing an exception when a CP-subsystem-enabled member was trying to report to Management Center for the first time. [#15609]

  • Made PagingPredicate a VisitablePredicate so the optimizer is able to visit its inner predicate to optimize it or to select an index to evaluate the inner predicate with. #15602

  • Fixed an issue where CP subsystem restarts were not being canceled or waited the running/scheduled discovery tasks. Otherwise, two different discovery tasks could overlap and run concurrently or one of the discovery tasks could run in a corrupted state. #15568

  • Fixed an issue where the cleaner task for expired records was logging the exceptions during migrations: the log level for PartitionMigratingException has been set to FINEST. #15543

  • Fixed an issue where the executor service message task was blocking the partition thread. #15526

  • Fixed an issue where normalFramesRead and priorityFramesRead were never incremented as seen in the diagnostic logs. #15479

  • Fixed an issue where the readBackupData and statisticsEnabled options were not respected when a new map configuration is dynamically added from a client to a running Hazelcast cluster. #15392

  • Fixed an issue where a lock was required when registering metrics on the happy path. #15378

  • Fixed an issue where some updates to the entries got lost from the write behind queue: It was a concurrency issue when there are updates to CoalescedWriteBehindQueue while StoreWorker is running. #15060

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.12.2

Enhancements

Hazelcast IMDG Enterprise Enhancements:

  • Enabled the REST endpoints for the cluster.sh, cp-subsystem.sh and healthcheck.sh scripts to use HTTPS. Before, they were using HTTP. See the cluster.sh, cp-subsystem.sh and healthcheck.sh sections.

Hazelcast IMDG Open Source Enhancements:

Fixes

  • Fixed an issue where the UNKNOWN current runtime version in the JavaVersion.isAtLeast() method was not handled properly. To avoid having runtime functionality, that relies on this method, broken with each new major Java release, UNKNOWN as a detected current runtime version is now considered to be at least any other version. #15362

  • Updated the JavaVersion class to support JDK 13 and 14 builds. #15354

  • Fixed the deserialization filtering for Externalizables and Deadlock in the map index. The deserialization filter was not properly protecting against the vulnerable Externalizable classes. The filtering has been extended. #15346

  • Fixed an issue where the comparators were not able to act on both keys and values. A custom paging predicate comparator may act on keys and values at the same time even if only the keys are requested, e.g., using the IMap.keySet() method. Before this fix only the keys were fetched for this method, making comparators unable to act on values. #15324

  • Fixed an issue where the locked entries with a time-to-live were not evicted. With this fix, the lock operation checks if an entry has already expired. #15310

  • Fixed an issue where the named scheduled tasks was not respecting the HazelcastInstanceAware marker. #15233

  • Fixed an issue where storing MapStore instances in MapStoreConfig could cause member failures when the configuration is added dynamically: When you configure the map store by the class name and start Hazelcast with this configuration, the MapStoreConfig#implementation field was altered to store the reference to the MapStore instance created by Hazelcast (this meant that someone can access the created map store instance via MapStoreConfig#getImplementation()). With this fix, MapStoreConfig behavior has become aligned with other data structures' configuration, i.e., it is not altered anymore and if you did not set anything as implementation then MapStoreConfig#getImplementation() returns null after Hazelcast is started. Also, this fix eliminates the side effects with the dynamically added map configurations, potentially breaking this functionality for the maps with map stores configured. #15225

  • Fixed an issue where IMap’s localLockedEntryCount metric was not considering the entries with null values. #15218

  • Fixed a possible NullPointerException for the remove variable in DeafultRecordStore. #15217

  • Fixed an issue where an executor was serialized multiple times when it is sent to multiple members by a Java client. Now, it is serialized only once as expected. #15007

3.12.1

New Features

Hazelcast IMDG Enterprise New Features:

  • Added support for updating the licenses of all the running members of a Hazelcast IMDG cluster using the REST API. See the License Information section.

Enhancements

  • Introduced complete example configurations (XML and YAML) for Hazelcast Java client and client failover. See the Configuring Declaratively section.

  • Updated Hazelcast JCache implementation to support JCache 1.1.1. This version of JCache does not introduce new functionalities; it resolves the errata and issues in JCache 1.1.0. See the Upgrading to JCache 1.1.1 section.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Improved Config#getConfigurationUrl()'s Javadoc to mention that it returns null if the Config instance has been built from a source different than URL or file. #15109

  • Improved the Raft snapshotting so that the old log entries are not kept when there is no follower with an unknown match index. #15068

  • Updated the client failover XSD to include targetNamespace that has been delivered with Hazelcast IMDG 3.12. #15057

  • Updated the Hazelcast Kubernetes dependency to the latest version. #14899

  • Added the getter method for the YAML configuration builder properties. #14884

  • Eliminated the excessive logging when NIO networking is shutdown by deregistering the IO threads. #14837

  • Introduced a warning log for illegal reflective access operation when using Java 9 and higher, and OpenJ 9. #14835

  • Updated the Hazelcast web session manager dependency to the latest version. #14822

Fixes

  • Fixed an issue where the queries like labels[any] = 0 and labels[any] = 1 were optimized only to false since labels[any] was interpreted as a regular attribute name having a single value. #15138

  • Fixed an issue which was causing OutOfMemoryException in a split-brain situation, due to the client listeners. #15042

  • Fixed the authentication mechanism between the clients and members by adding a check to prevent re-verification while the client is changing its owner member. #15015

  • Fixed an issue where the Hazelcast IMDG configuration files, that have an extension other than .xml, .yaml or .yml or do not have an extension, were ignored silently. This was happening when the configuration file is set by using the hazelcast.config system property. #14945

  • Some operations such as heartbeat checks and partition migrations share common threads with the client login module. In case of the long running client login module implementations, some symptoms such as split brain syndrome can be seen. This has been fixed by introducing a blocking executor which is used only for the client JAAS authentications. #14957

  • Fixed an issue where the client was not considering the new address of a restarted member, which has the same UUID but could have a different IP address after it is restarted. #14843

  • Fixed an issue where the migration operations were running before the previous finalization is completed. #14834

3.12

New Features

Hazelcast IMDG Enterprise New Features:

Hazelcast IMDG Open Source New Features:

  • CP Subsystem: Implementing the Raft consensus algorithm, Hazelcast introduces its CP subsystem which runs within a Hazelcast cluster and offers linearizable implementations of Hazelcast’s concurrency APIs. See the CP Subsystem chapter.

  • Querying JSON Strings: You can now query JSON strings stored inside your Hazelcast clusters. See the Querying JSON Strings section.

  • Pipelining: Introduced pipelining mechanism using which you can send multiple requests in parallel to Hazelcast members or clients, and read the responses in a single step. See the Pipelining section.

  • Support for Multiple Endpoints When Configuring Member’s Networking: Added the ability to configure the Hazelcast members with separate server sockets for different protocols. See the Advanced Network Configuration section.

  • YAML Configuration Support: Added the support for configuring Hazelcast in YAML. See the Configuring Declaratively with YAML section.

Breaking Changes

  • Support for JDK 6 and 7 has been dropped. The minimum Java version that Hazelcast supports now is Java 8. See the Supports JVMs section.

Enhancements

Hazelcast IMDG Enterprise Enhancements:

  • Sharing Hot Restart base-dir among Multiple Members: The base directory for the Hot Restart feature (base-dir) is now used as a shared directory between multiple members, and each member uses a unique sub-directory inside this base directory. This allows using the same configuration on all the members. Previously, each member had to use a separate directory which complicated the deployments on cloud-like environments. During the restart, a member tries to lock an already existing Hot Restart directory inside the base directory. If it cannot acquire any, then it creates a fresh new directory. See the Configuring Hot Restart section.

  • Lower Latencies and Higher Throughput in WAN Replication: Improved the design of the WAN replication mechanism to allow configuring it for lower latencies and higher throughput. See the Tuning WAN Replication For Lower Latencies and Higher Throughput section.

  • Add/Remove WAN Publishers in a Running Cluster: Introduced the ability to dynamically add or remove WAN publishers (target clusters). See the Dynamically Adding WAN Publishers section.

  • Automatic Removal of Stale Hot Restart Data: Introduced an option that allows the stale Hot Restart data to be removed automatically. See the description of the auto-remove-stale-data configuration element in the Configuring Hot Restart section.

  • Client Permission Handling When a New Member Joins: Introduced a declarative configuration attribute on-join-operation for the client permission in the Security configuration (its programmatic configuration equivalent is the setOnJoinPermissionOperation() method). This attribute allows to choose whether a new member joining to a cluster will apply the client permissions stored in its own configuration, or will use the ones defined in the cluster. See the Handling Permissions When a New Member Joins section.

  • Automatic Cluster Version Change after a Rolling Upgrade: Introduced the ability to automatically upgrade the cluster version after a rolling upgrade. See the Upgrading Cluster Version section.

  • FIPS 140-2 Validation: Hazelcast now can be configured to use a FIPS 140-2 validated module. See the FIPS 140-2 section.

Hazelcast IMDG Open Source Enhancements:

  • Client Instance Names and Labels: You can now retrieve the names of client instances on the member side. Moreover, client labels have been introduced so that you can group your clients and/or perform special operations for specific clients. See the Defining Client Labels section.

  • Composite Indexes: Introduced the ability to recognize the queries that use all the indexed properties and treat them as a composite, e.g., foo = 1 and bar = 2 and foobar = 3. See the Composite Indexes section.

  • REST Endpoint Groups: With this enhancement you can enable or disable:

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Improved the YAML configuration so that you can configure multiple WAN member sockets. #14800

  • Members now fail fast when the max-idle-seconds element for the entries in a map is set to 1 second. See the note in the Configuring Map Eviction section for this element. #14697

  • Removed group password from the Hazelcast’s default XML configuration file. Also improved the non-empty password INFO message. It’s now only logged if security is disabled, password is not empty and password is not the Hazelcast default one. #14603

  • Improved the code comments for the HazelcastInstance interface. #14439

  • Improved the speed of partition migrations. This has been achieved by sharing only the latest completed migration information with the members, instead of sending the whole partition table after each migration. Additionally, there was no need to send the completed migrations to all cluster members after each migration. Instead, completed migrations are sent to the source and destination members of the migration inside migration operations. Remaining members, now, receive the completed migrations in batches asynchronously. #14351

  • Improved the Javadoc of HazelcastClient so that the code comments now use "unisocket client" instead of "dumb client". #14213

  • Added the ability to perform an LDAP subtree search for groups in Hazelcast Management Center’s LDAP authenticator. #14118

  • Added the ability to set the EvictionConfig.comparatorClassName() in the client’s declarative configuration, too. #14093

  • Introduced the /ready endpoint to the REST API to allow checking a member if it is ready to be used after it joins to the cluster. #14089

  • Improved the syncing of XSD files. #14070

  • The IMap.removeAll() method now supports PartitionPredicate. #12238

  • Improved the diagnostics tool so that it automatically creates the configured directory for the diagnostic outputs. #11946

Fixes

  • Fixed an issue where the state of member list on the clients were broken after a hot restart in the cluster. #14839

  • Fixed an issue where the outbound pipeline was not waking up properly after merging the write-through changes. #14830

  • Fixed an issue where the like and ilike predicates didn’t catch any entity with the text field containing the \n character. #14751

  • Fixed an issue where NullPointerExceptions was thrown recursively when a client is connected to an unreachable member during a split-brain. #14722

  • Fixed an issue where Hazelcast running on RHEL (OpenJDK8) shows unknown gc in the logs, instead of major gc and minor gc. #14701

  • Fixed an issue where the IP client selector was not working for the local clients. #14654

  • Fixed the wording of a misleading error in the first attempt to connect to a wrongly configured cluster. The error message has been changed to “Unable to connect to any cluster”. #14574

  • Fixed an issue where the setAsync() method was throwing NullPointerException. #14445

  • Fixed an issue where the collection attributes indexed with [any] were causing incorrect SQL query results, if the first data inserted to the map has no value for the attribute or the collection is empty. #14358

  • Fixed an issue where mapEvictionPolicy couldn’t be specified in the JSON configuration file. #14092

  • Fixed an issue where the rolling upgrade was failing when all members change their IP addresses. #14088

  • Fixed an issue where the resources were not wholly cleared when destroying DurableExecutorService causing some resources to be left in the heap. #14087

  • Fixed an issue where the REST API was not handling the HTTP requests without headers correctly: when a client sends an HTTP request without headers to the Hazelcast REST API, the HttpCommand class was wrongly expecting an additional new line. #14353

  • Fixed an issue where QueryCache was not returning the copies of the found objects. #14280

  • Fixed an issue where the locks were not cleaned up after the members are restarted. #14215

  • Fixed an issue where the MultiMap’s RemoveOperation was iterating through the backing collection, which caused performance degradation (when using the SET collection type). #14145

  • Fixed an issue where the user code deployment feature was throwing NullPointerException while loading multiple nested classes and using entry processors. #14105

  • Fixed an issue where the newly joining members could not form a cluster when the existing members are killed. #14051

  • Fixed an issue where the IMap.get() method was not resetting the idle time counter when read-backup-data is enabled. #14026

  • Fixed an issue where the addIndex() method was performing a full copy of entries when a new member joins the cluster, which is not needed. #13964

  • Fixed an issue where the initialization failure of discoveryService was causing some threads to remain open and the JVM could not be terminated because of these threads. #13821

  • Fixed the discrepancy between the XSD on the website and the one in the download package. #13011

  • PagingPredicate with comparator was failing to serialize when sending from the client or member when the cluster size is more than 1. This has been fixed by making the PagingPredicateQuery comparator serializable. #12208

  • Fixed an issue where TcpIpConnectionManager was putting the connections in a map under the remote endpoint bind address but not under the address to which Hazelcast connects. #11256

Removed/Deprecated Features

  • ILock interface and implementation of ILock has been deprecated, and FencedLock has been introduced.

  • The original implementations of IAtomicLong, IAtomicReference, ISemaphore and ICountDownLatch have been deprecated. Instead, the implementations provided by the CP Subsystem have been introduced.

  • The following system properties are deprecated:

    • hazelcast.rest.enabled

    • hazelcast.mc.url.change.enabled

    • hazelcast.memcache.enabled

    • hazelcast.http.healthcheck.enabled

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.11.7

Fixes

  • Fixed an issue where the Near Cache invalidation was not working properly when used with Transactional Map. #16618

3.11.6

Fixes

  • Fixed an issue where the OSMBean.getFreePhysicalMemorySize() method was being excessively executed on AIX operating systems: added the option to disable the retrieval using this method. #16091

3.11.5

Fixes

  • Fixed an issue where the newly joining members could not form a cluster when the existing members go down. For this, the joiner mechanism has been improved, details of which can be seen in #15871.

3.11.4

Fixes

  • Some operations such as heartbeat checks and partition migrations share common threads with the client login module. In case of the long running client login module implementations, some symptoms such as split brain syndrome can be seen. This has been fixed by introducing a blocking executor which is used only for the client authentications. #14941

3.11.3

Enhancements

  • Added the security manager protection to member configuration methods so that the user code deployments can be used in a more secure way. #14634

Fixes

  • Fixed an issue where there was a leak in the socket channel when the networking is restarted after the cluster merge. #14699

  • Fixed an issue where the partition table was not being updated correctly on the client side when the connection was changed due to the changes in the member list. #14793

  • Disabled the migrations while the mastership claim is in progress to prevent the creation and submission of migrations before its claim is approved by all the final cluster members. #14745

  • Fixed an issue where some of the operation statistics were incorrectly reporting operations per partition. #14726

  • Fixed an issue where the entries in the Transactional Map were ignored while promoting from the backups. #14714

  • Fixed an issue during the backup expirations of maps with High-Density Memory Store when there is no eviction configured. #14626

  • Fixed an issue where the map statistics are not updated when the IMap.delete() method is called from the client. #14471

3.11.2

Enhancements

  • Introduced a new system property (hazelcast.security.permissions.operation.on-join) that allows to choose whether a new member joining to a cluster will apply the client permissions stored in its own configuration, or will use the ones defined in the cluster. See the Handling Permissions When a New Member Joins section.

Fixes

  • Fixed an issue where InitialMembershipEvent was being fired with empty member list, when client async start is enabled. #14364

  • Fixed an issue where the invokeOnPartitionsAsync() method was not returning a value when the memberPartitions is empty. #14266

  • Removed a misleading exception on the member side when the Java client is disconnected via the method hazelcast.shutdown(). #14244

  • Fixed an issue where the getOperationCount statistics were not updated as expected when performing get operations from the Java client. #14143

  • Difference between the clocks of target and source clusters had a potential to completely block the WAN communication which may cause the WAN queues to be filled up. This has been fixed by ignoring the call timeout check for WAN operations. #13301

Behavioral Changes

  • If you have multiple Hazelcast members on a single machine and you are using unisocket clients, we recommend you to set explicit ports for each member. Then you should provide those ports in your client configuration when you give the member addresses. Otherwise, all the load coming from your clients may go through a single member.

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.11.1

Enhancements

  • Toggle Scripting Support: Introduced the scripting-enabled configuration attribute so that you can allow or prevent sending commands to the members from the Hazelcast Management Center. See the Toggle Scripting Support section for information on how to configure it.

Fixes

  • Fixed an issue where the client’s Near Cache on Replicated Map with Binary storage was causing an exception on the member. #14210

  • Fixed an issue where the performance of DefaultQueryCache was degraded because of the flow in case of the index-aware queries. #14177

  • When the client (having a Reliable Topic) reconnects to the cluster, the MessageListener added to the client was no longer receiving messages, unlike the MapListener. This has been fixed by making the Reliable Topic structure to continue from the last known sequence ID, in case of an HazelcastClientOfflineException. #14123

  • Fixed an issue where the members and clients were not disconnecting the connection as soon as a heartbeat timeout is detected. The member side heartbeat monitor was still checking if the connection is the owner connection. This activity has also been removed. #14094

  • Fixed an issue where the mapEvictionPolicy configuration became null, when the JSON file is used to start a Hazelcast instance on the PCF environment. #14086

  • Fixed an issue where the failed store operations was filling the write-behind queues by duplicating themselves, in each run of the store-worker thread (this was happening when write-batch-size is larger than 1 and write-coalescing is disabled, there was and is no issue for the default values of these properties). #14019

  • Fixed an issue where the querying performance dropped when running Hazelcast within OSGi. #13785

Behavioral Changes

  • Starting with Hazelcast IMDG 3.11.1 and for the future releases, when you enable TLS/SSL and do not provide the trustStore property, the default trusted certificates are/will be used, i.e., the Java provided list of trusted CA certificates.

  • Due to the API change in evictionPolicy in maps (related issue is listed above as #14086), you may face with a configuration conflict while using dynamic configuration, i.e., the exception Cannot add a dynamic configuration. You can use the hazelcast.dynamicconfig.ignore.conflicts property when restarting your cluster after this conflict to heal it.

3.11

Hazelcast IMDG 3.11 and the upcoming releases are fully compatible with JDK 9, 10 and 11. For information on the other JDK compatibility details, see the Supported Java Virtual Machines section in Hazelcast IMDG Reference Manual.

Enhancements

Hazelcast IMDG Enterprise Enhancements:

  • Fine-Grained Control over WAN Replication Events: Introduced the new event type LOADED and the class EntryLoadedListener through which all the loads can be listened to. Also introduced the new configuration element persist-wan-replicated-data to specify whether to persist an incoming event over WAN replication or not. See the Configuring Consumer section.

  • License Enforcements and Warnings: Introduced a license monitor daemon that warns about expirations and instructs about the next steps. These information are also available through REST/JMX APIs. See the License Information section.

  • Delta WAN Synchronization: Introduced WAN anti-entropy mechanism utilizing Merkle Tree to synchronize only the affected subsets for Maps, improving network and heap utilization. See the Delta WAN Synchronization section.

Hazelcast IMDG Open Source Enhancements:

  • Ability to Set Custom TTL for Map and Cache Entries: Added a new method, setExpiryPolicy(key, expirePolicy), to allow interacting with the TTL value of an entry without updating the value. Also, the IMap interface for the operation of putting a value into a map was not clear where the resulting value of the entry’s TTL is concerned. This is fixed by adding the method setTTL() on both the member and client sides. See the here.

  • Ability to Set Custom Maximum Idle Timeouts for Map Entries: Extended the put operation so that now it has a maxIdle parameter that represents the idleness seconds for specific entries. See the Evicting Specific Entries section.

  • Configurable Backoff Strategy for Client Reconnections: Introduced a highly configurable exponential backoff mechanism for the client with which you can set the duration for waiting after connection failures, upper limit for the wait, etc. See the Client Connection Retry Configuration section.

  • Map Index Statistics: Introduced statistics related to indexes. To achieve this, map statistics have been extended with per index information about indexes associated with a certain map. See the Map Index Statistics section.

  • JCache Expired Entries Disposal: Improved JCache’s lazy eviction with a periodic cleanup that eventually removes all expired entries and frees up the occupied space.

  • Improved Network I/O: Enhanced network I/O to allow TLS handshake in a block-free manner and to support offloadable tasks, improving the latency and security.

  • Now, Hazelcast supports discovering members in the GCP Compute Engine environment. See the documentation for more information.

  • Introduced the ability which allows adding the user libraries to the classpath. See the Adding User Library to CLASSPATH section.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Added support to capture LOADED events for MapEventJournal. #13667

  • The method ClientCloudConfig.setEnabled() has been improved so that now it returns its own instance. #13640

  • The client was always constructed with an empty userContext. This has been improved by adding the method setUserContext() for the object ClientConfig. #13398

  • Added Java 9+ support (modularization), hence improved the user experience on Java versions 9+ which are based on Java Platform Module System. #13182

  • MigrationRequestOperation has been improved with the new Offload abstraction. #11989

  • Improved the multicast discovery strategy for clients. #11357

  • Removed group password from Hazelcast configuration. The password is not checked anymore during member joins. #11667

  • Introduced a more proper way of heap-data conversion: the method toHeapData. Before, ToHeapDataConverter was being used. #10249

  • The method EntryListener.entryAdded() is not called after IMap.evict() is triggered followed by the method IMap.get(). To fix this, a new entry event type has been introduced: LOADED. #1542

Fixes

  • PN counters were not replicating for the following scenario:

    • start a member

    • wait replication-period-millis to pass

    • start a second member

    • kill the first member

      PN counter data from the first member was lost. This is fixed by removing PN counter statistics on migrations. #13793

  • Fixed the incorrect TCP connection probe registration in outgoing member connections. It was [null→null] instead of [address→address]. #13768

  • Clients were sometimes failing to reconnect to another owner member with the ExecutionException. This is fixed by making ClientReauthOperation to be retryable. #13758

  • Event Journal was throwing StaleSequenceException when TTL is greater than 0. This is fixed by skipping expired events in the journal during reading. #13746

  • Fixed the HazelcastOverloadException when trying to shutdown the cluster. It was also not performing a graceful shutdown. Now cluster state operations are marked as UrgentSystemOperation since otherwise, these operations might get rejected by backpressure. #13730

  • Fixed the noisy health check logging when starting Hazelcast. #13662

  • When an unserializableResponse is tried to be sent to the client as a response from the executor service tasks, the exception was logged on the server side and there was no response returned back to the client. This has been fixed by removing the logging and sending HazelcastSerializationException to the client. #13639

  • Fixed the issue where IMap entries having a max-idle-timeout were not expiring when the member shuts down. #13630

  • Reliable Topic was not working after the correct partition migration to previous owner member (where it was created the first time and message listener was attached); there were no exceptions or warnings. This is fixed by removing stale caching of Ringbuffer in AbstractRingBufferOperation. #13602

  • Fixed the repeatedly thrown IllegalAccessException when the client statistics is enabled. #13565

  • Fixed the accidental leakage of InternalSerializationService through DataSerializable interface. #13505

  • Fixed the InaccessibleObjectException which is caused by the operating system level metrics silently dropping on Java 9 when a Hazelcast member is started. #13463

  • SerializationServiceBuilder does not ignore the method useNativeByteOrder() anymore. #13410

  • Attribute extractor now falls back to the user code deployment: it was not using the user code deployment to search for the extractor implementation. #13400

  • Fixed the connected clients being slow when the server port is connected without receiving anything. #13298

  • When using the hazelcast-all artifact for 3.10.1, the version of hazelcast-hibernate included within is the one built using Hibernate 3. This is fixed by updating the default Hibernate version to 5. #13250

  • Built-in predicate implementations now implement the equals/hashCode. #13245

  • When adding a dynamic data structure configuration, Hazelcast fails fast when the same structure is already configured statically (even when both configurations are equal. This is fixed so that the submitted dynamic configuration is silently ignored when it is equal to an existing static configuration, or Hazelcast fails with a ConfigurationException when a conflicting static configuration already exists. #13158

  • There was a warning when Hazelcast is used on Java 10 JRE: An illegal reflective access operation has occurred. This is fixed by making Hazelcast fully compatible with Java 10. #13151

  • Fixed the EOFException which is thrown when using PagingPredicate with Kryo in Hazelcast 3.8.7. #13147

  • Fixed the exception Cannot override cache’s CacheManager which is thrown when a cache is recreated after the restart of remote Hazelcast. #12975

  • When a member is killed, events are lost and the method QueryCache.tryRecover() is triggered in a loop even if data is not lost. This is fixed by resetting the query cache sequence numbers by the local promotions. #12928

  • Fixed the memory leak on NonBlockingSocketWriter when the client disconnects: the member instance was holding onto a write buffer when a client disconnects abruptly, while there is pending data to be sent. #12353

  • The comparison of values during the operation CacheRecordStore.merge() was incorrect. This is fixed by not firing an update event when merging values are equal. #12175

  • Hazelcast lacks a mechanism for the client/server recovery within Spring cache context. This is fixed by recreating the local cache configurations when the client is connecting to a restarted member. #12128

  • Ringbuffer.readManyAsync(): Client unwraps the ExecutionException before throwing it to the user cod; the member never does it. It also does not happen on other async proxies. This is fixed so that the client does not unwrap this exception. #12108

  • Removed group password based credentials check in for the client connections. #11867

  • When IPv6 is enabled for Hazelcast, the started member was still setting an IPv4 as a local address by default. This is fixed by improving the IPv6 bind address selection mechanism. #11820

  • Fixed an issue for hostname and local network interface matching in the DefaultAddressPicker. The member was picking the hostname which resolves to an IP not present locally. #8211

  • The method MapLoader.load() does not create an entry event, as it is essentially a get operation. But, the method MapLoader.loadAll() uses IMap.putTransient() and so it generates EntryAdded events. This was inconsistent and the latter was problematic for WAN replicated clusters. It is fixed by avoiding the invocation of MapLoader on containsKey(). #7771

Behavioral Changes

  • When the TLS/SSL is configured and no trustStore property is provided, no default trusted certificates are used; neither the keystore, nor the Java provided list of trusted CA certificates. Therefore, you ALWAYS have to configure the trustStore property. See the TLS/SSL section.

  • WAN Replication:

    • Loaded entries were listened using EntryAddedListener. Now, a new event type for the loaded entries is introduced (LOADED) and they are listened using the new EntryLoadedListener.

    • Loaded entries are not replicated to the target cluster by default anymore.

    • Replicated entries were being persisted at the target cluster (in its map store). Now, they are not being persisted by default anymore. You can use the newly introduced configuration element persist-wan-replicated-data and set it to true (the default is "false") to make these entries to be persisted.

  • Map entries timestamps: Entry timestamps (i.e., LastAccessedTime, LastUpdatedTime, CreationTime, etc.) are now using SECONDS resolution, rather than milliseconds.

Removed/Deprecated Features

  • The data structure IdGenerator is deprecated. Use Flake ID Generator.

  • Following system properties are removed:

    • hazelcast.master.confirmation.interval.seconds

    • hazelcast.max.no.master.confirmation.seconds

    • hazelcast.mc.max.visible.instance.count

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.10.7

Fixes

  • Fixed an issue where the Near Cache invalidation was not working properly when used with Transactional Map. #16617

3.10.6

Enhancements

  • Added the ability to log IOExceptions thrown by the ICMP ping task as ping failures. #13826

Fixes

  • Entry processors equipped with predicates try to utilize global indexes to narrow down the key set to operate on. #13892 fixes the following issues for this utilization:

    • There were no migration stamp validations performed to prevent entry processors from operating on stale index data.

    • WrongTargetException was not thrown for out-migrated partitions, as a result the caller (InvokeOnPartitions) was not retrying entry processors on such partitions.

    • Partition scope provided to PartitionIteratingOperation was not respected. As a result, operations produced by factories providing their own partition sets were running on more partitions than expected.

  • Fixed the incorrect dependencies in hazelcast-all.pom. #13885

  • Hazelcast lacks a mechanism for the client/server recovery within Spring cache context. This is fixed by recreating the local cache configurations when the client is connecting to a restarted member. #13811

  • Clients were sometimes failing to reconnect to another owner member with the ExecutionException. This is fixed by making ClientReauthOperation to be retryable. #13790

  • When an unserializableResponse is tried to be sent to the client as a response from the executor service tasks, the exception was logged on the server side and there was no response returned back to the client. This has been fixed by removing the logging and sending HazelcastSerializationException to the client. #13777

  • Queryable entries returned by High-Density Memory Store indexes were unaware of the map value extractors, so it was not possible to extract values provided by the extractors from the returned entries. This is fixed by forwarding the map value extractor to these indexes. #13775

  • Hazelcast 3.10.5 client was not being auto-discovered on Kubernetes. This is fixed by using a specific TLS version, namely v1.2, for SSLContext. #13760

  • There is a scheduled task to clean the client resources. When an endpoint is removed, this task keeps a reference to it and this was causing unnecessary leak. This is fixed by passing only the necessary field objects of the client endpoint to this task to avoid keeping reference to the client endpoint itself. #13755

  • When merging MultiMap entries, MERGE events were being fired. Now they are not since MultiMap does not have a merging listener. #13714

  • Fixed the heartbeat resume mechanism where the heartbeat of a connection was becoming unhealthy when the last read time is lower than the heartbeat timeout. #13684

  • Added the storage.clear operation before storage.destroy to release the High-Density (HD) Memory Store resources, since using only storage.destroy releases the internal resources of backing data structure, not the HD resources. #13659

  • Moved the alive check in ClientConnectionManager to a more centralized method and fixed the exception type. #13525

  • Fixed an issue where a joining member was not validating the initial cluster version. #13390

  • When a Near Cache is used for an IMap and a TransactionalMap is used on a client then the get(key) operation fails reporting a wrong key type. This is fixed by skipping the Near Cache lookup at the member side, for the client’s transaction request. #13371

  • HDTxnSetOperation was not taking the changes made by MapInterceptor into account while sending the backups. Now it replicates the result of interceptPut. #12705

3.10.5

Enhancements

  • Introduced OperationThreadSamples as a new diagnostics plugin. It samples the operation threads and checks the running operations/tasks. See the OperationThreadSamples section.

Fixes

  • Added a workaround for the Constructor cache, which performs caching under incorrect keys, such that you have now the option not to use that cache. #13679

  • A client connection could not detect whether it has been idle and hence the member was closing the connection. This is fixed now so that the member does not close the connections when the clients only listen. #13576

  • Fixed the MultiMapValue serialization error occurred when merging MultiMap entries. It was caused by firing MERGE events which expect the entry values to be serialized. Since MultiMap does not have a MERGE listener, there was no need to fire these events; this has been fixed and now no MERGE events are fired during the merging process. #13559

  • When a client is waiting for a lock to be released for a period more than invocation timeout seconds and the member dies, the client was getting an operation timeout exception. This behavior is fixed so that the operation is now retried on the new member of that lock. #13551

  • By default Hazelcast map evicts just a single entry in each eviction cycle. This was insufficient when inserting entries with increasingly larger payload size - as evicting a single entry cannot make sufficient room for a new (larger) entry. This is fixed by introducing a configurable batch size for map eviction. #13529

  • Fixed the incorrect unit declaration in EntryView class' Javadoc: the method getTtl() returns a value in milliseconds, it was declared that it is in seconds. #13523

  • When a client is disconnected, the member was cleaning up resources but not the pending invocations. This is fixed so that the member now cleans up the pending invocations. #13388

3.10.4

Enhancements

  • License information was not sent to Hazelcast Management Center. With this enhancement, Hazelcast Management Center shows information about the cluster license. #13451

Fixes

  • Fixed inconsistencies in mapEventJournal calls coming from DefaultRecordStore. #13470

  • Fixed index data loss after rolling upgrade: when a rolling update of an application is performed; first instance is turned off, updated and after it is done, the second instance takes place. After such a rolling update (without the downtime) some of the data was lost. It turned out that after the Split-Brain merge process, the index gets corrupted. #13456

  • Changed repartitioning mechanism so that it is triggered only when needed during the change of cluster state. #13446

  • Made attribute extractor to use User Code Deployment for searching extractor implementation. #13400

3.10.3

Starting with Hazelcast 3.10.3, Hazelcast IMDG Enterprise license keys are required only for members. You do not need to set a license key for your Java clients for which you want to use Enterprise features.

Enhancements

  • Descriptive null checks should be added when accessing a data structure on the client. #13256

Fixes

  • There is a memory leak in SecondsBasedEntryTaskScheduler of Replicated Map: assuming a TTL is set when an entry is put into the Replicated Map; when the map is cleared, the heap still contains a lot of instances of SecondsBasedEntryTaskScheduler. #13409

  • In AbstractInternalQueryCache, the private method getQueryCacheConfig() is called in multiple places. #13402

  • Permission type for PN Counter is missing. #13399

  • Setting ExpirationTimeMergePolicy throws exception during configuration validation. #13392

  • Close task problems in NioOutboundPipeline: There happens a temporary stall since a task is executed on an I/O thread from that same I/O thread and then the I/O thread waits for it. Because the thread will not execute its own task (yet) there is a 3 seconds of stall in the I/O pipeline. This is a problem since it blocks all the traffic for that I/O thread. Also the close task is scheduled as a packet. If the pipeline already is scheduled, scheduling the close task will not get picked up as part of an optimization. #13363

  • When a cache is closed, it is no longer managed by the CacheManager; its CacheManager should be reset to null and the method getCacheManager() should return null as per specification. #13236

  • Hazelcast client produces OutOfMemoryError when there is no Hazelcast member available at startup. #13186

Removed/Deprecated Features

  • The method 'ClientConfig.setLicenseKey()' is deprecated.

3.10.2

Enhancements

  • Added utility methods to drain a queue. #13108

Fixes

  • If the inner predicate of the paging predicate returns an empty result set, the paging predicate returns null as the indexed query result: this is interpreted as an absence of the indexed result by the query engine and the execution falls back to the full scan. #13159

  • While doing a rolling upgrade with a mixed 3.9 - 3.10 cluster running on 3.9 cluster version, cache configurations must be converted to CacheConfig before dispatched to 3.9 members. Otherwise, a PreJoinCacheConfig could be stored on a 3.9 member’s cache configurations registry (AbstractCacheService.configs) and from there it may leak to a client getting the cache configuration causing deserialization errors on the client side. #13137

  • NullPointerException is thrown when accessing endpoints on the message task using AbstractMessageTask.getClientVersion. #13119

  • NullPointerException is thrown during the authentication, if the connection is closed right before bind. #13118

3.10.1

Enhancements

Fixes

  • The method readManyAsync() throws different exceptions when there is no quorum: The client fails with QuorumException, and member fails with ExecutionException. #13065

  • It is expected to get HazelcastSerializationException from the method future.get(), however, it hangs when the response is not serializable. #12956

  • CachePutBackupOperation: When an entry is evicted on the primary member, it is also evicted on the backup members. This causes the method LocalCacheStats.getCacheEvictions() to return incorrect numbers. #12931

  • The connections to the members after they are removed from the cluster should be closed. #12926

3.10

New Features

  • PN Counter: Introduced CRDT PN Counter (Positive-Negative Counter), which allows incrementing and decrementing a counter value and propagating the results to all cluster members. See the PN Counter section.

  • Flake ID Generator: Introduced to overcome the duplicate ID generation in case of a network split, by the deprecated IdGenerator data structure. See the FlakeIdGenerator section.

Enhancements

Hazelcast IMDG Enterprise Enhancements:

  • Split-Brain Healing for Hazelcast Data Structures with High-Density (HD) Memory Store: Added implementations to allow the merging of HD Memory Store backed data structures.

  • Multiple Users/Roles in a Single Security Mapping: Added support for allowing multiple principals, i.e., users or roles/groups, to be attributed to a security realm. See the Authorization section for more information.

Hazelcast IMDG Open Source Enhancements:

  • Extended Split-Brain Protection: Split-Brain protection has been implemented for all the Hazelcast data structures (it was only for IMap, ICache, IQueue and ILock before 3.10). See the Distributed Data Structures chapter.

  • Shorter Split-Brain Detection Window: In addition to the current quorum function, which uses the list of members to determine whether a minimum number of members is available in the cluster, a new quorum implementation has been introduced to shorten the time between the occurrence of a network partitioning and its detection: it takes advantage of information about the cluster health (heartbeats and ICMP pings). See the Split-Brain Protection section for the new quorum functions.

  • Extended Merge Policies: Merge policies applied after a Split-Brain syndrome has been implemented for more Hazelcast data structures (it was for IMap, ICache and Replicated Map before 3.10). See the Split-Brain Recovery section for more information.

  • Health Check Script: Introduced a command line tool to check the health of your Hazelcast cluster. See the Health Check Script section.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Split-brain merge policy design issues: the SPI design requires downcasting in the merge policy classes. It is also unclear which merge policy is compatible with which data structure. #12721

  • Added ID count to flake ID generators statistics. #12612

  • MemberMap.SINGLETON_MEMBER_LIST_VERSION value should be configurable. #12528

  • It is still required to provide a group and password while configuring with Spring: the attribute required should be removed from the tag group both for name and password elements. #12462

  • Group passwords should not be serialized in ConfigCheck. #12303

  • When custom merge policy is not appropriate, there should be a fail-fast. #12284

  • Code comments for Hazelcast.shutdownAll() should be improved: it talks about shutting down all running Hazelcast instances in the JVM. However, native clients, which are also Hazelcast instances, are not shut down when this method is called. #12217

  • An initialization exception should be thrown when Symmetric Encryption is configured on Hazelcast Open Source edition. #12203

  • IAtomicLong and IAtomicReference miss MergePolicyConfig support in ClientDynamicClusterConfig. #12126

  • ConfigXmlGenerator masking should be optional: it masks certain fields like passwords. This is unnecessary since the user creating the hazelcast.xml should already know the values or they should be input by the user installing the application. By masking the fields automatically, the user is forced to manually edit the hazelcast.xml creating a manual step. #12012

  • The CharConverter conversion behavior should be improved: When given an empty string, IllegalArgumentException should be thrown instead of the obscure StringIndexOutOfBoundsException. Also, when given a Number, a Character should be returned instead of an Integer. #11981

  • Currently when reading from event journal, the reader throws StaleSequenceException when the consumer is behind the current head of the journal. The exception contains the new head of the journal, however by the time the client gets to read again this new head can be stale again. It can mean that the reader is never able to catch up once it falls behind. #11895

  • The Javadoc for IMap must be enhanced to clarify the value vs. reference semantics. #11816

  • The class com.hazelcast.query.Predicates contains public factory methods for query predicates. Most of the methods are missing Javadoc. #11734

  • The value of Main-Class in hazelcast-all.jar should be fixed. The class StartServer should be used instead of ConsoleApp. Currently, when the JAR is executed, the ConsoleApp demo application is started. #11655

  • The error message for failed client connections should not talk about "address in the config" as multicast discovery may have been used. #11652

  • Doing a rolling upgrade without a Hazelcast Enterprise key should produce a better error message. #11602

  • IMap contract should describe how it interacts with Map{Loader/Store}. #10969

  • New HTTP connections should be logged with DEBUG or a lower level. #10924

  • Out-of-the-box Hazelcast on Docker experience should be improved due to the "This node is not requested endpoint" errors. #9219

Fixes

  • Split-brain merge policy configuration validation is not properly working: The merge policy checks in ConfigValidator are based on false assumptions regarding SomeConfig.isStatisticsEnabled() and valid split-brain merge types. #12722

  • XmlConfigBuilder wrongly uses Locale dependent number format for nativeMemory.metadataSpacePercentage. #12656

  • NullPointerException is thrown when the method QueryCache#getAll is called. #12656

  • Query cache is not consistent with its IMap: After the construction of a continuous query cache, all changes on the underlying IMap are immediately reflected to this cache as a stream of events. Therefore, this cache will be an always up-to-date view of the IMap. This is not always true. #12599

  • When minority of members have network delays, there occurs transactional exceptions on the client. #12252

  • Delayed resolutions for other types in CacheConfig should be implemented: In addition to key and value types, the following need to be deferred: cacheLoaderFactory, cacheWriterFactory, expiryPolicyFactory and listenerConfigurations. #11905

3.10-BETA-2
  • ReplicatedMapConfig must not be mutated while constructing read-only variant. #12620

  • Added ID count to flake ID generators statistics. #12612

  • Exception in Node constructor doesn’t shutdown services properly. #12587

  • CacheNotExistsException must be thrown when cache configuration does not exist. #12593

  • Index configurations must be respected in query caches. #12577

  • Issues with replicated map and lite members: (i) replicated map cannot be used after lite member promotion, (ii) replicated map runs into NPE after a split-brain healing. #12537

  • It is still required to provide a group and password while configuring with Spring: the attribute required should be removed from the tag group both for name and password elements. #12462

  • Eviction causes absence of an entry in IMap and MapStore at the same time: just after the method evictAll is invoked, the map is cleared and the eviction is done in a different thread. This causes the objects not being available both IMap and MapStore. #12455

  • JCache 1.1 TCK: the test org.jsr107.tck.event.CacheListenerTest passes, but there are assertion errors in the log. #12390

  • An index aware predicate does not invoke its apply method: only the index operation is performed and the apply filter is silently ignored. #12352

  • Query with predicates on IMap does not use index when running locally. #12351

  • EventJournal loses data if two members terminate: Scenario is starting a cluster with four members and a client, producing data for the event journal, terminating one member and terminating another member after some time. When checking the total count of events, it is seen that some data is lost. #12300

  • When minority of members are removed from the network, read/write failures occur on the cluster. #12240

  • A merge policy cannot be defined for replicated maps using declarative configuration (XML). #12223

  • All dynamic configurations are sent out in a prejoin operation regardless of the cluster version. #12151

  • When using PagingPredicate, setting a too big page ends up with IllegalArgumentException. #12079

  • There is a performance regression on predicate queries for 3.9.1 and 3.8.7 versions. #12018

  • Each new configuration method introduced in 3.10 must be overridden in the client-side dynamic configuration support class. #12010

  • For some of the scheduled jobs, the method getLastRunDuration() returns negative values for quick executions. #11929

  • Exception in ResponseThreadRunnable causes the REST API to fail: When there is an exception while processing a single command, the entire thread is killed. It causes an unrecoverable condition when the system does not process any incoming REST commands, and hence the memory leak (incoming commands are accumulated in blockingQueue). #11722

  • Accessing a query cache with a reasonable amount of data can result in an NPE. #11675

  • IMap.add and IMap.remove: EntryListener randomly hangs. #11470

  • Members not rejoining cluster after an elongated network disconnectivity: Assuming that, initially there is a cluster of 8 members and when the connectivity for a member is broken it gets eliminated from the cluster. The original cluster contains now 7 members and this is working as expected. However, when the connectivity is restored, the disconnected member is not able to join the original cluster and remains isolated. #11267

  • There is a minor decrease in the performance of indexed searches for 3.7.6 and higher versions. #11231

  • The code comments for time-to-live expiration is not clear. #11787

  • When setting the same key value twice, NPE is thrown on the member. #10556

  • JCache fails to initialize when the type of a key or value is not available on a remote member. #8972

  • In Hazelcast 3.7, PollOperation invocation is failing to complete due to operation-heartbeat-timeout. #8831

  • Custom LoggerFactory is instantiating two times. #5641

3.10-BETA-1
  • Health Monitor reports load statistics incorrectly. #12190

  • Time-to-live for IMap is not working as documented: outdated warnings should be removed from the code comments. #12144

  • There are multiple top-level types missing in HazelcastNamespaceHandler, for example serializer. #12121

  • The full stack trace of QuorumException should be logged into the log file instead of the console. #12188

  • ConfigXmlGeneratorTest: There are missing tests for many attributes of multiple data structures. #12119

  • Beta annotation from DurableExecutor classes should be removed. #12083

  • Hazelcast should not allow to add dynamic configurations for concurrent data structures during rolling upgrades: new configurations have been introduced for IAtomicLong and IAtomicReference for the split-brain healing. There may come more for the split-brain protection; these should not be added dynamically when the cluster is on version 3.9, since old members will fail to process them. #12000

  • Using MemberAddressProvider with custom discovery strategy SPI does not seem to work. #11997

  • Hit count gets incremented by two (instead of one) when entry is updated using EntryProcessor. #11993

  • The objects BufferObjectDataInput and BufferObjectDataOutput are pooled. While being used for serialization, their version may be set, however when cleared (so they can be returned to the pool), their version is not reset to UNKNOWN. #11900

  • The default network interface selection chooses a wrong interface in some cases. #11795

  • The memory limit setting for a queue store is ignored and the entries are getting inserted into both the store and the queue, even If the limit is reached. #11682

  • When trying to integrate Spring and Hazelcast with MapStore implementation, the JdbcTemplate cannot be autowired in the MapStore class. #11656

  • PER_NODE capacity calculation algorithm is not precise. #11646

  • Currently ILock operations are not logged at any level. It would be very helpful diagnostic log output can be enabled, with the log lines containing the name of the ILock, the operation (create, lock, unlock, destroy) and the usual timestamp, and thread ID information. #11622

  • The Echo task used to execute the operations demo in ConsoleApp fails to deserialize because of missing no-arg constructor. #11612

  • Wrong Hazelcast configuration XSD schema is used when minor/major or minor has more than 1 digit (e.g., 3.1x). #11586

  • The XSD (hazelcast-spring.xsd) should work independently of the hazelcast-spring.jar. #11577

  • IMap with MapStore has duplicate keys. #11462

  • Backups' view of the Ringbuffer differs from that of the primary’s in case of an exception is thrown by the store underlying the Ringbuffer. #11209

  • It does not seem to be possible to configure a list of cipher suites nor a list of SSL/TLS protocols. Client authorization is implemented, but it has to be passed in a populated Properties object. #10750

  • Transaction could not obtain a lock for the key while calling the method TransactionalMap.getForUpdate(). #9374

  • JCache fails to initialize when the type of a key or value is not available on a remote member. #8972

  • There is no way to set the HazelcastClient name and/or InstanceName programmatically. So, the method HazelcastClient.getHazelcastClientByName(String name) does not work. #7289

  • IMap JMX statistics are evicted together with the map entry: When system starts and caches are populated, and get the hits, the statistics presents the relevant values for localHits and localGetOperationCount attributes. But after 1 hour of idle and some entries are evicted, these attributes become less and less, till 0. Looks like these statistics are stored together with the entry itself and they are evicted together with the entry. #4321

Coverity Fixes
  • The methods readObject(…​) and writeObject(…​) should call the method from the superclass instead of handling the attributeName field themselves in the classes LikePredicate and RegexPredicate. #11768

  • The field retryCounter is not atomically updated in the method RestartingMemberIterator.retry(Set). #11750

Behavioral Changes

  • Up to 3.9.x, the method Config.findCacheConfig looks up a CacheSimpleConfig: if none is found, then null is returned. This behavior has been not like all the other findXXConfig methods in the following ways:

  • findXXConfig never returns null, it falls back to defaults, creates the missing data structure configuration and stores it in Config.

  • findXXConfig returns a read-only copy of the actual configuration, while findCacheConfig returns the found configuration object itself.

    With 3.10, the method Config.findCacheConfig is aligned to behave like the other findXXConfig methods; it now returns a read-only copy and does fallback to defaults. If you who want the old behavior for this method in 3.10.x, you should use the method Config.findCacheConfigOrNull.

  • Starting with 3.10, any unknown property of Discovery SPI in the Hazelcast configuration will result in an exception blocking the Hazelcast member from starting. This approach restricts the configuration (both XML and Object-based) and prevents typos.

Known Problems

  • PartitionPredicates only works if you upgrade all of your members to 3.9.3. It may not work

    • when running a mixed cluster having members from 3.9.0, 3.9.1, 3.9.2 and 3.9.3, and

    • when rolling up from 3.8.x to 3.9.x.

Removed/Deprecated Features

  • The data structure IdGenerator is deprecated. It could produce duplicate IDs in case of a network split, even with split-brain protection being enabled. Use Flake ID Generator for an alternative implementation which does not suffer from the mentioned issue.

  • Following system properties are deprecated:

    • hazelcast.master.confirmation.interval.seconds

    • hazelcast.max.no.master.confirmation.seconds

    • hazelcast.mc.max.visible.instance.count

    • hazelcast.icmp.enabled

    • hazelcast.icmp.parallel.mode

    • hazelcast.icmp.timeout

    • hazelcast.icmp.max.attempts

    • hazelcast.icmp.interval

    • hazelcast.icmp.ttl

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.9.4

Enhancements

  • Introduced variable replacers. See its documentation here. #12809

  • Reporting of the ExecuteScriptRequest and ThreadDumpRequest: These two types of requests are sent from the Hazelcast Management Center to members. When they finish, there is no distinction between the logs (it is not obvious whether the responses end up with error or success). #12437

  • Version of the client should be added to the Statistics API. #12394

  • Outdated warnings regarding to time-to-live should be removed from IMap’s Javadoc. #12357

Fixes

  • Reliable Topic cannot continue to operate if owner and backup of the backed ringbuffer go down at the same time. #12919

  • NullPointerException is thrown when the method QueryCache.getAll() is called. #12918

  • Query cache listener which is configured declaratively is not registered properly. #12825

  • Query cache is created after each getQueryCache() call. #12814

  • Portable serialization is mistakenly reporting incompatible class definitions when nested portables are used in certain scenarios. #12733

  • ILock cannot the release lock properly based on the leasing time, if the lock name contains @. #12614

  • When map has a query cache, oldValue must be published after the method IMap.set(). #12576

  • The method IMap.addEntryListener throws NullPointerException when a null is provided instead of a listener object. The same check should be done in all similar addListener methods in all proxies. #12552

  • When an entry event listener is registered with oldValueRequired==false then both CacheEntryEvent#value and CacheEntryEvent#oldValue must be null or equal to the value that was removed/expired. #12507

  • When using AWS discovery on a Hazelcast client, if the initial connection fails due to a 503 error being returned from AWS, then the connection is not retried as per the configured retry count. #12457

  • IMap MBean is not created when a client reconnects to the cluster (when it uses cached map proxy). #12417

  • ClientJoinOperation should be moved to PreJoin: the client ownership information is carried via the method OnJoinClientOperation. It is invoked after the joins, which creates a potential race. In rare cases, it is possible that a recently joined member is not aware of the client, if a client is also joined recently. #12375

  • Authentication timeout logic must be revised to prevent the slow shutdowns: the clients need to wait all the scheduled tasks to finish before shutdown. #12362

  • The method HazelcastCache#getNativeCache should define a narrowed type: Currently, it does not narrow the return type to the actual native cache that is used internally. This requires unnecessary cast in the calling code. #12342

  • Shutting down members via console on Hazelcast Management Center should be disabled: it does not ask for credentials when stopping a member. #12328

  • The text connection send buffer is not correctly sized: it is based on the server setting, not on the client. #12326

  • Adding dynamic map configuration from HazelcastClient omits the time-to-live-seconds value. #12292

  • The methods QueryCache#destroy and then create does not give a fresh query cache. #12246

  • When using a HazelcastClient instance to add dynamic map configuration, it generally works for most of other parameters but the parameter timeToLiveSeconds is always 0. #12273

  • A dynamically added configuration for Replicate Map structure causes a ConfigurationException. #12222

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.9.3

Enhancements

  • Ping Failure Detector for Hazelcast Java Client: Ping Failure Detector provides a quicker and more deterministic detection of hardware and other lower level events. It has been available for Hazelcast members; now, it can also be used for Hazelcast Java clients. The related PR is #12049. See the Client Ping Failure Detector section.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Enhancements in the WAN Diagnostics: Introduced latency tracking for WAN endpoints and counters for sent and received WAN events. Note that WAN Replication is an Hazelcast IMDG Enterprise Edition feature. #12615

  • Introduced a system property to allow overriding the configured serialization byte order for better testing. #12145

  • Performed updates for JCache 1.1 compatibility. #12014

Fixes

  • CacheMergeOperation does not publish WAN events. #12186

  • Java serialization compression issue: Since GZipInputStream read more bytes than it needs, the fields coming after a compressed data is not able to be read. #12104

  • Setting the property hazelcast.mc.max.visible.instance.count to Integer.MAX or a huge value causes OOME. #12095

  • The Javadoc for SHUFFLE_MEMBER_LIST in the class ClientProperty is not correct. The shuffling is done when the related property (hazelcast.client.shuffle.member.list) is set to true. Also the description in the Hazelcast Reference Manual should be corrected accordingly. #12082

  • Endianness not taken into account consistently for serializer type IDs. #12077

  • Hazelcast client throws "AuthenticationException: Invalid credentials after owner member left". Clients should detect when a member is gone ungracefully. #12038

  • Query results should be converted to OBJECT in-memory format while creating the query-result-set, if the transaction is not originated from the client. #12050

  • IMap’s aggregate, projection, localKeySet and executeOnEntries methods throws exception when PartitionPredicate is being used. #11962

  • There is a race condition in the client’s Near Cache: "File is already being used by this Hazelcast instance". This was caused by the a problem with the synchronization between Near Cache creation and the destroy inside the Near Cache manager. #11648

  • The map screen of Management Center does not show throughput info for TransactionalMaps. #11640

Behavioral Changes

  • Default value of the system property hazelcast.mc.max.visible.instance.count changed from 100 to Integer.MAX_VALUE. So you may see more than usual data in your Management Center, if you have more than 100 data structures in your Hazelcast cluster.

3.9.2

Enhancements

  • Expiration operations can be stopped/canceled during the SHUTTING_DOWN state. Otherwise it causes graceful shutdowns which take longer, especially on clusters with a large amount of data, e.g., High-Density Memory Store backed maps. It is not necessary to block the shutdown for these operations since it will eventually be handled on the new owner of the entries. #11919

  • Changed Hazelcast base images from Debian to Alpine Linux. Hence, the size of the image is reduced. #48

Fixes

  • PartitionPredicate on MapAggregation throws exception when it is called from the client. #11923

  • Some client TransactionalMap operations deserializes on the server side, while IMap.values() does not do so. This causes serialization exceptions on a client-member setup when members do not have the serialization information, e.g., in case where portables are used. #11885

  • Replicated map’s Javadoc contains only member behaviors/requirements which the client cannot meet due to its technical difference. #11707

  • Ringbuffer access throws HazelcastInstanceNotActiveException when cluster state is PASSIVE. #11664

Coverity Fixes
  • The method addressTranslator.translate(target) of com.hazelcast.client.connection.nio.ClientConnectionManagerImpl.InitConnectionTask.run() may return null. The returned value is used as an argument for the method createSocketConnection(Address) which uses the value without a null-check. #11771

3.9.1

New Features

  • Ping Failure Detector: Ping Failure Detector provides a quicker and more deterministic detection of hardware and other lower level events. See the Ping Failure Detector section.

Fixes

  • MulticastJoiner may cause OutOfMemoryException due to the self-amplification in SplitBrainJoinMessage sending. #11836

  • Dynamic configuration added on the lite member is not found after a Split-Brain healing. #11815

  • Client transactions does not respect to unisocket behavior. #11802

  • Incorrect null log message in DelegatingAddressPicker. #11783

  • A high number of EntryEventType[] is created in a client application which receives many entry events. In a specific application, EntryEventType[] dominates the application’s overall memory allocation. #11777

  • Hazelcast client fails to connect to the cluster if the first cluster member is down. #11735

  • After promoting a member (HazelcastInstance.getCluster().promoteLocalLiteMember()) the instance of the member retrieved by getMembers() is not updated. #11720

  • Enqueued item gets lost when the client stops and reconnects sooner than hazelcast.client.endpoint.remove.delay.seconds. #11700

  • While trying to setup Hazelcast on AWS, the order of member initialization is wrong. #11681

  • Query cache on an IMap that is subsequently destroyed and recreated does not work. #11666

  • The method DefaultQueryCache::getAll does not check for null values. #11665

  • When the method HazelcastInstance.shutdown() is called, the following error is thrown: Could not retrieve ringbuffer service to destroy event journal. #11662

  • Defining management center in the configuration causes partition migrations even though there is no data. #11598

  • Weird Split-Brain like issue with Hazelcast 3.8.4. #11533

  • Consider logging new HTTP connections with DEBUG or lower level. #11196

  • HashSet and HashMap are initialized with a too small size. #9445

Coverity Fixes
  • Dereference null return value in the method`ClientConnectionManagerImpl.encodeAuthenticationRequest()`. #11767

  • Using invalid iterator in the method SmartClientListenerService.deregisterListenerInternal(String). #11760

  • Resource leak in client’s ProxyManager. #11751

3.9

New Features

Hazelcast IMDG Enterprise Features:

  • WAN Replication Using Discovery API: With this new feature, you can run multiple clusters within environments, that have Hazelcast’s Discovery SPI implemented, without manually configuring IP addresses and use Hazelcast’s WAN replication to keep them in sync. See the Defining WAN Replication Using Discovery SPI section.

Hazelcast IMDG Open Source Features:

Enhancements

Hazelcast IMDG Enterprise Enhancements:

Hazelcast IMDG Open Source Enhancements:

  • Client Support for User Code Deployment: You can now perform dynamic class loading from Hazelcast clients too. See the Client User Code Deployment section.

  • Event Journal: This new distributed data structure stores the history of mutation actions on data structures such as map or cache. See the Event Journal section.

  • Fine-Grained Anti-Entropy Mechanism: Allows to detect inconsistencies for each data structure in a single partition and replicate only the specific fragment of it instead of the whole partition. See the Consistency and Replication Model chapter.

  • Gigantic Cache Migration Enhancements: It enables manual control on the partition migrations. Partitions can be migrated/replicated in small fragments. See the hazelcast.partition.migration.fragments.enabled system property and NO_MIGRATION cluster state.

  • Storing Near Cache Keys by Reference: This enhancement provides a functionality to skip the serialization of keys for Near Caches to improve the performance of the Near Caches. See the serialize-keys option in the Near Cache section.

  • Client Connection Strategy Enhancements: Allows lazy initiation for Hazelcast Clients. See the Configuring Client Connection Strategy section.

  • Default Group Password and Symmetric Encryption Default Credentials Policy: Hazelcast requires the default group password and default symmetric encryption password/salt to be changed. See the Validating Secrets Using Strength Policy section.

  • Hazelcast Consistency Model Documentation: We added a new chapter explaining the full picture of Hazelcast’s consistency model. See the Consistency and Replication Model chapter.

  • Keeping JCache Implementation Updated: Hazelcast JCache implementation is updated for JSR 1.1 compliance when it is released.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Support for properties in RingbufferStoreConfig must be added. #11077

  • There should be the ability to add a timeout to get operations for Spring HazelcastCache. #10756

  • Exception causes get lost when using IExecutorService with the Hazelcast client. #10591

  • The attribute override-java-serialization should be added to the element global-serializer in hazelcast-client-config.xsd. #10449

  • Hazelcast should have a method like gerOrCreateHazelcastInstance() to detect a default configuration file and either create an instance or get an existing one. #10007

  • Allow binding any port without using the method setPortAutoIncrement() and specifying the port count. #9775

  • Consistent interface for IMap and TransactionalMap is needed. #8729

  • Hazelcast instance’s name should be added as a prefix to the name of the threads managed by Hazelcast. #8542

  • When back pressure is enabled members always use back-off and clients always throws HazelcastOverloadedException. This is not consistent and the behavior should be configurable on both members and clients. #8533

  • There should be a Spring property placeholder for cache-deserialized-values in the map configuration. #8487

  • For OperationService, the contention on the counters should be reduced. #4925

  • Some of the properties mentioned in GroupProperties are not used anymore, it should be cleaned up. #4505

  • When a grid member is configured to use port 0, it should be handled properly. #2506

  • There should be the ability to configure the distributed objects on the fly. #592

Fixes

  • Members cannot join to 3.9 cluster when using cache with a key/value type that is only resolvable from the user code deployment repository. #11505

  • Hazelcast listeners may fail to be registered to a new joined member. This is caused by a race condition between PostJoinOperations and the member doing the listener registration. #11490

  • Regression occurs after returning ResultSet instead of SortedQueryResultSet. #11489

  • Performance issue in MapProxyImpl.readFromEventJournal with projection. Projection cloning on the method readFromEventJournal must be skipped. #11410

  • There is an issue with MapStore: the method hcInstance.getMap(mapName) gets blocked when a new member joins and data is being loaded. #11407

  • Migration operation during the member startup raises ERROR level exception. #11406

  • SpringAware and NodeAware are not taken into account when injecting dependencies to various cache resources. #11406

  • When the method execute() from OperationServiceImpl is used and if it times out, the operation cannot send a response since responseHandler is null (assuming the caller did not set it initially). This leads to NullPointerException. #11375

  • In the method DataRecordComparator.isEqual(), the objects are always serialized so that the binary form is used to determine if something equals another object. This can lead to a lot of unwanted serializations. #11290

  • There is an inconsistent behavior for the method exists() of CacheEntryProcessorEntry. #11266

  • Hazelcast 3.8.4 source code does not match the JAR classes. #11261

  • OperationTimeoutException should include the cause of exception. #11233

  • There is no way to configure the port-count element within Spring context. #11224

  • Registration objects cause a memory leak while using ScheduledExecutor. #11221

  • Incorrect UnsupportedOperationException message: Event journal actions are not available when cluster version is 3.9 or higher. It should be .. are available ... #11220

  • loadClasses of client user code deployment throws ClassNotFoundException. #11217

  • Dead cluster member causes a high reconnect count. #11176

  • MapGetInvalidationMetaDataOperation throws NullPointerException in a Near Cache scenario. #11147

  • REST endpoints for cluster version management should be documented. #11139

  • CacheDestroyOperation must be invoked on the generic operation thread. #11137

  • Hazelcast members keep leaving and rejoining from/to the cluster in Docker (via the plugin Hazelcast AWS). #11118

  • The method ICache.getAndRemove() doesn’t invalidate the Near Cache. #11110

  • IScheduledExecutorService with @SpringAware does not inject ApplicationContext or Beans. #11108

  • IMap allows passing null collection in putAll and getAll, and null keys in loadAll. #11099

  • MapLoader NullPointerException occurs on loadAll(Collection) when the value is null and MapLoader implements PostProcessingMapStore or has an Interceptor. #11081

  • CacheNotExistsException is thrown when a new member joins a cluster where the primary member is creating and destroying JCaches. #11047

  • NullPointerException on ScheduledExecutor when handling multiple migration requests on the same source should be fixed. #11046

  • The exception NoClassDefFoundError: com/hazelcast/com/eclipsesource/json/JsonObject is thrown when using Payara Server and Hazelcast-AWS. #10994

  • There should be a warning or an information message when DiscoveryStrategy SPI is not on the classpath. #10993

  • There is a rolling upgrade compatibility issue for native queries in Hazelcast 3.9. It will fail when a 3.8 member send a QueryOperation to 3.9. #10973

  • SSL Incorrect Error Message: Memcached not enabled. Connecting to a Hazelcast Cluster that has SSL/Security disabled with a Hazelcast Client that has SSL enabled gives this error. #10971

  • Statistics getTotalGetLatency, getTotalRemoveLatency, and getTotalPutLatency works incorrectly. #10938

  • Map destroy doesn’t remove invalidation sequences and this causes OutOfMemoryException. #10936

  • Ensure dynamic configuration is available before HazelcastInstance is returned to user. #10926

  • Illegal character exception is thrown at index for the character ''^''. #10921

  • The method ScheduledTaskHandler.of(urn) fails when IPv6 address is used. #10898

  • There is a thread leak after calling the method HazelcastInstance.shutdown() in 3.8.2; so, JVM won’t exit. #10886

  • Transactional Queue: Backup reserve is failed, itemId: is not found. #10867

  • Map configurations inconsistency between members. #10860

  • PostJoinMapOperation is running on a generic thread; it means it cannot create high-density memory store indexes. It should probably spawn partition specific operations. #10841

  • The element config-permission cannot be configured in Hazelcast Spring XML. #10835

  • DynamicSecurityConfig and SecurityService lifecycle should be clarified. #10834

  • Eliminate synchronization when checking task permissions. #10833

  • Dynamic configuration tasks should be non-blocking. #10813

  • Hazelcast declares commons-logging (OSGi) as a required dependency, but it is not required. #10770

  • Comparing Version.UNKNOWN with other versions results in strange results. #10755

  • Multiple threads from Spring boot application are able to acquire lock on the same document id ( String ). #10754

  • The system property hazelcast.cache.invalidation.batchfrequency.seconds does not work for IMap. #10743

  • The method isEnterprise() for BuildInfoProvider does not work in Hazelcast 3.7.x. #10740

  • Infinite loop trying to initialize Cache configuration when deserialization fails. #10728

  • There is a race between CacheConfig addition and Proxy creation when a new member joins cluster. #10727

  • There is an infinite loop trying to initialize cache configuration when deserialization fails. #10723

  • Fail fast should occur when the cache’s in-memory format is incompatible with its eviction configuration. #10716

  • When a Java application is run with Hazelcast 3.8.2 within Docker, and after switching from openjdk:8u121-jre-alpine to openjdk:8u131-jre-alpine, IllegalArgumentException is thrown during cluster bootstrap. #10704

  • Performance Issue in Hazelcast 3.8.1: QueueStore.loadAll() is called way too often when draining items from Queue. #10621

  • Hazelcast instance is not injected into predicate. It occurs when the predicate is not serialized and deserialized because it was invoked on the partition owner. #10620.

  • Hazelcast client always tries to connect to localhost when using DiscoveryStrategy. #10606

  • Scheduled task remains cancelled after migration. #10603

  • SplitBrainMergeValidationOperation ignores the join check from [NODEB]:5701, because that node is not master. #10587

  • Problem setting up a cluster into Docker Swarm (overlay network).

  • New cache eviction is populated among nodes very slowly. #10470

  • WAN backup events are published twice. The publishWanReplicationEventBackups method is called in the run and afterRun method in the PartitionWideEntryBackupOperation. Elsewhere it is only called in afterRun. This needs to be checked but possibly the fix is just to remove the call in run. #10457

  • PortableFactory[-22] is already registered error when using Spring Boot 1.4.2 and Hazelcast 3.7.x. #10438

  • Source parameter is null on JCache.loadAll() call. #10328

  • TransactionalMapProxy depends on MapContainer. Also, creating a TransactionalMapProxy should not trigger creation of the backing MapContainer. #10254

  • It is impossible to stop a Hazelcast Client service if it has never connected to the server. #10237

  • TcpIpJoiner throws the exception`ConcurrentModificationException: null`. #10207

  • Cache.cacheManager may be overwritten with a different CacheManager. #10200

  • It seems like the Near Cache statistics seem to be off-by-one for at least the ownedEntryCount. Sometimes the ownedEntryMemoryCost seems to be affected as well. #10193

  • PagingPredicate does not work for the method executeOnEntries(). #10174

  • Setting up a Hazelcast listener in a Spring configuration format does not seem to work when using the class property of hz:listener. #10154

  • Test coverage for NearCacheClientCacheProxy should be increased. #10127

  • Fast Aggregations for any operator on empty arrays do not work. #10126

  • CachingProvider should attempt to get or create HazelcastInstance with the default configuration when only instance name is provided. #10094

  • MulticastDiscoveryStrategy does not work correctly with client discovery. #10089

  • Updates with IMap#putTransient should trigger EntryUpdatedListener. #10077

  • The method loadAll(boolean replaceExistingValues) does not reload the map store after the method cache.evictAll is called. #10057

  • Hazelcast XML configuration does not allow RANDOM eviction as an eviction policy. #10053

  • There is a race condition in TestClientRegistry where the tests are calling blockFrom and blockTo before any connection was made (via createSocketConnection). This causes NullPointerException in those block methods. #10021

  • Client side query cache declarative configuration does not support wildcard usage mapName configuration. #9990

  • When health check is enabled, the user can send a garbage request like http://<your member’s host IP>:5701/hazelcast/healthqqq and it returns a correct response. #9967

  • Durable Executor Service re-executes the completed tasks in case of a member failure. #9965

  • There is a problem with virtual IP assignments when Hazelcast is used into a Docker Swarm cluster. #9963

  • Currently there is no Spring support for Near Cache preloader. #9771

  • MapStore: write delay is not precisely respected as it has been in the previous Hazelcast releases (before 3.7.4). #9745

  • The option cache-local-entries is not supported at the client side Near Cache configuration. #9712

  • Latest member-list may not be received when FinalizeJoinOperation invocation timeouts. #9501

  • Cluster member-list update operations are not ordered, new member addition and removal operations can get reordered on receiving/processing side. Also periodic member-list publish operation has no order with other member adding/removing operations. That can cause having different member lists on different members. [#9486]

  • Hazelcast Client API (3.7.3) is not able to execute get/put/delete operations on the maps when used in AWS with smart-routing enabled (values and clear operation work). #9419

  • Single map with ten items and three members, Split-Brain case: After isolating one member, the merge occurs only with some of the keys, not all of them. #9358

  • When attempting to start a cluster member (3.6.5) with JMX enabled, IllegalArgumentException is thrown and the member is self-terminated immediately. #9293

  • Backup is lost if maxIdle property is used. #9153

  • MultiMap lock: Thread is getting stuck when calling lock(key). #9055

  • Near Cache on the Hazelcast Client side returns old values. #8838

  • Behavior of TTL when it is a negative value should be clarified. #7729

  • Heartbeat only removes a member if it is related to the master member. #5253

  • Multicast discovery does not work after loading some data. #4721

Behavioral Changes

  • Maximum timeout of heartbeat for a member to assume it is dead was 300 seconds. Starting with Hazelcast 3.9, it is reduced to 60 seconds. Related property is hazelcast.max.no.heartbeat.seconds. Also, starting with Hazelcast 3.9, maximum timeout of master confirmation from other members is reduced to 150 seconds from 450 seconds. Related property is hazelcast.max.no.master.confirmation.seconds.

  • Starting with Hazelcast 3.9, the default Cache merge policy is Put if Absent. It was Pass Through before Hazelcast 3.9.

  • Starting with Hazelcast 3.9, the format of member list shown in the logs is changed. Before Hazelcast 3.9, it was like the following:

    Members [3] {
        Member [127.0.0.1]:5701 - c1ccc8d4-a549-4bff-bf46-9213e14a9fd2 this
        Member [127.0.0.1]:5702 - 33a82dbf-85d6-4780-b9cf-e47d42fb89d4
        Member [127.0.0.1]:5703 - 813ec82f-9d9e-4712-bae1-6c95b32d6d7d
    }

    Starting with Hazelcast 3.9, it is shown as follows:

    Members {size:3, ver:3} [
      Member [127.0.0.1]:5701 - e40081de-056a-4ae5-8ffe-632caf8a6cf1 this
      Member [127.0.0.1]:5702 - 93e82109-16bf-4b16-9c87-f4a6d0873080
      Member [127.0.0.1]:5703 - 06fb4e61-9757-443b-a19f-7af1f3966f30
    ]

    Here, you can see the size of your cluster (size) and member list version (ver). The member list version will be incremented when changes happen to the cluster, e.g., a member leaving from or joining to the cluster.

    You can set the system property hazelcast.legacy.memberlist.format.enabled to true if you want to see the member list in its old format.

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.8.9

No changes for the Hazelcast IMDG Open Source edition. This version is released due to a change in the Hazelcast IMDG Enterprise Edition.

3.8.8

The following is the enhancement for Hazelcast 3.8.8 release.

  • MulticastJoiner’s split-brain handler should broadcast a new split-brain message only when it detects the other side should join, not on every merge interval. #11888

3.8.7

The following are fixed issues and enhancements for Hazelcast 3.8.7 release.

  • MapStore: Map initialization is blocked when new members join during the data loading. #11407

  • Split-brain is not working as expected on a 2-member cluster. There occurs a cluster merge deadlock when the core size is 1. #11403

  • Unable to connect any address in the configuration. #11288

  • OperationTimeoutException thrown at Hazelcast clients should include a more informative text. #11241, #11236

  • Stale reads must be guaranteed from the map inside a transaction, when Near Cache and delayed MapStore are enabled. #9248

  • A GroupProperty INDEX_COPY_BEHAVIOR was added to enable fine-tuning of index query behavior. Related to performance degradation reported in #11280

3.8.6

The following are fixed issues and enhancements for Hazelcast 3.8.6 release.

  • Fixed local execution Spring context injection on the Scheduled Executor tasks. #11374

  • toHeapData should be used when converting native data to heap data. #11361

  • Prefix for the diagnostics file name should be configurable. #11345

  • Client 3.8.4 (and newer versions) with near cache gets java.lang.UnsupportedOperationException when it is connected to 3.7.x cluster. #11343

  • Notifications from a Hazelcast member through ITopic listeners (MessageListener) are being lost. #11264

  • Address validation for the client network configuration is missing. #11242

  • The client is unable to connect to any address in the configuration, after some hours. #11194

  • There are errors when using discovery strategy in the Hazelcast client. #11116

  • Hazelcast terminates the listeners incorrectly (for 3.7.6). #10101

  • The explanation for InvalidConfigurationException (for query caches) should be more clear. #9988

3.8.5

The following are fixed issues and enhancements for Hazelcast 3.8.5 release.

  • Fix for map-loader put in case of size-based eviction NPE. #11253

  • Checked existence of key in-memory before proceeding. #11251

  • Registration objects cause memory leak while using ScheduledExecutor. #11227

  • Implement HTTPCommunicator using Apache HttpClient. #11214

  • Notify invocations when a member leaves in FROZEN/PASSIVE cluster state. #11202

  • The client should not use localhost if it is not explicitly configured and the discovery is configured. #11199

  • Query cache fix for map destroy. #11197

  • Add system property to process WAN map remove events as delete. #11178

  • Add forgotten public address check from cache. #11169

  • CacheDestroyOperation must be invoked from generic operation thread. #11165

  • RemoveOperation returns old value causing SerializationException in WAN replication when sources are not available and the store type is binary. #11157

  • Error in Spring configuration: ${hazelcast.queue.capacity} is not a valid value for ’integer''. The property queue-capacity’s type should be changed. #11150

  • The Javadoc for the method MapEvictionPolicy.compare is not correct. #11074

  • When handling multiple migration requests on the same source, NullPointerException can be thrown due to the null future reference from the previous migration processing. #11067

  • An exception can’t be rethrown from a different thread; it will lead to exceptions with very obscure stacktraces. #11022

  • Wrong property name partition.count in XSD files. #10544

  • Hazelcast 3.7: PollOperation invocation fails to complete due to operation-heartbeat-timeout. #8831

3.8.4

The following are fixed issues and enhancements for Hazelcast 3.8.4 release.

  • Added timeout option for the get operation when Hazelcast is used within the Spring context. #11001

  • HTTP HEAD support added to the REST interface. #11000

  • ClientProxy gets NullPointerException from the method getClient. #10961

  • Added support for collecting and tracking statistics for all reliable topic proxies on a local member. The statistics are destroyed when the proxy is destroyed. #10932

  • The method containsKey returns true even the key does not exist in the map. #10923

  • Assuming a scenario in which invalidations are generated previously for some other near caches and then later a new near cached map is created, all entries that are near cached before the first reconciliation period will be removed. #10913

  • HTTP upload to Hazelcast fails. The code fails with the exception BufferOverflowException. #10878

  • The method MapProxyImpl.executeOnKeys() does not work when ConcurrentSkipListSet is passed and throws NullPointerException. #10877

Known Issues

  • Upgrading from 3.6.x to 3.8.x (Hazelcast IMDG Enterprise) when using JCache: Due to a compatibility problem, CacheConfig serialization may not work if your member is 3.8.x where x < 5. Hence, you will need to use Hazelcast 3.8.5 or higher version where the problem is being fixed. This issue affects Hazelcast IMDG Enterprise only.

3.8.3

The following are the fixed issues for Hazelcast 3.8.3 release.

  • Hazelcast 3.8.1: calling replace(K, old, new) keeps the expiry time as creation time + duration. But Hazelcast doesn’t seem to honor this expiration even after calling get(). #10817

  • When using a client side Near Cache, the cache statistics seem incorrect when asking for a key that doesn’t exist. #10806

  • Ringbuffer is not adjusting headSeq while setting tailSeq due to capacity. #10774

  • Older versions of Mulesoft clash with Hazelcast Client classpath. #10706

  • Fixed an issue where NonBlockingIOThread was not shutting down since a thread interruption could be suppressed/cleared in the flow of operations; an explicit stop flag has been introduced to close it. #10651

  • In the case when the master node is terminated in the middle of a 2-phase transaction, the rollback for that transaction is not performed. This causes loss of data. #10637

  • Adding/removing listeners fails if a member is removed in the progress. #10595

  • When two callbacks are registered as callback to ClientDelegatingFuture, if one of them wants to deserialize the response while the other is does not, Hazelcast falsely gives null response to the one waiting a deserialized value. #10581

  • Eviction and max-size for less then 271 cause map to never populate. #10179

  • IOException at the client side must be produced only if hazelcast.client.invocation.timeout.seconds is passed. #9696

3.8.2

The following are the fixed issues for Hazelcast 3.8.2 release.

  • Even though a CachePermission already exists, the respective PermissionType does not exist and cannot be configured programmatically or declaratively. #10503

  • Management Center reports incorrect values for backups. Given that there is one copy of the data across a native map, it is expected to see identical memory footprint on the entry and the backups with identical sum across a server. But it is not the case. #10454

  • There is a memory leak caused by ThreadGroup. #10394

  • Split-Brain messages still occupy the memory after recovery. #10325

  • When shutting down a client with a topic subscription, it logs a warning level exception which seems harmless. It is not good to log a harmless exception as warning. #10291

  • There is a race condition in the back pressure at the client side which causes the back pressure not to work. #10253

  • Ringbuffer OutOfMemoryError: GC overhead limit exceeded. #10189

  • ClientContext is created for every proxy and it causes unneeded resource allocations. #10117

  • Hazelcast terminates listeners incorrectly. #10101

  • When hostname is used in the Hazelcast member configuration instead of IP address, NullPointerException is thrown. #9884

  • When using a Hazelcast Client instance, if the redoOperation is set to true, operations which failed to be sent to the cluster are retried, but failures from executing the operation itself in the cluster are not. #8545

Behavioral Changes

  • Starting with 3.8.2, members with the same group name but with different group passwords (that do not use authentication) form a cluster. Before this release, group passwords needed to be the same too. Please see Cluster Groups for more information on creating cluster groups.

3.8.1

The following are the enhancements introduced with 3.8.1 release.

  • Mutual authentication with TLS certificates: It allows the clients also to have their keyStores and members to have their trustStores so that the members can know which clients they can trust.

  • Introduced Offloadable and Readonly entry processors in order to significantly improve entry processors’ throughput. Please see the Entry Processor Performance Optimizations section.

The following are the fixed issues for Hazelcast 3.8.1 release.

  • BackupEntryProcessor returned from AbstractEntryProcessor does not support HazelcastInstanceAware. #10083

  • Updates performed with IMap’s putTransient method should trigger EntryUpdatedListener. #10077

  • Missing documentation for configuring log4j2 as logging framework. #9970

  • Management Center section in the Reference Manual is misleading when it explains tool’s behavior for clusters having more than two members. #9969

  • JCache bootstrap should have a way to reconciliate the existing HazelcastInstance. #9958

  • Multicast configuration’s trusted-interfaces element is not handled properly on startup of the members. #9953

  • For Hazelcast 3.7.*, MigrationListener’s migrationStarted and migrationCompleted methods are called for all partitions. #9919

  • While performing async operations test, OOME is thrown due to exceeding garbage collection overhead, at the client side. #9665

  • Behavior of aggregators when extracted value is null should be well-documented. #9858

  • Hazelcast Java client disconnects from the cluster with heartbeat problems. #8956

  • Currently the near cache updates does not guarantee the order of events for a certain key. The updates are performed via asynchronous executor runs and it does not guarantee the order of updates. Furthermore, the invalidations via event service is also not ordered. #8652

  • Client’s near cache implementation uses userExecutor which is also used for other purposes such as making callbacks to lifecycle listeners. A client may block an operation of the near cache updates by a slow lifecycle listener. This needs to be addressed. #8651

  • Client has a configurable hazelcast.client.max.concurrent.invocations property which limits the outstanding client requests. It is observed that at some cases, for async calls, it is possible that this limit may not work as expected and this may cause outstanding requests to grow which may cause OOM. #8568

  • When using a Hazelcast Client instance, if the redoOperation is set to true, operations which failed to be sent to the cluster are retried, but failures from executing the operation itself in the cluster are not. #8545

  • While updating the Hazelcast maps in the clustered environment, the exception TargetNotMember is thrown. #7584

3.8

This section lists the new features, enhancements, fixed issues and, removed or deprecated features for 3.8 release.

New Features

The following are the new features introduced with 3.8 release.

Hazelcast IMDG Enterprise Features:

  • Rolling Member Upgrades for Minor Releases: You can upgrade your Hazelcast members to the next minor release without service interruption. See the Rolling Member Upgrades chapter.

  • Restore Hot Restart Data in a Different Cluster: You will be able to restore Hot Restart data on a completely new cluster with different IP addresses or on a cluster of machines formed by existing and new IP addresses. Since this is a removal of a restriction or limitation, nothing will change for current Hot Restart Store users. See the Moving/Copying Hot Restart Data section.

Hazelcast IMDG Open Source Features:

  • Continuous Query Cache: With this release, the previously Enterprise feature Continuous Query Cache becomes open source. See the Continuous Query Cache section.

  • Cardinality Estimator: Cardinality Estimator data structure is implemented on top of Hazelcast to estimate cardinality. See the Cardinality Estimator Service section.

  • Scheduled Executor Service: Distributed implementation of Java’s ScheduledExecutorService API. See the Scheduled Executor Service section.

  • User Code Deployment: Load your new classes to Hazelcast IMDG members dynamically without restarting all of them.

Enhancements

The following are the enhancements introduced with 3.8 release.

  • Fast Aggregations for Queries: See the Fast Aggregations section.

  • Projections for Queries: See the Projections section.

  • Eventually Consistent Near Cache: With this improvement, possible losses of invalidation events are detected and stale reads are prevented. See the Near Cache Eventual Consistency section.

  • Cluster Split-Brain Protection for Queue and Lock: Split-brain protection is now available for the queue and lock data structures. See the sections Lock Split-Brain Protection and Queue Split-Brain Protection.

  • Unified Near Cache Implementation: Different near cache implementations for JCache, map, client side and member side are unified. See the Near Cache section.

  • High Performance Near Cache: A near cache preloader is introduced to preserve the information about which keys are used in a populated near cache. See the Near Cache Preloader section.

  • removeAll(predicate) for Map: You can remove all map entries that match your predicate. See the Removing Bulk Map Entries with Predicates section.

  • Ringbuffer with Persistent Datastore: Implemented for the Ringbuffer which is similar to the existing queue store mechanism. See the Ringbuffer with Persistent Datastore section.

  • Hazelcast Striim Hot Cache Adapter: Hot Cache functionality based on Striim’s Change Data Capture (CDC) for the underlying database updates. Note that this enhancement is available for Hazelcast Enterprise. See the Hazelcast Striim Hot Cache section.

  • One-off WAN Synchronization with Dynamically Added Endpoint: Improvement to allow copying the maps’ all content to a target cluster dynamically in runtime. Note that this enhancement is available for Hazelcast Enterprise.

  • Hot Restart Usability Improvements: When a cluster with Hot Restart enabled is restarted, it starts in the ACTIVE state. Note that this enhancement is available for Hazelcast Enterprise. See the Hot Restart Types section.

  • Hot Restart with Incomplete Members: The restart procedure for a cluster having Hot Restart enabled is now more flexible by allowing you to do a partial start, which basically means that once the cluster is shutdown, it can be restarted with some missing members. Note that this enhancement is available for Hazelcast Enterprise. See the Partial Start section.

  • Copy Hot Restart Store data from Source to Target: You can copy Hot Restart Store data from a running cluster to a target without shutting down the source cluster. Note that this enhancement is available for Hazelcast Enterprise. See the Moving/Copying Hot Restart Data section.

  • Improvement in WAN Queue Full Behavior: Now, Hazelcast throws an exception when the queue is full and replication is active, unless it is stopped on the Management Center. Note that this enhancement is available for Hazelcast Enterprise. See the Queue Full Behavior section.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Currently there are only MERGING and MERGED states for lifecycle, but no event is fired for a failed merge. This would be informative and further give an idea to user to understand a reason of a SHUTTING_DOWN event since a failed merge is followed by a shutdown. #9856

  • Tests are needed to prove that removeAll does not deserialize portables. #9474

  • Client support for distributed scheduled executor service. #9385

  • When a member without JCache service joins a cluster where JCache service is enabled but not used, the member should not fail. #9301

  • It would be nice to have names assigned to all threads created by Hazelcast (and not the default ones, e.g., pool-2-thread-1, pool-2-thread-2). #9210

  • There should be a more detailed and clarifying information in the diagnostics of PartitionIteratingOperation. #8855

  • When there is no access to NodeEngine, the logger should be able to be used from a static context without any side effects to the logging configuration. #8760

  • Client tasks should be bounced back when a member has not completely start yet. #8715

  • A method size(Predicate) should be added to determine the size of predicate results. #8555

  • Currently the property com.hazelcast.map.impl.eviction.ExpirationManager.PERIOD is hard-coded to 5 seconds. Making it a configurable property will help in tuning the eviction. #8277

  • The method com.hazelcast.topic.impl.TopicProxy#publish serializes the message even if the message is never sent due to having no subscribers. In the case of non-global-ordering and non-client use, the implementation should defer serialization until it has established that there are subscribers. #8262

  • Hazelcast should provide an abstract Mapper implementation which invokes the Supplier given to Aggregation#getMapper(). #7477

  • Configuration setting for WAN replication queue should be only for primary entries. #7124

  • Currently if you want to use Container-Managed Transactions with Hazelcast within a JEE Application Server, you need to use the provided JCA Adapter. This JCA Adapter creates a Hazelcast instance. This should be allowed: configuration of the JCA Adapter to be either Server or Client-based. #5667

  • Currently PartitionIteratingOperation consumes a generic operation thread, while the operations for the partitions are running. This means that the generic thread pool can dry out and this can cause a lot of problems for important tasks that cannot be executed since the thread pool is empty. A PartitionIteratingService should be introduced where the system can lookup TraceableOperations. #4889

  • Hazelcast Shutdown Hook calls the method terminate() instead of shutdown(). #4648

  • Hazelcast should support ScheduledExecutorService. #115

Fixes

The following are the issues solved for Hazelcast 3.8 release.

  • TaskDefinition should be removed from the client protocol. #9846

  • AggregationResult should be serialized via Codec instead of IDS. #9845

  • MetaDataResponse should be serialized via Codec instead of IDS. #9844

  • The script stop.bat does not stop the running Hazelcast instances. #9822

  • The script start.bat allows multiple instances to be started in the same folder; it should not. #9820

  • Ordering consistency for the the method writePortable should be explained in the Reference Manual. #9813

  • A client cannot add index to map using Portable values without registering Portable classes on member. #9808

  • Index is not updated after entry processors change values. #9801

  • When using Entry Processor and index on a field the Entry Processor uses, wrong data is returned when executing a Predicate. #9798

  • NullPointerException in LockServiceImpl: an NPE is thrown in LockServiceImpl when creating one map with a quorum configuration (which activates the QuorumServiceImpl for all QuorumAware operations) and then calling a QuorumAware operation on another data structure without a quorum configuration. #9792

  • ClusterVersion is missing convenience methods for comparing easily. #9789

  • When scheduling tasks with multiple executors, calling getAllScheduled on one of them returns tasks scheduled per member but for all executors, rather than for the one we called the getAllScheduled from. #9784

  • There are too many log entries when quorum is not present. #9781

  • Hazelcast terminates listener incorrectly. #9766

  • When storeAll/deleteAll methods are used in the MapStore interface, it is possible that the underlying implementation has stored/deleted some entries from the supplied batch and then gets an exception on some particular entry in the middle of the batch. In order to handle this scenario properly the MapStore could remove from the supplied batch entries which were properly stored in the underlying persistent store. The only thing to improve in the DefaultWiteBehindProcessor is to properly synchronize the map passed to processBatch method with the initial batchMap. #9733

  • If hazelcast.discovery.enabled on the client side: If neither DiscoveryServiceProvider nor DiscoveryStrategyConfig is specified, an NPE is thrown. A more helpful exception should be provided. #9722

  • The hazelcast-all.jar, which is the one included in Hazelcast Docker, downloaded from Maven repo, contains an outdated version of hazelcast-aws library. #9698

  • When scheduling on random partitions and trying to get all the scheduled tasks from the service, the number of tasks is wrong. The returned figure seems to match num_of_tasks * backup_count. #9694

  • When a scheduled Callable/Runnable throws an exception, invoking future.get() on the client side does not throw the expected ExecutionException. #9685

  • The client is not updating listener registrations when the member restarts with a new version. #9677

  • IScheduledExecutorService does not use HazelcastInstanceAware on local operations. #9675

  • Near cache compatibility with 3.7 clients is broken for TTL expirations. #9674

  • Fix inconsistent Ringbuffer TTL defaults: when starting a HazelcastInstance without an explicit configuration (HazelcastInstance hz = Hazelcast.newHazelcastInstance();), the default Ringbuffer TTL was 30 seconds, instead of correct value 0. This change may break existing applications relying on the default configuration. #9610

  • Replicated Map configured with async-fillup as false prevents further members from joining the cluster. Members are stuck in an endless loop. #9592

  • When an exception is thrown from an InitializingObject’s initialize method, it is logged in WARNING level but then swallowed and the proxy object is added to the ProxyRegistry. Instead, the exception should result in removing the object from the ProxyRegistry, as it is already done for exceptions thrown from service.createDistributedObject. #9578

  • Near cache pre-loader fails when invalidations are enabled. #9536

  • There is a memory leak in MetaDataGenerator when (1) near cache invalidations are enabled, (2) map/cache is created and destroyed, and (3) the map name is stays referenced from MetaDataGenerator. #9495

  • JCache client needs access to CacheLoader implementation to work. #9453

  • The matching wildcard is strict, but no declaration can be found for element hz:hazelcast. #9406

  • Connection Reset warning shows up in the class CloudyUtility. #9404

  • Key state marking in near cached getAll should be improved. Instead of key based marking, partition based approach can be used. #9403

  • Scheduled executor service tests are too slow. #9386

  • The method getAll is not updating the localMapStats. #9373

  • Hazelcast 3.7.3 and Spring Boot 1.4.2 get the error PortableFactory[-22] is already registered. #9353

  • There is an issue when integrating with WebLogic 12c. Classloading logging should be added when DefaultNodeExtension is loaded by multiple classloaders. #9291

  • The method MapLoader.loadAll(true) does not reload after reloading with new keys. #9255

  • Problems with timeout in blocking calls. #9250

  • Stale reads from Map inside a transaction should be guaranteed, when Near Cache and delayed MapStore are enabled. #9248

  • Client protocol cannot handle nulls in the collection. #9240

  • LifecycleState events are not received when Hazelcast docker image is used. #9189

  • If there is no network, multicast socket exception is thrown. #9081

  • Hazelcast is not handling XML configuration attribute for DurableExecutorService. #9078

  • If DurableExecutorService.retrieveResult(long) is run from a client for a task that does not exist, two exceptions appear at the client side. At the member side, it logs StaleTaskIdException. This should not be logged to the member if re-thrown to the client. #9051

  • When retrieving data from Java Hazelcast instance, HazelcastSerializationException is thrown from the .NET client. #8985

  • Callback for time to live (for map eviction) is invoked with a significant delay. #8894

  • Warning 2 in IMap::tryRemove method’s code comments is not correct. This method returns only a boolean. #8884

  • There are too many log messages when members of a cluster are not joined to another cluster (multicast) having a different group name in Hazelcast 3.6 and 3.7, which is already expected. #8867

  • When clients lose connection to the members in the cluster, even after the clients are reconnected to the cluster, the Entry/Message Listeners are never fired again. #8847

  • Nested queries on VersionedPortables with different versions cannot be performed. Hazelcast throws IllegalArgumentException. #8846

  • Fixed max-size value behavior for Near Cache configuration. If you set 0 for the max-size, now, it is valued as Integer.MAX_VALUE as expected and documented. #8826

  • While generating the XML string from a configuration object in ConfigXmlGenerator.generate() method, native-memory configuration is not appended to the result string. This causes the native-memory configuration for members not to be shown on the Management Center. #8825

  • Client heartbeat is timed out, closing connection warning shows up indefinitely on AWS. #8777

  • Spring XML configuration: The smart routing attribute defaults to false while the documentation says it defaults to true. #8746

  • Distributed objects present in the cluster do not show the same values. #8743

  • In Hazelcast 3.6.3, the error Current thread is not owner of the lock shows up when using Hazelcast client to perform the actions. #8730

  • There is an unchecked CancelledKeyException when client disconnects from the cluster. #8681

  • Bulk get operation with 100 keys fails. #8535

  • An IllegalThreadStateException may be thrown sometimes on a member of a cluster with two or more members when a Hazelcast client creates a new map with configured indexes. This exception is not thrown when indexes are created programmatically. #8492

  • Group and password attributes in Hibernate integration module should be validated. #7978

  • There appears to be an inconsistency where JCache uses the EvictionConfig object, but IMap directly uses properties of the NearCacheConfig. #7788

  • Topic listeners are lost after merging from a split-brain scenario. #7742

  • Member attributes go missing after merging from a split-brain scenario. #7697

  • There should be a distributed class loading feature to perform distributed loading of classes so that jobs like MapReduce and Aggregate can be done more effectively. #7394

  • MapLoader misses state updates on second load on the receiver members. #7364

  • If the system retries an operation because a member is leaving the cluster, the invocation can be retried due to the response but also due to the member-left event. In most cases this should not lead to a problem, but the invocation may be executed twice. This can be a problem and can lead to a permanent inconsistent system. #7270

  • Using Lock and Map, the heap memory on Hazelcast server keeps increasing. All the created Locks and Maps are destroyed by the method destroy(), but the memory cannot be destroyed by the garbage collector even via FGC. When an application runs for some time, Hazelcast server goes down. #7246

  • Hazelcast 3.5 throws InvalidConfigurationException. #6523

  • More granular mutexes should be used when creating and removing a set of closeable cache resources. #6273

  • Running PredicateBuilder queries when nodes join causes timeouts and cluster crash. #6242

  • The method setEvictionPolicy for near cache configuration wants a string argument whereas the same for map configuration wants an enum value. #2010

  • The configuration schemas Hazelcast Spring XSD and Hazelcast default configuration XSD are out of sync. They must have the same elements. #5034

Behavioral Changes

  • A separate thread pool for query requests coming from the clients has been introduced. The goal for this is to avoid OutOfMemoryException s under heavy query load. This separate thread pool’s size is configurable via the system property hazelcast.clientengine.query.thread.count.

Removed/Deprecated Features

  • Aggregators feature is deprecated. Instead, Fast-Aggregations feature should be used.

  • MapReduce feature is deprecated. See the MapReduce Deprecation section.

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.7.8

The following is the fixed issue for 3.7.8 release.

  • When the cluster state is PASSIVE, the background expiration task can get an exception (in ClearExpiredOperation) during execution. But after the cluster state transitions to ACTIVE, it should continue to process. The task’s progress conditions should be reset in case of any execution exception. #10636

3.7.7

The following is the fixed issue for 3.7.7 release.

  • Servers should throw exception when message type is not known. Client protocol is defined to throw an exception when a client sends a client message with unknown type to server. #10196, #10203

3.7.6

The following are the fixed issues for 3.7.6 release.

  • In the case of using multiple lite members with near cache, when a clear is called it tries to use the same operation and results in IllegalStateException. #9879

  • Near cache eviction does not work. #9723

  • Connection exception is logged in ClientListenerSupport after the client disconnects. #9709

  • IMap entry eviction issue; some keys stay forever. #9095

3.7.5

The following are the fixed issues for 3.7.5 release.

  • Map statistics are disappeared in Management Center when the member is restarted. #9590

  • Clients are not updated when DistributedObject is removed from the member. #9457, #9423

  • Hazelcast Management Center is not working with Tomcat 8 and higher. #9452

  • TextReadHandler (for REST and Memcache) commands has a size restriction. Large responses cannot be handled. There is a hardcoded buffer size in TextReadHandler but it should be converted as dynamic so that it can handle responses of any size. #9355

  • The configuration schema hazelcast-spring.xsd should use <xs:all> instead of <xs:sequence> for hazelcast-bean base type. #9289

  • Stuck threads are seen in Hazelcast 3.6.2 client when session replication is used. #9060

  • When cluster service starts when a member starts, it registers itself as the only member and publishes the membership event. QuorumService is a MembershipAwareService, therefore it is expected to update quorums when it receives these membership events. The problem is as follows: services are not registered to ServiceManager yet when the cluster service publishes this event on a member start. So, it seems that QuorumService is missing it and failing to update its internal quorum status. #8820

  • Client disconnection leads to a stuck clusterExecutor shutdown in Hazelcast 3.6.2. #8721

  • Near cache updates may cause OutOfMemoryException under high load and high number of ICache.getAsync calls. #8593

  • The tag hot-restart cannot resolve the placeholder for the value of enabled and fsync within Spring context. #8500

  • HazelcastInstanceAware should be supported for QuorumFunction. #7499

3.7.4

The following are the fixed issues for 3.7.4 release.

  • Client protocol should be updated to support old .NET clients that have an issue preventing them to connect to 3.7.3 and newer clusters. #9388

  • Fixed keyStateMarker problems. #9382

  • When nodes terminates concurrently, clusterService.removeAddress can fail because target node is already shutdown. #9324

  • Cache event listeners trigger multiple events when the listener is configured declaratively (XML). #9315

  • When client is connected and the client process is paused, e.g., using a debugger, then the member starts printing the warning message (Client heartbeat is timed out) every 10 seconds. #9308

  • Hazelcast client outside the AWS cloud cannot connect to Hazelcast member inside the AWS cloud. #9290

  • In case of a retrying operation, an old heartbeat could still be stored on the invocation. If the invocation monitor sees this, it does not understand that this is from an old invocation. It concludes that the invocation has not received any recent heartbeats and aborts the invocation. #9287

  • Client endpoint may have a different connection than the one it was initially constructed. #9282

  • The method loadAll(keys) has to notify RecordStores about the end of loading. #9260

  • Partitions should be unavailable/inaccessible until MigrationAwareServices complete commit/rollback during migration and/or promotion. #9235

  • When backups are reordered and a backup with a version greater than (localVersion + 1) is received, backup is applied but partition replica versions are not updated. #9206

3.7.3

The following are the fixed issues for 3.7.3 release.

  • MapLoader.loadAll(true) does not reload after reloading with given keys. #9263

  • The scheduler in TcpIpConnectionManager is not completing after the method instance.shutdown() is called. #9187

  • EntryView’s storeTime and expirationTime are not available even though they are set. #9166

  • NullPointerException is thrown while deleting an entry through Entry Processor by setting value to NULL. #9135

  • There is a discrepancy between the Reference Manual and source code in the default value of hazelcast.heartbeat.interval.seconds. #9070

  • When Hazelcast is upgraded from 3.6.5 to 3.7.1, calls to Spring cache generate exceptions. #9023

  • Hazelcast 3.6.x client cannot obtain cache proxy from 3.7.x cluster. Configuration creation and retrieval seem to be not working on 3.6.x clients when they connect to a 3.7.x cluster. Two new system properties added, namely hazelcast.compatibility.3.6.server and hazelcast.compatibility.3.6.client. When upgrading from (assuming that you are using ICache) 3.6.x to 3.7.3 then the servers should be started with -Dhazelcast.compatibility.3.6.client=true to enable existing 3.6.x clients work properly. When upgrading from 3.7.1 and 3.7.2 to 3.7.3 this is not needed.https://github.com/hazelcast/hazelcast/issues/9006[#9006]

    Please note that there is no change in the Hazelcast cluster member upgrade procedure; the cluster members cannot have mixed major versions (e.g., a cluster of 3.6.x member and 3.7.x member). The above change only relates to clients (the mixed clients are supported starting from version 3.6 with the Open Binary Client Protocol). Hence, you can have a 3.6.x client running with a 3.7.x cluster.
  • When clients lose connection to the nodes in the cluster, even after the clients reconnected to the cluster, the Entry/MessageListeners never fired again. When you register a listener, the registration to the members are invoked with a timeout value of connection-timeout. Keeping this value short can cause a timeout exception during a listener registration. Therefore, you may need to increase the default value (i.e., 5 seconds), see the #Setting Connection Timeout section. #8847

  • There is an unnecessary warning about member side near cache invalidation operations. #8841

3.7.2

The following are the fixed issues for 3.7.2 release.

  • The container WaitingOperation (which wraps BlockingOperation) is asked for call ID/caller address. But this container object does not have any sensible information, so the heartbeat is not constructed correctly for a blocked operation. #8979

  • JCache backup operation does not propagate the exception CacheNotExists. #8955

  • The log message unsafe unavailable is at WARNING level and it also displays the full stack trace. #8852

  • When a predicate for distributed query is executed on a lite member it throws the exception ClassNotFound. #8849

  • While running under non-HotSpot JVM, the exception NoSuchField is thrown. #8835

3.7.1

The following are the fixed issues for 3.7.1 release.

  • Behavior of JCache near cache invalidation events changed: (i) the method Cache.close now delivers near cache invalidation events, (ii) the method Cache.destroy no longer delivers invalidation events, (iii) the method CacheManager.destroyCache when invoked from a client-side cache manager no longer delivers invalidation events. #8801

  • Aggressive SQL predicate flattening may result in wrong parsing of SQL. #8799

  • To properly configure within Spring context, incorrect client defaults should be fixed. #8761

  • NullPointerException is thrown when the class DefaultNodeExtension has been loaded by two different class loaders. #8755

  • HazelcastInstanceNotActive exception is thrown which is misleading. When a remote member is shutdown while destroying a proxy, no warnings should be logged. #8736

  • Missing dependency for JSPC plugin should be added. #297

3.7

This section lists the new features, enhancements, fixed issues and, removed or deprecated features for 3.7 release.

New Features

The following are the new features introduced with 3.7 release.

  • Custom Eviction Policies: You can implement and use your own eviction policy. Please refer to the Custom Eviction Policy section.

  • Discovery SPI Implementation for Microsoft Azure Services: Hazelcast members can be discovered within your Azure resource group. You can add this implementation as a plugin to your projects. Please refer to Hazelcast-Azure plugin page.

  • Hazelcast for OpenShift: Hazelcast members on OpenShift can discover each other. Please refer to Hazelcast OpenShift plugin page.

  • Apache Spark Connector: A plugin which allows Hazelcast maps and caches to be used as shared RDD caches by Spark using the Spark RDD API. Please refer to the plugin’s repo at Spark Connector for Hazelcast.

  • Alignment of WAN Replication Clusters: This feature provides a mechanism to align or realign distributed objects in the clusters connected through WAN. WAN replication mirrors changes from a map or cache in one cluster to another. It was maintaining the alignment but was not attaining. Now when the receiving cluster is empty and the sending cluster is not, objects are aligned. And if changes have been dropped for any reason, realignment is attained. Please refer to the Synchronizing WAN Target Cluster section.

  • Fault Tolerant Executor Service: This new data structure is introduced to guarantee that no execution tasks will be lost when a member goes down. Please refer to the Durable Executor Service section.

Enhancements

The following are the enhancements introduced with 3.7 release.

  • Modularized Hazelcast: As stated in the above note, 3.7 is Hazelcast’s first modularized release. Clients, languages and plugins are now modules which have separate GitHub repositories, Maven modules, and release cycles. Please refer to the Clients & Languages and Plugins pages.

  • Near Cache (JCache) Notification Enhancements: You can disable the near cache notifications for each entry and enable/disable notifications of full-flush events. Please refer to the ICache Configuration section and see definition of the new configuration element disable-per-entry-invalidation-events.

  • Migration Algorithm Enhancements: With these improvements the possibility of a data loss due to a member crash while the partitions are being migrated is eliminated.

  • WAN Replication Integrated with Solace: This integration is achieved through a new JMS endpoint. Please refer to the Solace Integration section.

  • Cloud Discovery SPI Plugin for Multicast: You can use multicast discovery for your Hazelcast platform with client/server topology. Only Hazelcast Java client is supported for this release. Please refer to the Discovering Native Clients section.

  • IMap Eviction Sync with JCache: Hazelcast Map uses now Hazelcast JCache’s eviction algorithm when evicting map entries.

  • Docker Image Enhancements: Hazelcast Docker image is able to integrate with the Service Discovery products such as Zookeeper, Consul, Etcd, and Eureka. Please refer to its repo at Hazelcast Docker.

  • Phone Home Enhancements: Performed to collect new phone home data to learn more about the environment on which Hazelcast runs. Please refer to the Phone Home section.

  • IMap.putAll() Performance Enhancements: The performance of putAll operation is improved by reducing the number of remote operations and supporting batching.

  • Hazelcast Instance and JCache Integration Enhancements: A direct relation between a Hazelcast instance and JCache is established with this enhancement. You can retrieve and access caches via the method getCache(String name) over HazelcastInstance API. Please refer to the JCache - Hazelcast Instance Integration section.

  • Indexing with Predicates for Entry Processors: Entry Processor can use indexing when a predicate is used to filter entries that need to be processed. Please refer to the Using Indexes section.

  • Partition Grouping Enhancements: You can define partition groups, in the same way as the IP address network configuration with wildcard support. You can also configure zone-aware partition groups for your Hazelcast clusters on AWS. Please refer to the Partition Group Configuration section.

  • Prevention of Blocking Reads in Transactions: Now the read operations are blocked only during committing the transactions.

  • Jetty and Tomcat Based Web Session Replications: These features have been made open source. You can reach them at Tomcat Session Manager and Jetty Session Manager GitHub repos.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • Expanded async API support in IMap, IAtomicLong, and IAtomicReference; async methods now return ICompletable and enable a reactive programming style. #8580, #7967, #7960

  • Call reference IDs are included in the Lock operations issued from member to member, to prevent double locking when lock operation is retried. But, Lock at clients does not have these. #8359

  • ReadBackupData and MaxIdleSeconds related documentation should be more clarifying. #8354

  • Entry listener with predicates should be notified on value entry/exit from the predicate value space. #8340

  • Collection querying in Portables. This allows querying a collection like car.wheels[0].pressure or car.wheels[any].pressure in the Portable data format. It also allows using a ValueExtractor in the Portable data format. #8132

  • Performance improvements for the invocation system. #8009

  • The performance log should have an option to log to the `shared' logger instead of its own file. It would be useful for environments where Hazelcast cannot create or get a performance log file. #7973

  • The path for performance logs, which is currently fixed to the user’s working directory should be configurable. #7968

  • Hazelcast IAtomicLong data structure provides synchronous methods only; async versions already exist and are available to Hazelcast members via the AsyncAtomicLong interface. Lack of public a async API for IAtomicLong is impeding Hazelcast integrations. #7957

  • API documentation for QueueStore should be enhanced. #7775

  • It would be better to have a way where near cache notifications for each entry are disabled, but an ability to enable/disable notifications of full-flush events (clear, etc.). #7580

  • Hazelcast should support Transaction API of Spring. #7469,https://github.com/hazelcast/hazelcast/issues/611[#611]

  • For Hazelcast Topic, even the event service’s pool size is set to a number larger than one, all of the messages are consumed by only one Hazelcast event thread. The use case includes a single Hazelcast member, both producer and consumer being singletons within the member, and message rate of more than 1000 per second. #7443

  • Partition strategy should be able to be specified not only in the Hazelcast configuration, but also within the Spring context. In addition, an implementing instance should be specified besides the class which implements the partition strategy. #7363

  • Async put operations should be reflected at near cache as soon as the method future.get() returns. In the case of async put operations and LocalUpdatePolicy being CACHE at the client side, entries are put to the near cache asynchronously from another task when the response is received. But with this way, when future.get() returns, entry might not be inside the near cache (but it will be there eventually). #7155

  • For ICache.iterator(), fetchSize is not configurable. #7041

  • Unit tests should have a default timeout. #6978

  • Outgoing ports on Hazelcast clients should be configurable. #6845

  • The method IMap.set does not have a corresponding async version, unlike put and putAsync. The method putAsync is not entirely suitable as an async set, since put returns the previous value mapped to the key, and triggers EntryListeners which may not be desirable. IMap should expose a dedicated setAsync to fulfill the contract for set and have the means to do so asynchronously. #6726

  • Javadoc for EntryProcessor.java should be enhanced by adding notes related to its thread safety. #6593

  • Custom SPI services should be more Spring-friendly. #6567

  • The spring-aware should be enabled programmatically too. #6514

  • Hibernate 5 should be supported. #5633

  • The error This node is not requested endpoint is shown in Docker networking. #4537

  • It would be nice if the type parameters of Predicate were inherited by the IndexAwarePredicate. #1686

  • The class MigrationEndpoint should be a part of Hazelcast SPI package. #1427

  • When a task is submitted to all members, and an executeOnEntries is invoked in the call with a predicate that is based on an index, then the index is ignored and a full scan of the local members is performed. #1156

  • Inconsistency between the declarative and programmatic configuration of network elements should be solved. #945

Fixes

The following are the issues solved for Hazelcast 3.7 release.

  • Issue with continuous query natural filtering event types: When the property hazelcast.map.entry.filtering.natural.event.types is set to true, updating a map entry whose value did not match the predicate to a new value that matches the predicate should publish an event of type ADDED instead of UPDATED. #8648

  • The method ClientEngineImpl::getConnectedClientStats reuses the operation GetConnectedClientsOperation. This operation should not be reused for multiple invocations. #8628

  • There is a possible regression in PartitionAwareOperationFactory when missing code coverage of MultipleEntryWithPredicateOperation is tried to be increased. #8622

  • When a client is inside the cloud, DiscoveryAddressTranslator class does not look for a public/private address flag. Private address should be used when the client is inside the cloud. If not, then public address should be used. #8595

  • MigrationThread can leak after shutting down or even forcefully terminating Hazelcast. It retains the entire HazelcastInstance. #8560

  • QueueOperation keeps the state between executions. If there is a state, then it should always be set, if-null-then-set idiom should not be applied for operation state. #8546

  • When the connection is made, the connection type is not known yet. But the connection type is used in the metrics ID; this id is used as soon as the connection is made. So it defaults to NONE in the method getMetricsId. It would be better to add a probe connection type and remove the connection type from the method TcpIpConnection.getMetricsId. #8540

  • Commit failure on clients sets the transaction state to ROLLING_BACK, which makes the transaction non-rollbackable. This is not a problem for lock-based data structures (map, multimap, etc.) but it is problematic for collections. State of the transaction should be changed to COMMIT_FAILED rather than ROLLING_BACK upon a commit failure. #8483

  • MapAttributeConfig cannot be loaded in an OSGi environment. #8482

  • Using Hazelcast out of the box with no configuration creates a Peer-to-Peer cluster. As soon as the configuration file hazelcast.xml is created by the user, in which only the logging type is set, Hazelcast does not create a cluster but starts in standalone mode. #8481

  • NullPointerException at the method HazelcastTimeStamper.getNextTimeStamp. #8465

  • Portable EntryProcessor is not being called on portable objects. #8365

  • In Hazelcast Hibernate modules, there is no JCache dependencies in the classpath. When Hazelcast is upgraded to 3.7-SNAPSHOT, java.lang.NoClassDefFoundError: javax/cache/Cache error is thrown when mocking HazelcastInstance. #8352

  • The method QueryableEntry.serializationService throws NullPointerException when MapReduce is run with extractable entries. Mapping phase of the MapReduce for Portable data formats should be fixed. #8346

  • There is an error when configuring Replicated Map. The return of the method getConfig in ReplicatedMapMBean should be fixed. #8298

  • TargetDisconnectedException is thrown for the clients in 3.7-SNAPSHOT. #8261

  • TransactionalMap’s removeIfSame test fails. #8238

  • Distributed Executor Service does not take ExecutorConfig.isStatisticsEnabled into account. #8223

  • Using MapStoreConfig in a cluster raises the exception Transition not allowed from state NOT_LOADED to LOADED. #8196

  • The method ICache::destroy should remove the cache itself from the owner CacheManager because, otherwise, it causes memory leaks due to the cache proxies which are dead but deemed as working, in AbstractHazelcastCacheManager::caches. #8186

  • Partition promotion is skipped when a node is terminated during the commit. #8174

  • The tests check the messages of InvalidConfigurationExeption`s thrown by the `XmlConfigBuilder. But these messages are often extracted from `SAXParseException`s which are localized; Maven build is platform dependent and fails. #8169

  • The method IAtomicReference:alter does not persist the changes. When a reference is tried to be altered, no alteration happens. #8149

  • Cache should not expire entities when Duration value is 0. #8148

  • Deserialization of dynamic proxy instances ignores the configured class loader. #8033

  • The attribute binary is missing in the MultiMap configuration within Spring context. It does not exist in Hazelcast configuration schema either. #8000

  • If you setup an interceptor to change the data being inserted, the entry listeners still fire with the old value. #7991

  • Unlike the InvocationFuture at the server side, ClientInvocationFuture immediately propagates InterruptedException if the calling thread gets interrupted. This can be a problem when both caller and callee need to agree on whether the operation has executed or not. #7963

  • Hazelcast 3.2.6 uses too much CPU when it is idle. #7943

  • Old version of Portable object from a map cannot be read if new UTF_ARRAY type field is added. #7926

  • The method IMap.size() reports a stale result when blocked by the initialization of MapStore. #7905

  • Isolated thread pool for priority generic operations. #7857

  • There is an issue when detecting JCache in the classpath. The exception NoClassDefFound is thrown when upgrading to a newer Hazelcast version. #7810

  • Programmatic configuration of logging for Hazelcast client does not work. #7764

  • Better separators should be used in the exceptions for a clearer read between local and remote stacktraces. #7744

  • Under the section Operation Threading of Hazelcast Reference Manual, it states that the default number of partition-aware operation threads is (2 x number of cores). However, when looking at the code and observing the actual number of threads created runtime, it seems like the default value is instead 1 x number of cores instead. #7741

  • The method IMap.executeOnKeys() does not support the empty set (it throws a misleading NullPointerException), and is inconsistent with the method getAll(). #7631

  • Replicated map updates take a very long time. The problematic method is putAll(). The replication logic in this method checks whether the data owners are in sync with the replicas. If they are not, this logic syncs them every 30 seconds. This means, when the updates are not replicated to callers, it takes up to 30 seconds to make all the members synchronized. This period should be configurable. #7617

  • ScheduledExecutorServiceDelegate violates contract of ScheduledExecutorService. It wraps tasks in ScheduledTaskRunner which delegates to a different executor. As a consequence, a task can be executed concurrently and this is a violation of a contract of ScheduledExecutorService. #7611

  • If javax.cache.CacheManager is created with the default settings, the underlying HazelcastInstance is not shutdown when the method close is called on the CacheManager. #7606

  • The method containsKey() of TransactionalMap is blocked when the key was previously locked by the method getForUpdate(). #7588

  • There is an inconsistent behavior when removing from TransactionalMap while the key is locked on IMap. In order to avoid trying to remove an entry that may have already been removed in another uncommitted transaction, IMap.tryLock is used before performing TransactionalMap.remove. This works as expected if the operations occur on a member. But a TransactionException is thrown when it occurs on a client when using XA Transaction. #7587

  • Hazelcast instance should be exposed through com.hazelcast.spring.cache.HazelcastCacheManager. #7571

  • Instance name should not be overridden while creating cache manager from the specified configuration file. Also, it would be better to specify instance name via the HazelcastCachingProvider.HAZELCAST_INSTANCE_NAME property when instance configuration is taken from the specified configuration file via HazelcastCachingProvider.HAZELCAST_CONFIG_LOCATION. #7567

  • The addInterceptor() method in com.hazelcast.map.impl.MapContainer() is not thread safe. For example, if two concurrent attempts are made to inject the same interceptor, these will be different interceptor objects with the same ID. In this case, the call to interceptorMap.put(id, interceptor) will increase the map size by one, but the call to interceptors.add(interceptor) will increase the list size by two. #7520

  • All JMX Beans disappear when the single Hazelcast instance is shut down in the same JVM. #7467

  • There are unused elements for Management Center configuration: cluster-id and security-token. #7446

  • For clients, InitialMembershipListener.init is called after MembershipListener.memberAdded. This contradicts the content in the Reference Manual. #7430

  • DiscoveryService’s start and destroy methods should be called during the start and shutdown of client when Discovery SPI is enabled. #7347

  • Return cache config as response even though found and created cache config could not put into cache configs inside cache service. #7208

  • In Hazelcast Management Center shutting down a node seems to prevent a node from restarting. #7101

  • MapStoreConfig does not override hashCode and equals methods. Implementation for these two methods should be added. #7035

  • Data is lost when the member dies during repartitioning. #6628

  • Some of the map statistics, such as cost and last access time, are calculated by the traversing map entries. Therefore the calculation time exceeds the time interval reserved for management center state sending thread when entry count is too high. #6442, #5905

  • Eviction with eviction-percentage does not work. #6432

  • InvocationFuture’s asynchronous calls do not detect the lost operations. #6250

  • The invocation mechanism for blocking operations relies on a periodic timeout so that the operation gets retried. To prevent the calling thread (the thread for future.get) from waiting indefinitely, it will periodically ask the isstillrunning service if the operation is lost. 6248

  • Under some circumstances Hazelcast is getting a corrupt value for IAtomicLongs when a member leaves the cluster. #6074

  • When the client disconnects normally, the server logs an info and a warning message containing the text java.io.EOFException. #6035

  • Session ID management error (500 HTTP error) is shown by Hazelcast 3.5 filter based replication. #5660

  • Some operating systems (such as HPUX or Solaris) and hardware platforms have constraints about the aligned memory operations. In these architectures memory operations must be byte-by-byte as implemented in DirectByteBuffer. #5532

  • Data is lost when a member crashes or is killed during the repartitioning. #5444

  • Data is lost when a member is terminated. Related scenario is as followshttps://github.com/hazelcast/hazelcast/issues/5388[#5388]:

    1. Start the first member, and let it populate a map with 100k entries.

    2. Start the second member, and let it start joining the cluster.

    3. Terminate the second member during the join operation.

    4. Observe that data is lost from the first member.

  • As for now it is very complicated to listen a getAsync or putAsync result and to integrate it with completable futures or listenable futures. An ICompletableFuture should be returned since it is an interface which seems to extend JDK futures and is returned by an IMap. #5315

  • If multiple Hazelcast members attempt to remove values from a key of a multimap concurrently, and then the members are shut down, the multimap can remain in an inconsistent state with entries remaining after all have been removed. #5220

  • ClassNotFoundException is thrown when trying to get an entry from a TransactionalMap. #4969

  • Profiling a Hazelcast application reveals a thread contention in SpringManagedContext on java.lang.Class.getAnnotation(). And this calls a synchronized method called initAnnotationsIfNecessary(). #4506

  • Hazelcast IMap statistics show negative values. After heavy usage of the cache, the number of misses starts showing up negative. #4022

  • WebFilter may prevent requests from accessing request input stream. #3829

  • WrongTargetException for PollOperation in Queues. #3754

  • Hazelcast member continuously logs WrongTargetException. #3395

  • When there is a map with write-behind mode and a map store is configured (eviction is not needed); when the method flush is called in the IMap, the map store’s store method can be called concurrently for the same key, namely for those keys which are in the write-behind queue and then forcibly stored by the flush. This is because the flush operation storing all entries in the write-behind queue seems to be executed in the operation thread, while the periodic processing of the write-behind queue is done by an executor service defined in the WriteBehindQueueManager. #3338

Removed/Deprecated Features

  • WanNoDelayReplication implementation of Hazelcast’s WAN Replication has been removed. You can still achieve this behavior by setting the batch size to 1 while configuring the WanBatchReplication. Please refer to the Defining WAN Replication section for more information.

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.6.8

The following is the fixed issue for 3.6.8 release.

  • Servers should throw exception when message type is not known. Client protocol is defined to throw an exception when a client sends a client message with unknown type to server. #10196, #10202

3.6.7

The following is the fixed issue for 3.6.7 release.

  • When loading keys supplied by the user, keyLoaderState should not be changed. In some cases, MapLoader.loadAll(true) does not reload after reloading with new keys. #9279

3.6.6

The following are the fixed issues for 3.6.6 release.

  • NullPointerException is thrown while deleting an entry through Entry Processor by setting value to NULL. #9141

  • Indefinite reads of stale values from near cache should be prevented. #9075

  • Write behind MapStore is not retrying to store an entry indefinitely. When an entry cannot be persisted, the retry mechanism stops working after a while. This happens when the distributed map is configured to use the write-behind mechanism. #8918

3.6.5

The following are the fixed issues for 3.6.5 release.

  • When a client is inside the cloud, DiscoveryAddressTranslator class does not look for a public/private address flag. Private address should be used when the client is inside the cloud. If not, then public address should be used. #8595

  • The method ClusterMergeTask::waitOnFutureInterruptible has an almost endless loop. #8527

  • Cluster state should not be allowed to change before startup is completed. Normally, the method NodeExtension.isStartCompleted() returns true when a member is joined successfully. However, when Hot Restart Store is enabled, it returns false until Hot Restart process completes on the whole cluster. #8523

  • MapPartitionLostListener is not working. Upon migration commit, the flag SYNC_WAITING_REPLICA should not be overwritten. #8505

  • PartitonServiceBeanDTO always tries to resolve the domain name. Instead of calling the method getSocketAddress() every time, it can use the method getLocalAddress(). #8463

  • Hazelcast Demo application depends on JDK. JRE should be enough on all platforms. #8445

  • Excessive logging of is-executing': true checks against long running operations. #8386

  • There is a deadlock in MigrationQueue. The internal queue and counter are not synced in some situations and this causes deadlock. Usually it happens when there is a migration task failure. #8217

  • The method ClientSmartInvocationServiceImpl#invokeOnTarget does not throw TargetNotMemberException. #7997

  • Access data is not updated by local reads when reading from backup is enabled. #7354

  • Missing dependency for JSPC plugin should be added. #297

3.6.4

The following are the fixed issues for 3.6.4 release.

  • OperationFactoryWrapper should use callersUUID instead of overriding on the member side. #8437

  • HazelcastInstanceLoader ignores instance-name for clients. #8413

  • Usage of GroupProperties in Hibernate modules should be removed. #8398

  • Thread interrupt flag check must be added to StoreWorker. #8345

  • Partition proxies are created eventually on all members. However, this is an async process driven by the eventing system. If a member does not have a proxy in its registry, then it will not fire MapPartitionLostListener. Before firing a partition lost event, proxies should be waited to be created. #8293

  • Ensure skipping previously removed CacheEntryListenerConfiguration entries while iterating over added CacheEntryListenerConfiguration s in the CacheConfig to register listeners. #8283

  • Hazelcast instance is started without problems but Management Center does not launch and throws an exception when bundled Jetty is used with Java versions 1.8.0_91 and higher. #8248

3.6.3

The following are the fixed issues for 3.6.3 release.

  • MapStatisticsAwareService should obtain the map names from proxy registry. #8209

  • Cache should not expire the entities when Duration value is set to 0. #8148, #8206

  • Collection querying in Portables. #8172

  • Type extraction by the method extractedMultiResult should be fixed when the extraction result is null. #8134

  • Expiration time for the map entries should be calculated based on their latest update times. #8111, #8113

  • Support for the usage of IAM role’s defaults should be added. #8100

  • Binary property for MultiMap should be added to XmlConfigBuilder. #8094

  • EC2 auto-discovery in China throws an exception due to a bug in host header handling mechanism. #8073

  • The method Cluster.shutdown() ignores lifecycle listeners. #8070

  • Resource adapters should not cast to implementation when using a container to get a connection. #8019

  • Unnecessary deserialization step during the execution of put operations should be removed. #8018

  • A URL text with spaces should not be transformed into a URI. #8016

  • Unnecessary deserialization step in the replicated map data structure should be removed. #8014

  • When using InstanceOfPredicate, the method toObject at the server side causes NullPointerException, since the class loader is not set and it is null. Default class loader should be used if it is not set by the configuration to avoid the exception. #7977

  • There may be cases when the server may return a list of entries larger than the requested page size. In this case the client should not put any anchor into the list that is on a page greater than the requested page. #7976

  • When a remote invocation waits indefinitely on the internal executor, a deadlock may occur since its response would use the same executor and since the invocation monitor runs on the same executor. #7944

  • Already published messages should be retrieved after StaleSequenceException occurs when ReliableMessageListener is registered at the client side. #7928

  • The case of response being equal to null in the class ClientDelegatingFuture should be handled. #7901

  • Detection of JCache should be improved. Currently Hazelcast checks if the class javax.cache.Caching is available and, if so, Hazelcast creates the JCache service and configures the client protocol messages. However, some older snapshots of JCache JARs have the Caching class available, but other classes are missing. This causes Class not found exceptions when members are started. #7899

  • When connection gets an exception from the socket and the method destroyConnection is called before connection is authenticated, the client does not have an endpoint and it is not in the map of connections. Hence, the connection cannot be closed. #7866

  • Upon subsequent destroy and create operations of IMap, there can be more than one map container referenced by different record stores at the same time. Therefore, indexes can be created in an unexpected map container and this can lead to return less than expected number of results when IMap is queried. #7838

  • After the client is shut down, there should be no invocations left. The method assert() of ClientInvocationServiceSupport fails. #7836

The following are the enhancements performed for 3.6.3 release.

  • Phone home should include environment information, such as the operating system name, version and kernel architecture. #7970

  • Exceptions, which are thrown when the method isMemberSafe() is called, should go into finest level. When there are topology changes some exceptions are expected. The method getFutureResult() should log exceptions as finest(), not warn(). #7904

  • Protocol version should be 1 instead of 0. It was set as 0 for the response messages. #7900

3.6.2

The following are the fixed issues for 3.6.2 release.

  • Upon subsequent destroy and creation of IMap, there is a possibility that there can be more than one map-containers referenced by different record-stores at the same time. Hence, indexes can be created in an unexpected map-container and this can lead to return less than expected number of results when IMap is queried. #7874

  • Couple of issues related to client should be fixed: (a) There should be no invocations left after client is shut down. (b) When the client-member connection gets an exception from the socket and the method destroyConnection is called before the connection is authenticated, the client does not have an endpoint and it is not in the connections map anymore. Connection cannot be closed because of this. (c) Authentication future waits infinitely. Because of (b), heartbeat does not work on that connection yet. #7867

  • In Hazelcast 3.6.1, the OSGi bundle install fails. #7772

  • Timeout happens at the ClientMapBasicTest. #7718

  • IMap does not send invalidations to the client’s near caches after putAll/loadAll operations at member side. #7715

  • The test ClientXAStressTest.testCommitConcurrently fails due to an assertion error that reads as the size of the map is not correct. #7709

  • The test ClientTransactionalMapQuorumTest fails since it timed out. #7693

  • Wildcards do not work with permissions. For example, when a queue permission is defined using wildcard, i.e., <queue-permission name="secure.*" principal="admin"> and the queue secure.orders is created at the client, Hazelcast throws AccessControlException. #7616

  • The FREE_HEAP_PERCENTAGE eviction policy does not work as documented because the EvictionChecker is incorrectly calculating available memory percentage in checkFreeHeapPercentageEviction. #7599

  • DiscoveryStrategy’s destroy() method is not called on shutdown. #7545

  • The method MapStore.storeAll() is never called when the objects are updated with a delay. #7464

  • WebFilter(3.6-RC1): Session attributes are lost or overwritten with old values when using deferred-write by reading them. #7291

  • The method HazelcastHttpSession.getAttributeNames() returns entries that were only read but not written. #7033

  • Hazelcast 3.5.3 conflicts with FUSE 6.1. When Hazelcast is integrated into a system with JBoss Fuse 6.1, there appeared some warnings in the logs. #6821

  • An exception is thrown on the server when attempting to deserialize HIBERNATE4_TYPE_HIBERNATE_CACHE_ENTRY value. #6683

  • The test IOBalancerMemoryLeakTest.testMemoryLeak fails due to a socket exception that reads as unexpected end of file from server. #6496

The following are the enhancements performed for 3.6.2 release.

  • Hazelcast Docker images are big in size; it would be better to clean them up and reduce their sizes. #7553

  • It would be better to provide an enterprise-javadoc.jar in the Hazelcast Enterprise bundle. This way IDEs can auto-import the Javadocs for the Enterprise APIs and provide context sensitive completion/help within the IDE. #7245

  • Management Center WAN Replication metrics take a long time to stabilize. The responsiveness of the Outbound records/sec. metric should be improved. It currently seems to under-report WAN replication performance for 10s of seconds before reaching the same figure as the client side reported metrics.https://github.com/hazelcast/hazelcast/issues/7244[#7244]

3.6.1

The following are the fixed issues for 3.6.1 release.

  • Hazelcast 3.6 fails to create ObjectName for an instance created with JCache API. #7548

  • The test com.hazelcast.client.spi.impl.ClientInvocationTest.executionCallback_FailOnShutdown fails. #7462

  • There are failures in SystemClockChangeTest. #7444

  • When you use EntryProcessor with a map interceptor, it does not get the correct value. #7414

  • Backup records also increase the owned cache entry count but they should be ignored. Cache statistics show incorrect entry count via the method getOwnedEntryCount. #7403

  • When getting values from PredicateSupplier, the method DistinctValueAggregation.DistinctValueMapper.map() throws a ClassCastException. #7398

  • Timeout is not enough for the test TransferStressTest.testLargePackets. #7393

  • There are failures in the test ClientExceptionFactoryTest.testException. #7360

  • The method getReplicationImplObject in WanTargetClusterConfig is not used. #7353

  • Entry processor and map put/remove tests in WAN replication module fail. #7352

  • Hazelcast namespace for Spring configuration does not support Discovery SPI. #6913

  • When Hazelcast Spring configuration is used for the client and if a serialization configuration is present in the Spring client, then the client hangs forever during the authentication method. #5815

3.6

This section lists the new features, enhancements and fixed issues for 3.6 release.

New Features

The following are the new features introduced with 3.6 release.

  • High-Density Memory Store for Hazelcast Map: With this release, Hazelcast Map data structure is now equipped with the High-Density Memory Store, previously implemented for Hazelcast JCache. Please refer to the Setting In Memory Format section.

  • Discovery Service Provider Interface (Discovery SPI): You can use this SPI to discover Hazelcast instances on cloud environments provided by jclouds®, Kubernetes and many more. The existing discovery mechanisms that Hazelcast provides (Multicast, TCP/IP and Amazon EC2) have been re-implemented on top of this new Discovery SPI. See the Discovery SPI section.

  • Client Protocol: This feature presents the Hazelcast’s new open binary client protocol. See the Open Binary Client Protocol Documentation.

  • Client Cross Version Compatibility: Now you can upgrade your Hazelcast clients independently from servers and other clients. Please refer to Open Binary Client Protocol Documentation.

  • Support for cloud providers through jclouds®: Hazelcast now supports deployments on all the well-known cloud providers through the jclouds® open source library. See the Discovering Members with jclouds section.

  • Hot Restart Persistence: This new feature provides fast restarting of the Hazelcast clusters. This is achieved by storing the state of the cluster members to the disk. See the Hot Restart Persistence section for more details.

  • Ringbuffer and Reliable Topic in Hazelcast Clients: The data structures Ringbuffer and Reliable Topic recently introduced by Hazelcast (with the release 3.5) are now implemented for Hazelcast Java Client. Ringbuffer has also been implemented for .NET Client.

  • Cluster Quorum for Hazelcast JCache: Cluster Quorum checks are now provided for Hazelcast JCache implementations, too. See the Defining a Cluster Quorum section to refresh and the ICache Configuration section to learn configuring it for JCache.

  • Split Brain Syndrome handler for Hazelcast JCache: Now Split Brain Syndrome is handled in JCache as it is taken care in Hazelcast Map. See the JCache Split-Brain section.

  • Partition Lost Listener for Hazelcast JCache: You can listen to partition lost events fired in your Hazelcast JCache implementation. See the ICache Configuration section.

  • Hazelcast Docker image: Now you can run Hazelcast using our image in the Docker platform. See the Deploying using Docker section.

  • Lite Members: With the re-introduction of Hazelcast Lite Members (it was removed starting with Hazelcast 3.0 release), you are able to specify certain members in your cluster so that they do not store data. You can use these lite members mostly for your task executions and listener registrations. See the Enabling Lite Members section.

  • Querying in collections and arrays: Hazelcast is now able to query and index attributes of objects stored in a collection or array. Please refer to the Querying in collections section.

  • Custom attributes extraction: It is now possible to extract a value of an object’s attribute using a custom extractor class. See the the Custom attributes section.

  • Acquiring locks with a lease time: Now, you can try to acquire locks with a lease time. See the the comment for the method tryLock() in ILock code.

  • Monitoring the WAN replication: You can now monitor the state of your WAN replications using the Hazelcast Management Center. Please refer to the Monitoring WAN Replication section.

Enhancements

The following are the enhancements introduced with 3.6 release.

  • Replicated Map improvements: The implementation of Hazelcast replicated maps has been revisited. Please especially refer to the Considerations for Replicated Map section.

  • Management Center improvements: Alerting mechanism added. Please refer to the Management Center section.

  • Paging Predicate improvements: With the performed improvements, now random page accessing is supported. See the Filtering with Paging Predicates section.

  • Rule based query optimizations: This improvement introduces a query optimizer based on static rewriting rules. The optimizer treats predicates as immutable and returns a modified copy when the optimized one is found. See the hazelcast.query.optimizer.type property definition in the System Properties section.

  • WAN replication improvements: With the improvements performed on Hazelcast’s WAN replication feature, you can now monitor WAN replication events for each data structure and WAN replication now supports different acknowledge types for each target cluster group. Please refer to the WAN Replication Event Filtering API section and WAN Replication Acknowledge Types section for more information.

  • Improvements on Hazelcast’s OSGI support: With this improvement, Hazelcast bundles provide OSGI services so that the users can manage (create, access, shutdown) the Hazelcast instances through this service on OSGI environments. Having the hazelcast.osgi.start property enabled, when an Hazelcast OSGI service is activated, a default Hazelcast instance is created automatically. These instances can be served as an OSGI service to be accessed by other bundles. Registering the created Hazelcast instances behavior is enabled by default and can be disabled using the hazelcast.osgi.register.disabled property. Each Hazelcast bundle provides a different OSGI service and their instances can be grouped (clustered) together to prevent possible compatibility issues between different Hazelcast versions/bundles. This grouping behavior is enabled by default and can be disabled using the hazelcast.osgi.grouping.disabled property. Hazelcast OSGI service’s lifecycle (and also the owned/created instances’ lifecycles) are the same as the owner Hazelcast bundles. When the bundle is stopped (deactivated), owned service and Hazelcast instances are also deactivated/shutdown and deregistered automatically. Then, when the bundle is re-activated, its service is registered again. In addition, the Hazelcast Enterprise JAR file is also an OSGI bundle like the Hazelcast OSS JAR file.

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • On WAN Replication Queue overrun, a WANReplicationQueueFullException is expected, but instead, the client receives an UndefinedErrorCodeException. #7241

  • Approximate max-size calculation should be removed for IMap eviction. #6463

  • SpringAwareWebFilter should have a constructor which takes properties as arguments. #6438

  • Client side and server side cache proxies handle putAll operation one by one. This is not efficient. Records for this operation should be grouped as per their partitions and should be sent and processed in batches. #6367

  • Not requested events should not be sent to MapListener #6349

  • Inconsistent and potentially buggy design in BasicCompletableFuture. #6080

  • Starting with hazelcast-wm 3.3, OSGI Manifest Spring package imports should be optional. #6072

  • The new client determines the partition ID for every invocation for data structures like queue and list where the partition ID is static. There is no need for this behavior. It should calculate the partition ID for once when the proxy is created and continue to re-use it. #5848

  • Map.Entry supplied to Entry Processor is not Serializable any more. #5611

  • The configuration file minimal-json with the provided scope is not picked up by the shade plugin. #5543

  • In Spring configuration, when a boolean property is injected for hazelcast bean (<hz:hazelcast:…​.</hz:hazelcast) a SAXParse exception is thrown. #5528

  • Currently, key/value pairs are deserialized prior to the execution of entry processor by default. This leads to the need of domain object at the server side, even if entry processor never uses it. #5301

  • In Spring XML configuration, the attributes of socket-options should be of type xs:string. #4700

  • ClientMembershipEvent does not need to have the member field. #4282

  • Hazelcast has lock with lease time feature but does not support tryLock with lease time. #1564

Fixes

The following are the fixed issues solved for 3.6 release.

3.6 Fixes

  • Lock requests hang forever after a split-brain recovery: When a member hangs a few minutes, the cluster assumes that this frozen member as dead and the master kicks it. When the frozen member comes back, after some time they figure out it was a split-merge back activity. Problem is, some of the lock requests hang forever without a timeout after the merge. That means some of the previously acquired locks cannot be released and hang in the cluster infinitely; no one can acquire them and there is no one to release them anymore. #8647

  • In the manifest file, org.jclouds.* should be marked as optional dependencies. #7318

  • Tests are needed for WanReplicationPublisherDelegate, WanReplicationEvent, MapReplicationUpdate and AbstractMultipleEntryBackupOperation in the Open Source WAN API. #7315

  • Invocation of quorum listener requires at least an attempt to perform a map operation. But it should not require this; just the crash of nodes should be enough to use the quorum mechanism. #7300

  • Owned entry count to be used as the expected near cache hit count should be calculated by checking the partition ownership in the NearCacheTest::testGetAll. #7285

  • The parameter minEvictionCheckMillis controls the maximum frequency of evictions. It is 100ms by default. It means at most 1 eviction is executed in a 100ms interval. No other put() operation within this interval triggers an eviction. So, if the put rate is greater than 1 per 100ms, then the number of entries is growing regardless of the max-size-policy. This eventually triggers a forced eviction which will prevent OutOfMemoryException. Forced evictions are only hiding this issue. Another possible solution is to keep the default interval as it is and apply batching: When X eviction cycles are skipped due the minEvictionCheckMillis parameter, then during the next cycle X + 1 entries should be evicted instead of just 1. #7268

  • Descriptions of some maximum size policies defined in the com.hazelcast.config.EvictionConfig.MaxSizePolicy and com.hazelcast.config.MaxSizeConfig.MaxSizePolicy are not clear and confusing. They should be clarified. #7267

  • Tests under TopicOverloadDistributedTest are spuriously failing even on the local machine. They need to be reviewed. #7266

3.6-RC1 Fixes

This section lists the enhancements and fixed issues for 3.6-RC1 (Release Candidate 1) release.

  • Javadoc for IMap.putAll() does not mention the lack of atomicity in the invocation. #7256

  • When a WAN Queue overrun occurs (with exception enabled), the source cluster logs an excessive amount of noise. This should to be logged. #7242

  • When using Hazelcast as a JCache provider: As JSR-107 Javadoc states, an update should not reset expiry time for CreatedExpiryPolicy. However, when a cache entry is updated, it does not expire. #7236

  • Default WAN acknowledge type should be ACK_ON_RECEIPT. #7160

  • NullPointerException is thrown in ClientRegressionWithMockNetworkTest. #7148

  • Changing clusters in the Management Center does not update/refresh the cluster members in the Scripting tab. #7119

  • A fix is needed for operation retries in PartitionCheckIfLoadedOperation. #7114

  • WAN Queue counts in the Management Center for Hazelcast 3.6-EA3 are not correct. #7100

  • Hazelcast 3.6 Reference Manual is not correct for its Enterprise WAN Replication content. #7099

3.6-EA3 Fixes

This section lists the enhancements and fixed issues for 3.6-EA3 (Early Access 3) release.

  • NullPointerException is thrown for the thread cached4 in a test which uses MapLoader. #7098

  • The method loadInternal of MapProxySupport requires dataKeys. Hence, a serialization step should be added to MapProxy.loadAll(). #7090

  • Near cache heap cost calculation is not proper when the cache gets concurrent misses. #7057

  • IQueue accepts null values from the Hazelcast Java client. #7048

  • WriteBehindMapStore for a map that has OBJECT as the in-memory format causes the entry processors to serialize the objects. #7040

  • Latest code does not include the file com.hazelcast.client.impl.protocol.codec.CacheContainsKeyCodec and build fails. #7019

  • Two members of a cluster become masters and ignore each other. #7016

  • AbstractCacheRecordStore should update the field isOwner while it is being cleared after migration. #6983

  • There are memory leaks in the local map statistics and near cache invalidation queues. The map containers also leak memory caused either by the near cache invalidation mechanism (when re-creating objects to check whether the near cache is enabled) or MapPartitionDestroyOperation (when re-creating objects and trying to the backup count). #6972

  • When the lite-member flag is used within the Spring context, its enabled attribute does not work properly. #6945

  • LoadAllTask for the client and server side cache proxies should also handle the Throwable, not just the Exception. #6944

  • The enable attribute of the partition-group element in the Hazelcast-Spring.xsd scheme should have the type string, not boolean. #6927

  • There is a left-over method in the Discovery SPI configuration, namely addDiscoveryProviderConfig. #6911

  • InMemoryFormat.OBJECT does not work with the max-size policies USED_HEAP_SIZE and USED_HEAP_PERCENTAGE. #6875

  • PublicAddressTest has been ignored due to the running time. #6858

  • NullPointerException is thrown in ClientExecutionPoolSizeLowTest. #6853

3.6-EA2 Fixes

This section lists the enhancements and fixed issues for 3.6-EA2 (Early Access 2) release.

  • MapLoader may insert null values into IMap causing memory leak. #6830

  • When replicated map entries are migrated to a new destination; TTL eviction should be scheduled, eviction should be retried when a failure caused by the migration happens and the sync interval should be increased. #6799

  • There is a logical error in the method Ringbuffer.readManyAsync() when minSize = 0. In this case, the Ringbuffer is not read and nothing is returned. #6787

  • When a listener’s registration is made from the listener configuration, an error occurs during the listener initialization. #6784

  • Remaining cache invalidation messages should be flushed on the ICacheService while the member is in the SHUTTING_DOWN state. #6778

  • When a client cannot send a request to one of the connections, TargetNotMemberException is thrown. This name is confusing the Hazelcast users. #6766

  • ClassCastException is thrown when using Timestamp within DataSerializable. #6759

  • The method destroyDistributedObject() of ReplicatedMapService iterates over partition containers and record stores and destroys them. While destroying, record store calls destroyDistributedObject() which leads to an infinite loop. #6754

  • Hazelcast does not inject its instance into HazelcastInstanceAware registered via classname. #6697

  • There is a sporadic startup failure in 3.6-EA. #6684

  • There is no need to use CacheLoader inside the client/server side cache proxies. #6676

  • Fixed wrong calculation of eviction removal size when PER_NODE max-size policy is used. #6671

  • If the cluster state is not active RepartitioningTask should not be triggered. Otherwise, it causes infinite retries and prevents the member from shutdown. #6663

  • There are broken XML configuration tests in the Hazelcast client package. #6633

  • There is a memory leak since the method publishBathcedEvents does not remove the events from batchEvent. #6618

  • Custom credentials class is not de-serialized on the server side. #6615

  • Lite member element should be added to the Hazelcast Spring configuration. #6605

  • EntryListener shows the unprocessed value in combination with PostProcessingMapStore. #6588

  • Clients cannot submit HazelcastInstanceAware callables. #6570

3.6-EA Fixes

The following are the issues solved for Hazelcast 3.6-EA (Early Access) release.

  • The method map.size() waits indefinitely after the shutdown of a node. #6538

  • HazelcastCachingProvider does not use the specified instance (by the object) when instance-name is not specified. #6454

  • onExecutionFailure should be called before returning from run, if backup is not valid. #6420

  • OperationThread.priorityPendingCount() should return scheduleQueue.prioritySize() instead of scheduleQueue.normalSize(). #6318

  • There is a growth in heap usage caused by a memory leak in the following scenario: A node in the cluster regularly creates maps and puts entries into it, again in regular intervals. Another node removes the entries minutes after they were put, and if the map is empty, it destroys the map. #6317

  • Currently, there is an EntryEvictedListener that is notified both for expiration and eviction events. There should be a separate listener for expired entries: eviction happens due to size constraints, and expiry is once the entry has expired. #6311

  • `InvocationFuture`s async calls do not detect the lost operations. #6250

  • When the method setBooleanAttribute of the class Member is run, Null Pointer Exception is occurred on STDOUT. The problem is in the method sendMemberAttributeEvent of the class ClusterServiceImpl. #6223

  • IOBalancer keeps references of all the socket reader/writers but when destroying the connection, they release the references for only the ones which has endpoints. This causes a memory leak. #6199

  • ILIKE and Regex examples should be added to the Reference Manual under the Supported SQL Syntax section. #6190

  • GroupProperty defaulting does not work properly when programmatic configuration is used. #6174

  • When integrating Hazelcast in Spring Boot: if HazelcastInstance is created using the default newHazelcastInstance static method, then an HazelcastInstance whose Config has a valid configurationUrl property is created. However, XmlBuilder does not set this URL in the configuration it parses. #6061

  • Hazelcast’s latest snapshot run fails due to the introduction of ClientExceptionFactory which has been developed for exception processing and working well in that sense. #6010

  • The class HazelcastXATest has only fast and slow modes (nothing in between) and possibly due to this, sometimes a transaction is waiting for a timeout. Either the transaction recovery or the test class itself is racy. #5923

  • A memory leak occurs when a listener is added and removed from client. A remove runnable in the collection that is stored in ClientEndpointImpl is the leftover. This runnable collection is used to cleanup the listeners when client is disconnected, it should be removed too after the listener is removed. #5893

  • The class CacheRemoveAllOperation does not send the completed event in some cases, e.g. if CacheRecordStore for that partition is not created yet or if the filtered keys are empty. #5865

  • In the class MapProxyImpl, the methods executeOnKey and submitToKey create an EntryOperation with the thread ID set. This does not happen with the class ClientMapProxy. Therefore, the class MapExecuteOnKeyRequest should take a thread ID and set this on the generated EntryOperation. #5857

  • The method IndexImpl.getRecords() fails with Null Pointer Exception due to the inconsistency between the not(…​equals()) and notEquals(). #5807

  • The method HazelcastHttpSession.getAttribute() for WebFilter does not work when deferredWrite is set to true. #5798

  • When hazelcast.nio.faststring is enabled, UTFEncoderDecoder tries to create a FastStringCreator. However, if the reflection is not available due to the security manager, buildFastStringCreator returns null and consequently StringCreator becomes null. #5777

  • hazelcast-jca-rar/pom.xml references to src/main/rar/ra.xml which does not exist. #5760

  • The Maven profile mvn clean compile -Pqa does not exist but it is documented in the README of Hazelcast. #5746

  • PerformanceLogFile only compiles if JDK 1.7 or above is used. #5729

  • Currently, for every deserialization a BufferObjectDataInput is created. This generates waste since it is created with an array of data for every deserialization. The BufferObjectDataOutput is already cached; the input should use a similar approach. #5562

  • When any entities are defined as read only in the Hibernate L2 cache, an invalidation of the cache (such as caused by executing a native SQLQuery) leads to the error UnsupportedOperationException. #5561

  • The performance impacts of TWO_PHASE and LOCAL transaction types should be documented. #5075

  • Client requests are very inefficient when determining the partition ID. #4940

  • The method keySet() relies on QueryOperation. The QueryOperation does not accept IterationType - it always returns both keys and values. This can lead to unnecessary load and potentially even an OOM exception. #4642

  • Hazelcast is stuck in TIMED_WAITING when used as 2nd level cache for Hibernate. #4406

  • Management Center license loading problem when REST API is used. #189

  • Executor monitoring in Management Center does not show the cancelled operations" #177

  • When an alert for a data structure (map, queue, etc.) with its specific name is created, a NullPointerException is thrown after the cluster is reset. #175

  • Default directory name is hardcoded as mancenter3.5 and it needs to be maintained for every major release. This process should be dynamic. #174

  • Throughput statistics for Map shows nothing when the putAll() method is used. #159

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.5.5

The following are the issues solved for Hazelcast 3.5.5 release.

  • When hazelcast.jmx option is enabled, MBeans are created for every Hazelcast object but they are never removed. When destroying Hazelcast object with the method destroy(), DistributedObjectEvent::getDistributedObject() throws DistributedObjectDestroyedException and stops the process of unregistering MBeans. MBeans are left forever causing memory leaks and they can be seen in VisualVM. #7329

  • IdGenerator sometimes generates duplicate IDs if it is put under stress. #7299

  • The method IAtomicLong.compareAndSet() does not properly backup its updated state. In a two node cluster, if you use this method and then shutdown the owner of the IAtomicLong, the remaining node no longer sees the updated value. #7290

  • InvocationMonitor checks for the same invocations continuously. #7170

  • The methods IMap.getAsync and IMap.putAsync should update the statistics getCount and putCount, etc. #7109

  • NullPointerException is thrown for CoalescedWriteBehindQueue.removeFirstOccurrence(). #7082

  • The quorum definition in the Spring context is not correct. #6946

  • While publishing events, IMap operations convert the value to data even if the registered listener does not request the value. #6866

  • Map entry event listeners are not invoked on the clients of WAN replication target cluster. #6802

  • The method putAsync does not affect LocalMapStats.getPutOperationCount(). #6731

  • Possible memory leak when using IMap.containsKey. Problem happens when containsKey is used for a value that exists in the MapStore. Entry is loaded from the store but it does not get added to the map (no event is triggered in this case), eventually the system crashes with no memory. #6517

  • Performance test with Spring Batch throws TargetDisconnectedException. #4230

  • Web session replication does not work as expected during a shutdown. #3362

  • The parameter session-ttl-seconds is set after the instance creation. It should be set before the instance is created. #2377

3.5.4

The following are the issues solved for Hazelcast 3.5.4 release.

  • Fixed the wrong calculation of eviction removal size when PER_NODE max-size policy is used. #6674

  • Lazy deserialization is required while events are being processed. #6582

  • Thread Dumps freeze in the Management Center. It shows the same thread dumps for all connected members, same traces and same thread IDs. And they are shown on the same line numbers. #6536

  • In Hazelcast 3.4.*, the methods migrationStarted/migrationCompleted were only called once when a partition migration was done. But in 3.5, these methods are called twice for each partition. #6396

3.5.3

The following are the enhancements performed for Hazelcast 3.5.3 release.

  • Cache statistics are only supported at the server side and exposed by CacheProxy. At the client side, cache statistics are not calculated and supported so UnsupportedOperationException is thrown at ClientCacheProxy. #6262

  • Added iam-role support to Hazelcast Cloud module. So users can use Hazelcast Cloud module without configuring access keys. #2630

The following are the fixed issues for Hazelcast 3.5.3 release.

  • ClientInvocationFuture may hang when the deserialized response is null. #6363

  • The method CacheStatisticsImpl::getAverageRemoveTime() uses the get count on the cache but it must use the remove count. #6314

  • Hazelcast console.sh should support changing namespaces that contain space characters. #6307

  • The client fails to properly reconnect to a single node cluster after the Hazelcast server is restarted. #6168

  • Transactional Queue ordering on rollback can be violated. The reason is that the QueueContainer does not rollback the changes in the order that is opposite to the order of the items when they were added. It is a random order due to the fact that the changes are stored in a hashmap. #6156

  • When a field, that is only available in a subclass of an interface, is indexed, Null Pointer Exception is thrown. #6151

  • Extra \r\n in the body of the REST API responses causing warnings. #6144

  • Near cache on the client size for replicated map does not get invalidated after replicated map changes. #5694

  • IList.iterator() and listIterator() do not support the method iterator.remove(). #5508

3.5.2

The following are the fixed issues for Hazelcast 3.5.2 release.

  • There is a performance issue: Even when the Spring boot application is doing nothing, CPU consumption is very high. A thread named hazelcast-wm.ensureInstance consumes CPU around 70% because of the method ClusteredSessionService.run(). #6052

  • MapLoader blocks the entire partition when loading a single entry. #5818

  • The method IMap.getAll by-passes interceptors in the Hazelcast 3.3 and higher versions. #5775

  • AWSJoiner fails for the regions except us-east-1. #5653

  • Getting an instance of sun.misc.Unsafe class does not work on HP-UX operating system. #5518

  • AWSAddressTranslator always uses the default region and this causes the HazelcastClient to be unable to join a Hazelcast AWS cluster in a non-default region. #5446

  • The test code JettyWebFilterTest.java does not fail properly. #5188

  • Management Center behaves unfriendly when map entries increase. #4895

  • In hazelcast-client.xml, if the region is configured but host-header is not provided, the configuration gives a default endpoint value of ec2.amazonaws.com. It should give, for example, ec2.eu-west-1.amazonaws.com when the region is eu-west-1 and host-header is not provided. #4731

  • Too much CPU is used when Hazelcast is idle. #81

3.5.1

The following are the fixed issues for Hazelcast 3.5.1 release.

  • Hazelcast Management Center uses UpdateMapConfigOperation to update map configurations. This operation simply replaces the map configuration of the related map container. However, this replacement has no effect for maxIdleSeconds and timeToLiveSeconds properties of the map configuration since they are not used in the map container directly. They are assigned to the final variables during map container creation and never touched again. #5593

  • Destroying a map just after creating it produces double create/destroy events for DistributedObjectListener. #5592

  • Map does not allow changing its maximum size, TTL and maximum idle properties. However, these fields are editable in the Map Config popup of Management Center. These fields should be disabled to prevent misguiding. #5591

  • Map is destroyed using IMap.destroy() but then it is immediately recreated. #5554

  • There should be a better calculation when calling the method getApproximateMaxSize() related to casting. Its return type is int and this causes the map entries to be evicted all the time when, for example, the eviction policy for an IMap is set to heap percentage with the value 1%. #5516

  • All onResponse() calls on a MultiExecutionCallback should be made before the method onComplete() is called. There exists a race condition in ExecutionCallbackAdapterFactory which permits the method onComplete() to be called before all onResponse() calls are made. #5490

  • Hazelcast Management Center’s Scripting tab is not refreshed when a new node joins to the cluster. #4738

  • When updating a map entry which is replicated over WAN, the TTL (time to live) is not honored in the remote cluster map. When the timeout expires, the entry disappears from the cluster in which the key is owned, however it remains in the remote cluster. #254

The following are the enhancements performed for Hazelcast 3.5.1 release.

  • Client instances should spawn threads with their instance names added as prefix. #5671

  • The method com.hazelcast.spi.impl.classicscheduler.ResponseThread::process may catch throwables. When this occurs, it logs an unhelpful message, and ignores the actual exception. This method should be improved to additionally log the cause, or at least the exception class and message. #5619

  • The element min-eviction-check-millis in the map configuration does not exist in documentation. #5614

3.5

This section lists the new features, enhancements and fixed issues for 3.5 release.

New Features

The following are the new features introduced with 3.5 release.

  • Async Back Pressure: The Back Pressure introduced with Hazelcast 3.4 now supports async operations. For more information, please see the Back Pressure section.

  • Client Configuration Import: Hazelcast now supports replacing variables with system properties in the declarative configuration of Hazelcast client. Moreover, now you can compose the Hazelcast client declarative configuration out of smaller configuration snippets. For more information, please see the Composing Declarative Configuration section.

  • Cluster Quorum: This feature enables you to define the minimum number of machines required in a cluster for the cluster to remain in an operational state. For more information, please see the Cluster Quorum section.

  • Hazelcast Client Protocol: Starting with 3.5, Hazelcast introduces the support for different versions of clients in a cluster. Please keep in mind that this support is not valid for the releases before 3.5. Please see the important note at the last paragraph of the Hazelcast Java Client chapter’s introduction.

  • Listener for Lost Partitions: This feature notifies you for possible data loss occurrences. Please see the Partition Lost Listener section and MapPartitionLostListener section.

  • Increased Visibility of Slow Operations: With the introduction of the SlowOperationDetector feature, slow operations are logged and can be seen on the Hazelcast Management Center. Please see the SlowOperationDetector section and Management Center:Members section.

  • Enterprise WAN Replication: Hazelcast Enterprise implementation of the WAN Replication. Please see the Enterprise WAN Replication section.

  • Sub-Listener Interfaces for Map Listener: This feature enables you to listen to map-wide or entry-based events. With this new feature, the listener formerly known as EntryListener has been changed to MapListener and MapListener has sub-interfaces to catch map/entry related events. Please see the Map Listener section for more information.

  • Scalable Map Loader: With this feature, you can load your keys incrementally if the number of your keys is large. Please see the Incremental Key Loading section.

  • Near Cache for JCache: Now you can use a near cache with Hazelcast’s JCache implementation. Please see JCache Near Cache for details.

  • Fail Fast on Invalid Configuration: With this feature, Hazelcast throws a meaningful exception if there is an error in the declarative or programmatic configuration. Please see the note at the end of the Configuration Overview section.

  • Continuous Query Caching: (Enterprise only, since 3.5) Provides an always up to date view of an IMap according to the given predicate. Please see the Continuous Query Cache section

  • Dynamic Selector Rebalancing

  • Management of Unbounded Return Values

Enhancements

The following are the enhancements introduced with 3.5 release.

  • Eventing System Improvements: RingBuffer and Reliable Topic structures are introduced.

  • XA Transactions Improvements: With this improvement, you can now obtain a Hazelcast XA Resource instance through HazelcastInstance. For more information, please see XA Transactions.

  • Query and Indexing Improvements

The following are the other improvements performed to solve the enhancement issues opened by the Hazelcast customers/team.

  • While configuring JCache, duration of the ExpiryPolicy can be set programmatically but not declaratively. #5347

  • Since near cache is not supported as embedded but only at client, at the moment, there is no need for NearCacheConfig in CacheConfig. #5215

  • Support for parametrized test is needed. #5182

  • SlowOperationDetector should have an option to not to log the stacktraces to the log file. There is no need to have the stacktraces written to the normal log file if the Hazelcast Management Center or the performance monitor is being used. #5043

  • The batch launcher should include the JCache API. #4902

  • There are no Spring tags available for Native Memory configuration. #4772

  • In the class BasicInvocationFuture, there is no need to create an additional AtomicInteger object. It should be replaced with AtomicIntegerFieldUpdater. #4408

  • There is no need to use the class IsStillExecutingOperation to check if an operation is running locally. One can directly access to the scheduler. #4407

  • Configuring NearCache in a Client/Server system only talks about the programmatic configuration of NearCache on the clients. The declarative configuration (XML) of the same is not mentioned. #4376

  • XML schema and XML configuration validation is not compliant for AWS configuration. #4310

  • The JavaDoc for the methods KeyValueSource.hasNext/element/key and Iterator.hasNext/next should emphasize the differences between each other, i.e., the state changing behavior should be clarified. #4218

  • While migration is in progress, the nodes will have different partition state versions. If the query is running at that time, it can get results from the nodes at different stages of the migration. By adding partition state version to the query results, it can be checked whether the migration was happening and the query can be re-run. #4206

  • XML Config Schema does not allow to set a SecurityInterceptor Implementation. #4118

  • Currently, certain types of remote executed calls are stored into the executingCalls map. The key (and value) is a RemoteCallKey object. The functionality provided is the ability to ask on the remote side if an operation is still executing. For a partition-aware operation, this is not needed. When an operation is scheduled by a partition specific operation thread, the operation can be stored in a volatile field in that thread. #4079

  • The class TcpIpJoinerOverAWS fails at AWS’ recently launched eu-central-1 region. The reason for the fail is that the region requires v4 signatures. #3963

  • API change in EntryListener breaks the compatibility with the Camel Hazelcast component. #3859

  • The hazelcast-spring-<`version>.xsd` should include the User Defined Services (SPI) elements and attributes. #3565

  • XA Transactions run on multiple threads. #3385

  • Hazelcast client fails to connect when you provide variables from the system properties. #3270

  • Entry listeners are not called when the entries are modified by WAN replication. #2981

  • Map wildcard matching is confusing. There should be a pluggable wildcard configuration resolver. #2431

  • The method loadAllKeys() in map is not scalable. #2266

  • Back pressure feature should be added. #1781

Fixes

The following are the issues solved for Hazelcast 3.5 release.

  • Operation timeout mechanism is not working. #5468

  • MapLoader exception is not logged: Exception should be logged and propagated back to the client that triggered the loading of the map. #5430

  • Replicated Map documentation page does not mention that it is in the beta stage. #5424

  • The method XAResource.rollback() should not need the transaction to be in the prepared state when called from another member/client. #5401

  • The method XAResource.end() should not need to check threadId. #5400

  • The method IList::remove() should publish the event REMOVED. #5386

  • IllegalStateException with wrong partition is thrown when the method IMap::getOperation() is invoked. #5341

  • WrongTarget warnings appear in the log since the operations are not sent to the replicas when a map has no backups. #5324

  • When the method finalizeCombine() is used, Hazelcast throws NullPointerException. #5283

  • WanBatchReplication causes OutOfMemoryException when the default value for WAN Replication Batch Size (50) is used. #5280

  • When testing Hazelcast, it does not start as an OSGI bundle. After the OSGI package was refactored, the dynamic class loading of the Script engine was missed. #5274

  • XA Example from Section 11.3.5 in the Reference Manual broken after the latest XA Improvements are committed. #5273

  • XA Transaction throws TransactionException instead of an XAException on timeout. #5260

  • The test for unbounded return values runs forever with the new client implementation. #5230

  • The new client method getAsync() fails with a NegativeArraySizeException. #5229

  • The method putTransient actuated the MapStore unexpectedly in an environment with multiple instances. #5225

  • Changes made by the interceptor do not appear in the backup. #5211

  • The method removeAttribute will prevent any updates by the method setAttribute in the deferred write mode. #5186

  • Backward compatibility of eviction configuration for cache is broken since CacheEvictionConfig class was renamed to EvictionConfig for general usage. #5180

  • Value passed into ICompletableFuture.onResponse() is not deserialized. #5158

  • Map Eviction section in the Reference Manual needs more clarification. #5120

  • When host names are not registered in DNS or in /etc/hosts and the members are configured manually with IP addresses and while one node is running, a second node joins to the cluster 5 minutes after it started. #5072

  • The method OperationService.asyncInvokeOnPartition() sometimes fails. #5069

  • The SlowOperationDTO.operation shows only the class name, not the package. This can lead to ambiguity and the actual class cannot be tracked. #5041

  • There is no documentation comment for the MessageListener interface of ITopic. #5019

  • The method InvocationFuture.isDone returns true as soon as there is a response including WAIT_RESPONSE. However, WAIT_RESPONSE is an intermediate response, not a final one. #5002

  • The method InvocationFuture.andThen does not deal with the null response correctly. #5001

  • CacheCreationTest fails due to the multiple TestHazelcastInstanceFactory creations in the same test. #4987

  • When Spring dependency is upgraded to 4.1.x, an exception related to the putIfAbsent method is thrown. #4981

  • HazelcastCacheManager should offer a way to access the underlying cache manager. #4978

  • Hazelcast Client code allows to use the value 0 for the connectionAttemptLimit property which internally results in int.maxValue. However, the XSD of the Hazelcast Spring configuration requires it to be at least 1. #4967

  • Updates from Entry Processor does not take write-coalescing into account. #4967

  • CachingProvider does not honor custom URI. #4943

  • Test for the method getLocalExecutorStats() fails spuriously. #4911

  • Missing documentation of network configuration for JCache. #4905

  • Slow operation detector throws a NullPointerException. #4855

  • Consider use of System.nanoTime in sleepAtLeast test code. #4835

  • When upgraded to 3.5-SNAPSHOT for testing, Hazelcast project gives a warning that mentions a missing configuration for hazelcastmq.txn-topic. #4790

  • ClassNotFoundException when using WAR classes with JCache API. #4775

  • When Hazelcast is installed using Maven in Windows environment, the test XmlConfigImportVariableReplacementTest fails. #4758

  • When a request cannot be executed due to a problem (connection error, etc.), if the operation redo is enabled, request is retried. Retried operations are offloaded to an executor, but after offloading, the user thread still tries to retry the request. This causes anomalies like operations being executed twice or operation responses being handled incorrectly. #4693

  • Client destroys all connections when a reconnection happens. #4692

  • The size() method for a replicated map should return 0 when the entry is removed. #4666

  • NullPointerException on the CachePutBackupOperation class. #4660

  • When removing keys from a MultiMap with a listener, the method entryRemoved() is called. In order to get the removed value, one must call the event.getValue() instead of event.getOldValue(). #4644

  • Unnecessary deserialization at the server side when using Cache.get(). #4632

  • Operation timeout exception during IMap.loadAllKeys(). #4618

  • There have been Hazelcast AWS exceptions after the version of AWS signer had changed (from v2 to v4). #4571

  • In the declarative configuration; when a variable is used to specify the value of an element or attribute, Hazelcast ignores the strings that come before the variable. #4533

  • LocalRegionCache cleanup is working wrongly. #4445

  • Repeatable-read does not work in a transaction. #4414

  • Hazelcast instance name with Hibernate still creates multiple instances. #4374

  • In Hazelcast 3.3.4, FinalizeJoinOperation times out if the method MapStore.loadAllKeys() takes more than 5 seconds. #4348

  • JCache sync listener completion latch problems: Status of ICompletableFuture while waiting for completion latch in the cache must be checked. #4335

  • Classloader issue with javax.cache.api and Hazelcast 3.3.1. #3792

  • Failed backup operation on transaction commit causes Nested transactions are not allowed! warning. #3577

  • Hazelcast Client should not ignore the fact that the XML is for server and should not use default XML feature to connect to localhost. #3256

  • Owner connection read() forever. #3401

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.4.8

The following is the fixed issue for 3.4.8 release.

  • Prevents indefinite read of stale value from near-cache. #8091

3.4.7

There are some minor internal improvements.

3.4.6

There are some minor internal improvements.

3.4.5

There are some minor internal improvements.

3.4.4

The following is the fixed issue for 3.4.4 release.

  • MultiMap entry listener provides incorrect null values. #5538

3.4.3

The following is the the enhancement performed for 3.4.3 release.

  • Expose TcpIpJoiner.MAX_PORT_TRIES as a configurable property. #5062

The following are the fixed issues for 3.4.3 release.

  • Subsequent remove operations may cause reading of stale value from the map store. #5368

  • Write-behind may cause reading of stale value upon migration. #5339

  • Hazelcast client is unresponsive. OperationTimeoutException is seen in the logs. #5338

  • Last update time of an entry should not be changed after getAll() is invoked. #5333

  • AtomicReference.alterAndGet() throws HazelcastSerializationException. #5265

  • ICompletableFuture callback from the method getAsync is not always invoked. #5133

  • Warnings and exceptions are logged when closing the client connection. #4966

  • CacheConfig is not created on the cluster if the executer of CacheCreateConfigOperation has already a CacheConfig. #4960

  • The schema does not allow for an explicit hz:replicatedMap element to be created. One can be created inside hz:config but not as a definition for a concrete Replicated Map. Therefore, at present it is impossible to define a Replicated Map using Spring. #4958

  • ResponseThread and InvocationRegistry.InspectionThread reset and retry operations. Since these threads did not implement NIOThread, the OperationExecutor is free to execute tasks on these threads and that is not desirable. #4929

  • The method CacheManager.getCache() does not re-open the closed cache. It should let access to the closed cache and re-open it. Cache can be accessed by getCache but it is still closed. #4631

  • The method close() of a Closeable CacheLoader is called without explicitly calling the method Cache.close(). #4617

  • The method Cache.close() does not call the method close() of registered Closeable CacheEntryListener. #4616

  • The method awaitNanos() returns the wrong value for both the ClientConditionProxy and ConditionImpl classes. #4603

  • The method NotEqualPredicate should return false if entry is null (without index) and also if index is present, it should not throw an exception with null values. #4525

  • When running Hazelcast with Spring and Hibernate 4 and when an application is started, the error related to org/hibernate/cache/QueryResultsRegion is produced. #4519

  • OperationTimeoutException when calling get on task future after hazelcast.operation.call.timeout.millis. #4398

  • Predicates with null values throws exception for unordered indexes. #4373

  • The method queue.take() does not get interrupted on shutdown. #4143

3.4.2

The following is the enhancement performed for 3.4.2 release.

  • The method contains for ISet scans all the items. #4620

The following are the fixed issues for 3.4.2 release.

  • While executing unit tests, SlowOperationDetectorThread and CleanupThread may not be terminated before the next test is started. #4757

  • When multiple nodes join sequentially after partitions are assigned/distributed, old nodes fail to clean backup replicas larger than the configured backup count. This causes a memory leak. Also, when multiple nodes leave the cluster at the same time (or in a short period), the new partition owner looses some partition replica versions and this causes backup nodes for those specific replica indexes to fail synchronizing data from the owner node, although the owner node holds the whole partition data. #4687

  • After cluster merges due to a network-split, Hazelcast infinitely logs WaitNotifyServiceImpl$WaitingOp::WrongTargetException warnings. #4676

  • A strange mapName parameter occurred when using wildcard configuration for a custom MapStoreFactory. #4667

  • The method IExecutorService.submitToKeyOwner encountered two errors: the onResponse method is invoked with null and a cast exception is thrown in a Hazelcast thread. #4627

  • The method init in an implementation of the MapLoaderLifecyleSupport interface is not invoked. #4623

  • The method readData in NearCacheConfig reads the maxSize twice. #4609

  • The system property hazelcast.client.request.retry.count is not handled properly. #4592

3.4.1

The following are the enhancements performed for 3.4.1 release.

  • When the near cache is used, cached entries from the remote node are evicted by idleness, despite being read. #4358

  • HazelcastQueryResultsRegion is never expired/evicted. The cleanup() method in LocalRegionCache for the query cache instances is never called, thus the query cache lives forever. #3882

The following are the fixed issues for 3.4.1 release.

  • IMap.getAll does not put data to RecordStore upon loading from map store. #4458

  • In the ClientNearCache class, there is a comparator which is used in a TreeSet to find the entries that should be evicted. If there are CacheRecords with the same hit count or lastAccessTime (depending on the policy, i.e., LFU or LRU), all of them should be evicted. #4451

  • When using write-behind and the entries, which have not been stored yet, are evicted, duplicate calls to the map store is made. #4448

  • There is a memory leak caused by the empty await queues in WaitNotifyService. When more then one thread try to lock on an IMap key at the same time, a memory leak occurs. #4432

  • ClientListener is not configurable via ListenerConfig. HazelcastInstanceImpl.initializeListeners(Config config) does not honor ClientListener instances. #4429

  • The CacheConfig(CacheSimpleConfig simpleConfig) constructor is broken. Variable assignments should be fixed. #4423

  • In ReplicatedMap, the containsKey method should return false on the removed keys. #4420

  • During the Hazelcast.shutdownAll() process, LockService is shut down before the MapService and this may cause null pointer exception if there is something like isLocked check in some internal IMap operations. #4382

  • Hazelcast clients shut down in the case of an IP change of one or more of the configured node (DNS) addresses. #4349

  • Write-behind system coalesces all operations on a specific key in a configured write-delay-seconds window and it should also store only the latest change on that key in that window. Problem with the current behavior is; a continuously updated key may not be persisted ever due to the shifted store time during the updates. #4341

  • Issue with contains pattern in Config.getXXXConfig(). Since the actual wildcard search always does a contains matching, you cannot set a configuration for startsWith, for instance. #4315

  • ReplicatedMapMBean is not present in JMX. #4173

3.4

This section lists the new features, enhancements and fixed issues for 3.4 release.

New Features

The following are the new features introduced with 3.4 release.

  • High-Density Memory Store: Used with the Hazelcast JCache implementation, High-Density Memory Store is introduced with this release. High-Density Memory Store is the enterprise grade backend storage solution. This solution minimizes the garbage collection pressure and thus enables predictable application scaling and boosts performance. For more information, please see High-Density Memory Store section.

  • Jetty Based Session Replication: We have introduced Jetty-based web session replication with this release. This is a feature of Hazelcast Enterprise. It enables session replication for Java EE web applications that are deployed into Jetty servlet containers, without having to perform any changes in those applications. For more information, please see Jetty Based Web Session Replication section.

  • Hazelcast Configuration Import: This feature, which is an element named <import>, enables you to compose the Hazelcast declarative (XML) configuration file out of smaller configuration snippets. For more information, please see Composing XML Configuration section.

  • Back Pressure: Starting with this release, Hazelcast provides the back pressure feature which prevents the overload caused by pending asynchronous backups. For more information, please see Back Pressure section.

Enhancements

The following are the enhancements performed for 3.4 release.

  • Event packets sent to the client do not have partitionId. #4071

  • Spring Configuration for ReplicatedMap is Missing. #3966

  • NodeMulticastListener floods log file with INFO-level messages when debug is enabled. #3787

  • A Hazelcast client should not be a HazelcastInstance. It should be a factory and this factory should be able to shut down Hazelcast clients. #3781

  • InvalidateSessionAttributesEntryProcessor could avoid creating strings at every call to process. #3767

  • The timeout for SocketConnector cannot be configured. #3613.

  • The method MultiMap.get() returns collection, but this method should return the correct collection type (Set or List). #3214

  • HazelcastConnection is not aligned with HazelcastInstance. #2997

  • Support for Log4j 2.x has been implemented. #2345

  • Management Center console behavior on node shutdown. #2215

  • When queue-store is not enabled, QueueStoreFactory should not be instantiated. #1906

  • Management Center should be able to say when cluster is safe and all backups are up to date. #963

Fixes

The following are the fixed issues for 3.4 release.

  • Deadlock happens in MapReduce implementation when there is a high load on the system. The issue has been solved by offloading Distributed MapReduce result collection to the async executor. #4238

  • When the class ClientExecutorServiceSubmitTest.java is compiled using the Eclipse compiler, it gives a compile error: The method submit(Runnable, ExecutionCallback) is ambiguous for the type IExecutorService. The reason is that the IExecutorService.java class does not have some generics. The issue has been solved by adding these missing generics to the IExecutorService.java class. #4234

  • JCache declarative listener registration does not work. #4215

  • JCache evicts the records which are not expired yet. To solve this issue, the clear method should be removed that runs when the size is smaller than the minimum eviction element count (MIN_EVICTION_ELEMENT_COUNT). #4124

  • Hazelcast Enterprise Native Memory operations should be updated in relation with the Hazelcast sync listener changes. #4089

  • The completion listener (JCache) relies on event ordering but if the completion listener is registered in another node then event ordering is not guaranteed. #4073

  • AWS joiner classname should be fixed since EC2 discovery is not working after the restructure. #4025

  • If an IMap has a near cache configured, accessing the near cache via the method get(key) does not count as an access to the underlying IMap. The near cache has its own max-idle-seconds element. However, if an entry is expired/evicted in the IMap, it also causes a near cache removal operation for the entry regardless of the max-idle-seconds of that entry in the near cache. The entry expires and is evicted even if the near cache is being hit constantly. When a near cache is hit, the underlying map should reset the idle time for that key. #4016

  • Getting a pre-configured Cache instance is not working as expected. #4009

  • Bounded Queue section in the Reference Manual is unclear and wrong. #3995

  • The method checkFullyProcessed of MapReduce throws null pointer exception. The reason may be that multiple threads attempt to start the final processing state in the JobSupervisor. #3952

  • Merge operation after a split brain syndrome does not guarantee that the merging is over. #3863

  • When a client with near cache configuration enabled is shut down, RejectedExecutionException is thrown. #3669

  • In Hazelcast IMap and TransactionalMap, read-only operations such as get(), containsKey(), keySet(), and containsValue() break the transaction atomicity. #3191

  • Documentation should clearly list features of and differences between native clients. #2385

  • Sections of Hazelcast configuration should be able to be imported so that these sections can be shared between other Hazelcast configurations. #406

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.3.5

The following is the fixed issue for 3.3.5 release.

  • Make write-coalescing configurable for write-behind mapstores. #4438

3.3.4

The following are the fixed issues for 3.3.4 release.

  • Predicate with short values is not working. #4293

  • Hits statistics copy and paste error in the method ReplicatedRecord. #4254

  • Serialization error on the ReplicatedMap when in-memory format is set to BINARY. #4205

  • Too long exception stacktraces if the Hazelcast client fails to receive data, and this leads to a failure on the client. #4192

  • Hazelcast client registers the translated public address instead of its own private address to the list of connections. This causes the client not to be able to remove the connection correctly. #4190

  • TransactionType:Local emits exceptions while committing. The normal behavior should be throwing the exceptions to the user. #4160

  • Map replication should mark expirable recordstore. Otherwise, in some situations, if one does not set the map wide expiration or map wide TTL, the key based TTL expiration may not work. #4144

  • The method BasicInvocationFuture.response should be cleared when BasicInvocation.WAIT_RESPONSE is read by the waiter thread. Otherwise, when the retry operation takes too much time, the waiting thread sees the same wait response multiple times and the operation may not timeout forever. #4123

  • Topic listeners should be unregistered when topic is destroyed. #4117

  • Invocations (and their operations) remain in the invocations map forever if the operation timeouts without a response. #4113

  • Timeout is needed for parallel query operations. #4074

  • Initial map load and max-size-policy conflict. #4066

  • MapStore operations should be retried and performed eventually after a temporary failure. #4061

  • The class SynchronizedWriteBehindQueue (from com.hazelcast.map.mapstore.writebehind package) is declared threadsafe in JavaDocs, but it is not. #4039

  • The method RemoveIfSameOperation does not set dataOldValue for the entryRemoved event. #4037

  • When a new node with a new field is added to a cluster and when a query over this node is attempted, the old nodes throw an exception (com.hazelcast.query.QueryException: Unknown Portable field: newFieldName) and the query fails by throwing the same exception. #3927

  • At the moment, the internal state fields of a Reducer are required to be volatile to ensure the memory visibility effects after the suspension and continuation of a reducer. This requirement should be moved to be handled by the framework itself since it is tend to be forgotten. #3866

  • The method executeOnKey hangs when the server fails to handle a query. #3842

  • The GlobalSerializerConfig#setImplementation() parameter should be compatible with the implementation field. #3569

  • ClientConsoleApp should not define the file hazelcast-client.xml. #3554

  • When using a custom partitioning strategy and the configured backup count of a map cannot be fulfilled since a node defined in the custom partition group is down, a JMX service call is blocked in the while-loop at com.hazelcast.map.MapService.createLocalMapStats. #3526

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.3.3

The following are the fixed issues for 3.3.3 release.

  • JCache average put time statistic is not calculated correctly. #4029

  • When sending backup, the replica address can be seen as null. #4001

  • Evicted events are sent before the added events to EntryListeners. #3992

  • In Management Center, the default login credentials cannot be deleted. #3990

  • Logger for NodeMulticastListener does not belong to com.hazelcast hierarchy. #3941

  • MapInterceptors are not removed when a node leaves the cluster. #3932

  • MapInterceptors of same type (Class) are chained. #3931

  • Expiration Time should not be updated. Its value is updated on every set operation on a map, but Hazelcast uses only the first value set for ExpirationTime. So a getExpirationTime() operation returns a wrong and misleading value. #3923

  • When using the XML file to configure a Queue to use a QueueStoreFactory, a null pointer exception is thrown at QueueStoreWrapper. #3907

  • Excess logging on startup. #3869

  • LifecycleService should be terminated after the node cannot join to the cluster. #3843

  • The method MapProxyImpl.aggregate hangs sporadically. #3824

  • Currently, there is no class named com.hazelcast.nio.utf8.EnterpriseStringCreator in Hazelcast. So the class and its log messages should be removed from the code. #3819

  • Bad user interface experience in the management center. Maps menu item that contains maps with longer names cannot be expanded. #3815

  • When the shutdown button in the management center is hit multiple times, the nodes are shutdown again, after they are shutdown at the first place and restarted. #3718

  • Alert e-mails from the management center are not sent to the e-mail address. #3693

  • Instances with private IPs cannot be discovered on Amazon EC2. #3666

  • Null pointer exception in the method Records.buildRecordInfo from the stabilizer MapStoreTest. #2956

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.3.2

The following are the fixed issues for 3.3.2 release.

  • Reject multicast messages if the group configuration is not matching. #3806

  • Map#getEntryView should check expiration of a key. #3801

  • Hazelcast gets stuck in HazelcastInstanceNotActiveException loop during multicast join. #3732

  • Hazelcast fails to comply with maxIdleTime expiration when running EntryProcessors. A delay should be added to expiration times on backups. #3710

  • containsKey() in transactional context returns wrong value for keys deleted within transaction. #3682

  • TransactionalMap.values() returns stale values that was updated within the transaction boundary. #3668

  • Number of loaded keys should not exceed map’s maximum size. #3608

  • During client node shutdown, if the cluster happens to be down, Hazelcast logs some extra messages at SEVERE level. #3493

3.3.1

The following are the fixed issues for 3.3.1 release.

  • MapReduce Combiner creation is not threadsafe, but certain operations on mapping phase might need a concurrent creation of the combiners. #3625

  • When connectionTimeout property in ClientNetworkConfig is set to Integer.MAX_VALUE, the client could not connect to cluster since a default 2000 ms. extra value is added to connectionTimeout while connecting. #3615

  • User provided list results from combiner is colliding with the internally used multi-result list. #3614

  • While committing collection transactions, the collection item is being added to the collection container. However, this gives the warning There is no suitable de-serializer for type warning. Instead of collection item, transactional item should be added to the container. #3603

  • MaxSizeConfig constructor should convert zero size to Integer.MAX_VALUE. #3579

  • If deserialization of the client request fails, the exception is not propagated back to the client. #3557

  • Lock is not owned by by the transaction exception. This exception was received while testing how transactions are working with Map and MultiMap for some last Hazelcast releases. #3545

  • Main classes in manifest.mf files are not correctly set. #3537

  • Count of evicted events may exceed the map size when the "read backup data" feature is enabled. #3515

  • mancenter.war from Hazelcast release 3.2.5 cannot be deployed to Glassfish 3.1.2.2 and it fails to deploy. #3501

  • While evicting entries from a map with the method evictAll, locked keys should stay in the map. #3473

  • In hazelcast-vm module, before every test, new server container is started. And after every test, running server is terminated. This behavior causes a long test execution time. Server start-up and termination should be done before and after test class initialization and finalization. #3473

  • The method IQueue.take() method should throw InterruptedException, but throws HazelcastException instead. #3133

  • Multicast discovery doesn’t work without network. #2594

3.3

This section lists the new features, enhancements and fixed issues for 3.3 release.

New Features

The following are the new features introduced with 3.3 release.

  • Heartbeat for Java client: Before this release, a Java client could not detect a node as dead, if the client is not trying to connect to it. With this heartbeat feature, each node will be pinged periodically. If no response is returned from a node, it will be deemed as dead. Main goal of this feature is to decrease the time for detection of dead (disconnected) nodes by Java clients, so that the user operations will be sent directly to a responsive one. For more information, please see Client Properties.

  • Tomcat 6 and 7 Web Sessions Clustering: Please see Web Session Replication.

  • Replicated Map implemented: Please see Replicated Map

  • WAN Replication improved: Added configurable replication queue size WAN Replication Queue Size.

  • Data Aggregation implemented: Added common data aggregations, please find Aggregators documentation.

  • EvictAll and LoadAll features for IMap: evictAll and loadAll methods have been introduced to be able to evict all entries except the locked ones and that loads all or a set of keys from a configured map store, respectively. Please see Evicting All Entries and Forcing All Keys to be Loaded sections for more information.

  • Hazelcast JCache implementation introduced: Starting with release 3.3.1, Hazelcast offers its JCache implementation. Please see Hazelcast JCache Implementation for details.

Fixes

The following are the fixed issues 3.3 release.

  • TxQueue cannot find reserved items upon ownership changes. #3432

  • Documentation update is needed to tell that PagingPredicate is only supported for Comparable objects if there is no comparator. #3428

  • java.lang.NullPointerException is thrown when publishing an event in ClientEndPointImpl. #3407

  • The entryUpdated() callback of a listener during a transaction always has a null oldValue in the EntryEvent. #3406

  • Documentation update with the links to code samples for integration modules. #3389

  • Hazelcast write-behind with map.replace() stores replaced items. #3386

  • XAResource’s setTransactionTimeout() method is not correctly implemented. #3384

  • Hazelcast web session replication filter may die if response committed. #3360

  • Resource adapter state is never reset to isStarted == false, resulting in errors down the line. #3350

  • PagingPredicate.getAnchor does not return the correct value. #3241

  • If deserialization fails, calling node is not informed. #2509

  • CallerNotMemberException and WrongTargetException exceptions are thrown at random intervals. #2253

RC3 Fixes

This section lists issues solved for 3.3-RC3 (Release Candidate 3) release.

  • Parallel execution of MapStore#store method for the same key triggered by IMap#flush. #3338

  • When offering null argument in queue throws an exception but it adds null argument to collection, then addAll() performed on this list does not throw an exception. #3330.

  • java.io.FileNotFoundException thrown by MapLoaderTest #3324

  • MapMaxSizeTest Stabilizer test with SoftKill. #3291

  • Incompatible Spring and Hazelcast configuration XSDs. #3275

  • ExpirationManager partition sorting can fail. #3271

  • Configuration validation is broken. #3257

  • Code Samples for Spring Security and WebFilter Integration. #3252

  • WebFilter Test Cases are slow. #3250

  • Management Center and Weblogic Deployment Problem. #3247

  • Enabling Multicast and TCP/IP node discovery methods freeze the instances. #3246

  • getOldValue and getValue returns the same value when removing item from IMap. #3198

  • MapTransactionContextTest: member SoftKill and then HazelcastSerializationException and IegalStateException: Nested are thrown. #3196

  • IMap.delete() should not call MapLoader.load(). #3178

  • 3.3-RC3+: NPE in the method connectionMarkedAsNotResponsive. #3169

  • WebFilter.HazelcastHttpSession.isNew() does not check the Hazelcast Session Cache. #3132

  • Hazelcast Spring XSD files are not version agnostic. #3131

  • ClassCastException: java.lang.Integer cannot be cast to java.lang.String Query. #3091

  • Predicate returns a value not matching the predicate. #3090

  • Modifications made by Entry Processor are lost in 3.3-RC-2. #3062

  • Hazelcast Session Clustering with Spring Security Problem. #3049

  • PagingPredicate returning duplicated elements results in an infinite. loop #3047

  • expirationTime on EntryView is not set. #3038

  • BasicRecordStoreLoader cannot handle retry responses. #3033

  • Short await() on condition of contended lock causes IllegalStateException. #3025

  • Indices and Comparable: not documented. #3024

  • Marking Heartbeat as healthy is too late. #3014

  • 3.3-RC2: IMap#keySet triggers value deserialization. #3008

  • map.destroy() throws DistributedObjectDestroyedException. #3001

  • Stabilizer tests Final profile, Xlarge cluster OperationTimeoutException. #2999

  • com.hazelcast.jca.HazelcastConnection::getExecutorService returns plain ExecutorService. #2986

  • Serialization NPE in MapStoreTest stabilizer, 3.3-RC3-SNAPSHOT. #2985

  • Bug with IMap.getAll(). #2982

  • Client deadlock on single core machines. #2971

  • Retrieve number of futures in loop in calling thread. #2964

RC2 Fixes

This section lists issues solved for 3.3-RC2 (Release Candidate 2) release.

  • evictAll should flush to staging area. #2969

  • NPE exception in MapStoreTest. #2956

  • Fixed AddSessionEntryProcessor. #2955

  • Added StripedExecutor to WanReplicationService. #2947

  • All read operations of map should respect expired keys. #2946

  • Fix test EvictionTest#testMapWideEviction. #2944

  • Heartbeat check of clients from nodes. #2936

  • WebFilter does not clean up timed- out sessions. #2930

  • Fix leaking empty concurrent hashmaps. #2929

  • Data loss fix in hazelcast-wm module. #2927

  • Configured event queue capacity. #2924.

  • Client closes owner connection when a connection to the same address is closed. #2921

  • Close the owner connection if heartbeat timeout when client is smart. #2916

  • Set application buffer size to not exceed tls record size. #2914

  • EntryProcessor makes unnecessary serialization. #2913

  • Make evictable time window configurable. #2910

  • Fixes data loss issue when partition table is being synced and a node is gracefully shutdown. #2908

  • MapStoreConfig; implementation instance is not set, when configured via XML. #2898

  • LocalMapStats does not record stats about locked entries in 3.x. #2876

  • Concurrency security interceptor. #2874

  • Client hangs during split, if split occurs due to network error. #2850

  • Network connection loss does not release lock. #2818

RC1 Fixes

This section lists issues solved for 3.3-RC1 (Release Candidate 1) release.

  • It is not possible to copy the link from http://hazelcast.org/download/ and run wget on it. #2814

  • mapCleared method for EntryListener is needed. #2789

  • The method keySet with predicate should trigger loading of MapStore. #2692

  • MapStore with write-behind: The method IMap.remove() followed by IMap.putIfAbsent(key,value) still returns the old value. #2685

  • Hazelcast cannot read UTF-8 String if multiple-byte characters end up at position that is an even multiple of buffer size. #2674

  • Current implementation of record expiration relies on undefined behavior of System.nanoTime(). #2666

  • Inconsistency at Hazelcast Bootup Editions message. #2641

  • AbstractReachabilityHandler writes to standard output. #2591

  • IMap.set() does not not remove a key from write behind deletions queue. #2588

  • com.hazelcast.core.EntryView#getLastAccessTime is invalid. #2581

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.2.7

There are some minor internal improvements.

3.2.6

The following are the fixed issues for 3.2.6 release.

  • MapStore in write-behind mode throws Exception (Spring configured). #3397

  • Wildcard pattern of the map (map is configured with a wildcard in its name) should be respected during evictions. #3345

  • Map eviction does not work when the policy is USED_HEAP_PERCENTAGE or USED_HESAP SIZE. #3321

  • Exceptions when using Portable serialization. #3313

  • When Hazelcast is used as drop-in replacement for Memcached, it causes errors. #3182

  • Null Pointer Exception is thrown by MapService.dispatchEvent. #3101

  • PagingPredicate returns duplicated elements which result in an infinite loop. #3047

  • ContextClassLoader is by default only set on some cached operation threads, not on most others. #2721

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.2.5

The following are the fixed issues for 3.2.5 release.

  • Txn map keyset and values with portable entries is not working correctly. #3152

  • The TransactionalMap.{putIfAbsent(k, v), replace(k, v), replace(k, v1, v2), remove(k, v)} methods never release lock after transaction ends. #3149

  • Test failure at ClientMapTest.testMapStatistics. #3138

  • NetworkConfig.setReuseAddress is not available in the XML. #3122

  • When a selector fails to open, the AbstractSelector does not throw an exception, but logs it and then continues. Also, when select throws an IOException, this exception is not dealt correctly. #3105

  • Test failure at QueryBasicTest.testInPredicateWithEmptyArray. #3060

  • Hibernate cache flush leaves ClientMapProxy in an inconsistent state. This cache flush triggers IMapRegionCache.clear() and the implementation here does not look correct since it leaves the map field in the inconsistent state (context = null) and prevents any further use of it. #3004

  • Fixes operation execution/invocation on IO threads issue. #2994

  • Node cannot recover from MergeOperation if target node exits the cluster. #2937

  • Client fails to run due to the lack of ClientTestApp class. #2817

  • Using Hazelcast Queue, assume that there is a system in which messages are actively being consumed by one consumer. When a second Hazelcast instance is started, i.e., second consumer for the same queue, Hazelcast throws an exception, then continues normally and there are two competing consumers on the same queue. #2805

  • IMap.submitToKey and IMap.executeOnKey in combination with nodes joining/leaving the cluster result in data loss. #2785

  • Too much delay for deciding heartbeat timeout. #2766

  • When multiple predicates are combined by an AndPredicate, the first IndexAwarePredicate that is not indexed will be added to the no index list twice. #2531

  • There appears to be a leak in the memory in SecondsBasedEntryTaskScheduler when idle timeout is enabled on a map. #2343

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.2.4

The following are the fixed issues for 3.2.4 release.

  • Assigning wrong item ID on backup when instance is shutdown in QueueStore. #2842

  • IQueue.take throws HazelcastException instead of InterruptedException. #2833

  • Hazelcast distribution has some left-over files. #2821

  • Management Center cannot update map configuration on already created map proxies. #2810

  • Transient test failure: IMap.get(k) returns null. #2804

  • IllegalArgumentException: Target cannot be null! shows up when MultipleEntryBackupOperationForNullValue test is run by executeOnKeys() firstly. #2754

  • When creating an instance with EntryProcessor, the backup process does not work. #2752

  • Data loss happens in the web filter. #2746

  • BackupEntryProcessor stores the value even though it is not set explicitly. #2613

  • The test listenerAddStress fails often. #2611

  • Predicate should fail when null argument is passed. #2577

  • XAResourceWrapper does not honor contract of XAResource#setTransactionTimeout. #2569

  • Allow Predicates.and and Predicates.or for more than 2 arguments. #2563

  • Semaphore is given to the thread that is coming late. #2472

  • UnknownHostException is logged when using hostnames for seed addresses. #2125

  • The Java client seems to hang if there is blocking of a map call in map listener. #2106

  • The cluster is not responsive when 2nd node joins. #2105

  • Hibernate query caches are not configurable. #2064

  • XA Transactions should be explained in the Reference Manual. #2020

  • Encryption enabled results in cluster to hang under load. #1897

  • SSL connections are unreliable. #1896

3.2.3

The following are the fixed issues for 3.2.3 release.

  • The method TransactionalQueue.poll fails to find an item that is put into the queue in the same transaction with a non-zero timeout. #2669

  • IExecutorService fails when it is called with MemberSelector instance that returns 0 Members. #2650

  • If statistics are enabled, map’s InMemoryFormat.Object does not work. #2622

  • There is a memory leak in long running Hazelcast instances because of a bug in MapEvictionManager.MapEvictTask. #2596

  • Hazelcast client is missing an extensive XML Config Test and XSD Test. #2587

  • The client may consider the original address list if no cluster member can be reached. #2585

  • Locks are not cleaned upon the operation map.destroy. #2582

  • Classpath: Configuration file discovery is not working. #2565

  • The method ClientService.getConnectedClients() does not always return the correct number of clients. #2541

  • Nodes leaving and joining from/to the cluster can cause multiple subscription callbacks to be sent. #2539

  • Predicate fails with NullPointerException if the value is null. #2522

  • Messages’ order breaks with versions 3.2+. #2513

  • The method ClientProxy.destroy should always clean the resources. #2475

  • The method HazelcastHttpSession.getAttributeNames returns entries that were only read but not written. #2434

3.2.2

The following are the fixed issues for 3.2.2 release.

  • Client security callable fix. #2561

  • Updating a key in a transaction gives listeners an entryAdded() callback instead of entryUpdated(). #2542

  • Client ssl engine doesn’t need keyStore and keyStorePassword. #2525

  • Added support for Mapper, Combiner, Reducer, KeyValueSource to implement HazelcastInstanceAware. #2502

  • Fixed alter function. #2496

  • Return cached value upon IMap.get() if near cache is enabled. #2482

  • Exception initializing hz:client. #2480

  • Fixed portable serialization between different services versions. #2478

  • Resolves a data race in the client proxy that can lead to an NPE. #2474

  • Fixed partition group hostname matching. #2470

  • Client shutdown issue: Improve logging. #2442

  • Unnecessary synchronized lock when invoking com.hazelcast.instance.LifecycleServiceImpl.isRunning(). #2454

  • If MapStoreFactory throws exception, instance hangs. #2445

  • Semaphore is given to the thread that is coming late. #2443

  • Lots of exceptions when shutting down connection. #2441

  • Migration fails when statistics are disabled. #2436

  • 3.2.1 regression: nested transactions are not caught and prevented. #2404

  • Client proxy init synced. #2376

  • Fixes hostname matching problem when interface has wildcards. #2398

  • Fix weblogic shutdown backport. #2391

  • NotWritablePropertyException connectionAttemptLimit with ssl client config. #2335

  • Map-Reduce Operation fails, when another instance tries to form a cluster with an instance running a map reduce task. #2354

  • EntryEvent getMember returning null when a node leaves the cluster. #2358

  • NullPointerException in Bundle Activator. #2489

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.2.1

The following are the fixed issues for 3.2.1 release.

  • JCA problems have been fixed. #2025

  • C++ client compilation problems are fixed.

  • Redo problem about Java dummy client is fixed.

  • Round robin load balancer of Java client is improved.

  • Initial timeout is for the initial connections in Java clients.

  • Wildcard configuration improvement in near cache configuration.

  • Unneeded serializations in EntryProcessor should be removed when the object format is In-Memory. #2139

  • Race condition in near cache has been solved, immediate invalidation of local near cache was needed. #2163

  • Predicate issue seen in transactions is solved.

  • Comparator issue in map eviction is solved.

  • Map eviction part has been refactored due to a race condition on map listener. #2324

  • Stale data problem in client near cache has been solved. #2065

  • Many checkstyle and findbugs issues are solved.

3.2

This section lists the new features, enhancements and fixed issues for 3.2 release.

New Features

  • NIO Client: New architecture based on NIO introduced to support more scalable and concurrent client usage.

  • MapReduce Framework: MapReduce implemented for your key-value collections that need to be reduced by grouping the keys. Please see the interview and MapReduce section.

  • Order/Limit Support: Now you can order and limit results returned by queries performed on Hazelcast Distributed Map.

  • *C Client*: Native C client developed for C++ users which can connect to a Hazelcast cluster and realize almost all operations that a node can perform. Please see Native Clients.

  • C# Client: Also, Native C# client that has a very similar API with Native Java client developed. Please see Native Clients.

Enhancements

  • Size of a distributed queue via REST API can be returned. #1809

  • InitialLoadMode configuration parameter (having Lazy and Eager as values) added to MapStoreConfig. #1751

  • Tagging support for Executor Service introduced such that nodes can be tagged for IExecutorService. 1457

  • getForUpdate() operation for transactional map introduced. #1033

  • Entry processor can run on a set of keys with the introduction of executeOnKeys(keys,entryprocessor) method for IMap. 1423

  • getNearCacheStats() introduced. Statistics for near cache can be retrieved. #30

Fixes

  • LocalMapStats.getNearCacheStats() can return null when it is called before a map get that calls initNearCache(). #2009

  • testMapWithIndexAfterShutDown fails in OpenJDK. #2001

  • Portable Serialization needs objects to be shared between client and server. #1957

  • Near cache entries should be locally invalidate on IMap.executeOnKey(). #1951

  • OperationTimeoutException is thrown when executing task that runs longer than hazelcast.operation.call.timeout.millis. #1949

  • MapStore#store was called when executing AbstractEntryProcessor on backup. #1940

  • After an OperationTimeoutException is thrown from `ILock.tryLock() (and after the system is back in a normal state), the named lock remains locked. #1937

  • Hazelcast client needs OutOfMemoryErrorDispatcher. #1933

  • Near Cache: Caching of local entries may lead to race condition. #1905

  • After key owner node dies, it takes too much time for threads to wakeup from condition.await(). #1879

  • Possible improvements/fixes for NearCache. #1863

  • MultipleEntryBackupOperation does not handle deletion of entries. #1854

  • If topics are created/destroyed, then the statistics for that topic are not destroyed and this can cause a memory leak. #1847

  • PartitionService backup/replication fixes. #1840

  • Cached null values remain in near cache after evict is called. #1829

  • NullPointerException in MultiMap when the service is shutdown before the migration is processed. #1823

  • Network interruption causes node to continually warn with WrongTargetException. #1815

  • DefaultRecordStore#removeAll should be modified so that it keeps key objects to delete as a list, not a set. #1795

  • Very long operation.run() call stack especially when high partition count is used. #1745

  • When executing an entry processor with an index aware predicate, the index is not used, instead the predicate is applied to the entire entry set. #1719

  • When one node goes down in a cluster with 2 nodes (where near cache is enabled), containsKey call hangs in the second node. #1688

  • When deleting an entry from an entry processor by setting the value to null, it is not removed from the backup store. #1687

  • Client calls executed at server side cause unwanted (de)serialization. #1669

  • In TrackableJobFuture.get(long, TimeUnit), there is a 100 ms of sleep-spin while waiting for the result of a MapReduce task to be set. #1648

  • If storeAll takes much time and if instance terminates while map store is running, data can be lost. #1644

  • A missing Spring 4 Cache method added to hazelcast-spring package (namely public T get(Object key, Class type)). #1627

  • When eviction tasks are canceled, scheduledExecutorService is not cleaned. #1595

  • storeAll() with new value for the same key should not be executed until any previous storeAll() operations with the same key are not completed. #1592

  • When using native client to interact with Hazelcast cluster, some JMX MBean attribute values on cluster nodes are not set/updated. #1576

  • IMap.getAll(keys) method does not read from near cache. #1532

  • Near Cache cache-local-entries attribute is missing in hazelcast-spring-3.2 XSD. #1524

  • Exception while executing script in OpenJDK 8. #1518

  • Infinite waiting on merge operations when cluster shuts down. #1504

  • Client side socket interceptor is not needed to be MemberSocketInterceptor. #1444

  • Near cache on the local node should be enabled if its InMemoryFormat is different from that of the map. #1438

  • Async EntryProcessor does not deserialize the value before it is called back. #1433

  • A submitted task cannot be canceled via the native client. #1394

  • executeOnKeys(keys,entryprocessor) introduced on IMap. With this feature entry processor can be run on a set of keys. #1339

  • FINEST logging should be guarded where appropriate. #1332

  • False errors reported in Eclipse due to schema definition. #1330

  • Index based operations are not synchronized with partition changes. #1297

  • Management Center: InvocationTargetException in Tomcat console when a node is started and then stopped. #1267

  • The system property hazelcast.map.load.chunk.size is being ignored in Hazelcast 3.1. #1110

  • Master should fire repartitioning after getting confirmation from nodes. #1058

  • SqlPredicate does not Implement equals/hashCode. #960

  • DelegatingFuture.isDone seems to always return false until the method DelegatingFuture.get is called. #850

  • Predicate support for entry processor. #826

RC2 Fixes

The following are the fixed issues for 3.2-RC2 (Release Candidate 2) release.

  • ClientService.getConnectedClients returns all end points. #1883

  • MultiMap is throwing ConcurrentModificationExceptions. #1882

  • executorPoolSize field of ClientConfig cannot be configured using XML. #1867

  • Partition processing cannot be postponed. #1856

  • Memory leak at client endpoints. #1842

  • Errors related to management center configuration on startup. #1821

  • XML parsing error by client. #1818

  • ClientReAuthOperation cannot return response without call ID. #1816

  • MemberAttributeOperationType should be introduced to remove the dependency to MapOperationType. #1811

  • Entry listener removal from MultiMap. #1810

RC1 Fixes

The following are the fixed issues for 3.2-RC1 (Release Candidate 1) release.

  • TransactionalMap does not support put(K,V,long,TimeUnit). #1718

  • Entry is not removed from backup store when it is deleted using entry processor. #1687

  • Possibility of losing data when MapStore takes a long time. #1644

  • When eviction tasks are cancelled, scheduledExecutorService should be cleaned. #1595

  • A fix related to StoreAll is needed in a write-behind scenario. #1592

  • Update problem at map statistics. #1576

  • Exception while executing script in OpenJDK 8. #1518

  • StackOverflowError at AndResultSet. #1501

  • Near Cache using InMemoryFormat.OBJECT also for local node. #1438

  • Async entry processor is not deserializing the value before returning. #1433

  • Distributed Executor; Future Cancel is not working. #1394

  • HazelcastInstanceFactory$InstanceFuture.get() never returns when newHazelcastInstance() method fails/throws exception. #1253

  • Changes for Vertx on Openshift. #1176

  • Serialization should be performed after database interaction for MapStore. #1115

  • System property related to chunk size is passed over in Hazelcast 3.1. #1110

  • Map backups lack eviction of some specific data. #1085

  • DelegatingFuture.isDone always returns false until get is called. #850

  • Predicate support for entry processor. #826

  • Full replication of Maps should be performed. #360

Contributors

We would like to thank the contributors from our open source community who worked on this release:

3.1.9

The following is the enhancement performed for 3.1.9 release.

  • Exposed TcpIpJoiner.MAX_PORT_TRIES as a configurable system property. #9734

3.1.8

This section lists the new features, enhancements and fixed issues for 3.1 and 3.1.x releases.

New Features

This section provides the new features introduced with Hazelcast 3.1 release.

  • Elastic Memory (Enterprise Extensions Only) is now available.

  • Hazelcast Security is now available.

  • Hazelcast JCA integration is back.

  • Controlled Partitioning: Controlled Partitioning is the ability to control the partition of certain DistributedObjects like the IQueue, IAtomicLong or ILock. This will make collocating related data easier. Hazelcast map also supports custom partitioning strategies. A PartitioningStrategy can be defined in map configuration.

  • TransactionalMap now supports keySet(), keySet(predicate), values() and values(predicate) methods.

  • Eviction based on USED_HEAP_PERCENTAGE or USED_HEAP_SIZE now takes account real heap memory size consumed by map.

  • SqlPredicate now supports ’' as escape character.

  • SqlPredicate now supports regular expressions using REGEX keyword. For example, map.values(new SqlPredicate("name REGEX .*earl$")).

  • Hazelcast queue now supports QueueStoreFactory that will be used to create custom QueueStores for persistent queues. QueueStoreFactory is similar to map’s MapStoreFactory.

  • TransactionalQueue now supports peek() and peek(timeout, timeunit) methods.

  • Client now has SSL support.

  • Client also supports custom socket implementations using SocketFactory API. A custom socket factory can be defined in ClientConfig.

  • Hazelcast IList and ISet now have their own configurations. They can be configured using config API, XML and Spring.

  • HazelcastInstance.shutdown() method added back.

  • OSGI compatibility is improved significantly.

Fixes

This section lists issues solved for 3.1 and 3.1.x releases.

  • ClassCastException when using near cache. #1941

  • Hazelcast management console does not allow update of map configuration in runtime. #1898

  • MultiMap ConcurrentModificationExceptions. #1882

  • Near Cache: Race Condition. #1861

  • ClientReAuthOperation cannot return response without call ID. #1816

  • REST put overrides TTL from the configuration. #1783

  • Management Center usage should not trigger loading of map. #1770

  • ResourceAdaptor fix to handle external HazelcastInstance breaking scheme. #1727

  • There is no executeOnAllKey for IMap, the documentation should be changed. #1711

  • Memory leak; backupCalls record is not removed if Future.get() is not called during IMap.putAsync() or IMap.removeAsync(). #1693

  • ClassCastException when removing from Multimap. #1667

  • com.hazelcast.spi.exception.RetryableHazelcastException: Map is not ready. #1593

  • Member join while loading persistent maps throws NPEs. #1577

  • Near cache does not work properly for null values. #1570

  • Memory Leak on operation timeout. #1559

  • EOFException: Remote socket is closed. #1551

  • Severe NullPointerException but then everything works fine. #1548

  • IMap.putTransient(K,V) method (without TTL) is missing. #1526

  • Object is not an instance of declaring the class com.hazelcast.query.impl.QueryException. #1503

  • Setting socket timeout, Hazelcast client will throw SocketTimeoutException. #1435

  • QueueStore.store(Long, Object) is called multiple times from TransactionalQueue.offer. #1412

  • Warning: No transaction item for itemId: . #1393

  • StackOverflow in AndResultSet. #1349

  • maxIdleSeconds is set to DEFAULT_TTL_SECONDS in com.hazelcast.config.MapConfig. #1308

  • Serializer implementation registration with Spring config fails. #1294

  • ReadBackupData on Map exposes internally stored value. #1292

  • Multimap: Lock is not owned by the transaction. #1276

  • Java deserialization of non-static inner classes fails. #1265

  • HazelcastInstance is not injected into MapInterceptor when HazelcastInstanceAware is implemented. #1233

  • Remove compare optimization from map put. #1231

  • Unneeded deserialization in EntryOperation. #1182

  • Adding an InitialMembershipListener to ClientConfig using ClientConfig.addListenerConfig throws NullPointerException. #1181

  • Insufficient calculation of next key/value when doing a restore of message queue from database. #1165

  • Case insensitivity of configurations in XML. #1152

  • The scheme hazelcast-spring3.1.xsd throws SAXParseException when hz:interfaces have multiple hz:interface elements. #1145

  • Bundle start fails due to OSGiScriptEngineManager. #1136

  • Apparent memory leak when using near cache. #1087

  • Semaphore can be initialized multiple times if permit-count = 0. #1080

  • Exception when running continuous query. And non-existent key is deleted under transaction. #1076

  • Race condition between first map operation and re-populating a map after a cluster restart. #1070

  • Eviction tasks should be cancelled on remove(), evict(). #1068

  • Stale cache issues when using HazelcastLocalCacheRegionFactory. #1039

  • When multiple nodes start simultaneously, post join process duplicates some event registrations and causes warning logs on each publish. #1024

  • Multimap entry listener is called twice. #993

  • Clear logic on ByteArrayObjectDataOutput can create memory leak with large objects. #989

  • TransactionException: Lock is not owned by the transaction. #988

  • Spring schema declaration is missing. #982

  • Null object indexing exception. #978

  • Entries not changed by an EntryProcessor should not render an EntryListener event. #969

  • Unlock parameter in TxnSetOperation backup operation is not serialized. #956

  • PartitioningStrategy is not set in HazelcastClient. #923

  • Enhancement request: Consider extending SqlPredicate to allow using regexes. #914

  • Map getAll() is blocked with ``Map is not ready exception''. #887

  • QueueStoreConfig needs a factory to support wildcard creation. #884

  • Data is lost when transaction manager node terminates/crashes without preparing transaction. #863

  • Make cache-null-value-seconds customizable. #806

  • Reintroduce the Instanceof predicate. #790

  • Add IMap.addEntryListener() without key (continuous query). #710

  • The calculation used_heap_percentage should take backups into account. #403

3.0.3

This section lists the new features, enhancements and fixed issues for 3.0 and 3.0.x releases.

New Features

This section provides the new features introduced with Hazelcast 3.0 release.

  • Multi-thread execution: Operations are now executed by multiple threads (by factor of processor cores). With Hazelcast 2, there was a only single thread.

  • SPI: Service Programming Interface for developing new partitioned services, data structures. All Hazelcast data structures like Map, Queue are reimplemented with SPI.

  • IdentifiedDataSerializable: A slightly optimized version of DataSerializable that does not use class name and reflection for de-serialization.

  • Portable Serialization: Another Serialization interface that does not use reflection and can navigate through binary data and fetch/query/index individual field without having any reflection or whole object de-serialization.

  • Custom Serialization: Support for custom serialization that can be plugged into Hazelcast.

  • Entry Processor: Executing an EntryProcessor on the key or on all entries. Hazelcast implicitly locks the entree and guarantees no migration while the execution of the Processor.

  • In-Memory Format: Support for storing entries in Binary, Object and Cached format.

  • Continuous Query: Support for listeners that register with a query and are notified when there is a change on the Map that matches the Query.

  • Interceptors: Ability to intercept the Map operation before/after it is actually executed.

  • Lazy Indexing:Ability to index existing items in the map. No need to add indexes at the very beginning.

  • Queue: No more dependency on the distributed map

  • Queue: Scales really well as you have thousands of separate queues.

  • Queue: Persistence Support for persistence with QueueStore.

  • Multimap: Values can be Set/List/Queue.

  • Topic: Support for global ordering where all Nodes receive all messages in the same order.

  • Distributed Transaction: Support for both 1-phase (local) and 2 phase transactions with a totally new API.

  • New Binary Protocol: A new binary protocol based on portable serialization. The same protocol is used for Java/C/C# and other clients.

  • Smart client: Support for dummy and smart client. A dummy client will maintain a connection to only one member, whereas the smart client can route the operations to the node that owns the data.

Fixes

This section lists issues solved for 3.0 and 3.0.x releases.

  • A class cannot be LifecycleListener and MigrationListener at the same time. #970

  • EntryProcessor does not fire entry events. #950

  • IMap.localKeySet() is very slow after upgrade to 3.0. #946

  • Eviction logic error for USED_HEAP_PERCENTAGE and USED_HEAP_SIZE. #891

  • Failed to execute SqlPredicate with LIKE and OR keywords on the same field of the class. #885

  • Infinite loop in native client when cluster is gone. #821

  • TransactionalMap: DistributedObjectListener is not working. #812

  • Could not find PortableFactory for factoryId: -3. #800

  • Eviction occurs when it is not expected to occur. #776

  • Possible dead-lock while updating an entry which loaded by map-loader. #768

  • IMap.destroy() clears map instead of destroy. #764

  • IMap.destroy() throws an exception on client. #726

  • Client replace method issue. #724

  • Classloader issue in deserialization. #723

  • JMX preDeregister of some beans throws HazelcastIsNotActiveException during shutdownAll(). #713

  • Optional sequencing support for Hazelcast write-behind implementation is needed. #712

  • ManagementService.destroy fails to find HazelcastMBean on shutdown. #683

  • Client getDistributedObject() does not work. #678

  • Spring ManagedContext does not run on the local node when a task is executed. #659

  • MemberLeftException during SqlPredicate search. #654

  • NearCacheConfig: maxSize = 0 is no longer interpreted as Integer.MAX_VALUE. #650

  • statistics-enabled is missing for most items in the XSD scheme. #642

  • Could not find PortableFactory for factoryId: 1095521605 on SqlPredicate query. #617

  • Map.containsKey() should delay eviction. #614

  • Error when serializing portable nested inside other types: Could not find PortableFactory for factoryId: 0. #600

  • Client related configuration improvements. #599

  • Index on enum properties does not work. #597

  • Executing Query operation with a heavy CRUD load throws an IllegalArgumentException. #596

  • Map.set() always calls the onAdded entry event. #589

  • Client throws `Could not register listener' often - Near Cache entries get never invalidated after. #584

  • Unexpected authentication exception. #576

  • Map clear does not call MapStore. #572

  • Transactional collections should check the transaction state to validate proper access. #568

  • Client threads are not destroyed on authentication failures. #555

  • Map index from configuration is not working. #553

  • Client disconnects and never tries to reconnect. #538

  • Map local entry listener gives no Value on entryEvicted. #537

  • The method map.getAll() is not working if entry does not exist via Client. #535

  • EC2 Auto discovery bug: Instances do not see each other. #516

  • Custom types (other than primitives, string, date, enum, etc) can not be queried. #503

  • OOME is thrown when queues are not explicitly destroyed. #417

  • The method loadAll is called redundantly on a new node joining the cluster. #341

  • Support for Hibernate 4.0 cache is needed. #72

  • Add a functionality that performs entry processings. #71

  • Support for JVM system property reference in Hazelcast XML is needed. #59

  • Support invalidation-only 2nd level cache for Hibernate. #57

  • Hazelcast resource adapter does not work on WebSphere 6.1/7. #37

Contributors

We would like to thank the contributors from our open source community who worked on this release:

2.6.9

The following is the fixed issue for 2.6.9 release.

  • Hazelcast cannot match hostnames when wildcards are used in interfaces configuration element. #2396

2.6.8

The following is the fixed issue for 2.6.8 release.

  • OperationTimeoutException: [CONCURRENT_MAP_SET] Redo threshold[90] exceeded! Last redo cause: REDO_MAP_OVER_CAPACITY. #1685

2.6.6

The following are the fixed issues for 2.6.6 release.

  • Hazelcast 2.6.5 throws NullPointerException if the method MapStore.load() returns null during IMap.getAll() execution. #1605

  • In Hazelcast 2.x branch, Hazelcast instance is shutdown but exception is not logged. #1488

2.6.3

The following are the fixed issues for 2.6.3 release.

  • Memcache client content in the Reference Manual needs to be improved . #1047

  • Hazelcast Hibernate pom is using a retired Maven repo. #831

  • "Marked for removal" mechanism should be more clarified. #792

2.6.2

The following are the fixed issues for 2.6.2 release.

  • Client Listener: the Client argument has null Socketaddress for ClientDisconnected listener. #756

  • Entries put by the method IMap.putAll were never evicted from the map with the TTL configured. #670

  • The Java client is initialized with two cluster members. After killing one member in the cluster where it is connected, the client is unable to reconnect. #653

  • Put operation after a missed get operation in a transaction does not have any effect. #644

  • Near cache functionality does not work for Java client. MapConfig always returns the generic map name since in the client, the client prefix is added to map’s name. #620

Contributors

We would like to thank the contributors from our open source community who worked on this release:

2.6.1

The following are the fixed issues and enhancements for 2.6.1 release.

  • The method set(Key, Value, 0, Timeunit) applies the configured TTL values where it should cancel eviction. Also, the method set(Key, Value) should apply the inherited TTL if exists. #585

  • Operations like contains() should use a specific executor service. Calling the default executor service may cause a deadlock. #579

  • Empty array for the predicate in throws exception. #573

  • The methods ILock.destroy() and ISemaphore.destroy() throw exceptions on Hazelcast 2.5. #566

  • NullPointerException in com.hazelcast.nio.ConnectionManager in Hazelcast 2.5. #530

  • Hazelcast instance does not start with TCP configuration in Hazelcast 2.5 when the hosts are not reachable. #523

  • Hazelcast source archives in Maven central repository does not contain the Java source files but the class files. #514

  • There is an unexpected store call behavior in Hazelcast 2.6. Every time when the method get() is used in a transaction and when the method commit() is called, it throws ``Duplicated key order_id'' exception. #506

  • When the method getInstances() on HazelcastInstance is called (acquired via HazelcastClient.newHazelcastClient(clientConfig)), the collection is retrieved. After some time, NullPointerException is thrown when the same method is called. #478

  • There are some formatting problems when using SQL predicates with date. #473

  • There is a memory leak in the member when clients are connected. An OutofMemoryException is thrown related to a client authentication failure. #450

  • Hazelcast client’s putAll() method throws NullPointerException when called with an empty map. #397

  • The InputStream in the XmlConfigBuilder can never be closed. If the constructor which provides an XML file is used, it creates a new FileInputStream. There is no ability to close this input stream after it is created. #390

  • The heartbeat timer for Hazelcast client does not run again after a NullPointerException. #382

  • The queue SysmteLogService.joinLogs holds some data and causes memory consumption. #325

  • Doing a read of a value not modified in a transaction causes the listener to fire an update for that entry. #311

2.6

The following are the fixed issues and enhancements for 2.6 release.

  • Issues with Spring configuration and merge policies. #488

  • Trying to create a configuration object by API, serializing and deserializing the objects (by Hazelcast builder), due to an error of PartitionGroup, RuntimeException is thrown. If set by API, it works. #487

  • OSGi: The manifest data in hazelcast-all.jar for 2.5.1 is not correct. #484

  • Consistency of the set of members in the cluster should be guaranteed. #477

  • Hazelcast IMap’s `containsKey()`method does not reset the idle timeout. #472

  • @SpringAware annotation leads to memory leak when using Hazelcast distributed executor service. #470

Contributors

We would like to thank the contributors from our open source community who worked on this release:

2.5.1

The following are the fixed issues and enhancements for 2.5.1 release.

  • Deadlock happens when Hazelcast client is shut down. #466

  • There is a data inconsistency and loss following the ClassCastException: CONCURRENT_MAP_REPLACE_IF_SAME. #462

  • Put operations with zero TTL does not prevent the eviction. #455

  • NullPointerException is thrown at com.hazelcast.query.MapIndexService.remove during CONCURRENT_MAP_REMOVE operation. #454

  • The exception Failed migrating from Member is thrown when two clients are started in Eclipse and one is stopped before it is completely initialized. #452

  • The method IList.destroy() does not empty the list. #449

  • The package hazelcast-all.jar should build its manifest dynamically. #448

  • The package hazelcast-client.jar does not deploy as OSGi bundle in JBoss7. #447

  • The package hazelcast-wm.jar should not require servlet-api v3.0. #432

  • NullPointerException is thrown when trying to run the method clear() on an empty distributed set when migrating from Hazelcast 2.1.2 to 2.4 and 2.5 #430

  • The method getOldValue() consistently returns wrong results for entryUpdated listener notifications after the replace() operation. #418

  • In the Hazelcast configuration schema XSD, the tcp-ip and aws elements do not have conn-timeout-seconds sub-element. #410

  • It would be nice if the hazelcast.config option can be used in combination with the classpath. #408

  • Deadlock happens when adding an index to a map that is not empty. #310

  • When custom loader is used with write-behind persistence mode, every time store() is called in loader and then remove() is called on locked map, value for the key is read from the store and not from map. Consequently, put/set/update is overridden and data is lost. #187

Contributors

We would like to thank the contributors from our open source community who worked on this release:

2.5

This section lists the new features, enhancements and fixed issues for 2.5 release.

New Features

The following are the new features introduced with 2.5 release.

  • Added near cache support for Java Client.

  • Management Center alert system can now receive alerts based on custom filters.

  • Management Center has now better support for Hazelcast cluster running in OSGI environment.

  • Nodes can be easily shutdown or restarted using Management Center interface.

Fixes

The following are the fixed issues and enhancements for 2.5 release.

  • Management Center does not respond for the maps with names having underscore characters. #394

  • MultiTask operation returns Constants$Object$6 instead of a map. #392

  • In the method forceUnlock(), local lock is not removed. #386

2.4.1

This section lists the new features, enhancements and fixed issues for 2.4.1 release.

New Features

The following are the new features introduced with 2.4.1 release.

  • Added Hibernate 2nd level cache local/invalidation mode.

  • Added quick clear feature for maps.

Fixes

The following are the fixed issues and enhancements for 2.4.1 release.

  • After a split-brain syndrome, states sent by a non-member should be ignored. #374

  • Indexing: Negative double values comparison causes problems in SQL predicate. #368

  • Query on Hazelcast IMap with indexes returns stale data. #359

  • By default, the method map.clear() clears the map entry by entry since the entries should be checked if they have a related lock, should be persisted, etc. If you do not use map store and can assume that there is no locked entries, then there should be an option to clear the map directly. #356

  • System logs should be optional. #351

  • Management center should show the Hazelcast version. #349

  • UnsupportedOperationException is thrown when the method getConfig() is called during the startup of the web application. Used versions are Hazelcast 2.4 and Hibernate 3.6.6. #339

  • Documentation is wrong. Hazelcast supports REPEATABLE_READ instead of READ_COMMITTED. #337

  • When you need a lite member, since it cannot be specified in the configuration file, you need to do the whole loading mechanism yourself. #333

  • In the local map statistics, currently there is only getHits(). There should be statistics for misses (get requests which return null) to see hit/miss ratio. #328

  • When Hazelcast OSGi bundle is used in Glassfish 3.1.2, ClassNotFoundException is thrown. #322

  • The method MapConfig.toString() should report mapIndexConfig too. #321

  • When the connection between the client and member is lost, the client does not notice it and tries to send the data for a very long time and it does not shut down. #315

  • Hazelcast tends to consume large amounts of memory in com.hazelcast.nio.Packet instances. #312

  • Map index entries are not deleted upon migration. When using SQL predicate in the method localKeySet(), the same key can be retrieved on different nodes. #308

  • Hazelcast IMap entries are evicted even they are touched by the method put(). #466

2.4

This section lists the new features, enhancements and fixed issues for 2.4 release.

New Features

The following are the new features introduced with 2.4 release.

  • Client threads does not have fixed size anymore, now it uses internal Hazelcast cached thread pool.

  • Added ability to restrict outbound ports that Hazelcast uses to connect to other nodes.

Fixes

The following are the fixed issues and enhancements for 2.4 release.

  • Hazelcast Management Center’s Configuration has been successfully updated notification is never removed. #301

  • Map maximum size does not take into account that some cluster members are lite members and cannot hold any data. Thus, the map becomes smaller than the size it was configured. #292

  • The method map.containsKey() does not reset the idle time counter. #288

  • In Hazelcast 2.3.1, the property hazelcast.local.localAddress is used only when provided directly from System.properties. #282

  • Data removed under transaction sometimes could not be removed from the backup. #277

  • Map initialization should not use the default executor service. #276

  • In Hazelcast 2.3.1, Hazelcast got into a state where the list of members on some of the nodes do not match up. #274

  • Map EntryListener is not working correctly if one master is restarted. #269

  • IMap.tryLockAndGet: There is a concurrency error when the map has map store. #268

  • Error in com.hazelcast.query.Predicates.NotEqualPredicate. #262

  • Partition Group with group type CUSTOM should honor configured backups. #260

  • Messages are lost while moving them inside a transaction. #259

  • Enable Hazelcast to use specified port ranges to accommodate firewall policies. #251

  • Seeing multiple concurrent locks on a distributed lock. #168

Contributors

We would like to thank the contributors from our open source community who worked on this release:

2.3.1

The following are the fixed issues and enhancements for 2.3.1 release.

  • Changed hazelcast.partition.migration.interval property’s value from 1 to 0.

  • The method ILock.newCondition() now throws UnsupportedOperationException.

  • After upgrading Hazelcast to 2.3, IPv6 is no longer supported in TCP join mechanism. #258

  • ClassCastException is thrown when the method MultiMap.get() is run. #256

  • Currently, the method MultiMap.get(K key) returns null if there are no values associated with the key. However, it should return an empty collection instead. #167

2.3

This section lists the new features, enhancements and fixed issues for 2.3 release.

New Features and Changes

The following are the new features and changes introduced with 2.3 release.

  • Changed hazelcast.max.operation.timeout unit from seconds to milliseconds.

  • Added hazelcast.max.concurrent.operation.limit property to be able to limit the number of concurrent operations that can be submitted to Hazelcast.

  • Added hazelcast.backup.redo.enabled property to enable/disable redo for backup operations.

  • Added MultiMap and Distributed ExecutorService statistics to Management Center application.

  • MigrationListener has now an additional method to receive failed migration events; void migrationFailed(MigrationEvent migrationEvent).

  • ItemEvent has now an additional method returning Member firing that event; public Member getMember().

  • Improved out of memory (OOM) error detection and handling. Now it is possible to register a custom hook when OutOfMemoryError is thrown.

  • Fixed some issues related to domain name handling and networking/join.

  • During cluster merge after a network split-brain, merging side will now fire MERGING and MERGED before and after RESTARTING and RESTARTED LifecycleEvents.

Fixes

The following are the fixed issues and enhancements for 2.3 release.

  • Operation exceptions occur randomly while the SQL predicates are stress tested. #263

  • EOF exception is thrown in Management Center. #252

  • The method IMap.flush() delegates map store exceptions to the caller instead of suppressing them. #250

  • Node appears to be in multiple clusters. #247

  • It would be nice to have Spring support in Runnable tasks. #244

  • MERGING and MERGED events should be fired during a split-brain merging process. #241

  • A problem occurs during a quick restart of master node. #235

  • Distributed tryLock() throws NullPointerException from lite member. #233

  • Using domain instead of IP address is problematic. #230

  • The method MultiMap.lock() blocks forever. #228

  • SimpleMapTest uses deprecated method Hazelcast.getLoggingService() and annoys WebLogic 12. #227

  • ItemEvent should have a getMember method to return the member that offered/polled an item. #226

  • The consumers do queue.take() and block until something is put on the queue. In Hazelcast 2.1.2, this was fine, but in 2.2, it causes a massive amount of WARN-level log spam. #225

  • The method MultiMap.put() is inconsistent when the key is locked by another node/thread. #223

  • FAILED event should be fired when migration fails for some reason. #222

  • Runtime exception is thrown when accessing to queue after client restarted. #220

  • Using the client, once it tries to access the cache while the cache is unavailable, the client cannot be used anymore after the server cache becomes available. #218

  • The methods DistributedTask.get() and MultiTask.get() throw OperationTimeoutException. #217

  • There are a couple of classes that do not implement toString, that makes the Config.toString message incomplete. #209

  • Hazelcast IMap ignores the size capacity. #188

  • Hazelcast resource adapter package (version 2.1.2) does not deploy on JBoss 7.1.1. #182

Contributors

We would like to thank the contributors from our open source community who worked on this release:

2.2

This section lists the new features, enhancements and fixed issues for 2.2 release.

New Features and Changes

The following are the new features and changes introduced with 2.2 release.

  • Improved redo logs and added maximum call/operation timeout.

  • Improved domain name handling; Hazelcast will use defined addresses/domain-names in TCP-IP Config as they are, without resolving an IP address.

  • Added Cluster Health Check to Management Center application.

Fixes

The following are the fixed issues and enhancements for 2.2 release.

  • Hazelcast IMDG Reference Manual shows a default config for WAN replication that includes the config item <replication-impl>com.hazelcast.impl.wan.WanNoDelayReplication</replication-impl>. This item appears to be used for configuring WAN replication implementation to be used. This item is not being used at all. Instead the replication implementation is hard coded in WanReplicationService.java. #152

  • It would be nice to have possibilities to instantiate native client from java.util.Properties and/or from classpath-located configuration file. #93

  • The method ILock.isLocked() should have the ability to implement a spin lock / wait for a lock release to perform non-blocking code. #39

2.1.3

The following are the fixed issues and enhancements for 2.1.3 release.

  • Hazelcast IList evicts the items using the default map TTL. #196

  • Nodes with EC2 auto-discovery do not rejoin the cluster after a network outage. #195

  • The method remove() cannot be called on a Hazelcast Set iterator. #189

  • Topic does not receive messages when using Spring injection. #186

  • The method IMap.set() should not load data from map store. #185

  • EntryListener for native clients receives duplicate events. #183

  • Listeners do not work when used with locks and remove. #181

  • The package hazelcast-spring dependencies on Hibernate and MongoDB should be optional. #179

  • Backup entry count is smaller than the owned entries. #177

  • Queries on enum indexed types return an empty result set. #176

2.1.2

The following are the fixed issues and enhancements for 2.1.2 release.

  • The method containsKey() does not work on a single node with the default near cache configuration. #174

  • The method Transactional IMap.get(key) causes the key to be added to map. #161

2.1.1

The following are the fixed issues and enhancements for 2.1.1 release.

  • Item listener does not work when using client. #158

  • The method IMap.remove(key) returns incorrect object. #149

  • Map eviction policies are not documented. #148

  • When calling the method IQueue.remove(object), the item listener added for that queue is not invoked. #146

  • List is not keeping the order in transactional context. #73

2.1

This section lists the new features, enhancements and fixed issues for 2.1 release.

New Features and Changes

The following are the new features and changes introduced with 2.1 release.

  • Hazelcast now supports IPv6 addresses seamlessly.

  • Added async backup support.

  • Hazelcast now can be used as Spring Cache provider.

  • Spring Dependency Injection support: Hazelcast can apply bean properties or to apply factory callbacks such as ApplicationContextAware, BeanNameAware or to apply bean post-processing such as InitializingBean, @PostConstruct like annotations while using Hazelcast distributed ExecutorService or DistributedTasks or more generally any Hazelcast managed object.

  • Added persistence support with Spring-Data MongoDB and JPA integration.

  • Member.getUuid() now will return UUID for node.

  • Improved session clustering configuration.

Fixes

The following are the fixed issues and enhancements for 2.1 release.

  • Hazelcast client breaks the lifecycle listeners. #130

  • The Spring bean attributes lazy-init and scope should be added. #118

  • Spring configuration namespace is missing lock. #116

  • MaxSizeHeapPolicy issue causing improper evictions and freezes. #110

  • NullPointerException is thrown by the method tx.commit(). #108

  • Hazelcast client does not shutdown properly if no connection can be established. #101

  • It would be nice to have ``semi-asynchronous'' replication for Hazelcast. #92

2.0.4

The following are the fixed issues and enhancements for 2.0.4 release.

  • NullPointerException is thrown by the method AbstractRecord.getLockCount(). #166

  • There are ``Connection refused'' in Hazelcast 2.0.3. #140

  • Predicate value on the date field should not be converted from a String by the method getRealObject(). #135

  • Some IMap removed/evicted entries will go back when one cluster node crashes. #132

  • Predicates.GreaterLesserPredicate’s doApply operation is broken. #131

  • Incorrect selection by predicate with comparison of non-numerical fields and indexing problems. #98

  • Evicted/deleted entries should not be stored to the map store. #96

2.0.3

The following are the fixed issues and enhancements for 2.0.3 release.

  • Increasing amount of stuck Hazelcast threads in weblogic; in a clustered environment and also when running a single instance. #160

  • Hyphens in the group names confuse the Management Center. It treats the group name as two separate tokens. #128

  • Split-brain merge handling causes the locks never to be acquired again. #127

  • ConfigXmlGenerator does not handle the TTL correctly when multicast joining mechanism is used. #119

  • Data is lost in a map when adding a key the second time after a new member joins the cluster. #117

  • Listeners are not called when a queue is changed within a transaction. #114

  • Programmatic session destruction does not lead to the destruction of the clustered session. #104

  • The name for hazelcast.sessionId should be configurable. #103

  • There is an issue in concurrent queue.take() and txn.rollback(). #99

Contributors

We would like to thank the contributors from our open source community who worked on this release:

2.0

This section lists the new features, enhancements and fixed issues for 2.0 release.

New Features and Changes

The following are the new features and changes introduced with 2.0 release.

  • New Elastic Memory(Enterprise Edition Only): By default, Hazelcast stores your distributed data (map entries, queue items) into Java heap which is subject to garbage collection. As your heap gets bigger, garbage collection might cause your application to pause tens of seconds, badly effecting your application performance and response times. Elastic Memory is Hazelcast with off-heap memory storage to avoid GC pauses. Even if you have terabytes of cache in-memory with lots of updates, GC will have almost no effect; resulting in more predictable latency and throughput.

  • Security Framework(Enterprise Edition Only): Hazelcast Security is JAAS based pluggable security framework which can be used to authenticate both cluster members and clients and do access control checks on client operations. With the security framework, take control of who can be part of the cluster or connect as client and which operations are allowed or not.

  • Native C# Client(Enterprise Edition Only): Just like our Native Java Client, it supports all map, multimap, queue, topic operations including listeners and queries.

  • Distributed Backups: Data owned by a member will be evenly backed up by all the other members. In other word, every member takes equal responsibility to backup every other member. This leads to better memory usage and less influence in the cluster when you add/remove members. The new backup system makes it possible to form backup-groups so that you can have backups and owners fall into different groups.

  • Parallel IO: Number of socket selector threads can be configured. You can have more IO threads, if you have good number of CPU/cores and high-throughput network.

  • Connection Management: Hazelcast 2.0 is more tolerant to connection failures. On connection failure it tries to repair it before declaring the member as dead. So now it is ok to have short socket disconnections. No problem if your virtual server migrates to a new host.

  • Listeners such as migration, membership and map indexes can be added with configuration.

  • New Event Objects: Event Listeners for Queue/List/Set/Topic were delivering the item itself on event methods. That’s why the items had to be deserialized by Hazelcast Threads before invoking the listeners. Sometimes this was causing class loader problems too. With 2.0, we have introduced new event containers for Queue/List/Set and Topic just like Map has EntryEvent. The new listeners now receive ItemEvent and Message objects respectively. The actual items are deserialized only if you call the appropriate get method on the event objects. This is where we brake the compatibility with the older versions of Hazelcast.

  • ClientConfig API: We had too many of factory methods to instantiate a HazelcastClient. Now all we need isHazelcastClient.newHazelcastClient(ClientConfig).

  • SSL communication support among cluster members.

  • Distributed MultiMap value collection can be either List or Set.

  • SuperClient is renamed to LiteMember to avoid confusion. Be careful! It is a member, not a client.

  • New IMap.set(key, value, ttl, TimeUnit) implementation, which is optimized put(key, value) operation as Set does not return the old value.

  • HazelcastInstance.getLifecycleService().kill() will forcefully kill the member. Useful for testing.

  • forceUnlock, to unlock the locked entry from any member and any thread regardless of the owner.

  • Enum type query support: As an example; new SqlPredicate (“level = Level.WARNING”)

Fixes

The following are the fixed issues and enhancements for 2.0 release.

  • [459], [471], [567], [574], [582], [629], [632], [646], [666], [686], [669], [690], [692], [693], [695], [698], [705], [708], [711], [714], [715], [719], [721], [722], [727], [728], [729], [730], [731], [732], [733], [735], [738], [739], [740], [741], [742], [747], [751], [752], [758], [759], [760], [761], [765], [767], [770], [773], [779], [781],

1.9.4.9

This section lists the new features, enhancements and fixed issues for the releases starting from 1.9.4.9 to 1.0.

  • Added WAN Replication (synchronization of separate active clusters).

  • Added Data Affinity (co-location of related entries) feature.

  • Added EC2 Auto Discovery for your Hazelcast cluster running on Amazon EC2 platform.

  • Implemented Distributed CountDownLatch.

  • Implemented Distributed Semaphore implementation.

  • Hazelcast distribution now contains HTML and PDF documentation besides Javadoc.

  • Better TCP/IP and multicast join support. Handling more edge cases like multiple nodes starting at the same time.

  • Memcache protocol: Better integration between Java and Memcache clients. Put from memcache, get from Java client.

  • Monitoring Tool is removed from the project.

  • Re-implementation of distributed queue:

    • Configurable backup count and synchronous backup.

    • Persistence support based on backing MapStore.

    • Auto-recovery from backing MapStore on startup.

  • Re-implementation of distributed list supporting index based operations.

  • Optimized IMap.putAll for much faster bulk writes.

  • Added IMap.getAll for bulk reads which is calling MapLoader.loadAll if necessary.

  • Added IMap.tryLockAndGet and IMap.putAndUnlock methods.

  • Added IMap.putTransient API for storing only in-memory.

  • Added IMap.addLocalEntryListener() for listening locally owned entry events.

  • Added IMap.flush() for flushing the dirty entries into MapStore.

  • Added MapLoader.getAllKeys API for auto-pre-populating the map when cluster starts.

  • Support for minimum initial cluster size to enable equally partitioned start.

  • Introduced graceful shutdown.

  • Faster dead-member detection.

  • Memcache interface support. Memcache clients written in any language can access Hazelcast cluster.

  • RESTful access support, e.g., http://:5701/hazelcast/rest/maps/mymap/key1.

  • Added split-brain (network partitioning) handling.

  • Added LifecycleService API to restart, pause Hazelcast instances and listen for the lifecycle events.

  • Added asynchronous put and get support for IMap via IMap.asyncPut() and IMap.asyncGet().

  • Added AtomicNumber API; distributed implementation of java.util.concurrent.atomic.AtomicLong.

  • Significant performance gain for multi-core servers. Higher CPU utilization and lower latency.

  • Reduced the cost of map entries by 50 percent.

  • Better thread management. No more idle threads.

  • Added queue statistics API and the queue statistics panel on the Monitoring Tool.

  • Monitoring Tool enhancements. More responsive and robust.

  • Hazelcast distribution now contains hazelcast-all-<version>.jar to simplify the JAR dependency.

  • Sorted index optimization for map queries.

  • Added Hazelcast Cluster Monitoring Tool.

  • Added Partition API. Partition and key owner, migration listeners.

  • Added IMap.lockMap() method.

  • Added Multicast and TCP/IP join feature. Try multicast first, if not found, try TCP/IP.

  • Added Hazelcast.getExecutorService(name) API. You can have separate named executor services. Do not let your big tasks blocking your small ones.

  • Added Logging API. Build your own logging. or simply use Log4j or get logs as LogEvents.

  • Added MapStatistics API. Get statistics for your Map operations and entries.

  • Hazelcast client now automatically updates the member list. There is no need to pass the list to all members.

  • Added the ability to start the cluster members evenly partitioned. Hence, no migration.

  • Added Java clients for accessing the cluster remotely.

  • Added Distributed Query for maps. Both Criteria API and SQL are supported.

  • Added near cache feature for distributed maps.

  • Added TTL (time-to-live) property for each individual map entry.

  • Improved the put operation: IMap.put(key,value, ttl, timeunit).

  • Introduced the method IMap.putIfAbsent(key,value, ttl, timeunit).

  • Now, you can have multiple Hazelcast members on the same JVM. Introduced HazelcastInstance API.

  • Better API based configuration support.

  • Smoother data migration enabling better response times during joins.

  • Persistence via Loader/Store interface for distributed map.

  • Added Socket level encryption feature. Both symmetric and asymmetric encryption are supported.

  • Added support for JMX.

  • Added support for Hibernate second level cache provider.

  • Added instance events for getting notified when a data structure instance (map, queue, topic, etc.) is created or destroyed.

  • Added eviction listener: EntryListener.entryEvicted(EntryEvent).

  • Hazelcast is now fully Mavenized.

  • Added support for synchronous backups and configurable backup-count for maps.

  • Added eviction support: Timed eviction for queues. LRU, LFU and time based eviction for maps.

  • Added support for statistics/history for entries: create/update time, number of hits, cost.

  • Implemented MultiMap structure. Similar to google-collections and apache-common-collections, but distributed and thread-safe.

  • Now, you can destroy() the data structures when not needed anymore.

  • Now, you can shutdown the local member using Hazelcast.shutdown().

  • Now, you can get the list of all data structure instances via Hazelcast.getInstances().

  • Full implementation of java.util.concurrent.BlockingQueue. Now, queues can have configurable capacity limits.

  • Introduced Super Clients (a.k.a LiteMember): Members with no storage. If -Dhazelcast.super.client=true JVM parameter is set, that JVM will join the cluster as a `super client' which will not be a `data partition' (no data on that node) but will have super fast access to the cluster just like any regular member does.

  • Added HTTP Session sharing support for Hazelcast Web Manager. Different web applications can share the same sessions.

  • Added the ability to separate clusters by creating groups.

  • Added java.util.logging support.

  • Added the support for adding, removing and updating events for queue, map, set and list data structures.

  • Introduced Distributed Topic for pub/sub messaging.

  • Added integration with J2EE transactions via JCA complaint resource adapter.

  • Added ExecutionCallback interface for distributed tasks.

  • Introduced cluster-wide unique ID generator.

  • Implemented Transactional Distributed Queue, Map, Set and List.

  • Implemented Distributed Executor Service.

  • Added support for multi member executions.

  • Implemented key based execution routing.

  • Added task cancellation support.

  • Implemented Session Clustering with Hazelcast Webapp Manager.

  • Added full TCP/IP clustering support.

  • Introduced distributed implementation of java.util.{Queue,Map,Set,List}.

  • Introduced distributed implementation of java.util.concurrency.Lock.

  • Added support for retrieving cluster membership events.

  • 1000+ commits 100+ bug fixes and several other enhancements.

Take our Survey!