Package | Description |
---|---|
com.hazelcast.connector |
Hazelcast 3 connector provides sources, sinks and enrichment using data
in Hazelcast 3 remote cluster
|
com.hazelcast.jet.avro |
Apache Avro file read/write support for Hazelcast Jet.
|
com.hazelcast.jet.cdc |
Contains source/sink connectors that deal with Change Data Capture (CDC)
events from various databases as well as a generic connector for Debezium
CDC sources.
|
com.hazelcast.jet.elastic |
Contains sources and sinks for Elasticsearch 7
|
com.hazelcast.jet.hadoop |
Apache Hadoop read/write support for Hazelcast Jet.
|
com.hazelcast.jet.kafka |
Apache Kafka reader/writer support for Hazelcast Jet.
|
com.hazelcast.jet.kinesis |
Amazon Kinesis Data Streams producer/consumer support for Hazelcast Jet.
|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.pipeline.test |
This package contains various mock sources to help with pipeline testing
and development.
|
com.hazelcast.jet.s3 |
AWS S3 read/write support for Hazelcast Jet.
|
Modifier and Type | Method and Description |
---|---|
static <T,K,V> Sink<T> |
Hz3Sinks.remoteMap(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn,
String clientXml)
Returns a sink that uses the supplied functions to extract the key
and value with which to put to a Hazelcast
IMap with the
specified name. |
static <K,V> Sink<Map.Entry<K,V>> |
Hz3Sinks.remoteMap(String mapName,
String clientXml)
Returns a sink that puts
Map.Entry s it receives into a Hazelcast
IMap with the specified name. |
Modifier and Type | Method and Description |
---|---|
static <R> Sink<R> |
AvroSinks.files(String directoryName,
Class<R> recordClass,
org.apache.avro.Schema schema)
Convenience for
AvroSinks.files(String, Schema,
SupplierEx) which uses either SpecificDatumWriter or
ReflectDatumWriter depending on the supplied recordClass . |
static Sink<org.apache.avro.generic.IndexedRecord> |
AvroSinks.files(String directoryName,
org.apache.avro.Schema schema)
Convenience for
AvroSinks.files(String, Schema,
SupplierEx) which uses GenericDatumWriter . |
static <R> Sink<R> |
AvroSinks.files(String directoryName,
org.apache.avro.Schema schema,
SupplierEx<org.apache.avro.io.DatumWriter<R>> datumWriterSupplier)
Returns a sink that that writes the items it receives to Apache Avro
files.
|
Modifier and Type | Method and Description |
---|---|
static <K,V> Sink<ChangeRecord> |
CdcSinks.map(IMap<? super K,? super V> map,
FunctionEx<? super ChangeRecord,? extends K> keyFn,
FunctionEx<? super ChangeRecord,? extends V> valueFn)
Returns a sink that applies the changes described by a Change Data
Capture (CDC) stream to an
IMap . |
static <K,V> Sink<ChangeRecord> |
CdcSinks.map(String mapName,
FunctionEx<? super ChangeRecord,? extends K> keyFn,
FunctionEx<? super ChangeRecord,? extends V> valueFn)
Returns a sink that applies the changes described by a Change Data
Capture (CDC) stream to an
IMap . |
static <K,V> Sink<ChangeRecord> |
CdcSinks.remoteMap(String mapName,
ClientConfig clientConfig,
FunctionEx<? super ChangeRecord,? extends K> keyFn,
FunctionEx<? super ChangeRecord,? extends V> valueFn)
Returns a sink equivalent to
CdcSinks.map(java.lang.String, com.hazelcast.function.FunctionEx<? super com.hazelcast.jet.cdc.ChangeRecord, ? extends K>, com.hazelcast.function.FunctionEx<? super com.hazelcast.jet.cdc.ChangeRecord, ? extends V>) , but for a map in a remote
Hazelcast cluster identified by the supplied ClientConfig . |
Modifier and Type | Method and Description |
---|---|
Sink<T> |
ElasticSinkBuilder.build()
Create a sink that writes data into Elasticsearch based on this builder configuration
|
static <T> Sink<T> |
ElasticSinks.elastic(FunctionEx<? super T,? extends org.elasticsearch.action.DocWriteRequest<?>> mapToRequestFn)
Creates an Elasticsearch sink, uses a local instance of Elasticsearch
|
static <T> Sink<T> |
ElasticSinks.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn,
FunctionEx<? super T,? extends org.elasticsearch.action.DocWriteRequest<?>> mapToRequestFn)
Creates an Elasticsearch sink, uses a client obtained from
clientFn and maps items using given mapToRequestFn
|
Modifier and Type | Method and Description |
---|---|
static <K,V> Sink<Map.Entry<K,V>> |
HadoopSinks.outputFormat(org.apache.hadoop.conf.Configuration configuration)
Convenience for
HadoopSinks.outputFormat(Configuration, FunctionEx,
FunctionEx) which expects Map.Entry<K, V> as
input and extracts its key and value parts to be written to HDFS. |
static <E,K,V> Sink<E> |
HadoopSinks.outputFormat(org.apache.hadoop.conf.Configuration configuration,
FunctionEx<? super E,K> extractKeyF,
FunctionEx<? super E,V> extractValueF)
Returns a sink that writes to Apache Hadoop HDFS.
|
Modifier and Type | Method and Description |
---|---|
Sink<E> |
KafkaSinks.Builder.build()
Builds the Sink object that you pass to the
GeneralStage.writeTo(Sink) method. |
static <E,K,V> Sink<E> |
KafkaSinks.kafka(Properties properties,
FunctionEx<? super E,org.apache.kafka.clients.producer.ProducerRecord<K,V>> toRecordFn)
Returns a sink that publishes messages to Apache Kafka topic(s).
|
static <K,V> Sink<Map.Entry<K,V>> |
KafkaSinks.kafka(Properties properties,
String topic)
Convenience for
KafkaSinks.kafka(Properties, String, FunctionEx, FunctionEx)
which expects Map.Entry<K, V> as input and extracts its key and value
parts to be published to Kafka. |
static <E,K,V> Sink<E> |
KafkaSinks.kafka(Properties properties,
String topic,
FunctionEx<? super E,K> extractKeyFn,
FunctionEx<? super E,V> extractValueFn)
Convenience for
KafkaSinks.kafka(Properties, FunctionEx) which creates
a ProducerRecord using the given topic and the given key and value
mapping functions |
Modifier and Type | Method and Description |
---|---|
Sink<T> |
KinesisSinks.Builder.build()
Construct the sink based on the options provided so far.
|
Modifier and Type | Method and Description |
---|---|
Sink<T> |
FileSinkBuilder.build()
Creates and returns the file
Sink with the supplied components. |
Sink<T> |
SinkBuilder.build()
Creates and returns the
Sink with the components you supplied to
this builder. |
Sink<T> |
JmsSinkBuilder.build()
Creates and returns the JMS
Sink with the supplied components. |
Sink<T> |
JdbcSinkBuilder.build()
Creates and returns the JDBC
Sink with the supplied components. |
static <T extends Map.Entry> |
Sinks.cache(String cacheName)
Returns a sink that puts
Map.Entry s it receives into a Hazelcast
ICache with the specified name. |
static <T> Sink<T> |
Sinks.files(String directoryName)
Convenience for
Sinks.filesBuilder(java.lang.String) with the UTF-8 charset and with
overwriting of existing files. |
static <T> Sink<T> |
Sinks.fromProcessor(String sinkName,
ProcessorMetaSupplier metaSupplier)
Returns a sink constructed directly from the given Core API processor
meta-supplier.
|
static <T> Sink<T> |
Sinks.fromProcessor(String sinkName,
ProcessorMetaSupplier metaSupplier,
FunctionEx<? super T,?> partitionKeyFn)
Returns a sink constructed directly from the given Core API processor
meta-supplier.
|
static <T> Sink<T> |
Sinks.jdbc(String updateQuery,
String jdbcUrl,
BiConsumerEx<PreparedStatement,T> bindFn)
A shortcut for:
|
static <T> Sink<T> |
Sinks.jdbc(String updateQuery,
SupplierEx<? extends CommonDataSource> dataSourceSupplier,
BiConsumerEx<PreparedStatement,T> bindFn)
A shortcut for:
|
static <T> Sink<T> |
Sinks.jmsQueue(String queueName,
SupplierEx<javax.jms.ConnectionFactory> factorySupplier)
Convenience for
Sinks.jmsQueueBuilder(SupplierEx) . |
static <T> Sink<T> |
Sinks.jmsTopic(String topicName,
SupplierEx<javax.jms.ConnectionFactory> factorySupplier)
Shortcut for:
|
static <T> Sink<T> |
Sinks.json(String directoryName)
Convenience for
Sinks.filesBuilder(java.lang.String) with the UTF-8 charset and with
overwriting of existing files. |
static <T> Sink<T> |
Sinks.list(IList<? super T> list)
Returns a sink that adds the items it receives to the specified
Hazelcast
IList . |
static <T> Sink<T> |
Sinks.list(String listName)
Returns a sink that adds the items it receives to a Hazelcast
IList with the specified name. |
static <T> Sink<T> |
Sinks.logger()
|
static <T> Sink<T> |
Sinks.logger(FunctionEx<? super T,String> toStringFn)
Returns a sink that logs all the data items it receives, at the INFO
level to the log category
WriteLoggerP . |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.map(IMap<? super K,? super V> map)
Returns a sink that puts
Map.Entry s it receives into the given
Hazelcast IMap . |
static <T,K,V> Sink<T> |
Sinks.map(IMap<? super K,? super V> map,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn)
Returns a sink that uses the supplied functions to extract the key
and value with which to put to given Hazelcast
IMap . |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.map(String mapName)
Returns a sink that puts
Map.Entry s it receives into a Hazelcast
IMap with the specified name. |
static <T,K,V> Sink<T> |
Sinks.map(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn)
Returns a sink that uses the supplied functions to extract the key
and value with which to put to a Hazelcast
IMap with the
specified name. |
static <T,K,V,R> Sink<T> |
Sinks.mapWithEntryProcessor(IMap<? super K,? super V> map,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends EntryProcessor<K,V,R>> toEntryProcessorFn)
Returns a sink that uses the items it receives to create
EntryProcessor s it submits to a Hazelcast IMap with the
specified name. |
static <E,K,V,R> Sink<E> |
Sinks.mapWithEntryProcessor(int maxParallelAsyncOps,
String mapName,
FunctionEx<? super E,? extends K> toKeyFn,
FunctionEx<? super E,? extends EntryProcessor<K,V,R>> toEntryProcessorFn)
Returns a sink that uses the items it receives to create
EntryProcessor s it submits to a Hazelcast IMap with the
specified name. |
static <E,K,V,R> Sink<E> |
Sinks.mapWithEntryProcessor(String mapName,
FunctionEx<? super E,? extends K> toKeyFn,
FunctionEx<? super E,? extends EntryProcessor<K,V,R>> toEntryProcessorFn)
Convenience for
Sinks.mapWithEntryProcessor(int, String, FunctionEx, FunctionEx)
when the maximum number of async operations is not specified. |
static <T,K,V> Sink<T> |
Sinks.mapWithMerging(IMap<? super K,? super V> map,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn,
BinaryOperatorEx<V> mergeFn)
Returns a sink that uses the supplied functions to extract the key
and value with which to update a Hazelcast
IMap . |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.mapWithMerging(IMap<? super K,V> map,
BinaryOperatorEx<V> mergeFn)
Convenience for
Sinks.mapWithMerging(IMap, FunctionEx, FunctionEx,
BinaryOperatorEx) with Map.Entry as input item. |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.mapWithMerging(String mapName,
BinaryOperatorEx<V> mergeFn)
Convenience for
Sinks.mapWithMerging(String, FunctionEx, FunctionEx,
BinaryOperatorEx) with Map.Entry as input item. |
static <T,K,V> Sink<T> |
Sinks.mapWithMerging(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn,
BinaryOperatorEx<V> mergeFn)
Returns a sink that uses the supplied functions to extract the key
and value with which to update a Hazelcast
IMap . |
static <K,V,E extends Map.Entry<K,V>> |
Sinks.mapWithUpdating(IMap<? super K,? super V> map,
BiFunctionEx<? super V,? super E,? extends V> updateFn)
Convenience for
Sinks.mapWithUpdating(IMap, FunctionEx,
BiFunctionEx) with Map.Entry as the input item. |
static <T,K,V> Sink<T> |
Sinks.mapWithUpdating(IMap<? super K,? super V> map,
FunctionEx<? super T,? extends K> toKeyFn,
BiFunctionEx<? super V,? super T,? extends V> updateFn)
Returns a sink that uses the supplied key-extracting and value-updating
functions to update a Hazelcast
IMap . |
static <K,V,E extends Map.Entry<K,V>> |
Sinks.mapWithUpdating(String mapName,
BiFunctionEx<? super V,? super E,? extends V> updateFn)
Convenience for
Sinks.mapWithUpdating(String, FunctionEx,
BiFunctionEx) with Map.Entry as the input item. |
static <T,K,V> Sink<T> |
Sinks.mapWithUpdating(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
BiFunctionEx<? super V,? super T,? extends V> updateFn)
Returns a sink that uses the supplied key-extracting and value-updating
functions to update a Hazelcast
IMap . |
static <T> Sink<T> |
Sinks.noop()
Returns a sink which discards all received items.
|
static <T> Sink<T> |
Sinks.observable(Observable<? super T> observable)
Returns a sink that publishes to the provided
Observable . |
static <T> Sink<T> |
Sinks.observable(String name)
Returns a sink that publishes to the
Observable with the
provided name. |
static <T> Sink<T> |
Sinks.reliableTopic(ITopic<Object> reliableTopic)
Returns a sink which publishes the items it receives to the provided
distributed reliable topic.
|
static <T> Sink<T> |
Sinks.reliableTopic(String reliableTopicName)
Returns a sink which publishes the items it receives to a distributed
reliable topic with the specified name.
|
static <T extends Map.Entry> |
Sinks.remoteCache(String cacheName,
ClientConfig clientConfig)
Returns a sink that puts
Map.Entry s it receives into a Hazelcast
ICache with the specified name in a remote cluster identified by
the supplied ClientConfig . |
static <T> Sink<T> |
Sinks.remoteList(String listName,
ClientConfig clientConfig)
Returns a sink that adds the items it receives to a Hazelcast
IList with the specified name in a remote cluster identified by the
supplied ClientConfig . |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.remoteMap(String mapName,
ClientConfig clientConfig)
Returns a sink that puts
Map.Entry s it receives into a Hazelcast
IMap with the specified name in a remote cluster identified by
the supplied ClientConfig . |
static <T,K,V> Sink<T> |
Sinks.remoteMap(String mapName,
ClientConfig clientConfig,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn)
Returns a sink that uses the supplied functions to extract the key
and value with which to put to a Hazelcast
IMap in a remote
cluster identified by the supplied ClientConfig . |
static <E,K,V,R> Sink<E> |
Sinks.remoteMapWithEntryProcessor(String mapName,
ClientConfig clientConfig,
FunctionEx<? super E,? extends K> toKeyFn,
FunctionEx<? super E,? extends EntryProcessor<K,V,R>> toEntryProcessorFn)
Returns a sink equivalent to
Sinks.mapWithEntryProcessor(java.lang.String, com.hazelcast.function.FunctionEx<? super E, ? extends K>, com.hazelcast.function.FunctionEx<? super E, ? extends com.hazelcast.map.EntryProcessor<K, V, R>>) , but for a map
in a remote Hazelcast cluster identified by the supplied ClientConfig . |
static <K,V> Sink<Map.Entry<K,V>> |
Sinks.remoteMapWithMerging(String mapName,
ClientConfig clientConfig,
BinaryOperatorEx<V> mergeFn)
|
static <T,K,V> Sink<T> |
Sinks.remoteMapWithMerging(String mapName,
ClientConfig clientConfig,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn,
BinaryOperatorEx<V> mergeFn)
Returns a sink equivalent to
Sinks.mapWithMerging(String, BinaryOperatorEx) ,
but for a map in a remote Hazelcast cluster identified by the supplied
ClientConfig . |
static <K,V,E extends Map.Entry<K,V>> |
Sinks.remoteMapWithUpdating(String mapName,
ClientConfig clientConfig,
BiFunctionEx<? super V,? super E,? extends V> updateFn)
|
static <T,K,V> Sink<T> |
Sinks.remoteMapWithUpdating(String mapName,
ClientConfig clientConfig,
FunctionEx<? super T,? extends K> toKeyFn,
BiFunctionEx<? super V,? super T,? extends V> updateFn)
Returns a sink equivalent to
Sinks.mapWithUpdating(java.lang.String, com.hazelcast.function.FunctionEx<? super T, ? extends K>, com.hazelcast.function.BiFunctionEx<? super V, ? super T, ? extends V>) , but for a map
in a remote Hazelcast cluster identified by the supplied ClientConfig . |
static <T> Sink<T> |
Sinks.remoteReliableTopic(String reliableTopicName,
ClientConfig clientConfig)
Returns a sink which publishes the items it receives to a distributed
reliable topic with the provided name in a remote cluster identified by
the supplied
ClientConfig . |
static <T> Sink<T> |
Sinks.socket(String host,
int port)
Convenience for
Sinks.socket(String, int, FunctionEx,
Charset) with Object.toString as the conversion function and
UTF-8 as the charset. |
static <T> Sink<T> |
Sinks.socket(String host,
int port,
FunctionEx<? super T,? extends String> toStringFn)
Convenience for
Sinks.socket(String, int, FunctionEx,
Charset) with UTF-8 as the charset. |
static <T> Sink<T> |
Sinks.socket(String host,
int port,
FunctionEx<? super T,? extends String> toStringFn,
Charset charset)
Returns a sink that connects to the specified TCP socket and writes to
it a string representation of the items it receives.
|
Modifier and Type | Method and Description |
---|---|
SinkStage |
GeneralStage.writeTo(Sink<? super T> sink)
Attaches a sink stage, one that accepts data but doesn't emit any.
|
<T> SinkStage |
Pipeline.writeTo(Sink<? super T> sink,
GeneralStage<? extends T> stage0,
GeneralStage<? extends T> stage1,
GeneralStage<? extends T>... moreStages)
Attaches the supplied sink to two or more pipeline stages.
|
Modifier and Type | Method and Description |
---|---|
static <T> Sink<T> |
AssertionSinks.assertAnyOrder(Collection<? extends T> expected)
Asserts that the previous stage emitted the expected items in any order,
but nothing else.
|
static <T> Sink<T> |
AssertionSinks.assertAnyOrder(String message,
Collection<? extends T> expected)
Asserts that the previous stage emitted the expected items in any order,
but nothing else.
|
static <T> Sink<T> |
AssertionSinks.assertCollected(ConsumerEx<? super List<T>> assertFn)
Collects all the received items in a list and once the upstream stage is
completed it executes the assertion supplied by
assertFn . |
static <T> Sink<T> |
AssertionSinks.assertCollectedEventually(int timeoutSeconds,
ConsumerEx<? super List<T>> assertFn)
Collects all the received items into a list and runs the
assertFn
every time a new item is received. |
static <T> Sink<T> |
AssertionSinks.assertContains(String message,
Collection<? extends T> expected)
Asserts that the previous stage emitted all of the given items in any order.
|
static <T> Sink<T> |
AssertionSinks.assertOrdered(Collection<? extends T> expected)
Asserts that the previous stage emitted the exact sequence of expected
items and nothing else.
|
static <T> Sink<T> |
AssertionSinks.assertOrdered(String message,
Collection<? extends T> expected)
Asserts that the previous stage emitted the exact sequence of expected
items and nothing else.
|
Sink<T> |
AssertionSinkBuilder.build()
Creates and returns the
Sink with the components you supplied to
this builder. |
Modifier and Type | Method and Description |
---|---|
static <T> Sink<? super T> |
S3Sinks.s3(String bucketName,
String prefix,
Charset charset,
SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier,
FunctionEx<? super T,String> toStringFn)
Creates an AWS S3
Sink which writes items to files into the
given bucket. |
static <T> Sink<? super T> |
S3Sinks.s3(String bucketName,
SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier)
Convenience for
S3Sinks.s3(String, String, Charset, SupplierEx, FunctionEx)
Uses Object.toString() to convert the items to lines. |
Copyright © 2023 Hazelcast, Inc.. All rights reserved.