Uses of Interface
com.hazelcast.jet.pipeline.Sink

Packages that use Sink

Package	Description
com.hazelcast.connector	Hazelcast 3 connector provides sources, sinks and enrichment using data in Hazelcast 3 remote cluster
com.hazelcast.jet.avro	Apache Avro file read/write support for Hazelcast Jet.
com.hazelcast.jet.cdc	Contains source/sink connectors that deal with Change Data Capture (CDC) events from various databases as well as a generic connector for Debezium CDC sources.
com.hazelcast.jet.elastic	Contains sources and sinks for Elasticsearch 7
com.hazelcast.jet.hadoop	Apache Hadoop read/write support for Hazelcast Jet.
com.hazelcast.jet.kafka	Apache Kafka reader/writer support for Hazelcast Jet.
com.hazelcast.jet.kinesis	Amazon Kinesis Data Streams producer/consumer support for Hazelcast Jet.
com.hazelcast.jet.pipeline	The Pipeline API is Jet's high-level API to build and execute distributed computation jobs.
com.hazelcast.jet.pipeline.test	This package contains various mock sources to help with pipeline testing and development.
com.hazelcast.jet.s3	AWS S3 read/write support for Hazelcast Jet.

Uses of Sink in com.hazelcast.connector

Methods in com.hazelcast.connector that return Sink

Modifier and Type	Method and Description
static <T,K,V> Sink<T>	Hz3Sinks.remoteMap(String mapName, FunctionEx<? super T,? extends K> toKeyFn, FunctionEx<? super T,? extends V> toValueFn, String clientXml) Returns a sink that uses the supplied functions to extract the key and value with which to put to a Hazelcast IMap with the specified name.
static <K,V> Sink<Map.Entry<K,V>>	Hz3Sinks.remoteMap(String mapName, String clientXml) Returns a sink that puts Map.Entrys it receives into a Hazelcast IMap with the specified name.

Uses of Sink in com.hazelcast.jet.avro

Methods in com.hazelcast.jet.avro that return Sink

Modifier and Type	Method and Description
static <R> Sink<R>	AvroSinks.files(String directoryName, Class<R> recordClass, org.apache.avro.Schema schema) Convenience for AvroSinks.files(String, Schema, SupplierEx) which uses either SpecificDatumWriter or ReflectDatumWriter depending on the supplied recordClass.
static Sink<org.apache.avro.generic.IndexedRecord>	AvroSinks.files(String directoryName, org.apache.avro.Schema schema) Convenience for AvroSinks.files(String, Schema, SupplierEx) which uses GenericDatumWriter.
static <R> Sink<R>	AvroSinks.files(String directoryName, org.apache.avro.Schema schema, SupplierEx<org.apache.avro.io.DatumWriter<R>> datumWriterSupplier) Returns a sink that that writes the items it receives to Apache Avro files.

Uses of Sink in com.hazelcast.jet.cdc

Methods in com.hazelcast.jet.cdc that return Sink

Modifier and Type	Method and Description
static <K,V> Sink<ChangeRecord>	CdcSinks.map(IMap<? super K,? super V> map, FunctionEx<? super ChangeRecord,? extends K> keyFn, FunctionEx<? super ChangeRecord,? extends V> valueFn) Returns a sink that applies the changes described by a Change Data Capture (CDC) stream to an IMap.
static <K,V> Sink<ChangeRecord>	CdcSinks.map(String mapName, FunctionEx<? super ChangeRecord,? extends K> keyFn, FunctionEx<? super ChangeRecord,? extends V> valueFn) Returns a sink that applies the changes described by a Change Data Capture (CDC) stream to an IMap.
static <K,V> Sink<ChangeRecord>	CdcSinks.remoteMap(String mapName, ClientConfig clientConfig, FunctionEx<? super ChangeRecord,? extends K> keyFn, FunctionEx<? super ChangeRecord,? extends V> valueFn) Returns a sink equivalent to CdcSinks.map(java.lang.String, com.hazelcast.function.FunctionEx<? super com.hazelcast.jet.cdc.ChangeRecord, ? extends K>, com.hazelcast.function.FunctionEx<? super com.hazelcast.jet.cdc.ChangeRecord, ? extends V>), but for a map in a remote Hazelcast cluster identified by the supplied ClientConfig.

Uses of Sink in com.hazelcast.jet.elastic

Methods in com.hazelcast.jet.elastic that return Sink

Modifier and Type	Method and Description
Sink<T>	ElasticSinkBuilder.build() Create a sink that writes data into Elasticsearch based on this builder configuration
static <T> Sink<T>	ElasticSinks.elastic(FunctionEx<? super T,? extends org.elasticsearch.action.DocWriteRequest<?>> mapToRequestFn) Creates an Elasticsearch sink, uses a local instance of Elasticsearch
static <T> Sink<T>	ElasticSinks.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn, FunctionEx<? super T,? extends org.elasticsearch.action.DocWriteRequest<?>> mapToRequestFn) Creates an Elasticsearch sink, uses a client obtained from clientFn and maps items using given mapToRequestFn

Uses of Sink in com.hazelcast.jet.hadoop

Methods in com.hazelcast.jet.hadoop that return Sink

Modifier and Type	Method and Description
static <K,V> Sink<Map.Entry<K,V>>	HadoopSinks.outputFormat(org.apache.hadoop.conf.Configuration configuration) Convenience for HadoopSinks.outputFormat(Configuration, FunctionEx, FunctionEx) which expects Map.Entry<K, V> as input and extracts its key and value parts to be written to HDFS.
static <E,K,V> Sink<E>	HadoopSinks.outputFormat(org.apache.hadoop.conf.Configuration configuration, FunctionEx<? super E,K> extractKeyF, FunctionEx<? super E,V> extractValueF) Returns a sink that writes to Apache Hadoop HDFS.

Uses of Sink in com.hazelcast.jet.kafka

Methods in com.hazelcast.jet.kafka that return Sink

Modifier and Type	Method and Description
Sink<E>	KafkaSinks.Builder.build() Builds the Sink object that you pass to the GeneralStage.writeTo(Sink) method.
static <E,K,V> Sink<E>	KafkaSinks.kafka(Properties properties, FunctionEx<? super E,org.apache.kafka.clients.producer.ProducerRecord<K,V>> toRecordFn) Returns a sink that publishes messages to Apache Kafka topic(s).
static <K,V> Sink<Map.Entry<K,V>>	KafkaSinks.kafka(Properties properties, String topic) Convenience for KafkaSinks.kafka(Properties, String, FunctionEx, FunctionEx) which expects Map.Entry<K, V> as input and extracts its key and value parts to be published to Kafka.
static <E,K,V> Sink<E>	KafkaSinks.kafka(Properties properties, String topic, FunctionEx<? super E,K> extractKeyFn, FunctionEx<? super E,V> extractValueFn) Convenience for KafkaSinks.kafka(Properties, FunctionEx) which creates a ProducerRecord using the given topic and the given key and value mapping functions

Uses of Sink in com.hazelcast.jet.kinesis

Methods in com.hazelcast.jet.kinesis that return Sink

Modifier and Type	Method and Description
Sink<T>	KinesisSinks.Builder.build() Construct the sink based on the options provided so far.

Uses of Sink in com.hazelcast.jet.pipeline

Methods in com.hazelcast.jet.pipeline that return Sink

Modifier and Type	Method and Description
Sink<T>	SinkBuilder.build() Creates and returns the Sink with the components you supplied to this builder.
Sink<T>	JmsSinkBuilder.build() Creates and returns the JMS Sink with the supplied components.
Sink<T>	JdbcSinkBuilder.build() Creates and returns the JDBC Sink with the supplied components.
Sink<T>	FileSinkBuilder.build() Creates and returns the file Sink with the supplied components.
static <T extends Map.Entry> Sink<T>	Sinks.cache(String cacheName) Returns a sink that puts Map.Entrys it receives into a Hazelcast ICache with the specified name.
static <T> Sink<T>	Sinks.files(String directoryName) Convenience for Sinks.filesBuilder(java.lang.String) with the UTF-8 charset and with overwriting of existing files.
static <T> Sink<T>	Sinks.fromProcessor(String sinkName, ProcessorMetaSupplier metaSupplier) Returns a sink constructed directly from the given Core API processor meta-supplier.
static <T> Sink<T>	Sinks.fromProcessor(String sinkName, ProcessorMetaSupplier metaSupplier, FunctionEx<? super T,?> partitionKeyFn) Returns a sink constructed directly from the given Core API processor meta-supplier.
static <T> Sink<T>	Sinks.jdbc(String updateQuery, ExternalDataStoreRef externalDataStoreRef, BiConsumerEx<PreparedStatement,T> bindFn) A shortcut for:
static <T> Sink<T>	Sinks.jdbc(String updateQuery, String jdbcUrl, BiConsumerEx<PreparedStatement,T> bindFn) A shortcut for:
static <T> Sink<T>	Sinks.jdbc(String updateQuery, SupplierEx<? extends CommonDataSource> dataSourceSupplier, BiConsumerEx<PreparedStatement,T> bindFn) A shortcut for:
static <T> Sink<T>	Sinks.jmsQueue(String queueName, SupplierEx<javax.jms.ConnectionFactory> factorySupplier) Convenience for Sinks.jmsQueueBuilder(SupplierEx).
static <T> Sink<T>	Sinks.jmsTopic(String topicName, SupplierEx<javax.jms.ConnectionFactory> factorySupplier) Shortcut for:
static <T> Sink<T>	Sinks.json(String directoryName) Convenience for Sinks.filesBuilder(java.lang.String) with the UTF-8 charset and with overwriting of existing files.
static <T> Sink<T>	Sinks.list(IList<? super T> list) Returns a sink that adds the items it receives to the specified Hazelcast IList.
static <T> Sink<T>	Sinks.list(String listName) Returns a sink that adds the items it receives to a Hazelcast IList with the specified name.
static <T> Sink<T>	Sinks.logger() Convenience for Sinks.logger(FunctionEx) with Object.toString() as the toStringFn.
static <T> Sink<T>	Sinks.logger(FunctionEx<? super T,String> toStringFn) Returns a sink that logs all the data items it receives, at the INFO level to the log category WriteLoggerP.
static <K,V> Sink<Map.Entry<K,V>>	Sinks.map(IMap<? super K,? super V> map) Returns a sink that puts Map.Entrys it receives into the given Hazelcast IMap.
static <T,K,V> Sink<T>	Sinks.map(IMap<? super K,? super V> map, FunctionEx<? super T,? extends K> toKeyFn, FunctionEx<? super T,? extends V> toValueFn) Returns a sink that uses the supplied functions to extract the key and value with which to put to given Hazelcast IMap.
static <K,V> Sink<Map.Entry<K,V>>	Sinks.map(String mapName) Returns a sink that puts Map.Entrys it receives into a Hazelcast IMap with the specified name.
static <T,K,V> Sink<T>	Sinks.map(String mapName, FunctionEx<? super T,? extends K> toKeyFn, FunctionEx<? super T,? extends V> toValueFn) Returns a sink that uses the supplied functions to extract the key and value with which to put to a Hazelcast IMap with the specified name.
static <T,K,V,R> Sink<T>	Sinks.mapWithEntryProcessor(IMap<? super K,? super V> map, FunctionEx<? super T,? extends K> toKeyFn, FunctionEx<? super T,? extends EntryProcessor<K,V,R>> toEntryProcessorFn) Returns a sink that uses the items it receives to create EntryProcessors it submits to a Hazelcast IMap with the specified name.
static <E,K,V,R> Sink<E>	Sinks.mapWithEntryProcessor(int maxParallelAsyncOps, String mapName, FunctionEx<? super E,? extends K> toKeyFn, FunctionEx<? super E,? extends EntryProcessor<K,V,R>> toEntryProcessorFn) Returns a sink that uses the items it receives to create EntryProcessors it submits to a Hazelcast IMap with the specified name.
static <E,K,V,R> Sink<E>	Sinks.mapWithEntryProcessor(String mapName, FunctionEx<? super E,? extends K> toKeyFn, FunctionEx<? super E,? extends EntryProcessor<K,V,R>> toEntryProcessorFn) Convenience for Sinks.mapWithEntryProcessor(int, String, FunctionEx, FunctionEx) when the maximum number of async operations is not specified.
static <T,K,V> Sink<T>	Sinks.mapWithMerging(IMap<? super K,? super V> map, FunctionEx<? super T,? extends K> toKeyFn, FunctionEx<? super T,? extends V> toValueFn, BinaryOperatorEx<V> mergeFn) Returns a sink that uses the supplied functions to extract the key and value with which to update a Hazelcast IMap.
static <K,V> Sink<Map.Entry<K,V>>	Sinks.mapWithMerging(IMap<? super K,V> map, BinaryOperatorEx<V> mergeFn) Convenience for Sinks.mapWithMerging(IMap, FunctionEx, FunctionEx, BinaryOperatorEx) with Map.Entry as input item.
static <K,V> Sink<Map.Entry<K,V>>	Sinks.mapWithMerging(String mapName, BinaryOperatorEx<V> mergeFn) Convenience for Sinks.mapWithMerging(String, FunctionEx, FunctionEx, BinaryOperatorEx) with Map.Entry as input item.
static <T,K,V> Sink<T>	Sinks.mapWithMerging(String mapName, FunctionEx<? super T,? extends K> toKeyFn, FunctionEx<? super T,? extends V> toValueFn, BinaryOperatorEx<V> mergeFn) Returns a sink that uses the supplied functions to extract the key and value with which to update a Hazelcast IMap.
static <K,V,E extends Map.Entry<K,V>> Sink<E>	Sinks.mapWithUpdating(IMap<? super K,? super V> map, BiFunctionEx<? super V,? super E,? extends V> updateFn) Convenience for Sinks.mapWithUpdating(IMap, FunctionEx, BiFunctionEx) with Map.Entry as the input item.
static <T,K,V> Sink<T>	Sinks.mapWithUpdating(IMap<? super K,? super V> map, FunctionEx<? super T,? extends K> toKeyFn, BiFunctionEx<? super V,? super T,? extends V> updateFn) Returns a sink that uses the supplied key-extracting and value-updating functions to update a Hazelcast IMap.
static <K,V,E extends Map.Entry<K,V>> Sink<E>	Sinks.mapWithUpdating(String mapName, BiFunctionEx<? super V,? super E,? extends V> updateFn) Convenience for Sinks.mapWithUpdating(String, FunctionEx, BiFunctionEx) with Map.Entry as the input item.
static <T,K,V> Sink<T>	Sinks.mapWithUpdating(String mapName, FunctionEx<? super T,? extends K> toKeyFn, BiFunctionEx<? super V,? super T,? extends V> updateFn) Returns a sink that uses the supplied key-extracting and value-updating functions to update a Hazelcast IMap.
static <T> Sink<T>	Sinks.noop() Returns a sink which discards all received items.
static <T> Sink<T>	Sinks.observable(Observable<? super T> observable) Returns a sink that publishes to the provided Observable.
static <T> Sink<T>	Sinks.observable(String name) Returns a sink that publishes to the Observable with the provided name.
static <T> Sink<T>	Sinks.reliableTopic(ITopic<Object> reliableTopic) Returns a sink which publishes the items it receives to the provided distributed reliable topic.
static <T> Sink<T>	Sinks.reliableTopic(String reliableTopicName) Returns a sink which publishes the items it receives to a distributed reliable topic with the specified name.
static <T extends Map.Entry> Sink<T>	Sinks.remoteCache(String cacheName, ClientConfig clientConfig) Returns a sink that puts Map.Entrys it receives into a Hazelcast ICache with the specified name in a remote cluster identified by the supplied ClientConfig.
static <T> Sink<T>	Sinks.remoteList(String listName, ClientConfig clientConfig) Returns a sink that adds the items it receives to a Hazelcast IList with the specified name in a remote cluster identified by the supplied ClientConfig.
static <K,V> Sink<Map.Entry<K,V>>	Sinks.remoteMap(String mapName, ClientConfig clientConfig) Returns a sink that puts Map.Entrys it receives into a Hazelcast IMap with the specified name in a remote cluster identified by the supplied ClientConfig.
static <T,K,V> Sink<T>	Sinks.remoteMap(String mapName, ClientConfig clientConfig, FunctionEx<? super T,? extends K> toKeyFn, FunctionEx<? super T,? extends V> toValueFn) Returns a sink that uses the supplied functions to extract the key and value with which to put to a Hazelcast IMap in a remote cluster identified by the supplied ClientConfig.
static <E,K,V,R> Sink<E>	Sinks.remoteMapWithEntryProcessor(String mapName, ClientConfig clientConfig, FunctionEx<? super E,? extends K> toKeyFn, FunctionEx<? super E,? extends EntryProcessor<K,V,R>> toEntryProcessorFn) Returns a sink equivalent to Sinks.mapWithEntryProcessor(java.lang.String, com.hazelcast.function.FunctionEx<? super E, ? extends K>, com.hazelcast.function.FunctionEx<? super E, ? extends com.hazelcast.map.EntryProcessor<K, V, R>>), but for a map in a remote Hazelcast cluster identified by the supplied ClientConfig.
static <K,V> Sink<Map.Entry<K,V>>	Sinks.remoteMapWithMerging(String mapName, ClientConfig clientConfig, BinaryOperatorEx<V> mergeFn) Convenience for Sinks.remoteMapWithMerging(java.lang.String, com.hazelcast.client.config.ClientConfig, com.hazelcast.function.FunctionEx<? super T, ? extends K>, com.hazelcast.function.FunctionEx<? super T, ? extends V>, com.hazelcast.function.BinaryOperatorEx<V>) with Map.Entry as input item.
static <T,K,V> Sink<T>	Sinks.remoteMapWithMerging(String mapName, ClientConfig clientConfig, FunctionEx<? super T,? extends K> toKeyFn, FunctionEx<? super T,? extends V> toValueFn, BinaryOperatorEx<V> mergeFn) Returns a sink equivalent to Sinks.mapWithMerging(String, BinaryOperatorEx), but for a map in a remote Hazelcast cluster identified by the supplied ClientConfig.
static <K,V,E extends Map.Entry<K,V>> Sink<E>	Sinks.remoteMapWithUpdating(String mapName, ClientConfig clientConfig, BiFunctionEx<? super V,? super E,? extends V> updateFn) Convenience for Sinks.remoteMapWithUpdating(java.lang.String, com.hazelcast.client.config.ClientConfig, com.hazelcast.function.FunctionEx<? super T, ? extends K>, com.hazelcast.function.BiFunctionEx<? super V, ? super T, ? extends V>) with Map.Entry as input item.
static <T,K,V> Sink<T>	Sinks.remoteMapWithUpdating(String mapName, ClientConfig clientConfig, FunctionEx<? super T,? extends K> toKeyFn, BiFunctionEx<? super V,? super T,? extends V> updateFn) Returns a sink equivalent to Sinks.mapWithUpdating(java.lang.String, com.hazelcast.function.FunctionEx<? super T, ? extends K>, com.hazelcast.function.BiFunctionEx<? super V, ? super T, ? extends V>), but for a map in a remote Hazelcast cluster identified by the supplied ClientConfig.
static <T> Sink<T>	Sinks.remoteReliableTopic(String reliableTopicName, ClientConfig clientConfig) Returns a sink which publishes the items it receives to a distributed reliable topic with the provided name in a remote cluster identified by the supplied ClientConfig.
static <T> Sink<T>	Sinks.socket(String host, int port) Convenience for Sinks.socket(String, int, FunctionEx, Charset) with Object.toString as the conversion function and UTF-8 as the charset.
static <T> Sink<T>	Sinks.socket(String host, int port, FunctionEx<? super T,? extends String> toStringFn) Convenience for Sinks.socket(String, int, FunctionEx, Charset) with UTF-8 as the charset.
static <T> Sink<T>	Sinks.socket(String host, int port, FunctionEx<? super T,? extends String> toStringFn, Charset charset) Returns a sink that connects to the specified TCP socket and writes to it a string representation of the items it receives.

Methods in com.hazelcast.jet.pipeline with parameters of type Sink

Modifier and Type	Method and Description
SinkStage	GeneralStage.writeTo(Sink<? super T> sink) Attaches a sink stage, one that accepts data but doesn't emit any.
<T> SinkStage	Pipeline.writeTo(Sink<? super T> sink, GeneralStage<? extends T> stage0, GeneralStage<? extends T> stage1, GeneralStage<? extends T>... moreStages) Attaches the supplied sink to two or more pipeline stages.

Uses of Sink in com.hazelcast.jet.pipeline.test

Methods in com.hazelcast.jet.pipeline.test that return Sink

Modifier and Type	Method and Description
static <T> Sink<T>	AssertionSinks.assertAnyOrder(Collection<? extends T> expected) Asserts that the previous stage emitted the expected items in any order, but nothing else.
static <T> Sink<T>	AssertionSinks.assertAnyOrder(String message, Collection<? extends T> expected) Asserts that the previous stage emitted the expected items in any order, but nothing else.
static <T> Sink<T>	AssertionSinks.assertCollected(ConsumerEx<? super List<T>> assertFn) Collects all the received items in a list and once the upstream stage is completed it executes the assertion supplied by assertFn.
static <T> Sink<T>	AssertionSinks.assertCollectedEventually(int timeoutSeconds, ConsumerEx<? super List<T>> assertFn) Collects all the received items into a list and runs the assertFn every time a new item is received.
static <T> Sink<T>	AssertionSinks.assertContains(String message, Collection<? extends T> expected) Asserts that the previous stage emitted all of the given items in any order.
static <T> Sink<T>	AssertionSinks.assertOrdered(Collection<? extends T> expected) Asserts that the previous stage emitted the exact sequence of expected items and nothing else.
static <T> Sink<T>	AssertionSinks.assertOrdered(String message, Collection<? extends T> expected) Asserts that the previous stage emitted the exact sequence of expected items and nothing else.
Sink<T>	AssertionSinkBuilder.build() Creates and returns the Sink with the components you supplied to this builder.

Uses of Sink in com.hazelcast.jet.s3

Methods in com.hazelcast.jet.s3 that return Sink

Modifier and Type	Method and Description
static <T> Sink<? super T>	S3Sinks.s3(String bucketName, String prefix, Charset charset, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier, FunctionEx<? super T,String> toStringFn) Creates an AWS S3 Sink which writes items to files into the given bucket.
static <T> Sink<? super T>	S3Sinks.s3(String bucketName, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier) Convenience for S3Sinks.s3(String, String, Charset, SupplierEx, FunctionEx) Uses Object.toString() to convert the items to lines.