Package | Description |
---|---|
com.hazelcast.jet.aggregate |
Contains
AggregateOperation and several its variants, as well
as a builder object for the aggregate operations. |
com.hazelcast.jet.avro |
Apache Avro file read/write support for Hazelcast Jet.
|
com.hazelcast.jet.core.processor |
Contains static utility classes with factories of Jet processors.
|
com.hazelcast.jet.function |
Serializable and exception-declaring variants of functional interfaces from
java.util.function . |
com.hazelcast.jet.hadoop |
Apache Hadoop HDFS read/write support for Hazelcast Jet.
|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
Modifier and Type | Method and Description |
---|---|
static <T0,A0,R0,T1,A1,R1,R> |
AggregateOperations.aggregateOperation2(AggregateOperation1<? super T0,A0,? extends R0> op0,
AggregateOperation1<? super T1,A1,? extends R1> op1,
DistributedBiFunction<? super R0,? super R1,? extends R> exportFinishFn)
Returns an aggregate operation that is a composite of two independent
aggregate operations, each one accepting its own input.
|
static <T,A0,A1,R0,R1,R> |
AggregateOperations.allOf(AggregateOperation1<? super T,A0,? extends R0> op0,
AggregateOperation1<? super T,A1,? extends R1> op1,
DistributedBiFunction<? super R0,? super R1,? extends R> exportFinishFn)
Returns an aggregate operation that is a composite of two aggregate
operations.
|
Modifier and Type | Method and Description |
---|---|
<T> BatchSource<T> |
AvroSourceBuilder.build(DistributedBiFunction<String,? super D,T> mapOutputFn)
Builds a custom Avro file
BatchSource with supplied components
and the output function mapOutputFn . |
static <D> BatchSource<D> |
AvroSources.files(String directory,
DistributedBiFunction<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn)
Convenience for
AvroSources.filesBuilder(String, DistributedSupplier) which
reads all the files in the supplied directory as generic records and
emits the results of transforming each generic record with the supplied
mapping function. |
static <D,T> ProcessorMetaSupplier |
AvroProcessors.readFilesP(String directory,
String glob,
boolean sharedFileSystem,
DistributedSupplier<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier,
DistributedBiFunction<String,? super D,T> mapOutputFn)
Returns a supplier of processors for
AvroSources.filesBuilder(java.lang.String, java.lang.Class<D>) . |
Modifier and Type | Method and Description |
---|---|
static <K,A,R,OUT> |
Processors.aggregateByKeyP(List<DistributedFunction<?,? extends K>> keyFns,
AggregateOperation<A,R> aggrOp,
DistributedBiFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that groups items by key
and performs the provided aggregate operation on each group.
|
static <K,A,R,OUT> |
Processors.combineByKeyP(AggregateOperation<A,R> aggrOp,
DistributedBiFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for the second-stage vertex in a
two-stage group-and-aggregate setup.
|
static <C,T,R> ProcessorSupplier |
Processors.flatMapUsingContextP(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends Traverser<? extends R>> flatMapFn)
Returns a supplier of processors for a vertex that applies the provided
item-to-traverser mapping function to each received item and emits all
the items from the resulting traverser.
|
static <C,T,R> ProcessorSupplier |
Processors.mapUsingContextP(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends R> mapFn)
Returns a supplier of processors for a vertex which, for each received
item, emits the result of applying the given mapping function to it.
|
static <R> ProcessorMetaSupplier |
SourceProcessors.readFilesP(String directory,
Charset charset,
String glob,
boolean sharedFileSystem,
DistributedBiFunction<? super String,? super String,? extends R> mapOutputFn)
Returns a supplier of processors for
Sources.filesBuilder(java.lang.String) . |
static ProcessorMetaSupplier |
SourceProcessors.streamFilesP(String watchedDirectory,
Charset charset,
String glob,
boolean sharedFileSystem,
DistributedBiFunction<? super String,? super String,?> mapOutputFn)
Returns a supplier of processors for
Sources.filesBuilder(java.lang.String) . |
static <T,K,V> ProcessorMetaSupplier |
SinkProcessors.updateMapP(String mapName,
DistributedFunction<? super T,? extends K> toKeyFn,
DistributedBiFunction<? super V,? super T,? extends V> updateFn)
Returns a supplier of processors for
Sinks.mapWithEntryProcessor(String, DistributedFunction, DistributedFunction) . |
static <T,K,V> ProcessorMetaSupplier |
SinkProcessors.updateRemoteMapP(String mapName,
ClientConfig clientConfig,
DistributedFunction<? super T,? extends K> toKeyFn,
DistributedBiFunction<? super V,? super T,? extends V> updateFn)
Returns a supplier of processors for
Sinks.remoteMapWithUpdating(String, ClientConfig, DistributedFunction
, DistributedBiFunction) . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeJmsQueueP(DistributedSupplier<? extends javax.jms.Connection> connectionSupplier,
DistributedFunction<? super javax.jms.Connection,? extends javax.jms.Session> sessionF,
DistributedBiFunction<? super javax.jms.Session,? super T,? extends javax.jms.Message> messageFn,
DistributedBiConsumer<? super javax.jms.MessageProducer,? super javax.jms.Message> sendFn,
DistributedConsumer<? super javax.jms.Session> flushFn,
String name)
Returns a supplier of processors for
Sinks.jmsQueueBuilder(com.hazelcast.jet.function.DistributedSupplier<javax.jms.ConnectionFactory>) . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeJmsTopicP(DistributedSupplier<? extends javax.jms.Connection> connectionSupplier,
DistributedFunction<? super javax.jms.Connection,? extends javax.jms.Session> sessionF,
DistributedBiFunction<? super javax.jms.Session,? super T,? extends javax.jms.Message> messageFn,
DistributedBiConsumer<? super javax.jms.MessageProducer,? super javax.jms.Message> sendFn,
DistributedConsumer<? super javax.jms.Session> flushFn,
String name)
Returns a supplier of processors for
Sinks.jmsTopicBuilder(com.hazelcast.jet.function.DistributedSupplier<javax.jms.ConnectionFactory>) . |
Modifier and Type | Method and Description |
---|---|
default <V> DistributedBiFunction<T,U,V> |
DistributedBiFunction.andThen(DistributedFunction<? super R,? extends V> after)
Serializable variant of java.util.function.BiFunction#andThen(Function) . |
Modifier and Type | Method and Description |
---|---|
static <K,V,E> BatchSource<E> |
HdfsSources.hdfs(org.apache.hadoop.mapred.JobConf jobConf,
DistributedBiFunction<K,V,E> projectionFn)
Returns a source that reads records from Apache Hadoop HDFS and emits
the results of transforming each record (a key-value pair) with the
supplied mapping function.
|
static <K,V,R> com.hazelcast.jet.hadoop.impl.ReadHdfsP.MetaSupplier<K,V,R> |
HdfsProcessors.readHdfsP(org.apache.hadoop.mapred.JobConf jobConf,
DistributedBiFunction<K,V,R> mapper)
Returns a supplier of processors for
HdfsSources.hdfs(JobConf, DistributedBiFunction) . |
Modifier and Type | Method and Description |
---|---|
<R,OUT> BatchStage<OUT> |
BatchStageWithKey.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given group-and-aggregate operation.
|
default <T1,R0,R1,OUT> |
BatchStage.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
DistributedBiFunction<? super R0,? super R1,? extends OUT> mapToOutputFn)
Attaches a stage that co-aggregates the data from this and the supplied
stage by performing a separate aggregate operation on each and then
passing both results to
mapToOutputFn , which transforms them
into the final output. |
<T1,R,OUT> BatchStage<OUT> |
BatchStageWithKey.aggregate2(BatchStageWithKey<T1,? extends K> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
<T1,T2,R,OUT> |
BatchStageWithKey.aggregate3(BatchStageWithKey<T1,? extends K> stage1,
BatchStageWithKey<T2,? extends K> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from this stage as well as
stage1 and stage2 you supply. |
<R,OUT> BatchStage<OUT> |
GroupAggregateBuilder1.build(AggregateOperation<?,R> aggrOp,
DistributedBiFunction<? super K,? super R,OUT> mapToOutputFn)
Creates and returns a pipeline stage that performs the
co-grouping and aggregation of pipeline stages registered with this
builder object.
|
<OUT> BatchStage<OUT> |
GroupAggregateBuilder.build(DistributedBiFunction<? super K,ItemsByTag,OUT> mapToOutputFn)
Creates and returns a pipeline stage that performs the co-aggregation
of the stages registered with this builder object.
|
<T> BatchSource<T> |
FileSourceBuilder.build(DistributedBiFunction<String,String,? extends T> mapOutputFn)
Builds a custom file
BatchSource with supplied components and the
output function mapOutputFn . |
<R> StreamStage<R> |
StreamHashJoinBuilder.build(DistributedBiFunction<T0,ItemsByTag,R> mapToOutputFn)
Builds a new pipeline stage that performs the hash-join operation.
|
<R> BatchStage<R> |
HashJoinBuilder.build(DistributedBiFunction<T0,ItemsByTag,R> mapToOutputFn)
Builds a new pipeline stage that performs the hash-join operation.
|
<T> StreamSource<T> |
FileSourceBuilder.buildWatcher(DistributedBiFunction<String,String,? extends T> mapOutputFn)
Builds a source that emits a stream of lines of text coming from files in
the watched directory (but not its subdirectories).
|
<C,R> StreamStage<R> |
StreamStage.flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends Traverser<? extends R>> flatMapFn) |
<C,R> GeneralStage<R> |
GeneralStage.flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends Traverser<? extends R>> flatMapFn)
Attaches a flat-mapping stage which applies the supplied function to
each input item independently and emits all items from the
Traverser it returns as the output items. |
<C,R> BatchStage<R> |
BatchStage.flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends Traverser<? extends R>> flatMapFn) |
<K,T1_IN,T1,R> |
StreamStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
DistributedBiFunction<T,T1,R> mapToOutputFn) |
<K,T1_IN,T1,R> |
GeneralStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
DistributedBiFunction<T,T1,R> mapToOutputFn)
Attaches to both this and the supplied stage a hash-joining stage and
returns it.
|
<K,T1_IN,T1,R> |
BatchStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
DistributedBiFunction<T,T1,R> mapToOutputFn) |
<C,R> StreamStage<R> |
StreamStage.mapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends R> mapFn) |
<C,R> GeneralStage<R> |
GeneralStage.mapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends R> mapFn)
Attaches a mapping stage which applies the supplied function to each
input item independently and emits the function's result as the output
item.
|
<C,R> BatchStage<R> |
BatchStage.mapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn) |
default <K,V,R> GeneralStage<R> |
GeneralStage.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn)
Attaches a
GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap . |
default <K,V,R> BatchStage<R> |
BatchStage.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn) |
default <V,R> StreamStage<R> |
StreamStageWithKey.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super T,? super V,? extends R> mapFn) |
default <V,R> GeneralStage<R> |
GeneralStageWithKey.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super T,? super V,? extends R> mapFn)
Attaches a
GeneralStageWithKey.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap . |
default <V,R> BatchStage<R> |
BatchStageWithKey.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingIMap(String mapName,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn) |
default <K,V,R> GeneralStage<R> |
GeneralStage.mapUsingIMap(String mapName,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn)
Attaches a
GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap with the supplied name. |
default <K,V,R> BatchStage<R> |
BatchStage.mapUsingIMap(String mapName,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn) |
default <V,R> StreamStage<R> |
StreamStageWithKey.mapUsingIMap(String mapName,
DistributedBiFunction<? super T,? super V,? extends R> mapFn) |
default <V,R> GeneralStage<R> |
GeneralStageWithKey.mapUsingIMap(String mapName,
DistributedBiFunction<? super T,? super V,? extends R> mapFn)
Attaches a
GeneralStageWithKey.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap with the supplied name. |
default <V,R> BatchStage<R> |
BatchStageWithKey.mapUsingIMap(String mapName,
DistributedBiFunction<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingReplicatedMap(ReplicatedMap<K,V> replicatedMap,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn) |
default <K,V,R> GeneralStage<R> |
GeneralStage.mapUsingReplicatedMap(ReplicatedMap<K,V> replicatedMap,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn)
Attaches a
GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>) stage where the context is a
Hazelcast ReplicatedMap . |
default <K,V,R> BatchStage<R> |
BatchStage.mapUsingReplicatedMap(ReplicatedMap<K,V> replicatedMap,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingReplicatedMap(String mapName,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn) |
default <K,V,R> GeneralStage<R> |
GeneralStage.mapUsingReplicatedMap(String mapName,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn)
Attaches a
GeneralStage.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>) stage where the context is a
Hazelcast ReplicatedMap with the supplied name. |
default <K,V,R> BatchStage<R> |
BatchStage.mapUsingReplicatedMap(String mapName,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn) |
static <K,V,E extends Map.Entry<K,V>> |
Sinks.mapWithUpdating(IMap<? super K,? super V> map,
DistributedBiFunction<? super V,? super E,? extends V> updateFn)
Convenience for
Sinks.mapWithUpdating(IMap, DistributedFunction,
DistributedBiFunction) with Map.Entry as the input item. |
static <T,K,V> Sink<T> |
Sinks.mapWithUpdating(IMap<? super K,? super V> map,
DistributedFunction<? super T,? extends K> toKeyFn,
DistributedBiFunction<? super V,? super T,? extends V> updateFn)
Returns a sink that uses the supplied key-extracting and value-updating
functions to update a Hazelcast
IMap . |
static <K,V,E extends Map.Entry<K,V>> |
Sinks.mapWithUpdating(String mapName,
DistributedBiFunction<? super V,? super E,? extends V> updateFn)
Convenience for
Sinks.mapWithUpdating(String, DistributedFunction,
DistributedBiFunction) with Map.Entry as the input item. |
static <T,K,V> Sink<T> |
Sinks.mapWithUpdating(String mapName,
DistributedFunction<? super T,? extends K> toKeyFn,
DistributedBiFunction<? super V,? super T,? extends V> updateFn)
Returns a sink that uses the supplied key-extracting and value-updating
functions to update a Hazelcast
IMap . |
JmsSinkBuilder<T> |
JmsSinkBuilder.messageFn(DistributedBiFunction<javax.jms.Session,T,javax.jms.Message> messageFn)
Sets the function which creates the message from the item.
|
static <K,V,E extends Map.Entry<K,V>> |
Sinks.remoteMapWithUpdating(String mapName,
ClientConfig clientConfig,
DistributedBiFunction<? super V,? super E,? extends V> updateFn)
|
static <T,K,V> Sink<T> |
Sinks.remoteMapWithUpdating(String mapName,
ClientConfig clientConfig,
DistributedFunction<? super T,? extends K> toKeyFn,
DistributedBiFunction<? super V,? super T,? extends V> updateFn)
Returns a sink equivalent to
Sinks.mapWithUpdating(java.lang.String, com.hazelcast.jet.function.DistributedFunction<? super T, ? extends K>, com.hazelcast.jet.function.DistributedBiFunction<? super V, ? super T, ? extends V>) , but for a map
in a remote Hazelcast cluster identified by the supplied ClientConfig . |
<R,OUT> StreamStage<OUT> |
StreamStageWithKey.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn) |
<R,OUT> GeneralStage<OUT> |
GeneralStageWithKey.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a rolling aggregation stage.
|
<R,OUT> BatchStage<OUT> |
BatchStageWithKey.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn) |
Copyright © 2018 Hazelcast, Inc.. All rights reserved.