public interface Pipeline extends Serializable
The Pipeline
object is a container of all the stages defined on
a pipeline: the source stages obtained directly from it by calling readFrom(BatchSource)
as well as all the stages attached (directly or
indirectly) to them.
Note that there is no simple one-to-one correspondence between pipeline stages and Core API's DAG vertices. Some stages map to several vertices (e.g., grouping and co-grouping are implemented as a cascade of two vertices) and some stages may be merged with others into a single vertex (e.g., a cascade of map/filter/flatMap stages can be fused into one vertex).
Modifier and Type | Method and Description |
---|---|
static Pipeline |
create()
Creates a new, empty pipeline.
|
boolean |
isEmpty()
Returns
true if there are no stages in the pipeline. |
boolean |
isPreserveOrder()
Returns the preserve order property of this pipeline
|
<T> BatchStage<T> |
readFrom(BatchSource<? extends T> source)
Returns a pipeline stage that represents a bounded (batch) data source.
|
<T> StreamSourceStage<T> |
readFrom(StreamSource<? extends T> source)
Returns a pipeline stage that represents an unbounded data source (i.e., an
event stream).
|
Pipeline |
setPreserveOrder(boolean value)
Tells Jet whether or not it is allowed to reorder the events for better
performance.
|
DAG |
toDag()
Deprecated.
since Jet 4.3, Jet performs this transformation on the server-side.
|
String |
toDotString()
Returns a DOT format (graphviz) representation of the Pipeline.
|
<T> SinkStage |
writeTo(Sink<? super T> sink,
GeneralStage<? extends T> stage0,
GeneralStage<? extends T> stage1,
GeneralStage<? extends T>... moreStages)
Attaches the supplied sink to two or more pipeline stages.
|
boolean isPreserveOrder()
@Nonnull Pipeline setPreserveOrder(boolean value)
Keep in mind that there is often no total event order to begin with, for example in partitioned sources. In this case there is still partial order per each partition, and that is the order Jet preserves.
If you enable this property, you cannot use the rebalance
operator without a grouping key,
because it explicitly orders Jet to break the event order.
The default value is false.
this
, for fluent API@Nonnull <T> BatchStage<T> readFrom(@Nonnull BatchSource<? extends T> source)
T
- the type of source data itemssource
- the definition of the source from which the stage reads data@Nonnull <T> StreamSourceStage<T> readFrom(@Nonnull StreamSource<? extends T> source)
T
- the type of source data itemssource
- the definition of the source from which the stage reads data@Nonnull <T> SinkStage writeTo(@Nonnull Sink<? super T> sink, @Nonnull GeneralStage<? extends T> stage0, @Nonnull GeneralStage<? extends T> stage1, @Nonnull GeneralStage<? extends T>... moreStages)
SinkStage
representing the sink. You need this method when you
want to drain more than one stage to the same sink. In the typical case
you'll use GeneralStage.writeTo(Sink)
instead.T
- the type of data being drained to the sink@Nonnull @Deprecated DAG toDag()
@Nonnull String toDotString()
boolean isEmpty()
true
if there are no stages in the pipeline.Copyright © 2022 Hazelcast, Inc.. All rights reserved.