Package com.hazelcast.jet.core

Class DAG

  • All Implemented Interfaces:
    DataSerializable, IdentifiedDataSerializable, java.lang.Iterable<Vertex>
    public class DAG
extends java.lang.Object
implements IdentifiedDataSerializable, java.lang.Iterable<Vertex>
    Describes a computation to be performed by the Jet computation engine. A vertex represents a unit of data processing and an edge represents the path along which the data travels to the next vertex.

    The work of a single vertex is parallelized and distributed, so that there are several instances of the Processor type on each member corresponding to it. Whenever possible, each instance should be tasked with only a slice of the total data and a partitioning strategy can be employed to ensure that the data sent to each vertex is collated by a partitioning key.

    There are three basic kinds of vertices:

    1. source with just outbound edges;
    2. processor with both inbound and outbound edges;
    3. sink with just inbound edges.
    Data travels from sources to sinks and is transformed and reshaped as it passes through the processors.
    Since:
    Jet 3.0

    • Constructor Summary

      Constructors 
      Constructor Description
      DAG()  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      DAG edge​(Edge edge)
      Adds an edge to this DAG.
      java.util.Iterator<Edge> edgeIterator()
      Returns an iterator over the DAG's edges in unspecified order.
      int getClassId()
      Returns type identifier for this class.
      int getFactoryId()
      Returns DataSerializableFactory factory ID for this class.
      java.util.List<Edge> getInboundEdges​(java.lang.String vertexName)
      Returns the inbound edges connected to the vertex with the given name.
      java.util.List<Edge> getOutboundEdges​(java.lang.String vertexName)
      Returns the outbound edges connected to the vertex with the given name.
      Vertex getVertex​(java.lang.String vertexName)
      Returns the vertex with the given name, or null if there is no vertex with that name.
      java.util.Iterator<Vertex> iterator()
      Returns an iterator over the DAG's vertices in topological order.
      void lock()
      Used to prevent further mutations to the DAG after submitting it for execution.
      Vertex newUniqueVertex​(java.lang.String namePrefix, SupplierEx<? extends Processor> simpleSupplier)
      Creates a vertex from a Supplier<Processor> and adds it to this DAG.
      Vertex newUniqueVertex​(java.lang.String namePrefix, ProcessorMetaSupplier metaSupplier)
      Creates a vertex from a ProcessorMetaSupplier and adds it to this DAG.
      Vertex newUniqueVertex​(java.lang.String namePrefix, ProcessorSupplier processorSupplier)
      Creates a vertex from a ProcessorSupplier and adds it to this DAG.
      Vertex newVertex​(java.lang.String name, SupplierEx<? extends Processor> simpleSupplier)
      Creates a vertex from a Supplier<Processor> and adds it to this DAG.
      Vertex newVertex​(java.lang.String name, ProcessorMetaSupplier metaSupplier)
      Creates a vertex from a ProcessorMetaSupplier and adds it to this DAG.
      Vertex newVertex​(java.lang.String name, ProcessorSupplier processorSupplier)
      Creates a vertex from a ProcessorSupplier and adds it to this DAG.
      void readData​(ObjectDataInput in)
      Reads fields from the input stream
      java.lang.String toDotString()
      Returns a DOT format (graphviz) representation of the DAG.
      java.lang.String toDotString​(int defaultLocalParallelism, int defaultQueueSize)
      Returns a DOT format (graphviz) representation of the DAG, annotates the vertices using supplied default parallelism value, and the edges using supplied default queue size value.
      com.hazelcast.internal.json.JsonObject toJson​(int defaultLocalParallelism)
      Returns a JSON representation of the DAG.
      java.lang.String toString()  
      java.lang.String toString​(int defaultLocalParallelism)
      Returns a string representation of the DAG.
      DAG vertex​(Vertex vertex)
      Adds a vertex to this DAG.
      java.util.Set<Vertex> vertices()
      Returns a copy of the DAG's vertices.
      void writeData​(ObjectDataOutput out)
      Writes object fields to output stream

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

      • Methods inherited from interface java.lang.Iterable

        forEach, spliterator

    • Constructor Detail

      • DAG

        public DAG()

    • Method Detail

      • newVertex

        @Nonnull
public Vertex newVertex​(@Nonnull
                        java.lang.String name,
                        @Nonnull
                        SupplierEx<? extends Processor> simpleSupplier)
        Creates a vertex from a Supplier<Processor> and adds it to this DAG.
        Parameters:
        name - the unique name of the vertex
        simpleSupplier - the simple, parameterless supplier of Processor instances
        See Also:
        Vertex(String, SupplierEx)

      • newUniqueVertex

        @Nonnull
public Vertex newUniqueVertex​(@Nonnull
                              java.lang.String namePrefix,
                              @Nonnull
                              SupplierEx<? extends Processor> simpleSupplier)
        Creates a vertex from a Supplier<Processor> and adds it to this DAG. The vertex will be given a unique name created from the namePrefix.
        Parameters:
        namePrefix - the prefix for unique name of the vertex
        simpleSupplier - the simple, parameterless supplier of Processor instances
        Since:
        Jet 4.4
        See Also:
        Vertex(String, SupplierEx)

      • newVertex

        @Nonnull
public Vertex newVertex​(@Nonnull
                        java.lang.String name,
                        @Nonnull
                        ProcessorSupplier processorSupplier)
        Creates a vertex from a ProcessorSupplier and adds it to this DAG.
        Parameters:
        name - the unique name of the vertex
        processorSupplier - the supplier of Processor instances which will be used on all members
        See Also:
        Vertex(String, ProcessorSupplier)

      • newUniqueVertex

        @Nonnull
public Vertex newUniqueVertex​(@Nonnull
                              java.lang.String namePrefix,
                              @Nonnull
                              ProcessorSupplier processorSupplier)
        Creates a vertex from a ProcessorSupplier and adds it to this DAG. The vertex will be given a unique name created from the namePrefix.
        Parameters:
        namePrefix - the prefix for unique name of the vertex
        processorSupplier - the supplier of Processor instances which will be used on all members
        Since:
        Jet 4.4
        See Also:
        Vertex(String, ProcessorSupplier)

      • newVertex

        @Nonnull
public Vertex newVertex​(@Nonnull
                        java.lang.String name,
                        @Nonnull
                        ProcessorMetaSupplier metaSupplier)
        Creates a vertex from a ProcessorMetaSupplier and adds it to this DAG.
        Parameters:
        name - the unique name of the vertex
        metaSupplier - the meta-supplier of ProcessorSuppliers for each member
        See Also:
        Vertex(String, ProcessorMetaSupplier)

      • newUniqueVertex

        @Nonnull
public Vertex newUniqueVertex​(@Nonnull
                              java.lang.String namePrefix,
                              @Nonnull
                              ProcessorMetaSupplier metaSupplier)
        Creates a vertex from a ProcessorMetaSupplier and adds it to this DAG. The vertex will be given a unique name created from the namePrefix.
        Parameters:
        namePrefix - the prefix for unique name of the vertex
        metaSupplier - the meta-supplier of ProcessorSuppliers for each member
        Since:
        Jet 4.4
        See Also:
        Vertex(String, ProcessorMetaSupplier)

      • vertex

        @Nonnull
public DAG vertex​(@Nonnull
                  Vertex vertex)
        Adds a vertex to this DAG. The vertex name must be unique.

      • edge

        @Nonnull
public DAG edge​(@Nonnull
                Edge edge)
        Adds an edge to this DAG. The vertices it connects must already be present in the DAG. It is an error to connect an edge to a vertex at the same ordinal as another existing edge. However, inbound and outbound ordinals are independent, so there can be two edges at the same ordinal, one inbound and one outbound.

        Jet supports multigraphs, that is you can add two edges between the same two vertices. However, they have to have different ordinals.

      • getInboundEdges

        @Nonnull
public java.util.List<Edge> getInboundEdges​(@Nonnull
                                            java.lang.String vertexName)
        Returns the inbound edges connected to the vertex with the given name.

      • getOutboundEdges

        @Nonnull
public java.util.List<Edge> getOutboundEdges​(@Nonnull
                                             java.lang.String vertexName)
        Returns the outbound edges connected to the vertex with the given name.

      • getVertex

        @Nullable
public Vertex getVertex​(@Nonnull
                        java.lang.String vertexName)
        Returns the vertex with the given name, or null if there is no vertex with that name.

      • vertices

        @Nonnull
public java.util.Set<Vertex> vertices()
        Returns a copy of the DAG's vertices. Adding a vertex to or removing a vertex from the returned Set will not be reflected in the DAG, and vice-versa.

      • iterator

        @Nonnull
public java.util.Iterator<Vertex> iterator()
        Returns an iterator over the DAG's vertices in topological order.
        Specified by:
        iterator in interface java.lang.Iterable<Vertex>

      • edgeIterator

        @Nonnull
public java.util.Iterator<Edge> edgeIterator()
        Returns an iterator over the DAG's edges in unspecified order.
        Since:
        5.4

      • toString

        @Nonnull
public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object

      • toString

        @Nonnull
public java.lang.String toString​(int defaultLocalParallelism)
        Returns a string representation of the DAG.
        Parameters:
        defaultLocalParallelism - the local parallelism that will be shown if neither overridden on the vertex nor the preferred parallelism is defined by meta-supplier

      • toJson

        @Nonnull
public com.hazelcast.internal.json.JsonObject toJson​(int defaultLocalParallelism)
        Returns a JSON representation of the DAG.

        Note: the exact structure of the JSON is unspecified.

        Parameters:
        defaultLocalParallelism - the local parallelism that will be shown if neither overridden on the vertex nor the preferred parallelism is defined by meta-supplier

      • toDotString

        @Nonnull
public java.lang.String toDotString()
        Returns a DOT format (graphviz) representation of the DAG.

      • toDotString

        @Nonnull
public java.lang.String toDotString​(int defaultLocalParallelism,
                                    int defaultQueueSize)
        Returns a DOT format (graphviz) representation of the DAG, annotates the vertices using supplied default parallelism value, and the edges using supplied default queue size value.
        Parameters:
        defaultLocalParallelism - the local parallelism that will be shown if neither overridden on the vertex nor the preferred parallelism is defined by meta-supplier
        defaultQueueSize - the queue size that will be shown if not overridden on the edge

      • writeData

        public void writeData​(ObjectDataOutput out)
               throws java.io.IOException
        Description copied from interface: DataSerializable
        Writes object fields to output stream
        Specified by:
        writeData in interface DataSerializable
        Parameters:
        out - output
        Throws:
        java.io.IOException - if an I/O error occurs. In particular, an IOException may be thrown if the output stream has been closed.

      • readData

        public void readData​(ObjectDataInput in)
              throws java.io.IOException
        Description copied from interface: DataSerializable
        Reads fields from the input stream
        Specified by:
        readData in interface DataSerializable
        Parameters:
        in - input
        Throws:
        java.io.IOException - if an I/O error occurs. In particular, an IOException may be thrown if the input stream has been closed.

      • lock

        @PrivateApi
public void lock()
        Used to prevent further mutations to the DAG after submitting it for execution.

        It's not a public API, can be removed in the future.