Package com.hazelcast.jet.mongodb

Class MongoSources

  • public final class MongoSources
extends java.lang.Object
    Contains factory methods for MongoDB sources.

    See MongoSourceBuilder for creating custom MongoDB sources.

    Since:
    5.3

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static MongoSourceBuilder.Batch<org.bson.Document> batch​(SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)
      Creates as builder for new batch mongo source.
      static MongoSourceBuilder.Batch<org.bson.Document> batch​(DataConnectionRef dataConnectionRef)
      Creates as builder for new batch mongo source.
      static BatchSource<org.bson.Document> batch​(DataConnectionRef dataConnectionRef, java.lang.String database, java.lang.String collection, org.bson.conversions.Bson filter, org.bson.conversions.Bson projection)
      Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.
      static BatchSource<org.bson.Document> batch​(java.lang.String connectionString, java.lang.String database, java.lang.String collection, org.bson.conversions.Bson filter, org.bson.conversions.Bson projection)
      Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.
      static MongoSourceBuilder.Stream<org.bson.Document> stream​(SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)
      Creates as builder for new stream mongo source.
      static StreamSource<? extends org.bson.Document> stream​(java.lang.String connectionString, java.lang.String database, java.lang.String collection, org.bson.Document filter, org.bson.Document projection)
      Returns a MongoDB stream source which watches the changes on the collection.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • batch

        @Nonnull
public static MongoSourceBuilder.Batch<org.bson.Document> batch​(@Nonnull
                                                                SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)
        Creates as builder for new batch mongo source. Equivalent to calling MongoSourceBuilder.batch(java.lang.String, com.hazelcast.function.SupplierEx<? extends com.mongodb.client.MongoClient>).

        Example usage: 

        
 BatchSource<MyDTO> batchSource =
         MongoSources.batch(() -> MongoClients.create("mongodb://127.0.0.1:27017"))
                 .database("myDatabase")
                 .collection("myCollection", MyDTO.class)
                 .filter(new Document("age", new Document("$gt", 10)),
                 .projection(new Document("age", 1))
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
        Parameters:
        clientSupplier - a function that creates MongoDB client.
        Returns:
        Batch Mongo source builder
        Since:
        5.3

      • batch

        @Nonnull
public static MongoSourceBuilder.Batch<org.bson.Document> batch​(@Nonnull
                                                                DataConnectionRef dataConnectionRef)
        Creates as builder for new batch mongo source. Equivalent to calling MongoSourceBuilder.batch(java.lang.String, com.hazelcast.function.SupplierEx<? extends com.mongodb.client.MongoClient>).

        Example usage: 

        
 BatchSource<Document> batchSource =
         MongoSources.batch(dataConnectionRef("mongo"))
                 .database("myDatabase")
                 .collection("myCollection")
                 .filter(new Document("age", new Document("$gt", 10)),
                 .projection(new Document("age", 1))
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
        Connector will use provided data connection reference to obtain an instance of MongoClient. Depending on the configuration this client may be shared between processors or not.
        Parameters:
        dataConnectionRef - a reference to mongo data connection
        Returns:
        Batch Mongo source builder
        Since:
        5.3

      • batch

        @Nonnull
public static BatchSource<org.bson.Document> batch​(@Nonnull
                                                   java.lang.String connectionString,
                                                   @Nonnull
                                                   java.lang.String database,
                                                   @Nonnull
                                                   java.lang.String collection,
                                                   @Nullable
                                                   org.bson.conversions.Bson filter,
                                                   @Nullable
                                                   org.bson.conversions.Bson projection)
        Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.

        See MongoSourceBuilder for creating custom MongoDB sources.

        Here's an example which queries documents in a collection having the field age with a value greater than 10 and applies a projection so that only the age field is returned in the emitted document. 

        
 BatchSource<Document> batchSource =
         MongoSources.batch(
                 "mongodb://127.0.0.1:27017",
                 "myDatabase",
                 "myCollection",
                 new Document("age", new Document("$gt", 10)),
                 new Document("age", 1)
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
        Parameters:
        connectionString - a connection string URI to MongoDB for example: mongodb://127.0.0.1:27017
        database - the name of the database
        collection - the name of the collection
        filter - filter object as a Document
        projection - projection object as a Document
        Since:
        5.3

      • batch

        @Nonnull
public static BatchSource<org.bson.Document> batch​(@Nonnull
                                                   DataConnectionRef dataConnectionRef,
                                                   @Nonnull
                                                   java.lang.String database,
                                                   @Nonnull
                                                   java.lang.String collection,
                                                   @Nullable
                                                   org.bson.conversions.Bson filter,
                                                   @Nullable
                                                   org.bson.conversions.Bson projection)
        Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.

        See MongoSourceBuilder for creating custom MongoDB sources.

        Here's an example which queries documents in a collection having the field age with a value greater than 10 and applies a projection so that only the age field is returned in the emitted document. 

        
 BatchSource<Document> batchSource =
         MongoSources.batch(
                 dataConnectionRef("mongoDb"),
                 "myDatabase",
                 "myCollection",
                 new Document("age", new Document("$gt", 10)),
                 new Document("age", 1)
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);

        Connector will use provided data connection reference to obtain an instance of MongoClient. Depending on the configuration this client may be shared between processors or not.

        Parameters:
        dataConnectionRef - a reference to some mongo data connection
        database - the name of the database
        collection - the name of the collection
        filter - filter object as a Document
        projection - projection object as a Document
        Since:
        5.3

      • stream

        @Nonnull
public static MongoSourceBuilder.Stream<org.bson.Document> stream​(@Nonnull
                                                                  SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)
        Creates as builder for new stream mongo source. Equivalent to calling MongoSourceBuilder.stream(java.lang.String, com.hazelcast.function.SupplierEx<? extends com.mongodb.client.MongoClient>). Example usage: 
        
 StreamSource<Document> streamSource =
         MongoSources.stream(() -> MongoClients.create("mongodb://127.0.0.1:27017"))
                 .database("myDatabase")
                 .collection("myCollection")
                 .filter(new Document("fullDocument.age", new Document("$gt", 10)),
                 .projection(new Document("fullDocument.age", 1))
         );
 Pipeline p = Pipeline.create();
 StreamStage<Document> srcStage = p.readFrom(streamSource);
        Parameters:
        clientSupplier - a function that creates MongoDB client.
        Returns:
        Stream Mongo source builder
        Since:
        5.3

      • stream

        @Nonnull
public static StreamSource<? extends org.bson.Document> stream​(@Nonnull
                                                               java.lang.String connectionString,
                                                               @Nonnull
                                                               java.lang.String database,
                                                               @Nonnull
                                                               java.lang.String collection,
                                                               @Nullable
                                                               org.bson.Document filter,
                                                               @Nullable
                                                               org.bson.Document projection)
        Returns a MongoDB stream source which watches the changes on the collection. The source applies the given filter and projection on the change stream documents.

        Change stream is available for replica sets and sharded clusters that use WiredTiger storage engine and replica set protocol version 1 (pv1). Change streams can also be used on deployments which employ MongoDB's encryption-at-rest feature. You cannot watch on system collections and collections in admin, local and config databases.

        See MongoSourceBuilder for creating custom MongoDB sources.

        Here's an example which streams inserts on a collection having the field age with a value greater than 10 and applies a projection so that only the age field is returned in the emitted document. 

        
 StreamSource<? extends Document> streamSource =
         MongoSources.stream(
                 "mongodb://127.0.0.1:27017",
                 "myDatabase",
                 "myCollection",
                 new Document("fullDocument.age", new Document("$gt", 10))
                         .append("operationType", "insert"),
                 new Document("fullDocument.age", 1)
         );

 Pipeline p = Pipeline.create();
 StreamSourceStage<? extends Document> srcStage = p.readFrom(streamSource);
        Parameters:
        connectionString - a connection string URI to MongoDB for example: mongodb://127.0.0.1:27017
        database - the name of the database
        collection - the name of the collection
        filter - filter object as a Document
        projection - projection object as a Document
        Since:
        5.3