Package com.hazelcast.jet.mongodb

Class MongoSources

java.lang.Object
com.hazelcast.jet.mongodb.MongoSources
public final class MongoSources extends Object
Contains factory methods for MongoDB sources.

See MongoSourceBuilder for creating custom MongoDB sources.

Since:
5.3

  • Method Summary

    Modifier and Type
    Method
    Description
    static MongoSourceBuilder.Batch<org.bson.Document>
    batch(SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)
    Creates as builder for new batch mongo source.
    static MongoSourceBuilder.Batch<org.bson.Document>
    batch(DataConnectionRef dataConnectionRef)
    Creates as builder for new batch mongo source.
    static BatchSource<org.bson.Document>
    batch(DataConnectionRef dataConnectionRef, String database, String collection, org.bson.conversions.Bson filter, org.bson.conversions.Bson projection)
    Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.
    static BatchSource<org.bson.Document>
    batch(String connectionString, String database, String collection, org.bson.conversions.Bson filter, org.bson.conversions.Bson projection)
    Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.
    static MongoSourceBuilder.Stream<org.bson.Document>
    stream(SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)
    Creates as builder for new stream mongo source.
    static StreamSource<? extends org.bson.Document>
    stream(String connectionString, String database, String collection, org.bson.Document filter, org.bson.Document projection)
    Returns a MongoDB stream source which watches the changes on the collection.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • batch

      @Nonnull public static MongoSourceBuilder.Batch<org.bson.Document> batch(@Nonnull SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)
      Creates as builder for new batch mongo source. Equivalent to calling MongoSourceBuilder.batch(java.lang.String, com.hazelcast.function.SupplierEx<? extends com.mongodb.client.MongoClient>).

      Example usage: 

      
 BatchSource<MyDTO> batchSource =
         MongoSources.batch(() -> MongoClients.create("mongodb://127.0.0.1:27017"))
                 .database("myDatabase")
                 .collection("myCollection", MyDTO.class)
                 .filter(new Document("age", new Document("$gt", 10)),
                 .projection(new Document("age", 1))
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
      Parameters:
      clientSupplier - a function that creates MongoDB client.
      Returns:
      Batch Mongo source builder
      Since:
      5.3

    • batch

      @Nonnull public static MongoSourceBuilder.Batch<org.bson.Document> batch(@Nonnull DataConnectionRef dataConnectionRef)
      Creates as builder for new batch mongo source. Equivalent to calling MongoSourceBuilder.batch(java.lang.String, com.hazelcast.function.SupplierEx<? extends com.mongodb.client.MongoClient>).

      Example usage: 

      
 BatchSource<Document> batchSource =
         MongoSources.batch(dataConnectionRef("mongo"))
                 .database("myDatabase")
                 .collection("myCollection")
                 .filter(new Document("age", new Document("$gt", 10)),
                 .projection(new Document("age", 1))
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
      Connector will use provided data connection reference to obtain an instance of MongoClient. Depending on the configuration this client may be shared between processors or not.
      Parameters:
      dataConnectionRef - a reference to mongo data connection
      Returns:
      Batch Mongo source builder
      Since:
      5.3

    • batch

      @Nonnull public static BatchSource<org.bson.Document> batch(@Nonnull String connectionString, @Nonnull String database, @Nonnull String collection, @Nullable org.bson.conversions.Bson filter, @Nullable org.bson.conversions.Bson projection)
      Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.

      See MongoSourceBuilder for creating custom MongoDB sources.

      Here's an example which queries documents in a collection having the field age with a value greater than 10 and applies a projection so that only the age field is returned in the emitted document. 

      
 BatchSource<Document> batchSource =
         MongoSources.batch(
                 "mongodb://127.0.0.1:27017",
                 "myDatabase",
                 "myCollection",
                 new Document("age", new Document("$gt", 10)),
                 new Document("age", 1)
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
      Parameters:
      connectionString - a connection string URI to MongoDB for example: mongodb://127.0.0.1:27017
      database - the name of the database
      collection - the name of the collection
      filter - filter object as a Document
      projection - projection object as a Document
      Since:
      5.3

    • batch

      @Nonnull public static BatchSource<org.bson.Document> batch(@Nonnull DataConnectionRef dataConnectionRef, @Nonnull String database, @Nonnull String collection, @Nullable org.bson.conversions.Bson filter, @Nullable org.bson.conversions.Bson projection)
      Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.

      See MongoSourceBuilder for creating custom MongoDB sources.

      Here's an example which queries documents in a collection having the field age with a value greater than 10 and applies a projection so that only the age field is returned in the emitted document. 

      
 BatchSource<Document> batchSource =
         MongoSources.batch(
                 dataConnectionRef("mongoDb"),
                 "myDatabase",
                 "myCollection",
                 new Document("age", new Document("$gt", 10)),
                 new Document("age", 1)
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);

      Connector will use provided data connection reference to obtain an instance of MongoClient. Depending on the configuration this client may be shared between processors or not.

      Parameters:
      dataConnectionRef - a reference to some mongo data connection
      database - the name of the database
      collection - the name of the collection
      filter - filter object as a Document
      projection - projection object as a Document
      Since:
      5.3

    • stream

      @Nonnull public static MongoSourceBuilder.Stream<org.bson.Document> stream(@Nonnull SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)
      Creates as builder for new stream mongo source. Equivalent to calling MongoSourceBuilder.stream(java.lang.String, com.hazelcast.function.SupplierEx<? extends com.mongodb.client.MongoClient>). Example usage: 
      
 StreamSource<Document> streamSource =
         MongoSources.stream(() -> MongoClients.create("mongodb://127.0.0.1:27017"))
                 .database("myDatabase")
                 .collection("myCollection")
                 .filter(new Document("fullDocument.age", new Document("$gt", 10)),
                 .projection(new Document("fullDocument.age", 1))
         );
 Pipeline p = Pipeline.create();
 StreamStage<Document> srcStage = p.readFrom(streamSource);
      Parameters:
      clientSupplier - a function that creates MongoDB client.
      Returns:
      Stream Mongo source builder
      Since:
      5.3

    • stream

      @Nonnull public static StreamSource<? extends org.bson.Document> stream(@Nonnull String connectionString, @Nonnull String database, @Nonnull String collection, @Nullable org.bson.Document filter, @Nullable org.bson.Document projection)
      Returns a MongoDB stream source which watches the changes on the collection. The source applies the given filter and projection on the change stream documents.

      Change stream is available for replica sets and sharded clusters that use WiredTiger storage engine and replica set protocol version 1 (pv1). Change streams can also be used on deployments which employ MongoDB's encryption-at-rest feature. You cannot watch on system collections and collections in admin, local and config databases.

      See MongoSourceBuilder for creating custom MongoDB sources.

      Here's an example which streams inserts on a collection having the field age with a value greater than 10 and applies a projection so that only the age field is returned in the emitted document. 

      
 StreamSource<? extends Document> streamSource =
         MongoSources.stream(
                 "mongodb://127.0.0.1:27017",
                 "myDatabase",
                 "myCollection",
                 new Document("fullDocument.age", new Document("$gt", 10))
                         .append("operationType", "insert"),
                 new Document("fullDocument.age", 1)
         );

 Pipeline p = Pipeline.create();
 StreamSourceStage<? extends Document> srcStage = p.readFrom(streamSource);
      Parameters:
      connectionString - a connection string URI to MongoDB for example: mongodb://127.0.0.1:27017
      database - the name of the database
      collection - the name of the collection
      filter - filter object as a Document
      projection - projection object as a Document
      Since:
      5.3