com.hazelcast.jet.mongodb

Class MongoSources

java.lang.Object

com.hazelcast.jet.mongodb.MongoSources

public final class MongoSources
extends Object

Contains factory methods for MongoDB sources.

See MongoSourceBuilder for creating custom MongoDB sources.

Since:

5.3

Method Summary

Modifier and Type	Method and Description
static MongoSourceBuilder.Batch<org.bson.Document>	batch(String name, DataLinkRef dataLinkRef) Creates as builder for new batch mongo source.
static BatchSource<org.bson.Document>	batch(String name, DataLinkRef dataLinkRef, String database, String collection, org.bson.conversions.Bson filter, org.bson.conversions.Bson projection) Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.
static BatchSource<org.bson.Document>	batch(String name, String connectionString, String database, String collection, org.bson.conversions.Bson filter, org.bson.conversions.Bson projection) Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.
static MongoSourceBuilder.Batch<org.bson.Document>	batch(String name, SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier) Creates as builder for new batch mongo source.
static StreamSource<? extends org.bson.Document>	stream(String name, String connectionString, String database, String collection, org.bson.Document filter, org.bson.Document projection) Returns a MongoDB stream source which watches the changes on the collection.
static MongoSourceBuilder.Stream<org.bson.Document>	stream(String name, SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier) Creates as builder for new stream mongo source.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

batch

@Beta
 @Nonnull
public static MongoSourceBuilder.Batch<org.bson.Document> batch(@Nonnull
                                                                                String name,
                                                                                @Nonnull
                                                                                SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)

Creates as builder for new batch mongo source. Equivalent to calling MongoSourceBuilder.batch(java.lang.String, com.hazelcast.function.SupplierEx<? extends com.mongodb.client.MongoClient>).

Example usage:

 BatchSource<Document> batchSource =
         MongoSources.batch("batch-source", () -> MongoClients.create("mongodb://127.0.0.1:27017"))
                 .into("myDatabase", "myCollection")
                 .filter(new Document("age", new Document("$gt", 10)),
                 .projection(new Document("age", 1))
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
 

Parameters:

name - descriptive name for the source (diagnostic purposes) client.

clientSupplier - a function that creates MongoDB client.

Returns:

Batch Mongo source builder

Since:

5.3

batch

@Beta
 @Nonnull
public static MongoSourceBuilder.Batch<org.bson.Document> batch(@Nonnull
                                                                                String name,
                                                                                @Nonnull
                                                                                DataLinkRef dataLinkRef)

Creates as builder for new batch mongo source. Equivalent to calling MongoSourceBuilder.batch(java.lang.String, com.hazelcast.function.SupplierEx<? extends com.mongodb.client.MongoClient>).

Example usage:

 BatchSource<Document> batchSource =
         MongoSources.batch("batch-source", dataLinkRef("mongo"))
                 .into("myDatabase", "myCollection")
                 .filter(new Document("age", new Document("$gt", 10)),
                 .projection(new Document("age", 1))
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
 

Connector will use provided data link reference to obtain an instance of MongoClient. Depending on the configuration this client may be shared between processors or not.

Parameters:

name - descriptive name for the source (diagnostic purposes) client.

dataLinkRef - a reference to mongo data link

Returns:

Batch Mongo source builder

Since:

5.3

batch

@Beta
 @Nonnull
public static BatchSource<org.bson.Document> batch(@Nonnull
                                                                   String name,
                                                                   @Nonnull
                                                                   String connectionString,
                                                                   @Nonnull
                                                                   String database,
                                                                   @Nonnull
                                                                   String collection,
                                                                   @Nullable
                                                                   org.bson.conversions.Bson filter,
                                                                   @Nullable
                                                                   org.bson.conversions.Bson projection)

Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.

See MongoSourceBuilder for creating custom MongoDB sources.

Here's an example which queries documents in a collection having the field age with a value greater than 10 and applies a projection so that only the age field is returned in the emitted document.

 BatchSource<Document> batchSource =
         MongoSources.batch(
                 "batch-source",
                 "mongodb://127.0.0.1:27017",
                 "myDatabase",
                 "myCollection",
                 new Document("age", new Document("$gt", 10)),
                 new Document("age", 1)
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
 

Parameters:

name - a descriptive name for the source (diagnostic purposes)

connectionString - a connection string URI to MongoDB for example: mongodb://127.0.0.1:27017

database - the name of the database

collection - the name of the collection

filter - filter object as a Document

projection - projection object as a Document

Since:

5.3

batch

@Beta
 @Nonnull
public static BatchSource<org.bson.Document> batch(@Nonnull
                                                                   String name,
                                                                   @Nonnull
                                                                   DataLinkRef dataLinkRef,
                                                                   @Nonnull
                                                                   String database,
                                                                   @Nonnull
                                                                   String collection,
                                                                   @Nullable
                                                                   org.bson.conversions.Bson filter,
                                                                   @Nullable
                                                                   org.bson.conversions.Bson projection)

Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.

See MongoSourceBuilder for creating custom MongoDB sources.

Here's an example which queries documents in a collection having the field age with a value greater than 10 and applies a projection so that only the age field is returned in the emitted document.

 BatchSource<Document> batchSource =
         MongoSources.batch(
                 "batch-source",
                 dataLinkRef("mongoDb"),
                 "myDatabase",
                 "myCollection",
                 new Document("age", new Document("$gt", 10)),
                 new Document("age", 1)
         );
 Pipeline p = Pipeline.create();
 BatchStage<Document> srcStage = p.readFrom(batchSource);
 

Connector will use provided data link reference to obtain an instance of MongoClient. Depending on the configuration this client may be shared between processors or not.

Parameters:

name - a descriptive name for the source (diagnostic purposes)

dataLinkRef - a reference to some mongo data link

database - the name of the database

collection - the name of the collection

filter - filter object as a Document

projection - projection object as a Document

Since:

5.3

stream

@Beta
 @Nonnull
public static MongoSourceBuilder.Stream<org.bson.Document> stream(@Nonnull
                                                                                  String name,
                                                                                  @Nonnull
                                                                                  SupplierEx<? extends com.mongodb.client.MongoClient> clientSupplier)

Creates as builder for new stream mongo source. Equivalent to calling MongoSourceBuilder.stream(java.lang.String, com.hazelcast.function.SupplierEx<? extends com.mongodb.client.MongoClient>). Example usage:

 StreamSource<Document> streamSource =
         MongoSources.stream("batch-source", () -> MongoClients.create("mongodb://127.0.0.1:27017"))
                 .into("myDatabase", "myCollection")
                 .filter(new Document("fullDocument.age", new Document("$gt", 10)),
                 .projection(new Document("fullDocument.age", 1))
         );
 Pipeline p = Pipeline.create();
 StreamStage<Document> srcStage = p.readFrom(streamSource);
 

Parameters:

name - descriptive name for the source (diagnostic purposes) client.

clientSupplier - a function that creates MongoDB client.

Returns:

Stream Mongo source builder

Since:

5.3

stream

@Beta
 @Nonnull
public static StreamSource<? extends org.bson.Document> stream(@Nonnull
                                                                               String name,
                                                                               @Nonnull
                                                                               String connectionString,
                                                                               @Nonnull
                                                                               String database,
                                                                               @Nonnull
                                                                               String collection,
                                                                               @Nullable
                                                                               org.bson.Document filter,
                                                                               @Nullable
                                                                               org.bson.Document projection)

Returns a MongoDB stream source which watches the changes on the collection. The source applies the given filter and projection on the change stream documents.

Change stream is available for replica sets and sharded clusters that use WiredTiger storage engine and replica set protocol version 1 (pv1). Change streams can also be used on deployments which employ MongoDB's encryption-at-rest feature. You cannot watch on system collections and collections in admin, local and config databases.

See MongoSourceBuilder for creating custom MongoDB sources.

Here's an example which streams inserts on a collection having the field age with a value greater than 10 and applies a projection so that only the age field is returned in the emitted document.

 StreamSource<? extends Document> streamSource =
         MongoSources.stream(
                 "stream-source",
                 "mongodb://127.0.0.1:27017",
                 "myDatabase",
                 "myCollection",
                 new Document("fullDocument.age", new Document("$gt", 10))
                         .append("operationType", "insert"),
                 new Document("fullDocument.age", 1)
         );

 Pipeline p = Pipeline.create();
 StreamSourceStage<? extends Document> srcStage = p.readFrom(streamSource);
 

Parameters:

name - a descriptive name for the source (diagnostic purposes)

connectionString - a connection string URI to MongoDB for example: mongodb://127.0.0.1:27017

database - the name of the database

collection - the name of the collection

filter - filter object as a Document

projection - projection object as a Document

Since:

5.3