Package com.hazelcast.jet.elastic

Class ElasticSinkBuilder<T>

java.lang.Object
com.hazelcast.jet.elastic.ElasticSinkBuilder<T>
Type Parameters:
T -
All Implemented Interfaces:
Serializable
@Deprecated(forRemoval=true, since="5.7") public final class ElasticSinkBuilder<T> extends Object implements Serializable
Deprecated, for removal: This API element is subject to removal in a future version.
Elasticsearch 7 is no longer supported
Builder for Elasticsearch Sink

The Sink first maps items from the pipeline using the provided mapToRequestFn(FunctionEx) and then using BulkRequest.

BulkRequest() is used by default, it can be modified by providing custom bulkRequestFn(SupplierEx)

Usage: 


 Sink<Map<String, ?>> elasticSink = new ElasticSinkBuilder<Map<String, ?>>()
   .clientFn(() -> ElasticClients.client(host, port))
   .mapToRequestFn(item -> new IndexRequest("my-index").source(item))
   .build();

Requires clientFn(SupplierEx) and mapToRequestFn(FunctionEx).

Since:
Jet 4.2
See Also:

  • Constructor Summary

    Constructors
    Constructor
    Description
    ElasticSinkBuilder()
    Deprecated, for removal: This API element is subject to removal in a future version.
     

  • Method Summary

    Modifier and Type
    Method
    Description
    Sink<T>
    build()
    Deprecated, for removal: This API element is subject to removal in a future version.
    Create a sink that writes data into Elasticsearch based on this builder configuration
    ElasticSinkBuilder<T>
    bulkRequestFn(SupplierEx<org.elasticsearch.action.bulk.BulkRequest> bulkRequestFn)
    Deprecated, for removal: This API element is subject to removal in a future version.
    Set the supplier function for BulkRequest, defaults to new BulkRequest()
    ElasticSinkBuilder<T>
    clientFn(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn)
    Deprecated, for removal: This API element is subject to removal in a future version.
    Set the client supplier function
    <T_NEW> ElasticSinkBuilder<T_NEW>
    mapToRequestFn(FunctionEx<? super T_NEW,? extends org.elasticsearch.action.DocWriteRequest<?>> mapToRequestFn)
    Deprecated, for removal: This API element is subject to removal in a future version.
    Set the function mapping the item from a pipeline item to an index request
    ElasticSinkBuilder<T>
    optionsFn(FunctionEx<? super org.elasticsearch.action.ActionRequest,org.elasticsearch.client.RequestOptions> optionsFn)
    Deprecated, for removal: This API element is subject to removal in a future version.
    Set the function that provides RequestOptions
    ElasticSinkBuilder<T>
    retries(int retries)
    Deprecated, for removal: This API element is subject to removal in a future version.
    Number of retries the connector will do in addition to Elastic client retries

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • ElasticSinkBuilder

      public ElasticSinkBuilder()
      Deprecated, for removal: This API element is subject to removal in a future version.

  • Method Details

    • clientFn

      @Nonnull public ElasticSinkBuilder<T> clientFn(@Nonnull SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn)
      Deprecated, for removal: This API element is subject to removal in a future version.
      Set the client supplier function

      The connector uses the returned instance to access Elasticsearch. Also see ElasticClients for convenience factory methods.

      For example, to provide an authenticated client: 

      
 builder.clientFn(() -> client(host, port, username, password))
      This parameter is required.
      Parameters:
      clientFn - supplier function returning configured Elasticsearch REST client

    • bulkRequestFn

      @Nonnull public ElasticSinkBuilder<T> bulkRequestFn(@Nonnull SupplierEx<org.elasticsearch.action.bulk.BulkRequest> bulkRequestFn)
      Deprecated, for removal: This API element is subject to removal in a future version.
      Set the supplier function for BulkRequest, defaults to new BulkRequest()

      For example, to modify the BulkRequest used to index documents: 

      
 builder.bulkRequestFn(() -> new BulkRequest().setRefreshPolicy(IMMEDIATE))
      Parameters:
      bulkRequestFn - supplier function for the bulk request
      See Also:

    • mapToRequestFn

      @Nonnull public <T_NEW> ElasticSinkBuilder<T_NEW> mapToRequestFn(@Nonnull FunctionEx<? super T_NEW,? extends org.elasticsearch.action.DocWriteRequest<?>> mapToRequestFn)
      Deprecated, for removal: This API element is subject to removal in a future version.
      Set the function mapping the item from a pipeline item to an index request

      For example, to create an IndexRequest for a versioned document: 

      
 builder.mapToRequestFn((mapItem) ->
                      new IndexRequest("my-index")
                              .source(map)
                              .version((Long) map.get("version"))
      This parameter is required.
      Type Parameters:
      T_NEW - type of the items from the pipeline
      Parameters:
      mapToRequestFn - maps an item from the stream to an IndexRequest, UpdateRequest or DeleteRequest

    • optionsFn

      @Nonnull public ElasticSinkBuilder<T> optionsFn(@Nonnull FunctionEx<? super org.elasticsearch.action.ActionRequest,org.elasticsearch.client.RequestOptions> optionsFn)
      Deprecated, for removal: This API element is subject to removal in a future version.
      Set the function that provides RequestOptions

      It can either return a constant value or a value based on provided request.

      For example, to provide a custom authentication header: 

      
 sinkBuilder.optionsFn((request) -> {
     RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
     builder.addHeader("Authorization", "Bearer " + TOKEN);
     return builder.build();
 })
      Parameters:
      optionsFn - function that provides RequestOptions
      See Also:

    • retries

      @Nonnull public ElasticSinkBuilder<T> retries(int retries)
      Deprecated, for removal: This API element is subject to removal in a future version.
      Number of retries the connector will do in addition to Elastic client retries

      Elastic client tries to connect to a node only once for each request. When a request fails the node is marked dead and is not retried again for the request. This causes problems with single node clusters or in a situation where whole cluster becomes unavailable at the same time (e.g. due to a network issue).

      The initial delay is 2s, increasing by factor of 2 with each retry (4s, 8s, 16s, ..).

      Parameters:
      retries - number of retries, defaults to 5

    • build

      @Nonnull public Sink<T> build()
      Deprecated, for removal: This API element is subject to removal in a future version.
      Create a sink that writes data into Elasticsearch based on this builder configuration