Package com.hazelcast.vector.jet

Class VectorTransforms

java.lang.Object
com.hazelcast.vector.jet.VectorTransforms
@Beta public final class VectorTransforms extends Object
Transforms allowing to execute similarity search in Jet pipeline to enrich the stream.

Example usage: 


 StreamStage<String> inputStage = createInputStage();
 StreamStage<String> outputStage = inputStage.apply(
         mapUsingVectorSearch(collectionName,
                         // search options template
                         SearchOptions.of(new float[]{0}, 1, true, false),
                         // extract vector from the item
                         item -> item.getVector(),
                         // use results to enrich the item or replace it
                         (item, results) -> Map.entry(item, results)
                 ));
With batch stages, mapUsingVectorSearchBatch(java.lang.String, com.hazelcast.vector.SearchOptions, com.hazelcast.function.FunctionEx<T, com.hazelcast.vector.VectorValues>, com.hazelcast.function.BiFunctionEx<T, com.hazelcast.vector.SearchResults<K, V>, R>) has to be used instead mapUsingVectorSearch(java.lang.String, com.hazelcast.vector.SearchOptions, com.hazelcast.function.FunctionEx<T, com.hazelcast.vector.VectorValues>, com.hazelcast.function.BiFunctionEx<T, com.hazelcast.vector.SearchResults<K, V>, R>).
Since:
5.5

  • Method Details

    • mapUsingVectorSearch

      @Nonnull public static <T, K, V, R> FunctionEx<StreamStage<T>,StreamStage<R>> mapUsingVectorSearch(@Nonnull String collectionName, @Nonnull SearchOptions options, @Nonnull FunctionEx<T,VectorValues> toVectorFn, @Nonnull BiFunctionEx<T,SearchResults<K,V>,R> resultFn)
      A stage-transforming method that adds similarity search pipeline stage for streaming pipelines.
      Type Parameters:
      T - type of incoming stream item
      K - type of vector collection key
      V - type of vector collection value
      R - type of produced stream item
      Parameters:
      collectionName - vector collection name
      options - search options prototype (vectors will be supplied for each item)
      toVectorFn - function that extracts the vectors from the stream item. It must be stateless and cooperative
      resultFn - function that aggregates original item and search results. Search results are never null but may be empty. It must be stateless and cooperative
      Returns:
      the newly attached stage

    • mapUsingVectorSearch

      @Nonnull public static <T, K, V, R> FunctionEx<StreamStage<T>,StreamStage<R>> mapUsingVectorSearch(@Nonnull VectorCollection<K,V> collection, @Nonnull SearchOptions options, @Nonnull FunctionEx<T,VectorValues> toVectorFn, @Nonnull BiFunctionEx<T,SearchResults<K,V>,R> resultFn)
      A stage-transforming method that adds similarity search pipeline stage for streaming pipelines.
      Type Parameters:
      T - type of incoming stream item
      K - type of vector collection key
      V - type of vector collection value
      R - type of produced stream item
      Parameters:
      collection - vector collection
      options - search options prototype (vectors will be supplied for each item)
      toVectorFn - function that extracts the vectors from the stream item. It must be stateless and cooperative
      resultFn - function that aggregates original item and search results. Search results are never null but may be empty. It must be stateless and cooperative
      Returns:
      the newly attached stage

    • mapUsingVectorSearchBatch

      @Nonnull public static <T, K, V, R> FunctionEx<BatchStage<T>,BatchStage<R>> mapUsingVectorSearchBatch(@Nonnull String collectionName, @Nonnull SearchOptions options, @Nonnull FunctionEx<T,VectorValues> toVectorFn, @Nonnull BiFunctionEx<T,SearchResults<K,V>,R> resultFn)
      A stage-transforming method that adds similarity search pipeline stage for batch pipelines.
      Type Parameters:
      T - type of incoming stream item
      K - type of vector collection key
      V - type of vector collection value
      R - type of produced stream item
      Parameters:
      collectionName - vector collection name
      options - search options prototype (vectors will be supplied for each item)
      toVectorFn - function that extracts the vectors from the stream item. It must be stateless and cooperative
      resultFn - function that aggregates original item and search results. Search results are never null but may be empty. It must be stateless and cooperative
      Returns:
      the newly attached stage

    • mapUsingVectorSearchBatch

      @Nonnull public static <T, K, V, R> FunctionEx<BatchStage<T>,BatchStage<R>> mapUsingVectorSearchBatch(@Nonnull VectorCollection<K,V> collection, @Nonnull SearchOptions options, @Nonnull FunctionEx<T,VectorValues> toVectorFn, @Nonnull BiFunctionEx<T,SearchResults<K,V>,R> resultFn)
      A stage-transforming method that adds similarity search pipeline stage for batch pipelines.
      Type Parameters:
      T - type of incoming stream item
      K - type of vector collection key
      V - type of vector collection value
      R - type of produced stream item
      Parameters:
      collection - vector collection
      options - search options prototype (vectors will be supplied for each item)
      toVectorFn - function that extracts the vectors from the stream item. It must be stateless and cooperative
      resultFn - function that aggregates original item and search results. Search results are never null but may be empty. It must be stateless and cooperative
      Returns:
      the newly attached stage