Package com.hazelcast.vector.jet

Class VectorTransforms

java.lang.Object

com.hazelcast.vector.jet.VectorTransforms

@Beta
public final class VectorTransforms
extends Object

Transforms allowing to execute similarity search in Jet pipeline to enrich the stream.

Example usage:

 StreamStage<String> inputStage = createInputStage();
 StreamStage<String> outputStage = inputStage.apply(
         mapUsingVectorSearch(collectionName,
                         // search options template
                         SearchOptions.of(new float[]{0}, 1, true, false),
                         // extract vector from the item
                         item -> item.getVector(),
                         // use results to enrich the item or replace it
                         (item, results) -> Map.entry(item, results)
                 ));
 

With batch stages, mapUsingVectorSearchBatch(java.lang.String, com.hazelcast.vector.SearchOptions, com.hazelcast.function.FunctionEx<T, com.hazelcast.vector.VectorValues>, com.hazelcast.function.BiFunctionEx<T, com.hazelcast.vector.SearchResults<K, V>, R>) has to be used instead mapUsingVectorSearch(java.lang.String, com.hazelcast.vector.SearchOptions, com.hazelcast.function.FunctionEx<T, com.hazelcast.vector.VectorValues>, com.hazelcast.function.BiFunctionEx<T, com.hazelcast.vector.SearchResults<K, V>, R>).

Since:

5.5

Method Summary

Modifier and Type	Method	Description
static <T, K, V, R> com.hazelcast.function.FunctionEx<com.hazelcast.jet.pipeline.StreamStage<T>,com.hazelcast.jet.pipeline.StreamStage<R>>	mapUsingVectorSearch(VectorCollection<K,V> collection, com.hazelcast.vector.SearchOptions options, com.hazelcast.function.FunctionEx<T,com.hazelcast.vector.VectorValues> toVectorFn, com.hazelcast.function.BiFunctionEx<T,SearchResults<K,V>,R> resultFn)	A stage-transforming method that adds similarity search pipeline stage for streaming pipelines.
static <T, K, V, R> com.hazelcast.function.FunctionEx<com.hazelcast.jet.pipeline.StreamStage<T>,com.hazelcast.jet.pipeline.StreamStage<R>>	mapUsingVectorSearch(String collectionName, com.hazelcast.vector.SearchOptions options, com.hazelcast.function.FunctionEx<T,com.hazelcast.vector.VectorValues> toVectorFn, com.hazelcast.function.BiFunctionEx<T,SearchResults<K,V>,R> resultFn)	A stage-transforming method that adds similarity search pipeline stage for streaming pipelines.
static <T, K, V, R> com.hazelcast.function.FunctionEx<com.hazelcast.jet.pipeline.BatchStage<T>,com.hazelcast.jet.pipeline.BatchStage<R>>	mapUsingVectorSearchBatch(VectorCollection<K,V> collection, com.hazelcast.vector.SearchOptions options, com.hazelcast.function.FunctionEx<T,com.hazelcast.vector.VectorValues> toVectorFn, com.hazelcast.function.BiFunctionEx<T,SearchResults<K,V>,R> resultFn)	A stage-transforming method that adds similarity search pipeline stage for batch pipelines.
static <T, K, V, R> com.hazelcast.function.FunctionEx<com.hazelcast.jet.pipeline.BatchStage<T>,com.hazelcast.jet.pipeline.BatchStage<R>>	mapUsingVectorSearchBatch(String collectionName, com.hazelcast.vector.SearchOptions options, com.hazelcast.function.FunctionEx<T,com.hazelcast.vector.VectorValues> toVectorFn, com.hazelcast.function.BiFunctionEx<T,SearchResults<K,V>,R> resultFn)	A stage-transforming method that adds similarity search pipeline stage for batch pipelines.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

mapUsingVectorSearch

@Nonnull
public static <T,
K,
V,
R>
com.hazelcast.function.FunctionEx<com.hazelcast.jet.pipeline.StreamStage<T>,com.hazelcast.jet.pipeline.StreamStage<R>> mapUsingVectorSearch(@Nonnull
 String collectionName,
 @Nonnull
 com.hazelcast.vector.SearchOptions options,
 @Nonnull
 com.hazelcast.function.FunctionEx<T,com.hazelcast.vector.VectorValues> toVectorFn,
 @Nonnull
 com.hazelcast.function.BiFunctionEx<T,SearchResults<K,V>,R> resultFn)

A stage-transforming method that adds similarity search pipeline stage for streaming pipelines.

Type Parameters:

T - type of incoming stream item

K - type of vector collection key

V - type of vector collection value

R - type of produced stream item

Parameters:

collectionName - vector collection name

options - search options prototype (vectors will be supplied for each item)

toVectorFn - function that extracts the vectors from the stream item. It must be stateless and cooperative

resultFn - function that aggregates original item and search results. Search results are never null but may be empty. It must be stateless and cooperative

Returns:

the newly attached stage

mapUsingVectorSearch

@Nonnull
public static <T,
K,
V,
R>
com.hazelcast.function.FunctionEx<com.hazelcast.jet.pipeline.StreamStage<T>,com.hazelcast.jet.pipeline.StreamStage<R>> mapUsingVectorSearch(@Nonnull
 VectorCollection<K,V> collection,
 @Nonnull
 com.hazelcast.vector.SearchOptions options,
 @Nonnull
 com.hazelcast.function.FunctionEx<T,com.hazelcast.vector.VectorValues> toVectorFn,
 @Nonnull
 com.hazelcast.function.BiFunctionEx<T,SearchResults<K,V>,R> resultFn)

A stage-transforming method that adds similarity search pipeline stage for streaming pipelines.

Type Parameters:

T - type of incoming stream item

K - type of vector collection key

V - type of vector collection value

R - type of produced stream item

Parameters:

collection - vector collection

options - search options prototype (vectors will be supplied for each item)

toVectorFn - function that extracts the vectors from the stream item. It must be stateless and cooperative

resultFn - function that aggregates original item and search results. Search results are never null but may be empty. It must be stateless and cooperative

Returns:

the newly attached stage

mapUsingVectorSearchBatch

@Nonnull
public static <T,
K,
V,
R>
com.hazelcast.function.FunctionEx<com.hazelcast.jet.pipeline.BatchStage<T>,com.hazelcast.jet.pipeline.BatchStage<R>> mapUsingVectorSearchBatch(@Nonnull
 String collectionName,
 @Nonnull
 com.hazelcast.vector.SearchOptions options,
 @Nonnull
 com.hazelcast.function.FunctionEx<T,com.hazelcast.vector.VectorValues> toVectorFn,
 @Nonnull
 com.hazelcast.function.BiFunctionEx<T,SearchResults<K,V>,R> resultFn)

A stage-transforming method that adds similarity search pipeline stage for batch pipelines.

Type Parameters:

T - type of incoming stream item

K - type of vector collection key

V - type of vector collection value

R - type of produced stream item

Parameters:

collectionName - vector collection name

options - search options prototype (vectors will be supplied for each item)

toVectorFn - function that extracts the vectors from the stream item. It must be stateless and cooperative

resultFn - function that aggregates original item and search results. Search results are never null but may be empty. It must be stateless and cooperative

Returns:

the newly attached stage

mapUsingVectorSearchBatch

@Nonnull
public static <T,
K,
V,
R>
com.hazelcast.function.FunctionEx<com.hazelcast.jet.pipeline.BatchStage<T>,com.hazelcast.jet.pipeline.BatchStage<R>> mapUsingVectorSearchBatch(@Nonnull
 VectorCollection<K,V> collection,
 @Nonnull
 com.hazelcast.vector.SearchOptions options,
 @Nonnull
 com.hazelcast.function.FunctionEx<T,com.hazelcast.vector.VectorValues> toVectorFn,
 @Nonnull
 com.hazelcast.function.BiFunctionEx<T,SearchResults<K,V>,R> resultFn)

A stage-transforming method that adds similarity search pipeline stage for batch pipelines.

Type Parameters:

T - type of incoming stream item

K - type of vector collection key

V - type of vector collection value

R - type of produced stream item

Parameters:

collection - vector collection

options - search options prototype (vectors will be supplied for each item)

toVectorFn - function that extracts the vectors from the stream item. It must be stateless and cooperative

resultFn - function that aggregates original item and search results. Search results are never null but may be empty. It must be stateless and cooperative

Returns:

the newly attached stage