Package com.hazelcast.jet.python

Class PythonTransforms

java.lang.Object

com.hazelcast.jet.python.PythonTransforms

public final class PythonTransforms
extends Object

Transforms which allow the user to call Python user-defined functions from inside a Jet pipeline.

Since:

Jet 4.0

Method Summary

Modifier and Type	Method	Description
static <K> FunctionEx<StreamStage<String>,StreamStage<String>>	mapUsingPython(FunctionEx<? super String,? extends K> keyFn, PythonServiceConfig cfg)	Deprecated. Jet now has first-class support for data rebalancing, see GeneralStage.rebalance() and GeneralStage.rebalance(FunctionEx).
static FunctionEx<StreamStage<String>,StreamStage<String>>	mapUsingPython(PythonServiceConfig cfg)	A stage-transforming method that adds a "map using Python" pipeline stage.
static <K> FunctionEx<BatchStage<String>,BatchStage<String>>	mapUsingPythonBatch(FunctionEx<? super String,? extends K> keyFn, PythonServiceConfig cfg)	A stage-transforming method that adds a partitioned "map using Python" pipeline stage.
static FunctionEx<BatchStage<String>,BatchStage<String>>	mapUsingPythonBatch(PythonServiceConfig cfg)	A stage-transforming method that adds a "map using Python" pipeline stage.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

mapUsingPython

@Nonnull
public static FunctionEx<StreamStage<String>,StreamStage<String>> mapUsingPython(@Nonnull
 PythonServiceConfig cfg)

A stage-transforming method that adds a "map using Python" pipeline stage. Use it with stage.apply(PythonService.mapUsingPython(pyConfig)). See PythonServiceConfig for more details.

mapUsingPython

@Deprecated
@Nonnull
public static <K>
FunctionEx<StreamStage<String>,StreamStage<String>> mapUsingPython(@Nonnull
 FunctionEx<? super String,? extends K> keyFn,
 @Nonnull
 PythonServiceConfig cfg)

Deprecated.

Jet now has first-class support for data rebalancing, see GeneralStage.rebalance() and GeneralStage.rebalance(FunctionEx).

A stage-transforming method that adds a partitioned "map using Python" pipeline stage. It applies partitioning using the supplied keyFn. You need partitioning if your input stream comes from a non-distributed data source (all data coming in on a single cluster member), in order to distribute the Python work across the whole cluster.

Use it like this: stage.apply(PythonService.mapUsingPython(keyFn, pyConfig)). See PythonServiceConfig for more details.

mapUsingPythonBatch

@Nonnull
public static FunctionEx<BatchStage<String>,BatchStage<String>> mapUsingPythonBatch(@Nonnull
 PythonServiceConfig cfg)

A stage-transforming method that adds a "map using Python" pipeline stage. Use it with stage.apply(PythonService.mapUsingPythonBatch(pyConfig)). See PythonServiceConfig for more details.

mapUsingPythonBatch

@Nonnull
public static <K>
FunctionEx<BatchStage<String>,BatchStage<String>> mapUsingPythonBatch(@Nonnull
 FunctionEx<? super String,? extends K> keyFn,
 @Nonnull
 PythonServiceConfig cfg)

A stage-transforming method that adds a partitioned "map using Python" pipeline stage. It applies partitioning using the supplied keyFn. You need partitioning if your input stream comes from a non-distributed data source (all data coming in on a single cluster member), in order to distribute the Python work across the whole cluster.

Use it like this: stage.apply(PythonService.mapUsingPythonBatch(keyFn, pyConfig)). See PythonServiceConfig for more details.