Package com.hazelcast.jet.python
Class PythonTransforms
- java.lang.Object
-
- com.hazelcast.jet.python.PythonTransforms
-
public final class PythonTransforms extends java.lang.Object
Transforms which allow the user to call Python user-defined functions from inside a Jet pipeline.- Since:
- Jet 4.0
-
-
Method Summary
All Methods Static Methods Concrete Methods Deprecated Methods Modifier and Type Method Description static <K> FunctionEx<StreamStage<java.lang.String>,StreamStage<java.lang.String>>
mapUsingPython(FunctionEx<? super java.lang.String,? extends K> keyFn, PythonServiceConfig cfg)
Deprecated.Jet now has first-class support for data rebalancing, seeGeneralStage.rebalance()
andGeneralStage.rebalance(FunctionEx)
.static FunctionEx<StreamStage<java.lang.String>,StreamStage<java.lang.String>>
mapUsingPython(PythonServiceConfig cfg)
A stage-transforming method that adds a "map using Python" pipeline stage.static <K> FunctionEx<BatchStage<java.lang.String>,BatchStage<java.lang.String>>
mapUsingPythonBatch(FunctionEx<? super java.lang.String,? extends K> keyFn, PythonServiceConfig cfg)
A stage-transforming method that adds a partitioned "map using Python" pipeline stage.static FunctionEx<BatchStage<java.lang.String>,BatchStage<java.lang.String>>
mapUsingPythonBatch(PythonServiceConfig cfg)
A stage-transforming method that adds a "map using Python" pipeline stage.
-
-
-
Method Detail
-
mapUsingPython
@Nonnull public static FunctionEx<StreamStage<java.lang.String>,StreamStage<java.lang.String>> mapUsingPython(@Nonnull PythonServiceConfig cfg)
A stage-transforming method that adds a "map using Python" pipeline stage. Use it withstage.apply(PythonService.mapUsingPython(pyConfig))
. SeePythonServiceConfig
for more details.
-
mapUsingPython
@Deprecated @Nonnull public static <K> FunctionEx<StreamStage<java.lang.String>,StreamStage<java.lang.String>> mapUsingPython(@Nonnull FunctionEx<? super java.lang.String,? extends K> keyFn, @Nonnull PythonServiceConfig cfg)
Deprecated.Jet now has first-class support for data rebalancing, seeGeneralStage.rebalance()
andGeneralStage.rebalance(FunctionEx)
.A stage-transforming method that adds a partitioned "map using Python" pipeline stage. It applies partitioning using the suppliedkeyFn
. You need partitioning if your input stream comes from a non-distributed data source (all data coming in on a single cluster member), in order to distribute the Python work across the whole cluster.Use it like this:
stage.apply(PythonService.mapUsingPython(keyFn, pyConfig))
. SeePythonServiceConfig
for more details.
-
mapUsingPythonBatch
@Nonnull public static FunctionEx<BatchStage<java.lang.String>,BatchStage<java.lang.String>> mapUsingPythonBatch(@Nonnull PythonServiceConfig cfg)
A stage-transforming method that adds a "map using Python" pipeline stage. Use it withstage.apply(PythonService.mapUsingPythonBatch(pyConfig))
. SeePythonServiceConfig
for more details.
-
mapUsingPythonBatch
@Nonnull public static <K> FunctionEx<BatchStage<java.lang.String>,BatchStage<java.lang.String>> mapUsingPythonBatch(@Nonnull FunctionEx<? super java.lang.String,? extends K> keyFn, @Nonnull PythonServiceConfig cfg)
A stage-transforming method that adds a partitioned "map using Python" pipeline stage. It applies partitioning using the suppliedkeyFn
. You need partitioning if your input stream comes from a non-distributed data source (all data coming in on a single cluster member), in order to distribute the Python work across the whole cluster.Use it like this:
stage.apply(PythonService.mapUsingPythonBatch(keyFn, pyConfig))
. SeePythonServiceConfig
for more details.
-
-