Metadata Package¶

class kubeflow.metadata.metadata.ABC[source]¶

Bases: object

Helper class that provides a standard way to create an ABC using inheritance.

class kubeflow.metadata.metadata.Artifact[source]¶

Bases: abc.ABC

abstract property ARTIFACT_TYPE_NAME¶

classmethod(function) -> method

Convert a function to be a class method.

A class method receives the class as implicit first argument, just like an instance method receives the instance. To declare a class method, use this idiom:

class C:
@classmethod def f(cls, arg1, arg2, …):

…

It can be called either on the class (e.g. C.f()) or on an instance (e.g. C().f()). The instance is ignored except for its class. If a class method is called for a derived class, the derived class object is passed as the implied first argument.

Class methods are different than C++ or Java static methods. If you want those, see the staticmethod builtin.

static is_duplicated(a, b)[source]¶

Checks if two artifacts are duplicated.

The artifacts may be considered duplication even if not all the fields are the same as in mlpb.Artifact. For example, two models can be considered the same if they have the same uri, name and version.

Returns: True or False for duplication.

abstract serialization()[source]¶

Return type: Artifact

class kubeflow.metadata.metadata.DataSet(uri=None, name=None, workspace=None, description=None, owner=None, version=None, query=None, labels=None, **kwargs)[source]¶

Bases: kubeflow.metadata.metadata.Artifact

Dataset captures a data set in a machine learning workflow.

uri¶: Required uri of the data set.

name¶: Required name of the data set.

workspace¶: Optional name of the workspace.

description¶: Optional description of the data set.

owner¶: Optional owner of the data set.

version¶: Optional version tagged by the user.

query¶: Optional query string on how this data set being fetched from a data source.

labels¶: Optional string key value pairs for labels.

Example

>>> metadata.DataSet(description="an example data",
...                  name="mytable-dump",
...                  owner="owner@my-company.org",
...                  uri="file://path/to/dataset",
...                  version="v1.0.0",
...                  query="SELECT * FROM mytable",
...                  labels={"label1","val1"}))

ARTIFACT_TYPE_NAME = 'kubeflow.org/alpha/data_set'¶

static is_duplicated(a, b)[source]¶

Checks if two artifacts are duplicated.

The artifacts may be considered duplication even if not all the fields are the same as in mlpb.Artifact. For example, two models can be considered the same if they have the same uri, name and version.

Returns: True or False for duplication.

serialization()[source]¶

class kubeflow.metadata.metadata.Execution(name=None, workspace=None, run=None, description=None)[source]¶

Bases: object

Captures a run of pipeline or notebooks in a workspace and group executions.

Execution also serves as object for logging artifacts as its input or output.

EXECUTION_TYPE_NAME = 'kubeflow.org/alpha/execution'¶

log_input(artifact)[source]¶

Log an artifact as an input of this execution.

Parameters: artifact (Artifact) – Model, DataSet, Metrics or customized artifact type.
Return type: Artifact
Returns: The same artifact with artifact.id set.

log_output(artifact)[source]¶

Log an artifact as an input of this execution.

Parameters: artifact (Artifact) – Model, DataSet, Metrics or customized artifact type.
Return type: Artifact
Returns: The same artifact with artifact.id set.

serialized()[source]¶

class kubeflow.metadata.metadata.List[source]¶: Bases: list, typing.MutableSequence

class kubeflow.metadata.metadata.Mapping[source]¶: Bases: collections.abc.Mapping, typing.Collection

class kubeflow.metadata.metadata.Metrics(uri=None, name=None, workspace=None, description=None, owner=None, data_set_id=None, model_id=None, metrics_type=None, values=None, labels=None, **kwargs)[source]¶

Bases: kubeflow.metadata.metadata.Artifact

Captures an evaluation metrics of a model on a data set.

uri¶: Required uri of the metrics.

name¶: Required name of the metrics.

workspace¶: Optional name of the workspace.

description¶: Optional description of the metrics.

owner¶: Optional owner of the metrics.

data_set_id¶: Optional id of the data set used for evaluation.

model_id¶: Optional id of a evaluated model.

metrics_type¶: Optional type of the evaluation.

values¶: Optional map from metrics name to its value.

labels¶: Optional string key value pairs for labels.

Example

>>> metadata.Metrics(
...        name="MNIST-evaluation",
...        description=
...        "validating the MNIST model to recognize handwritten digits",
...        owner="someone@kubeflow.org",
...        uri="gcs://my-bucket/mnist-eval.csv",
...        data_set_id="123",
...        model_id="12345",
...        metrics_type=metadata.Metrics.VALIDATION,
...        values={"accuracy": 0.95},
...        labels={"mylabel": "l1"}))

ARTIFACT_TYPE_NAME = 'kubeflow.org/alpha/metrics'¶

PRODUCTION = 'production'¶

TESTING = 'testing'¶

TRAINING = 'training'¶

VALIDATION = 'validation'¶

serialization()[source]¶

class kubeflow.metadata.metadata.Model(uri=None, name=None, workspace=None, description=None, owner=None, version=None, model_type=None, training_framework=None, hyperparameters=None, labels=None, **kwargs)[source]¶

Bases: kubeflow.metadata.metadata.Artifact

Captures a machine learning model.

uri¶: Required uri of the model artifact, e.g. “gcs://path/to/model.h5”.

name¶: Required name of the model.

workspace¶: Optional name of the workspace.

description¶: Optional description of the model.

owner¶: Optional owner of the model.

model_type¶: Optional type of the model.

training_framework¶: Optional framework used to train the model.

hyperparameters¶: Optional map from hyper param name to its value.

labels¶: Optional string key value pairs for labels.

kwargs¶: Optional additional keyword arguments are saved as additional properties of this model.

Example

>>> metadata.Model(name="MNIST",
...                description="model to recognize handwritten digits",
...                owner="someone@kubeflow.org",
...                uri="gcs://my-bucket/mnist",
...                model_type="neural network",
...                training_framework={
...                   "name": "tensorflow",
...                   "version": "v1.0"
...                },
...                hyperparameters={
...                   "learning_rate": 0.5,
...                   "layers": [10, 3, 1],
...                   "early_stop": True
...                },
...                version="v0.0.1",
...                labels={"mylabel": "l1"}))

ARTIFACT_TYPE_NAME = 'kubeflow.org/alpha/model'¶

static is_duplicated(a, b)[source]¶

Checks if two artifacts are duplicated.

The artifacts may be considered duplication even if not all the fields are the same as in mlpb.Artifact. For example, two models can be considered the same if they have the same uri, name and version.

Returns: True or False for duplication.

serialization()[source]¶

class kubeflow.metadata.metadata.Run(workspace=None, name=None, description=None)[source]¶

Bases: object

Run captures a run of pipeline or notebooks in a workspace and group executions.

class kubeflow.metadata.metadata.Store(grpc_host='metadata-grpc-service.kubeflow', grpc_port=8080, root_certificates=None, private_key=None, certificate_chain=None)[source]¶

Bases: object

Metadata Store that connects to the Metadata gRPC service.

class kubeflow.metadata.metadata.Workspace(store=None, name=None, description=None, labels=None, reuse_workspace_if_exists=True, backend_url_prefix=None)[source]¶

Bases: object

Groups a set of runs of pipelines, notebooks and their related artifacts and executions.

CONTEXT_TYPE_NAME = 'kubeflow.org/alpha/workspace'¶

list(artifact_type_name=None)[source]¶

List all artifacts of a given type.

Parameters

{str} name of artifact type. (artifact_type_name) –
a list of artifacts of the provided typed associated (Returns) –
this workspace. Each artifact is represented as a dict. (with) –

Return type

List[Artifact]

Returns

A list of Artifact objects.

kubeflow.metadata.metadata.abstractmethod(funcobj)[source]¶

A decorator indicating abstract methods.

Requires that the metaclass is ABCMeta or derived from it. A class that has a metaclass derived from ABCMeta cannot be instantiated unless all of its abstract methods are overridden. The abstract methods can be called using any of the normal ‘super’ call mechanisms.

Usage:

class C(metaclass=ABCMeta):
@abstractmethod def my_abstract_method(self, …):

…

kubeflow.metadata.metadata.retry(*dargs, **dkw)[source]¶: Decorator function that instantiates the Retrying object @param *dargs: positional arguments passed to Retrying object @param **dkw: keyword arguments passed to the Retrying object