Metadata Package¶
-
class
kubeflow.metadata.metadata.
ABC
[source]¶ Bases:
object
Helper class that provides a standard way to create an ABC using inheritance.
-
class
kubeflow.metadata.metadata.
Artifact
[source]¶ Bases:
abc.ABC
-
abstract property
ARTIFACT_TYPE_NAME
¶ classmethod(function) -> method
Convert a function to be a class method.
A class method receives the class as implicit first argument, just like an instance method receives the instance. To declare a class method, use this idiom:
- class C:
@classmethod def f(cls, arg1, arg2, …):
…
It can be called either on the class (e.g. C.f()) or on an instance (e.g. C().f()). The instance is ignored except for its class. If a class method is called for a derived class, the derived class object is passed as the implied first argument.
Class methods are different than C++ or Java static methods. If you want those, see the staticmethod builtin.
-
static
is_duplicated
(a, b)[source]¶ Checks if two artifacts are duplicated.
The artifacts may be considered duplication even if not all the fields are the same as in mlpb.Artifact. For example, two models can be considered the same if they have the same uri, name and version.
- Returns
True or False for duplication.
-
abstract property
-
class
kubeflow.metadata.metadata.
DataSet
(uri=None, name=None, workspace=None, description=None, owner=None, version=None, query=None, labels=None, **kwargs)[source]¶ Bases:
kubeflow.metadata.metadata.Artifact
Dataset captures a data set in a machine learning workflow.
-
uri
¶ Required uri of the data set.
-
name
¶ Required name of the data set.
-
workspace
¶ Optional name of the workspace.
-
description
¶ Optional description of the data set.
-
owner
¶ Optional owner of the data set.
-
version
¶ Optional version tagged by the user.
-
query
¶ Optional query string on how this data set being fetched from a data source.
-
labels
¶ Optional string key value pairs for labels.
Example
>>> metadata.DataSet(description="an example data", ... name="mytable-dump", ... owner="owner@my-company.org", ... uri="file://path/to/dataset", ... version="v1.0.0", ... query="SELECT * FROM mytable", ... labels={"label1","val1"}))
-
ARTIFACT_TYPE_NAME
= 'kubeflow.org/alpha/data_set'¶
-
static
is_duplicated
(a, b)[source]¶ Checks if two artifacts are duplicated.
The artifacts may be considered duplication even if not all the fields are the same as in mlpb.Artifact. For example, two models can be considered the same if they have the same uri, name and version.
- Returns
True or False for duplication.
-
-
class
kubeflow.metadata.metadata.
Execution
(name=None, workspace=None, run=None, description=None)[source]¶ Bases:
object
Captures a run of pipeline or notebooks in a workspace and group executions.
Execution also serves as object for logging artifacts as its input or output.
-
EXECUTION_TYPE_NAME
= 'kubeflow.org/alpha/execution'¶
-
-
class
kubeflow.metadata.metadata.
Metrics
(uri=None, name=None, workspace=None, description=None, owner=None, data_set_id=None, model_id=None, metrics_type=None, values=None, labels=None, **kwargs)[source]¶ Bases:
kubeflow.metadata.metadata.Artifact
Captures an evaluation metrics of a model on a data set.
-
uri
¶ Required uri of the metrics.
-
name
¶ Required name of the metrics.
-
workspace
¶ Optional name of the workspace.
-
description
¶ Optional description of the metrics.
-
owner
¶ Optional owner of the metrics.
-
data_set_id
¶ Optional id of the data set used for evaluation.
-
model_id
¶ Optional id of a evaluated model.
-
metrics_type
¶ Optional type of the evaluation.
-
values
¶ Optional map from metrics name to its value.
-
labels
¶ Optional string key value pairs for labels.
Example
>>> metadata.Metrics( ... name="MNIST-evaluation", ... description= ... "validating the MNIST model to recognize handwritten digits", ... owner="someone@kubeflow.org", ... uri="gcs://my-bucket/mnist-eval.csv", ... data_set_id="123", ... model_id="12345", ... metrics_type=metadata.Metrics.VALIDATION, ... values={"accuracy": 0.95}, ... labels={"mylabel": "l1"}))
-
ARTIFACT_TYPE_NAME
= 'kubeflow.org/alpha/metrics'¶
-
PRODUCTION
= 'production'¶
-
TESTING
= 'testing'¶
-
TRAINING
= 'training'¶
-
VALIDATION
= 'validation'¶
-
-
class
kubeflow.metadata.metadata.
Model
(uri=None, name=None, workspace=None, description=None, owner=None, version=None, model_type=None, training_framework=None, hyperparameters=None, labels=None, **kwargs)[source]¶ Bases:
kubeflow.metadata.metadata.Artifact
Captures a machine learning model.
-
uri
¶ Required uri of the model artifact, e.g. “gcs://path/to/model.h5”.
-
name
¶ Required name of the model.
-
workspace
¶ Optional name of the workspace.
-
description
¶ Optional description of the model.
-
owner
¶ Optional owner of the model.
-
model_type
¶ Optional type of the model.
-
training_framework
¶ Optional framework used to train the model.
-
hyperparameters
¶ Optional map from hyper param name to its value.
-
labels
¶ Optional string key value pairs for labels.
-
kwargs
¶ Optional additional keyword arguments are saved as additional properties of this model.
Example
>>> metadata.Model(name="MNIST", ... description="model to recognize handwritten digits", ... owner="someone@kubeflow.org", ... uri="gcs://my-bucket/mnist", ... model_type="neural network", ... training_framework={ ... "name": "tensorflow", ... "version": "v1.0" ... }, ... hyperparameters={ ... "learning_rate": 0.5, ... "layers": [10, 3, 1], ... "early_stop": True ... }, ... version="v0.0.1", ... labels={"mylabel": "l1"}))
-
ARTIFACT_TYPE_NAME
= 'kubeflow.org/alpha/model'¶
-
static
is_duplicated
(a, b)[source]¶ Checks if two artifacts are duplicated.
The artifacts may be considered duplication even if not all the fields are the same as in mlpb.Artifact. For example, two models can be considered the same if they have the same uri, name and version.
- Returns
True or False for duplication.
-
-
class
kubeflow.metadata.metadata.
Run
(workspace=None, name=None, description=None)[source]¶ Bases:
object
Run captures a run of pipeline or notebooks in a workspace and group executions.
-
class
kubeflow.metadata.metadata.
Store
(grpc_host='metadata-grpc-service.kubeflow', grpc_port=8080, root_certificates=None, private_key=None, certificate_chain=None)[source]¶ Bases:
object
Metadata Store that connects to the Metadata gRPC service.
-
class
kubeflow.metadata.metadata.
Workspace
(store=None, name=None, description=None, labels=None, reuse_workspace_if_exists=True, backend_url_prefix=None)[source]¶ Bases:
object
Groups a set of runs of pipelines, notebooks and their related artifacts and executions.
-
CONTEXT_TYPE_NAME
= 'kubeflow.org/alpha/workspace'¶
-
list
(artifact_type_name=None)[source]¶ List all artifacts of a given type.
- Parameters
{str} name of artifact type. (artifact_type_name) –
a list of artifacts of the provided typed associated (Returns) –
this workspace. Each artifact is represented as a dict. (with) –
- Return type
List
[Artifact
]- Returns
A list of Artifact objects.
-
-
kubeflow.metadata.metadata.
abstractmethod
(funcobj)[source]¶ A decorator indicating abstract methods.
Requires that the metaclass is ABCMeta or derived from it. A class that has a metaclass derived from ABCMeta cannot be instantiated unless all of its abstract methods are overridden. The abstract methods can be called using any of the normal ‘super’ call mechanisms.
Usage:
- class C(metaclass=ABCMeta):
@abstractmethod def my_abstract_method(self, …):
…