Skip to content

Commit 9675480

Browse files
Nick-Harveyjlewi
authored andcommitted
minor update to the pachyderm seldon example (kubeflow#562)
* minor update to the pachyderm seldon exaple * Another minor update to the pipeline
1 parent a046db5 commit 9675480

File tree

2 files changed

+35
-5
lines changed

2 files changed

+35
-5
lines changed

github_issue_summarization/Pachyderm_Example/README.md

+33-3
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ If you get stuck on the example, please reach out the community via:
4242

4343
We also include some [resources](#resources) at the bottom of the tutorial, so you can dig in a little deeper.
4444

45-
## 1. Deploy KubeFlow
45+
## 1. Deploy KubeFlow and setup Ksonnet
4646

4747
The best instructions for deploying KubeFlow live on [KubeFlow.org](https://www.kubeflow.org). Follow the directions listed in the [Getting Started](https://www.kubeflow.org/docs/started/getting-started/) section for the platform of your choice (minikube, microk8s, GKE, etc.).
4848

@@ -52,6 +52,33 @@ The best instructions for deploying KubeFlow live on [KubeFlow.org](https://www.
5252
$ export NAMESPACE=kubeflow
5353
```
5454

55+
If you don't already have ksonnet setup on your local machine you'll want to install it via their getting started here: [ksonnet get started](https://ksonnet.io/get-started/)
56+
57+
Next, you'll want to export a few environment variables. You'll need a Github API token and if you don't already have one, you can get one [here](https://github.com/settings/tokens)
58+
59+
```
60+
$ export NAMESPACE=kubeflow
61+
$ export USER=<your github user id>
62+
$ export GITHUB_TOKEN=<your github token>
63+
$ export APP_NAME=my-kubeflow
64+
```
65+
66+
Now you're ready to proceed with the ksonnet setup.
67+
68+
```
69+
$ ks init ${APP_NAME}
70+
$ cd ${APP_NAME}
71+
$ ks env set default --namespace ${NAMESPACE}
72+
```
73+
It's recommended to install the core Kubeflow infrastructure, which includes the ability to train models with a TFJob CRD. In addition to that, we are going to go ahead and add in the Pachyderm and Seldon components:
74+
75+
```
76+
$ ks registry add kubeflow github.com/katacoda/kubeflow-ksonnet/tree/master/kubeflow
77+
$ ks pkg install kubeflow/core
78+
$ ks pkg install kubeflow/seldon
79+
$ ks pkg install kubeflow/pachyderm
80+
```
81+
5582
## 2. Deploy Pachyderm and Seldon on top of KubeFlow
5683

5784
We will be using Pachyderm and Seldon for data storage, versioning, pipelining, and serving. We already added these components to our ksonnet app, we just need to deploy/prep them for use.
@@ -79,6 +106,9 @@ seldon-cluster-manager-7f5ddbcf7d-trvfp 1/1 Running 0 1m
79106
To deploy Pachyderm, we follow a similar pattern:
80107

81108
```
109+
# Setup permissions for GKE. For local deployments just setup with the appropriate user
110+
kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=$(gcloud config get-value account)
111+
82112
# generate the template
83113
$ ks generate pachyderm pachyderm
84114
@@ -101,8 +131,8 @@ And you should be able to communicate with Pachyderm via their CLI `pachctl`:
101131
```
102132
$ pachctl version
103133
COMPONENT VERSION
104-
pachctl 1.7.5
105-
pachd 1.7.5
134+
pachctl 1.8.*
135+
pachd 1.8.*
106136
```
107137

108138
## 3. Create a versioned data repository with the training data set

github_issue_summarization/Pachyderm_Example/build.json

+2-2
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,13 @@
1515
"input": {
1616
"cross": [
1717
{
18-
"atom": {
18+
"pfs": {
1919
"repo": "train",
2020
"glob": "/"
2121
}
2222
},
2323
{
24-
"atom": {
24+
"pfs": {
2525
"repo": "pre_process",
2626
"glob": "/"
2727
}

0 commit comments

Comments
 (0)