Kserve-1.3

Step1. Minio 사용을 위한 Secret과 ServiceAccount 생성

이미 있다면 Skip

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: minio-secret    # Secret 명
  namespace: kubeflow-user-example-com    # Secret이 설치될 Namespace
  annotations:
    serving.kubeflow.org/s3-endpoint: minio-service.kubeflow:9000  # 설치된 Minio 서비스 endpoint
    serving.kubeflow.org/s3-usehttps: "0"   # http를 사용하지 않음 (설치된 Minio 설정)
    serving.kubeflow.org/s3-verifyssl: "0"  # ssy verify 하지 않음 (설치된 Minio 설정)
    serving.kubeflow.org/s3-region: us-east-0
type: Opaque
stringData:
  AWS_ACCESS_KEY_ID: minio
  AWS_SECRET_ACCESS_KEY: minio123
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kfserving-sa    # ServiceAccount 명
  namespace: kubeflow-user-example-com    # ServiceAccount가 설치될 Namespace
secrets:
- name: minio-secret    # 이 ServiceAccount가 사용할 Secret (위에서 생성한 Secret)
EOF

 

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: minio-secret                   # Secret 명
  namespace: kubeflow-user-example-com # Secret이 설치될 Namespace
  annotations:
    serving.kubeflow.org/s3-endpoint: minio-service.kubeflow:9000  # 설치된 Minio 서비스 endpoint
    serving.kubeflow.org/s3-usehttps: "0"   # http를 사용하지 않음 (설치된 Minio 설정)
    serving.kubeflow.org/s3-verifyssl: "0"  # ssy verify 하지 않음 (설치된 Minio 설정)
data:
  awsAccessKeyID: bWluaW8=           # 문자열 "minio"를 base64 인코딩
  awsSecretAccessKey: bWluaW8xMjM=   # 문자열 "minio123"를 base64 인코딩
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kfserving-sa                       # ServiceAccount 명
  namespace: kubeflow-user-example-com     # ServiceAccount가 설치될 Namespace
secrets:
- name: minio-secret                       # 이 ServiceAccount가 사용할 Secret (위에서 생성한 Secret)
EOF

 

*생성된 Secret 확인

$ kubectl get secrets -n kubeflow-user-example-com
NAME                         TYPE                                  DATA   AGE
default-editor-token-6hkgr   kubernetes.io/service-account-token   3      45h
default-token-dklvl          kubernetes.io/service-account-token   3      45h
default-viewer-token-4szx4   kubernetes.io/service-account-token   3      45h
kfserving-sa-token-bwfs4     kubernetes.io/service-account-token   3      23h
minio-secret                 Opaque                                2      23h
mlpipeline-minio-artifact    Opaque                                2      45h

* 생성된 Secret 상세 보기

kubectl describe secrets/minio-secret -n kubeflow-user-example-com
Name:         minio-secret
Namespace:    kubeflow-user-example-com
Labels:       <none>
Annotations:  serving.kubeflow.org/s3-endpoint: minio-service.kubeflow:9000
              serving.kubeflow.org/s3-region: us-east-0
              serving.kubeflow.org/s3-usehttps: 0
              serving.kubeflow.org/s3-verifyssl: 0

Type:  Opaque

Data
====
AWS_ACCESS_KEY_ID:      5 bytes
AWS_SECRET_ACCESS_KEY:  8 bytes

보안 문제로 시크릿 노출이 안된다. 확인하기 위해서는 다음과 같이 실행해야 한다.

$ kubectl get secret minio-secret -n kubeflow-user-example-com -o jsonpath='{.data}'
{"AWS_ACCESS_KEY_ID":"bWluaW8=","AWS_SECRET_ACCESS_KEY":"bWluaW8xMjM="}

 

* Secret 삭제

$ kubectl delete secret minio-secret -n kubeflow-user-example-com
secret "minio-secret" deleted

 

 

• 참고1: minio service 확인
  •  endpoit는 <Namespae>.<Service명>:<Port(내부)>
    • Namespae: kubeflow
    • Service명: minio-service
    • Port(내부): 9000

kubectl get svc -A | grep minio
kubeflow      minio-service       NodePort       10.99.254.139    <none>       9000:32001/TCP

 

Step2. InferenceService 배포

• kubeflow 1.4 에는 kfserving (inferenceservices.serving.kubeflow.org) 과 kserve (inferenceservices.serving.kserve.io) APIserver 가 모두 설치되어 있습니다.
• serving.kserve.io/v1beta1 사용 시 inferenceservices.serving.kserve.io 로 resource kind 를 가집니다.

kubectl 이용하여 배포

* kserve

cat << EOF | kubectl apply -f -
apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: covid-19                          # KFServing InferenceService 명
  namespace: kubeflow-user-example-com    # KFServing InferenceService가 배포될 Namespace
spec:
  predictor:
    serviceAccountName: kfserving-sa   # s3:// 레파지토리(Minio)를 사용하기 위해 이 KFServing InferenceService가 사용할 serviceAccount (Step1.에서 생성함)
    tensorflow:
      storageUri: 's3://model/covid-19/'  # 모델을 저장한 경로 s3://<bucket명>/<저장경로>
EOF

배포 확인

$ kubectl get inferenceservice -n kubeflow-user-example-com covid-19
NAME      URL                                                             READY   DEFAULT TRAFFIC   CANARY TRAFFIC   AGE
covid-19   http://covid-19.kubeflow-user-example-com/v1/models/covid-19   True    100                                125m

 

 

 

* kfserving

cat << EOF | kubectl apply -f -
apiVersion: serving.kubeflow.org/v1alpha2
kind: InferenceService
metadata:
  name: covid-19        # KFServing InferenceService 명
  namespace: kubeflow-user-example-com   # KFServing InferenceService가 배포될 Namespace
  annotations:
    sidecar.istio.io/inject: "false"
spec:
  default:
    predictor:
      serviceAccountName: kfserving-sa   # s3:// 레파지토리(Minio)를 사용하기 위해 이 KFServing InferenceService가 사용할 serviceAccount (Step1.에서 생성함)
      tensorflow:
        storageUri: 's3://model/covid-19/'  # 모델을 저장한 경로 s3://<bucket명>/<저장경로>
        resources:
          limits:
            cpu: 100m
            memory: 1Gi
          requests:
            cpu: 100m
            memory: 100Mi
EOF

*배포확인

$ kubectl get inferenceservices.serving.kubeflow.org -n kubeflow-user-example-com covid-19
NAME       URL                                                     READY   PREV   LATEST   PREVROLLEDOUTREVISION   LATESTREADYREVISION                AGE
covid-19   http://covid-19.kubeflow-user-example-com.example.com   True           100                              covid-19-predictor-default-00001   114s

 

kubeflow UI 를 통한 배포

Models > NEW MODEL SERVER 에 yaml 입력

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: covid-19                          # KFServing InferenceService 명
  namespace: kubeflow-user-example-com    # KFServing InferenceService가 배포될 Namespace
spec:
  predictor:
    serviceAccountName: kfserving-sa   # s3:// 레파지토리(Minio)를 사용하기 위해 이 KFServing InferenceService가 사용할 serviceAccount (Step1.에서 생성함)
    tensorflow:
      storageUri: 's3://model/covid-19/'  # 모델을 저장한 경로 s3://<bucket명>/<저장경로>

Step3. InferenceService 테스트

• kfserving-ingressgateway를 NodePort로 노출 (이미 변경하였다면 Skip)

kubectl edit svc -n istio-system kfserving-ingressgateway

...
  ports:
...
  - name: https-tracing
    nodePort: 31204
    port: 15032
    protocol: TCP
    targetPort: 15032
  - name: tls
    nodePort: 30103
    port: 15443
    protocol: TCP
    targetPort: 15443
  selector:
    app: kfserving-ingressgateway
    kfserving: ingressgateway
  sessionAffinity: None
  type: NodePort  # 변경
...

MODEL_NAME=covid-19
TEST_JSON="../01-prerequisite/image_data.json"
 
# Jupyter Notebook Terminal (K8s 내부)에서 실행 시
INGRESS_HOST=kfserving-ingressgateway.istio-system
INGRESS_PORT=80
 
# Host ssh 접속하여 (K8s 외부) 실행 시
# INGRESS_HOST=<VM 외부 IP>
# INGRESS_PORT=31380
 
SERVICE_HOSTNAME=$(kubectl get inferenceservice -n myspace $MODEL_NAME -o jsonpath='{.status.url}' | cut -d "/" -f 3)
SERVING_URL=http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/$MODEL_NAME:predict
 
curl -v -H "Host: ${SERVICE_HOSTNAME}" ${SERVING_URL} -d @${TEST_JSON}