Quickstart Guide
Welcome to the KServe Quickstart Guide! This guide will help you set up a KServe Quickstart environment for testing and experimentation.
By the end of this guide, you will have a fully functional KServe environment ready for experimentation.
KServe Quickstart Environments are for experimentation use only. For production installation, see our Administrator's Guide.
Prerequisites
Before you can get started with a KServe Quickstart deployment, you will need to ensure you have the following prerequisites installed:
Tools
Make sure you have the following tools installed:
- kubectl - The Kubernetes command-line tool
- helm - for installing KServe and other Kubernetes operators
- git - for cloning the KServe repository
Run the following commands to verify that you have the required tools installed:
To verify kubectl installation, run:
kubectl version --client
To verify helm installation, run:
helm version
To verify git installation, run:
git --version
Kubernetes Cluster
Kubernetes version 1.32 or higher is required.
You will need a running Kubernetes cluster with properly configured kubeconfig to run KServe. You can use any Kubernetes cluster, but for local development and testing, we recommend using kind (Kubernetes in Docker) or minikube.
- Local Cluster (Kind/Minikube)
- Existing Kubernetes Cluster
Using Kind (Kubernetes in Docker):
If you want to run a local Kubernetes cluster, you can use Kind. It allows you to create a Kubernetes cluster using Docker container nodes.
First, ensure you have Docker installed on your machine. Install Kind by following the Kind Quick Start Guide if you haven't done so already.
Then, you can create a local Kubernetes cluster with the following command:
kind create cluster
Using Minikube:
If you prefer to use Minikube, you can follow the Minikube Quickstart Guide to set up a local Kubernetes cluster.
First, ensure you have Minikube installed on your machine. Then, you can start a local Kubernetes cluster with the following command:
minikube start
If you have access to an existing Kubernetes cluster, you can use that as well. Ensure that your kubeconfig is properly configured to connect to the cluster. You can verify your current context with:
kubectl config current-context
Verify your cluster meets the version requirements by running:
kubectl version --output=json
The server version in the output should show version 1.32 or higher:
{
"serverVersion": {
"major": "1",
"minor": "32",
...
}
}
Install KServe Quickstart Environment
Once you have the prerequisites installed and a Kubernetes cluster running, you can proceed with the KServe Quickstart installation.
First, clone the KServe repository:
git clone https://github.com/kserve/kserve.git
cd kserve
Then choose your installation scenario:
- KServe Only
- KServe + LocalModel
- LLMIsvc Only
- KServe + LLMIsvc + LocalModel
- Dependencies Only
Install KServe controller only without additional components.
Using Quick Install Script
# Standard mode
curl -s "https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/kserve-standard-mode-full-install-with-manifests.sh" | bash
# Knative mode (serverless)
curl -s "https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/kserve-knative-mode-full-install-with-manifests.sh" | bash
Using kserve-install.sh
Helm Mode:
# Standard mode
./hack/kserve-install.sh --kserve-version v0.17.0 --type kserve --standard
# Knative mode (serverless)
./hack/kserve-install.sh --kserve-version v0.17.0 --type kserve --knative
Kustomize Mode:
# Standard mode
./hack/kserve-install.sh --kustomize --standard
# Knative mode (serverless)
./hack/kserve-install.sh --kustomize --knative
Install KServe controller with LocalModel support for local model caching.
Using kserve-install.sh
Helm Mode:
# Standard mode
./hack/kserve-install.sh --kserve-version v0.17.0 --type kserve,localmodel --standard
# Knative mode (serverless)
./hack/kserve-install.sh --kserve-version v0.17.0 --type kserve,localmodel --knative
Kustomize Mode:
# Standard mode
./hack/kserve-install.sh --kustomize --standard --type kserve,localmodel
# Knative mode (serverless)
./hack/kserve-install.sh --kustomize --knative --type kserve,localmodel
Install LLMInferenceService controller only for Generative AI model serving.
Using Quick Install Script
curl -s "https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/llmisvc-full-install-with-manifests.sh" | bash
Using kserve-install.sh
Helm Mode:
./hack/kserve-install.sh --kserve-version v0.17.0 --type llmisvc
Kustomize Mode:
# LLMIsvc
./hack/kserve-install.sh --kustomize --type llmisvc
Install all KServe components together.
Using Quick Install Script
Step 1: Install KServe Dependencies (Choose ONE)
Choose either Standard mode or Knative mode:
# Option A: Standard Mode Dependencies
curl -s "https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/kserve-standard-mode-dependency-install.sh" | bash
# OR
# Option B: Knative Mode Dependencies
curl -s "https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/kserve-knative-mode-dependency-install.sh" | bash
Step 2: Install All Components
After installing dependencies above, run these commands:
# Install LLMInferenceService Dependencies
curl -s "https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/llmisvc-dependency-install.sh" | bash
# Install CRDs
kubectl apply -f https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/kserve-crds.yaml
# Install Controllers
kubectl apply -f https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/kserve.yaml
# Install ClusterServingRuntimes/LLMIsvcConfigs
kubectl apply -f https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/kserve-cluster-resources.yaml
Using kserve-install.sh
Helm Mode:
# All components (Standard mode)
./hack/kserve-install.sh --kserve-version v0.17.0 --type kserve,llmisvc,localmodel --standard
# All components (Knative mode)
./hack/kserve-install.sh --kserve-version v0.17.0 --type kserve,llmisvc,localmodel --knative
Kustomize Mode:
# All components (Standard mode)
./hack/kserve-install.sh --kustomize --standard --type kserve,llmisvc,localmodel
# All components (Knative mode)
./hack/kserve-install.sh --kustomize --knative --type kserve,llmisvc,localmodel
Install only infrastructure dependencies without KServe components.
Using Quick Install Script
Standard Mode Dependencies:
curl -s "https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/kserve-standard-mode-dependency-install.sh" | bash
Knative Mode Dependencies:
curl -s "https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/kserve-knative-mode-dependency-install.sh" | bash
LLMIsvc Dependencies:
curl -s "https://raw.githubusercontent.com/kserve/kserve/refs/tags/v0.17.0/install/v0.17.0/llmisvc-dependency-install.sh" | bash
Using kserve-install.sh
Standard Dependencies
./hack/kserve-install.sh --deps-only --standard
Knative Dependencies
./hack/kserve-install.sh --deps-only --knative
LLMIsvc Dependencies:
./hack/kserve-install.sh --deps-only --type llmisvc
For detailed installation instructions, customization options, and troubleshooting:
Next Steps
Now that you have a KServe Quickstart environment set up, you can start deploying and testing machine learning models:
- 📖 First GenAI InferenceService - Deploy your first GenAI model using InferenceService
- 📖 First Predictive InferenceService - Deploy your first predictive model using InferenceService