Use Pub/Sub emulator in minikube

In a previous blog post Things I wish I knew about Pub/Sub, Part 3, I introduced Pub/Sub emulator for free local development and testing, and shared how to use it with Docker, which has gotten quite a few highlights from the community. In this blog post, I’m revisiting this topic on local testing with a focus on minikube.

Before I jump into the quickstart, let me quickly describe the diagram below, which is created by the quickstart.

Pub/Sub emulator deployed in minikube

There is just one node in this example deployment because minikube, by design, is a single-node Kubernetes cluster on your local machine. Inside this node, there is a Pub/Sub service of the name pubsub-service and its permanent IP. One pod has been deployed to run a containerized Pub/Sub emulator with an ephemeral IP.

The following is a step-by-step guide to start a Pub/Sub emulator on a minikube cluster as shown in the diagram:

STEP 1️⃣

Follow minikube documentation to install and start minikube.

Make sure minikube start is successfully running before you proceed.

STEP 2️⃣

In one terminal, apply the following minikube configuration to create a deployment named gcp-emulator-deployment and a service named pubsub-service.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gcp-emulator-deployment
  labels:
    app: pubsub
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pubsub
  template:
    metadata:
      labels:
        app: pubsub
    spec:
      containers:
        - name: pubsub-emulator
          image: gcr.io/google.com/cloudsdktool/cloud-sdk:414.0.0-emulators
          command: ["gcloud", "beta", "emulators", "pubsub", "start"]
          args: ["--project=abc", "--host-port=0.0.0.0:8085"]
---
apiVersion: v1
kind: Service
metadata:
  name: pubsub-service
spec:
  type: NodePort
  ports:
    - protocol: TCP
      port: 8008
      targetPort: 8085
      nodePort: 30001
  selector:
    app: pubsub

A few notes worth mentioning:

• The service pubsub-service is a NodePort type service. It’s not secure but it’s easy to use and great for demo purposes. You do not need to assign a port number to nodePort, which defaults to 30000–32767. I picked 30001 for illustration purposes only.
• The host port of the Pub/Sub emulator is set to be 0.0.0.0:8085. It means that the emulator is always accessible on container port 8085 no matter what the external IP that the pod has. targetPort must match the port number that the Pub/Sub emulator is listening to.
• At the time of this blog post creation, gcr.io/google.com/cloudsdktool/cloud-sdk:414.0.0-emulators is used. But you can check out the latest release version for the emulator, which is packaged in the gcloud SDK, in this Google Cloud public container registry. The image containers all the officially released emulators from Google Cloud and is approximately 1.1 GB.

Feel free to copy and paste the above configuration into your own pubsub-emulator-minikube.yaml, or clone mine on GitHub.

$ git clone https://github.com/anguillanneuf/pubsub-emulator-minikube.git
$ cd pubsub-emulator-minikube
$ kubectl apply -f pubsub-emulator-minikube.yaml

If the configuration file has been applied successfully, you should see:

deployment.apps/gcp-emulator-deployment created
service/pubsub-service created

STEP 3️⃣

Check if a pod is ready for the service. If it is, it should have an IP address assigned.

$ kubectl get service
$ kubectl describe service pubsub-service

Here is an example output where an IP address is ready in Endpoints:

Name: pubsub-service
 Namespace: default
 Labels: <none>
 Annotations: <none>
 Selector: app=pubsub
 Type: NodePort
 IP: 10.103.142.157
 Port: <unset> 8008/TCP
 TargetPort: 8085/TCP
 NodePort: <unset> 30001/TCP
 Endpoints: 172.17.0.4:8085
 Session Affinity: None
 External Traffic Policy: Cluster
 Events: <none>

A few notes:

• 172.17.0.4:8085 is the IP address assigned to the pod.
• 10.103.142.157:8008 is the internal IP and port number of the service.

STEP 4️⃣

Assign an external IP address to the service to which requests can be send.

$ minikube service pubsub-service

In the following example output, 192.168.49.2:30001 is the external IP address of the node and 127.0.0.1:64274 is the external IP address of the service.

| - - - - - -| - - - - - - - - | - - - - - - -| - - - - - - - - - - - - - -|
| NAMESPACE | NAME | TARGET PORT | URL |
| - - - - - -| - - - - - - - - | - - - - - - -| - - - - - - - - - - - - - -|
| default | pubsub-service | 8008 | http://192.168.49.2:30001 |
| - - - - - -| - - - - - - - - | - - - - - - -| - - - - - - - - - - - - - -|
🏃 Starting tunnel for service pubsub-service.
| - - - - - -| - - - - - - - - | - - - - - - -| - - - - - - - - - - - - |
| NAMESPACE | NAME | TARGET PORT | URL |
| - - - - - -| - - - - - - - - | - - - - - - -| - - - - - --------------|
| default | pubsub-service | | http://127.0.0.1:64274 |
| - - - - - -| - - - - - - - - | - - - - - - -| - - - - - - - - - - - - |
🎉 Opening service default/pubsub-service in default browser…
❗ Because you are using a Docker driver on darwin, the terminal needs to be open to run it.

STEP 5️⃣

In a second terminal, set the following environment variables before you create a topic and subscription in a dummy project and test publishing and receiving messages. This simulates sending requests from the outside world to the minikube cluster.

$ git cloud https://github.com/googleapis/python-pubsub.git
$ cd python-pubsub/samples/snippets/

$ export PUBSUB_PROJECT_ID=abc
$ export PUBSUB_EMULATOR_HOST=127.0.0.1:64274

$ python -m pip install --upgrade google-cloud-pubsub
$ python publisher.py abc create my-topic
$ python subscriber.py abc create my-topic my-sub
$ python publisher.py abc publish my-topic

Note:

• PUBSUB_PROJECT_ID must match the project ID used in args in pubsub-emulator-minikube.yaml. In this step-by-step guide, abc is used.
• Use the external IP of pubsub-service obtained in the previous step for PUBSUB_EMULATOR_HOST.
• Python is used here but feel free to program in any of the officially supported languages.

CLEAN UP 🫧

Stop minikube (this will stop the running Docker container) and optionally remove the Docker container and its image.

$ minikube stop
$ docker ps
$ docker ps -a
$ docker rm $(CONTAINER_ID)
$ docker images
$ docker rmi $(IMAGE_ID)

Before you go, please remember that the Pub/Sub emulator is written in Java and is designed to work on a single machine, so only single-node one-pod deployment can be achieved with minikube. The resources created by the Pub/Sub emulator, including topics, subscriptions, schemas, etc., are simple Java hashmaps that don’t communicate across machines. Therefore, using the emulator in a distributed way is not supported.

Lastly, if you want to see a guide on how to use the Pub/Sub emulator with other Google Cloud emulators such as the Spanner emulator, please leave a comment. Hope to get some signals from the developer community of your most interested use cases.