Part 2: A Comprehensive Guide to Setting Up Spicedb-operator with PostgreSQL and a Monitoring Stack

3 min readAug 11, 2023

In part 1, we reviewed the SpiceDB setup with the monitoring stack. This post will set up the SpiceDB Operator with the monitoring stack on a locally running Kubernetes cluster using kind. Additional tips for handling CA certs for the AWS EKS deployment of Spicedb Operator with RDS.

Prerequisites:

kind
kubectl
zed (Spicedb client)
spicedb-operator
helm
grpcurl (optional)
git
Docker

Setup code repository

Start by cloning the spicedb-postgres-kube repository from GitHub using the following command:

git clone https://github.com/akoserwal/spicedb-postgres-kube.git

Change the director to spicedb-operator-deployment

cd spicedb-operator-deployment

Run the setup script

./setup.sh

The script will set up the spicedb-operator with a locally running PostgreSQL and expose ingress using the project contour operator.

kubectl cluster-info --context kind-sp-cluster

Thanks for using kind! 😊
> waiting for kubernetes node(s) become ready
node/sp-cluster-control-plane condition met
deploy contour
namespace/projectcontour created
serviceaccount/contour created
serviceaccount/envoy created
configmap/contour created
customresourcedefinition.apiextensions.k8s.io/contourconfigurations.projectcontour.io created
customresourcedefinition.apiextensions.k8s.io/contourdeployments.projectcontour.io created
customresourcedefinition.apiextensions.k8s.io/extensionservices.projectcontour.io created
customresourcedefinition.apiextensions.k8s.io/httpproxies.projectcontour.io created
customresourcedefinition.apiextensions.k8s.io/tlscertificatedelegations.projectcontour.io created
serviceaccount/contour-certgen created
rolebinding.rbac.authorization.k8s.io/contour created
role.rbac.authorization.k8s.io/contour-certgen created
job.batch/contour-certgen-v1.22.0 created
clusterrolebinding.rbac.authorization.k8s.io/contour created
rolebinding.rbac.authorization.k8s.io/contour-rolebinding created
clusterrole.rbac.authorization.k8s.io/contour created
role.rbac.authorization.k8s.io/contour created
service/contour created
service/envoy created
deployment.apps/contour created
daemonset.apps/envoy created
Install spicedb-operator
namespace/spicedb-operator created
namespace/spicedb-operator serverside-applied
customresourcedefinition.apiextensions.k8s.io/spicedbclusters.authzed.com serverside-applied
serviceaccount/spicedb-operator serverside-applied
clusterrole.rbac.authorization.k8s.io/spicedb-operator serverside-applied
clusterrole.rbac.authorization.k8s.io/spicedb-operator-edit serverside-applied
clusterrole.rbac.authorization.k8s.io/spicedb-operator-view serverside-applied
clusterrolebinding.rbac.authorization.k8s.io/spicedb-operator serverside-applied
deployment.apps/spicedb-operator serverside-applied
create spicedb namespace
namespace/spicedb created
deploy postgres
secret/postgres-credentials created
persistentvolume/postgres-pv created
persistentvolumeclaim/postgres-pvc created
service/postgres created
deployment.apps/postgres created
deploy spicedb
spicedbcluster.authzed.com/spicedb-cr created
secret/spicedb-config created
ingress.networking.k8s.io/spicedb-https created
ingress.networking.k8s.io/spicedb-prometheus created
ingress.networking.k8s.io/spicedb-grpc created
httpproxy.projectcontour.io/spicedb created
still waiting for spicedb
still waiting for spicedb

still waiting for spicedb

spicedb is ready
NAME                 CLASS    HOSTS                             ADDRESS   PORTS   AGE
spicedb-grpc         <none>   spicedb-grpc.127.0.0.1.nip.io               80      46s
spicedb-https        <none>   spicedb-http.127.0.0.1.nip.io               80      46s
spicedb-prometheus   <none>   spicedb-metric.127.0.0.1.nip.io             80      46s
Install kube prometheus
NAME: kube-prometheus-stack
LAST DEPLOYED: Fri Aug 11 19:16:41 2023
NAMESPACE: monitoring
STATUS: deployed
REVISION: 1
NOTES:
kube-prometheus-stack has been installed. Check its status by running:
  kubectl --namespace monitoring get pods -l "release=kube-prometheus-stack"

Visit https://github.com/prometheus-operator/kube-prometheus for instructions on how to create & configure Alertmanager and Prometheus instances using the Operator.
pod/kube-prometheus-stack-kube-state-metrics-78c9594f8f-hfj2s condition met
pod/kube-prometheus-stack-operator-7845d55588-4wj7b condition met
pod/kube-prometheus-stack-prometheus-node-exporter-xwmnw condition met
Setup service monitor for spicedb
servicemonitor.monitoring.coreos.com/spicedb-cr-service-monitor created
ingress.networking.k8s.io/prom created
ingress.networking.k8s.io/grafana created
spicedb endpoints
NAME                 CLASS    HOSTS                             ADDRESS   PORTS   AGE
spicedb-grpc         <none>   spicedb-grpc.127.0.0.1.nip.io               80      112s
spicedb-https        <none>   spicedb-http.127.0.0.1.nip.io               80      112s
spicedb-prometheus   <none>   spicedb-metric.127.0.0.1.nip.io             80      112s
grafana and promethues
NAME      CLASS    HOSTS                      ADDRESS   PORTS   AGE
grafana   <none>   grafana.127.0.0.1.nip.io             80      0s
prom      <none>   prom.127.0.0.1.nip.io                80      0s

Verify the SpiceDB GRPC endpoint with the help of grpcurl command line.

$ grpcurl -plaintext spicedb-grpc.127.0.0.1.nip.io:80 list
authzed.api.v1.ExperimentalService
authzed.api.v1.PermissionsService
authzed.api.v1.SchemaService
grpc.health.v1.Health
grpc.reflection.v1alpha.ServerReflection

Verify the SpiceDB metrics in Prometheus: prom.127.0.0.1.nip.io

We have reviewed how to set up Spicedb Operator with PostgreSQL and a monitoring stack on a local Kubernetes cluster.

Deploying Spicedb Operator to AWS EKS with RDS backend

Update the “datastore_uri” in Spicedb-Cr

postgresql://postgres:PASSWORD@<rds-host>.amazonaws.com:5432/postgres?sslmode=verify-full&sslrootcert=/spicedb-db-tls/ca.crt

Create a secret with AWS certificates downloaded from https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html (https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem)

# apiVersion: v1
# kind: Secret
# metadata:
#   name: datastore-tls
#   namespace: spicedb
# stringData:
#   ca.crt: |
#     -----BEGIN CERTIFICATE-----
#    --- example aws rds ca cert
#     -----END CERTIFICATE-----
# ---

You can install the operator using the Spicedb-CR and RDS as the database. Ingress routes configuration will differ as compared to local setup.

Conclusion

I hope you find this post useful and enable you to set up spicedb-operator with a monitoring stack.

Please give me a cheer if you like the post. Thank you for reading!