Running Kubernetes on OCI Free Tier (Part 2): Multi-Cluster GitOps with Flux

In Part 1 I covered standing up a Kubernetes cluster on Oracle Cloud’s Always Free Tier — the OKE cluster, ARM nodes, and what you actually get for zero dollars. This post is about the part that made the whole thing worth building: managing both clusters from a single Git repository with Flux CD.

The Problem

I already had a homelab cluster running on Proxmox with Talos Linux. It was fully GitOps’d with Flux — HelmReleases, Kustomizations, SOPS-encrypted secrets, the works. When I spun up the OCI cluster, I didn’t want to maintain a second repo with duplicated manifests. I wanted one repo where shared app definitions live in a base directory, and each cluster pulls what it needs with environment-specific patches.

Turns out kustomize overlays are perfect for this.

Repository Structure

Here’s how the Flux repo is organized:

johnylab-flux/
  base/                          # Shared, reusable components
    apps/uptimekuma/             # App definition (HelmRelease, PVC, secrets, CronJob)
    infrastructure/              # cert-manager, external-dns, cilium, etc.
    networking/cloudflare/       # Tunnel configs

  clusters/
    homelab/                     # Proxmox/Talos cluster overlays
      infrastructure/            # Homelab-specific sources, secrets, patches
      network/                   # MetalLB, gateway configs

    oci/                         # OCI cluster overlays
      infrastructure/            # OCI-specific patches (remove nodeSelector, etc.)
      cloudflare/                # OCI tunnel config
      apps/uptimekuma/           # Kustomize overlay with OCI patches

Each cluster’s Flux instance is bootstrapped to its own path. The homelab Flux watches clusters/homelab/, the OCI Flux watches clusters/oci/. They never touch each other’s resources. The base/ directory is just regular kustomize bases that get referenced by overlays — Flux doesn’t sync it directly.

Overlay Pattern Example: Uptime Kuma

This is where it gets practical. Uptime Kuma is deployed on both clusters — the OCI instance monitors all homelab services from an external vantage point, so if my homelab goes down, OCI still reports it.

The base defines everything: the HelmRelease, PersistentVolumeClaim, SOPS-encrypted secrets, a CronJob that syncs monitor definitions from a shared ConfigMap. The OCI overlay references that base and applies JSON patches to change only what differs between environments.

Here’s what actually changes:

Setting	Base (Homelab)	OCI Override
Ingress class	`cilium`	`nginx`
Hostname	`uptimekuma-internal.xjohnyx.me`	`uptimekuma-oci.xjohnyx.me`
TLS secret	`uptimekuma-internal-tls`	`uptimekuma-oci-tls`
Node affinity	Proxmox preference	Removed
PVC storageClass	`synology-nfs`	`oci-bv`
PVC size	`4Gi`	`2Gi`

That last one is funny — OCI provisions a minimum of 50Gi of block storage regardless of what you request in the PVC. So 2Gi becomes 50Gi on disk. No cost impact on free tier, just surprising the first time you see it.

Everything else is reused as-is: the container image, resource limits, PodDisruptionBudget, SOPS secrets, the monitor sync CronJob and its ConfigMaps. The overlay is maybe 30 lines of YAML. The base is several hundred.

Flux Dependency Ordering

Both clusters use Flux Kustomizations with dependsOn to control reconciliation order, but the dependency chains look different because the infrastructure is different.

Homelab has a deeper chain:

infra -> net (MetalLB, gateways) -> cert-manager-issuers -> cloudflare -> apps

MetalLB has to be healthy before Cilium can assign LoadBalancer IPs to its shared Ingress. cert-manager has to be running before you can create ClusterIssuers. The Cloudflare tunnel needs working DNS before apps can be reached externally. Each step depends on the previous one.

OCI is flatter:

infra-oci -> cert-manager-issuers-oci -> apps-oci
          -> cloudflare-oci

The infra-oci Kustomization bundles more into a single step — namespaces, Helm sources, cert-manager, external-dns, and nginx-ingress all deploy together. There’s no MetalLB step because OCI uses a cloud-managed load balancer that the OCI Cloud Controller Manager provisions automatically. Fewer moving parts means fewer dependency stages.

The trick that made this work cleanly: keeping the Kustomization names distinct per cluster (e.g., infra-oci vs infra, apps-oci vs apps). This way I can tell at a glance which cluster a Flux resource belongs to when reading flux get kustomizations output.

What Connects the Two Clusters

Here’s what surprised people when I described this setup: the clusters have zero direct network connectivity. No VPN, no mesh, no federation. They’re completely independent Kubernetes clusters that happen to share some common touchpoints:

DNS domain — Both clusters use xjohnyx.me. Each runs its own ExternalDNS instance with a separate txtOwnerId (johnylab-flux vs oci-flux) so they don’t fight over TXT record ownership in Cloudflare.
Cloudflare account — Separate tunnels per cluster, separate credentials. The homelab tunnel uses per-hostname routing rules; the OCI tunnel uses a single wildcard *.xjohnyx.me rule pointing at nginx-ingress.
Git repository — Same Flux repo, different paths. This is the core of the multi-cluster pattern.
SOPS Age key — Both clusters decrypt secrets with the same Age key. One key to rule them all. This means I can define a secret once in base/ and both clusters can decrypt it.
Terraform — A single Terraform workspace manages tunnel infrastructure and DNS records for both clusters. One terraform apply updates everything.

That’s the entire integration surface. No service mesh spanning clusters, no cross-cluster service discovery. Each cluster is fully self-contained. The Git repo is the only thing that ties them together operationally.

Why This Pattern Works

The kustomize overlay approach scales well because adding a new app to the OCI cluster follows a repeatable process:

Define the app in base/apps/<app-name>/ with all the manifests
Create clusters/oci/apps/<app-name>/kustomization.yaml referencing the base
Add JSON patches for OCI-specific values (ingress class, storage class, hostnames)
Push to Git

Flux picks it up, reconciles, done. The base manifests get validated on the homelab first (since that’s where I develop), and the OCI overlay only changes what’s environment-specific.

What’s Next

In Part 3, I’ll dig into the networking differences between the two clusters — Cilium eBPF vs Flannel iptables, how external traffic enters each cluster through Cloudflare Tunnels, and why the OCI cluster’s wildcard tunnel pattern is actually simpler than the homelab’s per-hostname approach.

Infrastructure: Terraform + OCI Provider | GitOps: Flux CD v2.7 | Kubernetes: OKE v1.32 on ARM64 | Secrets: SOPS + Age