OAuth2 Proxy and Traefik logos.

This is how to protect your website with Google’s OAuth 2.0, using pusher/oauth2_proxy behind a containous/traefik cloud native edge router.

Objectives

For this tutorial, we will build the following on an existing docker swarm cluster:

containous/traefik will receive all http and https requests
pusher/oauth2_proxy will authenticate only the requests for the protected domains
alex.ix.ai will be protected (Sign in with Google)
alex.thom.ae will be unprotected
oauth.ix.ai will handle the OAUTH responses
The authentication domain is .ix.ai (the cookie domain)

A couple of more goodies will be added as well:

all HTTP traffic will be redirected to HTTPS
automates certificate generation with Let’s Encrypt for the domains hosted with Cloudflare
using docker secrets to manage the Cloudflare credentials
using docker configs to manage the traefik configuration files
not exposing the docker socket directly to traefik, but over Tecnativa/docker-socket-proxy
chaining of traefik middlewares, so that you get compression and secure headers

Notes

Networks

The following networks will need be created by the stack deployment:

oauth-web - for the traffic to the oauth2_proxy
traefik-web - for the traffic to the containers without authentication
traefik-oauth - for the traffic to the containers that have to be authenticated
traefik-docker - for traefik to communicate with the docker socket proxy

In order to see the real IP of the visitors, this example publishes the service ports directly on the swarm node. If you don’t want this, you’ll need to create one more network, with external access. See the comments below in the stack.yml file (look for traefik-external).

traefik

I’ll be using /var/storage/docker/traefik/acme.json as storage for the certificates. Make sure that you either modify it according to your path (and make sure to keep it in sync on all the nodes that might be running traefik) or you use a shared file system for it. I’m using Syncthing anyhow, so I’m using it for traefik as well.

Configuration files

We create five files:

secrets/cf_api_email
secrets/cf_api_key
configs/config.yml
configs/traefik.yml
stack.yml

secrets/cf_api_email

Add here your Cloudflare account e-mail address:

cloudflare-address@example.com

secrets/cf_api_key

Add here the Cloudflare API key:

e6QH61YbWa8SnCEiYun1yzxoVszWTqK7udE1W

configs/config.yml

http:
  middlewares:
    default-http:
      redirectScheme:
        scheme: https
        permanent: true
    default-https:
      chain:
        middlewares:
          - default-headers-https@file
          - default-compress@file
    default-headers-https:
      headers:
        browserXssFilter: true
        contentTypeNosniff: true
        customResponseHeaders:
          server: ""
        forceSTSHeader: true
        frameDeny: true
        sslRedirect: true
        stsSeconds: 31536000
        stsPreload: true
    default-compress:
      compress: {}

configs/traefik.yml

global:
  checkNewVersion: false
  sendAnonymousUsage: false

entryPoints:
  http:
    address: ":80"
  https:
    address: ":443"

providers:
  file:
    filename: /config.yml
    watch: true
  docker:
    endpoint: "tcp://dockersocket:2375"
    exposedByDefault: false
    swarmMode: true
    network: traefik-web
    # only exposes the services with the label `ai.ix.expose=true`
    constraints: "Label(`ai.ix.expose`, `true`)"
    # if no rule exists, it uses the value of the label `ai.ix.fqdn` for Host(``)
    defaultRule: "Host(`{{ index .Labels \"ai.ix.fqdn\"}}`)"

log:
  level: 'INFO'

accessLog:
  format: json
  fields:
    defaultMode: keep
    names:
      Set-Cookie: redact
      Etag: redact
      RequestLine: drop
    headers:
      defaultMode: keep
      names:
        Set-Cookie: redact
        Cookie: redact
        Etag: redact
        Authorization: redact
        X-Auth-Request-User: redact
        X-Auth-Request-Email: redact

metrics:
  prometheus:
    buckets:
      - 0.1
      - 0.3
      - 1.2
      - 5.0

certificatesResolvers:
  default:
    acme:
      # For the example only. Remove it for production
      caserver: 'https://acme-staging-v02.api.letsencrypt.org/directory'
      dnsChallenge:
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"
        provider: cloudflare
      storage: /acme.json

tls:
  options:
    default:
      minVersion: VersionTLS12
      sniStrict: true
      cipherSuites:
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256

stack.yml

version: "3.7"

services:
  traefik:
    image: traefik:latest
    networks:
      - oauth-web
      # Enable the next line if you don't expose the service directly on the swarm node
      #- traefik-external
      - traefik-oauth
      - traefik-docker
      - traefik-web
    # Comment out the following section, if you don't want to expose the service directly on the swarm node
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
    configs:
      - traefik.yml
      - config.yml
    secrets:
      - cf_api_key
      - cf_api_email
    environment:
      - CF_API_KEY_FILE=/run/secrets/cf_api_key
      - CF_API_EMAIL_FILE=/run/secrets/cf_api_email
    volumes:
      - '/var/storage/docker/traefik/acme.json:/acme.json:rw'
    command:
      - --configfile=/traefik.yml
  dockersocket:
    image: tecnativa/docker-socket-proxy
    deploy:
      mode: global
      placement:
        constraints: [node.role == manager]
    networks:
      - traefik-docker
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      NETWORKS: 1
      SERVICES: 1
      TASKS: 1
  oauth:
    image: quay.io/pusher/oauth2_proxy:latest
    networks:
      - oauth-web
    deploy:
      labels:
        ai.ix.expose: 'true'
        traefik.enable: 'true'
        traefik.docker.network: oauth-web
        traefik.http.middlewares.oauth-verify.forwardAuth.address: http://oauth:4180/oauth2/auth
        traefik.http.middlewares.oauth-verify.forwardAuth.trustForwardHeader: 'true'
        traefik.http.middlewares.oauth-verify.forwardAuth.authResponseHeaders: X-Auth-Request-User,X-Auth-Request-Email,Authorization,Set-Cookie
        traefik.http.middlewares.oauth-signin.errors.service: oauth@docker
        traefik.http.middlewares.oauth-signin.errors.status: '401'
        traefik.http.middlewares.oauth-signin.errors.query: /oauth2/sign_in
        traefik.http.routers.oauth.entrypoints: https
        traefik.http.routers.oauth.rule: Host(`oauth.ix.ai`) || PathPrefix(`/oauth2`)
        traefik.http.routers.oauth.tls.certResolver: 'default'
        traefik.http.routers.oauth.service: oauth@docker
        traefik.http.services.oauth.loadbalancer.server.port: '4180'
    environment:
      OAUTH2_PROXY_CLIENT_ID: 'FIXME: Google Client ID for Web application'
      OAUTH2_PROXY_CLIENT_SECRET: 'FIXME: Google Client secret'
      OAUTH2_PROXY_COOKIE_DOMAIN: '.ix.ai'
      OAUTH2_PROXY_COOKIE_REFRESH: '1h'
      OAUTH2_PROXY_COOKIE_SECURE: 'true'
      OAUTH2_PROXY_COOKIE_SECRET: 'FIXME: Cookie secret'
      OAUTH2_PROXY_EMAIL_DOMAINS: '*'
      OAUTH2_PROXY_FOOTER: '-'
      OAUTH2_PROXY_HTTP_ADDRESS: '0.0.0.0:4180'
      OAUTH2_PROXY_PASS_BASIC_AUTH: 'false'
      OAUTH2_PROXY_PASS_USER_HEADERS: 'true'
      OAUTH2_PROXY_PROVIDER: 'google'
      OAUTH2_PROXY_REVERSE_PROXY: 'true'
      OAUTH2_PROXY_SET_AUTHORIZATION_HEADER: 'true'
      OAUTH2_PROXY_SET_XAUTHREQUEST: 'true'
      OAUTH2_PROXY_WHITELIST_DOMAIN: '.ix.ai'
  alex-thom-ae:
    image: registry.gitlab.com/ix.ai/int/alex-thom-ae:latest
    networks:
      - traefik-web
    environment:
      NGINX_HOST: alex.thom.ae
      NGINX_PORT: 80
    deploy:
      restart_policy:
        delay: 5s
      replicas: 2
      labels:
        ai.ix.expose: 'true'
        ai.ix.fqdn: 'alex.thom.ae'
        traefik.enable: 'true'
        traefik.docker.network: traefik-web
        traefik.http.routers.alex-thom-ae-http.middlewares: default-http@file
        traefik.http.routers.alex-thom-ae-http.entrypoints: http
        traefik.http.routers.alex-thom-ae.entrypoints: https
        traefik.http.routers.alex-thom-ae.middlewares: default-https@file
        traefik.http.routers.alex-thom-ae.service: alex-thom-ae@docker
        traefik.http.routers.alex-thom-ae.tls.certResolver: 'default'
        traefik.http.services.alex-thom-ae.loadbalancer.server.port: '80'
  alex-ix-ai:
    image: registry.gitlab.com/ix.ai/int/alex-thom-ae:dev
    networks:
      - traefik-oauth
    environment:
      NGINX_HOST: alex.ix.ai
      NGINX_PORT: 80
    deploy:
      restart_policy:
        delay: 5s
      labels:
        ai.ix.expose: 'true'
        ai.ix.fqdn: 'alex.ix.ai'
        traefik.enable: 'true'
        traefik.docker.network: traefik-oauth
        traefik.http.routers.alex-ix-ai-http.middlewares: default-http@file
        traefik.http.routers.alex-ix-ai-http.entrypoints: http
        traefik.http.routers.alex-ix-ai.entrypoints: https
        traefik.http.routers.alex-ix-ai.middlewares: oauth-signin,oauth-verify,default-https@file
        traefik.http.routers.alex-ix-ai.service: alex-ix-ai@docker
        traefik.http.routers.alex-ix-ai.tls.certResolver: 'default'
        traefik.http.services.alex-ix-ai.loadbalancer.server.port: '80'
networks:
  oauth-web:
    driver: overlay
    driver_opts:
      encrypted: 'true'
    internal: true
  # Enable the next five lines if you don't expose the service directly on the swarm node
  # traefik-external:
  #   driver: overlay
  #   driver_opts:
  #     encrypted: 'true'
  #   internal: false
  traefik-oauth:
    driver: overlay
    driver_opts:
      encrypted: 'true'
    internal: true
  traefik-docker:
    driver: overlay
    driver_opts:
      encrypted: 'true'
    internal: true
  traefik-web:
    driver: overlay
    driver_opts:
      encrypted: 'true'
    internal: true
configs:
  config.yml:
    file: ./configs/config.yml
  traefik.yml:
    file: ./configs/traefik.yml
secrets:
  cf_api_key:
    file: ./secrets/cf_api_key
  cf_api_email:
    file: ./secrets/cf_api_email

Deploy

Last step: deploy it all. On one of the docker swarm masters, where you also have the files, run:

sudo docker stack deploy --compose-file stack.yml traefik

Updates

This article was updated on:

2020-01-29: Added OAUTH2_PROXY_REVERSE_PROXY: 'true' to the oauth2_proxy environment (due to changes brought by oauth2_proxy v5.0.0)

Protect your websites with oauth2_proxy behind traefik (docker stack edition)