OAuth2 Proxy and Traefik logos. OAuth2 Proxy and Traefik logos.

This is how to protect your website with Google’s OAuth 2.0, using pusher/oauth2_proxy behind a containous/traefik cloud native edge router.

Table of Contents

Objectives

For this tutorial, we will build the following on an existing docker swarm cluster:

A couple of more goodies will be added as well:

  • all HTTP traffic will be redirected to HTTPS
  • automates certificate generation with Let’s Encrypt for the domains hosted with Cloudflare
  • using docker secrets to manage the Cloudflare credentials
  • using docker configs to manage the traefik configuration files
  • not exposing the docker socket directly to traefik, but over Tecnativa/docker-socket-proxy
  • chaining of traefik middlewares, so that you get compression and secure headers

Notes

Networks

The following networks will need be created by the stack deployment:

  • oauth-web - for the traffic to the oauth2_proxy
  • traefik-web - for the traffic to the containers without authentication
  • traefik-oauth - for the traffic to the containers that have to be authenticated
  • traefik-docker - for traefik to communicate with the docker socket proxy

In order to see the real IP of the visitors, this example publishes the service ports directly on the swarm node. If you don’t want this, you’ll need to create one more network, with external access. See the comments below in the stack.yml file (look for traefik-external).

traefik

I’ll be using /var/storage/docker/traefik/acme.json as storage for the certificates. Make sure that you either modify it according to your path (and make sure to keep it in sync on all the nodes that might be running traefik) or you use a shared file system for it. I’m using Syncthing anyhow, so I’m using it for traefik as well.

Configuration files

We create five files:

  • secrets/cf_api_email
  • secrets/cf_api_key
  • configs/config.yml
  • configs/traefik.yml
  • stack.yml

secrets/cf_api_email

Add here your Cloudflare account e-mail address:

[email protected]

secrets/cf_api_key

Add here the Cloudflare API key:

e6QH61YbWa8SnCEiYun1yzxoVszWTqK7udE1W

configs/config.yml

http:
  middlewares:
    default-http:
      redirectScheme:
        scheme: https
        permanent: true
    default-https:
      chain:
        middlewares:
          - [email protected]
          - [email protected]
    default-headers-https:
      headers:
        browserXssFilter: true
        contentTypeNosniff: true
        customResponseHeaders:
          server: ""
        forceSTSHeader: true
        frameDeny: true
        sslRedirect: true
        stsSeconds: 31536000
        stsPreload: true
    default-compress:
      compress: {}

configs/traefik.yml

global:
  checkNewVersion: false
  sendAnonymousUsage: false

entryPoints:
  http:
    address: ":80"
  https:
    address: ":443"

providers:
  file:
    filename: /config.yml
    watch: true
  docker:
    endpoint: "tcp://dockersocket:2375"
    exposedByDefault: false
    swarmMode: true
    network: traefik-web
    # only exposes the services with the label `ai.ix.expose=true`
    constraints: "Label(`ai.ix.expose`, `true`)"
    # if no rule exists, it uses the value of the label `ai.ix.fqdn` for Host(``)
    defaultRule: "Host(`{{ index .Labels \"ai.ix.fqdn\"}}`)"

log:
  level: 'INFO'

accessLog:
  format: json
  fields:
    defaultMode: keep
    names:
      Set-Cookie: redact
      Etag: redact
      RequestLine: drop
    headers:
      defaultMode: keep
      names:
        Set-Cookie: redact
        Cookie: redact
        Etag: redact
        Authorization: redact
        X-Auth-Request-User: redact
        X-Auth-Request-Email: redact

metrics:
  prometheus:
    buckets:
      - 0.1
      - 0.3
      - 1.2
      - 5.0

certificatesResolvers:
  default:
    acme:
      # For the example only. Remove it for production
      caserver: 'https://acme-staging-v02.api.letsencrypt.org/directory'
      dnsChallenge:
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"
        provider: cloudflare
      storage: /acme.json

tls:
  options:
    default:
      minVersion: VersionTLS12
      sniStrict: true
      cipherSuites:
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256

stack.yml

version: "3.7"

services:
  traefik:
    image: traefik:latest
    networks:
      - oauth-web
      # Enable the next line if you don't expose the service directly on the swarm node
      #- traefik-external
      - traefik-oauth
      - traefik-docker
      - traefik-web
    # Comment out the following section, if you don't want to expose the service directly on the swarm node
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
    configs:
      - traefik.yml
      - config.yml
    secrets:
      - cf_api_key
      - cf_api_email
    environment:
      - CF_API_KEY_FILE=/run/secrets/cf_api_key
      - CF_API_EMAIL_FILE=/run/secrets/cf_api_email
    volumes:
      - '/var/storage/docker/traefik/acme.json:/acme.json:rw'
    command:
      - --configfile=/traefik.yml
  dockersocket:
    image: tecnativa/docker-socket-proxy
    deploy:
      mode: global
      placement:
        constraints: [node.role == manager]
    networks:
      - traefik-docker
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      NETWORKS: 1
      SERVICES: 1
      TASKS: 1
  oauth:
    image: quay.io/pusher/oauth2_proxy:latest
    networks:
      - oauth-web
    deploy:
      labels:
        ai.ix.expose: 'true'
        traefik.enable: 'true'
        traefik.docker.network: oauth-web
        traefik.http.middlewares.oauth-verify.forwardAuth.address: http://oauth:4180/oauth2/auth
        traefik.http.middlewares.oauth-verify.forwardAuth.trustForwardHeader: 'true'
        traefik.http.middlewares.oauth-verify.forwardAuth.authResponseHeaders: X-Auth-Request-User,X-Auth-Request-Email,Authorization,Set-Cookie
        traefik.http.middlewares.oauth-signin.errors.service: [email protected]ocker
        traefik.http.middlewares.oauth-signin.errors.status: '401'
        traefik.http.middlewares.oauth-signin.errors.query: /oauth2/sign_in
        traefik.http.routers.oauth.entrypoints: https
        traefik.http.routers.oauth.rule: Host(`oauth.ix.ai`) || PathPrefix(`/oauth2`)
        traefik.http.routers.oauth.tls.certResolver: 'default'
        traefik.http.routers.oauth.service: [email protected]
        traefik.http.services.oauth.loadbalancer.server.port: '4180'
    environment:
      OAUTH2_PROXY_CLIENT_ID: 'FIXME: Google Client ID for Web application'
      OAUTH2_PROXY_CLIENT_SECRET: 'FIXME: Google Client secret'
      OAUTH2_PROXY_COOKIE_DOMAIN: '.ix.ai'
      OAUTH2_PROXY_COOKIE_REFRESH: '1h'
      OAUTH2_PROXY_COOKIE_SECURE: 'true'
      OAUTH2_PROXY_COOKIE_SECRET: 'FIXME: Cookie secret'
      OAUTH2_PROXY_EMAIL_DOMAINS: '*'
      OAUTH2_PROXY_FOOTER: '-'
      OAUTH2_PROXY_HTTP_ADDRESS: '0.0.0.0:4180'
      OAUTH2_PROXY_PASS_BASIC_AUTH: 'false'
      OAUTH2_PROXY_PASS_USER_HEADERS: 'true'
      OAUTH2_PROXY_PROVIDER: 'google'
      OAUTH2_PROXY_SET_AUTHORIZATION_HEADER: 'true'
      OAUTH2_PROXY_SET_XAUTHREQUEST: 'true'
      OAUTH2_PROXY_WHITELIST_DOMAIN: '.ix.ai'
  alex-thom-ae:
    image: registry.gitlab.com/ix.ai/int/alex-thom-ae:latest
    networks:
      - traefik-web
    environment:
      NGINX_HOST: alex.thom.ae
      NGINX_PORT: 80
    deploy:
      restart_policy:
        delay: 5s
      replicas: 2
      labels:
        ai.ix.expose: 'true'
        ai.ix.fqdn: 'alex.thom.ae'
        traefik.enable: 'true'
        traefik.docker.network: traefik-web
        traefik.http.routers.alex-thom-ae-http.middlewares: [email protected]
        traefik.http.routers.alex-thom-ae-http.entrypoints: http
        traefik.http.routers.alex-thom-ae.entrypoints: https
        traefik.http.routers.alex-thom-ae.middlewares: [email protected]
        traefik.http.routers.alex-thom-ae.service: [email protected]
        traefik.http.routers.alex-thom-ae.tls.certResolver: 'default'
        traefik.http.services.alex-thom-ae.loadbalancer.server.port: '80'
  alex-ix-ai:
    image: registry.gitlab.com/ix.ai/int/alex-thom-ae:dev
    networks:
      - traefik-oauth
    environment:
      NGINX_HOST: alex.ix.ai
      NGINX_PORT: 80
    deploy:
      restart_policy:
        delay: 5s
      labels:
        ai.ix.expose: 'true'
        ai.ix.fqdn: 'alex.ix.ai'
        traefik.enable: 'true'
        traefik.docker.network: traefik-oauth
        traefik.http.routers.alex-ix-ai-http.middlewares: [email protected]
        traefik.http.routers.alex-ix-ai-http.entrypoints: http
        traefik.http.routers.alex-ix-ai.entrypoints: https
        traefik.http.routers.alex-ix-ai.middlewares: oauth-signin,oauth-verify,[email protected]
        traefik.http.routers.alex-ix-ai.service: [email protected]
        traefik.http.routers.alex-ix-ai.tls.certResolver: 'default'
        traefik.http.services.alex-ix-ai.loadbalancer.server.port: '80'
networks:
  oauth-web:
    driver: overlay
    driver_opts:
      encrypted: 'true'
    internal: true
  # Enable the next five lines if you don't expose the service directly on the swarm node
  # traefik-external:
  #   driver: overlay
  #   driver_opts:
  #     encrypted: 'true'
  #   internal: false
  traefik-oauth:
    driver: overlay
    driver_opts:
      encrypted: 'true'
    internal: true
  traefik-docker:
    driver: overlay
    driver_opts:
      encrypted: 'true'
    internal: true
  traefik-web:
    driver: overlay
    driver_opts:
      encrypted: 'true'
    internal: true
configs:
  config.yml:
    file: ./configs/config.yml
  traefik.yml:
    file: ./configs/traefik.yml
secrets:
  cf_api_key:
    file: ./secrets/cf_api_key
  cf_api_email:
    file: ./secrets/cf_api_email

Deploy

Last step: deploy it all. On one of the docker swarm masters, where you also have the files, run:

sudo docker stack deploy --compose-file stack.yml traefik