Since the update to Prometheus 2.0, the documentation still has to catch up. Because I’ve been debugging a custom written exporter, some unwanted metrics landed in my Prometheus.

This is how to delete those series.

Update: The official API documentation can be found here: prometheus.io/docs

Enable the Prometheus Admin API

Deleting the metrics is only available over the Admin API. You can enable it with a flag. Just edit your Prometheus start-up script and add:

--web.enable-admin-api

My full CLI command now looks like this:

/usr/bin/prometheus --web.enable-admin-api --config.file=/etc/prometheus/prometheus.yaml --storage.tsdb.path=/var/lib/prometheus2 --storage.tsdb.retention=365d --web.external-url=https://xxx[redacted]xxx/prometheus/ --web.console.templates=/usr/share/prometheus/consoles --web.console.libraries=/usr/share/prometheus/console_libraries

Delete the Time Series

As you’ve probably noticed, I use a custom URL for Prometheus. Showing an example on how to delete the up series for the instance amazing_instance:9922

If you’re not using a custom URL:

curl -s -X POST -g 'http://localhost:9090/api/v1/admin/tsdb/delete_series?match[]=up{instance="amazing_instance:9922"}'

If you’re using a custom URL (example: /prometheus):

curl -s -X POST -g 'http://localhost:9090/prometheus/api/v1/admin/tsdb/delete_series?match[]=up{instance="amazing_instance:9922"}'

Not working anymore

Keeping these here for historical reference, however these commands don’t work anymore

If you’re not using a custom URL:
curl --silent -H "Content-Type: application/json" -X POST -d '{"matchers":[{"type":"EQ","name":"__name__","value":"up"}]}' 'http://localhost:9090/api/v2/admin/tsdb/delete_series'|jq .
If you’re using a custom URL (example: /prometheus):
curl --silent -H "Content-Type: application/json" -X POST -d '{"matchers":[{"type":"EQ","name":"__name__","value":"up"}]}' 'http://localhost:9090/prometheus/api/v2/admin/tsdb/delete_series'|jq .

Other Matchers

Looking at the Source Code, the following matchers are available:

  • Equals: EQ
  • Not equal: NEQ
  • Regex match: RE
  • Regex not match: NRE

Updates

This article was updated on:

  • 2017: Changed the link to the source code, since it’s bound to be fluid. This way, you will know what to look for in the current master.
  • 2020-02-17: Updated the delete commands with the ones that work now. Added the link to the official API documentation.