Exporting Prometheus Data

If Prometheus data needs to be exported (such as to supply to machine learning), adapt the Python 2 script below to run a Prometheus 2.x query that outputs to a CSV file.

Tip

By default, Prometheus retains your metrics data for 14 days (which is configurable). Use this script in a cron job to collect and archive all historical data.

ExportToCsv.py

!/usr/bin/env python

import csv
import requests
import sys

if len(sys.argv) != 3:
    print('Usage: {0} http://prometheus:9090 a_query'.format(sys.argv[0]))
    sys.exit(1)

response = requests.get('{0}/api/v1/query'.format(sys.argv[1]),
        params={'query': sys.argv[2]})
results = response.json()['data']['result']

labelnames = set()
for result in results:
      labelnames.update(result['metric'].keys())

labelnames.discard('__name__')
labelnames = sorted(labelnames)

writer = csv.writer(sys.stdout)
writer.writerow(['name', 'timestamp', 'value'] + labelnames)

for result in results:
  for avalue in result['values']:
    l = [result['metric'].get('__name__', '')] + avalue
    for label in labelnames:
        l.append(result['metric'].get(label, ''))
    writer.writerow(l)

The script expects two arguments:

the URL to the Prometheus endpoint, which defaults to http://127.0.0.1:9090
a PromQL (native Prometheus) query
- See the reference for the PromQL query language: https://prometheus.io/docs/prometheus/latest/querying/examples/

This is the output that the above script generates:

Results

[root@swarmtelemetry ~]# ./ExportToCsv.py http://127.0.0.1:9090 caringo_swarm_scsp_gets[5m]
name,timestamp,value,instance,job
caringo_swarm_scsp_gets,1584979640.11,382,10.10.10.84:9100,swarm
caringo_swarm_scsp_gets,1584979670.11,382,10.10.10.84:9100,swarm
caringo_swarm_scsp_gets,1584979700.11,382,10.10.10.84:9100,swarm
caringo_swarm_scsp_gets,1584979730.11,294,10.10.10.84:9100,swarm
caringo_swarm_scsp_gets,1584979760.11,234,10.10.10.85:9100,swarm
caringo_swarm_scsp_gets,1584979790.11,284,10.10.10.85:9100,swarm
caringo_swarm_scsp_gets,1584979820.11,294,10.10.10.85:9100,swarm