If you need to export your Prometheus data (such as to supply to machine learning), you can adapt the Python 2 script below to run a Prometheus 2.x query that outputs to a CSV file.
Tip
By default, Prometheus retains your metrics data for 14 days (which is configurable). You can use this script in a cron job to collect and archive all of your historical data.
ExportToCsv.py
!/usr/bin/env python
import csv
import requests
import sys
if len(sys.argv) != 3:
print('Usage: {0} http://prometheus:9090 a_query'.format(sys.argv[0]))
sys.exit(1)
response = requests.get('{0}/api/v1/query'.format(sys.argv[1]),
params={'query': sys.argv[2]})
results = response.json()['data']['result']
labelnames = set()
for result in results:
labelnames.update(result['metric'].keys())
labelnames.discard('__name__')
labelnames = sorted(labelnames)
writer = csv.writer(sys.stdout)
writer.writerow(['name', 'timestamp', 'value'] + labelnames)
for result in results:
for avalue in result['values']:
l = [result['metric'].get('__name__', '')] + avalue
for label in labelnames:
l.append(result['metric'].get(label, ''))
writer.writerow(l)
The script expects two arguments:
- the URL to the Prometheus endpoint, which defaults to
http://127.0.0.1:9090 - a PromQL (native Prometheus) query
- See the reference for the PromQL query language: https://prometheus.io/docs/prometheus/latest/querying/examples/
This is the output that the above script generates:
Results
[root@swarmtelemetry ~]# ./ExportToCsv.py http://127.0.0.1:9090 caringo_swarm_scsp_gets[5m] name,timestamp,value,instance,job caringo_swarm_scsp_gets,1584979640.11,382,10.10.10.84:9100,swarm caringo_swarm_scsp_gets,1584979670.11,382,10.10.10.84:9100,swarm caringo_swarm_scsp_gets,1584979700.11,382,10.10.10.84:9100,swarm caringo_swarm_scsp_gets,1584979730.11,294,10.10.10.84:9100,swarm caringo_swarm_scsp_gets,1584979760.11,234,10.10.10.85:9100,swarm caringo_swarm_scsp_gets,1584979790.11,284,10.10.10.85:9100,swarm caringo_swarm_scsp_gets,1584979820.11,294,10.10.10.85:9100,swarm