Elasticsearch queries- direct to ES

This article will give a few examples of doing direct-to-Elasticsearch queries. This is normally not necessary and not advised as queries should be done through the Swarm Search API.

Preparation

Set a Swarm Node IP as SCSP_HOST, Elasticsearch IP and the Elasticsearch Index variables. These presume the Support tools are installed at /root/dist

export SCSP_HOST=<any Swarm node>
export ESIP=$(python /root/dist/platform-get-default-search-feed-ip.py $SCSP_HOST | awk '{ print $1 }')
export ESINDEX=index_$(python /root/dist/platform-get-default-search-index-name.py $SCSP_HOST)

Calls direct to ES

ES information by stream name

FILENAME=<filename> && curl -s "$ESIP:9200/$ESINDEX/_search?q=stype:NAMED+AND+name:$FILENAME&pretty" | jq  -r '.hits.hits[]'

This will return all named streams by that name across buckets/ domains. You can delineate by bucket or domain id as seen in other examples.

ES information by etag

ETAG=<etag> && curl -s "$ESIP:9200/$ESINDEX/_search?pretty&q=etag:$ETAG" | jq '.hits.hits[]'

Get bucket name, etag, and tmBorn by bucket name

This might be handy for recreating a recently deleted bucket

BUCKETNAME=<bucket name>;curl -s "$ESIP:9200/$ESINDEX/_search?q=stype:BUCKET+AND+name:$BUCKETNAME"  | jq -r '.hits.hits[]._source | "\(.name), \(.etag), \(.tmBorn), \(.contextid)"'

Get Domain id

from Swarm

DOMAIN=<domain name> && curl -sIL $SCSP_HOST?domain=$DOMAIN | grep ^Castor-System-Alias | awk '{ print $2 }'

from ES directly

DOMAIN=<domain name> && curl -s "$ESIP:9200/$ESINDEX/_search?q=stype:DOMAIN+AND+name:$DOMAIN&pretty" | jq -r '.hits.hits[]._id'

Get Bucket id (context id)

from Swarm by domain name

BUCKET=<bucket name> && DOMAIN=<domain name> && curl -sIL "$SCSP_HOST/$BUCKET?domain=$DOMAIN" | grep ^Castor-System-Alias | awk '{ print $2 }'

from ES directly- by domain id

BUCKET=<bucket name> && DOMAINID=<domainid> && curl -s "$ESIP:9200/$ESINDEX/_search?q=domainid:$DOMAINID+AND+stype:BUCKET+AND+name:$BUCKET&pretty" | jq  -r '.hits.hits[]._id'

Stream count by bucket (context id)

CONTEXTID=<contextid> && curl -s -XPOST -H "Content-Type: application/json" --data-binary '{"query":{"bool":{"must":[{"match":{"contextid":"'"$CONTEXTID"'"}}]}}}' "$ESIP:9200/$ESINDEX/_count" | jq '.count'

Streams by bucket (context id)

BUCKETID=<contextid) &&curl -s "$ESIP:9200/$ESINDEX/_search?q=stype:NAMED+AND+contextid:$BUCKETID&pretty";

Stream count by Domain (domain id)

DOMAINID=<domainid> && curl -s -XPOST -H "Content-Type: application/json" --data-binary '{"query":{"bool":{"must":[{"match":{"domainid":"'"$DOMAINID"'"}}]}}}' "$ESIP:9200/$ESINDEX/_count" | jq '.count'

Knowledge Base