Custom Metadata Typing
- Rumena Zlatkova (Deactivated)
- Bala Harish A(AC)
- Kyle Miller (Deactivated)
- Anjali Tyagi
Info
The user can disable metadata typing to save space with the search.enableCustomMetadataTyping setting:search.enableCustomMetadataTyping = false
The user must review this setting before creating any new search feed. This setting is available since Swarm v14.1, but the default is ‘False’ in v15.0.
Swarm indexes custom metadata as strings by default. Swarm types the custom metadata if the data follows standards for numeric, time, or geospatial formats to perform query operations and to manage data by type. Swarm applies typing to Custom Metadata Headers on all context objects (domains, buckets) and content objects (named, alias, immutable, untenanted). (v9.2)
Tip
Including a non-digit, non-"e" character (such as ~) in the string to prevent additional typing if the string data exists that may match the pattern for a number or date.
Data Types for Custom Headers
On POST and COPY, data types Swarm parses and recognizes in custom headers are:
String (default)
Numeric (double and long)
Date (ISO 8601)
Geo-point (latitude, longitude)
Swarm Search indexes on these types to run queries against these types in the custom metadata.
Important
Convert dashes (hyphens) to underscores which is how the headers are indexed for Elasticsearch if querying directly to Elasticsearch rather than through Swarm.
Type | Example Header | Example Search | Notes |
|---|---|---|---|
String | "x-foo-meta: ASCII string." | &x-foo-meta=ASCII%20string | The default datatype. See the Elasticsearch documentation onhttps://www.elastic.co/guide/en/elasticsearch/reference/2.3/string.html. |
Double | "x-foo-meta: 34567.123" | &x-foo-meta:double=34567.123 | See the Elasticsearch documentation on https://www.elastic.co/guide/en/elasticsearch/reference/2.3/number.html. |
Long | "x-foo-meta: 93e27" | &x-foo-meta:long=93e27 | To prevent overflow, Swarm indexes numeric values only up to 64 bits of precision. See the Elasticsearch documentation on https://www.elastic.co/guide/en/elasticsearch/reference/2.3/number.html. |
Geo-point | "x-foo-meta: 11, -33" | &x-foo-meta:geo=11,-33 | A 2-tuple of two numeric values. The first value is For search operations, evaluations are performed element by element. See the Elasticsearch documentation on https://www.elastic.co/guide/en/elasticsearch/reference/2.3/geo-point.html. |
Date | "x-foo-meta: 1420070400001" (milliseconds) | &x-foo-meta:date=1420070400001 | Counts of time since the epoch can be numbers up to 13 digits:
For dates, the time and zone are optional. An offset from UTC (Zulu) can be appended to the time as ±[hh]:[mm], ±[hh][mm], or ±[hh]. See the https://en.wikipedia.org/wiki/ISO_8601 and the Elasticsearch documentation on https://www.elastic.co/guide/en/elasticsearch/reference/2.3/date.html. |
Querying Custom Metadata by Type
Query for custom metadata typed by using the additional fields when making SCSP queries to Swarm:
&x-foo-metaperforms string-based queries&x-foo-meta:longperforms integer/long typed queries&x-foo-meta:doubleperforms double typed queries&x-foo-meta:dateperforms date-typed queries&x-foo-meta:geoperforms geo-point-typed queries
Here is an example of a query that filters between a range of timestamps, stored in custom headers:
curl -i --location-trusted "http://192.168.1.11/surveillance?domain=example.com&format=json&fields=all\
&content-type=video/mp4\
&x_example_meta_Building=Annex%202\
&x_example_meta_Start_Time:date=>2017-03-11T12:00Z\
&x_example_meta_End_Time:date=<2017-03-11T12:30Z"Upgrading an Existing Index
The index needs to be rebuilt if existing custom metadata is currently stored as strings and it is desired to take advantage of typing across all existing custom metadata.
© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.