What is the 404 Per Overlay message?

When performing operations on streams in a cluster, you may see the response includes the phrase "PER OVERLAY".  This means the response came from the overlay index.

The overlay index lives in RAM and has a mapping of all of the streams in the cluster and where they currently reside. If the overlay index is enabled and in an authoritative state, any requests to read a stream check with the overlay first since the response is much faster than the non-overlay queries. The overlay index reaches an authoritative state when it has accounted for all streams in the cluster.

When a cluster responds with a 404 "PER OVERLAY" response, it means that the overlay is enabled, has a mapping of every stream in the cluster and the stream was not found.

When a cluster returns a regular 404 response, it means either the overlay is not enabled (perhaps too few nodes in the cluster) or it is not in an authoritative state so queries the cluster for the stream instead.