...
...
...
...
...
...
...
...
...
| Table of Contents | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Gateway's audit log of user actions is designed for machine parsing so it can be used for auditing, compliance monitoring, API request analysis, and SLA reporting.
See Gateway Configuration for configuring the logging output.
...
Audit Log Message Fields
This section focuses on the format of the audit logs to allow for integration and development of applications that use them.
These are the fields that appear in logging output. These are only definitions and not the format of any particular log message.
Field Name | Description |
|---|---|
Auth Domain | Tenant or storage domain name used to authenticate user; tenant names prefixed with "+" |
Auth User | User ID used to authenticate; empty if anonymous |
Bucket | Name of bucket |
DNS Domain | Origin DNS domain; value of Host header from the request |
Domain | Swarm domain name to which operation refers to |
Elapsed Time | Transaction time in milliseconds |
HTTP Code | Request response code. Exceptions in request handling return a 500. All SCSP requests with authorization errors output a 401. |
Log Level | Logging level for the audit log entry |
Message Type | Message category to simplify filtering |
Object Name or UUID | Named of object, excluding bucket name, or UUID for unnamed streams |
Operation | The operation. Examples: POST, HEAD, DELETE, INVOKE |
Record Format Version | Audit log record format version. This changes if format of the output records is different from the previous release. |
Request ID | A unique identifier for client request attached to all associated audit messages. This value matches the HTTP response header Gateway-Request-Id given to the client and is used in the server log. |
Response Bytes | Number of bytes sent to Source IP in the HTTP response body |
Source Bytes | Number of bytes received from Source IP in the message body |
Source IP | IP address from which a request originated |
Timestamp | High resolution timestamp up to millisecond |
Audit Log Message Formats
Following are the output formats for all event types. All log messages share a common set of prefix fields, which includes a message type. The suffix fields in a log message are variable based on the message type. This allows for automated parsing of log messages.
...
Alphanumeric characters "a" through "z", "A" through "Z" and "0" through "9" remain unchanged
Characters ".", "-", "*", and "_" remain unchanged
Space character is converted into a plus sign "+"
All other characters are converted into %HH byte values using UTF-8 encoding
...
Note
The "/" character in an object's name appear as "%2F" in the log, based on the previous rules.
Common Prefix Fields
All messages are prefixed by the following fields in this order:
Timestamp
Log Level
Request ID
Record Format Version
Source IP
DNS Domain
Message Type
Operation
Auth User
Auth Domain
HTTP Code
Source Bytes
Response Bytes
Elapsed Time
Suffix Fields
This table defines the suffix fields that are included with each log message following the common prefix fields.
Event | Message Type | Operation | Suffix Fields |
|---|---|---|---|
User requests token | Auth | GET | |
User deletes token | DELETE | ||
List available domains | Admin | LIST_DOMAINS | |
Domain creation | Domain | POST | Domain |
Domain policy create/ update | POLICY_PUT | ||
Domain policy read | POLICY_GET | ||
Domain policy delete | POLICY_DELETE | ||
Domain copy | COPY | ||
Domain delete | DELETE | ||
Domain read | GET | ||
Domain info | HEAD | ||
List buckets in a domain | LIST_BUCKETS | ||
Bucket creation | Bucket | POST | Domain, Bucket |
Bucket policy create/ update | POLICY_PUT | ||
Bucket policy read | POLICY_GET | ||
Bucket policy delete | POLICY_DELETE | ||
Bucket copy | COPY | ||
Bucket delete | DELETE | ||
Bucket read | GET | ||
Bucket info | HEAD | ||
List objects in a bucket | LIST_OBJECTS | ||
S3 list multiparts | LIST_MULTIPARTS | ||
Object creation | Scsp | POST | Domain, Bucket, Object name or UUID |
Object update | PUT | ||
Object append | APPEND | ||
Object copy | COPY | ||
Object delete | DELETE | ||
Object read | GET | ||
Object info | HEAD | ||
S3 multipart initiate | MULTIPART_INITIATE | Domain, Bucket, Object name | |
S3 multipart put | MULTIPART_PUT | ||
S3 multipart copy | MULTIPART_COPY | ||
S3 multipart abort | MULTIPART_ABORT | ||
S3 multipart complete | MULTIPART_COMPLETE | ||
S3 list multipart | LIST_MULTIPART |
Example Log Messages
These are examples of a variety of audit log messages.
...
| Code Block | ||
|---|---|---|
| ||
2019-10-16 10:37:29,719 INFO [D580617E135E35DF] 2 172.30.1.1 172.20.1.2 Domain POLICY_PUT !superuser@ nom.dom.com 201 123 0 1.08 nom.dom.com |
Behaviors of Operations
Interrupted GET — : When a GET operation is interrupted, such as if the socket closed unexpectedly prior to reading all data, the audit log may record an HTTP 200 response with response bytes equal to the size of the object. When interruption takes place, an HTTP 500 response is logged with response bytes equal to the actual number of bytes transmitted.
Duplicate Request IDs — : All messages have the same Request ID so they can be correlated with the client request if multiple messages are logged from one client operation. For example, the recursive delete operation generates synthetic delete requests all with the same Request ID.
INVOKE operations — : The optional feature Video Clipping (v11.0) logs INVOKE operations. Each video clipping event logs multiple events to provide auditing through the process, which may take a while to complete. When you create a video clip, Gateway acknowledges the request with an INVOKE message. See [Archived] Video Clipping for Partial File Restore.
Application-Supplied Tag
Gateway's audit logging allows for the client application to supply a custom tag that can be used to correlate multiple audit log entries to one application-level transaction. The application specifies this tag in a Gateway-Audit-Id request header and it must be alpha-numeric and is truncated at 32 characters. When this optional tag is received, the Request ID field of the audit log entry contains the automatically-generated request identifier from the Gateway, a dash ("-"), and the application-supplied tag.
...
When the application-supplied tag is used for multiple operations, even across multiple Gateway servers, the request identifiers remain unique with a common suffix.
...