Logging

WAF Uses Amazon Kinesis Firehose to ingest logs. This allows logs to be passed to any Kinesis Firehose destination, such as Amazon S3, Amazon Redshift or Amazon Elastic Search. To enable logging of requests in your Web ACL, you must first create a Kinesis Data Firehose.

Here is an example WAF log of a request. Note that you receive details on the rule that terminated the evaluation of the request. The log also contains the action taking on that request

{
  "timestamp": 1576280412771,
  "formatVersion": 1,
  "webaclId": "arn:aws:wafv2:ap-southeast-2:EXAMPLE12345:regional/webacl/STMTest/1EXAMPLE-2ARN-3ARN-4ARN-123456EXAMPLE",
  "terminatingRuleId": "STMTest_SQLi_XSS",
  "terminatingRuleType": "REGULAR",
  "action": "BLOCK",
  "terminatingRuleMatchDetails": [
    {
      "conditionType": "SQL_INJECTION",
      "location": "HEADER",
      "matchedData": ["10", "AND", "1"]
    }
  ],
  "httpSourceName": "-",
  "httpSourceId": "-",
  "ruleGroupList": [],
  "rateBasedRuleList": [],
  "nonTerminatingMatchingRules": [],
  "httpRequest": {
    "clientIp": "1.1.1.1",
    "country": "AU",
    "headers": [
      {
        "name": "Host",
        "value": "localhost:1989"
      },
      {
        "name": "User-Agent",
        "value": "curl/7.61.1"
      },
      {
        "name": "Accept",
        "value": "*/*"
      },
      {
        "name": "x-stm-test",
        "value": "10 AND 1=1"
      }
    ],
    "uri": "/foo",
    "args": "",
    "httpVersion": "HTTP/1.1",
    "httpMethod": "GET",
    "requestId": "rid"
  }
}

Analysing and Visualising Logs

Log analysis is key to ensure the effectiveness of WAF Rules, in addition to troubleshooting of individual issues. Logs exported to Amazon Elastic Search Service can be queried. Kibana can be used alongside Amazon Elastic Search to visualise the WAF logs S3 Select can be used to perform SQL queries against individual WAF Log files in S3. Queries can be performed in the console, or using the AWS CLI or SDKs.

For example, the following S3 Select query counts how many requests in the log file were blocked.

SELECT *
FROM S3Object s
WHERE  s."action" = 'BLOCK'

/*
Result

{
    "count": 18
}
*/

Challenge

The Juice Shop is growing rapidly. Great work! Now that you have a set of rules, it is becoming more difficult to reason which rule is responsible for blocking a request. It would be helpful to have some logs. To do this, you need to enable logging for your Web ACL to an S3 Bucket.
Your logs contain a sensitive header, named Cookie. You don’t want this to be stored in your logs. You will need configure the redaction of this header in the logs.

1. Create an S3 bucket. This will be the destination of your Kinesis Data Firehose
   • Prefix the S3 bucket with aws-waf-logs-workshop-. This is to help you find it later.

2. Create a Kinesis Data Firehose delivery stream. Make sure to create the resource in us-east-1. This is required when capturing logs for CloudFront. Use the S3 bucket from the previous step as your destination.
   • Prefix the Kinesis Data Firehose with aws-waf-logs-workshop-. This is required by the WAF service
3. Enable logging for your WAF
4. Redact the cookie, titled Cookie (the Juice Shop Cookie) in your logs.

5. Generate some traffic using the following requests. Some of these will be blocked by the rules you’ve previously created

curl "$JUICESHOP_URL?username=admin"
curl "${JUICESHOP_URL}?milkshake=banana&favourite-topping=sauce"
curl -H "x-milkshake: chocolate" "${JUICESHOP_URL}"

6. Download the log file from the S3 bucket used as the Kinesis Data Firehose destination.
7. Inspect the logs. Can you find the redacted Cookie field?

Answer

Conclusion

WAF allows you to capture request logs and store them in any Kinesis Data Firehose destination. The logs provide information of the request. The logs also provide the action and rule involved for a request. This information can be invaluable when running a WAF. Use field redaction to avoid logging sensitive information