How do I troubleshoot a dateparsefailure when using Logstash's date plugin?

[magpie]

I'm getting a date parse failure when trying to parse a log that has the following format:

172.20.21.10 - - [29/Mar/2022:12:41:27 -0400] "some invalid request" 400 226

The event structure looks like this:

{
        "request" => "/um/",
           "auth" => "-",
          "ident" => "-",
      "@metadata" => {
        "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
        "host" => "M6"
    },
           "verb" => "GET",
        "message" => "172.20.21.10 - - [29/Mar/2022:12:44:36 -0400] \"GET /um/ HTTP/1.1\" 404 17787",
           "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
          "bytes" => "17787",
       "response" => "404",
       "clientip" => "172.20.21.10",
       "@version" => "1",
           "host" => "M6",
    "httpversion" => "1.1",
      "timestamp" => "29/Mar/2022:12:44:36 -0400"
}

To convert the timestamp field from text to a date I have a filter in the logstash configuration file that looks like this:

filter {
        grok {
              match => {"message" => "%{COMMONAPACHELOG}"}
        }
        mutate {
                remove_field => [ "@timestamp"]
        }
        date {
              match => ["timestamp", "dd/MM/yyyy:HH:mm:ss Z"]
              target => ["timestamp"]
        }
}

I have used the correct syntax for the format according to the https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html for the date filter. I'm not sure whats causing the parse failure since the error msg does not show exactly why the parse failed. It looks like this:

{
        "request" => "/wp-includes/IXR/login",
           "auth" => "-",
          "ident" => "-",
      "@metadata" => {
        "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
        "host" => "M6"
    },
           "verb" => "GET",
        "message" => "172.20.21.10 - - [29/Mar/2022:12:47:57 -0400] \"GET /wp-includes/IXR/login HTTP/1.1\" 404 17787",
           "tags" => [
        [0] "_dateparsefailure"
    ],
           "path" => "C:/net-logs/m4-httpd-logs/wordpress_access.log-20220403",
       "response" => "404",
          "bytes" => "17787",
       "clientip" => "172.20.21.10",
       "@version" => "1",
           "host" => "M6",
    "httpversion" => "1.1",
      "timestamp" => "29/Mar/2022:12:47:57 -0400"
}

[esther]

Fixed by creating a mapping for the timestamp field:

{
  "index_patterns": "httpd-*",
  "mappings": {
    "dynamic": true,
    "properties": {
        "request": {
          "type": "text"
        },
        "auth": {
          "type": "text"
        },
        "ident": {
          "type": "text"
        },
        "verb": {
          "type": "text"
        },
        "response": {
          "type": "text"
        },
        "bytes": {
          "type": "integer"
        },
        "clientip": {
          "type": "ip"
        },
        "httpversion": {
          "type": "text"
        },
        "timestamp": {
          "type": "date",
          "format": "dd/MMM/yyyy:HH:mm:ss Z"
        }
    }
  }