DBCC CHECKDB getting terminated abnormally
We have a SQL Server 2014 set up on Always on AG with one primary node and 2 secondary nodes. On the primary node, DBCC CHECKDB for one of the databases is scheduled on SQL agent job. This job was running fine till last month and now its getting failed and the job logs read "DBCC CHECKDB terminated abnormally due to error state 6." when running the DBCC CHECKDB command manually, getting the same result.
During the job/Command runtime , there is a spike in I/O , Memory and CPU utilization is full. Storage team had also checked and reported that there are no storage related errors
On the Secondary nodes, the job runs fine without any issues.
I did try to restore the DB on a test instance and the DBCC CHECKDB runs fine.
Could anyone help me to find the cause of this.
DBCC CHECKDB terminated abnormally due to error state 6
The error state 6 means process is killed or terminated by external process at least as far as my experience goes, this is not documented though. I have witnessed this and in my case this was due to a monitoring query running killing long running job. Please double, triple check is any user process/agent job/third part software (in some case) is killing the process.