I have progressDeadlineSeconds set to 120 seconds
I deploy and run kubectl rollout status deployment mydeployment
The deployment failed with 0 of 1 updated replicas available - CrashLoopBackOff
But the kubectl is still hanging forever with message: Waiting for deployment "mydeployment" rollout to finish: 0 of 1 updated replicas are available...
Why is this happening, progressDeadlineSeconds is suppose to force it to fail and cause kubectl rollout status deployment
to exit with a non-zero return code right?
1 Answer
You are correct, kubectl rollout status
returns a non-zero exit code if the Deployment has exceeded the progression deadline. Progress Deadline Seconds:
.spec.progressDeadlineSeconds
is an optional field that specifies the number of seconds you want to wait for your Deployment to progress before the system reports back that the Deployment has failed progressing - surfaced as a condition with Type=Progressing
, Status=False
. and Reason=ProgressDeadlineExceeded
in the status of the resource. The Deployment controller will keep retrying the Deployment. This defaults to 600. In the future, once automatic rollback will be implemented, the Deployment controller will roll back a Deployment as soon as it observes such a condition.
If specified, this field needs to be greater than .spec.minReadySeconds
.
Which brings us to Min Ready Seconds
.spec.minReadySeconds
is an optional field that specifies the minimum number of seconds for which a newly created Pod should be ready without any of its containers crashing, for it to be considered available. This defaults to 0 (the Pod will be considered available as soon as it is ready).
Without your exact deployment configs it is hard to tell where the problem is but there are few things to check out:
Try setting both the
progressDeadlineSeconds
andminReadySeconds
remembering that the later must have a smaller value.progressDeadlineSeconds
might not be respected when used withReplicas: 1
. Check yourmaxUnavailable
param in order to see if it allows deployment full unavailability.As a workaround you can specify a timeout period for your
kubectl rollout status
command. For example:kubectl rollout status deployment mydeployment --timeout=120s
If you don't want to wait for the rollout to finish then you can use
--watch=false
. You will than have to check the status manually by runningkubectl describe deployment
andkubectl get deployment
commands which might not be ideal.
Please let me know if that helps.
ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJobWtpZWiAeoWOqamon6KawLSwxJqbpaGemsCmr86nm6xlmah6r7vTZpqoqqKasLW42GaroqWZo7Ruu9St