Rebooting the instance during deployment causes an error

[tomyam1]

Trying to reboot the instance the instance during deployment causes an error:

Script - scripts/reboot.sh
[stderr]+ reboot
[stderr]
[stderr]Session terminated, terminating shell... ...terminated.

It'd be nice to have a way to tell the CodeDeploy that the instance is going to reboot and it has to wait for it to come up, or is there already a method for doing that?

[tangerini]

Our agent doesn't support rebooting an instance and resuming an in-progress deployment after the reboot. What's your use case for doing this?

[tomyam1]

The heavy lifting of our deploy process is done using an ansible playbook.
Part of that playbook is to update all the system packages.
As a result, sometimes after the playbook has finished, there is a need for a reboot.

[rohkat-aws]

@tomyam1 adding it as a feature request for now .

[chharish]

IIS website will mount FSx as a virtual directory
FSx needs Active Directory.
Active Directory needs a reboot.

[hikeeba]

Adding this would simplify a lot of items that force a reboot during installation.

[avineshwar]

Trying to reboot the instance the instance during deployment causes an error:

Script - scripts/reboot.sh
[stderr]+ reboot
[stderr]
[stderr]Session terminated, terminating shell... ...terminated.

It'd be nice to have a way to tell the CodeDeploy that the instance is going to reboot and it has to wait for it to come up, or is there already a method for doing that?

Encountered a similar issue today. Would be actually useful to have something like this.

[afinzel]

I got hit by this gem today as well. It would be good to have it recover.

[andrewlorien]

My use case :
Windows/IIS environment requires IIS, AD, CodeDeploy, CloudWatch, and various other security/monitoring agents to be installed and configured on boot - requiring either one or two reboots (I know, windows ◔_◔ ). When our ASG scales up, the deployment begins as soon as the agent registers itself, which may be before AD registration is complete. In this case, deployments fail with no output and no error message - just the step they were at when the reboot happened.
If it's too hard to continue the deployment after a reboot, another option would be to add a message like "codedeploy agent did not respond" so that it's clear the problem was with agent/instance rather than the appspec_ script.

[philstrong]

We're looking into this to provide support like this and general patching of instances.

[pixeltrix]

@philstrong this is particularly an issue with the deb package not honouring policy-rc.d (see #44 and #107) during an instance refresh. Our ansible config tries to stop and disable it before the deployment kicks in but it's not always successful - especially on larger instances. It seemed worse today with our first instance refresh after the recent 1.3.2 release - maybe it's booting quicker with the SDK v3?

I get the comments in those issues about wanting to provide a system agnostic option to the installer about not starting on installation but it seems even without that, the postinst script should honour policy-rc.d anyway? Having looked at the postinst script it seems an easy enough change:

  if systemctl >>/dev/null 2>/dev/null; then
    systemctl enable codedeploy-agent
    systemctl start codedeploy-agent
  else
    update-rc.d codedeploy-agent defaults
    service codedeploy-agent start-no-update
  fi

Instead of systemctl and service the block should use deb-systemd-invoke and invoke-rc.d like this:

  if systemctl >>/dev/null 2>/dev/null; then
    deb-systemd-invoke enable codedeploy-agent
    deb-systemd-invoke start codedeploy-agent
  else
    update-rc.d codedeploy-agent defaults
    invoke-rc.d codedeploy-agent start-no-update
  fi

Happy to help test this if you need someone with the appropriate setup.

[nicholas78719]

Any progress on this?

[pixeltrix]

@nicholas78719 it appears not as yet again I had deployment failures during an instance refresh this morning due to the agent starting the deploy automatically after it was installed.