Skip to content

Can't create servers on Digital Ocean #14554

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
penultimonkey opened this issue Nov 10, 2022 · 0 comments
Open

Can't create servers on Digital Ocean #14554

penultimonkey opened this issue Nov 10, 2022 · 0 comments

Comments

@penultimonkey
Copy link

The scripts get as far as creating the droplet but then hang or crash, depending on region.

To Reproduce

On two Ubuntu machines, 20.04 and 22.04, I have done:
git clone https://github.com/trailofbits/algo.git
sudo apt install -y --no-install-recommends python3-virtualenv
[remaining dependencies]
set config opts

  • four users, all new names
  • unattended_reboot:
    enabled: true

./algo

Expected behavior

Creation of droplet in selected region, configs for other users, etc.

Additional context

Love your work; I've used algo with zero trouble two or three times before (2-3 years ago). Now the scripts hang or fail at some point after setting up the droplets. I'm left with running servers that I can ping but can't ssh into (:4160) -- so I haven't been able to inspect their condition much.

The behavior is the same from both local machines but different depending on the region I pick, which makes me think it's a DO problem -- but their hands-off support declines to get too involved and doesn't seem aware of anything weird in their system or different from the last time I was doing this.

Below I've pasted first the output for an sfo3 droplet (hangs indefinitely at Reboot), then the output for an nyc3 (fails at Wait until SSH becomes ready...) (sorry for the ton of pastage). I also tried tor1, which behaved the same as nyc3. One further thing I tried, probably inappropriately but based on another issue thread, was removing "IdentitiesOnly=yes" from ansible.cfg; that made sfo3 hang at a different spot, marked with a comment.

Full log

~/tools/algo$ ./algo

PLAY [localhost] ***************************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [Playbook dir stat] *******************************************************
ok: [localhost]

TASK [Ensure Ansible is not being run in a world writable directory] ***********
ok: [localhost] => {
"changed": false,
"msg": "All assertions passed"
}
[DEPRECATION WARNING]: Use 'ansible.utils.ipaddr' module instead. This feature
will be removed from ansible.netcommon in a release after 2024-01-01.
Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
[WARNING]: The value '' is not a valid IP address or network, passing this
value to ipaddr filter might result in breaking change in future.

TASK [Ensure the requirements installed] ***************************************
ok: [localhost]

TASK [Set required ansible version as a fact] **********************************
ok: [localhost] => (item=ansible==6.1.0)

TASK [Just get the list from default pip] **************************************
ok: [localhost]

TASK [Verify Python meets Algo VPN requirements] *******************************
ok: [localhost] => {
"changed": false,
"msg": "All assertions passed"
}

TASK [Verify Ansible meets Algo VPN requirements] ******************************
ok: [localhost] => {
"changed": false,
"msg": "All assertions passed"
}
[WARNING]: Found variable using reserved name: no_log

PLAY [Ask user for the input] **************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]
[Cloud prompt]
What provider would you like to use?
1. DigitalOcean
2. Amazon Lightsail
3. Amazon EC2
4. Microsoft Azure
5. Google Compute Engine
6. Hetzner Cloud
7. Vultr
8. Scaleway
9. OpenStack (DreamCompute optimised)
10. CloudStack (Exoscale optimised)
11. Linode
12. Install to existing Ubuntu 18.04 or 20.04 server (for more advanced users)

Enter the number of your desired provider
:
1^M
TASK [Cloud prompt] ************************************************************
ok: [localhost]

TASK [Set facts based on the input] ********************************************
ok: [localhost]
[VPN server name prompt]
Name the vpn server
[algo]
:
Algox^M
TASK [VPN server name prompt] **************************************************
ok: [localhost]
[Cellular On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to cellular networks?
[y/N]
:
n^M
TASK [Cellular On Demand prompt] ***********************************************
ok: [localhost]
[Wi-Fi On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to Wi-Fi?
[y/N]
:
n^M
TASK [Wi-Fi On Demand prompt] **************************************************
ok: [localhost]
[Retain the PKI prompt]
Do you want to retain the keys (PKI)? (required to add users in the future, but less secure)
[y/N]
:
y^M
TASK [Retain the PKI prompt] ***************************************************
ok: [localhost]
[DNS adblocking prompt]
Do you want to enable DNS ad blocking on this VPN server?
[y/N]
:
y^M
TASK [DNS adblocking prompt] ***************************************************
ok: [localhost]
[SSH tunneling prompt]
Do you want each user to have their own account for SSH tunneling?
[y/N]
:
y^M
TASK [SSH tunneling prompt] ****************************************************
ok: [localhost]

TASK [Set facts based on the input] ********************************************
ok: [localhost]

PLAY [Provision the server] ****************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

--> Please include the following block of text when reporting issues:

Algo running on: Ubuntu 20.04.5 LTS
Created from git clone. Last commit: 651f949 Update cloud-hetzner.md (#14450)
Python 3.8.10
Runtime variables:
algo_provider "digitalocean"
algo_ondemand_cellular "False"
algo_ondemand_wifi "False"
algo_ondemand_wifi_exclude "X251bGw="
algo_dns_adblocking "True"
algo_ssh_tunneling "True"
wireguard_enabled "True"
dns_encryption "True"

TASK [Display the invocation environment] **************************************
changed: [localhost]

TASK [Install the requirements] ************************************************
ok: [localhost]

TASK [Generate the SSH private key] ********************************************
changed: [localhost]

TASK [Generate the SSH public key] *********************************************
changed: [localhost]

TASK [Copy the private SSH key to /tmp] ****************************************
changed: [localhost]

TASK [Include a provisioning role] *********************************************
[cloud-digitalocean : pause]
Enter your API token. The token must have read and write permissions (https://cloud.digitalocean.com/settings/api/tokens):
(output is hidden):

TASK [cloud-digitalocean : pause] **********************************************
ok: [localhost]

TASK [cloud-digitalocean : Set the token as a fact] ****************************
ok: [localhost]

TASK [cloud-digitalocean : Get regions] ****************************************
ok: [localhost]

TASK [cloud-digitalocean : Set facts about the regions] ************************
ok: [localhost]

TASK [cloud-digitalocean : Set default region] *********************************
ok: [localhost]
[cloud-digitalocean : pause]
What region should the server be located in?
1. ams3 Amsterdam 3
2. blr1 Bangalore 1
3. fra1 Frankfurt 1
4. lon1 London 1
5. nyc1 New York 1
6. nyc3 New York 3
7. sfo3 San Francisco 3
8. sgp1 Singapore 1
9. tor1 Toronto 1

Enter the number of your desired region
[6]
:
7^M
TASK [cloud-digitalocean : pause] **********************************************
ok: [localhost]

TASK [cloud-digitalocean : Set additional facts] *******************************
ok: [localhost]

TASK [cloud-digitalocean : Upload the SSH key] *********************************
changed: [localhost]

TASK [cloud-digitalocean : Creating a droplet...] ******************************
changed: [localhost]

TASK [cloud-digitalocean : set_fact] *******************************************
ok: [localhost]

TASK [cloud-digitalocean : set_fact] *******************************************
ok: [localhost]

TASK [Set subjectAltName as a fact] ********************************************
ok: [localhost]

TASK [Add the server to an inventory group] ************************************
changed: [localhost]

TASK [Additional variables for the server] *************************************
changed: [localhost]

TASK [Wait until SSH becomes ready...] *****************************************
ok: [localhost]

TASK [debug] *******************************************************************
ok: [localhost] => {
"IP_subject_alt_name": "159.223.195.158"
}

HANGS HERE IF NOT IdentitiesOnly=yes IN ansible.cfg

TASK [Wait 600 seconds for target connection to become reachable/usable] *******
ok: [localhost -> 159.223.195.158] => (item=159.223.195.158)

PLAY [Configure the server and install required software] **********************

TASK [Wait until the cloud-init completed] *************************************
ok: [159.223.195.158]

TASK [Ensure the config directory exists] **************************************
changed: [159.223.195.158 -> localhost]

TASK [Dump the ssh config] *****************************************************
changed: [159.223.195.158 -> localhost]

TASK [common : Check the system] ***********************************************
ok: [159.223.195.158]

TASK [common : include_tasks] **************************************************
included: /home/clarke/tools/algo/roles/common/tasks/ubuntu.yml for 159.223.195.158

TASK [common : Gather facts] ***************************************************
ok: [159.223.195.158]

TASK [common : Install software updates] ***************************************
ok: [159.223.195.158]

TASK [common : Check if reboot is required] ************************************
changed: [159.223.195.158]

TASK [common : Reboot] *********************************************************
changed: [159.223.195.158]

HANG FOREVER

#########

Second verse, much like the first:

#########

~/tools/algo$ ./algo

PLAY [localhost] ***************************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [Playbook dir stat] *******************************************************
ok: [localhost]

TASK [Ensure Ansible is not being run in a world writable directory] ***********
ok: [localhost] => {
"changed": false,
"msg": "All assertions passed"
}
[DEPRECATION WARNING]: Use 'ansible.utils.ipaddr' module instead. This feature
will be removed from ansible.netcommon in a release after 2024-01-01.
Deprecation warnings can be disabled by setting deprecation_warnings=False in
ansible.cfg.
[WARNING]: The value '' is not a valid IP address or network, passing this
value to ipaddr filter might result in breaking change in future.

TASK [Ensure the requirements installed] ***************************************
ok: [localhost]

TASK [Set required ansible version as a fact] **********************************
ok: [localhost] => (item=ansible==6.1.0)

TASK [Just get the list from default pip] **************************************
ok: [localhost]

TASK [Verify Python meets Algo VPN requirements] *******************************
ok: [localhost] => {
"changed": false,
"msg": "All assertions passed"
}

TASK [Verify Ansible meets Algo VPN requirements] ******************************
ok: [localhost] => {
"changed": false,
"msg": "All assertions passed"
}
[WARNING]: Found variable using reserved name: no_log

PLAY [Ask user for the input] **************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]
[Cloud prompt]
What provider would you like to use?
1. DigitalOcean
2. Amazon Lightsail
3. Amazon EC2
4. Microsoft Azure
5. Google Compute Engine
6. Hetzner Cloud
7. Vultr
8. Scaleway
9. OpenStack (DreamCompute optimised)
10. CloudStack (Exoscale optimised)
11. Linode
12. Install to existing Ubuntu 18.04 or 20.04 server (for more advanced users)

Enter the number of your desired provider
:
1^M
TASK [Cloud prompt] ************************************************************
ok: [localhost]

TASK [Set facts based on the input] ********************************************
ok: [localhost]
[VPN server name prompt]
Name the vpn server
[algo]
:
Algox^M
TASK [VPN server name prompt] **************************************************
ok: [localhost]
[Cellular On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to cellular networks?
[y/N]
:
n^M
TASK [Cellular On Demand prompt] ***********************************************
ok: [localhost]
[Wi-Fi On Demand prompt]
Do you want macOS/iOS clients to enable "Connect On Demand" when connected to Wi-Fi?
[y/N]
:
n^M
TASK [Wi-Fi On Demand prompt] **************************************************
ok: [localhost]
[Retain the PKI prompt]
Do you want to retain the keys (PKI)? (required to add users in the future, but less secure)
[y/N]
:
y^M
TASK [Retain the PKI prompt] ***************************************************
ok: [localhost]
[DNS adblocking prompt]
Do you want to enable DNS ad blocking on this VPN server?
[y/N]
:
y^M
TASK [DNS adblocking prompt] ***************************************************
ok: [localhost]
[SSH tunneling prompt]
Do you want each user to have their own account for SSH tunneling?
[y/N]
:
y^M
TASK [SSH tunneling prompt] ****************************************************
ok: [localhost]

TASK [Set facts based on the input] ********************************************
ok: [localhost]

PLAY [Provision the server] ****************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

--> Please include the following block of text when reporting issues:

Algo running on: Ubuntu 20.04.5 LTS
Created from git clone. Last commit: 651f949 Update cloud-hetzner.md (#14450)
Python 3.8.10
Runtime variables:
algo_provider "digitalocean"
algo_ondemand_cellular "False"
algo_ondemand_wifi "False"
algo_ondemand_wifi_exclude "X251bGw="
algo_dns_adblocking "True"
algo_ssh_tunneling "True"
wireguard_enabled "True"
dns_encryption "True"

TASK [Display the invocation environment] **************************************
changed: [localhost]

TASK [Install the requirements] ************************************************
ok: [localhost]

TASK [Generate the SSH private key] ********************************************
changed: [localhost]

TASK [Generate the SSH public key] *********************************************
changed: [localhost]

TASK [Copy the private SSH key to /tmp] ****************************************
changed: [localhost]

TASK [Include a provisioning role] *********************************************
[cloud-digitalocean : pause]
Enter your API token. The token must have read and write permissions (https://cloud.digitalocean.com/settings/api/tokens):
(output is hidden):

TASK [cloud-digitalocean : pause] **********************************************
ok: [localhost]

TASK [cloud-digitalocean : Set the token as a fact] ****************************
ok: [localhost]

TASK [cloud-digitalocean : Get regions] ****************************************
ok: [localhost]

TASK [cloud-digitalocean : Set facts about the regions] ************************
ok: [localhost]

TASK [cloud-digitalocean : Set default region] *********************************
ok: [localhost]
[cloud-digitalocean : pause]
What region should the server be located in?
1. ams3 Amsterdam 3
2. blr1 Bangalore 1
3. fra1 Frankfurt 1
4. lon1 London 1
5. nyc1 New York 1
6. nyc3 New York 3
7. sfo3 San Francisco 3
8. sgp1 Singapore 1
9. tor1 Toronto 1

Enter the number of your desired region
[6]
:
6^M
TASK [cloud-digitalocean : pause] **********************************************
ok: [localhost]

TASK [cloud-digitalocean : Set additional facts] *******************************
ok: [localhost]

TASK [cloud-digitalocean : Upload the SSH key] *********************************
changed: [localhost]

TASK [cloud-digitalocean : Creating a droplet...] ******************************
changed: [localhost]

TASK [cloud-digitalocean : set_fact] *******************************************
ok: [localhost]

TASK [cloud-digitalocean : set_fact] *******************************************
ok: [localhost]

TASK [Set subjectAltName as a fact] ********************************************
ok: [localhost]

TASK [Add the server to an inventory group] ************************************
changed: [localhost]

TASK [Additional variables for the server] *************************************
changed: [localhost]

TASK [Wait until SSH becomes ready...] *****************************************
fatal: [localhost]: FAILED! => {"changed": false, "elapsed": 321, "msg": "Timeout when waiting for search string OpenSSH in 142.93.67.37:4160"}

TASK [include_tasks] ***********************************************************
included: /home/clarke/tools/algo/playbooks/rescue.yml for localhost

TASK [debug] *******************************************************************
ok: [localhost] => {
"fail_hint": [
"Sorry, but something went wrong!",
"Please check the troubleshooting guide.",
"https://trailofbits.github.io/algo/troubleshooting.html"
]
}

TASK [Fail the installation] ***************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Failed as requested from task"}

PLAY RECAP *********************************************************************
localhost : ok=40 changed=8 unreachable=0 failed=1 skipped=3 rescued=1 ignored=0

PUT THE OUTPUT HERE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant