Terraform init fails with x509: certificate signed by unknown authority when using Zscaler

[flyhard]

Hi there,
I am trying to run terraform 0.10.0, but terraform init fails with

$ terraform init

Initializing provider plugins...
- Downloading plugin for provider "aws"...

Error installing provider "aws": Get https://releases.hashicorp.com/terraform-provider-aws/: x509: certificate signed by unknown authority.

Terraform analyses the configuration and state and automatically downloads
plugins for the providers used. However, when attempting to download this
plugin an unexpected error occured.

This may be caused if for some reason Terraform is unable to reach the
plugin repository. The repository may be unreachable if access is blocked
by a firewall.

If automatic installation is not possible or desirable in your environment,
you may alternatively manually install plugins by downloading a suitable
distribution package and placing the plugin's executable file in the
following directory:
    terraform.d/plugins/darwin_amd64

My company is using ZScaler to mess with CA certificates, so I know that the reason is that the certificate used by HashiCorp is not the one I will see, but most applications have a switch to ignore it to let me run it anyways.

Could we get a 'ignore certificate errors' switch, please?

[ghost]

Same issue here, also Mac OS. This behavior appears to be a regression in 0.10.0.

$ brew switch terraform 0.9.11
Cleaning /usr/local/Cellar/terraform/0.10.0
Cleaning /usr/local/Cellar/terraform/0.9.11
2 links created for /usr/local/Cellar/terraform/0.9.11

$ terraform init

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your environment. If you forget, other
commands will detect it and remind you to do so if necessary.

$ brew switch terraform 0.10.0
Cleaning /usr/local/Cellar/terraform/0.10.0
Cleaning /usr/local/Cellar/terraform/0.9.11
2 links created for /usr/local/Cellar/terraform/0.10.0

$ terraform init

Initializing provider plugins...
- Downloading plugin for provider "aws"...

Error installing provider "aws": Get https://releases.hashicorp.com/terraform-provider-aws/: x509: certificate signed by unknown authority.

Terraform analyses the configuration and state and automatically downloads
plugins for the providers used. However, when attempting to download this
plugin an unexpected error occured.

This may be caused if for some reason Terraform is unable to reach the
plugin repository. The repository may be unreachable if access is blocked
by a firewall.

If automatic installation is not possible or desirable in your environment,
you may alternatively manually install plugins by downloading a suitable
distribution package and placing the plugin's executable file in the
following directory:
    terraform.d/plugins/darwin_amd64

- Downloading plugin for provider "null"...

Error installing provider "null": Get https://releases.hashicorp.com/terraform-provider-null/: x509: certificate signed by unknown authority.

Terraform analyses the configuration and state and automatically downloads
plugins for the providers used. However, when attempting to download this
plugin an unexpected error occured.

This may be caused if for some reason Terraform is unable to reach the
plugin repository. The repository may be unreachable if access is blocked
by a firewall.

If automatic installation is not possible or desirable in your environment,
you may alternatively manually install plugins by downloading a suitable
distribution package and placing the plugin's executable file in the
following directory:
    terraform.d/plugins/darwin_amd64```

[ghost]

Oh, not quite new. This was probably an issue but wasn't visible until this change:

Terraform providers are no longer distributed as part of the main Terraform distribution. Instead, they are installed automatically as part of running terraform init. It is therefore now mandatory to run terraform init before any other operations that use provider plugins, to ensure that the required plugins are installed and properly initialized.

[jbardin]

Hi @flyhard,

Thanks for filing the issue, and this may be something we can add in in the future. In the meantime,
is it possible for you to either add releases.hashicorp.com to a whitelist, or add the ZScalar certificates to the trusted CAs on your system?

@dsnellgrove, Yes, Terraform now only pulls the needed providers at init time. For now that they are only pulled from releases.hashicorp.com.

Another option is always to manually install the providers. You can fetch the desired versions from the releases site and unpack them along side the terraform binary, or specify a location using init -plugin-dir.

[ghost]

@jbardin The ZScalar certificates are already trusted CAs on the system (otherwise not much would get done :) ), Terraform doesn't seem to be using them.

[jbardin]

@dsnellgrove, what is the system running terraform in your case?

[ghost]

Mac OS 10.12.6

[ghost]

It looks like Terraform no longer loads the correct parts of the Keychain due to changes in Go. This prevents TLS validation when performing inspection (which is the ZScaler feature impacting us):
#12711
golang/go#16532

[jbardin]

Yes, I'll take a look at the Go issue. The go1.9 code looks like it may be loading the correct keychains now even though that issue is still open, so hopefully that release will clear this up.

Until then the quickest way forward is to fetch the providers manually.

[jbardin]

Hi,

The linked Go issue is fixed by 2 separate pieces, one of which has been recently merged and should fix our particular issue. That issue as a whole is marked as a go1.9 release blocker, so I'm fairly confident that the next Go release will close this for us.

[bitbrain]

I am also experiencing the exact same issue as explained by @dsnellgrove.

[pgr-mattgartman]

We are seeing this issue still as well and originally reported it in #12711.

Looking forward to a resolution.

[bitbrain]

@pgr-mattgartman Which terraform version are you using? I have upgraded to the latest version and for me it works now. Also make sure to update Go to the latest version, there was a bug which caused it.

[pgr-mattgartman]

@bitbrain I was on an older version, I pulled down 0.10.6 and things seem to work fine on the mac now. Looks like v0.10.2 switched the version over to go 1.9.

Thanks!

[jbardin]

Thanks for the confirmation @bitbrain.
While the upstream Go fix isn't complete, the terraform releases don't use cgo, and should now be getting the correct set of root certs.

[stancy007]

check proxy settings. I got this error but once I checked on proxy settings,I was able to work on TF

[omarabid]

I'm getting the same issue using the latest Terraform version on OSX

Error installing provider "null": Get https://releases.hashicorp.com/terraform-provider-null/: x509: certificate signed by unknown authority.

[dmceathron]

Also getting this issue using WSL(Ubuntu)

[bitbrain]

@dmceathron @omarabid which Terraform version are you using exactly? Also, what is the version of your local Go installation?

[omarabid]

@bitbrain Pretty sure was using the latest when I posted the comment. I don't have a Go installation as far as I'm aware.

[dmceathron]

@bitbrain
Terraform v0.11.11

• provider.aws v1.57.0
  This issue has been resolved for me, I downloaded the zip of the provider I needed and put it in my terraform app dir.

[mmack]

Same problem here... 1.11.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.