Accessing a Rancher cluster fails with "unable to get issuer certificate"

[gautaz]

Hello,

This is very similar to #327 but the infrastructure used is public and the certificate is not self signed.

I am using the following piece of code:

const {join} = require('path');
const k8s = require('@kubernetes/client-node');
const kc = new k8s.KubeConfig();

kc.loadFromFile(join(__dirname, 'kubeconfig.yaml'));
kc.setCurrentContext('kubedev');

const k8sApi = kc.makeApiClient(k8s.CoreV1Api);

k8sApi.listPodForAllNamespaces()
  .then(console.log.bind(console))
  .catch(console.error.bind(console));

This fails this way:

Error: unable to get issuer certificate
    at TLSSocket.onConnectSecure (_tls_wrap.js:1474:34)
    at TLSSocket.emit (events.js:310:20)
    at TLSSocket._finishInit (_tls_wrap.js:917:8)
    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:687:12) {
  code: 'UNABLE_TO_GET_ISSUER_CERT'
}

But succeeds if insecure-skip-tls-verify: true is added to the kubeconfig.yaml file.

The kubeconfig.yaml file comes directly from the Rancher UI and works well with kubectl.

Any idea how I can diagnose this issue any further ?

[brendandburns]

Can you send the contents of your kubeconfig file (with personal information removed/obfuscated)

Somehow the certificate authority isn't getting loaded correctly (I think)

[gautaz]

@brendanburns Sorry for answering so lately, I have been a bit too busy these last days.

Here is the kubeconfig file used:

apiVersion: v1
kind: Config
clusters:
  - name: 'kubedev'
    cluster:
      server: 'https://SOMEPUBLICURL'
      certificate-authority-data: "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUdFekNDQ\
        S91Z0F3SUJBZ0lRZlZ0UkpyUjJ1aEhiZEJZTHZGTU5wekFOQmdrcWhraUc5dzBCQVF3RkFEQ0IKa\
        URFTE1Ba0dBMVVFQmhNQ1ZWTXhFekFSQmdOVkJBZ1RDazVsZHlCS1pYSnpaWGt4RkRBU0JnTlZCQ\
        WNUQzBwbApjbk5sZVNCRGFYUjVNUjR3SEFZRFZRUUtFeFZVYUdVZ1ZWTkZVbFJTVlZOVUlFNWxkS\
        GR2Y21zeExqQXNCZ05WCkJBTVRKVlZUUlZKVWNuVnpkQ0JTVTBFZ1EyVnlkR2xtYVdOaGRHbHZia\
        UJCZFhSb2IzSnBkSGt3SGhjTk1UZ3gKTVRBeU1EQXdNREF3V2hjTk16QXhNak14TWpNMU9UVTVXa\
        kNCanpFTE1Ba0dBMVVFQmhNQ1IwSXhHekFaQmdOVgpCQWdURWtkeVpXRjBaWElnVFdGdVkyaGxjM\
        1JsY2pFUU1BNEdBMVVFQnhNSFUyRnNabTl5WkRFWU1CWUdBMVVFCkNoTVBVMlZqZEdsbmJ5Qk1hV\
        zFwZEdWa01UY3dOUVlEVlFRREV5NVRaV04wYVdkdklGSlRRU0JFYjIxaGFXNGcKVm1Gc2FXUmhkR\
        2x2YmlCVFpXTjFjbVVnVTJWeWRtVnlJRU5CTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQwpBU\
        ThBTUlJQkNnS0NBUUVBMW5NejF0YzhJTkFBMGhkRnVOWStCNkkveDBIdU1qREpzR3o5OUovTEVwZ\
        1BMVCtOClRRRU1nZzhYZjJJdTZiaEllZnNXZzA2dDF6SWxrN2NIdjdsUVA2bE13MEFxNlRuLzJZS\
        EtIeFl5UWRxQUpya2oKZW9jZ0h1UC9JSm84bFVSdmgzVUdrRUMwTXBNV0NSQUlJejdTM1ljUGIxM\
        VJGR29LYWNWUEFYSnB6OU9UVEcwRQpvS01iZ242eG1ybnR4WjdGTjNpZm1nZzArMVl1V01RSkRnW\
        mtXN3czM1BHZktHaW9WckNTbzF5ZnU0aVlDQnNrCkhhc3doYTZ2c0M2ZWVwM0J3RUljNGdMdzZ1Q\
        kswdStRRHJUQlFCYndiNFZDU21UM3BEQ2cvcjh1b3lkYWpvdFkKdUszREdSZUVZKzF2VnYyRHkyQ\
        TB4SFMrNXAzYjRlVGx5Z3hmRlFJREFRQUJvNElCYmpDQ0FXb3dId1lEVlIwagpCQmd3Rm9BVVUzb\
        S9XcW9yU3M5VWdPSFltOENkOHJJRFpzc3dIUVlEVlIwT0JCWUVGSTJNWHNSVXJZcmhkK21iCitac\
        0Y0YmdCaldIaE1BNEdBMVVkRHdFQi93UUVBd0lCaGpBU0JnTlZIUk1CQWY4RUNEQUdBUUgvQWdFQ\
        U1CMEcKQTFVZEpRUVdNQlFHQ0NzR0FRVUZCd01CQmdnckJnRUZCUWNEQWpBYkJnTlZIU0FFRkRBU\
        01BWUdCRlVkSUFBdwpDQVlHWjRFTUFRSUJNRkFHQTFVZEh3UkpNRWN3UmFCRG9FR0dQMmgwZEhBN\
        kx5OWpjbXd1ZFhObGNuUnlkWE4wCkxtTnZiUzlWVTBWU1ZISjFjM1JTVTBGRFpYSjBhV1pwWTJGM\
        GFXOXVRWFYwYUc5eWFYUjVMbU55YkRCMkJnZ3IKQmdFRkJRY0JBUVJxTUdnd1B3WUlLd1lCQlFVS\
        E1BS0dNMmgwZEhBNkx5OWpjblF1ZFhObGNuUnlkWE4wTG1OdgpiUzlWVTBWU1ZISjFjM1JTVTBGQ\
        lpHUlVjblZ6ZEVOQkxtTnlkREFsQmdnckJnRUZCUWN3QVlZWmFIUjBjRG92CkwyOWpjM0F1ZFhOb\
        GNuUnlkWE4wTG1OdmJUQU5CZ2txaGtpRzl3MEJBUXdGQUFPQ0FnRUFNcjlodlE1SXcwL0gKdWtkT\
        itKeDRHUUhjRXgyQWIvekRjTFJTbWpFem1sZFMrekdlYTZUdlZLcUpqVUFYYVBnUkVIelN5ckh4V\
        lliSAo3ck0ya1liMk9WRy9ScjhQb0xxMDkzNUp4Q28yRjU3a2FEbDZyNVJPVm0reWV6dS9Db2E5e\
        mNWM0hBTzRPTEdpCkgxOSsyNHJjUmtpMmFBclBzclcwNGpUa1o2azRaZ2xlMHJqOG5TZzZGMEFud\
        25KT0tmMGhQSHpQRS91V0xNVXgKUlAwVDdkV2JxV2xvZDN6dTRmK2srVFk0Q0ZNNW9vUTBuQm56d\
        mc2czFTUTM2eU9vZU5EVDUrK1NSMlJpT1NMdgp4dmNSdmlLRnhtWkVKQ2FPRURLTnlKT3VCNTZEU\
        GkvWitmVkdqbU8rd2VhMDNLYk5JYWlHQ3BYWkxvVW1HdjM4CnNiWlhRbTJWMFRQMk9SUUdna0U0O\
        Vk5WTNJQmJwTlY5bFhqOXA1di8vY1dvYWFzbTU2ZWtCWWRicWJlNG95QUwKbDZsRmhkMnppK1dKT\
        jQ0cERmd0dGL1k0UUE1QzVCSUcrM3Z6eGhGb1l0L2ptUFFUMkJWUGk3RnAyUkJndkdRcQo2akczN\
        UxXak9oU2JKdU1MZS8wQ2pyYVp3VGlYV1RiMnFIU2loclplNjhaazZzK2dvL2x1bnJvdEViYUdtQ\
        WhZCkxjbXNKV1R5WG5XME9NR3VmMXBHZytwUnlyYnhtUkUxYTZWcWU4WUFzT2Y0dm1TeXJjakM4Y\
        XpqVWVxa2srQjUKeU9HQlFNa0tXK0VTUE1GZ0t1T1h3SWxDeXBUUFJwZ1NhYnVZME1MVERYSkxSM\
        jdsazhReUtHT0hRK1N3TWo0SwowMHUvSTVzVUtVRXJtZ1Fma3kzeHh6bElQSzFhRW44PQotLS0tL\
        UVORCBDRVJUSUZJQ0FURS0tLS0tCi0tLS0tQkVHSU4gQ0VSVElGSUNBVEUtLS0tLQpNSUlGZHpDQ\
        0JGK2dBd0lCQWdJUUUrb29jRnYwN08wTU5tTUpnR0ZETmpBTkJna3Foa2lHOXcwQkFRd0ZBREJ2C\
        k1Rc3dDUVlEVlFRR0V3SlRSVEVVTUJJR0ExVUVDaE1MUVdSa1ZISjFjM1FnUVVJeEpqQWtCZ05WQ\
        kFzVEhVRmsKWkZSeWRYTjBJRVY0ZEdWeWJtRnNJRlJVVUNCT1pYUjNiM0pyTVNJd0lBWURWUVFER\
        XhsQlpHUlVjblZ6ZENCRgplSFJsY201aGJDQkRRU0JTYjI5ME1CNFhEVEF3TURVek1ERXdORGd6T\
        0ZvWERUSXdNRFV6TURFd05EZ3pPRm93CmdZZ3hDekFKQmdOVkJBWVRBbFZUTVJNd0VRWURWUVFJR\
        XdwT1pYY2dTbVZ5YzJWNU1SUXdFZ1lEVlFRSEV3dEsKWlhKelpYa2dRMmwwZVRFZU1Cd0dBMVVFQ\
        2hNVlZHaGxJRlZUUlZKVVVsVlRWQ0JPWlhSM2IzSnJNUzR3TEFZRApWUVFERXlWVlUwVlNWSEoxY\
        zNRZ1VsTkJJRU5sY25ScFptbGpZWFJwYjI0Z1FYVjBhRzl5YVhSNU1JSUNJakFOCkJna3Foa2lHO\
        XcwQkFRRUZBQU9DQWc4QU1JSUNDZ0tDQWdFQWdCSmxGellPdzlzSXM5Q3NWdzEyN2MwbjAweXQKV\
        UlOaDRxb2dUUWt0WkFuY3pvbWZ6RDJwN1BiUHdkengwN0hXZXpjb0VTdEgyam5HdkRvWnRGK212W\
        DJkbzJOQwp0bmJ5cVRzcmtmamliOURzRmlDUUNUN2k2SFRKR0xTUjFHSmsyMytqQnZHSUdHcVFJa\
        nk4L2hQd2h4Ujc5dVFmCmp0VGtVY1lSWjBZSVVjdUdGRlEvdkRQK2ZteWMveGFkR0wxUmpqV21wM\
        mJJY21mYklXYXgxSnQ0QThCUU91ak0KOE55OG5reityd1dXTlI5WFdyZi96dms5dHl5MjlsVGR5T\
        2NTT2sydVRJcTNYSnEwdHlBOXluOGlOSzUrTzJobQpBVVRuQVU1R1U1c3pZUGVVdmxNM2tITkQ4e\
        kxEVSsvYnF2NTBUbW5IYTR4Z2s5N0V4d3pmNFRLdXpKTTdVWGlWClo0dnVQVmIrRE5CcER4c1A4e\
        VVtYXpOdDkyNUgrbk5ENVg0T3BXYXhLWHd5aEdOVmljUU53Wk5VTUJrVHJOTjkKTjZmclhUcHNOV\
        npiUWRjUzJxbEpDOS9ZZ0lvSmsyS090V2JQSllqTmhMaXhQNlE1RDlrQ251c1NUSlY4ODJzRgpxV\
        jRXZzh5NForTG9FNTNNVzRMVFRMUHRXLy9lNVhPc0l6c3RBTDgxVlhRSlNkaEpXQnAva2pibVVaS\
        U84eVo5CkhFMFh2TW5zUXliUXYwRmZRS2xFUlBTWjUxZUhubEFmVjFTb1B2MTBZeSt4VUdVSjVsa\
        ENMa01hVExUd0pVZFoKK2dRZWs5UW1Sa3BRZ2JMZXZuaTMvR2NWNGNsWGhCNFBZOWJwWXJyV1gxV\
        XU2bHpHS0FnRUpUbTREaXVwOGt5WApIQWMvRFZMMTdlOHZnZzhDQXdFQUFhT0I5RENCOFRBZkJnT\
        lZIU01FR0RBV2dCU3R2Wmg2TkxRbTkvckVKbFR2CkE3M2dKTXRVR2pBZEJnTlZIUTRFRmdRVVUzb\
        S9XcW9yU3M5VWdPSFltOENkOHJJRFpzc3dEZ1lEVlIwUEFRSC8KQkFRREFnR0dNQThHQTFVZEV3R\
        UIvd1FGTUFNQkFmOHdFUVlEVlIwZ0JBb3dDREFHQmdSVkhTQUFNRVFHQTFVZApId1E5TURzd09hQ\
        TNvRFdHTTJoMGRIQTZMeTlqY213dWRYTmxjblJ5ZFhOMExtTnZiUzlCWkdSVWNuVnpkRVY0CmRHV\
        nlibUZzUTBGU2IyOTBMbU55YkRBMUJnZ3JCZ0VGQlFjQkFRUXBNQ2N3SlFZSUt3WUJCUVVITUFHR\
        0dXaDAKZEhBNkx5OXZZM053TG5WelpYSjBjblZ6ZEM1amIyMHdEUVlKS29aSWh2Y05BUUVNQlFBR\
        GdnRUJBSk5sOWplRApsUTlldzRJY0g5WjM1enlLd0tvSjhPa0xKdkhnd21wMW9jZDV5YmxTWU1nc\
        EVnN3dyUVBXQ2NSMjMrV21nWlduClJ0cUNWNm1Wa3NXMmp3TWliRE4zd1hzeUYyNEh6bG9VUVRvR\
        kpCdjJGQVk3cUNVa0Rydk1LblhkdVhCQlAzelEKWXpZaEJ4OUcvMkNra2VGbnZONGZmaGtVeVdOb\
        mtlcG5CMnUwajR2QWJrTjl3NkdBYkxJZXZGT0ZmZHlRb2FTOApMZTlHY2xjMUJiKzdScnR1YlRlW\
        nR2OGprcEhHYmtENGp5bFc2bC9WWHhSVHJQQlBZZXIzSXN5blZndml1RFFmCkp0bDdHUVZvUDdvO\
        DFEZ0dvdFBtanc3anRIRnRRRUxGaExSQWxTdjBaYUJJZWZZZGdXT1duVTkxNFBoODVJNnAKMGZLd\
        GlyT014eUhOd3U4PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t"
  - name: 'kubedev-kubedev1'
    cluster:
      server: 'https://SOMEIPADDRESS:6443'
      certificate-authority-data: "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN3akNDQ\
        WFxZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFTTVJBd0RnWURWUVFERXdkcmRXSmwKT\
        FdOaE1CNFhEVEU1TURjd01qQTVORFExTTFvWERUSTVNRFl5T1RBNU5EUTFNMW93RWpFUU1BNEdBM\
        VVFQXhNSAphM1ZpWlMxallUQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ\
        0VCQU5jbkRmZ3VqTk0rCkxId2dWSGVXbHpjTVh1N2YrODIrU1BsVU1lYVZPb2thZ2pSR2xFWkNEY\
        Stwd0FyazFwZ21zVGxFK2cxaStpdVIKUmkxbkFobENwOW5VQlZrMk5ZVGtnK0swanFHcmhRQWI5d\
        U5iR0grczl0QlpKNFVUcnlBeWw0UXBSS2tweHd0cApYd0xjdHZ0by8xbEg0QTJsR0xUOFZFNnhCU\
        FlyS3JqVm91QjhIQmROTkJib2xYcVpsaTNsM2tudmxYcXJrci9zClNUT2JGa3FVMXdiMjNvM3l1a\
        klNK2dEek85K2FIS0FrWnpzcEFnMDBRZGlZS21BSURlbXI2T3NCc1czVGJtanMKcjUyM21uc3Vzb\
        mdhaU9oL1p1MUdsUzI1YWV5L2ZUYWVEWkZFOE8zRlR5UG95bW9SZ1dFNVBwdnVqMngvNWpQbAp5c\
        U90WjlJbUQ0VUNBd0VBQWFNak1DRXdEZ1lEVlIwUEFRSC9CQVFEQWdLa01BOEdBMVVkRXdFQi93U\
        UZNQU1CCkFmOHdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBTGl1Q0xZYWRXWEpKa3FYdWhRMG1oR\
        ms3bEg3bXdEeHBzSXEKTFRkRE1QOWJwOVVHSzRpcDhmNVZ0V2dTK0Z3Z2Z3UytFU3FCZ3ozSnJwL\
        2oraVN0WktlM2F6UXlGSElrc01iUApIQkFIcUpES2xmYmhCSnVwNkFIZmcvZDVaalI3azFwb3RtR\
        mtib0dCdUFlYmVpMDUrL2laOWxKZ0FxUlFMb1ZqCndoVjd2cWQ0L0ljTTI2NUs5UldsUXdiR3dDW\
        VpjRjU1ekY2eHgyS2VBT1NUZXVFd2xYNWIySjdzSkFTL2NzVTAKUmtEaG90Uzl5czVFeTAzUHlRc\
        FBmaDRsTmtDS2dvSEFVeW5vWjkxODFpSWZJdFNJZm9VaTl2aGpORVlZRW45bwpscWJ2VkFnKzRCR\
        EdPUDM3TXRGaEU4KzBhRVBjRDlkZWNwWVk3ejFQRnBlY2Q5cmtqUEk9Ci0tLS0tRU5EIENFUlRJR\
        klDQVRFLS0tLS0K"

users:
  - name: 'USER'
    user:
      token: 'THE USER TOKEN'

contexts:
  - name: 'kubedev'
    context:
      user: 'USER'
      cluster: 'kubedev'
  - name: 'kubedev-kubedev1'
    context:
      user: 'USER'
      cluster: 'kubedev-kubedev1'

current-context: 'kubedev'

Hope you'll find something...

[brendandburns]

I'll try to load up this certificate authority and see if there are any obvious errors.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[gautaz]

/remove-lifecycle stale

[brendandburns]

/lifecycle frozen

[able8]

The root cause is that the cluster CA is an intermediate CA certificate which are not trusted by Nodejs nodejs/node#36453

[gautaz]

Thanks @able8 for this analysis.
Let's hope that linking these types of issue to the NodeJS one will pave the way to a NodeJS option controlling the openssl X509_V_FLAG_PARTIAL_CHAIN flag.

[zgaduj]

For temporary solution:
process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0";

[gautaz]

Spring cleaning time!
There is no need to keep this issue anymore, feel free to reopen a new one if needed.