Unable to provision custom bucket, error "does not have storage.buckets.create access to project ..."

[tommed]

[REQUIRED] Step 2: Describe your environment

• Operating System version: Windows 10 (latest updates)
• Firebase SDK version: 3.18.4 (firebase --version)
• Library version: 5.12.0 (firebase-admin in packages.json)
• Firebase Product: storage (via npm package firebase-admin)

Steps to reproduce:

Trying to create a custom bucket via the firebase-admin npm module and even though my service account 100% has the storage.buckets.create permission (I've even created a custom role and explicitly assigned it this along with having the standard Storage Admin permission too).

I get the following error:

firebase-adminsdk-lwphn@XXX.iam.gserviceaccount.com does not have storage.buckets.create access to project 265909605284.

I believe this error is not accurate - or I'm doing something really stupid??

Relevant Code:

const admin = require('firebase-admin');

const serviceAccount = require('./storage-service-account.json');

admin.initializeApp({
  credential: admin.credential.cert(serviceAccount),
  projectId: serviceAccount.project_id,
  authDomain: 'XXX.firebaseapp.com',
  storageBucket: 'XXX.appspot.com',
});

const bucket = admin.storage().bucket('my.custom.bucket');
bucket.create() // OR bucket.get({ autoCreate: true });

[hiranya911]

Can you try the same directly with the GCS client library?

const storage = require('@google-cloud/storage');
const cert = require('./storage-service-account.json');
const storageClient = storage({
        credentials: {
          private_key: cert.private_key,
          client_email: cert.client_email,
        },
});
const bucket = storageClient.bucket('my.custom.bucket');
bucket.create();

This will help us localize the error a little bit better.

[tommed]

Thanks. I still get this:

(node:15068) UnhandledPromiseRejectionWarning: ApiError: firebase-adminsdk-lwphn@XXXX.iam.gserviceaccount.com does not have storage.buckets.create access to project 265909605284.
    at Object.parseHttpRespBody (C:\Users\Me\Desktop\test-google-storage\node_modules\@google-cloud\common\src\util.js:199:30)
    at Object.handleResp (C:\Users\Me\Desktop\test-google-storage\node_modules\@google-cloud\common\src\util.js:137:18)
    at C:\Users\Me\Desktop\test-google-storage\node_modules\@google-cloud\common\src\util.js:502:12
    at Request.onResponse [as _callback] (C:\Users\Me\Desktop\test-google-storage\node_modules\retry-request\index.js:195:7)
    at Request.self.callback (C:\Users\Me\Desktop\test-google-storage\node_modules\request\request.js:186:22)
    at emitTwo (events.js:126:13)
    at Request.emit (events.js:214:7)
    at Request.<anonymous> (C:\Users\Me\Desktop\test-google-storage\node_modules\request\request.js:1163:10)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
(node:15068) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:15068) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will
terminate the Node.js process with a non-zero exit code.

[hiranya911]

Ok, I managed to reproduce this error. I believe this happens because the Admin SDK does not specify a project ID when initializing the Storage client. This is good enough to access an existing bucket (whose names are global), but not sufficient to create new buckets. I think there's a simple fix.

@tommed can you modify the above code as follows, and see if that works?

const storage = require('@google-cloud/storage');
const cert = require('./storage-service-account.json');
const storageClient = storage({
        projectId: cert.project_id, // Set project ID
        credentials: {
          private_key: cert.private_key,
          client_email: cert.client_email,
        },
});
const bucket = storageClient.bucket('my.custom.bucket');
bucket.create();

If this works out, I can do the same fix in the Firebase Admin SDK.

[tommed]

Ok thanks!! Next error: The bucket you tried to create is a domain name owned by another user. - though I used a domain which I do own (and Google DNS is managing), though I haven't updated the DNS to point to storage.google..... - do I need to do this prior to creating the bucket?

[tommed]

...Ok so quick update, I tried this with the domain name tomklmnb.vipro.online. which has a CNAME registered with c.storage.googleapis.com. (within the same Google Cloud project) and got the same error:

PS C:\Users\Me\test-google-storage> node .\index.js
(node:12944) UnhandledPromiseRejectionWarning: ApiError: The bucket you tried to create is a domain name owned by another user.
    at Object.parseHttpRespBody (C:\Users\Me\test-google-storage\node_modules\@google-cloud\common\src\util.js:199:30)
    at Object.handleResp (C:\Users\Me\test-google-storage\node_modules\@google-cloud\common\src\util.js:137:18)
    at C:\Users\Me\test-google-storage\node_modules\@google-cloud\common\src\util.js:502:12
    at Request.onResponse [as _callback] (C:\Users\Me\test-google-storage\node_modules\retry-request\index.js:195:7)
    at Request.self.callback (C:\Users\Me\test-google-storage\node_modules\request\request.js:186:22)
    at emitTwo (events.js:126:13)
    at Request.emit (events.js:214:7)
    at Request.<anonymous> (C:\Users\Me\test-google-storage\node_modules\request\request.js:1163:10)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
(node:12944) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:12944) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

In my service account, I am using the project vapps-online which is the same project that owns and manages the vipro.online domain name. Do I need to alter the service account's permissions in order to provision a bucket which has the same name as a domain the admin account owns?

[hiranya911]

I'm not familiar with this issue, and it's not something that can be addressed in the SDK as far as I can tell. Some Googling around revealed this SO post, which suggests verifying the domain and then waiting for 24 hours. In your case you will have to also add your service account as a domain owner.

If that doesn't work out please reach out to GCP support.

[tommed]

Thanks @hiranya911 really helpful links and insight. Feels like I'm getting close now... I have verified ownership of the domain, I then added an owner with the email address XXXXXX@MYDOMAIN.iam.gserviceaccount.com (which is the service account). But unfortunately I'm still getting the same error.

The domain I have verified is the main domain DOMAIN.TLD whereas the bucket I'm trying to create is for a sub-domain (e.g. SUB.DOMAIN.TLD) but according to the docs you sent - what I've done should be sufficient?

Error is still: ApiError: The bucket you tried to create is a domain name owned by another user.

[hiranya911]

GCP support should be able to help you out. We have limited experience dealing with this particular issue. Firebase users typically use the default bucket associated with the project, which is what's exposed via the Firebase Storage client API.

The Cloud Storage Node.js client team might also have a rough idea what's going on here. They can be reached at https://github.com/googleapis/nodejs-storage

[tommed]

Thanks. I emailed them on the 4th, but had no response yet. Unfortunate as I was really hoping to get a workaround for this for a work project. If you have any way of nudging them about this - I'd be extremely grateful!

[hiranya911]

@tommed I filed a bug report internally as well. I'll keep an eye on it.