t-SNE script errors

[cweinard]

I'm not able to produce a tsne.csv file with the script. The terminal error messages are below. (I wouldn't be surprised if it's a problem with my local setup. 🤔)

Thanks in advance for any suggestions.

cw11-C02T250DGVC1:tsne cw11$ node tsne.js -i /Users/localmacaccount/Desktop/data/1024
starting with [ '/Users/localmacaccount/.nvm/versions/node/v11.11.0/bin/node',
  '/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/tsne.js',
  '-i',
  '/Users/localmacaccount/Desktop/data/1024' ]

============================
Hi there 👋. Looks like you are running TensorFlow.js in Node.js. To speed things up dramatically, install our node backend, which binds to TensorFlow C++, by running npm i @tensorflow/tfjs-node, or npm i @tensorflow/tfjs-node-gpu if you have CUDA. Then call require('@tensorflow/tfjs-node'); (-gpu suffix for CUDA) at the start of your program. Visit https://github.com/tensorflow/tfjs-node for more details.
============================

(node:39309) UnhandledPromiseRejectionWarning: Error: browserHTTPRequest is not supported outside the web browser without a fetch polyfill.
    at new BrowserHTTPRequest (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-core/dist/io/browser_http.js:48:23)
    at Object.browserHTTPRequest (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-core/dist/io/browser_http.js:278:12)
    at Object.<anonymous> (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:104:50)
    at step (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:42:23)
    at Object.next (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:23:53)
    at /Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:17:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:13:12)
    at Object.loadModelInternal (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:98:12)
    at Object.loadModel (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/exports.js:17:21)
(node:39309) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:39309) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
cw11-C02T250DGVC1:tsne

[cpietsch]

Hi @cweinard, it looks like loading the mobilenet falls back to the online version of loading tf-models via fetch. Can you try node v10.3.0 (nvm use v10.3.0) ? More on tensorflow/tfjs#410

[cweinard]

Thanks so much for the quick response @cpietsch. Really appreciate it.

I installed v10.3.0, re-installed the modules, and still see a problem somewhere. (same error message)

I noticed there was a warning about tensorflow mobilenet peer dependencies? (not sure what that means)

cw11-C02T250DGVC1:tsne cw11$ npm i

> @tensorflow/tfjs-node@0.2.3 install /Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-node
> node scripts/install.js

* Downloading libtensorflow
[==============================] 3944118/bps 100% 0.0s
* Building TensorFlow Node.js bindings

> canvas@2.3.1 install /Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/canvas
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using needle for node-pre-gyp https download 
[canvas] Success: "/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/canvas/build/Release/canvas.node" is installed via remote

> protobufjs@6.8.8 postinstall /Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/protobufjs
> node scripts/postinstall

npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN @tensorflow-models/mobilenet@0.2.2 requires a peer of @tensorflow/tfjs@^0.12.0 but none is installed. You must install peer dependencies yourself.

added 144 packages from 97 contributors and audited 293 packages in 22.977s
found 0 vulnerabilities

cw11-C02T250DGVC1:tsne cw11$ node tsne.js -i /Users/localmacaccount/Desktop/data/1024
starting with [ '/Users/localmacaccount/.nvm/versions/node/v10.3.0/bin/node',
  '/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/tsne.js',
  '-i',
  '/Users/localmacaccount/Desktop/data/1024' ]

============================
Hi there 👋. Looks like you are running TensorFlow.js in Node.js. To speed things up dramatically, install our node backend, which binds to TensorFlow C++, by running npm i @tensorflow/tfjs-node, or npm i @tensorflow/tfjs-node-gpu if you have CUDA. Then call require('@tensorflow/tfjs-node'); (-gpu suffix for CUDA) at the start of your program. Visit https://github.com/tensorflow/tfjs-node for more details.
============================

(node:44726) UnhandledPromiseRejectionWarning: Error: browserHTTPRequest is not supported outside the web browser without a fetch polyfill.
    at new BrowserHTTPRequest (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-core/dist/io/browser_http.js:48:23)
    at Object.browserHTTPRequest (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-core/dist/io/browser_http.js:278:12)
    at Object.<anonymous> (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:104:50)
    at step (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:42:23)
    at Object.next (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:23:53)
    at /Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:17:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:13:12)
    at Object.loadModelInternal (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/models.js:98:12)
    at Object.loadModel (/Users/localmacaccount/Downloads/vikus-viewer-script-master/tsne/node_modules/@tensorflow/tfjs-layers/dist/exports.js:17:21)
(node:44726) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:44726) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
cw11-C02T250DGVC1:tsne cw11$ node -v
v10.3.0

[cpietsch]

I will be back on my computer in the next days and gonna try to fix it.
Meanwhile you could try adding global.fetch = require('node-fetch') to the top of tsne.js and install https://www.npmjs.com/package/node-fetch

[cweinard]

Will do. Many thanks!

[cpietsch]

Just had time to look into it. I accidentally comment out tfjs-node which enables non browser fetch for tfjs.
Reason: I am still running on an OSX 10.11.6 which tfjs-node doesn't support. Crippled like that I didn't see the error :)
@cweinard can you give it another try ?

[cweinard]

Thank you so much! I've got a tsne.csv

[cpietsch]

Cool. What did you visualize? Chad Weinard <notifications@github.com> schrieb am Sa., 16. März 2019, 14:10:

…

Thank you so much! I've got a tsne.csv — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAH5-UeOnSwNv1On7WRVkM8wDSIa3Mrsks5vXO0qgaJpZM4bmL4g> .

[cweinard]

I started with a small part of our art collection that we loan to students. I'm running a collection data project at the Williams College Museum of Art, and we're looking into web-based visualization tools. Next, I'll load in our entire collection and see how it goes. (~13k images) This is fantastic—looks like a great tool to build on. Thanks for sharing it, and thanks again for your help.

[cpietsch]

Thanks for the kudos. I am eager to see what your 13k collection will look like! Make sure to generate <128px (even 100px) spritesheet textures via node textures.js -i /path/to/images -s 128 -f jpg.