RedHat R Install with no Internet Access #7055
Description
Hi there!
Looking to install arrow via R but running into some issues.
The RedHat 7.6 server is behind a firewall with no external internet access. Think that might relevant in this situation.
The arrow packages have been installed via internal repo so should be available. I don't think they are being picked up by the R.
[root@rminer-dev library]# yum list installed 'ar*' 'par*'
Installed Packages
arrow-devel.x86_64 0.17.0-1.el7 @arrow
arrow-glib-devel.x86_64 0.17.0-1.el7 @arrow
arrow-glib-libs.x86_64 0.17.0-1.el7 @arrow
arrow-libs.x86_64 0.17.0-1.el7 @arrow
parquet-devel.x86_64 0.17.0-1.el7 @arrow
parquet-glib-devel.x86_64 0.17.0-1.el7 @arrow
parquet-glib-libs.x86_64 0.17.0-1.el7 @arrow
parquet-libs.x86_64 0.17.0-1.el7 @arrow
When I install via our internal repo in R I am getting the following error.
I added the following flag for verbose logging.
export ARROW_R_DEV=true
-sh-4.2$ R
R version 3.6.0 (2019-04-26) -- "Planting of a Tree"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-redhat-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> install.packages("/data/software/r_packages/arrow_0.17.0.tar.gz", lib="/usr/lib64/R/library")
inferring 'repos = NULL' from 'pkgs'
* installing *source* package ‘arrow’ ...
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation
*** Generating code with data-raw/codegen.R
Fatal error: cannot open file 'data-raw/codegen.R': No such file or directory
trying URL 'https://dl.bintray.com/ursalabs/arrow-r/libarrow/src/arrow-0.17.0.zip'
Error in download.file(from_url, to_file, quiet = quietly) :
cannot open URL 'https://dl.bintray.com/ursalabs/arrow-r/libarrow/src/arrow-0.17.0.zip'
trying URL 'https://www.apache.org/dyn/closer.lua?action=download&filename=arrow/arrow-0.17.0/apache-arrow-0.17.0.tar.gz'
^C
Execution halted
./configure: line 132: cd: libarrow/arrow-0.17.0/lib: No such file or directory
PKG_CFLAGS=-I/tmp/RtmpYUUPaA/R.INSTALL1092b73ddc984/arrow/libarrow/arrow-0.17.0/include -DARROW_R_WITH_ARROW
PKG_LIBS=-L/tmp/RtmpYUUPaA/R.INSTALL1092b73ddc984/arrow/libarrow/arrow-0.17.0/lib -larrow_dataset -lparquet -larrow
** libs
g++ -m64 -std=gnu++11 -I"/usr/include/R" -DNDEBUG -I/tmp/RtmpYUUPaA/R.INSTALL1092b73ddc984/arrow/libarrow/arrow-0.17.0/include -DARROW_R_WITH_ARROW -I"/usr/lib64/R/library/Rcpp/include" -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -c array.cpp -o array.o
In file included from array.cpp:18:0:
./arrow_types.h:203:31: fatal error: arrow/dataset/api.h: No such file or directory
#include <arrow/dataset/api.h>
^
compilation terminated.
make: *** [array.o] Error 1
ERROR: compilation failed for package ‘arrow’
* removing ‘/usr/lib64/R/library/arrow’
* restoring previous ‘/usr/lib64/R/library/arrow’
Warning message:
In install.packages("/data/software/r_packages/arrow_0.17.0.tar.gz", :
installation of package ‘/data/software/r_packages/arrow_0.17.0.tar.gz’ had non-zero exit status
So it is not finding the Linux dependencies and is going off to try and download then from internet?
I have also tried installing with following enviornment variables with no luck.
export LIBARROW_MINIMAL=false
export NOT_CRAN=true
Any suggestions would be greatly appreciated. I have a feeling I might be missing something.
The following gives a similar message
export LIBARROW_BUILD=false
> install.packages("/data/software/r_packages/arrow_0.17.0.tar.gz", lib="/usr/lib64/R/library")
inferring 'repos = NULL' from 'pkgs'
* installing *source* package ‘arrow’ ...
** package ‘arrow’ successfully unpacked and MD5 sums checked
** using staged installation
*** Generating code with data-raw/codegen.R
Fatal error: cannot open file 'data-raw/codegen.R': No such file or directory
*** Proceeding without C++ dependencies
./configure: line 132: cd: libarrow/arrow-0.17.0/lib: No such file or directory
PKG_CFLAGS=-I/tmp/RtmpIZM0Fn/R.INSTALL112a67397e15d/arrow/libarrow/arrow-0.17.0/include -DARROW_R_WITH_ARROW
PKG_LIBS=-L/tmp/RtmpIZM0Fn/R.INSTALL112a67397e15d/arrow/libarrow/arrow-0.17.0/lib -larrow_dataset -lparquet -larrow
** libs
g++ -m64 -std=gnu++11 -I"/usr/include/R" -DNDEBUG -I/tmp/RtmpIZM0Fn/R.INSTALL112a67397e15d/arrow/libarrow/arrow-0.17.0/include -DARROW_R_WITH_ARROW -I"/usr/lib64/R/library/Rcpp/include" -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -c array.cpp -o array.o
In file included from array.cpp:18:0:
./arrow_types.h:203:31: fatal error: arrow/dataset/api.h: No such file or directory
#include <arrow/dataset/api.h>
^
compilation terminated.
make: *** [array.o] Error 1
ERROR: compilation failed for package ‘arrow’
* removing ‘/usr/lib64/R/library/arrow’
* restoring previous ‘/usr/lib64/R/library/arrow’
Warning message:
In install.packages("/data/software/r_packages/arrow_0.17.0.tar.gz", :
installation of package ‘/data/software/r_packages/arrow_0.17.0.tar.gz’ had non-zero exit status