openMP Mac install not working

[bhive01]

I've been trying to get timing comparisons of various file IO functions today and managed to get everything working except the OpenMP implementation of fwrite(). When I run fwrite() in my benchmark I get the following:

library(data.table)

set.seed(3-29-16)

rows <- 100000

x <- data.frame(ints = round(runif(rows, -100, 100)), stringsAsFactors = FALSE)
x$floats <- runif(rows, -100, 100)
x$bools <- sample(c(TRUE, FALSE), rows, replace = TRUE)
x$dates <- as.POSIXct(runif(rows, 100000000, 1459293171), origin = "1970-01-01")
x$categories <- as.factor(sample(c(LETTERS, 0:9), rows, replace = TRUE))
x$strings <- replicate(rows, paste0(sample(letters, sample(1:10, 1), replace = TRUE), collapse = ""))

DTCSV =fwrite(x, "x.DT.csv")
#Your platform/environment has not detected OpenMP support. fwrite() will still work, but slower in single threaded mode.

I followed the instructions here: https://github.com/Rdatatable/data.table/wiki/Installation
Created a Makevar file in ~/.R/ with the following flags:

cat ~/.R/Makevars
CC=clang-omp -fopenmp
CXX=clang-omp++ -fopenmp

Running the script above after installation and creation of Makevar file gives the above message.

When I tried to reinstall Rcpp or data.table from source (hoping that it would compile against the OpenMP capable compilier) I get an error.

install.packages("data.table", type = "source",
     repos = "https://Rdatatable.github.io/data.table")

trying URL 'https://Rdatatable.github.io/data.table/src/contrib/data.table_1.9.7.tar.gz'
Content type 'application/octet-stream' length 2721566 bytes (2.6 MB)
==================================================
downloaded 2.6 MB

* installing *source* package ‘data.table’ ...
** libs
clang-omp -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c assign.c -o assign.o
In file included from assign.c:1:
In file included from ./data.table.h:1:
/Library/Frameworks/R.framework/Resources/include/R.h:45:12: fatal error: 'stdlib.h' file not found
#  include <stdlib.h> /* Not used by R itself, but widely assumed in packages */
           ^
1 error generated.

The downloaded source packages are in
    ‘/private/var/folders/zg/hy0k4g312536_4l1qc1w27640000gn/T/RtmpXM09oF/downloaded_packages’
make: *** [assign.o] Error 1
ERROR: compilation failed for package ‘data.table’
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table’
Warning message:
In install.packages("data.table", type = "source", repos = "https://Rdatatable.github.io/data.table") :
  installation of package ‘data.table’ had non-zero exit status

#############################
install.packages("Rcpp", type = "source")
trying URL 'http://cran.rstudio.com/src/contrib/Rcpp_0.12.4.tar.gz'
Content type 'application/x-gzip' length 2402065 bytes (2.3 MB)
==================================================
downloaded 2.3 MB

* installing *source* package ‘Rcpp’ ...
** package ‘Rcpp’ successfully unpacked and MD5 sums checked
** libs
clang-omp++ -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include/ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c Date.cpp -o Date.o
In file included from Date.cpp:31:
In file included from ../inst/include/Rcpp.h:27:
In file included from ../inst/include/RcppCommon.h:29:
In file included from ../inst/include/Rcpp/r/headers.h:48:
In file included from ../inst/include/Rcpp/platform/compiler.h:100:
/usr/local/Cellar/clang-omp/2015-04-01/libexec/bin/../include/c++/v1/cmath:301:10: fatal error: 'math.h' file not found
#include <math.h>
         ^
1 error generated.
make: *** [Date.o] Error 1
ERROR: compilation failed for package ‘Rcpp’
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/Rcpp’

The downloaded source packages are in
    ‘/private/var/folders/zg/hy0k4g312536_4l1qc1w27640000gn/T/RtmpXM09oF/downloaded_packages’
Warning message:
In install.packages("Rcpp", type = "source") :
  installation of package ‘Rcpp’ had non-zero exit status
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/Rcpp’

I googled around and found that g++-5 on homebrew comes with openMP so I tapped it. Rcpp compiles, but then data.table does not.

cat ~/.R/Makevars
CC=g++-5 -fopenmp
CXX=g++-5 -fopenmp

install.packages("Rcpp", type = "source")
trying URL 'http://cran.rstudio.com/src/contrib/Rcpp_0.12.4.tar.gz'
Content type 'application/x-gzip' length 2402065 bytes (2.3 MB)
==================================================
downloaded 2.3 MB

* installing *source* package ‘Rcpp’ ...
** package ‘Rcpp’ successfully unpacked and MD5 sums checked
** libs
g++-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include/ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c Date.cpp -o Date.o
g++-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include/ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c Module.cpp -o Module.o
g++-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include/ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c Rcpp_init.cpp -o Rcpp_init.o
g++-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include/ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c api.cpp -o api.o
g++-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include/ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c attributes.cpp -o attributes.o
g++-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I../inst/include/ -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c barrier.cpp -o barrier.o
g++-5 -fopenmp -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o Rcpp.so Date.o Module.o Rcpp_init.o api.o attributes.o barrier.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/3.3/Resources/library/Rcpp/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (Rcpp)

The downloaded source packages are in
    ‘/private/var/folders/zg/hy0k4g312536_4l1qc1w27640000gn/T/RtmpXM09oF/downloaded_packages’


############################

install.packages("data.table", type = "source",
     repos = "https://Rdatatable.github.io/data.table")
trying URL 'https://Rdatatable.github.io/data.table/src/contrib/data.table_1.9.7.tar.gz'
Content type 'application/octet-stream' length 2721566 bytes (2.6 MB)
==================================================
downloaded 2.6 MB

* installing *source* package ‘data.table’ ...
** libs
g++-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c assign.c -o assign.o
In file included from assign.c:1:0:
data.table.h:73:29: error: two or more data types in declaration of 'parameter'
 SEXP which(SEXP x, Rboolean bool);
                             ^
assign.c: In function 'int _selfrefok(SEXP, Rboolean, Rboolean)':
assign.c:115:26: error: invalid conversion from 'void*' to 'SEXP {aka SEXPREC*}' [-fpermissive]
     p = R_ExternalPtrAddr(v);
                          ^

The downloaded source packages are in
    ‘/private/var/folders/zg/hy0k4g312536_4l1qc1w27640000gn/T/RtmpXM09oF/downloaded_packages’
Warning message:
In install.packages("data.table", type = "source", repos = "https://Rdatatable.github.io/data.table") :
  installation of package ‘data.table’ had non-zero exit status
assign.c: In function 'Rboolean selfrefok(SEXP, Rboolean)':
assign.c:137:44: error: cannot convert 'bool' to 'Rboolean' in return
     return(_selfrefok(x, FALSE, verbose)==1);
                                            ^
assign.c: In function 'Rboolean selfrefnamesok(SEXP, Rboolean)':
assign.c:140:43: error: cannot convert 'bool' to 'Rboolean' in return
     return(_selfrefok(x, TRUE, verbose)==1);
                                           ^
assign.c: In function 'SEXPREC* alloccol(SEXP, R_len_t, Rboolean)':
assign.c:195:17: error: expected unqualified-id before 'class'
     SEXP names, class;
                 ^
assign.c:199:5: error: expected primary-expression before 'class'
     class = getAttrib(dt, R_ClassSymbol);
     ^
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c:200:16: error: expected primary-expression before 'class'
     if (isNull(class)) error("dt passed to alloccol has no class attribute. Please report result of traceback() to datatable-help.");
                ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:303:21: note: in definition of macro 'TYPEOF'
 #define TYPEOF(x) ((x)->sxpinfo.type)
                     ^
assign.c:200:9: note: in expansion of macro 'isNull'
     if (isNull(class)) error("dt passed to alloccol has no class attribute. Please report result of traceback() to datatable-help.");
         ^
assign.c:200:16: error: expected ')' before 'class'
     if (isNull(class)) error("dt passed to alloccol has no class attribute. Please report result of traceback() to datatable-help.");
                ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:303:21: note: in definition of macro 'TYPEOF'
 #define TYPEOF(x) ((x)->sxpinfo.type)
                     ^
assign.c:200:9: note: in expansion of macro 'isNull'
     if (isNull(class)) error("dt passed to alloccol has no class attribute. Please report result of traceback() to datatable-help.");
         ^
assign.c:200:133: error: expected ')' before ';' token
     if (isNull(class)) error("dt passed to alloccol has no class attribute. Please report result of traceback() to datatable-help.");
                                                                                                                                     ^
assign.c:200:133: error: expected ')' before ';' token
assign.c:200:133: error: expected ')' before ';' token
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c: In function 'SEXPREC* alloccolwrapper(SEXP, SEXP, SEXP)':
assign.c:227:76: error: invalid conversion from 'int' to 'Rboolean' [-fpermissive]
     SEXP ans = PROTECT(alloccol(dt, INTEGER(newncol)[0], LOGICAL(verbose)[0]));
                                                                            ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:661:31: note: in definition of macro 'PROTECT'
 #define PROTECT(s) Rf_protect(s)
                               ^
assign.c:193:6: note:   initializing argument 3 of 'SEXPREC* alloccol(SEXP, R_len_t, Rboolean)'
 SEXP alloccol(SEXP dt, R_len_t n, Rboolean verbose)
      ^
assign.c: In function 'Rboolean isDatatable(SEXP)':
assign.c:252:10: error: expected primary-expression before 'class'
     SEXP class = getAttrib(x, R_ClassSymbol);
          ^
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c:253:28: error: expected primary-expression before 'class'
     for (int i=0; i<length(class); i++) {
                            ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:1222:30: note: in definition of macro 'length'
 #define length(x)  Rf_length(x)
                              ^
assign.c:254:36: error: expected primary-expression before 'class'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) return(TRUE);
                                    ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:382:41: note: in definition of macro 'DATAPTR'
 #define DATAPTR(x) (((SEXPREC_ALIGN *) (x)) + 1)
                                         ^
assign.c:254:20: note: in expansion of macro 'CHAR'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) return(TRUE);
                    ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:389:35: note: in expansion of macro 'DATAPTR'
 #define STRING_ELT(x,i) ((SEXP *) DATAPTR(x))[i]
                                   ^
assign.c:254:25: note: in expansion of macro 'STRING_ELT'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) return(TRUE);
                         ^
assign.c:254:36: error: expected ')' before 'class'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) return(TRUE);
                                    ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:382:41: note: in definition of macro 'DATAPTR'
 #define DATAPTR(x) (((SEXPREC_ALIGN *) (x)) + 1)
                                         ^
assign.c:254:20: note: in expansion of macro 'CHAR'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) return(TRUE);
                    ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:389:35: note: in expansion of macro 'DATAPTR'
 #define STRING_ELT(x,i) ((SEXP *) DATAPTR(x))[i]
                                   ^
assign.c:254:25: note: in expansion of macro 'STRING_ELT'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) return(TRUE);
                         ^
assign.c:254:80: error: expected ')' before ';' token
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) return(TRUE);
                                                                                ^
assign.c:254:80: error: expected ')' before ';' token
assign.c:254:80: error: expected ')' before ';' token
assign.c:254:80: error: expected ')' before ';' token
assign.c:254:80: error: expected ')' before ';' token
assign.c:254:80: error: expected ')' before ';' token
assign.c:254:80: error: expected ')' before ';' token
assign.c:254:80: error: expected ')' before ';' token
assign.c: In function 'SEXPREC* selfrefokwrapper(SEXP, SEXP)':
assign.c:272:63: error: invalid conversion from 'int' to 'Rboolean' [-fpermissive]
     return ScalarInteger(_selfrefok(x,FALSE,LOGICAL(verbose)[0]));
                                                               ^
assign.c:104:12: note:   initializing argument 3 of 'int _selfrefok(SEXP, Rboolean, Rboolean)'
 static int _selfrefok(SEXP x, Rboolean checkNames, Rboolean verbose) {
            ^
assign.c: In function 'SEXPREC* assign(SEXP, SEXP, SEXP, SEXP, SEXP, SEXP)':
assign.c:285:93: error: expected unqualified-id before 'class'
     SEXP targetcol, RHS, names, nullint, thisvalue, thisv, targetlevels, newcol, s, colnam, class, tmp, colorder, key, index, a;
                                                                                             ^
assign.c:287:39: error: invalid conversion from 'int' to 'Rboolean' [-fpermissive]
     Rboolean verbose = LOGICAL(verb)[0], anytodelete=FALSE, isDataTable=FALSE;
                                       ^
assign.c:297:5: error: expected primary-expression before 'class'
     class = getAttrib(dt, R_ClassSymbol);
     ^
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c:298:16: error: expected primary-expression before 'class'
     if (isNull(class)) error("Input passed to assign has no class attribute. Must be a data.table or data.frame.");
                ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:303:21: note: in definition of macro 'TYPEOF'
 #define TYPEOF(x) ((x)->sxpinfo.type)
                     ^
assign.c:298:9: note: in expansion of macro 'isNull'
     if (isNull(class)) error("Input passed to assign has no class attribute. Must be a data.table or data.frame.");
         ^
assign.c:298:16: error: expected ')' before 'class'
     if (isNull(class)) error("Input passed to assign has no class attribute. Must be a data.table or data.frame.");
                ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:303:21: note: in definition of macro 'TYPEOF'
 #define TYPEOF(x) ((x)->sxpinfo.type)
                     ^
assign.c:298:9: note: in expansion of macro 'isNull'
     if (isNull(class)) error("Input passed to assign has no class attribute. Must be a data.table or data.frame.");
         ^
assign.c:298:115: error: expected ')' before ';' token
     if (isNull(class)) error("Input passed to assign has no class attribute. Must be a data.table or data.frame.");
                                                                                                                   ^
assign.c:298:115: error: expected ')' before ';' token
assign.c:298:115: error: expected ')' before ';' token
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c:303:24: error: expected primary-expression before 'class'
     for (i=0; i<length(class); i++) {   // There doesn't seem to be an R API interface to inherits(), but manually here isn't too bad.
                        ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:1222:30: note: in definition of macro 'length'
 #define length(x)  Rf_length(x)
                              ^
assign.c:304:36: error: expected primary-expression before 'class'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) break;
                                    ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:382:41: note: in definition of macro 'DATAPTR'
 #define DATAPTR(x) (((SEXPREC_ALIGN *) (x)) + 1)
                                         ^
assign.c:304:20: note: in expansion of macro 'CHAR'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) break;
                    ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:389:35: note: in expansion of macro 'DATAPTR'
 #define STRING_ELT(x,i) ((SEXP *) DATAPTR(x))[i]
                                   ^
assign.c:304:25: note: in expansion of macro 'STRING_ELT'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) break;
                         ^
assign.c:304:36: error: expected ')' before 'class'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) break;
                                    ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:382:41: note: in definition of macro 'DATAPTR'
 #define DATAPTR(x) (((SEXPREC_ALIGN *) (x)) + 1)
                                         ^
assign.c:304:20: note: in expansion of macro 'CHAR'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) break;
                    ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:389:35: note: in expansion of macro 'DATAPTR'
 #define STRING_ELT(x,i) ((SEXP *) DATAPTR(x))[i]
                                   ^
assign.c:304:25: note: in expansion of macro 'STRING_ELT'
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) break;
                         ^
assign.c:304:73: error: expected ')' before ';' token
         if (strcmp(CHAR(STRING_ELT(class, i)), "data.table") == 0) break;
                                                                         ^
assign.c:304:73: error: expected ')' before ';' token
assign.c:304:73: error: expected ')' before ';' token
assign.c:304:73: error: expected ')' before ';' token
assign.c:304:73: error: expected ')' before ';' token
assign.c:304:73: error: expected ')' before ';' token
assign.c:304:73: error: expected ')' before ';' token
assign.c:304:73: error: expected ')' before ';' token
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c:306:18: error: expected primary-expression before 'class'
     if (i<length(class))
                  ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:1222:30: note: in definition of macro 'length'
 #define length(x)  Rf_length(x)
                              ^
assign.c:309:28: error: expected primary-expression before 'class'
         for (i=0; i<length(class); i++) {
                            ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:1222:30: note: in definition of macro 'length'
 #define length(x)  Rf_length(x)
                              ^
assign.c:310:40: error: expected primary-expression before 'class'
             if (strcmp(CHAR(STRING_ELT(class, i)), "data.frame") == 0) break;
                                        ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:382:41: note: in definition of macro 'DATAPTR'
 #define DATAPTR(x) (((SEXPREC_ALIGN *) (x)) + 1)
                                         ^
assign.c:310:24: note: in expansion of macro 'CHAR'
             if (strcmp(CHAR(STRING_ELT(class, i)), "data.frame") == 0) break;
                        ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:389:35: note: in expansion of macro 'DATAPTR'
 #define STRING_ELT(x,i) ((SEXP *) DATAPTR(x))[i]
                                   ^
assign.c:310:29: note: in expansion of macro 'STRING_ELT'
             if (strcmp(CHAR(STRING_ELT(class, i)), "data.frame") == 0) break;
                             ^
assign.c:310:40: error: expected ')' before 'class'
             if (strcmp(CHAR(STRING_ELT(class, i)), "data.frame") == 0) break;
                                        ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:382:41: note: in definition of macro 'DATAPTR'
 #define DATAPTR(x) (((SEXPREC_ALIGN *) (x)) + 1)
                                         ^
assign.c:310:24: note: in expansion of macro 'CHAR'
             if (strcmp(CHAR(STRING_ELT(class, i)), "data.frame") == 0) break;
                        ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:389:35: note: in expansion of macro 'DATAPTR'
 #define STRING_ELT(x,i) ((SEXP *) DATAPTR(x))[i]
                                   ^
assign.c:310:29: note: in expansion of macro 'STRING_ELT'
             if (strcmp(CHAR(STRING_ELT(class, i)), "data.frame") == 0) break;
                             ^
assign.c:310:77: error: expected ')' before ';' token
             if (strcmp(CHAR(STRING_ELT(class, i)), "data.frame") == 0) break;
                                                                             ^
assign.c:310:77: error: expected ')' before ';' token
assign.c:310:77: error: expected ')' before ';' token
assign.c:310:77: error: expected ')' before ';' token
assign.c:310:77: error: expected ')' before ';' token
assign.c:310:77: error: expected ')' before ';' token
assign.c:310:77: error: expected ')' before ';' token
assign.c:310:77: error: expected ')' before ';' token
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c:312:25: error: expected primary-expression before 'class'
         if (i == length(class)) error("Input is not a data.table, data.frame or an object that inherits from either.");
                         ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:1222:30: note: in definition of macro 'length'
 #define length(x)  Rf_length(x)
                              ^
assign.c:350:17: error: 'tmp' was not declared in this scope
         PROTECT(tmp = chmatch(cols, names, 0, FALSE));
                 ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:661:31: note: in definition of macro 'PROTECT'
 #define PROTECT(s) Rf_protect(s)
                               ^
assign.c:598:54: warning: deprecated conversion from string constant to 'char*' [-Wwrite-strings]
                             if (isReal(thisvalue)) s3="; may have truncated precision"; else s3="";
                                                      ^
assign.c:598:96: warning: deprecated conversion from string constant to 'char*' [-Wwrite-strings]
                             if (isReal(thisvalue)) s3="; may have truncated precision"; else s3="";
                                                                                                ^
assign.c:607:5: error: 'key' was not declared in this scope
     key = getAttrib(dt,install("sorted"));
     ^
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c:611:17: error: 'tmp' was not declared in this scope
         PROTECT(tmp = allocVector(STRSXP, LENGTH(cols)));
                 ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:661:31: note: in definition of macro 'PROTECT'
 #define PROTECT(s) Rf_protect(s)
                               ^
assign.c:625:11: error: assignment of function 'char* index(const char*, int)'
     index = getAttrib(dt,install("index"));
           ^
assign.c:625:11: error: cannot convert 'SEXP {aka SEXPREC*}' to 'char*(const char*, int)' in assignment
assign.c:626:18: error: comparison between distinct pointer types 'char* (*)(const char*, int)' and 'SEXP {aka SEXPREC*}' lacks a cast [-fpermissive]
     if (index != R_NilValue) {
                  ^
In file included from data.table.h:3:0,
                 from assign.c:1:
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:300:25: error: request for member 'attrib' in 'index', which is of non-class type 'char*(const char*, int)'
 #define ATTRIB(x) ((x)->attrib)
                         ^
assign.c:627:13: note: in expansion of macro 'ATTRIB'
         s = ATTRIB(index);
             ^
assign.c:629:13: error: 'a' was not declared in this scope
             a = TAG(s);
             ^
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c:666:17: error: 'colorder' was not declared in this scope
         PROTECT(colorder = duplicate(cols));
                 ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:661:31: note: in definition of macro 'PROTECT'
 #define PROTECT(s) Rf_protect(s)
                               ^
assign.c: At global scope:
assign.c:876:42: error: expected ',' or '...' before 'new'
 SEXP setcharvec(SEXP x, SEXP which, SEXP new)
                                          ^
In file included from data.table.h:3:0,
                 from assign.c:1:
assign.c: In function 'SEXPREC* setcharvec(SEXP, SEXP, SEXP)':
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:303:22: error: expected type-specifier before ')' token
 #define TYPEOF(x) ((x)->sxpinfo.type)
                      ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:1374:22: note: in expansion of macro 'TYPEOF'
 #define isString(s) (TYPEOF(s) == STRSXP)
                      ^
assign.c:881:10: note: in expansion of macro 'isString'
     if (!isString(new)) error("'new' must be a character vector");
          ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:338:44: error: expected type-specifier before ')' token
 # define SHORT_VEC_LENGTH(x) (((VECSEXP) (x))->vecsxp.length)
                                            ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:337:26: note: in expansion of macro 'SHORT_VEC_LENGTH'
 # define IS_LONG_VEC(x) (SHORT_VEC_LENGTH(x) == R_LONG_VEC_TOKEN)
                          ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:344:21: note: in expansion of macro 'IS_LONG_VEC'
 # define LENGTH(x) (IS_LONG_VEC(x) ? R_BadLongVector(x, __FILE__, __LINE__) : SHORT_VEC_LENGTH(x))
                     ^
assign.c:882:9: note: in expansion of macro 'LENGTH'
     if (LENGTH(new)!=LENGTH(which)) error("'new' is length %d. Should be the same as length of 'which' (%d)",LENGTH(new),LENGTH(which));
         ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:344:55: error: expected type-specifier before ',' token
 # define LENGTH(x) (IS_LONG_VEC(x) ? R_BadLongVector(x, __FILE__, __LINE__) : SHORT_VEC_LENGTH(x))
                                                       ^
assign.c:882:9: note: in expansion of macro 'LENGTH'
     if (LENGTH(new)!=LENGTH(which)) error("'new' is length %d. Should be the same as length of 'which' (%d)",LENGTH(new),LENGTH(which));
         ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:338:44: error: expected type-specifier before ')' token
 # define SHORT_VEC_LENGTH(x) (((VECSEXP) (x))->vecsxp.length)
                                            ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:344:79: note: in expansion of macro 'SHORT_VEC_LENGTH'
 # define LENGTH(x) (IS_LONG_VEC(x) ? R_BadLongVector(x, __FILE__, __LINE__) : SHORT_VEC_LENGTH(x))
                                                                               ^
assign.c:882:9: note: in expansion of macro 'LENGTH'
     if (LENGTH(new)!=LENGTH(which)) error("'new' is length %d. Should be the same as length of 'which' (%d)",LENGTH(new),LENGTH(which));
         ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:338:44: error: expected type-specifier before ')' token
 # define SHORT_VEC_LENGTH(x) (((VECSEXP) (x))->vecsxp.length)
                                            ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:337:26: note: in expansion of macro 'SHORT_VEC_LENGTH'
 # define IS_LONG_VEC(x) (SHORT_VEC_LENGTH(x) == R_LONG_VEC_TOKEN)
                          ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:344:21: note: in expansion of macro 'IS_LONG_VEC'
 # define LENGTH(x) (IS_LONG_VEC(x) ? R_BadLongVector(x, __FILE__, __LINE__) : SHORT_VEC_LENGTH(x))
                     ^
assign.c:882:110: note: in expansion of macro 'LENGTH'
     if (LENGTH(new)!=LENGTH(which)) error("'new' is length %d. Should be the same as length of 'which' (%d)",LENGTH(new),LENGTH(which));
                                                                                                              ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:344:55: error: expected type-specifier before ',' token
 # define LENGTH(x) (IS_LONG_VEC(x) ? R_BadLongVector(x, __FILE__, __LINE__) : SHORT_VEC_LENGTH(x))
                                                       ^
assign.c:882:110: note: in expansion of macro 'LENGTH'
     if (LENGTH(new)!=LENGTH(which)) error("'new' is length %d. Should be the same as length of 'which' (%d)",LENGTH(new),LENGTH(which));
                                                                                                              ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:338:44: error: expected type-specifier before ')' token
 # define SHORT_VEC_LENGTH(x) (((VECSEXP) (x))->vecsxp.length)
                                            ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:344:79: note: in expansion of macro 'SHORT_VEC_LENGTH'
 # define LENGTH(x) (IS_LONG_VEC(x) ? R_BadLongVector(x, __FILE__, __LINE__) : SHORT_VEC_LENGTH(x))
                                                                               ^
assign.c:882:110: note: in expansion of macro 'LENGTH'
     if (LENGTH(new)!=LENGTH(which)) error("'new' is length %d. Should be the same as length of 'which' (%d)",LENGTH(new),LENGTH(which));
                                                                                                              ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:382:42: error: expected type-specifier before ')' token
 #define DATAPTR(x) (((SEXPREC_ALIGN *) (x)) + 1)
                                          ^
/Library/Frameworks/R.framework/Resources/include/Rinternals.h:389:35: note: in expansion of macro 'DATAPTR'
 #define STRING_ELT(x,i) ((SEXP *) DATAPTR(x))[i]
                                   ^
assign.c:886:32: note: in expansion of macro 'STRING_ELT'
         SET_STRING_ELT(x, w-1, STRING_ELT(new, i));
                                ^
make: *** [assign.o] Error 1
ERROR: compilation failed for package ‘data.table’
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table’

[arunsrinivasan]

This has come up before in #1652. What's your OS X version? Assuming you've updated homebrew before installing clang-omp, do you have xcode command line tools installed as well?

[MichaelChirico]

If this is going to keep resurfacing, I guess we should include something about it in the start-up banner?

[bhive01]

Should be good on both of those things. Did brew update before clang-omp tap. Command line tools installed.

Brandons-iMac:~ brandonhurr$ xcode-select -version
xcode-select version 2343.


Brandons-iMac:~ brandonhurr$ xcode-select -p
/Applications/Xcode.app/Contents/Developer
Brandons-iMac:~ brandonhurr$ brew update
==> Migrating Homebrew to v0.9.9
warning: no common commits
remote: Counting objects: 469, done.
remote: Compressing objects: 100% (426/426), done.
remote: Total 469 (delta 27), reused 262 (delta 17), pack-reused 0
Receiving objects: 100% (469/469), 759.83 KiB | 254.00 KiB/s, done.
Resolving deltas: 100% (27/27), done.
From https://github.com/Homebrew/brew
 + 21ce7a5...d887dd3 master     -> origin/master  (forced update)
HEAD is now at d887dd3 brew install: make -s apply only to given formula, not deps (#205)
==> Homebrew has enabled anonymous aggregate user behaviour analytics
Read the analytics documentation (and how to opt-out) here:
  https://git.io/brew-analytics
==> Tapping homebrew/core
Cloning into '/usr/local/Library/Taps/homebrew/homebrew-core'...
remote: Counting objects: 3697, done.
remote: Compressing objects: 100% (3585/3585), done.
remote: Total 3697 (delta 14), reused 641 (delta 1), pack-reused 0
Receiving objects: 100% (3697/3697), 2.83 MiB | 407.00 KiB/s, done.
Resolving deltas: 100% (14/14), done.
Checking connectivity... done.
Tapped 3573 formulae (3,723 files, 8.8M)
Already up-to-date.


Brandons-iMac:~ brandonhurr$ brew list
brew-cask       gdal            graphicsmagick      libiomp         opencv3         readline
cairo           gdbm            harfbuzz        liblwgeom       openexr         sdl2
clang-omp       geos            icu4c           libmpc          openssl         sqlite
cmake           gettext         ilmbase         libpng          optipng         tcl-tk
eigen           ghostscript     imagemagick     libspatialite       pango           v8-315
ffmpeg          giflib          isl         libtiff         pdflib-lite     webp
fftw            gist            isl014          libtool         pixman          wget
fontconfig      git         jbig2dec        libvo-aacenc        pkg-config      wxmac
freetype        glew            jpeg            libxml2         proj            x264
freexl          glib            json-c          little-cms2     protobuf        xvid
gcc         gmp         lame            lzlib           pymol           xz
gcc5            gnuplot         libffi          mpfr            python          zeromq
gd          gobject-introspection   libgeotiff      numpy           qt

[bhive01]

OK, so I read the other issue and realized I didn't get the Makevar quite right for gcc5. With the Makevar below it compiles but then can't load. I installed gcc 5 from brew via these instructions.

cat ~/.R/Makevars
CC=gcc-5 -fopenmp
CXX=g++-5 -fopenmp

I had g++-5 in both places. now it's gcc-5 and g++-5.

install.packages("data.table", type = "source",
     repos = "https://Rdatatable.github.io/data.table")
trying URL 'https://Rdatatable.github.io/data.table/src/contrib/data.table_1.9.7.tar.gz'
Content type 'application/octet-stream' length 2721566 bytes (2.6 MB)
==================================================
downloaded 2.6 MB

* installing *source* package ‘data.table’ ...
** libs
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c assign.c -o assign.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c bmerge.c -o bmerge.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c chmatch.c -o chmatch.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c dogroups.c -o dogroups.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fastmean.c -o fastmean.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fcast.c -o fcast.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fmelt.c -o fmelt.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c forder.c -o forder.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c frank.c -o frank.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fread.c -o fread.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fwrite.c -o fwrite.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c gsumm.c -o gsumm.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c ijoin.c -o ijoin.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c init.c -o init.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c openmp-utils.c -o openmp-utils.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c quickselect.c -o quickselect.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c rbindlist.c -o rbindlist.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c reorder.c -o reorder.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c shift.c -o shift.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c subset.c -o subset.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c transpose.c -o transpose.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c uniqlist.c -o uniqlist.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c vecseq.c -o vecseq.o
gcc-5 -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c wrappers.c -o wrappers.o
gcc-5 -fopenmp -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o data.table.so assign.o bmerge.o chmatch.o dogroups.o fastmean.o fcast.o fmelt.o forder.o frank.o fread.o fwrite.o gsumm.o ijoin.o init.o openmp-utils.o quickselect.o rbindlist.o reorder.o shift.o subset.o transpose.o uniqlist.o vecseq.o wrappers.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
mv data.table.so datatable.so
if [ `uname -s` = 'Darwin' ]; then install_name_tool -id datatable.so datatable.so; fi
installing to /Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table/libs/datatable.so':
  dlopen(/Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table/libs/datatable.so, 6): Symbol not found: _GOMP_parallel
  Referenced from: /Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table/libs/datatable.so
  Expected in: flat namespace
 in /Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table/libs/datatable.so
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table’

The downloaded source packages are in
    ‘/private/var/folders/zg/hy0k4g312536_4l1qc1w27640000gn/T/RtmpXM09oF/downloaded_packages’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/data.table’
Warning message:
In install.packages("data.table", type = "source", repos = "https://Rdatatable.github.io/data.table") :
  installation of package ‘data.table’ had non-zero exit status

[arunsrinivasan]

I'm not sure how you figure if command line tools is installed by that statement. Looking at this issue, it seems you need to run xcode-select --install from the terminal. I don't know your OS X version. So it's hard to search for OS specific ways to get Xcode up and running correctly.

[bhive01]

Well, in Xcode it shows that I have them, which should be the case. I build nearly every package from source and you can't do that without them. When I tell it to install it tries to get them, which will take a while because of my quite slow connection. I will try a reinstallation of the tools and then data.table with gcc-5 and clang-omp again.

The latest error suggests that gcc-5 is able to build it, but something is not right with _GOMP_parallel.

[arunsrinivasan]

In that case, I don't know yet what's the issue as well. Once again, what's your OS X version?

[bhive01]

Well, good news is that reinstallation of the tools worked. I don't understand why because I was already compiling things using them. But, when I paste the following from here I got:
No receipt for 'com.apple.pkg.CLTools_Executables' found at '/'.

After reinstall:

pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 7.3.1.0.1.1461711523
volume: /
location: /
install-time: 1462568157
groups: com.apple.FindSystemFiles.pkg-group com.apple.DevToolsBoth.pkg-group com.apple.DevToolsNonRelocatableShared.pkg-group 

data.table installation with those flags and clang-omp works fine. It does not work with gcc-5 still, but that's irrelevant.

I'm sorry that's my fault. I did not follow instructions. I skipped the first line because I thought I had the tools already.

OSX version 10.11.4

[arunsrinivasan]

Great! IIUC xcode-select --install should be run after installing clang-omp seems to be the issue?

[bhive01]

Nope, exactly how you have them. Sorry again for not running everything as it was written.

[arunsrinivasan]

Thanks. Updated https://github.com/Rdatatable/data.table/wiki/Installation with a note that this step is essential.

[bhive01]

Yes, if you do have it installed, it will tell you so there is no harm in running it anyway:

Brandons-iMac:~ brandonhurr$ xcode-select --install 
xcode-select: error: command line tools are already installed, use "Software Update" to install updates

[arunsrinivasan]

Thanks. Clarified it further.

[botanize]

The brew clang-omp formula has been moved to the bone-yard (deleted). It looks like llvm is the replacement formula (Homebrew/homebrew-core#3266). Anyone have Mac build tips for llvm?