Improve the "tried to allocate too many buffers" error message

[martin-frbg]

for #3321 (leaving the original message intact despite dubious grammar, as it has accumulated lots of search engine hits over the years)

[brada4]

It reverts #3326

[martin-frbg]

Yes I need to rebase it when I'm back home tomorrow

[matthew-brett]

Sorry if I have misunderstood - but for the case where the number of CPUs is higher than the compiled number of threads, is it reasonable to use the compiled number of threads by default, rather than segfaulting?

[martin-frbg]

This is supposed to happen already, i.e. the number of threads (cores) found on startup is compared against the compile-time limit and should be capped at that value. Unfortunately there must be a logic flaw elsewhere in the code (and basically present since the days of GotoBLAS) but I have not managed to find it.

[ogrisel]

The problem happens when the threads are created externally to OpenBLAS but the user code:

• user code calls openblas_set_num_threads(1) to run OpenBLAS in sequential mode,
• user code creates a pool of threads of size num_threads == num_cpus > NUM_BUFFERS in its own program,
• user code call OpenBLAS routines in each of its threads triggering the crash.

@martin-frbg It there a some public API for the user program to introspect at run time the value of NUM_BUFFERS set at OpenBLAS built time?

[ogrisel]

BTW in the pattern I describe above, setting OPENBLAS_NUM_THREADS to a small value will not fix the problem. It's up to the user program it-self to avoid creating too many threads. But it can only do so it has a robust way to introspect NUM_BUFFERS.

Introspecting the value of openblas_get_num_threads is not enough because this value can have been changed by a previous call to openblas_set_num_threads.

[martin-frbg]

openblas_get_config() will return either SINGLE_THREADED or MAX_THREADS=<number> as part of the configuration string