Incorrect CMake module paths since aws-crt-cpp dependency was added

[glaubitz]

In bb96468, CMakeLists.txt was modified to use non-standard CMake modules paths which is incompatible with most standard distributions.

Trying to configure aws-sdk-cpp on openSUSE/SLE therefore currently fails with:

glaubitz@suse-laptop:~/upstream/aws-sdk-cpp/build> cmake -G Ninja ..
-- TARGET_ARCH not specified; inferring host OS to be platform compilation target
-- Building AWS libraries as shared objects
-- Generating linux build config
-- Building project version: 1.9.21
-- The CXX compiler identification is GNU 10.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11") 
--   Zlib library: /usr/lib64/libz.so
-- Encryption: Openssl
-- Found OpenSSL: /usr/lib64/libcrypto.so (found version "1.1.1k")  
--   Openssl include directory: /usr/include
--   Openssl library: /usr/lib64/libssl.so;/usr/lib64/libcrypto.so
-- Http client: Curl
-- Found CURL: /usr/lib64/libcurl.so (found version "7.76.1")  
--   Curl include directory: /usr/include
--   Curl library: /usr/lib64/libcurl.so
-- Performing Test HAVE_ATOMICS_WITHOUT_LIBATOMIC
-- Performing Test HAVE_ATOMICS_WITHOUT_LIBATOMIC - Success
CMake Error at CMakeLists.txt:174 (include):
  include could not find requested file:

    AwsFindPackage


CMake Error at CMakeLists.txt:184 (add_subdirectory):
  The source directory

    /home/glaubitz/upstream/aws-sdk-cpp/crt/aws-crt-cpp

  does not contain a CMakeLists.txt file.


-- Add s3-crt:s3 to C2J_SPECIAL_NAME_LIST
-- Looking for pathconf
-- Looking for pathconf - found
-- Looking for umask
-- Looking for umask - found
-- The C compiler identification is GNU 10.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done

Please see also: awslabs/aws-c-event-stream#15

[anuj812]

I am getting this issue as well for 1.9.19 as well. Where as it works perfectly for 1.8.186

[glaubitz]

Yep, it was introduced with version 1.9.0 which contains the commit bb96468.

[KaibaLopez]

Hi @glaubitz ,
Yea we got a few problems with adding the dependencies as submodules, still working on that.
Having said that I can make it work on my environment, so unless there is something specific going on in openSUSE (which I think shouldn't be the case) it could just be an environment configuration problem.
The most common cause of these errors is not having done the clone --recursive or if you already had version 1.8 git submodule update --init --recursive, remember to uninstall the previous crt version otherwise it will also fail.
If that doesn't work, can you walk me through every step you took to reproduce?

[glaubitz]

On May 20, 2021, at 7:24 PM, Gilberto Lopez Zayas ***@***.***> wrote: Hi @glaubitz , Yea we got a few problems with adding the dependencies as submodules, still working on that. Having said that I can make it work on my environment, so unless there is something specific going on in openSUSE (which I think shouldn't be the case) it could just be an environment configuration problem.

Well, I have been in touch with CMake upstream regarding the CMake issues with the various AWS C++ packages and they explicitly told me that the paths used and the way packages are included is wrong. I’ll try to loop in someone from CMake upstream in this discussion to help resolve this issue. FWIW, distributions don’t use git submodules but we rather split every dependency into a separate package to ease security maintenance.

[ncoder-1]

I'm having the same issue. I'm using the tar.gz release (and not a git clone).

For 1.9.25:

cd /tmp
curl https://codeload.github.com/aws/aws-sdk-cpp/tar.gz/refs/tags/1.9.25 -o aws-sdk-cpp-1.9.25.tar.gz
tar xvf aws-sdk-cpp-1.9.25.tar.gz
cd aws-sdk-cpp-1.9.25
mkdir build
cd build
cmake .. -G Ninja

Then I get the same error as above. This used to work with 1.8 releases. I'm also pulling specifc tar.gz releases in my own CMakeLists.txt files, so this is quite surprising that it stopped working.

[KaibaLopez]

@ncoder-1 ,
the tar.gz release is going to have the problem of not having the submodules included, for the moment this way of installing is not supported.
@glaubitz when you say:

distributions don’t use git submodules but we rather split every dependency into a separate package to ease security maintenance.

Does this mean that you just downloaded the project and built? (either via git or zip file), because then yea the same answer is going to apply to you and the only workaround for now is to keep using 1.8 since 1.9 requires the submodules to run.

[glaubitz]

Does this mean that you just downloaded the project and built?

Yes, downloaded from the tarball. Most distributions will use the tarball for creating a package, not check out the sources from git.

Distributions usually create individual packages for every upstream project which includes shared libraries. Having shared libraries in separate packages and not bundling them is an important security aspect for a distribution.

See: https://fedoraproject.org/wiki/Bundled_Libraries

[ncoder-1]

@KaibaLopez ,
So what is the use of the tar.gz releases if they are unsupported and not expected to work? For the reasons mentioned by @glaubitz , a lot of organizations require the use of tar.gz/zip releases on systems that build offline or for security reasons.

[KaibaLopez]

To clarify, I am not saying this is correct, I am trying to verify that was the cause and give a recommendation so this doesn't block you.
But yea, we are working to properly support this.