Investigate hardware accelerated Spark

[razvan]

Intel AVX instruction set

Intel OEP - Optimized Analytics Package
https://github.com/oap-project
https://github.com/oap-project/gazelle_plugin/blob/master/docs/User-Guide.md

NVidia RAPIDS

https://nvidia.github.io/spark-rapids/

Cloud how-to

• Google Kubernetes Engine (GKE)
• Amazon Elastic Kubernetes Service (EKS)
• Azure Kubernetes Service (AKS)

Articles

• https://www.inovex.de/de/blog/getting-started-with-the-rapids-accelerator-for-apache-spark-on-azure-databricks/
• https://www.inovex.de/de/blog/data-processing-dask-rapids-kubernetes-cluster-gpu-support-13/