[Epic] DataFusion Blogs

[alamb]

@pauldix says the recipe for great products is:

1. Write great software ✅
2. Tell people about it 📖

I immodestly think we already do the great software part in DataFusion 😊

However, I think the project would benefit from more telling. This ticket organizes various potential blog posts that we might want to add to the DataFusion blog: https://datafusion.apache.org/blog/

In think in general the requirements for posts on the blog are:

1. Related to DataFusion (obviously)
2. Not commercial marketing for vendor products (I think it is fine to mention features vendors build using DataFusion, etc)
3. Reposts of articles that appeared on vendor blogs, assuming they meet the above criteral

Some example PRs to create blog posts

• DataFusion 45 blog post datafusion-site#57
• Blog: Using Ordering for Better Plans in Apache DataFusion datafusion-site#58

Please leave a comment (or start a PR!) if you have an idea for a blog.

• Blog post about processing larger than memory queries #17089
• [BLOG] Blog post about writing your own SQL dialect / extending SQL with DataFusion #16756
• Blog post about TopK filter pushdown #15513
• Blog post about using external indexes with Parquet #17010
• Blog / Example of how to compile DataFusion to WASM #13715
• [Blog] Async Scalar User Defined Functions #16525
• Blog post about user defined window functions #6781
• write blog post about Substrait donation #5201
• Blog for DataFusion 46.0.0 #15053
• Blog: Using Ordering for Better Plans in Apache DataFusion datafusion-site#58
• Blog about DataFusion correlated subquery support #16084
• Blog post about parquet vs custom file formats / test Optimized Parquet with ClickBench #16149
• Blog post about DataFusion Async / Stream execution model / cancellation #16396
• Blog Post for Accelerating Query Processing with Specialized Indexes #16372
• Blog post about DataFusion Async / Stream execution model / cancellation #16396

[alamb]

In general, I also think @andygrove 's pattern of creating a blog post for each comet release is amazing. For example:

• Comet 0.6.0 datafusion-site#56

I general I would love to help support anyone who wanted to create posts for DataFusion releases. The high level flow is:

1. Create a post following the same skeleton as previous posts (e.g. DataFusion 45 blog post datafusion-site#57)
2. Update sections when possible (contributors, events, etc)
3. Leave "TODO"s in the sections where more specific content is needed

Feel free to @ mention me on a Pull Request with that basic framework, I would be more than thrilled to help fill in the more technical content / solicit others to help.

[akurmustafa]

Hi @alamb, I had some blog posts in my personal website such as below:

• Query Optimization Introduction
• Filter Pushdown
• Ordering Analysis

These are like introduction blogs for some of the functionalities in Datafusion or databases in general. If you think their scope iis suitable for the datafusion blogs, I can move them to the datafusion blog page with appropriate formatting. If you think, their scope is not suitable for the website. Feel free to ignore.

[alamb]

Hi @alamb, I had some blog posts in my personal website such as below:

• Query Optimization Introduction
• Filter Pushdown
• Ordering Analysis

These are like introduction blogs for some of the functionalities in Datafusion or databases in general. If you think their scope iis suitable for the datafusion blogs, I can move them to the datafusion blog page with appropriate formatting. If you think, their scope is not suitable for the website. Feel free to ignore.

I think the content looks good to me -- if you are open to updates / edits, I think porting them to the Datafusion blog sounds like a good idea to me.

I also have a writeup on Query optimziation I am working on that I can share with you if you want to collaborate. Let me know and I can share it

[akurmustafa]

I think the content looks good to me -- if you are open to updates / edits, I think porting them to the Datafusion blog sounds like a good idea to me.

I also have a writeup on Query optimziation I am working on that I can share with you if you want to collaborate. Let me know and I can share it

I welcome updates, edits, suggestions. Also, I would like to collaborate on that writing also. Feel free to share with me.