Another year passes. I was hoping to write more articles instead of just these end-of-the-year screeds, but I almost died in the spring semester, and it sucked up my time. Nevertheless, I will go ...

Community driven content discussing all aspects of software development from DevOps to design patterns. In helping students prepare for this exam, I have identified a number of commonly misunderstood ...

Community driven content discussing all aspects of software development from DevOps to design patterns. The AWS Certified Data Engineer Associate exam validates your ability to design, build, and ...

After Tempo compacts some data into a data.parquet file, I'd like to view this data with an external tool, but I have not been able to do so. I've tried parquet-tools, bdt, pandas (with both parquet ...

Abstract: In the domain of big data analytics, the bottleneck of converting storage-focused file formats to in-memory data structures has shifted from the bandwidth of storage to the performance of ...

When ingesting parquet files from S3, I receive the following exception: It seems that the parquet version needs to be upgraded (currently 1.8, should be at least 1.11). Here's the relevant JIRA My ...