Monday, December 10, 2018 - 4:30pm to 6:00pm
Location:8102 Gates Hillman Centers
Speaker:TIANYU LI , MATTHEW BUTROVICH and SIVAPRASAD SUDHIR, Masters Students /TIANYU%20LI%20%2C%20MATTHEW%20%20BUTROVICH%20and%20SIVAPRASAD%20SUDHIR
Project 1: Storage Engine
— Tianyu Li & Matt Butrovich
In this talk, we will discuss the work we've done on terrier's storage engine over the semester. We will cover the implementation of write-ahead logging and our proposed model for recovery, implementation of indexes, and our roadmap for the storage engine next semester. The immediate future direction for the storage work is to support Apache Arrow natively as our storage format to reduce ETL overhead to a data science pipeline, while relaxing some of the Arrow format's constraints for transactionally hot data to maintain high transaction throughput. We will briefly introduce Apache Arrow and present our proposed system architecture for achieving Arrow interoperability in the storage layer.