What is Lakehouse? How to Create Lakehouses in Microsoft Fabric?
Download Dataset for practice
🙋♂️Hi there. I am Atikant Jain (AJ). Welcome to my newsletter, where I talk about career in Analytics & Data Science. Currently spreading love about Microsoft Power BI & Microsoft Fabric.
The foundation of Microsoft Fabric is a Lakehouse, which is built on top of the OneLake scalable storage layer and uses Apache Spark and SQL compute engines for big data processing. A Lakehouse is a unified platform that combines:
The flexible and scalable storage of a data lake
The ability to query and analyze data of a data warehouse
Imagine your company has been using a data warehouse to store structured data from its transactional systems, such as order history, inventory levels, and customer information. You have also collected unstructured data from social media, website logs, and third-party sources that are difficult to manage and analyze using the existing data warehouse infrastructure. Your company's new directive is to improve its decision-making capabilities by analyzing data in various formats across multiple sources, so the company chooses Microsoft Fabric.
Lakehouse = Data Lake + Data Warehouse
Create and explore a lakehouse
You create and configure a new Lakehouse in the Data Engineering workload. Each lakehouse produces three named items in the Fabric-enabled workspace:
Lakehouse is the lakehouse storage and metadata, where you interact with files, folders, and table data.
Semantic model (default) is an automatically created semantic model based on the tables in the lakehouse. Power BI reports can be built from the semantic model.
SQL analytics endpoint is a read-only SQL analytics endpoint through which you can connect and query data with Transact-SQL.
I have explained vividly on how to Create your First Lakehouse in Microsoft Fabric, also we have played around with PySpark and Spark SQL a bit to show how powerful spark is in Big Data landscape.
Links for the Dataset and the Video:
I hope this post was useful, let’s up-skill ourselves with one of the most important skills for Data folks!
Please write to admin@analyticalguy.tech if there’s anything you would like to share with us.