DirectLake in Power BI – Direct Access to Data Without Duplication
DirectLake is a new technology in the Microsoft Power BI ecosystem that is transforming the way we work with large-scale data. It enables direct reading of data from the OneLake store (the central data lake of the Microsoft Fabric platform) in the form of Delta tables, without the need to duplicate data into a data model. This approach combines the benefits of both Import and DirectQuery modes—it offers performance close to that of an imported model, yet with timeliness nearly as good as a direct query to the source..
What is DirectLake and how does it work
DirectLake is a new data model storage mode in Power BI. Model tables can be configured so that Power BI loads data directly from OneLake instead of using the traditional import method. Data remains stored only once in OneLake, and Power BI loads it into memory as needed when queries are run. Once the data is loaded, queries are processed in memory just as they would be with an imported model. DirectLake thus achieves performance comparable to import mode, but without the need to preload large volumes of data into the model.
DirectLake vs. DirectQuery – Key Differences
DirectQuery mode keeps the data in an external database and sends a query to the source system with every interaction. DirectLake ensures data freshness in a different way—it stores the data in OneLake and loads it into Power BI’s memory. Key differences:
-
Performance and Data: DirectLake executes queries in memory (providing fast response times even with large datasets) and, after a brief metadata synchronization, delivers near-real-time data. In contrast, DirectQuery places a load on the external database with every query, which slows down response times (even though it always reads data directly from the primary source).
-
Infrastructure load: With DirectQuery, the source database must handle all queries from Power BI, and it is often necessary to upgrade it at considerable cost. DirectLake shifts the computational load to Power BI’s cloud capacity, thereby reducing the burden on production systems—but it requires sufficient memory and CPU resources on the BI side to handle large volumes of data.
DirectLake vs. Traditional Data Import
In a traditional data import, the information is copied into Power BI and the entire data model is stored in memory. DirectLake differs from this in several ways:
-
Query and update speed: Imported models support very fast queries, but the data must first be fully loaded, and all data is reloaded during updates. DirectLake achieves similar performance without pre-loading—the first query on a large table may be slightly slower, but subsequent queries are just as fast—and new records are reflected in reports within seconds of being processed in the lake (no need to wait for a long refresh).
-
Model size: Imported models run into memory limits—all data must fit within Power BI’s capacity, and the model size is restricted. DirectLake removes this barrier, as the data remains in OneLake and only the necessary segments are loaded into memory. This allows you to work with terabytes of data without having to significantly increase memory capacity.
Business Benefits of DirectLake
Implementing DirectLake in Power BI offers the following benefits:
-
Lower data costs: Eliminating duplicate storage reduces both storage and maintenance requirements. You only need to maintain a single primary data source (OneLake) instead of operating an additional data warehouse for BI. The load on source systems is also reduced, as there is no longer a need for large data exports when refreshing reports.
-
Faster decision-making: Thanks to DirectLake, fresh data reaches users more quickly. Managers can work with up-to-date figures in near real time, which shortens response times and enables more agile, informed decisions.
-
Better user experience: Fast-loading reports and up-to-date figures boost user confidence in BI. Users aren’t frustrated by slow queries or outdated data, which encourages greater adoption of analytical tools across the company.
-
Scalability: DirectLake can handle massive volumes of data and hundreds of concurrent queries. The analytics platform can grow alongside the data repository—as data volumes increase, you simply need to scale up capacity without having to change the architecture or rewrite data models.
DirectLake in Microsoft Fabric and the Medallion Architecture
DirectLake is ideally suited for use in a Microsoft Fabric environment. In the medallion architecture (Bronze–Silver–Gold), the “gold” tables in OneLake represent the final data ready for analysis. Using DirectLake, Power BI can connect directly to this Gold layer in OneLake. This eliminates the need to move data to another storage location, and OneLake serves as the single source of truth for all analytical tools, including Power BI.
DirectLake bridges the worlds of data lakes and business intelligence. By eliminating unnecessary replication and accelerating access to information, it enables organizations to use their data more effectively.