The Dataminds Connect 2024 event, held in the picturesque city of Mechelen, Belgium, is a highly anticipated three-day gathering for IT professionals and Microsoft data platform enthusiasts. This year, the focus was on innovative technologies, including Microsoft Fabric, Power BI, and Azure cloud solutions. The event provided an invaluable opportunity for our consultants to gain insights from leading experts in the field and stay abreast of the latest advancements in data management. In this two-part series, we will be sharing the key takeaways and experiences of the event. This first part will cover feedback from the first two days of the seminar, highlighting some of the most impactful sessions and insights.
Further insights and experiences will be shared in the second part of this series, which will cover the feedback from the third and final day of the event. This day was particularly valuable, offering even more lessons and cutting-edge discussions.
Upgrading Entreprise Power BI Architecture – Steve Campbell & Mathias Thierbach
During this training, we gained insight into the internal workings of Power BI, enabling us to optimize our models more effectively. The instructor explained how Power BI compresses data using techniques like run-length encoding and dictionaries (hash encoding for text fields and value encoding for numeric fields). By understanding these mechanisms, we learned how to structure our models to maximize compression efficiency, especially by managing column cardinality. For instance, limiting high-cardinality columns, favoring integer formats, and disabling attribute hierarchies are key steps in optimizing dataset performance.
Build a Microsoft Fabric Proof of Concept in a Day – Cathrine Wilhelmsen, Emilie Rønning & Marthe Moengen
I recently had the opportunity to attend the “Build a Microsoft Fabric Proof of Concept in a Day” seminar, hosted by Cathrine, Emilie, and Marthe. It was an extremely engaging experience. The three presenters contributed a wealth of knowledge from their distinct professional backgrounds, which greatly enhanced the training. It was particularly beneficial to gain insights from individuals occupying pivotal roles within the Fabric ecosystem. This approach enabled us to engage in critical analysis of key aspects such as Fabric’s technical architecture, data modeling, and data architecture design.
Saying no is OK – Sander Star
While declining is not an easy task, it is a crucial skill in professional settings. It is not merely a necessity; it is, in fact, a skill. Knowing how to decline offers protection from conditions such as depression, overwork, and reduced productivity. It also means maintaining a healthy work-life balance, establishing clear limits and boundaries, and maintaining consistency in terms of quality, effectiveness, and efficiency. This highly practical training course is suitable for all audiences and provides participants with the opportunity to experience a variety of situations. It offers detailed explanations of these situations and provides guidance on how to implement them effectively.
Become a metadata driven DBA – Magnus Ahlkvist
Are you a DBA whose day-to-day work is full of repetitive tasks, monitoring and running scripts in different places? Then this course would have been made for you. His slogan : ‘Automation is about turning something boring and repetitive into something more fun’ and to achieve this, Mr Ahlkvist suggested to combine DBA Tool and DBA Checks with an overlay of Pode (which creates REST APIs in powershell).
SQL Server Infernals – A Beginner’s Guide to SQL Server Worst Practices – Gianluca Sartori
With no pre-requisites, this course provides a comprehensive overview of everything you need to avoid with databases. For young and old alike, it’s often a good idea to go back to basics, to remember the little things that have a big impact.
Fabric: adoption roadmap: Napoleon’s success story – Jo David
In a departure from the typical technological focus of our industry, Jo David invited us to immerse ourselves in the history of 18th and 19th century France through the story of Napoleon Bonaparte. Mr. David demonstrated that adopting a fabric in a company is a relatively straightforward process, comparable to the challenges of waging a war. Once the key elements of success have been identified, it becomes easier to prepare for change.
What’s wrong with the Medallion Architecture? – Simon Whiteley
Behind this big title, Simon Whiteley tackled a genuine issue that affects companies when layering their Lakehouses. The “medallion architecture” approach may not be the optimal solution for complex real-life data structures, and the distinction between layers may not be readily apparent to non-data collaborators. By presenting the broad stages of data curation in a step-by-step manner and emphasizing the importance of proper naming, Whiteley provided a more grounded approach to Lakehouse design that more closely aligns with the reality of data.
The Sixth Sense: Using Log Analytics to Efficiently Monitor Your Azure Environment – Abhinav Jayanty
In this presentation, Jayanty outlined the general steps for developing the monitoring component of an Azure environment. He began by presenting the process for monitoring activity logs of Azure objects, querying resources using KQL (Kusto Query Language), and determining pricing options based on data retention requirements. The latter part provides visual examples of KQL queries on Azure objects to extract metrics, log onto SQL tables, and implement message-based alerting. Given the extensive range of analytical tools available in Azure Monitor, it was not feasible for Jayanty to cover each one in detail. However, he provided a comprehensive overview of the monitoring tool and its integration within the Azure platform, which left attendees with a solid grasp of the subject matter.
EFSA implements a data mesh at scale with Databricks practical insights – Sebastiaan Leysen, Giancarlo Costa and Jan Van Meirvenne
In 2019, Zhamak Dehghani proposed the data mesh architecture, which suggests organizing domain-based teams (business and technical profiles) around a central data team with expertise in data ingestion. To more effectively accommodate the expansion in the number of sources, teams, and data, as well as the demand for greater business autonomy, bespoke data transformation and scheduling, the EFSA (European Food Safety Authority) teams have transitioned to a data mesh architecture for their data organisation. They outlined how data is shared with different teams using the new share functionality on Databricks, how teams are organized by domain, and the need for a data governance team that oversees security, access, and monitoring.
Effective Data Quality Checks and Monitoring with Databricks and Pandas – Esa Denaux
Quality is defined as meeting a predefined standard and prioritizing both correctness and transparency. In the session on data quality using Pandas and Databricks, I explored strategies to ensure high data quality throughout the data lifecycle, using De Lijn’s reference architecture and technology stack as an example. During the session with Esa, we discussed the use of visualization techniques like histograms, box plots, and scatter plots for detecting anomalies. We also considered summary statistics and data quality reports as tools for gaining deeper insight into data quality. This session has provided me with a comprehensive approach to data quality management, from the initial profiling and validation of data sets to the deployment of automated testing and monitoring systems. By focusing on both technical validation (through Pandas and Databricks) and strategic practices (like naming conventions and business rule enforcement), organizations can ensure that their data remains a valuable and reliable asset.
Exploring the art of Conditional Fromatting in Power BI – Anastasia Salari
I really appreciated the session given by Anastasia Salari. As an introduction, Anastasia explained the importance of conditional formatting through interactive examples that could easily be part of a business presentation to raise awareness on the use of appropriate visuals. She introduced effective techniques and uncovered the strategic value behind them, enhancing our understanding of both the ‘how’ and the ‘why.’ We learned how to use this simple yet powerful feature to streamline complex information and make reports not only visually appealing but also fundamentally more effective. Afterwards, a very interesting and detailed demo was given, showcasing a significant number of Power BI visuals featuring visual formatting. Anastasia demonstrated each time how she had implemented it, which gave us ideas for possible applications at the client’s site. The session provided immediate insights into how conditional formatting can improve how reports communicate data and elevate the overall impact of data visualization.
Optimizing Power BI Development: Unleashing the Potential of Developer Mode – Rui Romano
This session provided an insightful look into Developer Mode in Power BI, focusing on how it integrates developer-centric features such as source control and Azure DevOps. The presenter demonstrated how these tools enable better team collaboration and the creation of CI/CD pipelines, enhancing the scalability and reliability of Power BI projects. It was a very interesting presentation that highlighted powerful new features in Power BI, some of which are already partially available and will likely transform how we work with Power BI in the future.
In conclusion
This first part of our seminar feedback highlights just a glimpse of the rich knowledge and experiences we gained over this outstanding event. The insights shared were invaluable and have provided us with new perspectives on several key topics. Stay tuned for the second part, where we will continue to explore more of the seminars and share additional takeaways that will certainly fuel our future growth!
Authors: Alexe Deverdenne, Hugo Henris, Martin Izquierdo, Pierre-Yves Richer, Sophie Opsommer, Thibaut De Carvalho