The data landscape experienced significant changes in 2023, presenting new opportunities (and potential challenges) for data engineering teams. I believe we will see the following this year in the ...

Dbt isn’t the only vendor to find that data quality is getting worse. Data observability vendor Monte Carlo published a report a year ago that came to a similar conclusion. The vendor’s State of Data ...

In fact, storing data in Hadoop using those raw formats is terribly inefficient. Plus, those file formats cannot be stored in a parallel manner. Since you’re using Hadoop in the first place, it’s ...

Access to more information does not necessarily lead to better decision making, according to a new study from Oracle. Though 83% of those surveyed agree that access to more data should make decisions ...

The data we have now is huge. But size, it turns out, is a relative thing. And according to the IDC, the sum of the world’s data – the DataSphere — will grow from 33 zettabytes in 2018 to a ...

First, we’ll define and demystify these terms. Second, I’ll share some key business use cases that cannot be solved with traditional relational data catalogs. Finally, I’ll wrap it up by getting ...

Data scientists spend about 45% of their time on data preparation tasks, including loading and cleaning data, according to a survey of data scientists conducted by Anaconda. The company also analyzed ...

In this age of information, to say that the volume of data is exploding is a stark understatement. This big bang of big data is estimated to grow from 33 zettabytes in 2018 to 175 zettabytes by 2025, ...

Should you strive to centralize your data, or leave it scattered about? It seems like it should be a simple question, but it’s actually a tough one to answer, particularly because it has so many ...

Anaconda created a stir over a year ago when it began charging large commercial users a fee for access to its popular collection of Python tools. The change, it said, was necessary to offset the costs ...

During a presentation at Nvidia’s GPU Technology Conference (GTC) this week, the director of data science for Walmart Labs shared how the company’s new GPU-based demand forecasting model achieved a ...

Never before have the challenges of big data–how we store it, manage it, govern it, and use it–been so pressing. Advances in artificial intelligence may be the driving force in 2024, but that doesn’t ...