NetApp Acquires DataPelago for Storage-Layer AI Processing

According to NetApp, Nucleus can reduce infrastructure costs by up to 80% and deliver performance up to 10 times faster than conventional architectures. Those are company-reported, workload-dependent performance claims, rather than independently benchmarked results.

DataPelago's technology is intended to work between storage systems and data-processing frameworks. Blocks & Files reports that Nucleus uses the open-source projects Gluten, Velox, and Substrait, and integrates with engines and frameworks including Apache Spark, Trino, Flink, Ray, and Dask. The publication also reports integrations with SQL, Python, Airflow, Tableau, and Power BI.

CRN reports that DataPelago Accelerator for Spark can be deployed without rewriting applications or migrating data. That deployment model is relevant to teams with existing Spark pipelines whose data preparation stages are constrained by transfers between storage and GPU-backed compute environments.

TechTarget characterized the acquisition as an expansion of NetApp's support for distributed data management, an area enterprises confront when preparing governed data for AI applications. The publication noted that NetApp had previously released data-management tools for collecting, curating, synchronizing, protecting, and preparing data for AI use, as well as an AFX all-flash array with disaggregated storage capabilities.

Rob Strechay, founder and principal at Smuget Consulting, told TechTarget that the technology could become a connective layer between file and object storage, enterprise data, and compute engines such as Spark. "By bringing GPU- and CPU-accelerated processing... directly to the storage layer, organizations should be able to execute analytics and AI data preparation without constantly copying data into separate compute clusters," Strechay said.

For data engineering and ML platform teams, the technical premise is familiar: movement, serialization, format conversion, and repeated copies can consume time and infrastructure budget before GPUs execute a training, feature-engineering, or inference-adjacent workload. Companies pursuing comparable storage-adjacent compute designs commonly seek to reduce those pipeline stages while maintaining governance controls around operational data.

The practical questions remain integration depth, hardware support, workload coverage, and whether reported gains hold across customer environments. NetApp declined to provide CRN with details beyond its press release and indicated that additional information would be forthcoming. StorageReview reports that DataPelago is operating as a wholly owned NetApp subsidiary following the transaction.