Storage
different environments… on Google Cloud, working on Intel CPU.” Expanding compatibility, including TCP-based fabric integrations, remains a key goal, though Lombardi acknowledges resource challenges: “All these require hardware resources that are not necessarily easy.” DAOS was developed to “rewrite a distributed file system from a
blank page,” focusing on an object-layer or key-value store model. Leveraging persistent memory (PMDK) and SPDK, the architecture bypasses the kernel to enhance streamlined operations. Te transition to an independent solution under the PSS IO program in 2015 led to rapid development, culminating in a successful 2017 proof-of- concept. Lombardi says, “Legacy storage models forced unnecessary transaction overhead. With DAOS, we created our backend, independent of CFS or existing file systems.” Adaptability remains central to DAOS’s development. “Optane’s
cancellation forced us to rethink metadata separation, but we still maintained distinct metadata structures for optimisation,” Lombardi notes. DAOS employs optimistic concurrency control to enhance performance. “I take a risk that there will be no conflict, so I do the operation, tag it with a version, and if necessary, redo the operation later. 99.9% of the time, there is no conflict, so there is no contention.” DAOS integrates with high-level languages and frameworks,
ensuring flexibility across AI, HPC, and enterprise workloads. It natively supports Python, PyTorch, and HDF5, optimising data handling and serialisation. “You just create a namespace, mount it, and access it like a file system. But DAOS can do more, incorporating a key-value store designed for direct integration with high-level languages.” DAOS employs interception libraries for improved application performance, bypassing the kernel. “We inject DAOS into the application, bypassing the kernel, and send the I/O directly – no modification needed. Instead of requiring developers to rewrite their applications, we intercept system calls transparently, making DAOS fully compatible.” To optimise data querying, DAOS eliminates inefficient file system
scans. “Typically, when querying a file system, you must scan all data sets across all users. With DAOS, we provide a structured query language that allows targeted searches.” Instead of analysing billions of files, DAOS enables metadata queries: “You can run a query like ‘Give me all data sets accessed in the last 24 hours.’ We scan only the metadata, not the file contents, making searches dramatically faster.” Future developments focus on middleware expansion and
enhanced adaptability. “We’re working on making DAOS more adaptable across different applications while ensuring it maintains high-performance scalability.” With advancements in GPU integration, metadata optimisations, and distributed storage models, DAOS strengthens its position as a next-generation storage solution.
Future development and expanding adoption Te DAOS Foundation continues to shape the technical direction of DAOS, adapting to evolving demands in storage and computing. Lombardi notes: “Tere are different options, whether the focus is on AI or other commercial applications, it depends on the market organisations want to address.” To ensure sustainable development, the foundation emphasises clear documentation and structured oversight. “We are trying to simplify the value of adoption and organising everything with gatekeepers,” Lombardi says, reinforcing
www.pcr-online.biz
the importance of maintaining code consistency across contributions from different companies. Community engagement is a priority, with outreach initiatives
including DAOS user groups, global conferences, and collaborative efforts across industry partners. “All great parts of the community collaborate,” Lombardi explains, highlighting the role of collective participation in fostering growth and long-term sustainability. Te foundation also manages roadmap planning, ensuring technical flexibility while defining DAOS’s future. “Te foundation’s role is to create a community roadmap and define the technical direction of DAOS,” Lombardi says. Expanding DAOS’s presence remains a priority. “We are working
on recruiting new members and registering trademarks in the US, Europe, and Asia,” Lombardi states, reflecting efforts to strengthen both technical influence and commercial reach. Te foundation’s roadmap includes future releases such as DAOS 2.8, scheduled for Q4 2025, which will introduce optimised object placement and client telemetry, while DAOS 3.0 in Q2 2026 will add SSD hotplug support. Building a strong user base is central to DAOS’s strategy. “For the
time being, we are focused on building our community. Tere’s strong interest in different workloads, and we’re aligning our development accordingly,” Lombardi says. “Te foundation’s goal is to maintain a scalable architecture that can evolve with future developments.” Improvements in network infrastructure, metadata management, and storage adaptability further reinforce DAOS’s position in distributed storage. DAOS’s adoption is growing across research and enterprise sectors.
“Google has a managed service based on DAOS called Pilot Store that is GA today and accessible on GCP,” Lombardi notes. Meanwhile, Argonne National Laboratory hosts the largest DAOS deployment, with “an expansive petabyte in a single storage system that can deliver more than 25 terabytes.” Intel has played a role in DAOS’s development, particularly in its work with Argonne. Looking ahead, the DAOS Foundation aims to expand adoption
further and ensure sustainability. “Te foundation’s goal is to promote the use of DAOS and provide a roadmap for future development,” Lombardi says. Te roadmap for 2024 includes strengthening industry partnerships, expanding parallel storage deployments, and refining infrastructure testing. “Project maturity takes around ten years, and DAOS has now hit that mark,” Lombardi says. With advancements in disaggregated architectures and metadata
optimisation, DAOS remains positioned as a scalable, low-latency storage framework for next-generation computing.
May/June 2025 | 41
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44 |
Page 45 |
Page 46 |
Page 47 |
Page 48 |
Page 49 |
Page 50 |
Page 51 |
Page 52
