Constellation Network and Common Crawl Foundation Revolutionize Web Data Accessibility and AI Development with Blockchain Technology
SAN FRANCISCO, October 24, 2024 /PRNewswire/ — The Common Crawl Foundation, a non-profit organization founded in 2007, dedicated to providing a copy of the Internet to the public, and Constellation Network, a Web3 blockchain ecosystem known for providing solutions to the ministry today American Defense. announced a strategic partnership to democratize and improve the accessibility and utility of web-crawled data on blockchain technology for artificial intelligence (AI) and data applications.
This collaboration will explore potential opportunities to improve large language models used by AI, starting with Common Crawl’s vast dataset which is used by 80% of large language models, has crawled over 250 billion web pages to date (19 billion in 2024 alone) and consists of an archive of nearly 9 petabytes of crawled archived data. By leveraging Constellation decentralized network, Hypergraph, to add immutability, provenance and auditability around the data that the partnership aligns to provide joint solutions around responsible and transparent AI.
With AI expected to be a $3 trillion industry by 2030, there is growing demand for secure solutions to share common datasets used for training large language models, improve data storage queried and cleaned, data monetization opportunities and increased transparency. with the data source. With Constellation’s unique approach of providing tools to converge existing infrastructure with distributed infrastructures and decentralized networks, and Common Crawl’s data history and growth in data utility, this partnership aligns to further democratize data.
“This partnership represents a significant step forward in ensuring reliable distribution of Common Crawl,” said Rich Skrenta, executive director of the Common Crawl Foundation. “By combining our comprehensive web archives with Constellation’s proven implementation of blockchain technology, researchers and developers around the world can trust what they get from Common Crawl and have a model for authenticating large open data sets, such as those used for AI training. “.
Ben JorgensenCEO of Constellation Network, says: “The partnership between Constellation Network and Common Crawl highlights the widespread adoption of Web3 solutions outside of the echo chambers of crypto. This alignment continues Constellation’s mission to use our Zero Trust network as a public good for a data-driven future. » Jorgensen continues: “Our goal is to attract more new developers by introducing capabilities, such as integrating immutability into digital workflows, and thus further differentiate ourselves from previous generations of blockchain technology.
The two organizations will begin a phased approach to implementing this initiative, starting with a customizable subnet, called a metagraph, which will integrate a subset of Common Crawl data. This subnet is currently active on their test network and will soon be deployed on Constellation’s public network, Hypergraph. Further details on the live metagraph will be presented in the coming weeks, along with information on how organizations and developers can participate.
For more information, please visit:
About the Common Crawl Foundation
The Common Crawl Foundation is a 501(c)(3) nonprofit organization dedicated to providing a free copy of the Internet to the public. Their web archives consist of petabytes of data collected over years of web exploration, serving as an essential resource for researchers, businesses, and developers around the world.
About the Constellation Network
Constellation Network is a Web3 blockchain ecosystem that connects crypto savings with traditional businesses. Its flagship network, Hypergraph, offers a solution for fast, scalable and fee-free transactions. The Constellation network is validated by the US Department of Defense, which has been a customer since 2019.
Note: This press release contains forward-looking statements. Actual results may differ materially from those projected.
SOURCE Constellation Network Inc.
YOU WANT NEWS ABOUT YOUR COMPANY FEATURED ON PRNEWSWIRE.COM?
440,000+
Press rooms and
Influencers
9k+
Digital media
Points of sale
270,000+
Journalists
Registration
