Exploring the Use of BitTorrent as the Basis for a Large Trace Repository

Motivated by the need to deploy a public repository of multi-gigabyte trace files, we studied the BitTorrent protocol’s ability to disseminate very large files among peers. BitTorrent is a popular peer-to-peer protocol that allows parallel downloads of large files. In this paper, we analyzed user activity on BitTorrent over a four-month period with respect to supportable file sizes, file popularity, session lengths, transfer speeds, and the likelihood of service-interrupting flash crowds. Our results show that file sizes tend to be on the order of gigabytes, far larger than other peer-to-peer applications. File popularity has a distribution similar to other peer-to-peer file sharing systems. Unlike other systems, the majority of users require multiple sessions to retrieve a file, and they are willing to remain connected to the system for a very long time. Most users we observed appear to have asymmetric Internet connections, and their generally poor upload performance is mitigated by their willingness to remain connected to the system and upload for an amount of time far longer than they spent downloading. We found that service disruption due to flash crowds is unlikely, as the vast majority of users were able to begin contributing resources back to the system within seconds of connecting. Our results indicate that BitTorrent provides an effective foundation for dissemination of files that are multi-gigabyte or larger, provided more sophisticated features are added like versioning, availability, and content management.