Observing the BitTorrent Universe Through Telescopes

Transcription

1 Observing the BitTorrent Universe Through Telescopes Rodrigo B. Mansilha, Leonardo R. Bays, Matheus B. Lehmann, Alan Mezzomo, Giovani Facchini, Luciano P. Gaspary, Marinho P. Barcellos IM 2011 May 26, 2011 Dublin, Ireland Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 1/21

2 Outline 1 Introduction 2 BitTorrent 3 TorrentU 4 Evaluation 5 Conclusion Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 2/21

3 Context BitTorrent is the most popular file sharing protocol Despite several isolated studies about the BitTorrent Universe", there exists no methodology to systematically observe it and obtain results Results would be useful for several reasons Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 3/21

4 Relevancy of results Marketing campaigns based on content popularity Estimate financial losses due to piracy Help improve the efficiency of sharing and reduce the costs of BitTorrent users to ISPs Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 4/21

5 Challenges The dimension, complexity, heterogeneity of BitTorrent networks provide limited opportunities of observation In contrast to others P2P networks, BitTorrent is a global network comprised of millions of smaller networks (swarms) There are several distinct implementations and protocol extensions and all of them need to be taken into account Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 5/21

6 TorrentU Monitoring system: systematically observe parts of the BitTorrent universe Flexible: different observation strategies can be combined according to the desired information Scalable: strategies can be employed by a set of observation agents Extensible: new strategies can be easily added Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 6/21

7 Universe Elements Peer: user agent that participates in swarm(s) Swarm: set of peers that share a given content.torrent: metadata necessary for the peers to enter the swarm Tracker: meeting point of peers Community: website dedicated to promote file sharing by publishing torrents BitTorrent Universe content 4 community 1 community 2 swarm 4 swarm 5 content 1 swarm 1 content 2 swarm 2 content 3 swarm 3 Legend torrent tracker peer Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 7/21

8 Strategies 1 Community consulting the websites downloading and parsing of webpages and torrents. 2 Tracker performing announce to obtain a list of peers requesting scrape to obtain the index of tracked swarms 3 Peer connecting with peers to identify the user agent, extensions and possession of pieces close the connection immediately or to keep it open and exchange messages to monitor the distribution of pieces Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 8/21

9 Architecture - Overview Typical architecture of network monitoring systems Observer manages a distributed set of Telescopes Telescopes allow different image resolutions, from a constellation view to a close look at one star only Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 9/21

10 Architecture - Observer Observer allows configuring system and presenting real-time, trending and historical data User Interface: manages telescopes, monitoring tasks, and presents data Observer Repository: stores information obtained from Telescopes Requester: communicates with the Telescopes Strategist: chooses the set of strategies and their parameters considering three aspects about results: richness, coverage and accuracy Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 10/21

11 Architecture - Telescope Telescope is responsible for looking at some specific part of the BitTorrent universe Provider: communicates with Observer Telescope Repository: stores information obtained from the Universe Monitor: contacts the elements of the Universe Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 11/21

12 Monitor - Community Lens Examples of parameters: Richness: information about communities and their peers Coverage: communities and content types Accuracy: frequency in which torrents are obtained Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 12/21

13 Monitor - Tracker Lens Examples of parameters: Richness: information on trackers, addresses of peers Coverage: tracker locallity, swarm size Accuracy: frequency of announces and scrapes Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 13/21

14 Monitor - Peer Lens Examples of parameters: Richness: information about peers and their pieces Coverage: peer locality Accuracy: duration of connections with peers Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 14/21

15 Evaluation - Overview Case study to show the feasibility of the proposed solution in terms of flexibility, accuracy and scalability Up to 50 PlanetLab nodes Three monitoring campaigns: 1 Followed the publication of torrents 2 Increased the richness of detail for a specific content 3 Captured a constellation view of the Universe Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 15/21

16 Evaluation - Case Study 1/3 Identify the differences in torrent publication on observed communities Observed the available data on recently published torrents, every five minutes, during 140 hours Used 1 telescope Results: 15,071 torrents, of which 11,024 were unique 363,251 files 5,98 TB Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 16/21

17 Evaluation - Case Study 2/3 Illustrate how to achieve a greater level of detail for a certain content Observed all torrents, including those already published, related to 11 episodes of a TV series to obtain information on peers which were sharing that content Using 11 telescopes, one for each episode 10 6 Quantity of Peers Communities Trackers 10 3 IP:port Monitoring Time (hours) Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 17/21

18 Evaluation - Case Study 3/3 Rapidly capture a snapshot of the swarms in the universe Divided the 750 trackers in 50 equal groups Each of them to be observed by one telescope Within 30 minutes Results: 4 million swarms 1.1 million unique IP addresses Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 18/21

19 Conclusion Defined the concept of BitTorrent Universe and its elements Listed the existing strategies to extract information from the Universe Developed TorrentU, a flexible architecture to observe this universe Evaluation show how scalability of TorrentU can be increased by: combining different strategies using a coordinated set of telescopes Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 19/21

20 Ongoing work Developing an algorithm to optimize the use of observation strategies Deploying a network of telescopes in Brazil to achieve long-term monitoring Studying the possibility of extending the architecture in order to share telescopes with multiple observers Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 20/21

21 End Acknowledgements RNP Academic Network (GT-UniT WG) CNPq Research Agency (Project P2P- SeC) Contacts GT-UniT: gtunit Rodrigo B. Mansilha Observing BitTorrent Universe Through Telescopes 21/21