Submissions/Autonomous Detection of Collaborative Link Spam
|
Presentation Media 30px No slides known
|
View the slides from this presentation (PDF)
- Review no.
114
- Title of the submission
- Autonomous Detection of Collaborative Link Spam
- Type of submission (workshop, tutorial, panel, presentation)
- Presentation
- Author of the submission
- Andrew G. West (west.andrew.g on en.wiki)
- E-mail address or username (if username, please confirm email address in Special:Preferences)
- westand@cis.upenn.edu
- Country of origin
- United States of America
- Affiliation, if any (organization, company etc.)
- University of Pennsylvania -- Philadelphia, PA
- Personal homepage or blog
- http://www.cis.upenn.edu/~westand/
- Abstract (please use no less than 300 words to describe your proposal)
(click to view this submission in PDF format)
Collaborative environments, such as Wikipedia, are no stranger to inappropriate contributions -- collectively known as "vandalism." Most often vandalism manifests itself as immature, offensive, or experimental insertions. As a result, virtually all existing anti-vandalism systems lack explicit functionality to detect inappropriate hyperlinks to destinations outside the host environment (i.e., external link spam) [4].
However, link spam is perhaps the most dangerous subset of vandalism: (1) Spam contributors are likely to be well-motivated (possibly financially) making them intelligent and evasive attackers. (2) Spam behavior is non-obvious in "diffs", the standard means of edit review. (3) Recent research suggests viable attack vectors (e.g., labor shortages [1] and automated spamming software [2]). While Wikipedia does employ anti-spam functionality (e.g., blacklists, account blocks), these require multiple instances of abuse to be triggered and are thus *reactive* and inherently latent in nature.
Thus, we were motivated to create a *proactive* autonomous link spam detector. Our efforts are described extensively in [5], and the proposed presentation will begin by summarizing that work. Briefly, a spam corpus was assembled from over 250,000 (English) Wikipedia edits. Using this, 40+ indicative features were identified and combined into a machine-learning classifier. Features draw from three main areas: (1) Wikipedia metadata, (2) landing site analysis (processing the HTML link destination), and (3) third-party data (Alexa web services, Google Safe-Browsing lists). Offline evaluation showed great potential, detecting 65% of spam at a 0.5% false-positive rate.
Novel to the proposed presentation, we will next describe and demonstrate the *live implementation* of this technique (in progress; will be completed prior to Wikimania 2011). Every edit to Wikipedia is scanned for new external links and assigned a corresponding "spam score." The most egregious edits can be undone automatically in a bot-like fashion, per a community-set false-positive tolerance. Where confidence is insufficient for automatic reversion scores can be used to interface with the STiki GUI tool [4], allowing human inspectors to review likely spam edits in a crowd-sourced fashion. Regardless, a spam edit can be easily reverted and the offending user warned, with persistent abuse resulting in blacklisting/blocks.
Manually inspected edits will require humans to visit the landing site to make a spam/ham distinction, potentially exposing them to dangerous/unsavory content. To this end, alerts have been installed to warn patrollers of adult content, potential malware destinations, etc.. Finally, the definitive human classifications received in this fashion (via the GUI) can be used to refine future scoring techniques.
- Track (People and Community/Knowledge and Collaboration/Infrastructure)
- Wiki Infrastructure and Technology
- Will you attend Wikimania if your submission is not accepted?
- Yes. Thank you to the committee for a partial scholarship.
- Slides or further information (optional)
[1] E. Goldman. Wikipedia’s labor squeeze and its consequences. Journal of Telecommunications and High Technology Law, Volume 8, 2009.
[2] Y. Shin, M. Gupta, and S. Myers. The nuts and bolts of a forum spam automator. In LEET’11: Proc. of the 4th Workshop on Large-Scale Exploits and Emergent Threats, 2011.
[3] A.G. West. STiki: A vandalism detection tool for Wikipedia. http://en.wikipedia.org/wiki/Wikipedia:STiki
[4] A.G. West, J. Chang, K. Venkatasubramanian, and I. Lee. Trust in Collaborative Web Applications. To appear in Future Generation Computer Systems, special section on Trusting Software Behavior, Elsevier Press, 2011. (A preliminary version was published as University of Pennsylvania Technical Report MS-CIS-10-33, 2010).
[5] A.G. West, A. Agarwal, P. Baker, B. Exline, and I. Lee. Autonomous Link Spam Detection in Purely Collaborative Environments. To appear at WikiSym 2011. http://www.cis.upenn.edu/~westand/docs/wikisym_11_spam_abstract.txt
Interested attendees
If you are interested in attending this session, please sign with your username below. This will help reviewers to decide which sessions are of high interest. Sign with four tildes. (~~~~).
- West.andrew.g 03:41, 23 April 2011 (UTC) -- Interested, by default!
- Vibhijain 15:39, 5 May 2011 (UTC)