Spam-filter for websites

Since I felt a bit ill tonight, I stayed at home and fiddled around with naive Bayes classifiers for spam filters (wiki). I had the idea that it should be possible to build a greasemonkey/tampermonkey script which removes (or greys out) all the unwanted and attention seeking stuff in the internet besides ads. Things like “related”-stuff, “you also like”, news which are not really news and all that unrelated content, maybe it could be expanded to trolls in forums. I thought about a filter based on the text and html-classes which classifies the content as spam or not.

As a first try I built a filter for a community where some users are known to spam with sexist or rude content. This is the formula of the classifier (it’s actually this one from the wiki article):

[

](https://fabiodelia.files.wordpress.com/2015/04/screen-shot-2015-04-17-at-23-56-39.png) (As a picture, since I found it to be more clearly arranged than pasted code.

wordsSPAM[word] is the occurrence of the word in the spam-bucket; wordsNOT is the non-spam-bucket.)

Here are some screenshots of the result: All you can see is the rating of the posts (where the time is displayed), higher % means it’s rather spam, and if a post is in the spam- or nonspam-bucket. After setting about 10 posts as spam/nonspam, the classifier already starts to do interesting stuff, however there are still false negatives and positives.

The handling is as known from e-mail: alt-click on a post sets it as nonspam, alt-shift-click as spam. The setting of a post overrides any classification. I’m looking forward to fill the buckets and see what’s going to happen. (The community is www.spocal.net, – I only made the opacity of the posts and the text in brackets, the rest is their work.)

[

](/wp-content/uploads/2015/04/screen-shot-2015-04-17-at-23-42-12.png)

However, there are some problems… I’ll write about them later. (Gary Dusbabek uses a slightly better approach here, avoiding one of the problems – it’s the same theory though.)