[Solved] Spam - straight to delete instead of quarantine?

[Hagar Delest]

I've checked the admin panel and the users in NRU gets automatically in RU (standard user) when they reach the post limit we set.
Moderation of the posts depends on the group and is forum sensitive.
What is not clear is if you set 1 as the limit, that post will be queued for moderation. I guess that as long as it has not been approved it won't allow the user to get the RU status by posting a mere second post. In this case, we would have to moderate only the first post of each new user.

NB: from the profiles reported by /a3, I've moved posts to the Deleted section because those users have been able to post messages with BBCode inside! There is something wrong. I checked the permissions for General Discussion where they were and settings are correct, BBCode disabled...

I haven't activated the NRU moderation yet but if there is another flood, will just do it.

[RoryOF]

I haven't read all of the 650 postings on the Forum during my absence, but I applaud the work of the mods and admins in dealing with the flood of spam that has occurred! [Thinks: didn't I choose the best week to go on holiday?!!] When I get Real Life sorted, I'll come back actively on online in Moderator mode.

To clarify a point: are we agreed that obvious spam should be deleted immediately?

[JohnSUN-Pensioner]

Thank you, Hagar and Acknak for all your efforts to clean out the spammers.

I haven't read all of the 650 postings on the Forum... I applaud the work of the mods and admins in dealing with the flood of spam that has occurred!

+100
Many-many thanks!

To clarify a point: are we agreed that obvious spam should be deleted immediately?

Together with its authors...

[Hagar Delest]

To clarify a point: are we agreed that obvious spam should be deleted immediately?

Yes.
Yesterday has been especially critical. At evening (Paris time), I've been welcomed by a 1845 new posts count! Non spam posts were less than 30...
Note that there has been also a most users online count above 1800 too, it may be linked. I've noticed that many users put in the signature: "OpenOffice 3.1 on Windows Vista / NeoOffice 2.2.3 with MacOS 10.4 / OpenOffice 2.4 on Ubuntu 9.04". So there may be some new bots that fool our registration process.

[RoryOF]

I've noticed that many users put in the signature: "OpenOffice 3.1 on Windows Vista / NeoOffice 2.2.3 with MacOS 10.4 / OpenOffice 2.4 on Ubuntu 9.04". So there may be some new bots that fool our registration process.

I had noticed that signature in common use by spammers before my recent trip. I think you are right that there may be some spambots involved and that this is their stamp.

[acknak]

I suspect that some of the different spam registrations are all manned by the same person. That would allow them to get around any posting frequency limits. If the same person was running the accounts, he/she would likely copy/paste the same signature.

[RoryOF]

For information: here is a recent relevant news item on BBC
http://www.bbc.com/news/technology-17813300

[floris v]

for the moderators.

[RoryOF]

Note that some of the spam User names are of the form Abcedxxxn where n is incremented after the earlier name has been Banned. Other spams originate from Users with random character names.

[/a3]

I've noticed that many users put in the signature: "OpenOffice 3.1 on Windows Vista / NeoOffice 2.2.3 with MacOS 10.4 / OpenOffice 2.4 on Ubuntu 9.04". So there may be some new bots that fool our registration process.

Yes, I'm starting to think so. Three of these have "OpenOffice 3.1 on Windows Vista" and one has the one you posted above.

memberlist.php?mode=viewprofile&u=53908
memberlist.php?mode=viewprofile&u=53910
memberlist.php?mode=viewprofile&u=53912
memberlist.php?mode=viewprofile&u=53913


I should also note that "OpenOffice 3.1 on Windows Vista / NeoOffice 2.2.3 with MacOS 10.4 / OpenOffice 2.4 on Ubuntu 9.04" is the example shown on the registration screen as well.

[kingfisher]

I do not understand the hesitancy to take preventative action. Why wait for the next enormous barrage?

EDIT: I notice that the spammers are not resting. We do NOT have to have this problem.

[Hagar Delest]

I've used the disallow usernames to prevent come back of orpon## and so on.

I'm still reluctant to moderate the NRU post because it will have a negative impact for the newcomers. So as long as we can handle a rather low level of spam, let's not confuse normal user.

But if the consensus is to activate the moderation, no problem.

[RoryOF]

I agree wth Hagar about reluctance to moderate NRU posts; If we are experiencing spam floods so also are other locations - I have seen several recent spams on a previously spam free list to which I am subscribed. I think we should continue as at present, where spamers are banned as soon as may be, in the hope that they will shift their target. If there is an increase in spam floods it should surely affect other Apache forums/lists and there may be offered a unified solution for Apache hosting, which we should use, rather than try reinventing the wheel.

It may be possible to increase the complexity of the Captcha image although I don't like this - I personally have moved away from purchasing from some commercial sites where this has happened on the grounds that I have better things to do than be deciphering obscure images. I prefer a question which requires an intelligent answer.

[kingfisher]

I agree with you about captcha. It can take me many attempts to get the right answer.

For ordinary users it will only be the first post which needs to be moderated. After that, they can be moved to the members' group.

There does not have to be any confusion. It should be a straightforward matter to inform registrants of the requirement for moderation initially and the reason.

As far as comparisons with other forums go, this and oooforum are the only forums I have seen which have a spam problem of any significant degree. I seldom see any spam elsewhere.

[RoryOF]

Am I conscious of a spam peak occurring daily about 14:10 (Irish/British summer time=UTC+1)? If so, I'll try to be online about then with my little hatchet

[floris v]

The best thing would be that a new member will only leave the new member group when at least some of his posts have been approved. That way a spammer will never see his posts become visible/available, removing the point of posting at all.

[Hagar Delest]

The confusion I fear with the first post(s) moderation is that the real new user may wonder if there has been a problem or not since the message will not appear. Perhaps there is a clear warning about that. And then perhaps he will fear about the time needed to get approved.

NB: back about disallowing usernames, it works only once we spot them sadly, that is when accounts like username## have already been created. We can't prevent that.

[RoryOF]

I agree with Hagar; often a new User posts in panic and dispair. Their document is in difficulties, they have a deadline (were we not all there in our younger days?); they want instant help and will not appreciate the delay for moderation. Could we disable URL links in first posts?

[Hagar Delest]

Could we disable URL links in first posts?

This is the critical point. And I've been in all the options of the admin panel several times and it seems we can't. According to some web searches, we have to use a phpBB MOD for that! Hence my call for a new admin some weeks ago!

[RoryOF]

I'm not au fait with phpBB code and haven't read this entire thread with the attention it deserves, so this suggestion may already have ben made: might it be easily possible to make a new level of user, such that all New Users have not access to URL links, but after a threshold number of posts (or Moderator approval if that level not reached) they are automatically advanced to Regular User and get URL link privileges? If it has been suggested, disregard this post.

[RGB]

I'm not au fait with phpBB code and haven't read this entire thread with the attention it deserves, so this suggestion may already have ben made: might it be easily possible to make a new level of user, such that all New Users have not access to URL links, but after a threshold number of posts (or Moderator approval if that level not reached) they are automatically advanced to Regular User and get URL link privileges? If it has been suggested, disregard this post.

AFAIK, the only part that cannot be done right now is precisely the "have not access to URL links". It's easy to create a group that it is automatically promoted after a number of post that, for example, do not show signatures or cannot attach files (as a matter of fact, that group already exists but it is not used), but I cannot find a way to disable urls on messages.

[RoryOF]

@RGB: there is discussion at
http://www.phpbb.com/community/viewtopic.php?f=64&t=2130000
which may be helpful.

[Hagar Delest]

I've already implemented the NRU (Newly Registered Users) with restricted features (post limit is 10 for the moment). It prevents the use of BBCode. But I noticed that some spammers still manage to post BBCode so I guess that it disables the BBCode toolbar but if you know the BBCodes (or if you've a post that has already been formatted with the relevant BBCodes, then, it doesn't prevent the forum to parse them! So it's a quite weak feature (I intend to make some tests with a new user but no time yet).
NB: once the post limit reached, NRU get automatically to RU status with the standard features.

As for the interesting discussion you've linked: it is exactly the point we are making! Currently, you've to use MODS (phpBB extensions that need to be installed by an admin who knows what he's doing. As said by the posters, this problem (url in new users posts) is now a basic and such counter-measure should be delivered by default in phpBB...

As for the "Allow links in posts/private messages" option, as said in the other topic, it's too restrictive since it will affect the whole board. It should be part of the groups permissions.

The signature is not really the problem, few spammers use it for spam in fact. And it would prevent us from knowing the basic information requested to help.

[floris v]

I read the Wikipedia article on spam bombs (okay, Wikipedia not 100 % reliable as source alert) that said that posts containing lots of links are not intended to be viewed by the visitors but are meant to be parsed by search engine bots to improve the ranking of the advertised websites. Disabling the url tag won't stop spammers from posting the actual urls, they just wouldn't be converted into live links, but if the links aren't intended to be clicked anyway, that wouldn't matter to the spammers. What we need to stop the spammers is a way to detect urls in posts by new members and move them to the quarantine section/moderating queue.

[acknak]

A "no hyperlinks" feature would be perfect--as long as you could apply it to specific groups.

I can't see any pressing need for new & regular users to post live hyperlinks. They can just insert the link as text and anyone who needs to follow the link can copy/paste into the browser. If a moderator decides it's important (and legitimate), they can convert it to a live link. We already manage such manual tweaks now and that works just fine.

Once someone is promoted to Volunteer status, then live links should be allowed.

That would remove the incentive for almost all the spam, not to mention other mischief.

[acknak]

... Disabling the url tag won't stop spammers from posting the actual urls, they just wouldn't be converted into live links, but if the links aren't intended to be clicked anyway, that wouldn't matter to the spammers. ...

A plain text link (a bare web address) is not only non-clickable, it is also ignored by search engines. As long as the forum software does not convert the web address to a link link, then there's no value to posting spam/SEO links here.

That may not stop the spammers right away--they're paid to post, not for getting results--but eventually, if the spam links are no longer boosting the search results, the people paying for the posting will catch on and stop hiring people to do it.

[floris v]

... Disabling the url tag won't stop spammers from posting the actual urls, they just wouldn't be converted into live links, but if the links aren't intended to be clicked anyway, that wouldn't matter to the spammers. ...

A plain text link (a bare web address) is not only non-clickable, it is also ignored by search engines. As long as the forum software does not convert the web address to a link link, then there's no value to posting spam/SEO links here.

Duh, thanks for pointing that out.

That may not stop the spammers right away--they're paid to post, not for getting results--but eventually, if the spam links are no longer boosting the search results, the people paying for the posting will catch on and stop hiring people to do it.

This link (alert: Dutch) explains how that's done: on the General tab of the ACO you can disable links in) private) messages for all users, then afterwards you have to add hyperlinks to the permissions of moderators - except that I can't find anything about links in the permissions for groups or users (I admit that I can't find pretty much anything in the ACO, which is horrible, convoluted, anti-intuitive etc, -- end rant).

[FJCC]

I posted a note on the dev list about the events last weekend. I know Hagar and Rory have seen it and there was also a helpful response from Rob Weir. One of them is intended to eliminate the incentive to post links, as you have just been discussing. I have no idea what it takes to implement any of this stuff.

What you are seeing is odd. A successful spammer does not work this
way. They want their posts to survive and persist, to have impact. To
build up Google Pagerank they want posts on 400 different websites
rather than 400 posts on one website. It doesn't make sense to send
400 to one website, since that will obviously draw attention from
moderators. This sounds more like a denial of service attack than
spam.

But a few ideas that might work, based on my experience running forums:

1) Change the CAPTCHA used in your registration. What you have right
now is too easy.

2) Much forum spam is targeted at getting links to raise their search
engine position. You can remove that incentive by ensuring that all
links given by users are given the rel="nofollow" attribute. Most
major sites, like Wikipedia, online newspapers, etc., do this in order
to reduce the incentive to add spam. I have the impression that the
spammers search the web for high Pagerank websites that do not cloak
their URL's with nofollow. These sites are targeted by spammers. If
we get off that list, then we'll get less spam.

3) Longer term, maybe there is some way we can run forum posts through
Apache's SpamAssasin? It would probably require some custom app dev
with phpBB, but it could result in a very sophisticated anti-spam
solution.

-Rob

[RoryOF]

My feeling (and it is no more than that) is that an Apache-wide solution will eventually be offered (by Infra?) and that we should then go with that. They like that things are done their way. In the meantime we just carry on and if there are any quick fixes we can apply, we apply them. Even if a quick fix doesn't eliminate _all_ spam, if it makes a reasonable reduction in the quantity it will be worthwhile.

[Hagar Delest]

I don't know how we can implement the SpamAssasin for forum posts. Perhaps it's rather easy. Else, we need the anti-spam mods from phpBB.