Cafe Announcements 📣 Cafe is now on Cloudflare - stop AI bots scraping and training

[Pete Thomas]

This is just to let you all know we are now using Cloudflare to combat bots, blocking them from the site including model training. Hopefully we are not blocking the good bots that we need for SEO.

 

[atwl77]

One annoying thing with Cloudflare to be aware of, if your IP address is blocked for whatever reason, they make it incredibly difficult to get it resolved.

You only have a "Ray ID" to report the instance, but they tell you to contact the website admin -- but the problem is, because you're blocked, you can't actually access the website to contact the admin.

I had this issue with another forum once, but fortunately I could find the admin's contact e-mail from elsewhere (Reddit), in order to actually inform the admin of the issue. I was effectively banned from the forum until the issue could be resolved. Of course, a VPN will instantly solve the issue, but at that time I didn't have a VPN and didn't know enough about them to circumvent the block.

I notice the Contact Admin link here is just a webform, which could potentially make this issue harder to resolve if you get blocked, since there's no actual e-mail address to contact from the outside... so something to keep in mind and possibly work out a solution?

 

[Pete Thomas]

One annoying thing with Cloudflare to be aware of, if your IP address is blocked for whatever reason, they make it incredibly difficult to get it resolved.

I'll do some more research but as far as I know Cloudflare itself does not block IP addresses, BUT it may use 3rd party blacklists such as ProjectHoneypot or Stopforumspam. This cafe may well also do that or at least flag attempted registration any blacklisted IPs of spammers who get on those lists.

If we ban a spammer soon after registration they automatically get added to those lists if they aren't already on them. However we usually catch spammers before they post anyway.

 

[Colin the Bear]

What is scraping?

 

[Grumps]

And now @PigSquealer can't get in!!!

I was always suspicious of that guy...

 

[Pete Thomas]

What is scraping?

Scraping is where content, entire pages or an entire website is taken and published elsewhere. It has happened to many pages of TTS and in some cases a take down notice was effective. Some people get away with it by paraphrasing. very easy with AI.

Use of a website to train AI will often result in paraphrasing of content, hence we want to stop that as well. Some may say you cannot effectively block the bots, but there are some clever tactics to confuse them that we can do.

 

[Ghr9204]

Use of a website to train AI will often result in paraphrasing of content, hence we want to stop that as well. Some may say you cannot effectively block the bots, but there are some clever tactics to confuse them that we can do.

I don't know if this is a case of paraphrasing, but I found an instance a few days ago provided by AI that was totally "off the wall". It was related to motorcycles. The AI based its findings on information most readily available, forum replies. Wished I captured the AI reply and links then because now can't remember the details, but at the time I read it was a bit hilarious and could lead someone down the wrong rabbit hole.

 

[Ivan]

What is scraping?

It is manoeuvre No. 2 when you are in trouble with the significant other

Scraping comes after manoeuvre No.1; bowing

 

[Pete Thomas]

What is scraping?

Another example is Waybackmachine. It takes a site and publishes snapshots of it at various times. It does this with no regard to copyright and without any kind of consent from the rights owner.

 

[Grumps]

Another example is Waybackmachine.

Does tend to keep folks honest though.

 

[PigSquealer]

What is scraping?

Trying to earn a living as a musician

 

[PigSquealer]

And now @PigSquealer can't get in!!!

I was always suspicious of that guy...

I have nosed my way back in…slippery swine talent.
My family best I know has no ties to Orwells timeline.
As family lore goes. My family comes from humble instrument makers of a unique style. Essentially the needs of a distress call.

PS-92 Pig Squealer Call

The PS-92 Pig Squealer Call. This call is designed to duplicate the distress squeals of a feral hog which attracts others. This call is very useful along edges of thickets where stalking is not possible.

www.haydels.com

Looking forward to seeing you at next BBQ.

 

[atwl77]

Interestingly, I'm not noticing any "verify you're human" screen before accessing the forum, so I guess we're on a less intrusive (and hopefully less problematic) Cloudflare system?

 

[Pete Thomas]

Interestingly, I'm not noticing any "verify you're human" screen before accessing the forum, so I guess we're on a less intrusive (and hopefully less problematic) Cloudflare system?

It is there on the registration page, contact form (if not logged in) and on the lost password form. Maybe some other areas also.

 

[atwl77]

I see... I guess that's the difference as some sites just put the verification right in front of the main website, so if you can't get past that for whatever reason, you are practically banned from that website.

 

[Pete Thomas]

so if you can't get past that for whatever reason, you are practically banned from that website.

I don’t think I’ve needed a CF captcha just view a site.

I'll do some more research but as far as I know Cloudflare itself does not block IP addresses,

But I think individual site owners may be able block. Again more research to be done…
IP addresses

 

[rhysonsax]

Another example is Waybackmachine. It takes a site and publishes snapshots of it at various times. It does this with no regard to copyright and without any kind of consent from the rights owner.

Waybackmachine is an interesting one and something that I have found useful in the past. I don't know what their financial model is, but do think that there is value in having an archive or record of the everchanging web.

I wouldn't be surprised if governments and some large corporations are building their own archives of what has been on the web, but not making that available to anyone else.

Copyright and consent are important concepts but can they keep up with changing technologies ? In the same way that lots of media companies deny being publishers and so say they have no responsibility for the rubbish that they host and push at users, copyright frameworks from pre-internet eras don't seem to be working well nowadays.

Rhys