Lemmy
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
fattyfoods@feddit.nl to Open Source@lemmy.ml · 1 day ago

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

external-link
message-square
87
fedilink
  • cross-posted to:
  • opensource@programming.dev
485
external-link

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

fattyfoods@feddit.nl to Open Source@lemmy.ml · 1 day ago
message-square
87
fedilink
  • cross-posted to:
  • opensource@programming.dev
Anubis, which block AI scrapers from scraping websites to death, has been downloaded almost 200,000 times.
  • bdonvr@thelemmy.club
    link
    fedilink
    arrow-up
    13
    ·
    20 hours ago

    Thanks. Anything special configuring it?

    • Captain Beyond@linkage.ds8.zone
      link
      fedilink
      arrow-up
      14
      ·
      edit-2
      19 hours ago

      I keep my server config in a public git repo, but I don’t think you have to do anything really special to make it work with lemmy. Since I use Traefik I followed the guide for setting up Anubis with Traefik.

      I don’t expect to run into issues as Anubis specifically looks for user-agent strings that appear like human users (i.e. they contain the word “Mozilla” as most graphical web browsers do) any request clearly coming from a bot that identifies itself is left alone, and lemmy identifies itself as “Lemmy/{version} +{hostname}” in requests.

Open Source@lemmy.ml

opensource@lemmy.ml

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !opensource@lemmy.ml

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

  • Open Source Initiative
  • Free Software Foundation
  • Electronic Frontier Foundation
  • Software Freedom Conservancy
  • It’s FOSS
  • Android FOSS Apps Megathread

Rules

  • Posts must be relevant to the open source ideology
  • No NSFW content
  • No hate speech, bigotry, etc

Related Communities

  • !libre_culture@lemmy.ml
  • !libre_software@lemmy.ml
  • !libre_hardware@lemmy.ml
  • !linux@lemmy.ml
  • !technology@lemmy.ml

Community icon from opensource.org, but we are not affiliated with them.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 803 users / day
  • 1.49K users / week
  • 3.25K users / month
  • 6.76K users / 6 months
  • 1 local subscriber
  • 38.7K subscribers
  • 457 Posts
  • 2.95K Comments
  • Modlog
  • mods:
  • Evan@lemmy.ml
  • kevincox@lemmy.ml
  • CrypticCoffee@lemmy.ml
  • Lettuce eat lettuce@lemmy.ml
  • BE: 0.19.9
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org