Quasarke Forums
  • Communities
  • Multi-communities
  • Support Lemmy
  • Search
  • Login
Selfhosted@lemmy.worldbytofu@lemmy.nocturnal.garden
12 days

[AIT] Creepy crawlies (AI crawlers on git.kernel.org)

people.kernel.org English

Cross posted from: https://lemmy.world/post/53750210

9
    Creepy crawlies
    people.kernel.org
    You've probably heard me complain about the "AI crawlers" before, but now I actually have some hard numbers I can put up to show their im...
    You must log in or register to comment.

    • gheesh@lemmy.worldEnglish
      12 days

      Should be cross-posted to fuck_ai@lemmy.world too!

        • curbstickle@anarchist.nexusEnglish
          12 days

          I’d prefer a new post there over a cross-post from here, personally.

            • tofu@lemmy.nocturnal.gardenEnglish
              12 days

              Where do you see the difference?

                • curbstickle@anarchist.nexusEnglish
                  12 days

                  Client dependent as to whether it shows the community its cross posted from.

                  Mostly because that crowd likes to come in and downvote anything they notice ai-related in any way, shape, or form, or they post comments I’ll need to remove. Despite tags being added so they could choose to filter those posts out instead.

                  Its less about the post and more about the general behavior of some people in that comm.

            • Possibly linux@lemmy.zipEnglish
              12 days

              Once the VC money runs out it will be less of an issue

              In the meantime we are screwed

              • IanTwenty@piefed.socialEnglish
                12 days

                Didn’t know this:

                Shallow clones are awful. Run your own damn mirror if you’re going to do something nasty like that.

                Shallow is the default/recommended in a lot of places though apparently:

                Shallow wins the clone, every time, by 4-6x. Which is exactly why actions/checkout defaults to depth=1, why GitLab CI ships GIT_DEPTH=20, and why Vercel hard-codes --depth=2 with no way to change it.

                https://bex.co/blog/2026/08/08/git-push-paas-shallow-vs-partial-clone-build-checkout

                Separately, one source of the scraping:

                welcome to the wonderful world of “proxy SDK monetization.” It’s big business, and your TV is probably doing it.

                The link in the article is useful, https://spur.us/blog/smart-tv-apps-residential-proxy-sdks

                Amazon makes it explicit: its Device and System Abuse Policy prohibits apps that facilitate proxy services for third parties. Roku has reportedly shut the door too: Lowpass, syndicated at The Verge, reported that Roku bars developers from using Bright SDK and similar proxy services, and that Roku apps using the SDK disappeared after the company was contacted.

                LG and Samsung have not drawn an equivalent public line. That is the gap these apps are living in. The same business model that Amazon bans and Roku reportedly blocks is still showing up at scale on webOS and Tizen.

                It concludes:

                A TV app should not be able to quietly turn a living-room device into residential proxy infrastructure. Screensavers, games, clocks, and novelty apps can be boring, cheap, or ad-supported. If an app is going to monetize a household’s internet connection, the user should be clearly informed about what that means, how the connection will be used, and what risks and tradeoffs they are accepting.

                We need to help everyone in the world block their TVs from doing this, this is such an abuse of trust.

                • CaptainShardmind@lemmy.worldEnglish
                  12 days

                  The part that got me is a single fork turning into billions of valid URLs, because every commit, diff and plain render is its own page. I run a small Gitea behind Nginx Proxy Manager, and a forge is exactly the kind of app where that math applies even at hobby scale. This has me checking whether anonymous visitors need the commit and diff views at all, or whether putting the web UI behind a login would cost me anything. Clones over git would still work for anyone who actually wants the code.

                  • lntl@lemmy.mlEnglish
                    11 days

                    Probably a silly question: Is serving HTML required? git exists after all

                      • tofu@lemmy.nocturnal.gardenEnglish
                        11 days

                        Yes, that’s the difference between a pure git server and what’s usually called a forge

                      Selfhosted@lemmy.world

                      selfhosted@lemmy.world

                      Subscribe from remote instance

                      Create post

                      Report community

                      Modlog
                      You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !selfhosted@lemmy.world

                      A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don’t control.

                      Rules:

                      Detailed Rules Post

                      1. Be civil.

                      2. No spam.

                      3. Posts are to be related to self-hosting.

                      4. Don’t duplicate the full text of your blog or readme if you’re providing a link.

                      5. Submission headline should match the article title.

                      6. No trolling.

                      7. Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.

                      8. AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.

                      Resources:

                      • selfh.st Newsletter and index of selfhosted software and apps
                      • awesome-selfhosted software
                      • awesome-sysadmin resources
                      • Self-Hosted Podcast from Jupiter Broadcasting

                      Any issues on the community? Report it using the report flag.

                      Questions? DM the mods!

                      Visibility: Public

                      This community is visible to everyone.

                      174 users / Day1.18K users / Week4.7K users / Month6.8K users / 6 months559 posts6.12K comments1 local subscriber63.7K subscribers
                      Mods: Ruud@lemmy.world
                      • BE: 1.0.0-beta.2
                      • Modlog
                      • Instances
                      • Docs
                      • Code
                      • join-lemmy.org