Azekeil (azekeil) wrote,
Azekeil
azekeil
  • Mood: shocked

Disturbing!

As a direct result of my last post, I've discovered LJ has about 10 'spiders' monitoring new posts for images and then grabbing them from the servers they're hosted on. That means there are at least 10 bots downloading every new post; the amount of traffic this generates must be considerable! I wonder if this is an LJ-provided function, or if they're individually going to grab their own copies of the pages. Now I think about it, it MUST be LJ-provided, surely, because I don't know of a function or way to grab all new pages.

UPDATE: Apparently LJ does support the use of bots, and even hosts XML showing the URLs of the latest X pictures. Mystery solved, although it's a little anti-social for people just to copy and host the PHP code to do this. Kills home web servers for a while...
Subscribe

  • First full day at home

    This is my first full day at home. Gardening leave. Such an innocuous term for the hunt for another source of income. I was fairly discouraged when I…

  • Exhausted

    After a fairly emotionally exhausting day, I had to do The Great Vehicle Shuffle™. My car's exhaust 'broke' on Saturday evening, and my bike's…

  • An update and some pictures

    Turns out I badly misjudged my ability to cope with the last few days mainly due to Christmas stresses. Anyway that seems to be mainly out of the way…

  • Post a new comment

    Error

    switch
    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
    Help
  • 10 comments

  • First full day at home

    This is my first full day at home. Gardening leave. Such an innocuous term for the hunt for another source of income. I was fairly discouraged when I…

  • Exhausted

    After a fairly emotionally exhausting day, I had to do The Great Vehicle Shuffle™. My car's exhaust 'broke' on Saturday evening, and my bike's…

  • An update and some pictures

    Turns out I badly misjudged my ability to cope with the last few days mainly due to Christmas stresses. Anyway that seems to be mainly out of the way…