If you liked this post, you may also be interested in...
- Hertz Ordered To Tell Court How Many Thousands Of Renters It Falsely Accuses Of Theft Every Year
- Even As Trump Relies On Section 230 For Truth Social, He's Claiming In Lawsuits That It's Unconstitutional
- Letter From High-Ranking FBI Lawyer Tells Prosecutors How To Avoid Court Scrutiny Of Firearms Analysis Junk Science
- FTC Promises To Play Hardball With Robocall-Enabling VOIP Providers
- FOIA Lawsuit Featuring A DC Police Whistleblower Says PD Conspired To Screw Requesters It Didn't Like
Reader Comments
Subscribe: RSS
View by: Time | Thread
Robots.txt?
[ link to this | view in chronology ]
Re: Robots.txt?
If you tell Google to disallow certain areas, it will not cache those areas (allegedly), but it will still go into them. In order to truly disallow Google, one needs to add special META tags to all pages -- tags that only Google honors. Shame on Google for not adhering to the established standard on this front.
Not that this changes anything with AFP.
[ link to this | view in chronology ]
Re: Robots.txt?
[ link to this | view in chronology ]
No Subject Given
[ link to this | view in chronology ]
Re: No Subject Given
Seems to me that (at least from a legal point of view), AFP's licensing agreement should simply oblige its partners to include an appropriate robots.txt with the licensed stories. Then, if Google doesn't honour these, AFP might have a case.
[ link to this | view in chronology ]