There is a file, called robots.txt, that when associated with a web site instructs search engine robots not to catalog the file or files disallowed. I have a teeny, tiny one here on zanshin.
The White House has a huge one, covering a huge list of topics.
I think this is wrong. The White House works for me, I’m a member of the board of the United States of America, along with nearly 300 million other citizens. The President, and his staff, are temporary appointments put there by us to oversee the running of the country for four years. How dare they play keep away with information.