Say you’d like to find out the IP addresses of lines in your apache access.log (or any log file with a similar format, really) that contain “Googlebot”:
grep 'Googlebot' access.log | cut -d' ' -f1 | sort | uniq
which finds the lines via grep, uses cut to extract the first field (space delimited), sorts the IP addresses and then uniqifies them.
Dirt simple, stupidly powerful.
This work, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.