My area of expertise is in web based applications. The languages I know best are pretty much designed for the web. Even though this program is web related, it obviously shouldn't be a web based application. This is the sort of program that will need to run for several hours before it completes its task. That's not a task you want a web server to perform, that wouldn't make any sense. So I decided to make this program in Python. I know the syntax of python. I've made a couple simple applications in it, but the difference between the ease of me coding in PHP and Python is comparable to the difference between swimming upstream and swimming downnstream.
Luckily there are resources that are designed to help out people like me! The first website I visited was Rosetta code. First thing I needed to know is how python handles opening webpages.
For example in PHP the code would be:
but in python it would be:
import urllib url = urllib.urlopen("http://www.rosettacode.org") print url.read() url.close()
Eventually my loop got to looking like this:
while(i < len(poop) and cont): print str(i)+".) opening page "+str(poop[i])+' -- '+time.strftime("%H:%M:%S") try: resp = opener.open('http://www.mysearch.com/pagenum?id='+str(poop[i]), , 10) res = resp.read() print "finding match" checked = open('db/'+section+"_checked.txt", "a") found = open('db/'+section+"_found.txt", "a") if re.search('my search term', res): print "found!" found.write(poop[i]+'\n') ratioless.append(poop[i]) else: print "nope" checked.write(poop[i]+'\n') time.sleep(6) i = i + 1 failStreak = 0 checked.close() found.close() except: if(errorOn == poop[i]): i = i + 1 print "still can't open, moving on" else: errorOn = poop[i] print "couldn't open" failStreak = failStreak + 1 print str(failStreak) + " failures" time.sleep(60) continue;
Rosetta code is a good site, but PHP is so easy because of the number of functions that just do everything for you. Eventually I got so uncreative with trying to describe my problem to google that I just typed in something along the lines of "php explode function for python", and such a query led me directly to Php2Python. A website designed to give you all the PHP functions in python syntax.
Using Php2Python I learned that Python has an equivalent explode function and now I use it for DB management:
poop = open('db/'+section+".txt", "r").read().split('\n')