Python Selenium | Best Practice for Web Automation Bot s 2018

So what is the best practice for bot building in 2018 on the WEB 3.0?  I’m certainly not an expert on anything, as I’ve stated before.  I just dabble at this point.  Having no university level computer science background. I just sort of muddle things together until they work.  So I really have no clue as to the best practices of web bot design in 2018.  I thought I would share my methodology for a automated web bot in Python and Selenium for  WEB 3.0 .

As always  import Selenium

1
2
3
4
5
6
7

print("*" * 60)
print("MI PYTHON COM SEO TOOL BOT")
print("*" * 60)
 
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

Now lets define our SEO TOOL BOT class.

1
2
3
4
5
6
7
8

class SEO_BOT(object):
 
    def __init__(self,browser,anon_url):
        self.browser = browser
        self.anon_url = anon_url
 
    def main(self):
        pass

Above we have defined our class with class wide variables of “browser” and “anon_url”.
We will use the first class wide variable “browser” to instantiate our web browser below.
The second variable “anon_url” is the Url redirect service we are going to use.

Now instead of just having a huge “main” function of a big mass of jumbled code. I’ve found it easier, in my experience debugging. To split the classes tasks into different functions. So we are just going to pass the main function for the time being. We can always come back and add whatever we want later. Below we will start on our “go_anon” function.

1
2
3
4
5
6

def go_anon(self):
        ###   SELENIUM IMPLICIT WAIT  !!! IMPORTANT !!!
        self.browser.implicitly_wait(300)
        print("GETTING " + str(self.anon_url))
        ###  GO TO ANON URL
        self.browser.get(self.anon_url)

In this function above we are calling the class wide variables that will instantiate the web driver and the Anon URL. Now lets instantiate our class below.

1

bot = SEO_BOT(webdriver.Firefox(),"https://www.kproxy.com")

To instantiate the class we imputed the Selenium Web Driver for Firefox. Next we add the url string to get to the anon redirect kproxy. Lets put everything together. Run it and see what happens.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

print("*" * 60)
print("MI PYTHON COM SEO TOOL BOT")
print("*" * 60)
 
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
 
class SEO_BOT(object):
 
    def __init__(self,browser,anon_url):
        self.browser = browser
        self.anon_url = anon_url
 
    def main(self):
        pass
 
    def go_anon(self):
        ###   SELENIUM IMPLICIT WAIT  !!! IMPORTANT !!!
        self.browser.implicitly_wait(300)
        print("GETTING " + str(self.anon_url))
        ###  GO TO ANON URL
        self.browser.get(self.anon_url)
 
bot = SEO_BOT(webdriver.Firefox(),"https://www.kproxy.com")
bot.go_anon()

So we should have our SEO TOOL BOT for the WEB 3.0 opening a browser window and going to kproxy. Congratulations we just defined a function in our bot class, passed variables to it and made it do something.

Now, lets define another function for our Python Selenium 3.0 Web Bot to do. Lets actually go some where.

Read the next tutorial here.