driver.execute_script("arguments[0].innerHTML = arguments[1]", element, HTML)

I figured out the answer for anyone that finds this later. managed_insecure_content_allowed_for_urls was the correct syntax. List<string> insecureUrls = new List<string>() { "https://www.someplace.com" }; options.AddUserProfilePreference("profile.managed_insecure_content_allowed_for_urls", insecureUrls);

I would recommend not to use time.sleep() with selenium. Instead WebDriverWait / presence_of_element_located() can be used: from selenium import webdriver from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.common.exceptions import TimeoutException from bs4 import BeautifulSoup DELAY = 30 wd = webdriver.Chrome("<PATH_TO_CHROME_DRIVER>") wd.maximize_window() url = 'https://www.aodour.pk/brand/hemani' wd.get(url) try: # waiting for pop-up landing_popup = WebDriverWait(wd, DELAY).until(EC.presence_of_element_located((By.CLASS_NAME, 'landing_popup'))) # waiting for loader to vanish page_loader = WebDriverWait(wd, DELAY).until_not(EC.presence_of_element_located((By.CLASS_NAME, 'pageloader'))) # waiting for button to be clickable close_button = WebDriverWait(wd, DELAY).until(EC.element_to_be_clickable((By.XPATH, '//button[./span[contains(@class, "icon-close")]]'))) wd.execute_script("arguments[0].click();", close_button) print("Ad skipped") except TimeoutException: print("Ad pop-up didn't appeared") except Exception as e: print("Error: " + str(e)) html = wd.page_source soup = BeautifulSoup(html, 'html.parser') print(soup)

Maybe using webdriver as a variable name for the webdriver is confusing. Try using driver = webdriver.Chrome(executable_path=chromedriver_path) And then you can resize window size with the following driver.set_window_size(340, 695)

This is while driver.find_element_by_class_name ("unviewed") == True: always False. I shoul do it just like this: def look_for_element (self): driver = self.driver print len (driver.find_elements_by_class_name ("unviewed"))) Or better by xpath: def look_for_element (self): driver = self.driver print len (driver.find_elements_by_xpath ("// tr [contains (@class, 'unviewed') and @ data-rowkey]")))

Finding and checking items is awfully hard I advise you to start by studying page locators, if you don't have time - use Selenium Builder (extension for FireFox) to find elements. If possible, agree with the developers to assign IDs to unique elements and use them as locators. If ID is not possible, then the alternative is CSS locators or XPath (More CSS locators and XPath). The code becomes cumbersome in just a couple of steps. The solution is page object pattern. Better use Selenium + Python + PyTest + OOP, the tests are as readable as possible.

Thanks. It seems to be working