I do not recommend using Selenium because according to https://hackernoon.com/do-not-use-selenium-for-web-scraping-de90cca7c804 he claims that Selenium is not a specialized web scraping tool (data extraction technique used to collect data from websites), but a tool to perform automated web application tests. It is recommended to use tools like Scrapy or Beautiful Soup + Requests.The Datasus website finds it difficult to use Selenium because the site has many check boxes, generating many combinations to download all content, it will be very laborious to do so in Selenium, with other tools being better for this purpose.I've done something like that to get all Enem's results using Bash Script and cURL, as follows:Use Google ChromeOpen the Datasus websiteRight click, and select "Inspect", will open the developer's tools to the right of the browser.Click the "Network" tab on developer toolsOn the Datasus website on the left, select the options, and click the "Show" button that appears on the siteIn the developer tools click on the request sent to the server called "tabcgi. exe? sih/cnv/nrbr. def" with the RIGHT button, select COPY -> Copy the...Depending on where to develop the script, if it is in Linux select "Copy as cURL (Bash)" if it is in Windows "Copy as cURL (cmd)".With the copied cURL command just paste on the Linux Bash and it will make the request equal to the Browser. You can go modifying the parameters of the cURL request to go looking for other site information.Example below the cURL request:curl 'http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sih/cnv/nrbr.def' \ -H 'Connection: keep-alive' \ -H 'Pragma: no-cache' \ -H 'Cache-Control: no-cache' \ -H 'Upgrade-Insecure-Requests: 1' \ -H 'Origin: http://tabnet.datasus.gov.br' \ -H 'Content-Type: application/x-www-form-urlencoded' \ -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36' \ -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \ -H 'Referer: http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sih/cnv/nrbr.def' \ -H 'Accept-Language: pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7' \ -H 'Cookie: TS014879da=01e046ca4c72569773aca201f18700eeeba156dca36d80d2164402d50541a167b0fe28c1eed5e284f878cbb97def8098f34d4600bd' \ --data 'Linha=Macrorregi%E3o_de_Sa%FAde&Coluna=--N%E3o-Ativa--&Incremento=Interna%E7%F5es&Arquivos=nrbr2003.dbf&pesqmes1=Digite+o+texto+e+ache+f%E1cil&SMunic%EDpio=1&pesqmes2=Digite+o+texto+e+ache+f%E1cil&SCapital=1&pesqmes3=Digite+o+texto+e+ache+f%E1cil&SRegi%E3o_de_Sa%FAde_%28CIR%29=1&pesqmes4=Digite+o+texto+e+ache+f%E1cil&SMacrorregi%E3o_de_Sa%FAde=TODAS_AS_CATEGORIAS__&pesqmes5=Digite+o+texto+e+ache+f%E1cil&SMicrorregi%E3o_IBGE=TODAS_AS_CATEGORIAS__&pesqmes6=Digite+o+texto+e+ache+f%E1cil&SRegi%E3o_Metropolitana_-_RIDE=TODAS_AS_CATEGORIAS__&pesqmes7=Digite+o+texto+e+ache+f%E1cil&STerrit%F3rio_da_Cidadania=TODAS_AS_CATEGORIAS__&pesqmes8=Digite+o+texto+e+ache+f%E1cil&SMesorregi%E3o_PNDR=TODAS_AS_CATEGORIAS__&SAmaz%F4nia_Legal=TODAS_AS_CATEGORIAS__&SSemi%E1rido=TODAS_AS_CATEGORIAS__&SFaixa_de_Fronteira=TODAS_AS_CATEGORIAS__&SZona_de_Fronteira=TODAS_AS_CATEGORIAS__&SMunic%EDpio_de_extrema_pobreza=TODAS_AS_CATEGORIAS__&SCar%E1ter_atendimento=TODAS_AS_CATEGORIAS__&SRegime=TODAS_AS_CATEGORIAS__&pesqmes16=Digite+o+texto+e+ache+f%E1cil&SCap%EDtulo_CID-10=TODAS_AS_CATEGORIAS__&pesqmes17=Digite+o+texto+e+ache+f%E1cil&SLista_Morb__CID-10=TODAS_AS_CATEGORIAS__&pesqmes18=Digite+o+texto+e+ache+f%E1cil&SFaixa_Et%E1ria_1=3&pesqmes19=Digite+o+texto+e+ache+f%E1cil&SFaixa_Et%E1ria_2=TODAS_AS_CATEGORIAS__&SSexo=TODAS_AS_CATEGORIAS__&SCor%2Fra%E7a=TODAS_AS_CATEGORIAS__&zeradas=exibirlz&formato=prn&mostre=Mostra' \ --compressed \ --insecure In the field --date you have going to change the values for all possibilities and go making new cURL requests until you download all the information from the site.The return of cURL requests comes in HTML, if you prefer to remove all HTML tags you can use a pipe with lynx.$ curl <comando> | lynx --dump -stdin > resultado1.txt

Yeah, just put the PhantomJS start event on Click and make a new call.Ex: Do what you have to do with PhantomJS and finish it.Call the screen for the User to do its operation.Call PhantomJS again and continue.Just an Idea.

You can create a new method:public void setNewInputValue(WebElement elem, String value) { elem.clear(); elem.sendKeys(value); }

You can try this way:while True: num_current_tabs = len(browser.window_handles) # Fechar todas - 1 abas, deixando uma aberta para não fechar o navegador. for i in range(1, num_current_tabs): browser.window_handles[i] browser.close() time.sleep(2) # Script para abrir as páginas. for total_new_tabs in range(1, num_of_tabs): self.browser.execute_script("window.open('','_blank');") # Foca na última aba aberta browser.switch_to_window(browser.window_handles[-1]) time.sleep(2) # Fecha a aba remanescente das anteriormente fechadas. browser.windows_handles[0].close()

The problem probably occurs because your JUnit is as test dependency:<scope>test</scope>` This means that it will only be available when you are running a test, for example by running a command mvn test in the project.However, the method main used reveals that you are not running the test really as a test, but rather as a normal program.To actually perform unit tests you must create a class in src/test/java. It can be in any package, ams has to be inside that directory. Then create public and non-static methods annotated with @Test.Example:@Test public void testarTitulo() { ... } Then, to perform the methods of the test classes, you must enter the command mvn test on the console for Maven to run the tests. If you are using an IDE as Eclipse, you can also run the test classes by right-clicking them and accessing the menu Run as > Junit Test.Read a little more about unit tests http://www.vogella.com/tutorials/JUnit/article.html .

This is a cookie https://en.wikipedia.org/wiki/HTTP_cookie#HttpOnly_cookie , that is, it cannot be accessed via javascript. To catch it, you can access the HTTP headers of the request. I use for this the lib in python seleninum-requests, which is similar to Python lib requests.from seleniumrequests import Firefox webdriver = Firefox() response = webdriver.request('GET', 'https://www.google.com/') print (response.cookies) But testing on the web.whatsapp. with, really does not take any cookies, because the web.whatsapp. with does not send any cookies, at least not via HTTP. Which means he can only be setting this up in Javascript.Taking a look at the javascript codes looking for Cookie, discovered one of the methods of a javascript class called setRefTokCookies. The method is undermined, but I gave a formatted in it and he stayed like this:setRefTokCookies:function(e, p){ d.setCookie({ name:g.COOKIE_REF, value:e, path:g.PP_REF, domain:g.COOKIE_DOMAIN, secure:!0 }); var n={ name:g.COOKIE_TOK, value:p, path:g.PP_TOK, domain:g.COOKIE_DOMAIN, secure:!0 }; d.setCookie(n) } Opa, now it's interesting. This method arrows 2 cookies, being the name of one coming from the constant COOKIE_REF and the other coming from the constant COOKIE_TOK. Much like the names of the cookies stored, right?Now we have to find out who the value is p and e passed to that method. The problem is that, as the code is minified, it is not trivial to understand it. Looking calmly at this code, to see if it discovered the class name of this method, I realized that it always did a test before continuing to call the rest of the code. This test is:if (!Store.Conn.me) return r.error("userPrefs: Me has not loaded yet.") Bacana. We got a guy now called Store, global, which seems to have interesting data and is used by our class of interest. Opening the Developer Toolbar on the page https://web.whatsapp.com and analyzing the content of the object Store.Conn, we have a pleasant surprise. 2 variables contain the exact content of these 2 tokens, are they Store.Conn.__x_ref and Store.Conn.__x_serverToken. Knowing this, making the code that takes this information becomes trivial:import selenium.webdriver driver = selenium.webdriver.Firefox() driver.get("https://web.whatsapp.com/") driver.execute_script('console.log(Store.Conn.__x_ref);') driver.execute_script('console.log(Store.Conn.__x_serverToken)') After past QR Code, it brings the values equal to those of cookies. ps: There are other interesting authentication data in this Store.Conn, maybe the most interesting thing is to save it whole and then just reschedule the content in Store.Conn, since this variable is global.

Can you clone your base in parallel? Positive case you could have two bases to run parallel. In that case one would not interfere with the other because they are pointing to different bases

I found the solution by putting a line like this:processosAlvo[y][1].send_keys('\ue007') He's with "Send" before, and then click.

import os from selenium import webdriver dir_path = os.getcwd() profile = os.path.join(dir_path, "profile", "wpp") options = webdriver.ChromeOptions() options.add_argument( r"user-data-dir={}".format(profile)) browser = webdriver.Chrome("./chromedriver.exe", chrome_options=options) browser.get("https://web.whatsapp.com") I was having this problem and I sorted it that way. You just need to log in once, dps it enters without needing to scan the QR Code dnv

Try treating the KeyboardInterrupt exception when running the program, e.g.try: w = WhatsappAPI() w.comeca_acao() except KeyboardInterrupt: w.driver.quit() print "Ate logo!"

I didn't do that, but according to http://docs.codehaus.org/display/SONAR/Analyzing+with+SonarQube+Runner#AnalyzingwithSonarQubeRunner-Multi-moduleProject you can configure a multi-module project and include multiple projects through certain file settings sonar-project.properties. For example, consider the following folder structure:Configuration file:# Root project information sonar.projectKey=org.mycompany.myproject sonar.projectName=My Project sonar.projectVersion=1.0 Some properties that will be inherited by the modules sonar.sources=src List of the module identifiers sonar.modules=module1,module2

I've managed to solve people. I manually imported OpenQA dll. Selenium. Support. UI and solved. Putting down Nuget didn't work.

I really don't understand the problem that's happening to you, it would help if you add your code, but here's an example:from selenium import webdriver driver = webdriver.PhantomJS() driver.get('https://pt.wikipedia.org/wiki/Transportes_de_Lisboa') lis = driver.find_elements_by_css_selector('#toc ul li') for item in reversed(lis): print(item.text) Note: that my driver (PhantomJS) may not be the same as yours, adjust to what you're wearing.

Solution found:def DistribuiProcesso(): try: log = open('c:\\E-Proc\\Distribuicao ' + str(hoje) + '.txt', 'w+') #Cria log de controle. except FileNotFoundError: log = open('c:\\E-Proc\\Distribuicao ' + str(hoje) + '.txt', 'w+') distribuir = Select(browser.find_element_by_id('selProNov771230778800100040000000000014')) #Abre o seletor dos Procuradores gerenciar = Select(browser.find_element_by_id('selTipo771230778800100040000000000014')) procuradores = distribuir.options for x in range(len(procuradores)): distribuir = Select(browser.find_element_by_id('selProNov771230778800100040000000000014')) #Abre o seletor dos Procuradores gerenciar = Select(browser.find_element_by_id('selTipo771230778800100040000000000014')) procuradores = distribuir.options nome = str(procuradores[x].text) if x != 0 and x != 1 and x != 26 and x != 63 and x != 94: #Regra interna de exceção. controle = 0 listaProcessos = [] for y in range(len(processosAlvo)): #Encontrando na lista os processos que correspondem ao procurador do laço. if processosAlvo[y][2] in nome: controle = controle + 1 processosAlvo[y][1].send_keys('\ue007') processosAlvo[y][1].click() listaProcessos.append(processosAlvo[y]) if controle &gt; 0: gerenciar.select_by_index(1) distribuir.select_by_index(x) for x in range(1): try: executar = browser.find_element_by_id('btnExecutar') executar.send_keys('\ue007') executar.click() time.sleep(10) except: break log.writelines("Para o(a) procurador(a): " + nome) log.writelines('\n') for x in range(len(listaProcessos)): log.writelines(str(listaProcessos[x][0])) log.writelines('\n') voltar = browser.find_element_by_id('btnVoltar') voltar.click() time.sleep(4) LimpaVariaveis() #Limpa a base da informação. SetaConsulta() #Refaz as consultas e realimenta a base. log.writelines('\n') log.close()

Check the name of the element you want to access. The element Dt_Ref does not exist according to the error: data = driver.find_element_by_name('Dt_Ref') "errorMessage":"Unable to find element with name 'Dt_Ref'",

Question solved with the following code: [Test] [TestCaseSource(typeof(massaBase), "ArquivoTXT")] public void AutomacaoLogin(massaBase.DadosTXT coluna)

There are several ways to validate if the file is full. One way to validate is to take the checksum https://en.wikipedia.org/wiki/Cyclic_redundancy_check of the BEFORE file and validate the file you downloaded and compare the values.In Linux, the command is cksum arquivo.txt The exit will be linux@ubuntu:~$ cksum arquivo.txt 3474990042 214 arquivo.txt Where the first code is the CRC32 file, the second is the number of bytes and the third is the file name.Another, which I particularly like most, is to use the https://en.wikipedia.org/wiki/Md5sum of the file, as follows:linux@ubuntu:~$ md5sum arquivo.txt and the answer will be cbc8ccd6c7f5a111f494cfd4e0aea86e arquivo.txt where the first column will be the MD5 hash of the file and the second, the name of the file itself.

Seleneium has a class to work with select->option, try this:from selenium import webdriver from selenium.webdriver.support.ui import Select driver = webdriver.Firefox() driver.get('url') select = Select(driver.find_element_by_id('key')) selecionando pelo texto visivel select.select_by_visible_text('TEIXEIRA') Selecionando pelo valor select.select_by_value('30632-N-PR-2')

You may be taking the wrong field to facilitate, you can take this field in the form to the bottom:driver.findElement(By.id("nomeCliente")).sendKeys("Nome Cliente");In this way you will find the element via ID and pass what you want to fill with sendKeys

From what I've read I don't know if I understand correctly what you want to do, but Selenium has several specific modules to do what you want... the problem is that you would need to go in the html of the page and see which element is which to be able to capture with the selenium.from selenium.webdriver.common.keys import Keys #importa a habilidade de input de chaves e senhas from selenium.webdriver.support.ui import Select #importa a habilidade de usar o select em boxes e pontos from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait #importa a habilidade de setar o 'wait time' do browser from selenium.webdriver.support import expected_conditions as EC #importa a biblioteca de condições esperadas are here some useful libraries of selenium... Now to check the date of the day and check whether the day is the current or the next I would recommend seeing the Id, the name or the id and using the commandvariavel = driver.find_element_by_name('elemento'). Now... if you've already captured the information and played them in a file or variable then I suggest using Pandas to organize the information as dataframes.To check the dates of a link I would pick up the link with find_element_by and then analyze what pixel the date starts and what the pixel it ends (link[n:m]) and thus use the datetime to compare the date sought with the current date.to get the current dateimport datetime from datetime import timedelta data_hoje = (datetime.datetime.now()).strftime("%d%m%Y") data_ontem = (datetime.datetime.now() - timedelta(days = 1)).strftime("%d%m%Y") data_um_dia_n_dias_atras = (datetime.datetime.now() - timedelta(days = n)).strftime("%d%m%Y")