javascript - CasperJS click() doesn't load new html - Stack Overflow

admin管理员组

文章数量:1023803

I'm trying to use CasperJS' click() to follow a link which generates a modal on the current screen. When I query the proper selector and click it in the browser console using document.querySelector().click() it works, but even when I casper.evaluate() this it doesn't work. I found someone who had a very similar problem, but his question remains unanswered, and I am experiencing almost identical problems. casperjs button click doesn't navigate to next page the code I'm currently using is

this.waitForSelector('div.talk-sharing__tools a.rate-button', function() {
    this.then(function() {
        this.evaluate(function() {
            document.querySelector('a.rate-button').click();
});

the page I'm trying to scrape is

I'm trying to use CasperJS' click() to follow a link which generates a modal on the current screen. When I query the proper selector and click it in the browser console using document.querySelector().click() it works, but even when I casper.evaluate() this it doesn't work. I found someone who had a very similar problem, but his question remains unanswered, and I am experiencing almost identical problems. casperjs button click doesn't navigate to next page the code I'm currently using is

this.waitForSelector('div.talk-sharing__tools a.rate-button', function() {
    this.then(function() {
        this.evaluate(function() {
            document.querySelector('a.rate-button').click();
});

the page I'm trying to scrape is http://www.ted./talks/uri_alon_why_truly_innovative_science_demands_a_leap_into_the_unknown

• javascript
• html
• phantomjs
• casperjs

Share Improve this question Follow edited Jun 13, 2014 at 15:10 ragingSloth asked Jun 13, 2014 at 4:13 ragingSlothragingSloth 1,10488 silver badges2323 bronze badges 3

• Did you try with casper.click('a.rate-button'); – Fanch Commented Jun 13, 2014 at 7:34
• 2 The untold truth about the casperjs and phantomjs tags is that nobody can help you solve the problem as long as you don't provide the link to the page in question. The pages you scrape behave very differently. That is why the SO format of posting does not really work for such question. You would have to post the plete page code here, which is really not encouraged. Also, your code should work as is, the page does something funny. We don't want to dream up something that might work. – Artjom B. Commented Jun 13, 2014 at 8:56
• @ArtjomB. I actually disagree with you on this one. This is a pretty general question, and while scraping questions aren't ideal for SO, there is a concrete usefulness here and I've been able to answer a few of these questions in the past. Also the page is now linked, but I assume that's a more recent development. – Slater Victoroff Commented Jun 13, 2014 at 15:27

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 6

It is really impossible to do those navigation steps with the phantomjs engine. It seems that the QtWebkit fork of phantom (version 1.9.7) is not up to the task anymore. Although your code works fine as is with slimerjs. You can fortably install slimerjs through npm:

npm install -g slimerjs

and run your script with

casperjs --engine=slimerjs script.js

---

I tried several things with phantomjs that did not work. I added casper.on listeners for remote.message and resource.error, but those didn't show any errors when running the script in phantom. logLevel: "debug" also didn't show anything.

First: Using a casperjs function.

casper.thenClick("div.talk-sharing__tools a.rate-button");

Second: Trying to explicitly show the modal for the button (specific to the page).

casper.then(function(){
    var name = this.evaluate(function(){
        var modal = document.querySelectorAll(".modal-wrapper > .modal")[0];
        modal.className += " modal--show";
        return modal.className;
    });
    this.echo("name: "+name); // null
});

casper.waitUntilVisible(".modal__head__title");

Third: Let jQuery do the work.

var casper = require('casper').create({
    remoteScripts: [ "http://code.jquery./jquery-1.11.1.min.js" ]
});

casper.waitForSelector(selector, function() {
    this.evaluate(function() {
        jQuery("a.rate-button").click();
    });
});

Encountered similar issue today. After executing this.click() on start page (which I verified that PhantomJS is executing - by hooking into resource.requested and resource.received events), current page remained on start page (and it should go to clicked link page).

Solution was to update to the latest PhantomJS (1.9.8 at this moment).

Please note that I had PhantomJS 1.9.7 when the above described problem occurred.

I'm trying to use CasperJS' click() to follow a link which generates a modal on the current screen. When I query the proper selector and click it in the browser console using document.querySelector().click() it works, but even when I casper.evaluate() this it doesn't work. I found someone who had a very similar problem, but his question remains unanswered, and I am experiencing almost identical problems. casperjs button click doesn't navigate to next page the code I'm currently using is

this.waitForSelector('div.talk-sharing__tools a.rate-button', function() {
    this.then(function() {
        this.evaluate(function() {
            document.querySelector('a.rate-button').click();
});

the page I'm trying to scrape is

I'm trying to use CasperJS' click() to follow a link which generates a modal on the current screen. When I query the proper selector and click it in the browser console using document.querySelector().click() it works, but even when I casper.evaluate() this it doesn't work. I found someone who had a very similar problem, but his question remains unanswered, and I am experiencing almost identical problems. casperjs button click doesn't navigate to next page the code I'm currently using is

this.waitForSelector('div.talk-sharing__tools a.rate-button', function() {
    this.then(function() {
        this.evaluate(function() {
            document.querySelector('a.rate-button').click();
});

the page I'm trying to scrape is http://www.ted./talks/uri_alon_why_truly_innovative_science_demands_a_leap_into_the_unknown

• javascript
• html
• phantomjs
• casperjs

Share Improve this question Follow edited Jun 13, 2014 at 15:10 ragingSloth asked Jun 13, 2014 at 4:13 ragingSlothragingSloth 1,10488 silver badges2323 bronze badges 3

• Did you try with casper.click('a.rate-button'); – Fanch Commented Jun 13, 2014 at 7:34
• 2 The untold truth about the casperjs and phantomjs tags is that nobody can help you solve the problem as long as you don't provide the link to the page in question. The pages you scrape behave very differently. That is why the SO format of posting does not really work for such question. You would have to post the plete page code here, which is really not encouraged. Also, your code should work as is, the page does something funny. We don't want to dream up something that might work. – Artjom B. Commented Jun 13, 2014 at 8:56
• @ArtjomB. I actually disagree with you on this one. This is a pretty general question, and while scraping questions aren't ideal for SO, there is a concrete usefulness here and I've been able to answer a few of these questions in the past. Also the page is now linked, but I assume that's a more recent development. – Slater Victoroff Commented Jun 13, 2014 at 15:27

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 6

It is really impossible to do those navigation steps with the phantomjs engine. It seems that the QtWebkit fork of phantom (version 1.9.7) is not up to the task anymore. Although your code works fine as is with slimerjs. You can fortably install slimerjs through npm:

npm install -g slimerjs

and run your script with

casperjs --engine=slimerjs script.js

---

I tried several things with phantomjs that did not work. I added casper.on listeners for remote.message and resource.error, but those didn't show any errors when running the script in phantom. logLevel: "debug" also didn't show anything.

First: Using a casperjs function.

casper.thenClick("div.talk-sharing__tools a.rate-button");

Second: Trying to explicitly show the modal for the button (specific to the page).

casper.then(function(){
    var name = this.evaluate(function(){
        var modal = document.querySelectorAll(".modal-wrapper > .modal")[0];
        modal.className += " modal--show";
        return modal.className;
    });
    this.echo("name: "+name); // null
});

casper.waitUntilVisible(".modal__head__title");

Third: Let jQuery do the work.

var casper = require('casper').create({
    remoteScripts: [ "http://code.jquery./jquery-1.11.1.min.js" ]
});

casper.waitForSelector(selector, function() {
    this.evaluate(function() {
        jQuery("a.rate-button").click();
    });
});

Encountered similar issue today. After executing this.click() on start page (which I verified that PhantomJS is executing - by hooking into resource.requested and resource.received events), current page remained on start page (and it should go to clicked link page).

Solution was to update to the latest PhantomJS (1.9.8 at this moment).

Please note that I had PhantomJS 1.9.7 when the above described problem occurred.

本文标签： javascriptCasperJS click() doesn39t load new htmlStack Overflow