javascript - Getting the "text" from a script element using Selenium - Stack Overflow

admin管理员组

文章数量:1022623

I'm working on a web scraper, and the site I'm scraping has a script element on the page that looks like this:

<script type="text/javascript">
                        jQuery(window).load(function($) {
                        Morris.Line({
                          element: 'mpr-graph',
                          data: [
                            {'date': '25-04-2017','y':'1.05'},
                            {'date': '25-04-2017','y':'1.50'},
                            ...

What I want:

I want to get to the data property of the object passed to Morris.Line so that I can turn the data into something usable.

I've managed to select the correct element as a Selenium WebElement using the surrounding div's id and the tag name script, but now I'm stuck.

Is there a way to get the text of a script element using Selenium? The text property is empty since it only returns the text shown on the page for a given element.

What I've tried:

Since I was able to get the text in the browser console by grabbing the text property of the element, I tried using execute_script.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script');")

This returns a WebElement, so we're back to square 1, but at least we know it's working so we can move on to:

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').text;")

I thought this might work since it works in the browser console, but Selenium returns nothing.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').innerHTML;")

As above.

I'm working on a web scraper, and the site I'm scraping has a script element on the page that looks like this:

<script type="text/javascript">
                        jQuery(window).load(function($) {
                        Morris.Line({
                          element: 'mpr-graph',
                          data: [
                            {'date': '25-04-2017','y':'1.05'},
                            {'date': '25-04-2017','y':'1.50'},
                            ...

What I want:

I want to get to the data property of the object passed to Morris.Line so that I can turn the data into something usable.

I've managed to select the correct element as a Selenium WebElement using the surrounding div's id and the tag name script, but now I'm stuck.

Is there a way to get the text of a script element using Selenium? The text property is empty since it only returns the text shown on the page for a given element.

What I've tried:

Since I was able to get the text in the browser console by grabbing the text property of the element, I tried using execute_script.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script');")

This returns a WebElement, so we're back to square 1, but at least we know it's working so we can move on to:

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').text;")

I thought this might work since it works in the browser console, but Selenium returns nothing.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').innerHTML;")

As above.

• javascript
• python
• selenium

Share Improve this question Follow edited Jun 1, 2018 at 1:10 Greg K asked Nov 16, 2017 at 1:12 Greg KGreg K 2,00011 gold badge1717 silver badges3333 bronze badges

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 5

You should be able to use an XPath to find the SCRIPT tag based on its contents

script_text = driver.find_element_by_xpath("//script[contains(.,'mpr-graph')]").text

If for some reason that isn't specific enough (more than one SCRIPT tag contains "mpr-graph") then you can adjust it to whatever text is unique among SCRIPT tags.

I'm working on a web scraper, and the site I'm scraping has a script element on the page that looks like this:

<script type="text/javascript">
                        jQuery(window).load(function($) {
                        Morris.Line({
                          element: 'mpr-graph',
                          data: [
                            {'date': '25-04-2017','y':'1.05'},
                            {'date': '25-04-2017','y':'1.50'},
                            ...

What I want:

I want to get to the data property of the object passed to Morris.Line so that I can turn the data into something usable.

I've managed to select the correct element as a Selenium WebElement using the surrounding div's id and the tag name script, but now I'm stuck.

Is there a way to get the text of a script element using Selenium? The text property is empty since it only returns the text shown on the page for a given element.

What I've tried:

Since I was able to get the text in the browser console by grabbing the text property of the element, I tried using execute_script.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script');")

This returns a WebElement, so we're back to square 1, but at least we know it's working so we can move on to:

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').text;")

I thought this might work since it works in the browser console, but Selenium returns nothing.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').innerHTML;")

As above.

I'm working on a web scraper, and the site I'm scraping has a script element on the page that looks like this:

<script type="text/javascript">
                        jQuery(window).load(function($) {
                        Morris.Line({
                          element: 'mpr-graph',
                          data: [
                            {'date': '25-04-2017','y':'1.05'},
                            {'date': '25-04-2017','y':'1.50'},
                            ...

What I want:

I want to get to the data property of the object passed to Morris.Line so that I can turn the data into something usable.

I've managed to select the correct element as a Selenium WebElement using the surrounding div's id and the tag name script, but now I'm stuck.

Is there a way to get the text of a script element using Selenium? The text property is empty since it only returns the text shown on the page for a given element.

What I've tried:

Since I was able to get the text in the browser console by grabbing the text property of the element, I tried using execute_script.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script');")

This returns a WebElement, so we're back to square 1, but at least we know it's working so we can move on to:

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').text;")

I thought this might work since it works in the browser console, but Selenium returns nothing.

script_text = driver.execute_script("return document.getElementById('avg').getElementsByTagName('script').innerHTML;")

As above.

• javascript
• python
• selenium

Share Improve this question Follow edited Jun 1, 2018 at 1:10 Greg K asked Nov 16, 2017 at 1:12 Greg KGreg K 2,00011 gold badge1717 silver badges3333 bronze badges

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 5

You should be able to use an XPath to find the SCRIPT tag based on its contents

script_text = driver.find_element_by_xpath("//script[contains(.,'mpr-graph')]").text

If for some reason that isn't specific enough (more than one SCRIPT tag contains "mpr-graph") then you can adjust it to whatever text is unique among SCRIPT tags.

本文标签： javascriptGetting the quottextquot from a script element using SeleniumStack Overflow