I am using this code in Try Puppeteer:

const browser = await puppeteer.launch();

const page = await browser.newPage();
await page.goto('');

const linkHandlers = await page.$x("//li[contains(text(), '1D')]");

if (linkHandlers.length > 0) {
  await linkHandlers[0].click();
} else {
  throw new Error("Link not found");
}

await page.$eval('input[name="fieldInput"]', el => el.value = '1');

console.log(await page.content())
// const text = page.evaluate(() => document.querySelector('rect'))
// text.then((r) => {console.log(r[0])})

await page.screenshot({path: 'screenshot.png'});

await browser.close();

The same page loaded in the Chrome browser shows the bars indicating price movements, but in the screenshot obtained in Puppeteer the chart is empty.

Also page.content() gives an html that is pletely different from the one I see when inspecting the element in Chrome.

I am using this code in Try Puppeteer:

const browser = await puppeteer.launch();

const page = await browser.newPage();
await page.goto('https://www.barchart./futures/quotes/ESM19/interactive-chart/fullscreen');

const linkHandlers = await page.$x("//li[contains(text(), '1D')]");

if (linkHandlers.length > 0) {
  await linkHandlers[0].click();
} else {
  throw new Error("Link not found");
}

await page.$eval('input[name="fieldInput"]', el => el.value = '1');

console.log(await page.content())
// const text = page.evaluate(() => document.querySelector('rect'))
// text.then((r) => {console.log(r[0])})

await page.screenshot({path: 'screenshot.png'});

await browser.close();

The same page loaded in the Chrome browser shows the bars indicating price movements, but in the screenshot obtained in Puppeteer the chart is empty.

Also page.content() gives an html that is pletely different from the one I see when inspecting the element in Chrome.

• javascript
• node.js
• web-scraping
• puppeteer

Share Improve this question Follow edited Apr 7, 2019 at 16:33 Thomas Dondorf 25.4k66 gold badges9696 silver badges112112 bronze badges asked Apr 7, 2019 at 14:49 user3755529user3755529 1,20611 gold badge1515 silver badges3737 bronze badges 1

• Can you share your screenshot.png? Also what happens if you launch with {headless:false}? – Md. Abu Taher Commented Apr 7, 2019 at 15:38

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 6

Problem

You are not waiting for the request to resolve when the input is changed. As a change will trigger a request, you should use page.waitForResponse to wait until the data is loaded.

In addition, this is an Angular application, which does not seem to like it if you simply change the value of the field via el.value = '1'. Instead you need to try to behave more like a human (and hit backspace and type the input value).

Solution

First, you get the element handle (input[name="fieldInput") from the document. Then, you focus the element, remove the value inside by pressing backspace. After that you type the desired input value.

The input field now has the correct value, now we need to trigger the blur event by calling blur() on the element. In parallel, we wait for the request to the server to finish. After the request finishes, we should give the page a few milliseconds to render the data.

All together, the resulting code looks like this:

const browser = await puppeteer.launch();

const page = await browser.newPage();
await page.goto('https://www.barchart./futures/quotes/ESM19/interactive-chart/fullscreen');

// wait until the element appears
const linkHandler = await page.waitForXPath("//li[contains(text(), '1D')]");
await linkHandler.click();

// get the input field, focus it, remove what's inside, then type the value
const elementHandle = await page.$('input[name="fieldInput"]');
await elementHandle.focus();
await elementHandle.press('Backspace');
await elementHandle.type('1');

// trigger the blur event and wait for the response from the server
await Promise.all([
    page.waitForResponse(response => response.url().includes('https://www.barchart./proxies/timeseries/queryminutes.ashx')),
    page.evaluate(el => el.blur(), elementHandle)
]);

// give the page a few milliseconds to render the diagram
await page.waitFor(100);

await page.screenshot({path: 'screenshot.png'});
await browser.close();

---

Code improvement

I also removed the page.$x function and replaced it with the page.waitForXPath function. This makes sure that your scripts waits until the page is loaded and the element you want to click is available before the script continues.