Handle lxml raising ValueError on node.itertext()

[yprez]

Fixes #143

[yprez]

Parsing now works:

from newspaper import Article
article = Article(url='http://www.bostonglobe.com/news/world/2015/05/13/obama-discusses-iran-with-saudi-leaders/YQVleMtSURxsYgxUOznzFO/story.html')
article.download()
article.parse()

article.text
#[Out]# u'US President Barack Obama shook hands with Saudi Crown Prince Mohammed
#[bin Nayef as Foreign Minister Adel al-Jubeir (L) looked on
#[Wednesday.\n\nWASHINGTON (AP) \u2014 President Barack Obama welcomed Saudi
#[Arabian leaders to the White House for discussions on his overtures to
#[Iran.\n\nObama met in the Oval Office with the Saudi crown prince and deputy
#[crown prince on Wednesday.\n\nContinue reading below\n\nObama says the U.S.
#[and the Saudis are building on a strong relationship \u2018\u2018during a very
#[challenging time.\u2019\u2019\n\nThe president says they will discuss the
#[situation in Yemen as well as broader issues for the gulf region.\n\nThe Saudi
#[officials are representing the kingdom in Washington after King Salman
#[announced that he no longer would make the trip.\n\nLater Wednesday, leaders
#[from other Gulf nations with join Obama and the Saudis for a dinner at the
#[White House. They\u2019ll head to Camp David Thursday for broader talks on
#[Iran and regional security.'

The same fix should work for the Python3 branch.

[codelucas]

I think we should at least print out warnings + fail loudly if this happens. Maybe notify the url that failed so the developer can triage further? The ideal is to debug urls that don't work so the library can slowly extract more successfully.

[codelucas]

See my inline comment, otherwise thanks for this!

[yprez]

@codelucas printing a warning makes sense here, but failing to parse the entire article because of one bad xml node is a bit drastic IMO.

Would you accept this fix if I add a print() or perhaps use warnings.warn() when this happens?
Any other way to push this forward?

[codelucas]

@yprez Good point. Hmm, how about:

import traceback
traceback.print_exc()

A traceback will be a big/loud and also imformative warning that something is deeply wrong but won't kill the entire program. After that, I'd feel comfortable merging this.

Ideally we'd also output the URL that it failed on so the developer knows what went wrong and where to debug

[yprez]

@codelucas Great. How about using logging, instead of printing to stdout?

Then we can use something like: log.warning('error msg', exc_info=True), which will log the traceback as well.
What do you think?

[yprez]

Also, I can't find a way to reach the article instance from OutputFormatter, so I can't append the URL when needed. And moving it to a higher level won't do the trick, because then the other nodes will be skipped, not just the broken one.

I'll make an intermediate commit with what I've got for now...

[codelucas]

Thanks!