Multi-threading article downloads not working

[projectweekend]

I am following the example at http://newspaper.readthedocs.org/en/latest/user_guide/advanced.html#advanced. After calling news_pool.join(), I then attempt to call parse() on an article belonging to a paper from the pool. It fails with the ArticleException: "You must download() an article before parsing it!".

[codelucas]

Thanks for submitting this issue. I'll look into it right now.

If you don't mind can you provide the source url(s) you are attempting to multithread from?
Also any config inputs if you have any.

[projectweekend]

Source urls are: http://www.theverge.com & http://arstechnica.com/. No special configs.

[codelucas]

Here is my attempt of replicating what happened. It worked for me, can you check which version of newspaper you installed via pip to make sure its the 0.0.6 version? I'm aware that this actually fails in the 0.0.5 version!

>>> import newspaper
>>> from newspaper import news_pool

>>> verge = newspaper.build('http://www.theverge.com')
>>> verge.size()
338

>>> ars = newspaper.build('http://arstechnica.com/')
>>> ars.size()
315

>>> papers = [verge, ars]
>>> news_pool.set(papers, threads_per_source=2) 
>>> news_pool.join()

>>> len(verge.articles[0].html)
94207 

# it worked, for the first article at least, now let's test them all
>>> verge.articles[0].parse()
>>> verge.articles[0].text[:50]
u'Richie Zyontz likes to talk about stories, about s'

>>> verge.parse_articles() # analogous to  for a in verge.articles: a.parse()

>>> for a in verge.articles:
...    print len(a.text)
4160  
992
1292
4686
321
321
321
... And so on, all text extractions via .parse() worked.

Test your version:

>>> from newspaper import version
>>> version.__version__
'0.0.6'

[projectweekend]

Indeed. I was on 0.0.5. Sorry about the false alarm! I really appreciate your time pointing me in the right direction. I'm really digging newspaper so far!

[codelucas]

No worries, glad you like it!

[Shubhashri1310]

Hi neither I am getting any output nor am I able to print anything. What do I do? Please Help

import newspaper
from newspaper import Article
from newspaper import news_pool

ct=0
l=[]
for i in parent_urls:
ct+=1
if cnt>11:
break
else:
p=newspaper.build(i)
a=Article(i)
l.append(p)
news_pool.set(l,threads_per_source=2)
news_pool.join()
a.parse()
print(a.text)

---

print(newspaper.version)-0.2.8

Please help @codelucas