Skip to content

Extract Meta Tags in structured way - #28

Merged
codelucas merged 1 commit into
codelucas:masterfrom
voidfiles:master
Jan 27, 2014
Merged

Extract Meta Tags in structured way#28
codelucas merged 1 commit into
codelucas:masterfrom
voidfiles:master

Conversation

@voidfiles

Copy link
Copy Markdown
Contributor

There is a lot of meta data that poeple put in meta tags. Espcially
when you realize that Open Graph, and Twitter Cards are very popular.
Therefore I suggest that should be a way to extract the tags from
an article.

Here is an example:

import newspaper
article = newspaper.build_article('http://www.buzzfeed.com/tabirakhter/epic-literary-love-tattoos')
article.download()
article.parse()
print pprint.pformat(dict(article.meta_data))

{'article': {'publisher': 'https://www.facebook.com/BuzzFeed',
             'section': 'DIY',
             'tag': 'Tattoos'},
 'og': {'description': 'All is fair in love and ink.',
        'image': {'height': 625,
                  'url': 'http://s3-ak.buzzfeed.com/static/2014-01/enhanced/webdr06/22/16/enhanced-buzz-8304-1390426280-19.jpg',
                  'width': 625},
        'site_name': 'BuzzFeed',
        'title': '23 Epic Literary Love Tattoos',
        'type': 'article',
        'url': 'http://www.buzzfeed.com/tabirakhter/epic-literary-love-tattoos'},
 'twitter': {'app': {'id': {'googleplay': 'com.buzzfeed.android',
                            'ipad': 'id352969997',
                            'iphone': 'id352969997'},
                     'url': {'googleplay': 'http://www.buzzfeed.com/tabirakhter/epic-literary-love-tattoos'}},
             'card': 'summary_large_image',
             'creator': '@tabooradley',
             'description': 'All is fair in love and ink.',
             'image': 'http://s3-ak.buzzfeed.com/static/2014-01/campaign_images/webdr06/26/14/23-epic-literary-love-tattoos-1-1410-1390763440-3_big.jpg',
             'site': '@buzzfeed',
             'title': '23 Epic Literary Love Tattoos',
             'url': 'http://www.buzzfeed.com/tabirakhter/epic-literary-love-tattoos'}}

There is a lot of meta data that poeple put in meta tags. Espcially
when you realize that Open Graph, and Twitter Cards are very popular.
Therefore I suggest that should be a way to extract the tags from
an article.

Here is an example:

```sh
import newspaper
article = newspaper.build_article('http://www.buzzfeed.com/tabirakhter/epic-literary-love-tattoos')
article.download()
article.parse()
print pprint.pformat(dict(article.meta_data))

{'article': {'publisher': 'https://www.facebook.com/BuzzFeed',
             'section': 'DIY',
             'tag': 'Tattoos'},
 'og': {'description': 'All is fair in love and ink.',
        'image': {'height': 625,
                  'url': 'http://s3-ak.buzzfeed.com/static/2014-01/enhanced/webdr06/22/16/enhanced-buzz-8304-1390426280-19.jpg',
                  'width': 625},
        'site_name': 'BuzzFeed',
        'title': '23 Epic Literary Love Tattoos',
        'type': 'article',
        'url': 'http://www.buzzfeed.com/tabirakhter/epic-literary-love-tattoos'},
 'twitter': {'app': {'id': {'googleplay': 'com.buzzfeed.android',
                            'ipad': 'id352969997',
                            'iphone': 'id352969997'},
                     'url': {'googleplay': 'http://www.buzzfeed.com/tabirakhter/epic-literary-love-tattoos'}},
             'card': 'summary_large_image',
             'creator': '@tabooradley',
             'description': 'All is fair in love and ink.',
             'image': 'http://s3-ak.buzzfeed.com/static/2014-01/campaign_images/webdr06/26/14/23-epic-literary-love-tattoos-1-1410-1390763440-3_big.jpg',
             'site': '@buzzfeed',
             'title': '23 Epic Literary Love Tattoos',
             'url': 'http://www.buzzfeed.com/tabirakhter/epic-literary-love-tattoos'}}
```
@codelucas

Copy link
Copy Markdown
Owner

This is very cool, thanks! This will most likely be merged by tonight after I look it over.

codelucas added a commit that referenced this pull request Jan 27, 2014
Extract Meta Tags in structured way
@codelucas
codelucas merged commit f0d5fae into codelucas:master Jan 27, 2014
@codelucas

Copy link
Copy Markdown
Owner

This will be in the 0.0.7 release + the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants