Wikipedia Image Scraper

  • Print
Details
Category: Powershell
Written by Winfred de Kreij
I've written a "Wikipedia Featured Pictures" scraper in PowerShell, using DotNetWikiBot. It downloads all 2500+ featured images (featured images are the images that are shown on the front page of Wikipedia).
 
$LocalFolderToSaveTo = "C:\DownloadLocation"
[System.Reflection.Assembly]::LoadFile("D:\Documents\WindowsPowerShell\test\dotnetwikibot\DotNetWikiBot.dll")
$EnWiki = new-object DotNetWikiBot.Site("http://en.wikipedia.org", "WikipediaUserName", "WikipediaPassword") 
$PageList = new-object DotNetWikiBot.PageList($enWiki);
$PageList.FillFromCategory("Featured pictures")
$ImageExts = ".jpg", ".png", ".jpeg"
Foreach($Page in $PageList.pages)
{
 if(($ImageExts -eq $Page.Title.substring($Page.Title.length - 4, 4)) -and (!($Page.Title.startswith("File talk"))) )
 {
 $DownloadTo = $Null
 $RenamedPage = $Page.Title.replace("File:", "")
 $RenamedPage = $RenamedPage.replace("`"", "") # "
 $RenamedPage = $RenamedPage.replace("`'", "")
 $DownloadTo = join-path $LocalFolderToSaveTo ($RenamedPage)
 $page.DownloadImage($DownloadTo)
 Start-Sleep 1 #Build in delay to prevent hammering and/or Ban. 
 }
}