You go to books.google.com and you select “full view”. Then pick your search term. You can download PDFs of the public domain books Google’s got up.
I don’t think much of their viewer format — or of PDFs, for that matter. But… any free book in a storm, ne?
Layamon’s Brut, in the original, with a prose translation at the bottom of each page.
The huge A Menology of England and Wales, Or, Brief Memorials of the Ancient British and English Saints. And the Cornish ones. Yep, it’s actually a Catholic book of UK saints! Awesome! Highly recommended for parents who want to name their kids something weird yet still have a valid saint’s name. Sidwell is a lovely modern girl’s name, ne? Or how about Avrildis? :)
People wishing to avoid treading on USCCB territory yet wishing to podcast encyclicals can find a good number of public domain translations, if you poke about in libraries or online. Here’s Pope Gregory XVI (with anti-Catholic annotations!). Watch out for mouth foam flying from the footnotes.
Seriously, though, podcasters can find plenty of apparently public domain material on the EWTN website or papalencyclicals.net, as well as vatican.va. (Vatican.va only goes back to Pope Leo XIII or so, at this point.) The problem is that none of these sites feel like telling you where they got their stuff and who translated it when. Sigh.