Erik Gnupsson was the first bishop of a North American diocese. He was named bishop of Garda, aka Gardar, a see with jurisdiction over Greenland and Vinland, in 1121, under Pope Paschal II. He was consecrated by Archbishop Adzar. The diocese went out of business when the Greenland colony did, but it is currently a titular diocese being used by Bishop Clark.
Man, I love the Internet.
Ignore the bit in the article where they think Vinland is Newport, Rhode Island. Although that does explain some Lovecraft stuff.