Well, I learned something. There is a method to get a URL to an embedded resource, it's just something I'd never used. This makes sense though, Telerik uses a lot of embedded resources, you'll see a lot of WebResource.axd requests.
Although I did have to manually add a reference like below for each file within the AssemblyInfo.cs file which is kind of a pain when you have a lot of them. It still generates a URL, but it didn't seem to actually point to anything until I did this. I'm not sure if this is just from having to specify the mime type, or if it simply doesn't recognize the reference. I also had to close the project and re-open it after modifying the AssemblyInfo.cs file to get it to pick up the changes.
[assembly: System.Web.UI.WebResource("ProjectDefaultNamespace.FolderPath.FileName.fileextension", "img/png")]
If someone knows of a better way to get it to pick up those embedded references without the manual editing I'd be happy to know what it might be.