Quitting it from Activity Monitor works.
What is this happening?
Side note: It appears that PodcastContentService continues to fetch data via nsurlsessiond after the Podcasts app has been quit. These outgoing requests by nsurlsessiond—to hosts such as wnyc.org—were made apparent by Little Snitch permission prompts.
Update: PodcastContentService continues to run >5 minutes after quitting Podcasts, so perhaps it's a service that is always running, and maybe is unrelated to OP's question. I haven't checked whether PodcastContentService was running before I opened the Podcasts app.
Update two: I saw the Podcasts app in Activity Monitor a few minutes later. It disappeared within a few seconds though. It seems to have started on its own.