Bumper Co-Founder Dan Misener on the Death of the Download

Although the transcription is largely accurate, in some cases it may be incomplete or inaccurate due to inaudible passages or transcription software errors.

Jeff Umbro: Today on Podcast Perspectives, I am chatting with Dan Misener, co-founder of Bumper. 

What is one myth about podcast data or measurement that you wish would go away?

Dan Misener: That downloads represent people.

Jeff Umbro: You might know Dan from his time at CBC and Pacific Content, but recently he's been challenging everyone from indie creators to big podcast networks to move past the download metric and start tracking who's really listening for how long and where the value truly lies in audience engagement.

Together Dan and I are going to dig into the biggest changes in podcast data, how shows can prove their impact and why building engaged audience has never been more essential. 

Welcome to the show, Dan.

Dan Misener: I'm glad to be here. I'm a fan of the show. I am a sometimes listener, sometimes viewer of the show, depending on who's on, and it's a thrill to be here.

Jeff Umbro: You wanna give us like the thousand foot view of, of your audio career, which brought you to today?

Dan Misener: Yeah, I come from radio. I came up in campus community radio at the Mighty 50 Watt, CKDU in my hometown of Halifax. So I just fell in love with campus community radio, where as a volunteer programmer, you could walk through the doors and people would teach you. How to make radio, how to do interviews, how to cut tape.

We were cutting tape. Then how to layer different elements together to tell stories on the radio, and I loved this idea of independent DIY media. If you wanted to make something that wasn't on the radio, you could do that. If you were interested in something that nobody else was talking about, you could make a show about that.

So much of what I loved about campus community radio and that spirit is, I think, very alive and well in podcasting today, particularly independent podcasting. And so came up in campus community radio, had the first part of my career in public radio, working for the CBC here in Canada. And then in the mid 2010s, my friend Steve Pratt called me up and said, we're gonna start this company called Pacific Content and we're gonna make podcasts with brands; wanna join us? And I left my job in public radio and went and worked in brand land for about six years prior to founding Bumper, which is where I work today.

Jeff Umbro: I'm curious how you learned what you do know today about data and computer science and that kind of thing. You understand how this works in a way that not many people do, even in the world of engineering and data. So I'm just curious where that came from?

Dan Misener: I think a lot of my interest in podcast measurement and podcast data stems from my experience as a producer, right? I worked at the CBC for 10 years. I made a show all day long. It was available as a podcast, but it was mostly heard, at least as far as I understand, by people over the airwaves coming from broadcast towers across the country.

We made a national show and I was always frustrated by the lack of a feedback loop. We only ever heard from the people who wrote in. We only ever heard from the people who bothered to send us an email or engage with us on social. And I remember making shows at the CBC and knowing that they were heard by an awful lot of people that I never really got to hear from.

And I remember feeling very frustrated by that. And I remember how crude so much of the measurement was at the time, right? I worked on a weekly show and we would get a once a month report from the audience insight team that showed how many downloads we got, and our weekly show had a number and then the daily show that published three episodes a day, five days a week, also had a number and they were right next to each other. And I remember thinking, that feels so unfair. Why is the weekly show and the daily show, a show that publishes once a week versus a show that publishes 15 episodes a week, why are they being held to the same standard? Why are we using the same yardstick for those things? Why is this feedback loop of wanting to hear from an audience and wanting to know how an audience uses a media product that you are involved in the production of? I wanted more of that data, so I think a lot of my interest in this stems from my frustration from wanting to make better episodes, wanting to make better shows, and feeling like I didn't have access to the kinds of information that would help me do that.

Which is why in podcasting, the fact that we have such actionable, such rich, such useful data available to us I think is a wonderful thing. And a big part of our work at Bumper is helping teams better understand what the audience is telling them. Not when they write in, not when they leave a comment on YouTube, but what the audience is telling them when they vote with their play buttons because there's an awful lot of really useful signal in there.

Jeff Umbro: How did you learn that though? Did you take a class? Just because, honestly, I bet you 80% of podcasting couldn't tell you what an API is.

Dan Misener: I've long been interested in teaching myself to code, and in the year 2012, my wife and I moved to France and I took a year off and did French things and taught myself Python with some online courses and have continued to be just knowledgeable enough to be dangerous in the Python programming language ever since.

Jeff Umbro: So today you are a co-founder at Bumper. How do you describe Bumper? If somebody asks you at, at a bar.

Dan Misener: We talk about Bumper as a podcast data and growth company, and practically speaking, what that means is we use data to help our clients become more successful in podcasting. Usually that means growing something, growing revenue, growing audience size, and often those two things go together.

Jeff Umbro: You just hit on something that I think is kind of essential to what you do, what we do at Podglomerate and what a lot of people, I think should be doing more of looking at the data behind a show to discover listenership or potential revenue opportunities, but in service of a business's goals for growth. a question I, I often ask people when they knock on our door, it, it's ultimately like, what do you hope to get out of this? There's a lot of different answers to that question, but having that answer is essential for the most part. Correct me if I'm wrong, but you are not working with people these days who are, are terribly interested in advertising as like a revenue stream.

Is that correct?

Dan Misener: We work with a fairly large number of ad supported shows and we work with a number of shows and teams that make shows that don't carry any advertising whatsoever. And a lot of people somewhere in the middle, right? So some of our clients are large, well-known media companies whose names you would know, whose names are listed on our website and a hundred percent they sell ads and that is their primary mode of generating revenue from podcasts.

We also work with some nonprofits and not-for-profits who don't run ads in their shows, aside from house inventory for their own initiatives or their own other shows on the same network. And then we work with a really interesting kind of spot in the middle. You know, companies like HubSpot or companies like BiggerPockets where there's ads in the shows and sometimes those are ads for outside products and services, and sometimes those are ads for, in the case of HubSpot, HubSpot's own product or in the case of BiggerPockets, BiggerPockets own downstream businesses. Right? And so I tend to think of this as a spectrum where you can be kind of all in on ads and ad revenue. You can be out of that or you can be anywhere along.

And that could be sort of a piece or the entirety of your business model. But one of the great things that I love about our client base right now is everybody's got a slightly different model and we get to sort of see what's working for different folks along that spectrum.

Jeff Umbro: One of the things I run into fairly often is there are people who will come to us looking for production growth or anything in between who don't necessarily understand or know how to measure what it, they could constitute success. When somebody comes to you that just, they have a clear vision of what they want, but they don't necessarily know how to like track success there. What are some of the things that you ask them about in terms of like how they can solve that problem?

Dan Misener: I think for us, a big piece of this comes down to what you were talking about earlier. What are your goals or what is this podcast supposed to do for you? And there's a bunch of questions. I call them podcast strategy questions that I've adapted from some prior work with a company called Postlight, sort of a digital agency out of New York.

These questions about like, what are we making? Who are we making it for? Why is it good for the audience we're making it for? Why is it good for us and who's gonna make it? If you've got good answers to those five questions, especially the why is it good for the audience and why is it good for us as a company, as an organization, that is a really great place to start from.

And if you do not have good, clear answers to those questions, particularly. Why is it good for listeners? Why is it good for the audience? Why is it good for viewers, and why is it good for us as the company, if you don't have clarity around that stuff, that feels like a recipe for a mediocre show that it will be difficult to grow or maybe even impossible to grow.

And I think one of the biggest challenges that I see teams make is conflating. Why is it good for listeners or viewers and why is it good for us as the publisher or producer? If those are the same answer for both sides of the, the equation that tends not to lead to success, at least in my experience.

Jeff Umbro: Why don't you walk us through what is the Bumper dashboard, and you mentioned there are some signals that it helps you guys, you know, address in the process. What are some of those signals?

Dan Misener: We built the Bumper dashboard because we were frustrated having to check four or five or six or seven separate data sources regularly in order to triangulate the truth of the shows that we work with on a good day. Without something like the Bumper dashboard, you might wake up and check your audio hosting provider to get your download number, and then you might check Spotify for creators to see how your Spotify cohort of listeners or viewers is engaging with your show.

You might then check Apple Podcasts Connect. You might then check YouTube Studio. You might then go on to check something like Podscribe or Magellan for marketing effectiveness or any of those kinds of measures, you might have a login to something like Flightpath to sort of do revenue optimization and forecasting, right?

It was not uncommon for our team and for many of our clients to have dashboard overwhelm, right? 5, 6, 7 dashboards just to get the pulse of a show. And so we were feeling really, really frustrated by this. So we started to build some internal tools to help us aggregate and harmonize the data so that we could actually turn it into the kind of action that we want, whether that's editorial action or marketing action.

And we should get into the important distinction between those two things, and so we were frustrated because the truth of your show is out there. It's just scattered across several different dashboards. We wanted to bring it all under one roof. We realized as we were building this set of tools internally that there was a whole lot of noise, particularly noise in the form of the download and how the download isn't a great proxy for people, time spent or regularity or habitual listening. So we've started to design the Bumper dashboard around these core things that we know podcasters and podcast networks are really after people. Did we reach real human beings who with intent spent time with our show? So like people, numbers, what we call playback numbers, which is Jeff hit play on five episodes of his favorite show last week.

Well, he's one person, but he hit play five times. He might want to count five times. Right? So looking at people, looking at playback and then looking at time spent, an hour on YouTube, an hour on Apple Podcasts, an hour on Spotify. I think I can add those three hours together and call them three hours. And so we started to really identify these three kind of categories people play back in time spent from, which all other good things flow.

And if you can reach more people, get them to hit play more often or more times per week or more times per month and get them to spend more time with you. All the other stuff figures itself out. We were seeing so many people being misled by the download number that they saw on their hosting provider, which tells you nothing about people, tells you nothing about playback, and tells you nothing about time spent.

And so we, we really zoomed in on this idea of let's find some better proxies for those things and use those to make editorial decisions and marketing decisions.

Jeff Umbro: One of the big problems that this brings up is we are an industry currently that is built around the download and, and I know a big conversation you have is the death of the download. Explain why exactly a download is not something that is kind of an accountable figure for what people are hoping to do. And then do you have any broad vision or manifesto for the death of the download and what replaces it in the future?

Dan Misener: We're actively trying to figure out what replaces the download, and I don't think it's gonna be a light switch where we flip it off and the entire industry moves to something new. This is gonna be gradual. It's gonna take time, and it's gonna take a tremendous amount of education on both the publisher and the advertiser side.

Jeff Umbro: That said, if a year ago when you started beating this drum, I remember thinking, Dan's crazy. This is never gonna pick up. And now today is kind of like just an accepted fact at this point in, in some, in some circles.

Dan Misener: I think it's really worth teasing apart the different uses people have for the download figure. I'm gonna break it down into two main use cases. One, people use the download as a signal of health. Is my show healthy? Am I growing? Is it successful? Am I reaching more people this week than last week? This season than last season?

I think the download has historically been used as a health signal. I think there's an entirely separate use, which is the download as currency for transacting. Among ad supported shows, and those are two very different use cases, but the download is kind of a faulty metric, I would argue, a very deeply broken metric for both.

And we can start with the download as a health metric. Used to be downloads were an okay proxy. They were at least what we had and what we could as an industry agree on. There were guidelines from the IAB and more or less, platform to platform. Service to service. A download is a download, is a download, and I think it worked that way for a lot of years, and then maybe two years ago.

A lot of people started to see the impact of Apple's changes to automatic downloads with iOS17. People started questioning, wait a minute, if half my downloads went away, but my audience is still the same size, how useful is the download as a proxy for audience?

Jeff Umbro: Just to give like a very brief overview of what happened. Essentially Apple changed the way in which they, they gave auto downloads to listeners of shows and a lot of folks, especially legacy publications, had a very significant drop off in total downloads on their show.

Dan Misener: iOS17 and automatic download changes. Big haircut for a lot of shows. I think that caused a lot of people to question the download as a proxy for audience size. Then over the past couple of years, we've seen Spotify with what they call passthrough and with their strong push into video podcasting and encouraging publishers to upload video directly to them, thereby cutting out downstream downloads.

As more people opted into, and as more publishers opted into Spotify video or migrated to hosting platforms where pass through was disabled, well suddenly a whole lot of Spotify downloads that used to happen stopped happening. This continues to happen as more people sign up and opt into things like the Spotify partner program or uploading Spotify video and then of course YouTube, which, in the context of YouTube main, has never generated and will likely never generate a download for anybody. And so yeah, three years ago we were talking about the death of the download and some people said, nice idea.

Jeff Umbro: I was one.

Dan Misener: But we're gonna stick. We're gonna stick with the download for now. And then iOS17, and then Spotify Video, and then the rise of YouTube. Plus, along the way, the IAB continues to change the guidelines for what counts as a download. It becomes ever stricter, ever more exclusive, more downloads that used to count don't count anymore, as the guidelines get tighter and tighter and tighter. So you take those three or four or five different things that have just happened over the last 24 months, and a whole lot more people are I think waking up to the idea that the download was never an especially good proxy for audience size, audience composition, or audience health or show health, and I think folks are starting to wake up to that and look at some of the other metrics, like I mentioned, people playback and time. So that's the sort of download as health signal side of it.

Jeff Umbro: In a lot of ways what people really care about with their podcast is just engagement metrics, and those are the three things, three of the things that you. Chatting about right now, it's the same as on social media. If somebody posts a video and it gets five views and zero likes or comments if you're in that game to try and gain influence via scale and engagement with listeners, this is the same kind of idea with a podcast.

If you're getting a million downloads but you're not getting a lot of emails or comments or verified listeners, then it's probably not doing much. So, okay, so that's downloads as a health metric, and the next is downloads as a transactional metric.

Dan Misener: Downloads as a currency, right? I mean, and, and I do think of downloads as a currency and they operate much like a currency. And you can in fact exchange downloads for money and you can turn money into downloads, right? It really is currency like, and that is so deeply broken. It is so incredibly broken. I was listening to an episode that you did with Adam from ADOPTER. You were talking about essentially what advertisers buy when they buy a lot of podcast advertising. They're buying access to influence. The download as a measure of a file being transferred from a hosting provider to somebody's phone is a terrible way to measure influence.

Jeff Umbro: Well, it, it is and something we talked about in that episode with Adam McNeil it in a lot of ways is still a better way to. Purchase influence than other mediums. You know, if you're buying like a television ad or a.

Dan Misener: Yeah.

Jeff Umbro: You're not necessarily getting any kind of traction or metrics that give you any better data. But to your point, I do 100% agree with you. As you know, there are ways in which to do that smarter and, and make sure that you're purchasing ads that are actually, hopefully going to be heard and to, to iterate on that process. But yeah, you're often buying just something, no one's ever gonna listen to.

Dan Misener: I think there's a tremendous amount of misleading language and misleading information in the ecosystem that does not help us as an industry. I think there's a reason why so much podcast advertising is direct response. It's related to what you can measure and how you can prove return on it. And I would send people to that episode with Adam from ADOPTER to explain exactly why that is, right?

So it's really a great episode from this show. A download tells you nothing about whether somebody played the file that landed on their phone, and it tells you nothing about whether anybody actually made it to the ad marker or made it through the ad break. It's really, it doesn't measure what I think advertisers actually want.

And what I find fascinating about this is that this story has played out in other forms of digital advertising many times. Maybe most recently with online display ads, right? You can read about how over the last 20 years, the entire online display ad world went through what I imagine is gonna happen with podcasting in the next couple of years.

They went from transacting on ads served or ads delivered to a world of what they call viewability metrics. I'm not gonna pay for ads that were loaded onto a webpage, but nobody ever actually scrolled into view. Just because the ad was served doesn't mean that somebody saw it. And I think there's a very similar thing going on with a tremendous amount of podcast ads where they're inserted into downloads that sit unlistened on people's devices or get inserted into episodes and nobody actually makes it to the moment in the episode where the ad actually appears. I think there's a huge delta, and we've proven this out a number of times, using consumption data from the platforms and download and impression data from the ad servers and the hosting providers.

Jeff Umbro: So I think that's a really interesting problem that you've identified, and it's something that is a hundred percent true. Where this gets tricky is, and you, you wrote about this recently in your beef paste blog post. The issue is that the most likely scenario in my mind is if there is some way to purchase verified listeners or verified impressions for your advertising unit, even if you charge something that has many multiples of what you would charge for a verified download, you still will not necessarily make the same amount of money through that process.

And you're gonna tell me that you, that there's a way in which you still can. And that's the thing that I don't know if I'm on the same page with you about. So as opposed to me rambling, let, let's hear from you as to why you think that I'm incorrect.

Dan Misener: So in an ideal world as an advertiser and Bumper buys a lot of podcast ads, we run tune-in campaigns for our clients. In an ideal world, I would only buy the ads that I knew were gonna get hurt. Sadly, I don't think that that exists or is likely to exist soon. I don't think this idea of what we're calling a verified impression works as a targeting mechanism.

Wouldn't it be so nice if I could only pay for the ads that I know in advance are somehow going to be heard by a human being? I think that would be lovely. That's not the case. Moreover there's a whole lot of downloads and impressions that are, are inserted into downloads or ads served or ads delivered to use the IAB’s language.

There's a whole lot of that stuff that gets inserted into downloads that we have very, very, very low confidence, actually got played auto loading audio players on websites. We see that much more frequently than we should. Mystery meat downloads in the form of Apple Core Media, or Dalvik, or unknown.

Jeff Umbro: Dalvik is a fun one.

Dan Misener: If you log into your hosting provider and you see a whole lot of downloads attributed to Google Chrome I don't have a high degree of confidence that anybody heard those ads or made it to the, to the ad marker. What we've identified is that there are certain platforms, and I will name Apple, Spotify, and YouTube right now as, I would call them the big three, that also happened to give publishers data back about minute by minute, second by second, where drop off happens the number of people, basically a histogram view of how many people were present during various moments of the episode.

Between Apple, Spotify, and YouTube, you can calculate a verified impression number. That is an impression that was inserted into an episode or baked into an episode that has a much higher likelihood of actually being heard by people who are present during the playback of that than an Apple Core Media or a Dalvik download, right?

And we're suggesting that verified impressions should cost more. And could cost more if publishers were willing to sell those and package those up separately. But what about all my pocket casts downloads? But what about all my overcast downloads? But what about all my cast box downloads? That may have gone up very considerably for some reason I'm not aware of.

Right? What about all these downloads? I don't get telemetry from many of those apps. So what am I supposed to do? Just write those off? I don't think what I'm suggesting is that you write those off. The beauty of what we're suggesting with verified impressions is that you get to sell both. You can sell classic impressions, the stuff people are used to buying and selling today, you can sell those and maybe you can still sell those at the same rates that you've been selling them, and you package up all your cast box and Apple Core Media and Overcast and Pocket Casts.

You just package those up and we say. Maybe somebody heard the ads in this, maybe they didn't. We don't know. But that's the status quo. And then there's this whole other thing, I'm gonna call it a premium product. What you're buying is ads inserted into episodes where we can prove there was a person present during the playback of that episode.

And during the exact ad break, you mentioned the beef paste analogy I've been using. This is selling filet mignon and selling hot dogs. You get to sell both, but you're not charging the same price for filet mignon and hot dogs.

Jeff Umbro: This dream world is a great one and I can't wait to see it. The issue in my mind is twofold. In addition to this being a product that may be. Too expensive to be appetizing, even if it is filet mignon. I think that there is a lot more administrative overhead that would go into doing something like this.

And we've talked about this. You're essentially selling a baked in ad and then double checking all the numbers and, and so until somebody, which I suspect and hope maybe you builds some new platform to automate this as much as possible. I do think that there are other considerations to be taken into account when purchasing or trying to run these kinds of swaps or buys. The second thing that I really want to hone in on in other industries in order to combat selling a zillion impressions on a display ad, are platforms that are doing what are called pay per click ads. So they're essentially charging based on engagement and they're doing it on an auction-based system. The more popular keywords you have to spend more in order to get those, those keywords, think that that is in some ways the future of what we may see when it comes to podcast advertising. The issue is that you can't really do that with an RSS feed because there's just not a lot of ways for the technology to actually track that, except for Spotify who is starting to build their own system outside of what solutions you are posing.

Are there other people in the industry that are trying to solve similar problems, and how are they doing?

Dan Misener: We've been talking about verified impressions.

Jeff Umbro: Yeah.

Dan Misener: Which simplest version is you don't pay for ads that people skip. You don't pay for ads that people don't hear. You don't pay for ads inserted into downloads that never get played, right? That's that's a verified impression. That's not far off dreamland that's available today.

I can go to Spotify right now. I can pull out my credit card, I can go to their self-serve ad platform, and I can buy ads that I am highly certain got played back and I get really, really good reporting from Spotify directly. They're dealing entirely with logged in users. They've got unique identifiers for, right?

It's really quite something. Similarly, on YouTube, right? I can go to YouTube and I can buy ads, and last I checked, I'm not paying a dime for ads that people skipped. So in some ways, I think you're right to bring up the sort of the classic RSS distribution of podcasting and how some of what we might like to do from a transaction perspective might be difficult to implement.

But I wanna, I wanna push back a bit on this notion that the idea of like verified impressions are only paying for ads that people see or hear is like a, a dream world. I can go today to Spotify and to YouTube and I can buy exactly that. And so there's this whole world of podcast ad buyers and podcast buy side agencies.

You've had some of them on this show. And I think part of the existential threat for those kinds of operations, particularly the smaller of those operations, the ones that are not sort of publicly traded companies, the existential threat is if I'm a large consumer packaged goods manufacturer and I am dealing with a very big media spend, why wouldn't I write one big check to Spotify, one big check to YouTube, and call it a day?

Why do I need a buy-side agency to manage direct buys with networks or direct buys with individual shows? It's slower. It's more human powered. I can just write one check to Spotify and call it a day, and so there's a really big threat to independent, smaller, independent companies that transact in this space, represented by those major platforms who have a data advantage.

And so part of what we're trying to do with verified impressions is bring some of that mentality of only pay for ads that actually get played, which I can buy today, and not make that only something that I can buy from Spotify, YouTube, or another big platform.

Jeff Umbro: When it comes to the example of Spotify and YouTube, just anecdotally, I've just found that the targeting is not as good as like when you're able to curate something on your own via RSS and that's why.

Dan Misener: We've seen the same. Yeah.

Jeff Umbro: Yeah. I, I wish it worked better. I would do it in our, I would be one of those companies that you're talking about right now and, and there are plenty of companies that are going to Spotify and YouTube and saying, take my a hundred million dollars. They don't really care if it's gonna be like 0.1% less effective. Some of them do, I don't know.

Dan Misener: Where else can you get the kind of scale that those platforms represent?

Jeff Umbro: And that's exactly where like an iHeart and a SiriusXM come in, in the audio space.

Dan Misener: Mm-hmm.

Jeff Umbro: Because they're the other platforms that can bring that kind of scale and curated systems. But I really want to touch on the idea of marketing versus editorial devices, how you can pay attention to it. I think of it as listener retention versus net new listeners. But how does the Bumper dashboard help you think about bringing in both of those kinds of listeners?

Dan Misener: Often people reach out to bumper because they feel like they've hit a plateau. Or they feel like they've stalled out or they've taken their show as far as they can take it on their own, and they want an outside perspective. They want some measurement. They want somebody who can help them level up, get to the next level, whatever that is for them.

We have so frequently seen marketing problems, confused with editorial problems and editorial problems, confused with marketing problems. Why is my show not growing? Could be that you have a marketing problem and you have not put the show in front of the right people or enough of the right people. Maybe your megaphone is just not loud enough and you need to investigate other tactics.

Maybe it's paid, maybe it's a big earned campaign, whatever. Maybe you do have a marketing problem and the top of your funnel is just not wide enough. We've seen that. We have also seen examples of shows where marketing is not the problem. There's something deeper going on with the show. There is not a clear product market fit.

I say this with a lot of love to branded shows. I used to work for a branded podcast production company. This is just especially prevalent with branded shows. In my experience, there are a whole bunch of shows that the world is not clamoring for.

Jeff Umbro: Yeah.

Dan Misener: There are a bunch of shows where why it's good for the company or organization.

Producing it is clear, but why it's good for listeners and why anybody would wanna listen to it is not clear and they've leaned too hard into the why is it good for us and not hard enough into the why. Is it good for anybody who doesn't work here or anybody who doesn't have you know our company's name on their business card when we are working with a client for the very first time.

It's really important to triage these different kinds of growth challenges. Are we not growing because our marketing hasn't been as effective as it should be, or our marketing investment has not been as strong as it should be? Or are we not growing because our show is mediocre? Those require very different interventions, and I would not advise dumping a bunch of paid media on a show that has fundamental product market fit or editorial structure or host decision problems. Fix your show before you spend a whole bunch of money on marketing. 

And so you talked about listener retention, which I think is a really helpful signal, is the thing that we currently make well received and well used by its current audience, right? And so we tend to look at things like, we call it average listen time or minute by minute, second by second retention rates. We look at how those numbers change over time. We look at things like episode release volume. How many episodes per week do we publish? How many episodes per month do we publish? What's the average number of episodes per listener per month? If you're releasing 12 episodes a month and the average number of episodes per listener per month is two, you are way overproducing. You're making more stuff than the world seems to want from you. 

And so there's a bunch of different ways to slice and dice, not just episode by episode performance, but to pull back and look at the show as a whole. We look at those sort of audience engagement metrics, episode level, show level, and then you can even ladder this up to network level if you're running multiple shows and identifying challenges or areas of improvement there can be so important before you move on to any of the marketing interventions that might be required to, to get a show bigger. And not all of them are editorial. Some of these are just show structure. We do a lot of analysis of like, Hey, where are the ad breaks? How is our positioning of ad breaks impacting our average time spent listening? How is the ad break position influencing? Whether or not people use the ad break as a convenient excuse to bail on an episode, you know, so there's some structural things that we can often look at. And more and more, as I think about YouTube and I think about Spotify and the algorithmic or the alligatorial style discovery features that are built into those platforms.

And increasingly, you look at the front page of Spotify or you look at your, you know, the front page of your own YouTube experience, like it's increasingly customized and driven by all kinds of signals. It's all connected, right? So like the placement of our ad markers and our episode has an impact on retention. Our retention has an impact on our discoverability in these apps, right? So like there's a whole bunch of stuff where it's not just an editorial decision or a show structure decision, it's actually tied up in product, packaging and sort of the, the, the reach of the show. So there's a bunch of diagnostic tools that we use there, but it's like, make sure that your show is well used by your existing audience before you try and get more of those people.

Jeff Umbro: You have more data access than many folks in the podcast industry, just by nature of the tools that you've built. Are there any metrics that you would consider to be kind of standard to show the health of a podcast? Are there things that you see that indicate to you that things need to change in a significant manner?

Dan Misener: When I'm trying to figure out does this show have a marketing problem or does, does this show have an editorial problem? I'm looking at a couple of things. Average listen time is absolutely one of those things, and a big warning sign that I see is when average listen, time falls off a cliff suddenly because somebody went and bought some listeners somewhere because they significantly changed the editorial of a show. We sometimes see this when a host changes, right? If there's personnel change on the show, you know, a beloved host leaves and a new unknown host shows up, right? 

So if there's a sudden change, a sudden negative trend in average time spent listening, that's a big warning sign for me. Also, if there is a very significant change in the average number of episodes per listener over a period of time, that is also often a really big warning sign and we've seen shows grapple with or try and contend with decrease in downloads by making more stuff and publishing more episodes, which seems like it should work. Oh, we're getting fewer downloads. We'll publish twice as much. That'll get us somehow ahead of the, the tide that's going out often. 

That can really backfire. We've seen that backfire and it shows up as backfiring in sudden increase in volume of episodes published and sudden decrease in average number of episodes per listener. Over time, you made more stuff and you turned a bunch of people off, and so there aren't a whole lot of numbers that are directly available in the Apple or Spotify or YouTube dashboard that can give you kind of super actionable insights. It's the derived stuff I pay most attention to and can often be most helpful when you're trying to figure out is the change that we made pushing us in the right direction or the wrong direction?

Because there are some really perverse kinds of incentives that people choose. Like our downloads decreased as a result of iOS17. Let's just publish more episodes. I've seen that one go wrong a lot of times.

Jeff Umbro: I just saw this example the other day where somebody was publishing essentially two different kinds of shows. There was like a chat show and then there was like a narrative show. They were publishing it in the same feed and, and using tools, actually, not Bumper’s Dashboard, but a very similar tool. They were able to identify that there was not a lot of overlap in who was listening to like.

Dan Misener: I think I know exactly the example you're talking about. Yeah.

Jeff Umbro: There was a, a handful of editorial decisions that were made in terms of how did they structure this and the, and the answer was that they'd set up a second RSS feed. So there's a lot of different ways to kind of address this. It's all about like, what goals are you trying to address. 

One final question for you. What are you noticing in the world of AI in terms of like, uh, measuring data sets as opposed to the.

Dan Misener: Yeah. Yeah.

Jeff Umbro: And how are you thinking about that?

Dan Misener: This is maybe more of an AI feature of a platform that has really caught my attention over the past couple of weeks. I've been really interested in what Spotify is doing with their automatic chapter markers. I'm sure you've seen this. Spotify seems to be for many popular shows, creating robo transcripts, which are really good, and from an accessibility standpoint, incredibly helpful for so many people.

I love them. They're taking these robo transcripts and they are using some system on their end to add automatic chapter markers to videos that did not specify chapter markers. They will listen to your whole episode to break it up into chunks and make timestamped chapter markers fascinating that they're able to do this.

I mean, I don't think it's especially sophisticated. It's summarization and breaking it up. Here's what I find fascinating. We have started to see evidence that those AI generated chapter markers are prompting user behavior that the producer of the episode might not have ever expected. When we look at those minute by minute, second by second drop off charts, we start to see little spikes at the AI generated chapter marker beginnings.

People are for their first time, so we know this because we've been running some paid campaigns, driving net new listeners to shows. We are seeing that on those episodes that have the AI chapter markers, when you send brand new fresh listeners to those episodes, little tiny spikes, but very noticeable, little tiny spikes right around the AI chapter markers.

So as an audio storyteller, as somebody who creates a beautifully crafted episode for somebody, you're thinking about at the first 10 seconds, you're thinking about the first 30 seconds. You're thinking about how are we gonna open this episode up and how are we gonna introduce people to the characters and how are we gonna do this? And along comes AI generated chapter markers and it completely short circuits some kind of experience that the producer is trying to create. I find it fascinating. I don't really know what to make of it yet, but I think it is remarkable that a chapter marker that no producer explicitly put on the episode is being used by net new listeners to skip over the intro, to skip over the boilerplate, to skip over the explanation of what the show is and who the personnel are and get right to the thing that they were interested in.

I think that is remarkable, and it's an example of the kind of producers using great editorial judgment and making really strong shows and AI tools that they may not even know are at work in the platforms. In some cases augmenting and in other cases superseding what they might have intended as the the listener experience.

So this maybe not a classic example of, you know, computers and humans working together, but we are seeing this borne out in the data for a bunch of shows that we work on right now. And I'm fairly certain that many of the producers had no idea that the AI chapter markers were there, or that people were using them to skip over their lushly crafted intros.

Jeff Umbro: Is there anything that you want our listeners to check out?

Dan Misener: I always want people to check out Bumper, a podcast data and growth company where I work. We have a website. We write things from time to time, and sometimes we use strange analogies about butcher shops.

Jeff Umbro: I do think if anybody listening to this is not already subscribed, you should be reading everything that Dan and his colleague Jonas, and the rest of their team, right. At the very least, it's gonna make you think differently about a lot of the things that we take for granted in the industry.

Dan Misener: We like to write.

Jeff Umbro: Yeah.

Dan Misener: Jeff. Thanks for having me. Super fun.

Jeff Umbro: Thank you so much for joining us for this week's episode of Podcast Perspectives. You can find Dan Misener on LinkedIn or at wearebumper.com. 

For more podcast related news info and takes you can follow me on LinkedIn at Jeff Umbro. Podcast Perspectives is a production of the Podglomerate. If you're looking for help producing marketing or monetizing your podcast, you can find us at podglomerate.com.

Shoot us an email at listen@thepodglomerate.com or follow us on all socials @podglomeratepods. This episode was produced by Chris Boniello and myself, Jeff Umbro. This episode was edited and mixed by Henry Lavoie. Thank you to our marketing team, Joni Deutsch, Madison Richards, Morgan Swift, Erin Weiss, and Stephan Moore, and a special thank you to Dan Christo.