A behind-the-scenes look at bulk declines, decline reasons, a content calendar, and one very bad hour
A few months ago, we took an honest look at publications on Medium and asked if they were succeeding. Were users incentivized to create quality publications on Medium?
We had the Boost Nomination Program, which was helpful for the cohort that was selected, but we didn’t have any signal that the BNP was motivating for other users. Were we getting higher quality articles as a result of BNP?
So we asked ourselves “How can we reward good editors for their editorial work and make it clear how others can earn that reward?”
That’s when we came up with the Editor Partner Program (EPP).
The first thing we introduced for EPP was the ability to assign an editor to a given submission. Before, there was no way to tell if a publication editor was actively working on a post. And since we wanted to start compensating editors directly, we needed a way (in the data) to know who worked on a given submission.
So we introduced the ability for editors to assign themselves to submissions in the inbox.
Even for those not in the EPP, this was a useful organizational tool.
I’m not going to go in depth on EPP, but I wanted to flag it as the catalyst for a lot of the recent changes we’ve made and the ones we’re making in the near future.
We knew we wanted to encourage publications to be “open” because we wanted more writers to find a good community of readers. But we knew what being “open” could mean for editors. Lots of slop submissions.
Open submissions doesn’t feel fair … unless we give them something that makes their lives better.— Direct quote from a meeting
Scott, just solve AI slop… done
I wish I had a magic wand to remove all AI slop. We’ve looked into a lot of tools and they just aren’t reliable. They flag human writers as AI a lot.
Some publications are also fine with AI-written articles. Who am I to tell them no?
Rather than throwing up our hands and saying “this is unsolvable,” we thought about mitigation tools. The first was obvious: bulk blocking and bulk declining.
That was something I could write and ship in a day. It was out the door so fast that we thought “what else can we do?” And “how do we decide what to do?”
Workshop on how we might improve editors’ lives
I’m a sucker for a good workshop. Coworkers coming together and throwing ideas at the wall to see what sticks.
We had an hour-long workshop with people from all over the organization. It was important that it wasn’t just engineers. We pulled people from our community team, trust and safety, design, content, and more.
None of us know everything. But we all see the product at different altitudes. And it was important that we had people there who actually talk to users regularly.
Manage and reduce spam
The first thing we focused on was to build on top of the bulk block and remove feature, and focus on other ways to reduce and mitigate slop.
We came out of the workshop with some easy wins. The first was a way to more easily see what submissions were related to your publications’ topics.
We had ways to quickly sift out the spam, but could we introduce more tools to cut down on the spam in the first place?
The next thing we wanted to give editors was the ability to set stricter submission rules for their publication. These would be hard stops that prevent people from submitting until all the boxes are checked. We gave editors tools to (hopefully) spot and remove slop, next let’s give them a tool to keep the slop out of the inbox in the first place.
This also helped writers. Some would have their stories rejected immediately by editors because they didn’t include a subtitle or an image.
Now editors were only getting stories that more closely followed their rules. Of course, that doesn’t mean we removed all spam and it certainly didn’t mean that every submission was high quality (even if well-intentioned).
Continuing down our list of mitigation tools, one point of frustration editors run into is a user who publishes too much. When you see a submission, you only see that post in a vacuum. There’s no way to see if that person publishes constantly (i.e. they are trying to spam).
So we added more to the inbox by giving you a quick glimpse at their recent publishing history:
Writers who earnestly submitted something and got declined kept telling us the same thing: they didn’t know why. Why was my story rejected? Unless the editor left a private note, there was no indication whatsoever.
We introduced preset “reasons” for declining. It made the ecosystem feel less anonymous and gave writers something actionable to work on.
Make running a publication easier
With some of those light, mitigation tools in place, we also wanted to focus on ways we could improve managing a publication (that had nothing to do with removing spam).
We had feedback from editors that poor images and the inability to easily set how images are displayed made their publications look unprofessional.
That’s what inspired us to introduce a tool that easily allows you to set the focal point of an article’s preview image. This is a change that made it to every user, not just editors.
Listening to more feedback, we heard from editors that scheduling could be a pain. It required clicking into each post to make changes, and there wasn’t an easy way to see your schedule other than a list.
So we built out a content calendar. You could see the full schedule for your publication and drag-and-drop items between days to reschedule. No more submenus or guesswork.
We also made it easier for you to edit and manage the writers for your publication. The new “Editors & Writers” page lets you quickly search your list of editors and writers, add new ones, and even download a CSV of everyone who contributes to your publication.
Misses Things we learned
The catch-22 of listening to user feedback is that sometimes you get feedback on the solution to the prior feedback.
We shipped the ability to assign editors, which we used to mark payouts for the editor partner program. But some publications only have 1 editor. A few users left comments saying it didn’t make sense to have to assign themselves, given they were the only editors.
So we shipped “auto assignment” for publications with only 1 editor.
When we showed off the “off topics” flag, most people liked the idea, but the biggest piece of feedback we heard was “5 tags isn’t enough.”
Why could publications only set 5 tags? I dug into it and… no one knows. That decision was made 10 years ago 🤷. Nobody who worked on it still works at Medium, and the ticket linked in the comments was for an old Jira instance I couldn’t access. So we upped it to 10 to give publications more tags to choose from.
The “decline reasons” were a pretty well-received feature. Only problem was, I completely broke submissions for an hour. Oops.
For about an hour, nobody could submit a story to a publication because a migration hadn’t run when I deployed that feature, so no one could write to the submissions table.
It was an easy fix, but it’s the price of moving fast sometimes…
So, what did we actually learn? Sometimes fixes need fixes, and we have to create bandwidth for that. Our users have a great sense of what is working, so we try our best to listen and think through the impact. That could mean changing something we literally just built, and that’s okay.
What’s next?
People want a customizable “decline reason.” i.e. an open text box to describe why something was declined. I do too.
The problem is that as soon as we introduce an open text field, we introduce a vehicle for harassment. So we’d have to build reporting and Trust & Safety tools into it. That turns a small quality-of-life feature into a much bigger project.
We’re going to work on a Publications Directory. A better way for writers to find publications that resonate with their writing. We’re still early in planning, but we want to help people find your publication!
We’re working on editor tools that better help you communicate with authors in the draft. This would let you have a faster back-and-forth with authors and show them the changes you’ve made to their draft before publishing.
Where can you send feedback?
Use Medium’s support form to send feedback or report an issue. For tips on writing a useful ticket, see Contact Medium Support.
Or you can leave a comment here! I swear, I do read them. Just be forewarned, I might use them in a blog post in the future.
What We Built for Medium Editors This Year (and What I Broke) was originally published in Medium Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.
Source: medium.engineering
