• SOLUTION

    A status-marking tool that allows creators to communicate their reward fulfillment status to backers

  • ROLE

    OKR lead/Design lead

  • YEAR

    2021

Fulfillment failure is the biggest threat to our business

Approximately 13% of all successfully funded Kickstarter projects failed to fulfill. While this might not seem like an alarming number, these projects accounted for over 30% of backers on our platform. Some of Kickstarter’s highest funding projects—the darlings of our backer community—unfortunately fall into this category.

From Jan to Aug 2020, fulfillment-related tickets made up 22% of all Zendesk support tickets—the highest of all support categories. A look at the comment hot words such as “refund”, “scam”, “fraud”, etc. suggests an escalated wave of negative fulfillment-related sentiment in the past 10 years. In physical-reward categories such as Design & Tech and Food, backer sentiment score deteriorates as rewards fail to be delivered on time, or for that matter, fulfilled at all.

In order to address larger fulfillment workflow issues, first, we need to gain scalable, systematic signals into fulfillment and backer sentiment. Having reliable fulfillment signals will enable us to invest in the right solution to our flawed workflow, be it building our own label/postage product or even just restricting creators’ risky promises early on.

The further a project is from its expected delivery date, the more backer sentiment score deteriorates

Project Conclusion Signals

In November 2020, I was tasked as OKR lead on Project Conclusion Signals—a H1 2021 product solution effort to quantify and eventually increase fulfillment signals on our platform.

Objective: Gain scalable, systematic signal into reward-tier level fulfillment and backer sentiment at project conclusion

KR1: Fulfillment signal coverage of at least 75% of all successful pledges, moving forward

KR 2: Backer satisfaction score coverage of at least 5% for fulfilled and unfulfilled

Current fulfillment signals were low

Currently, our set of signal tooling was weak. On-platform got-it tools give us 16% from backers, while send-it tools give us a meager 8% from creators.

There’re a couple of reasons why we’re not getting the coverage we want:

1.

Backers have to come to us for feedback; we’re not coming to them

Because of a lack of mobile-first approach and ineffective placement of the Pulse bar, we’re not engaging backers for Pulse feedback in the right place, at the right time.

2.

Creators aren’t incentivized to give us signals

Our current send-it tool is utterly manual. Many creators would rather use a spreadsheet to fulfill. On top of that, with robust third-party fulfillment tools like Backerkit and Shipstation, creators just aren’t incentivized to come back to our platform after they’ve received their money.

Fixed goal + small team = variable scope

Despite the importance and urgency of this project, it was assigned to my team, Creator Tools, in our most unfortunate state: our PM had just left, so did 2 engineers.

Working with a fixed business goal, as OKR lead, I had to be scrappy and flexible. In the next few months, I took on the role of a design lead, PM, and even dabbled in some coding.

I led various brainstorming sessions with stakeholders, including the Director of Risks, VP of Insights, Director of Design, and Group Product Manager.

We were able to outline 3 areas we’d like to tackle and potential features to experiment with:

Having only worked in small startups previously, I’ve fortunately picked up some PM skills. Perhaps the most useful one among them is Looker, a business intelligence tool.

I was able to run queries and model out signal coverage for each out our proposed solutions

Using existing proxy signals, I was able to model out how much coverage each potential approach would give us.

A generative design approach

In the early months of this project, I really struggled to be generative with my design. It took me some time to shed off the restrictive, business-oriented thinking patterns that came with wearing an acting PM hat. Once I learned to let loose, I was able to be much more generative with my design.

BACKER SIGNALS

An idea I wanted to experiment with was a Backed Project Carousel. Currently, the Backed Projects Page where backers go to manage their pledges, mark as received, give feedback, etc. is quite buried, especially on mobile web.

By giving major real estate on the homepage to the carousel, I hoped to increase backers’ interaction with their past projects.

Concept 1: Backed Projects Carousel

Another idea I had was a compact Got-it/Sentiment combo in the top nav, which had about 100k+ unique clicks per day.

Concept 2: Capture signals in the global top nav

One of our best bets was a “Did you receive X?” backer email. Traditionally, Kickstarter’s mailer has performed well.

Concept 3: Got-it email

I also wanted to experiment with different messaging levels and how they alter backer sentiments. What’s the difference between a standard “thank you” message versus a “we’ll disqualify this creator”? How does that alter backers’ perception of our platform?

I plan to follow up with each backer cohort with a Qualtrics survey to measure the effectiveness of these different messaging levels.

Concept 4: Different backer messaging levels

While we did some concept testing on the backer side, the only way to measure the effectiveness of these approaches was through experimentation.

I got to put my frontend skills to use a little bit and built several simple experiments on Optimizely.

An Optimizely experiment I built

CREATOR SIGNALS

Capturing signals on the creators side was tricky. Many of our creators abandon the platform after receiving their funds (which we give in full sum). In addition, Kickstarter’s neglection of post-campaign creator functionalities over the past decade has given rise to a suite of 3rd party fulfillment products that cater specifically to crowdfunding campaigns — we call these “pledge managers.”

One of the earliest concepts I developed was a Tweet-like status module that collects fulfillment stage data from creators.

Concept 1: Creator status bar

Creators can quickly post a status on top of their funded project page to update backers on their fulfillment progress

I also toyed with the idea of a “report card” concept: using some high-level fulfillment metrics such as percentage of rewards sent, number of backers waiting, peer creators’ got-it rate, etc. to create platform-wide high standards for fulfillment rate reporting.

Concept 2: Report card

Or I like to call it, the "peer pressure" concept.

Another concept that I explored was creator status update on our native app. Traditionally, Kickstarter has fallen short on creator functionalities on Native. If we could leverage creators’ need to send out a quick, convenient fulfillment update via the native app to collect fulfillment signals from them, we’d hit 2 birds with one stone.

Unfortunately, this concept was quickly scrapped because we didn’t have Native development support.

Concept 3: Creator status update on Native

I also explored a more versatile and utilitarian concept: allow creators to select pre-canned reward fulfillment status.

I made an assumption that if we present this as a tool that benefits creators’ workflow, we’ll be able to convince creators to give us signals.

Concept 4: Fulfillment Status Table

Another concept that I explored was incentives. What if we make fulfillment rate a permanent fixture in our search and filters?

Would that incentivize creators to take fulfillment more seriously?

Concept 5: Fulfillment rate filters

Creator concept testing

With a newfound focus on creator signals (and a new PM hired!), we set out to concept test several creator signal concepts that I designed.

We interviewed 16 creators across various funding buckets and categories with hopes that we’d identify a concept that benefits their workflows while helping acquire the fulfillment signals we needed.

Creator concept testing repo

Perhaps the biggest findings we came across was that creators wanted an easy way to communicate with backers about their rewards’ fulfillment status.

If we could position this tool as a fulfillment status communication tool, we would be able to increase adoption and acquire more signals.

MVP!

Among the concepts tested, the “quick status tweet” and the “fulfillment status table” tested best thanks to their communication capabilities.

While both tested equally well, only the fulfillment table, with its “rich signal” dropdown, met stakeholders’ needs. We decided that this would be our MVP.

I explore several surfaces that could host this table. We ended up going with surface 3, since it already had most of the heavy-lifting filtering functionalities that would allow creators to flexibly select the backer cohort they wanted to communicate with, which was a top request from creators.

Creators can bulk select and set a fulfillment status from the backer report table.

The status tool on mobile

Backers will be able to see their reward fulfillment status on the Backed Projects page and the View Pledge modal.

Retro: Liked, lacked, learned

LIKED
  • Leading an important, taxing initiative for Kickstarter
  • Dabbling as a PM, scoped out solutions and modeled effectiveness. I’ve always love spreadsheets :)
  • Looker!
  • Collaboration with UX Research on concept testing
  • Putting my frontend skills to good use and built out a couple of Optimizely experiments
LACKED
  • Stakeholder alignment on what qualified as fulfillment signals
  • A dedicated PM partner in the early stage of this project
  • A more adventurous spirit during early explorations
  • Engineering support during exploration
LEARNED
  • Build relationship with stakeholders early on, catch up 1on1, and get alignment on product decisions
  • How to design and get buy-in for low-lift frontend experiments to validate my assumptions
  • Leading is hard, but it gets better with practice, just like any skills :)
RESUME

LINKEDIN

BEHANCE

GET IN TOUCH