Experiment like a boss
Five years ago I had a great opportunity to run an innovation project at a well-known financial institution. While I had several years of experience innovating within established organizations and flexing my intrepreneurial muscles, I was definitely not prepared for the challenges I would face.
But, my experience at the bank helped me grow as a product manager and as a champion for experimentation.
Regardless of whether you’re building a new product in stealth mode in a far corner of the office (like I was), or starting small by experimenting with your existing feature set, these 9 tips are for you.
1. Let the vision lead
Get started by aligning around the vision for the product. Your team needs to be laser-focused on the “why” behind what you’re building, who you’re building it for, and what you’re trying to achieve with your innovation project and the longer-term product.
We kicked off our project with a planning week. We started with Simon Sinek’s “Golden Circle”, moved on to a vision statement workshop, sketching sessions, and then finished by creating learning objectives for the first phase.
A clear vision will keep your team focused and your stakeholders reassured. And, most importantly, it will help you make tough decisions as you forge ahead.
2. Create a new vocabulary
Words matter. The vocabulary you use to describe what you’re doing, how you’re doing it, and the product you’re building will impact your team’s success.
During our kickoff week, we decided to start calling our innovation project “the experiment”. The word experiment allowed us to try something new, to potentially fail, and to work outside the core digital team operations and procedures. In essence, an experiment wasn’t scary to stakeholders in the same way that a project was.
Additionally, we were careful about the terms we used during our first weeks. We didn’t want to fall into a trap where our vocabulary accidentally prescribed the user experience. The problem was, we still needed a shorthand to refer to what we were building. So, until we settled on specific concept we called the product “panther”.
3. Get comfortable with “I don’t know”
One of the biggest challenges associated with experimentation is the volume of unknowns. This applies to every aspect of the experiment, starting with planning and project management all the way through to execution and wrap up.
You have to show your team and stakeholders that you’re not only comfortable with the unknowns, you embrace them. This will help build a culture of trust and safety, which is integral to the success of your project.
So, how do you do this?
- Always be forthcoming with what you do and do not know.
- Develop a strong methodology for finding answers. If you’re not getting the results you’re after, iterate. Learn to trust your process.
- Remind your team and stakeholders that there’s value in the journey. If the team had all of the answers up front you wouldn’t have been asked to run an experiment.
4. Plan (…but not too far ahead)
We divided our 5-month experiment into 3 research phases: exploration with employees, co-creation and concept testing with recruited participants, and an 8-week pilot where the new app was distributed to test users.
At the start of each research phase, we defined a high-level plan for achieving our research goals. Specifics for each round of testing were determined several days in advance and built on findings from previous rounds.
Then, during sprint planning, we discussed the goals of the sprint and identified our top 3 focus areas. These were displayed prominently at the top of our Kanban board.
You will need to provide enough structure so that your team and stakeholders feel comfortable, but not so much that you’re boxed in. If you need to make changes to the project plan as you progress, do it.
5. Park questions about the future
Stakeholders are going to want to know what’s next, and they will often shoot down ideas because they can’t abstract themselves from current constraints. They’ll say things like, “We couldn’t possibly do that because of X,” or “How will you manage X once this is in production.”
Future-focused thinking is essential for successful product delivery, but it’s death to your experiment.
To protect your experiment, research learnings, and ability to pivot as needed on the fly, you need to find a way to let stakeholders feel heard while also avoiding talking about the future.
Enter the parking lot. This is a space, either physically in your team’s working areas or online in your project wiki, for keeping track of all of the future concerns.
Do not get bullied into thinking more than one sprint ahead!
6. Prioritize the experiment
Design your experiment to maximize learnings! Sometimes this means making UX compromises to ensure you get the information you need. Warning: this approach may make you unpopular with the team.
Keep the experiment vision, objectives, sprint goals, and hypotheses top of mind and visibly displayed. In moments of disagreement, refer back to your objectives, and design experiences that best achieve those objectives.
The way you implement something during the experiment doesn’t 1:1 dictate how it would be implemented in production.
7. Do all the things
When it comes to innovation and experimentation, you can do more with less. This means looking for T-shaped individuals (meaning: deep expertise in one area and shallow expertise in several other areas) with complementary skill sets. By taking this approach in spinning up our project team, we were able to accomplish a tremendous amount with only 4.5 resources.
Foster a collaborative spirit by creating a culture where taking ownership is encouraged and sentiments of “not my job” are quickly squashed. During an experiment, everyone needs to step up and do all the things.
8. Find new ways of keeping your stakeholders involved
While stakeholders were generally enthusiastic about our experiment and new app concept, when the time came for checkpoints our project often got trumped by other responsibilities. Poor stakeholder attendance at the start of the project had a negative impact on team morale.
So, we got creative with how we engaged our stakeholders. We reached out to them 1:1 and personally invited them to the fortnightly meetings. For VIPs, we booked in separate catch ups. Additionally, giving “tours” of our team space (walk the wall) helped to tell the story and pique stakeholder interest.
Ultimately it’s up to you to get the right people in front of your results. Otherwise, you won’t have the necessary buy in when it’s time to implement in production.
9. Have fun!
We made an effort not to take ourselves too seriously, both while working and during breaks.
On the work side, we brainstormed, sketched, journey mapped, and analyzed results together. We made avatars for our team, and we used celebrity names for our participants. We gave our features fun, descriptive names.
On the personal side, we had lunch together, celebrated birthdays and personal wins, came together for happy hour, documented funny quotes and gifs in a “wall of fame”, and never missed a Donut Friday.
A happy team is a productive team.
About the author
Hi, I’m Merissa. I have 10+ years of experience in digital product management in the US, Australia, and Germany. I specialize in new concept validation using Lean UX, and I love working with small, fast-moving cross-functional teams.
