The Evolution of Auth0’s Insights Repository
How We Chose the Right Tools for Us
The Benefits of An Insights Repository
Prior to joining Auth0, I had little knowledge about what an insights repository (“repo”) was and the benefits of having one. However, when I started my internship on the Research Team in June 2021, I found myself immersed in the subject. Little did I know I would be the one to manage and garden our current repo as a research ops professional.
The insights repository is the database that stores, organizes, and makes all the knowledge collected from any completed research accessible to others within the company. The biggest advantages we’ve found to having a repo at Auth0 include:
- Helping evangelize the importance of research
- Linking research activity to product decisions
- Aiding in the democratization of research
- Creating easier mechanism for sharing out research
- Avoiding duplication of research
Below we’ll explore the different phases our repo has had throughout the years, the pros and cons to each solution, and where we’re headed in the future.
The Starting Point: Confluence
The research team is relatively new to Auth0. Without a process for documenting research, it made sense to use a tool the organization was already using often. We used Confluence as a repo for about a year. At the time we didn’t have the operational capabilities to manage the repo all on our own, and the simplicity of Confluence allowed anyone that conducted research to add in their insights from their studies. Once our operational excellence improved, we knew it was time to move onto a more complex system.
Pros
All documentation at Auth0 already existed within Confluence
With a need for a quick solution, it made sense to use a tool that everyone already had experience with in their day-to-day work.
Easy to update with research
Everyone knew how to use Confluence, so it was very simple for folks to go in and add the research they had completed. It saved time for the team since they weren’t focusing on data entry.
Cons
Limited amount of information allowed due to table size constraints
Tables in Confluence only allow for eight columns’ worth of information that can be viewed without scrolling. With that, prioritization of information was key, and it kept our tagging system very elementary.
Difficult to find specific information
The search functionality in Confluence is not particularly helpful for finding information. Using the browser’s native search worked better in most cases, and that’s how information was primarily found when we were using Confluence for our repo.
Current System: Jira
For our next iteration, we wanted to fix the search issues we experienced while using a Confluence table to store insights. Enter Jira, another tool the company had been using for a long time. Switching to Jira also coincided with the addition of a full-time research ops member to our team, so it made sense to use a more complex platform to bring more benefits to our team. We have been using Jira as our repo for nearly a year, and it allows for a more rich repository of data than Confluence.
Pros
Findability increased significantly for studies
The search functionality in Jira is solid, and it includes the option to do advanced searches. The ability to tailor a search for a stakeholder helped our credibility and evangelized our team’s capabilities.
Fields and tags are readily available for use in Jira
Unlike Confluence, fields and tags are easy to use in Jira. There is no limit, so creating a more advanced taxonomy is possible. We were able to add the type of research that was being conducted, ie. behavioral research, as well as emotions participants felt and the frequency of said emotions. It added a whole new layer of depth to the research our teams conducted.
Cons
The learning curve for the repo is high
When I was learning how to use our repository, I struggled a bit at first. There is a lot of interplay between the studies and the child issues (the research “nuggets”), and I struggled to distinguish between the two categories. The added capability to attach studies together because they are related also requires a lot of leg work to discover why they are connected. To ease this issue I looked at past studies in the repo to reverse engineer how the information was connected and stored as data. It’s a time consuming process, but it increased my understanding and efficiency going forward.
The data entry is as tedious as it is helpful
The data from our repo is priceless, but the data entry is a bear of a task. Jira does have the capability to automate some tasks, which is lovely, but a large portion of the data still needs to be manually added in. However, it is more manageable when it is spaced out appropriately.
Looking Towards the Future: Great Question?
We recently added Great Question (a tool, not a statement) to handle panel management and participant outreach. Little did we know that we were getting a near full stack tool with built in integrations to help automate away some of the tedious parts of research, like scheduling and incentive delivery. We released the tool to our product team recently, and the experience has been overly positive so far. With a repository already built in, it seems like it would be easy and convenient to have all of our studies and knowledge live in one tool.
Pros
Full stack research tool
As it stands, Great Question (GQ) is set to become a full stack tool. It includes panel management, screening capabilities, scheduling, rewards delivery, support for multiple study types (e.g. surveys, interviews, and unmoderated research), as well as a repository. It is appealing to have all of our information live in one place. Before GQ, we used three different tools to do exactly what we are currently able to accomplish through their tool.
Ease of storing data
When a study is completed through GQ, the recordings and transcripts are automatically added into the repository. It’s a great mechanism to help save time for folx who are not dedicated Researchers, which is the majority of our People Who Do Research (PWDR).
Cons
Early stages of development
The current state of the GQ repo is not as sophisticated as we would like it to be. One of the biggest pain points we notice is the inability to add studies that took place outside of GQ into the repo. The Jira system we have now meets our needs, but the temptation of a full stack tool is ever present.
Conclusion
At the beginning of our repo journey we needed a simple solution to keep track of our insights. As the team grew and operations became more prominent, our ability to manage our insights grew. This created an opportunity for a more complex tool to be used. Jira is not perfect, but it is a system that works for us. Great Question could potentially hold our insights one day, but only time will tell on how the repository gets built out by their team. The most important question for you to answer as you get started on your repo journey is “how much time do we have to dedicate to this tool, and how complex do we need it to be?”
At the end of the day, the most important question we will always ask ourselves is “how can we make research at Auth0 accessible to anyone who wants to do it?” Research at Auth0 was built operations-first: continuing to support democratization of research is part of our DNA.
