In this blog post, we chat with two members of our developer team who conduct our Developer Experience Friction Audits - or DEFT Reports as we call them.
Dana Fujikawa and Steve Pousty review a lot of Developer programs, so we thought it would be fun and insightful to ask them for the inside scope of what they see out in the wild.
Let’s get started.
What do you enjoy most about pulling together a DEFT report?
Dana:
I love learning new things. Getting the chance to try out different technologies across a breadth of verticals broadens my industry knowledge.
I also feel that many companies overlook DX and usability, so it’s rewarding to highlight areas of opportunity, especially those small ‘nuggets’ that can have a big impact on a client’s developer outreach program.
Steve:
I love getting the chance to learn technology AND at the same time help someone improve their product’s developer experience.
There is so much technology I would love to explore, that I have the intention to explore, and yet, never explore. Doing a DEFT gives me a concrete reason to get hands-on with a technology. It is how I finally used Kafka, gained a lot more knowledge about payment systems, and even learned more about PostgreSQL.
The added bonus is that I get to help and teach someone else how to make things better for developers. I have been doing DevRel for almost 20 years, which has led to me having an oversized concern for the ease of use for developers. Doing DEFTs I get to learn something new while engaging in my passion for helping developers.
What are some of the common issues you come across?
Dana:
The top issues we find with client’s products and outreach are:
The home page is saturated with marketing speak, value propositions, and other vague information that doesn’t help the developer.
The “Developer” menu is missing or buried on the home page.
SEO is lacking, making it difficult to discover the company and their product.
The documentation fails to guide the developer.
Steve:
Bad product naming, leading to poor SEO. Even if someone has heard of the product but is not exactly sure of the spelling or the full name, you will lose them before they even have a chance to meet you… ships, passing in the night.
Not enough focus on complete and usable reference documentation.
A lacking or poorly implemented Getting Started (sometimes referred to as a Quickstart).
Lack of linking between the different parts of content. You may know where all that content lies and how to approach it, but the external developer does not. Support them in exploring and understanding more of your product by generously linking within your content.
What is your personal bugbear?
Dana:
I once had a client with a REST API that didn’t follow REST best practices. Unfortunately, they had an emotional connection to their API design and didn’t take kindly to my findings. My intention was to simply surface what I found, but since that experience, I always try to be extra sensitive when it comes to critiquing people’s creations.
Steve:
I have two, sorry I can’t just stick with just one 😀
The poor Getting Started/Quickstart documentation. You can have the most powerful, exciting project in the world, but, if your users can’t get started, none of that matters. It is good developer retention and just plain kindness to clearly walk through at least one basic workflow with your product.
Companies think examples, samples, and tutorials are more important, or can actually replace excellent reference documentation. If the developer is lucky, your example exactly matches their problem. If not, well then wish them good luck in trying to figure out how to adapt your example to their use case. Without the reference doc, they are forced to discover functionality by guessing and feeling around. NOBODY likes that.
Which company would you say has an awesome DX?
Dana:
Having spent a good chunk of my development career working with Microsoft products, I have to say Microsoft. You can always count on them to have extensive documentation and code samples for all of their products, albeit overwhelming at times. And events like Microsoft Build show just how dedicated they are to developer success.
Steve:
It’s not a company, but I believe the Django Python project has amazing DX. Their documentation is clearly laid out in the ways developers expect. They hit all the necessary pieces for a documentation site. They have a friendly and welcoming community. They have done a great job of making their framework match the way Python developers want to work.
Have you ever found anything notable or unique, perhaps even easter eggs, when reviewing a client’s Developer Experience?
Dana:
You’d be surprised how often we encounter products, APIs, and/or learning resources (e.g., documentation) that aren’t as ‘complete’ as the client thought. This can range from hardware dev kits that don’t work as intended to APIs that are not fully understood since their designers have long since left. Between my time as a developer, technical writer, and developer relations consultant, I feel like I can truly say I’ve seen it all!
Steve:
I am always pleasantly surprised when a product gives me an easily found, clear pricing page before I have to give them my email. It is surprising how many products:
Put pricing behind a sign-up
Have arcane and confusing pricing schemes - looking at you Hyperscalers
Even if their pricing is public, they make it difficult to find the pricing page
“How much is this going to cost me” is right behind “Does it solve my problem.”
Any tips for someone trying to do their own DX Audit?
Dana:
View the Developer Journey Map as a framework that categorizes touchpoints into journey stages., (in practice, developers will jump between stages and touchpoints.) The Developer Journey Map provides a way to categorize and organize both your audit steps and the findings that you document in your final report, from the developer's perspective.
Also, start at the bottom and work up. Start with your in-depth technical audit, and then “bubble” your findings up into a summary that maps each to a consequence and recommendation. This thought process is probably the most critical phase since it gives birth to your recommendations. From there, you can prioritize these items and derive high-level recommendations, strategies, and tactics.
Steve:
First and foremost you need to be able to put yourself in the shoes of a developer coming to use your product. Often this might mean hiring an external developer to try and build something with your product. Internal Engineers and Product Managers are typically living in the world of “expert users,” meaning they can lack the objectivity to interact with your product as if they have no background or inside knowledge.
Second, I would look to The Book and its Developer Journey Map. While the journey might not be linear, that chapter and framework capture all the things you are going to need in place for an exemplary DX. You don’t have to do it all at once, but if your goal is to build excellent DX, you are going to need most of it.
What actions have clients taken based on your findings to improve their DX?
Dana:
It often depends on what the client was hoping to achieve. Some already know that their DX needs improvement, so the DEFT report provides concrete evidence to secure budget, resources, or alignment to fix it. In other cases, the report serves as the basis for a strategy and tactical plan to be executed over the next year.
Some clients use it as a benchmark, with the goal of repeating the audit annually to measure improvements.
Steve:
I would say one of the biggest is allowing them to convince people with budget that the company really does need to invest in good DX. Since DevRel.agency has deep and broad experience in DX, the DEFT reports we generate can bring more legitimacy and specificity in terms of what needs to get done.
What makes for a successful DX audit and a smooth DX audit process?
Dana:
First is to ensure that all stakeholders understand the purpose and importance of DX and DevRel in general, so as to align around the audit.
Next, the product and learning resources need to be in a good state to audit. If there are bugs, gated learning resources, incomplete products, and other blockers that require client intervention to get working, then the audit becomes more difficult to conduct.
The stages of the developer journey might end early, or the journey might diverge from the intent, thus hindering and fragmenting the scope of the audit.
Steve:
The most important factor is close collaboration between the people asking for the DEFT and the team leader managing the DEFT. There is such a missed opportunity if the DEFT is undertaken without a clear understanding of the clients:
Expected use of the product
Expected users of the project
Know problems or unimplemented functionality
Immediate and long term goals for the client DX program
The vision goals of the client’s larger company.
Nailing down these shared understandings can, surprisingly, be quite tricky. Spending time with our team, making sure we are all on the same page is definitely one of those “an ounce of prevention is worth a pound of cure" situations.
Do you need help with your Developer Experience?
Book a no-obligation meeting with us to discuss your Developer Program. We will walk you through the content and structure of an example DEFT report and get to know your needs.