RFC-44: Marketplace Store Revamp

[ChoppaAditya]

RFCs are a way for Atlassian to share what we’re working on with our valued developer community.

It’s a document for building shared understanding of a topic. It expresses a technical solution, but can also communicate how it should be built or even document standards. The most important aspect of an RFC is that a written specification facilitates feedback and drives consensus. It is not a tool for approving or committing to ideas, but more so a collaborative practice to shape an idea and to find serious flaws early.

Please respect our community guidelines: keep it welcoming and safe by commenting on the idea not the people (especially the author); keep it tidy by keeping on topic; empower the community by keeping comments constructive. Thanks!

Summary of Project:

As part of our ongoing efforts to enhance and boost app discovery, we are in the process of revamping the Atlassian Marketplace store experience. Our current focus is on refining the navigational properties across the Marketplace, streamlining the app categorization system, and improving the search experience. You can find the detailed changes outlined in this RFC.

• Publish: Mar 26, 2024
• Discuss: Apr 3, 2024
• Resolve: Apr 10, 2024

Problem

The Atlassian Marketplace offers a wide array of apps but still faces challenges in effectively surfacing relevant options to users based on their specific needs and preferences due to fragmented navigation experience and disoriented app categorization.

In order to address these issues and provide a seamless user experience, we have recognized a few enhancements to the user experience. This would help us gear towards the direction of delivering a holistic platform that attends to user needs efficiently with future iterations and developments.

Some of the areas that we have already taken up as the primary scope of improvement include:

Navigation

1. The current experience in Marketplace lacks a robust navigation framework, which has led to a
   poor and inconsistent wayfinding experience. Customers are forced to rely on filters to navigate across the experience. With inadequate guidance, app discovery may pose a struggle for users to find specific apps or categories, leading to frustration and reduced engagement.

2. Navigation on mobile devices (which constitutes ~18% of the traffic) in the Marketplace is more problematic due to a lack of optimization for smaller screens and the absence of intuitive interaction patterns. Limited screen real estate may result in important navigation elements being hidden or inaccessible, contributing to decreased usability and potentially discouraging users from exploring the Marketplace.

App categorization

Atlassian Marketplace currently has 37 categories and 33 collections which overlap with each other
and offer a disoriented experience. The existing taxonomy is inconsistent with some being too broad and some being too specific based on use cases, functions, and features. This hinders the app discovery process for customers as they are unable to find the right apps that align with their needs.

Search

The current search functionality in the Atlassian Marketplace exhibits several shortcomings, contributing to suboptimal user experiences and challenges in app discoverability. Firstly, the reliance on keyword-based search limits users’ ability to find apps based on their specific needs expressed in natural language. Additionally, the absence of auto-complete suggestions hampers users’ ability to form the right search query and further refine it efficiently. Moreover, an over-reliance on app names in search queries incentivizes Marketplace partners to use longer, potentially convoluted names to increase visibility, detracting from the user experience. These issues collectively diminish the effectiveness of search functionality, impeding users’ ability to find relevant apps tailored to their requirements.

Proposed Solution

As a proposed solution, we are offering a new landing experience with a revamped home page that encompasses a robust navigation framework, well-defined app categorization, and enhanced search experience.

Expand to view new Homepage design

• Streamlined navigation: We are improving the navigation so that it enables users to maneuver efficiently through the site and find apps as per their requirements. This persistent navigation bar will guide users to quickly access different sections such as apps for products, categories, and resources while on the go.

• Personalization and app recommendations: Provisions are being made to introduce personalization to the Marketplace Homepage. This aims to promote relevant content and apps to logged-in users by allowing changes in section order, and prominence, and displaying only relevant content. We are also planning to include a new section on the homepage to recommend apps based on user segments, preferences, and other parameters.

• Additional resources and company stats: We are also planning to include a resources section both in the top navigation as well as the footer section to guide users as they navigate through the site. It will include links to documentation and other resources that can contribute to better engagement with Marketplace. We are also introducing a section to include Marketplace statistics that will help to build trust with users who have never made a purchase from Marketplace before.

App Categories

• Introducing categories on the top navigation bar: To enable users to easily access and navigate to category pages from anywhere across the Marketplace
• Creating category-specific landing pages: To enhance app discovery within the context of each category
• Introducing use-case filter on the category page: To assist users in refining their search based on specific use-case needs.

Collections

• Distinct collections: Different types of collections based on well-defined parameters, industry-based trends, and popular use cases.
• Enhanced collection landing pages: Revamped pages for collections with better information
  architecture.
• Highlighted content: To enhance the visual prominence of the collection on the home page based on customer interactions.

Search experience

• Introducing auto-complete suggestions: To help and guide users in curating their search queries and further refinements.

• Natural Language Search: We are planning to introduce a search feature that supports natural language queries. This will be rolled out as part of our upcoming releases.

Note: All the proposed changes are optimized for smaller screens and designed to scale well across all devices.

Disclaimer for images: Please note that the designs provided are tentative and subject to change. They are not final and are provided for the purpose of discussion and feedback.

Asks

While we would appreciate any feedback you have on this RFC, we’re particularly interested in hearing more about:

1. Any concerns regarding the new navigation experience?
2. Any specific suggestions for improving search functionality on Marketplace?

We appreciate your time and any feedback you could provide, whether it be on the specific points highlighted above or anything else outlined in the RFC. Your valuable insights will play a significant role in shaping better app discoverability in the Marketplace.

[JacopoLanzoni1]

Sure this is not RFC-44? We already have RFC-42: Deprecation of mirroring Third Party Dependencies via packages.atlassian.com

[ibuchanan]

@JacopoLanzoni1, good catch! And my fault. I’m supposed to check that and slipped up today.

[anton2]

Thanks for moving that forward!

Few points:

1. The home page has a huge banner with the search on top that does not give any value, it just wastes the space, IMHO. Probably we can put some valuable content, like recommendation, featured collections, some app hero banners etc

2. Maybe we can introduce some collections by industry, for example, IT service companies would like to see time tracking, vcs integrations and invoicing solutions.

3. Maybe we can make collection tiles/images more informative, like Monosnap

4. I think this block does not help people find the app they need, it is a marketing noise Monosnap

[nathanwaters]

I think you need to make efforts to stop promoting the same old incumbents on the homepage.

Every time you open the homepage it’s the same dozen apps cycled through those homepage categories. It’s incredibly boring, makes the Marketplace look like it’s dying, and it’s unfair to other apps.

They’ll kick and scream about any changes here, but those incumbents don’t need more installs. They need competition. The way you’ve structured the homepage and search result rankings ensures these incumbents will always get the majority of clicks.

Also removing install count and reviews would level the playing field so that the marketplace functions more like a free market where the best products win. Those figures are very poor signals for the most part. The older the app the more installs. And incumbents were allowed to keep all reviews (real or fake) prior to a recent policy change.

As a first step you could perhaps AB test removing these figures (and their ranking weights) from the search results but keep them on the listing page. I’d bet that’ll result in a wider distribution of clicks which will encourage a fairer and more healthy ecosystem.

[JensSchumacher1]

I agree that removing install count would be a good first step, but I would hate to see reviews go. Customers will need some way to differentiate quality apps from the rest, and reviews is the best we’ve got.

[nathanwaters]

@JensSchumacher1

Customers will need some way to differentiate quality apps from the rest, and reviews is the best we’ve got.

Yeah agree, though I’ve got many ideas on this (sorry to derail the thread):

1. Instead of reviews on search results, simply hook into JSM to show average support response time and time to resolution. That’ll encourage vendors to be more proactive and is a very good signal to separate vendors who are customer-focused and those who are not.
2. And turn static reviews into dynamic public support tickets segmented into either bugs or feature requests. You can then surface that data to differentiate vendors based on how fast they fix those bugs and deploy those most requested features.

Positive reviews are super difficult to get and the incumbents are riding on a decade or more of real+fake reviews. No matter what policy is tried, reviews will always be gamed. 100-person vendor can always just ask their 100 employees to create real reviews with their private emails.

It’s just impossible for any new app to compete on that data point or for Atlassian to realistically enforce any policy, particularly in this exponential era of LLMs.

And most negative reviews I see are simply because the product either has a bug or it’s missing a feature the customer wants. Instead of contacting support they leave a negative review, you fix the bug or add the feature within 24 hours, inform the reviewer, and they don’t update their review.

The two ideas above (and I’m sure the community can think of more) make things much more congruent with agile software development, levels the playing field for healthy competition, and aligns incentives for fast customer-focused iterative product cycles.

[aragot]

Nathan is right about having too much promotion for incumbents. There should be a section about top earners, sure, but it should only be a section, not permanent promotion at the top of the Marketplace.

• Any product with more than 100 paid installs has proven its stability, and thus, should be promoted to large companies.
• There should be a section to discover newcomers, although it’s not in my interest,
• Maybe companies should be able to filter by number of installs as a trust signal: Very large companies will only install products with more than 1000 medium customers; While small companies will trust us as soon as we have 100 customers, etc.

In the spirit of rotating the promotion of incumbents, would the Atlassian Marketplace accept paid promotions / ads to be put on the homepage?

The “staff picks” was the section that came closest to that: If you argumented nicely that your app had new features, Atlassian could give us a new moment of fame on the homepage, and that was useful. Maybe Atlassian should promote release notes / new features of apps when they have a nice image and have been marked as important new releases by the vendor?

[dmitry.astapkovich]

I kind of struggle to imagine how a page/section for specific Atlassian will look like, which a way more important filter than Category. The only way to select the product is the top navigation, while more than a full screen height is wasted on meaningless banners.

I understand that banners look “cooler” but honestly I’d rather have a simple Product selector like this right on the page, not in the dropdown or menu:

Same issue with the apps cards. Unless the target Product is in the name, it’s impossible to say where that app works. Which is a bit confusing.

To add up to that confusion, search results are mix of apps and workflows/templates:

I did this search recently, and it took me quite a few moments to release what are those results actually. Turns out it’s a mix of everything basically, and unless you narrow down your search, it’s impossible to tell which card is what exactly in terms of app/asset, product and platform (Cloud or DC).

That is the real issue, one of many, which can’t be fixed with a cosmetic revamp. Unfortunately it doesn’t look like Atlassian will focus on that.

So yes, please go forward with moving the pictures on the first screen. Whatever it is, we are fine with it.

[MattiaLomazzo]

Thanks for sharing this RFC. We really appreciate that you want to improve the Marketplace for both customers and vendors.

Some points and suggestions for improvement have already been mentioned. The following thoughts and wishes crossed my mind while reading.

From a customer perspective, I have the following suggestion regarding the search field. Instead of just entering the search word and submitting the search, it would be an advantage if you could also already set the filters (product, hosting, etc.). Instead, you must first submit the search and filter on the results page. There are enough marketplaces that provide examples of how a filter can be set up in the homepage’s search mask.

It would also be interesting for me to know what criteria are used to suggest apps to me as a customer. Presumably, this is also of interest from a vendor perspective.

Although I understand their argument, I would not remove the installs and reviews, as some have suggested. From the customer’s point of view, it makes a difference whether an app has had 100 installations in x years or just 1. The reviews can also help to gain trust in a vendor and their app.

As far as improving the search is concerned.
What criteria are used to sort the search suggestions?
And as a vendor, I have one wish. The texts from the “More details” section of the Marketplace Listing should also be included in the search index and considered for the search.

[MarkusDoll1]

We appreciate improvements on the marketplace, as user experience should indeed be a priority on the Atlassian Marketplace - hopefully all updates significantly enhance the user experience and help customers discover the best app solutions for their needs. When potential customers visit the marketplace, they should be able to navigate seamlessly and find relevant apps without any hassle. Visibility for marketplace vendors is also crucial, especially for their relevant search terms.

Having access to metrics like monthly search volumes, impressions/clicks/average position for specific keywords, number of results for particular keywords, app positioning within categories and appearance on the marketplace homepage would be immensely valuable. Conversion data on a keyword level, would be a game-changer.
With this data, marketplace vendors - of all sizes/age/status - could fine-tune their listings, focus on their specific niches, and optimize their offerings to better serve potential customers.

I agree with Mattia’s perspective. The number of installations provides crucial context. Customers often want to know if an app has been widely adopted. A higher installation count often implies reliability, stability, and trustworthiness. Reviews offer insights into an app’s performance, usability, and customer satisfaction. Positive reviews build trust, while negative ones raise red flags. Removing these factors from listings will not help customers decide on the right product for their needs.

More insights regarding “Personalization and app recommendations” would be necessary and very much appreciated.

[rlander]

I assume this all relates to the marketplace website, and I guess the in-app marketplace in cloud?

Would be lovely to see a better in-app experience in Data Center still plenty of marketplace revenue coming in on DC for Atlassian

Bit of a tongue in cheek response, but the UPM in DC is slow and dated, and the “incumbent” problem noted by @nathanwaters and others is more severe when browsing in the UPM.

[BrittaNeugebauer]

Thank you so much @ChoppaAditya for reaching out. I’d like to +1 to @MattiaLomazzo and @MarkusDoll1.

From a vendor perspective, I’d like to learn more about optimised search and how we can organically keep up with new criteria for search. It would be very helpful if we could adapt our marketplace listing to provide customers with relevant information and a healthy keyword density. Will there be opportunities for us as vendors to do enhanced search optimisation on the marketplace?

[EmilOdagiu]

@ChoppaAditya

Asks

While we would appreciate any feedback you have on this RFC, we’re particularly interested in hearing more about:

1. Any concerns regarding the new navigation experience?
2. Any specific suggestions for improving search functionality on Marketplace?

We appreciate your time and any feedback you could provide, whether it be on the specific points highlighted above or anything else outlined in the RFC. Your valuable insights will play a significant role in shaping better app discoverability in the Marketplace.

Hello everyone,
It’s encouraging to see the Marketplace experience receiving significant attention in recent months. I’d like to contribute my thoughts on enhancing the search functionality, building on the points you’ve outlined.

1. Reevaluation of Searchable Attributes and Their Weightings: The current reliance on app titles for search visibility, as highlighted, leads to unnecessarily lengthy and complex titles. A reassessment of the attributes considered in searches and their respective weightings could mitigate this issue. For instance, the inclusion of ‘Partner name’ as a distinct searchable attribute may not be as beneficial as it seems, given its likely presence in the app’s title, category, or tagline. This redundancy could inadvertently increase the likelihood of false positive matches, though such instances might be rare. Partner name being a searchable attribute with a high weighting opens the possibility for partners to include a high-traffic keyword in the name to gain an additional boost from this field. I’m prepared to offer specific examples if needed.

2. Revamping the Search Ranking Formula: The existing method of ranking search results, based solely on match rate, does not adequately serve customer needs. The ranking process should incorporate additional attributes, beyond mere match rate to ensure that general search results are both consistent and relevant. For example, the discrepancy between the results for selecting the ‘Macros’ category and those for searching ‘Macros’ suggests a need for a more nuanced approach. Incorporating factors used in tie-breaking situations into the primary ranking formula could enhance the relevance and consistency of search results, preventing the prioritization of apps that merely accumulate keywords across searchable attributes.

3. Enabling Partial Match in Searches: The lack of support for partial match searches, leading to instances where longer queries return no results, is a significant limitation. Implementing partial match functionality, as offered by Algolia, could represent a straightforward improvement. For instance, if separate searches for ‘A’ and ‘B’ yield results, but a combined search for ‘A B’ does not, the system should offer a selection of products related to ‘A’ and ‘B’ along with a note explaining the absence of direct matches for ‘A B’. This approach acknowledges that while an imperfect match may have a lower conversion rate, it is preferable to presenting no options at all, which guarantees no conversion.

These suggestions aim to refine the search experience, making it more intuitive and effective for users. I’m also considering several ‘quality of life’ enhancements that could further improve the platform. Please let me know if you’d be interested in discussing these additional ideas.

[CraigWillson]

Thanks for sharing this RFC. It’s great to see the Marketplace is being considered for a long-overdue revamp, which will definitely help improve the end-customer experience.

First of all, I agree with many of the comments already shared by other Marketplace Partners. I think the initial concepts are a great starting point, and have added some of my thoughts below:

• Homepage design: the proposed new version still uses over 50% of the ‘above the fold’ space with information that is not particularly useful. I’d argue that the Featured Collections section is unnecessary (unless these are rotated) and could be removed to save space. You already have “Collections” in the new navigation bar.
  • Marketplace app tile: two bits of feedback here:
    • 1. I don’t think the vendor name is needed at this point, and it’s a waste of screen space. Do most customers really care who the vendor is at the beginning of their search? Also, if a prospective customer wants to find out who the vendor is, it’s just one click away within the product listing itself.
    • 2. the “Cloud Fortified” badge needs to be better integrated so it’s not just floating at the bottom of the tile. I know you said these are initial concepts and not final, so I appreciate that may change in future iterations
• Filters: a couple of comments here:
  • Left-hand navigation: on the new Marketplace homepage designs it seems the filters and categories in the left-hand navigation is disappearing completely. I’m not sure this is the best idea. Other app marketplaces still make use of this space and it’s right in front of the customer’s eyes when they first access the Marketplace. I think the main problem with the current Atlassian Marketplace is that there were just too many category options and the filters could do with a bit of refining. This doesn’t mean they should be removed entirely.
  • Keywords (not SEO related): As part of the app categorization changes recently rolled out, Marketplace Partners were required to choose up to 4 ‘keywords’ per app. Are there any plans to use these within the new filtering and search experience?
• Search: like what other Partners have said, it’d be great to see more improvements with the search experience. However, it’s important that any changes are communicated clearly (and early) as this will directly impact app discoverability. Being kept in the loop will ensure we can optimise our listings and align with any changes.
  • Just to upvote one of the previous comments, it would be great if the ‘More details’ section within app listings were finally included in search discoverability. This has been a gap for a long time now.
• Category pages: I feel these could be improved further to give customers more visibility of newer/lesser-known apps. Why not bring some of the sub-sections (e.g., Spotlight, Rising Stars, etc.) into the category pages themselves? This would give people more choice.
  • In its current implementation, my concern is that within the category pages, you’ll just have the same old apps appearing in the first 1-2 rows. Don’t get me wrong, I have nothing against any of the long-standing apps and they’re popular for a reason. But if we’re truly focused on improving the customer search experience and app discoverability, it would be nice to give the customer more ways to discover new apps, as well as those that have existed for a long time.
• Resources: what is intended to go into this section? Based on the current mock-ups, it does not seem like it will add much value. Use case related content that introduces different apps within that category would be useful, but it would need to be fair and balanced.
  • Company stats: I may have misunderstood this one so please excuse me if that’s the case. In theory, this is a good idea, but it could create a very uneven playing field. The larger Marketplace Partners can naturally post more “impressive” stats due to their size and longevity in the Marketplace. This creates an uneven and possibly unfair environment for the small/medium-sized Marketplace Partners

[EmilOdagiu]

@CraigWillson

Company stats: I may have misunderstood this one so please excuse me if that’s the case. In theory, this is a good idea, but it could create a very uneven playing field. The larger Marketplace Partners can naturally post more “impressive” stats due to their size and longevity in the Marketplace. This creates an uneven and possibly unfair environment for the small/medium-sized Marketplace Partners

I think they are intending to publish Atlassian Marketplace related stats, not individual partner stats. The logic is to boost the credibility of the marketplace as a whole.

@CraigWillson

Just to upvote one of the previous comments, it would be great if the ‘More details’ section within app listings were finally included in search discoverability. This has been a gap for a long time now.

This is a great point, though it should be tested properly for how it affects accuracy and precision of search results, and if the increase in coverage offsets any decrease in accuracy and precision. Descriptions usually being a ‘wall of text’ introduce a lot of opportunities for false matches. As long as description as an attribute is penalised (for example contributing 0.1 instead of 1 to the match rate) this should be an overall win as the false matches will be pushed to the bottom of the list.

[j.riedinger]

Hey there,
As vendors on the Atlassian Marketplace, we fully support the initiative to revamp the marketplace to enhance user experience and streamline processes. Many points of improvement we’ve considered important have already been discussed in this forum.

However, we express concerns regarding the proposed removal of installation numbers and reviews. These metrics are crucial for establishing trust and aiding in customer decision-making. *Despite acknowledging the risk of fake reviews, our experience with genuine customer feedback through quality products and excellent support has been overwhelmingly positive - and a and motivator for all those who put their heart and soul into working on the products or in support… Nevertheless, we strongly support Atlassian’s efforts to identify fake reviews and the vendors behind them and to take measures to prevent or sanction them. Fake reviews are fraud against all those honest vendors who invest a lot of time and resources in their support and really focus on the individual customer and not just their ranking.

For new apps, even from established vendors, gaining initial traction is challenging. Yet, these challenges are common across all marketplaces and industries, whether dealing with physical or digital goods and services, and form a basis for customer expectations. Installation counts and reviews serve as primary indicators of trust for customers.

Maintaining transparency in these aspects not only aids potential users in making well-informed decisions but also helps vendors uphold their credibility and trustworthiness.

[marc]

@ChoppaAditya

App categorization

Atlassian Marketplace currently has 37 categories and 33 collections which overlap with each other
and offer a disoriented experience. The existing taxonomy is inconsistent with some being too broad and some being too specific based on use cases, functions, and features. This hinders the app discovery process for customers as they are unable to find the right apps that align with their needs.

How does this relate to RFC-30: RFC-30: Restructuring Categories on Atlassian Marketplace ? We’ve added the required categories and keywords to our apps, but it seems the marketplace does not show the categories of RFC-30.

[UlrichKuhnhardtIzym1]

As a vendor we will be monitoring our apps discoverability and mapping to collections. How can vendors ensure that apps are listed correctly? How is Atlassian MP implementing transparency w.r.t to listings and what is the feedback loop for vendors to influence the mappings?

[nathanwaters]

@nathanwaters

They’ll kick and scream about any changes here…

@MattiaLomazzo

Although I understand their argument, I would not remove the installs and reviews, as some have suggested

@MarkusDoll1

I agree with Mattia’s perspective. The number of installations provides crucial context

@j.riedinger

However, we express concerns regarding the proposed removal of installation numbers and reviews

Called it