[RESOLVED] RFC-3: Enhanced reports visualisation

@remie,

Maybe “too soon” for RFCs? Please understand that @MalathiVangalapati is the brave first author (who didn’t also create the “RFC play”). Yes, over your many posts, I can agree that Atlassian needs to figure out how to “level up” our engagement with developers but is this the right time and place for this leveling up? Can we get a little slack while we figure out this new practice? I ask because you eventually point out RFC-3 (this RFC)…

Maybe you can let a few of these early “no-ops” just go? Especially when, by your own admission, they won’t affect you.

I accept your critique that we might be “dressing up” a technical chore as a feature. Or, that with a little more consideration, it could have been something more than a technical chore. But it doesn’t seem right to be so critical when we are starting this RFC practice. Meanwhile, here in the community and in other channels, you have rightly pointed out that we should assume that UI changes will affect developers. Here we are trying to signal that kind of a change, to which you react:

I can’t tell if this is meant as a “serious flaw” in the RFC. I don’t have enough understanding of Atlaskit myself to know what is the misuse. Or are you proposing that we should not use Atlaskit at all in this effort? It seems like you want to make this a conversation about Atlaskit but you already know that’s off-topic.

I also realize you are suggesting the “serious flaw” is prioritization and effort spent (wasted, from your perspective) on features that have no value (to you). While I want to be able to have that discussion (and we can try in other categories), I’m going to call priorities “off topic” for RFCs (I’ll clarify in RFC-1, and you can disagree with me over there, if I’m wrong).

Overall, I recognize some genuine engagement in there, but at the same time, this doesn’t feel like a great start to a new kind of community engagement. Was your comment a good example of “keeping it welcoming and safe”? Or was it a simple misstep as both authors and commenters alike learn how to work together?

3 Likes

Hi @marc ,

Thanks for the feedback, Can you please elaborate on the ask?
Are you referring to having an option to filter the current licenses and transactions based on monthly and annual subscriptions?

Hi @yvesriel ,

Thanks for flagging this, The issue with SEN, is already being addressed in the upcoming rollout.

2 Likes

Thanks @chhantyal , we have added this feedback to our backlog. Will update this thread as we make progress.

1 Like

keeping it welcoming and safe

This is a two way street @ibuchanan. As you might have digested from my comment, I do not consider the initial RFC to be welcoming nor safe.

Can we get a little slack while we figure out this new practice?

Actually no, Atlassian can’t. At least not from me. Although I fully support the format of RFC, it does not exist in a void, nor does it mean that the unequal power dynamics between Atlassian and Marketplace Partners is no longer applicable.

Please understand that @MalathiVangalapati is the brave first author (who didn’t also create the “RFC play”).

If Atlassian truly wants to make this safe & welcoming, please note that the author for any RFC will forever and always be Atlassian, with @MalathiVangalapati (or any other staff member) solely being the representative for the company.

It is unfair to put the burden of “figuring out” this practice on the Atlassian Marketplace community, especially given the history of both the state of communication and specifically the contents of this RFC.

The burden of guidance for Atlassian staff with regard to working with RFC lies solely with Atlassian. Atlassian could have reviewed the RFC prior to publishing or chosen a different (smaller, less problematic) topic for RFC.

But it doesn’t seem right to be so critical when we are starting this RFC practice.

Asking the Atlassian Marketplace community to hold back on their critique with regard to these RFCs event though they have a direct impact on their business, within the context of the unequal power dynamics of our relationship, is bordering emotional blackmail.

Now with regard to the contents of the RFC:

Of course I will be impacted. It means that I, and many other vendors, will have to continue to support my own haphazard solutions. Do you really think I want to maintain internal tooling for sales reporting? I get a far better sales dashboard from Stripe for a fraction of the revenue share I pay them.

No, my point is was that the RFC is saying that the overall experience will be improved because of the use of ADG3/AtlasKit. However, this is a false statement, as the vendor pages will not entirely be converted to ADG3 and past experiences have showcased a lack of understanding of how to work with AtlasKit components by the Atlassian Marketplace team. The serious flaw is the disconnect between the statement The implementation of the “Atlassian Design System” to ensure uniformity and consistency in all the marketplace touch points. will not be achieved by the proposed solution.

Yes, I think we should have this discussion in RFC-1 because I do wish to argue my case that we should be able to discuss prioritisation as part of an RFC.

Again, I really like the process of using RFCs, but if they are only going to be used to sugar-coat the underlying communications problems between Atlassian and the Atlassian Community, we might as well not spent any time on this. Unless Atlassian is genuinely interested in listening to and acting on feedback (even if this does not suit you), we are still left screaming in the void.

We are coming from a place of zero confidence. I can already guarantee you that whatever anyone says the Atlassian Marketplace team will continue moving forward with this RFC, and given that we still have no means to escalate this, why should we even bother to comment?

3 Likes

Just to clarify, I vote to reject this RFC based on the following serious flaws:

1. The problem statement does not actually describe a problem

The only possible hint of a problem I can find is that this is the mention of upgrading our tech stack to build a reliable and robust foundation for the marketplace partner portal.

However, this does not describe a problem? It does mention that upgrading to a newer tech stack will allow Atlassian to enable

  • The implementation of the “Atlassian Design System” to ensure uniformity and consistency in all the marketplace touch points.
  • Enhanced analytics and instrumentation capabilities to facilitate faster learning and the creation of tailored user experiences.
  • Additionally, improvement in the overall performance and page loading speed.

However, there is no problem statement describing the necessity for these three enhancements. Which problem does migrating to ADG3 solve? Which problem does enhanced analytics an instrumentation capabilities solve? Which problem does overall performance and page loading speed solve?

These statements are just put there in a matter of fact way as if they are self-explanatory and desirable. However, there is no mention of any customer demand for any of these three bullet points.

2. The proposed solution does not relate to the problem statement

The proposed solution mentions that it resolves feedback from the previous attempt to rollout this experience. Many of the features mentioned (like a trial period with the ability to switch back/forth) are not mentioned in the problem statement itself. It tries to sell the initial redesign (that was rolled back) based on it’s features and how it addresses the previous feedback.

However, because the problem statement never mentions any of these features to be an actual problem of the current experience, the redesign in and of itself is presented again as matter of factly and not something up for debate. Again, what problem does the proposed solution of the redesign try to resolve?

3. The actions do not clarify the process of gathering feedback
Although the actions call for partners to give feedback to the new redesign, with a 3 month trial period, there is no mention on how the Atlassian Marketplace team will respond to this feedback, whether it will be made public, if we can vote on it, etc. Nor does the team mention anything with regard to either extending the trial period or even rolling back the entire experience based on the feedback. As it is currently stated, the feedback loop is completely gratuitous. Atlassian does not commit in any way to doing anything with that feedback.

4 Likes

Hi @MalathiVangalapati ,
Yes, I would like to see which licenses and transactions are annual, and filter on annual and monthly.
At this moment we can see which transaction is monthly/annual by clicking at the details view, but it is not possible to e.g. list all monthly/annual transactions or list all monthly/annual licenses.

Similar, it would be great if there is a filter in the sales graph, which allows to show monthly/annual sales (similar in principle on how you can view sales by region, but using monthly/annual instead of regions).

Thanks for following up!

2 Likes

Hi Remie,

Thank you for the detailed feedback!

Our focus is primarily on upgrading our tech stack to build a reliable and robust foundation for reports. In the process, we see a merit in enhancing the user experience.
There is wasted effort in trying to recreate older user experience on the new stack hence we prioritised the partial makeover of experience along with the tech stack upgrade.

We’ve had limitations with the old tech stack and upgrading the experience will aid us in delivering value faster to the partners.In this process, we are incrementally migrating the experience to adhere to Atlaskit guidelines and hence you will see a few components that are still a part of the old guideline.

With these changes, we are enhancing our internal analytics and instrumentation to facilitate faster learning to be able to serve partners better with tailored experiences.

Again to reiterate, this is just a FIRST STEP and not the end of Reports enhancement.

On the point of ARR/MRR, a RFC is in works and I will be publishing it in ~4weeks.

6 Likes

Morning @MalathiVangalapati - appreciate you publishing this RFC.

Just to clarify, would this redesign accommodate any changes in the underlying data and insights? Specifically a shift from cash to accrual reporting?

For instance, we are leveraging SaaS metrics on an accrual basis (MRR, Logo Churn, $ Churn, etc) to smooth out seasonality. Today that understanding is provided in our own systems, so I’m not fussed either way, yet I wonder if that is the direction Marketplace is heading.

Thank you Malathi, have a good day,
Nick Muldoon
Easy Agile

3 Likes

What kind of “warranty” commitment is Atlassian making during this period? 3 months and nothing being improved and then forced out to prod is something we have experienced in the past.

Will there be any ability to toggle from old to new and back if the new one has critical flaws that impact us?

1 Like

Unclear whether I will still be able to:

  • easily see the monthly sales sum by hosting type,
  • back to 2 years or more,
  • up to the immediate time (It was a drawback of the last redesign),
  • and save that URL so I don’t have to select all boxes all the time,
  • Plus, I gave up hoping for quality polish of UIs,

Therefore, if one asked me to vote, I’d vote against.

If they want to upgrade, they could have just created a new chart type and see whether people select that chart type by default.

Further to @nick’s comments, the existing Marketplace charts are not particularly useful for subscription revenue. Cloud is the future, so dressing up the old charts by adding titles and legends does not make them any more useful. For cash-based reporting, having (say) the net sales number for a specific month is helpful, but that is best expressed as a tabular report and not a chart.

What’s the whole point of displaying any of this information graphically? To visualize trends. But the Marketplace reports don’t help you visualize trends, because the hodgepodge of monthly and annual purchases do not let you see MRR or any vaguely-equivalent metric that might actually expose a trend.

Let me give you an example. Here is what Marketplace output looks like for Cloud sales:

This is not particularly helpful. If I want to use it to calculate cashflow, I need the actual numbers (so again, we’re talking about a table) rather than a pretty chart. And the line does not tell any story, other than cashflow being inconsistent.

Here is what that exact same data looks like when the sales are broken down to express MRR and to allocate subscription revenue to the month in which it belongs:

This is actually useful and actionable. This is also why I go elsewhere for Marketplace data reporting. Investing time in making the charts look nicer doesn’t help unless you are charting relevant data.

6 Likes

@scott.dudley - thanks for the detailed examples, you explained why it is important to have the accrual reports. Champion!

Hi @nick ,

This RFC is currently not addressing any changes to the underlying data.
That being said , totally agree with you on the need for surfacing more SaaS metrics .
This is on our roadmap and we will publish an RFC as we make progress.

Best,
Malathi

1 Like

@scott.dudley , appreciate the detailed examples.
We will share more updates on the SaaS metrics as we make progress.

Best,
Malathi

2 Likes

@MalathiVangalapati from your responses to this thread, it is clear that the Atlassian Marketplace team is taking an iterative approach to refactoring the vendor reporting pages.

However, what is not clear is whether Atlassian has heard and understood what the Atlassian Marketplace Partner community is trying to say in this thread.

We understand that Atlassian is dealing with a legacy tech stack and that it is imperative that this needs to be resolved. But there is no reason to decouple this from a complete rethinking of the Atlassian Marketplace vendor reporting pages.

The RFC tries to sell an intermediate solution as a given: we need to do A in order to be able to do B. But that is a false dilemma. You can actually come up with a meaningful dashboard, SaaS metrics, reporting, etc AND combine that with the tech stack upgrade.

The current proposed (intermediate) solution brings little to no benefit and potentially a worse experience than the current situation. There is no gain for us.

Based on the responses in this thread, I think Atlassian should be open to the idea that the Atlassian Marketplace Partner community would rather wait another 6 months for a significant redesign than have the proposed (intermediate) solution now.

Thanks @MalathiVangalapati focusing the RFC format (being an early adopter is never easy) and for continuously driving the improvement of partner experience for reporting in the past years. As for RFC-3 I have only a few remarks:

1) Priority
I understand that Atlassian invests in unified partner experience and different partners will see different value in that. You list “The implementation of the “Atlassian Design System” to ensure uniformity and consistency in all the marketplace touch points.” as the first (primary?) reason for this change and “Consistent partner experience” as the first (primary?) result for partners. I would have loved to see more substantial changes being proposed than fresh paint. I do, however, understand that bigger changes and more RFCs are ahead, covering some of the asks above (many of those Communardo would also back). It’s been covered above and I understand Atlassians position here although it might not be what some of us have hoped for.

2) Data changes
I understand that underlying data will not change. Neither will the API behave differently, nor will the values presented in the new UI differ from what we are seeing now. If that is indeed the case that’s great and important. Otherwise longer planing cycles for partners would be required as it’s been the case in the past.

3) Search
I’m not sure if this is covered by the point above: the search (licenses/transactions) is broken. Any special character (&, ü, ä, ß, any many others that are very common) will break the result. Will this change improve any of that?

4) Transaction details
This is a very welcome change, looking forward to play with that. Using a new tab makes sense here as it does not remove the previous search results. Thanks for that one!

Thanks and have great day!
David

2 Likes

We’re a little overdue but today I’m closing this thread as the discussion phase has expired. As of today, this thread had 865 views, with 46 likes. I’d like to thank the dozenish commenters who contributed 22 comments. We really appreciate that you took the time to tell us what you think about our reporting improvements in Marketplace, and to learn about the RFC process itself. For anyone with thoughts about this example in context of the overall RFC process, RFC-1 about RFCs is still open for another week (and we’ll be open to input on RFCs at any point).

I’ve had a quick conversation with @MalathiVangalapati who is preparing a resolution to this topic. Please stay tuned for that on or before 10 March 2023.