RFC-29: App Access Rule - Revised followup to New Data Access APIs

Project update

This is a revision of RFC-25 App Access Rule Update we issued regarding the implementation details of two new APIs announced in RFC-14 App Access Rule. Based on feedback regarding our previous update we have cancelled RFC-25 and replaced it with this one - a revised update if you will. :slight_smile:

Please respect our community guidelines : keep it welcoming and safe by commenting on the ideas not the people; keep it tidy by keeping on topic; empower the community by keeping comments constructive. Thanks!

RFC timeframe

  • Publish: 16 Oct 2023
  • Resolve: 23 Oct 2023

RFC changes

For those familiar with the now-cancelled RFC-25, the substantive changes are:

  • The architectural flow for an app to become aware of which objects have been blocked has changed - rather than publish a notification event and expect each subscribing app to resolve this to whichever objects are affected for them, instead we will publish notification events that identify the objects to which the app has lost access.
  • We will not publish notification events when an app gains access to an object for which it was previously blocked.
  • We will not build an API endpoint to determine for an individual page or issue whether it is blocked by policy.

APIs summary

Semantics

In our initial RFC, we outlined how the app access rule would block app access to specific Confluence spaces or Jira projects. At that time we planned to implement the block at the space and project level.
We have since decided that we can better optimize for customer flexibility in future iterations of this feature if we instead block apps from accessing data at one layer lower in granularity, meaning apps will not be able to access content on pages, issues, in comments, or in attachments, nor in other content types such as Whiteboards, Blog posts or Databases. Customers will currently still set rules at the space and project level and will be notified about what data is covered or excluded via documentation and in the UI directly.
While this will leave apps with access to a few pieces of user-generated content like space names, the majority of user-generated content in these spaces and projects will still be protected, and we will be able to enable more granular controls for customers in the future to better align with other rules under data security policies.

Note that ‘access to an object’ also implies access to the object’s comments, attachments, etc.

Note also the use of terms such as workspace, container and object - please see the appendix for more information.

Events/webhooks

An app’s access to data within a workspace may be affected whenever an Administrator activates, deactivates or updates an app access rule within a data security policy. Should an app lose access to the pages or issues in a space/project within a workspace, events which identify the inaccessible pages/issues will be published, allowing the app to react to the change accordingly.

API endpoints

Apps are provided with an API to determine if App Access Rules are active for a given workspace, so they can display appropriate messaging to end-users.

Apps are also provided an API to query by space or project, if this is known, whether pages and issues in that space/project are still accessible, again so they can display appropriate messaging to end-users.

We previously signalled an intention to provide an API which would allow an app to query whether an individual page or issue is blocked by policy or not. We believe that the events/webhooks provided meet the need of an app to know when a page or issue has become blocked. An app is always able to fetch a page or issue from the respective product API - if the object cannot be retrieved (whether a 403 or 404 response code is received), the net effect is that the app is not able to access the object.

Interactions

This high level diagram shows how the interactions between Apps and App Access Rules are mediated through web APIs and events delivered via webhooks.


Events/webhooks

An app’s access to data may be affected by changes directly upon a policy (i.e. policy rule configuration), or by changes to configuration data (such as classification of containers or objects) that is passed in to policy evaluation. There are various ways through which these changes may be effected, and these will likely increase over time. It is also notable that changes to policy rules or configuration data is asymmetric with regard to the impact on apps and their access to individual pages or issues - i.e. one small configuration change can result in widespread changes to apps’ access to data.

Rather than publishing ‘policy change’ events we will publish events that describe the actual effects on an app, that result from those policy changes. Given the asymmetry between policy changes and (an app’s access to) the pages and issues affected, we will publish notification events that contain sets of object identifiers. One policy configuration change may result in many events being published, each of these events containing many object identifiers - all the events together will cover the entire set of objects the app has lost access to.

AppAccessToObjectsBlocked event

An app that needs to know when it has been blocked from accessing objects will need to subscribe to the AppAccessToObjectsBlocked event which will include an array of identifiers. The app should iterate through the object identifiers in the event (all of which are inaccessible to the app), and for each object respond appropriately, such as deleting data or disabling background synchronization.

Sample event

An event will look similar to this:

{
  "specversion": "1.0",
  "type": "avi:ecosystem.app_policy:blocked:app_access_to_objects.v1",
  "source": "com.atlassian/ecosystem.app_policy",
  "id": "7a6796c0-746d-4504-92cd-819eca234306",
  "time": "2023-10-24T08:08:08Z",
  "data": {
    "site": { 
      "url": "https://site_name.atlassian.net"
    },
    "blockedObjects": [
      {type: "page", id: "12345"},
      {type: "page", id: "23456"},
      {type: "blog", id: "34567"}
    ]
  }
}

Event schema

The proposed event schema is defined using the AsyncAPI specification. The nicest way to read this schema is to go to AsyncAPI Studio and replace the sample schema with (copy/paste) the following schema:

asyncapi: '2.6.0'
tags:
  - name: ecosystem
id: 'urn:com.atlassian.ecosystem.app_policy.webhooks'
info:
  title: App Policy Service
  version: 0.1.0
  description: Atlassian service which publishes App Policy events to App webhooks.
  license: 
    name: Apache 2.0
    url: https://www.apache.org/licenses/LICENSE-2.0
  contact:
    name: Justin Thirkell
    email: jthirkell@atlassian.com
    url: https://community.developer.atlassian.com/t/rfc-27

channels:
  atlassian/webhooks:
    subscribe:
      bindings:
        http:
          type: request
          method: POST
          headers:
            content-type:
              type: string
              enum:
                - 'application/cloudevents+json; charset=UTF-8'
      summary: Receive notifications of changes to App Access Rules that affect App access to data.
      message:
        oneOf:
          - $ref: '#/components/messages/appAccessToObjectsBlocked'

components:
  messages:
    appAccessToObjectsBlocked:
      title: Event - App Access to Objects Blocked
      description: |-
       Published when an App's access to a set of objects such as Confluence pages, blogs or whiteboards, or Jira issues, has been blocked.

       It could be that the App has been added or removed from a Policy, or the Containers covered by a Policy associated with that App have changed, or some other cause.
      payload:
        type: object
        required:
          - specversion
          - id
          - source
          - type
          - time
          - data
        properties:
          specversion:
            type: string
            description: The version of the CloudEvents specification which the event uses.
            enum:
              - "1.0"
          id:
            type: string
            minLength: 1
            description: Uniquely identifies the event.
          source:
            type: string
            format: uri-reference
            minLength: 1
            description: Identifies the context within which an event occurred.
          type:
            type: string
            minLength: 1
            description: Describes the type of event related to the originating occurrence. The event type is suffixed by the event type version.
            examples:   
              - "avi:ecosystem.app_policy:blocked:app_access_to_objects.v1"
          time:
            type: string
            format: date-time
            description: Timestamp of when the occurrence happened. Must adhere to RFC 3339.
          data:
            type: object
            properties:
              site:
                type: object
                properties:
                  url:
                    type: string
              blockedObjects:
                type: array
                items:
                  type: object
                  description: Set of blocked objects and their types
                  properties: 
                    type:
                      type: string
                      description: Object type
                    id:
                      type: string
                      description: Object identifier

      examples:
        - name: AppAccessToObjectsBlocked
          headers:
            content-type: application/cloudevents+json; charset=utf-8
          payload:
            specversion: "1.0"
            type: "avi:ecosystem.app_policy:blocked:app_access_to_objects.v1"
            source: "atlassian.com/ecosystem"
            id: "7a6796c0-746d-4504-92cd-819eca234306"
            time: "2023-10-24T08:08:08Z"
            data:
              site: 
                url: "https://site_name.atlassian.net"
              blockedObjects: 
                - type: "Page"
                  id: "123456"
                - type: "Page"
                  id: "234567"
                - type: "Blog"
                  id: "345678"

Additional information

Event AVI

The event AVI is still subject to confirmation but at this stage will be avi:ecosystem.app_policy:blocked:app_access_to_objects.v1.

Cloudevents

We’ve noted Partners’ concerns about the use of another standard and potential complexities involved. We are nevertheless comfortable that aligning event schema to the Cloudevents specification should be largely transparent, with negligible impact on Partners. We know that Atlassian lacks consistent standards for error statuses and messages (including event metadata) across our public APIs, and we recognise this is a longstanding pain-point for developers, particularly developers attempting to build resilient apps against APIs they have not worked with previously. We believe use of a standard to consistently represent event metadata can only be a good thing. We’re choosing to use a mature public standard here because we believe it is fit for purpose and prefer to adopt a public standard where possible. Our reasoning for selecting Cloudevents specifically is as follows:

When we consider the properties one would normally expect to see in an event we could say;

  • time and id are typically required.
  • Separation of header concerns from a message payload has proven to be a useful pattern - so we could add another attribute called data to encapsulate the event payload.
  • We would like to support event schema versioning - so we could add another attribute called the event type, within which we specify the event schema version.

Now we have defined an event schema that is going to look something like this…

{
  "type": "the.event.type.v1",
  "id": "7a6796c0-746d-4504-92cd-819eca234306",
  "time": "2023-10-24T08:08:08Z",
  "data": {
  }
}

And at this point we have (apart from specversion) replicated the event schema specified by the Cloudevents structured format. If events are (as they will be) formatted as json, it is not necessary to know about or use any Cloudevents libraries - at the end of the day the event is still a regular json object. (Well, to be perfectly accurate the media type of the event will be application/cloudevents+json but that should make no difference to json parsers.)

When we consider the sample event above we expect Partners will be able to parse and handle events in the same way they normally do. And those who have need for Cloudevents, either now or in the future will be able to do so. (We can confirm the CloudEvents javascript SDK is compatible with Forge. The uuid package is only used by CloudEvents to create an event id if one does not already exist.)

Semantically duplicate events

We expect that our initial implementation of events for App Access Rule will result in sending AppAccessToObjectsBlocked events containing some page or issue identifiers for which the app was already blocked from accessing, regardless of the latest policy or policy configuration data change. This is a side-effect of our having internalised the iterative loop where we determine the effects of policy changes. We plan to improve and optimise this loop which will result in efficiency gains for all subscribing apps, but apps should expect to receive events identifying blocked objects to which the app already did not have access.
If an app is notified that it is blocked from a given page or issue and the app was already blocked from accessing that object, the effective change is a noop.

AppAccessToObjectsUnblocked

From replies to the original RFC and subsequent conversations with Partners and Customers we understand that the most problematic scenario is when an app is blocked from accessing an issue or page. Firstly, customers expect that apps will be afforded the means to remove data related to objects the app has been blocked from accessing. Secondly, if the app is not advised the object is blocked users may be confused when presented with app functionality that is broken or displays stale data. It is imperative that apps be enabled to know when access to an object has been blocked.

Feedback received and the use cases we are aware of, so far appear to be heavily weighted towards the need to know when content becomes blocked (inaccessible), as opposed to needing to know when content becomes unblocked. Therefore we propose to publish AppAccessToObjectsBlocked events when content is blocked, and not send apps equivalent notifications when content is unblocked. Please let us know if you have a specific use case for requiring ‘unblocked’ events that cannot be resolved by a user triggering a resync.

We would like to provide some additional explanation here. There is an asymmetry in our ability to determine the effect a policy or configuration data change may have on an app’s access to pages or issues. Because App Access Rules are defined via blocklists, it is relatively easy to determine that the effect of a change is to block an app’s access to objects. However, the reverse is not true - determining exactly if (and which) pages or issues are now unblocked is more computationally expensive, requiring iteration through all potentially affected objects and evaluation of App Access Rules to determine which of these are not blocked.

In summary there are several reasons we are choosing not to deliver AppAccessToObjectsUnblocked events in the first version of App Access Rules:

  1. An app’s ability to recover in the case of an unblocked object being out of sync is relatively simple - just refetch the page or issue. This is not a recovery mechanism available to an app when presented with a stale object that is potentially blocked.
  2. As discussed, determining when an app’s access to an object has been truly unblocked is more complex due to the potential existence of multiple policies and blocklist-based policy rules. We would like to defer this functionality so we can concentrate on delivering a good AppAccessToObjectsBlocked event/webhook experience.
  3. Partner needs appear to be heavily weighted towards receiving AppAccessToObjectsBlocked events, and much less so towards AppAccessToObjectsUnblocked events.

Rationale behind changes in approach

It is clear that our approach to meeting Partner requirements for publishing events has varied wildly between iterations - this has been as frustrating for us as it will have been perplexing for you. :sweat_smile: To give you some assurance that we have been a) listening and b) working hard to provide the best possible solution, we describe here the broad approaches we considered.

When policy changes, someone somewhere has to iterate over a bunch of pages or issues and for each, determine if an app has been blocked from accessing it - an iterative loop if you will.

  1. Our first solution was not published but worked on internally - and it attempted to fully encapsulate this iterative loop, publishing granular AppAccessToObjectBlocked and AppAccessToObjectUnblocked events (note: “object” not “objects”). We would publish one event for each blocked or unblocked object. While the self-contained nature of these events (not requiring any additional calls to resolve which object was affected) was appealing, the sheer volume of events that could be generated and published was of concern, not only for our event publishing capabilities but also the ability of apps to actually handle all these events. In addition we were concerned about the complexity of determining for sure that an app’s access to an object was truly unblocked, given our fundamental blocklist approach to App Access Rules, and the potential for multiple policies to be in effect.
  2. Our second solution swung the pendulum the other way. We would publish a simple ‘something has changed’ event and fully externalise the iterative loop. While apps would receive just one event, they would be responsible for executing the loop - checking all known objects against an API to determine if they were blocked or not. If an app contains data on every page or issue in a workspace that’s still a lot of traffic… A comment on our previous RFC queried the feasibility of implementing the iterative loop within Forge, and after investigating this we agree that implementing a very large scale event loop within Forge has some limitations we would need to solve.
  3. Our third attempt as presented in this RFC, strikes a balance which we believe provides Partners with the functionality they have asked for, while regaining the encapsulation that will allow us to take on more of the load ourselves and over time build additional efficiencies into the iterative loop. The net result should be events that are simple to process because they contain all the information needed by the app to take action without needing callbacks to retrieve additional data. We have also taken the time to ensure this solution can handle any changes to App Access Rules or Policy in the future without requiring changes to the event schema or app-side implementation.

App Policy API

Http response codes

Partner replies to our first RFC indicated a preference for product APIs to alter their response codes to indicate when an app has been blocked from doing something it otherwise would be allowed by its permissions to do.
And indeed our first solution attempt here did look at whether we could modify API response codes to allow apps to know when and how API responses were affected by the presence of App Access Rules. This would ideally have been a 403 response with a body that indicated the precise authorization failure reason, although that would still not have been sufficient for handling when items in a collection response were filtered by App Access Rules. Anyhow, it turned out to be unviable to have both Jira and Confluence make the required changes in the timeframe available to us.

Use of GraphQL

There are an increasing number of graphQL endpoints available through Atlassian APIs and this is largely unavoidable - by using graphQL we are choosing to align with the overall direction of Atlassian APIs.

However, Forge apps will be provided an sdk which abstracts away the mechanics of invoking a graphQL endpoint so it is transparent to apps. And Connect apps will be able to use the REST API endpoint.

Endpoint to get the status of provided objects

Our work to define solutions has looked at specific use cases and how to solve for these, including reviewing some apps ourselves and talking to Partners. There are no changes to the App Policy API described in the previous RFC except that we have now decided not to build an API endpoint to allow an app to determine, for an individual page or issue whether it is blocked by policy. Our reasoning is as follows:

The use case here is that the app needs to know if an object is not accessible so that it can provide informed messaging to the user.
We believe simply calling an existing Confluence or Jira product API results in the same net effect that we had intended for this proposed new API endpoint - that is to allow the app determine if an object is accessible or not.
Whether the response is a 403or 404, and whether this is caused by an object simply not existing, being inaccessible because of permissions, or being inaccessible because of policy - the result is the same; the object is inaccessible.

We have evaluated the benefit of such an endpoint to be outweighed by the additional cost and have accordingly deprioritised it. The proposed APIs are otherwise unchanged and are as follows.

APIs schema

The proposed API schema is defined using the OpenAPI specification. The nicest way to read this schema is to go to Swagger Editor and replace the sample schema with (copy/paste) the following schema:

openapi: 3.0.3

info:
  title: App Policies API
  version: "0.1.0"
  description: Ecosystem service which publishes events to App webhooks.
  contact:
    name: Justin Thirkell
    email: jthirkell@atlassian.com
    url: <https://community.developer.atlassian.com/t/rfc-27>

servers:
  - url: "<https://your-site.atlassian.net/rest/atlassian-connect/latest/app-policies/data-classifications>"
    description: Jira
  - url: "<https://your-site.atlassian.net/wiki/rest/atlassian-connect/latest/app-policies/data-classifications>"
    description: Confluence
    

paths:
  /containers:
    get:
      tags:
        - containers
      summary: Get the status of provided containers
      operationId: getContainerDecisionStatus 
      parameters:
        - name: projects
          in: query
          description: Project ids. Accepted only if app is installed into Jira.
          explode: false
          schema:
            type: array
            maxItems: 20
            minItems: 1
            items:
              type: integer
              minimum: 1
        - name: spaces
          in: query
          description: Spaces ids. Accepted only if app is installed into Confluence.
          explode: false
          schema:
            type: array
            maxItems: 20
            minItems: 1
            items:
              type: integer
              minimum: 1
      responses:
        '200':
          description: Successful operation
          content:
            application/json:
              schema:
                type: object
                properties:
                  containers: 
                    type: array
                    maxItems: 20
                    items:
                      $ref: '#/components/schemas/Result'
                
        '400':
          description: Invalid ID supplied
      security:
        - atlassian_connect_jwt:
            - READ
            
  /constraints:
    get:
      tags:
        - constraints
      summary: Check if the app is subject to policy restrictions in a given installation context
      operationId: checkActiveConstraints 
      responses:
        '200':
          description: Successful operation
          content:
            application/json:
              schema:
                type: object
                properties:
                  constraints:
                    $ref: '#/components/schemas/Constraints'
      security:
        - atlassian_connect_jwt:
            - READ
  
components:
  schemas:
    Result:
      required:
        - id
        - decision
      type: object
      properties:
        id:
          type: integer
          format: int64
          example: 14800
        decision:
          $ref: '#/components/schemas/Decision'
    Decision:
      required:
        - status
      type: object
      properties: 
        status:
          type: string
          description: Policy decision status
          enum:
            - ALLOWED
            - BLOCKED
    Constraints:
      type: object
      properties:
        hasConstraints:
          type: boolean
  securitySchemes:
    atlassian_connect_jwt:
      type: apiKey
      name: authorisation
      in: header

Alternatively, here is a short overview of new API methods:

Get the status of provided containers

Request (Jira Connect App):

GET /rest/atlassian-connect/latest/app-policies/data-classifications/containers ?projects=id1,id2

Request (Confluence Connect App):

GET /wiki/rest/atlassian-connect/latest/app-policies/data-classifications/containers ?spaces=id1,id2

Response:

{
  "containers": [
    {
      "id": "id1",
      "decision": {
        "status": "ALLOWED"
      }
    },
    {
      "id": "id2",
      "decision": {
        "status": "BLOCKED"
      }
    }
  ]
}

Check if the app is subject to policy restrictions in a given installation context

Request (Jira Connect App):

GET /rest/atlassian-connect/latest/app-policies/data-classifications/constraints

Request (Confluence Connect App):

GET /wiki/rest/atlassian-connect/latest/app-policies/data-classifications/constraints

Response:

{
  "constraints": {
    "active": true
  }
}

GraphQL API for Forge and OAuth 2.0 apps

Forge and OAuth 2.0 apps will not be able to access the REST API described above, it is reserved for Connect apps only. A GraphQL API will be provided instead. Additionally, we are going to provide a new SDK wrapper as part of the @forge/api package, similar to Storage API, to simplify integration for Forge apps.

API contract

Type definitions

type Query {
  ecosystem: EcosystemQuery
}
type EcosystemQuery {
  appPolicies: EcosystemAppPolicies
}
type EcosystemAppPolicies {
  dataClassifications(id: ID!): EcosystemDataClassificationsContext
}
type EcosystemDataClassificationsContext {
  id: ID!
  containers(ids: [ID!]!): [EcosystemDataClassificationPolicyResult]
  objects(ids: [ID!]!): [EcosystemDataClassificationPolicyResult]
  hasConstraints: Boolean
}
type EcosystemDataClassificationPolicyResult {
  id: ID!
  decision: EcosystemDataClassificationPolicyDecision!
}
type EcosystemDataClassificationPolicyDecision {
  status: EcosystemDataClassificationPolicyDecisionStatus!
}
enum EcosystemDataClassificationPolicyDecisionStatus {
  ALLOWED
  BLOCKED
}

Get the status of provided containers

Query:

query getContainersDecisions($installationContext: ID!, $containerIds: [ID!]!) {
  ecosystem {
    appPolicies {
      dataClassifications(id: $installationContext) {
        containers(ids: $containerIds) {
          id
          decision {
            status
          }
        }
      }
    }
  }
}

Input:

This example demonstrates usage of confluence space ARI to uniquely identify scopes

{
  "installationContext": "ari:cloud:confluence::site/{siteId}",
  "containerIds": [
    "ari:cloud:confluence:{siteId}:space/{spaceId}"
  ]
}

Output:

{
  "data": {
    "ecosystem": {
      "appPolicies": {
        "dataClassifications": {
          "containers": [
            {
              "id": "ari:cloud:confluence:{siteId}:space/{spaceId}",
              "decision": {
                "status": "ALLOWED"
              }
            }
          ]
        }
      }
    }
  }
}

Get if there are any constraints affecting the app in a given context

Query:

query getAppConstraints($installationContext: ID!) {
  ecosystem {
    appPolicies {
      dataClassifications(id: $installationContext) {
        hasConstraints
      }
    }
  }
}

Input:

This example demonstrates usage of workspace ARI to uniquely identify app installation context.

{
  "installationContext": "ari:cloud:confluence::site/{siteId}"
}

Output:

{
  "data": {
    "ecosystem": {
      "appPolicies": {
        "dataClassifications": {
          "hasConstraints": true
        }
      }
    }
  }
}

Scopes

For Connect apps, the App must have the relevant Connect READ scope. We do not expect this to be a change for any affected Apps.

For Forge and OAuth 2.0 apps, read:confluence-space.summary or read:project:jira scopes will be required to access container-level information (depending on the type of container). No additional scopes will be required to check if the app is affected by policy constraints in a workspace.


Appendix

ARIs

This RFC mentions ARIs and these may be unfamiliar to readers. An ARI (Atlassian resource identifier) is a globally unique identifier. ARIs can identify any entity, for example:

  • Jira issues
  • Jira projects
  • Confluence pages
  • Trello boards
  • Atlassian users, (global entities)

Workspace ARI examples:
ari:cloud:jira:ad95fada:workspace/114dfb50
ari:cloud:confluence:1d52f2ee:workspace/cfdef2de

Object ARI examples:
ari:cloud:confluence:ad95fada:page/123456
ari:cloud:jira:ad95fada:issue/123456
ari:cloud:confluence:ad95fada:blogpost/123456
ari:cloud:confluence:ad95fada:whiteboard/123456

ARIs are particularly important when providing APIs that span multiple products, such as in this RFC which specifies an API that will be used for both Confluence and Jira.

An ARI is an opaque identifier, meaning that although it is human-readable, it should only ever be handled atomically, and not parsed into its constituent segments except via Atlassian sdks. The structure of ARIs may change over time and therefore manual parsing of ARIs is strongly discouraged.

Containers

Both Confluence and Jira define a model that involves grouping objects into containers. For Jira a container of issues is called a project, for Confluence a container of pages is a space.

When defining APIs that span multiple products such as in this RFC, it is useful to define a term that may be used for any and all products. In this case, it is a container, and so the APIs specify the use of containerAri in various places.

A containerAri is an abstraction that in practice maps to concrete ARIs such as a spaceAri or a projectAri. So whenever you see container in API definitions you can think of this as a space or a project, whatever makes sense in your particular context.

Objects

The corollary of using the term container as an abstraction for projects and spaces is the term objects as an abstraction for Jira issues and Confluence pages.

So whenever you see object in API definitions you can think of this as an issue, a page, a whiteboard, a blog, whatever makes sense in your particular context.

2 Likes

I’m not sure if this is appropriate within the scope of this RFC or not, feel free to tell me so.

Has this proposal been evaluated from an economic impact perspective? Clearly no hard numbers since this doesn’t exist yet, but based on my experience with the Allow Access UI in Forge, I conservatively expect this to change to double our support costs and have a significant negative impact eval to net new conversions.

2 Likes

Now that this RFC is trending towards object-level granularity rather than container-level granularity, this seemingly creates a new requirement to handle all of the different possible object types.

For example, the proposal refers only to “pages” in Confluence…but Confluence spaces also support blog posts, whiteboards and databases. All of these are independent object types that cannot be commingled in the V2 APIs, so they need to be broken out separately. How will this be handled?

3 Likes

Thanks for pointing that out Scott - you are correct.

While internally we have been working with product-agnostic identifiers we were trying to make the explanations product-specific to improve understandability as this is a complex area. We then inadvertently drew this product specificity back into our event schema and examples.

I have updated the RFC - you will see the AppAccessToObjectsBlocked event now reflects again the non-specificity of the solution we designed. Thanks again. :slight_smile:

1 Like

@ibuchanan, can you please outline the process for escalating an RFC?

As mentioned on other RFC threads as well, individual Atlassian teams make overarching decisions that might seem logical to them as a team (and often involve a New Shiny Thing :tm:) but erode the trust in the system architecture as a whole.

It would be great if we could somehow signal this to a higher level, some sort of architectural committee or something, which would be able to override these decisions on behalf of the partner community.

In this case, the majority of partners that responded to the previous RFC said that they were against using Yet Another Standard unless there would also be a plan to migrate other parts of the Atlassian event system. Atlassian has decided to move forward regardless of our concerns, which means that I would like to know if there is any arbitration process available for partners.

EDIT: this post was flagged by the community as Off Topic, which I can somewhat understand. I have created a new topic for this but I hope it can stay here as I would like to maintain context. Also, once the policy is clear, I do want to apply it to this RFC, making it somewhat on-topic again.

@JustinThirkell you are now again mentioning ARI, but you have not responded to any of the questions with regard to ARI in the previous RFC

Can you please make sure that you address these questions in this RFC?

1 Like

RFC-25 was (rightfully) closed, so I’d like to ask here on RFC-29 for feedback on my follow-up question regarding locked custom fields, (@AngelinaIgnatova, @JustinThirkell. Here my question on RFC-25:

Thanks.

Hi @JustinThirkell

Thanks! One other concern: in Confluence, the type of the “id” field needs to be string and not integer, because some contentIds already exceed the JavaScript MAXINT value.

2 Likes

Hi @scott.dudley

Thanks - that I did not know! I have updated the event schema to reflect this constraint.

cheers

Hi Boris,

I conservatively expect this to change to double our support costs

Oh, that’s interesting. These APIs are being provided to alleviate Partner concerns that App Access Rules will result in an increased support burden. I hope that is the case for you also. :slight_smile:
If you are questioning the App Access Rule initiative itself, this is being provided to meet Customer demand - we acknowledge App Access Rules do result in additional complexity and implementation costs for both Atlassian and our Partners.

I expect this to have a significant negative impact eval to net new conversions

We expect App Access Rules to unblock additional cloud Customers - perhaps Partners who integrate with App Access Rules may expect a positive impact for net new conversions? To your point, I agree it is difficult to know for sure until we have some hard numbers.

Just like @BorisBerenberg I believe that this will increase our support cost a lot even if we implement the appropriate api’s since we’ll get support tickets of "Why can’t I find this [project|this issue|board|filter] in your app. We get this to some degree today and spend a fair bit of time debugging with the customer with limitations in the the various search api’s that we have access to (granted we’ve gotten really good at it now and have good documentation now).

While we will more than likely put in appropriate warning messages etc - there will be corner cases and there will always be end users that skip the warnings and in-app messaging.

Can we get an understanding of what the in-app experience is for the end user/container owner/instance admin? Ideally there will be tools such as the “Why can’t I see my field” tool on issues for this type of thing (Why can’t app X see this item). And documentation - the more documentation Atlassian can provide the better so we can deflect the end user to this.

Will this feature be available in the developer canary instances?

It is for this use case (i.e. apps which present to a user the set of containers in a workspace) that we are providing an API endpoint to retrieve containers and whether they are blocked by policy or not.

For individual objects (the same use case but presenting to a user a set of objects rather than containers) we believe the overhead of evaluating policy for every object in the view is impractical. For this reason we are providing an API endpoint to retrieve an indication of whether policy is in effect at all within the workspace.

In both cases the purpose is to give the app enough context to know whether to display messaging to the end-user.
And of course if an app is blocked by policy from rendering at all then the UI will render an explanatory message.

If these APIs are insufficient for your use cases then we welcome additional concrete examples. :slight_smile:

This reply is in response to ongoing expressions of concern about Cloudevents. In conjunction with the explanatory note about Cloudevents in this RFC I will explain by way of example show how the use of Cloudevents should be completely transparent to Partners, who should be able to work with plain json objects as usual.

Schema diff

If we were to ignore Cloudevents entirely (while still including event schema properties we feel are desirable), an example AppAccessToObjectsBlocked event would look like this;

{
  "type": "avi:ecosystem.app_policy:blocked:app_access_to_objects.v1",
  "id": "7a6796c0-746d-4504-92cd-819eca234306",
  "time": "2023-10-24T08:08:08Z",
  "data": {
    "site": { 
      "url": "https://site_name.atlassian.net"
    },
    "blockedObjects": [
      {"type": "page", "id": "12345"},
      {"type": "page", "id": "23456"},
      {"type": "page", "id": "34567"}
    ]
  }
}

If we then add whatever is necessary to align with Cloudevents, an example AppAccessToObjectsBlocked event (and what we propose in this RFC) then looks like this;

{
  "specversion": "1.0",
  "type": "avi:ecosystem.app_policy:blocked:app_access_to_objects.v1",
  "source": "com.atlassian/ecosystem.app_policy",
  "id": "7a6796c0-746d-4504-92cd-819eca234306",
  "time": "2023-10-24T08:08:08Z",
  "data": {
    "site": { 
      "url": "https://site_name.atlassian.net"
    },
    "blockedObjects": [
      {"type": "page", "id": "12345"},
      {"type": "page", "id": "23456"},
      {"type": "page", "id": "34567"}
    ]
  }
}

These two event schemas are clearly very similar.

The specific changes are;

  1. The http Content-Type header value changing from application/json to application/cloudevents+json.
  2. Addition of a specversion property - which may be ignored if not relevant to the event consumer.
  3. Addition of a source property - which may be ignored if not relevant to the event consumer.

We can now make the following observations;

  • The effective change to json schema made in order to align our event schema to the Cloudevents specification is the addition of two properties - which may be completely ignored.
  • If an app’s http request handler does not register a Cloudevents content type parser it will fallback to the json parser that will already be registered for Atlassian webhooks.
  • Every one of these schema changes are completely transparent to the app. An app can perform ordinary json parsing of the event payload, as it does today for all other Atlassian events.

Binary content mode

While we have withdrawn the proposed binary content mode, it seems this was misunderstood and so it may help to clear this up.

Cloudevents ‘Binary Content Mode’ simply means the event metadata and data are not carried together; metadata is carried via transport headers and data in the message payload.

For http webhooks this means the id, type, time properties are carried as http headers, and the data property is the http message body. It is not binary encoding, it is binary (two-pieces) transport. (As opposed to ‘Structured Content Mode’ which keeps event metadata and data together in the payload.)

Broader rollout

To be clear, we have not committed to evolving all existing event schema to align with Cloudevents, and there is no intent to do so. While consistency of event metadata in schema would be wonderful we believe mandating this would be an enormously disruptive change.

So the issue is that we’re having to implement these and then guide the customer to troubleshoot things to make sure that they’re seeing these items. This increases support costs since we’ll be having to do hand holding with every customer (and every vendor has to implement the feature).

The ask(s):

  1. Please provide standardized troubleshooting tools for the end user to see why something isn’t available to an app (but they can see it).
  2. Please provide standardized documentation explaining how this works and who they should be contacting to fix.

Us implementing the api’s will work somewhat but without #2 - we’re left having to teach the users how to resolve things.

The issue for us is that we won’t know about the “hidden” containers since we might not ever have seen them before.

1 Like

If you are questioning the App Access Rule initiative itself, this is being provided to meet Customer demand - we acknowledge App Access Rules do result in additional complexity and implementation costs for both Atlassian and our Partners.

If you want to make this claim then show us the data. What % of customers and what % of revenue will be impacted by this feature. Show us that it’s worth it.

2 Likes

Hi Boris,

Thank you for raising your concerns.

Note that the App Access Rule feature has been requested by our mutual enterprise customers as a necessary step for them to adopt more apps or as a requirement that needs to be met in order for them to move to cloud. Some of our largest customers are yet to migrate to cloud. They have identified apps to be business critical and would not move without them, eagerly waiting to get increased control over what apps can access in their environment.

based on my experience with the Allow Access UI in Forge, I conservatively expect this to change to double our support costs

Can you elaborate a bit on this, is it that the end user consent flow initially introduced for Forge apps have caused an increase in your support cost or the removal of it?

have a significant negative impact eval to net new conversions

As I mentioned above, we’ve heard from our customers that the “all or nothing” access that apps currently have deters them from using apps in cloud. As such, we expect this rule to help more customers feel confident installing cloud apps who may not otherwise install them today.
I’d like to unpack the support concern a bit here… With the app access rule in place, apps will still be allowed access in all spaces by default. A customer must actively enable an app block for them to experience any change to functionality, which they should expect once they’ve taken action to block an app.

We are also working on proactively notify end users in-product that their admin has blocked an app in places where the app functionality may be impacted (some examples mentioned here), so this should also limit support impact on partners.

Thanks heaps in advance!

Cheers,

Angelina

Hi @JustinThirkell,

Thank you for the clarification.

I’m just afraid that you are missing the bigger picture here. Obviously we understand that this is just going to be delivered as plain JSON. And obviously we can work with CloudEvents SDK. This is not a technical issue we are discussing.

The point is: you are working on this for what? 6 months? A year? According to CDAC you only joined in June this year. The Atlassian Marketplace Partner community consists of partners working with Atlassian architectures for more than a decade. We’ve seen every tech fad come and go within Atlassian, often introduced by engineers that recently started. Those engineers got promoted away and after a year are working in other teams pursuing other Shiny New Things.

You are introducing a new architecture for a very little small subset of the entire Atlassian Cloud ecosystem. You are even very open about this being specific for just this very little small subset:

And in a year, you will be gone. And this little inconsistency within the context of the entire Atlassian Cloud ecosystem will remain. At some point, CloudEvents might be upgraded. It might change. It might become deprecated. And another engineer comes along who will think “why are we using this? We should be using the AwesomeEvents standard instead”. And they migrate. And we follow along to pursue yet another fad.

And you will be gone. And we will be here. Just like we’ve done for the past decade(s).

This is why I’m appealing to you, no begging you, to look beyond your nice little project working with this Shiny New Thing and be open to ALL. THE. FEEDBACK. you received that ask you not to do this.

3 Likes

I totally get this. My question is how many have requested this? Is it 1%? Is it 10%? Is it 100% of people to whom you have suggested the idea in the first place? But they didn’t ask for it till they were told it would be possible?

Without these numbers, it’s hard to understand if the juice is worth the squeeze. Given that Atlassian is a data driven company, I feel like it would help for vendors to be on the same page with the opportunity this brings as Atlassian.

The recent change improved the situation after years of impact on both net new customers (eval → new churn highest of any of our apps) and support costs (nearly 100% of our support tickets were caused by this UX).

My gut feel is that this is technically true but practically false. This is going to get packaged into Access or Premium or something as an upsell functionality. As a result, there will be proactive marketing and communication informing customers about this functionality, even if they never asked for it. My guess is it will also be highlighted to existing customers via emails / change logs / etc. Now you have a large population of customers who don’t actually NEED it, but will enable it anyways because it’s security and that’s the thing we do now.


This RFC focuses on our technical ability to work with Atlassian at an API level once this scenario happens, but my concern is that even before this comes to us, the UX of how this is configured on the customer side, and the UX of how they can troubleshoot it, will be poor. This isn’t a hypothetical concern, Atlassian has a history of poor UX for complex configuration challenges. And also a history of shipping half finished products and then not delivering on the important features for supportability. In addition to the Forge example, in Access we have 5 years for audit logs: https://jira.atlassian.com/browse/ACCESS-577

I am not saying this functionality can’t be implemented well. I am saying that the evidence I have implies it won’t. And, recent evidence supports that when it’s done wrong, it will take years to resolve and vendors will be the ones paying the cost in the meantime.

As such, to my original question, please help us understand what % of customers actually need this. I want to model out if there is an actual financial positive scenario here, or if I need to open headcount for support and start reducing R&D headcount in our 2024 hiring plan.


Taking a step back to look at this problem from the customer perspective rather than the technical one, this feels to me like customers fall into a few groups:

  • I don’t trust vendors, Atlassian should host 100% of everything (solved with Forge)
  • I trust Atlassian, I also trust vendors, but not with everything (solved with this issue, or Forge)
  • I trust vendors with everything (status quo)

Based on this, Atlassian should focus on getting Forge to be the best platform for us and help us get to there so we can support the superset of customers rather than expanding serious effort and onboarding serious risk to only appease a subset of enterprise customers. Even assuming this implementation goes well, the best case scenario is we have to build support for this, and then we have to migrate to Forge anyways down the line.

2 Likes

Hello @JustinThirkell,

I would first like to echo a couple of things already mentioned in previous comments. The first being the question of if this will be available to app vendors before the feature reaches our customers? The second one being the concern of increased support to educate our customers in why our app isn’t showing an Atlassian entity. As @danielwester suggests it would be nice if Atlassian could provide tooling instead of each vendor doing their own implementation/support effort.

We have not decided how we will handle this update internally yet, but we store data in our DB that is connected to Confluence pages. Our app display Confluence pages in a customised site so we have data for likes, feedback on the content, meta data for display in social media etc. in our database. If we would go with the option of removing this data when the app no longer has access, then there wouldn’t be a way to restore it if the customer unblocks our app. Another option would be to keep the data forever, with the drawback of having data stored that is never used and that it might not be what the customer expects when blocking our app. There are of course also more complex solutions to make the customer decide what to do or to keep the data for a while and remove it if the app isn’t unblocked in that time (but there’s no unblock event so it would be difficult to implement). I’d like to hear Atlassian’s input on how you suggest we deal with data stored in the app but connected to Atlassian entities that might be blocked.

Thanks!

1 Like

@remie the tone of your comments is getting very close to ad hominem. Usually you are fairly good at avoiding deriding specific individuals, so please continue to focus on the proposed changes rather than the individual proposing them.

As to your concerns, I do agree that API inconsistencies make aspects of our platform unintuitive and harm the developer experience. While I’m proud of the depth and breadth of the Atlassian platform (and the amazing things that you and other partners build on top of it), we do not yet have a consistent approach to technical API considerations across the board.

This is an area we are exploring, alongside more standardised approaches to partner API collaboration (such as this very RFC process). However to set expectations, it will be some time before we have a consistent set of standards defined and longer still before it is rolled out across Atlassian’s broad API surface area. For eventing, CloudEvents is a possible standard that we will evaluate and may align on more broadly on in the future, but that commitment is outside the scope of this RFC.

For now, the most effective way to use the RFC process is to provide feedback on the specific application of CloudEvents to this API. Please do continue to share constructive criticism on this front. If there are further concrete issues with the API that would inhibit its adoption beyond concerns about consistency and the technical concerns regarding CloudEvents and ARIs that have already been articulated, we would love to hear them.

2 Likes