RFC-25: [Superseded by RFC-29] App Access Rule - followup to New App Data Access APIs

Project update


RFC status: superseded

In order to enable a clean set of feedback and comments we have decided to close this RFC entirely, and replace it. Please see RFC-29 Revised followup to New Data Access APIs.


This is an update on the implementation details of two new APIs announced in RFC-14 App Access Rule.

In our previous RFC, we outlined how the app access rule would block app access to specific Confluence spaces or Jira projects. At that time we planned to implement the block at the space and project level.

We have since decided that we can better optimize for customer flexibility in future iterations of this feature if we instead block apps from accessing data at one layer lower in granularity, meaning apps will not be able to access content on pages, issues, in comments, or in attachments. Customers will still set rules at the space and project level and will be notified about what data is covered or excluded via documentation and in the UI directly.

While this will leave apps with access to a few pieces of user-generated content like space names, the majority of user-generated content in these spaces and projects will still be protected, and we will be able to enable more granular controls for customers in the future to better align with other rules under data security policies.

Please respect our community guidelines : keep it welcoming and safe by commenting on the ideas not the people; keep it tidy by keeping on topic; empower the community by keeping comments constructive. Thanks!

Updated RFC timeframe

After originally publishing this followup RFC, we received feedback which resulted in our revising the solution (yes, again!). We intend to resubmit this followup RFC in a week, and resolve this RFC a week later.
In order to enable a clean set of feedback and comments we have decided to close this RFC entirely, and replace it. Thank you for your patience. :slight_smile:

  • Publish: 18 Sep 2023
  • Resolve: 02 Oct 2023
  • Publish update: 12 Oct 2023
  • Resolve: 19 Oct 2023

Please see RFC-29 Revised followup to New Data Access APIs.

The new APIs

An apps access to data within a workspace may be affected whenever an Administrator adds or updates an app access rule within a data security policy. Should an app lose or gain access to objects within a workspace, an event will be published allowing the app to react to the change accordingly.

Apps are provided an API to query by container, if this is known, whether objects in that container are still accessible. There is also an API to query by object whether the object is still accessible. Note that ā€˜access to an objectā€™ also implies access to the objectā€™s comments, attachments, etc.

Apps are also provided with an API to query rules that are active in a given workspace, so they can display a warning to end-users.

(See the appendix for more information about workspaces, containers and objects.)

New event/webhook

Previously specified:

New product trigger events for Forge apps and webhooks for Atlassian Connect apps to be notified when a policy change causes a space or project to become inaccessible or accessible (e.g. SPACE_RESTRICTED and SPACE_UNRESTRICTED).

After evaluating technical implementation options, we propose a simplified application flow and event schema.

Application flow

App components are in blue, new Atlassian components are in green.

An app that needs to know when it has been blocked/unblocked from accessing objects will subscribe to this event. As a policy may specify containers from multiple workspaces for protection, one event will be published for each workspace covered by the policy. The app should retrieve a policy result (i.e. is the object blocked or not blocked) for each object it knows about within the workspace (specified by an event), and for each policy result respond appropriately, such as deleting data for blocked objects, or including an object in a list once it is unblocked.

(See the appendix for more information about workspaces, containers and objects.)

Event schema

The proposed event schema is defined using the AsyncAPI specification. The nicest way to read this schema is to go to AsyncAPI Studio and replace the sample schema with (copy/paste) the following schema:

asyncapi: '2.6.0'
tags:
  - name: ecosystem
id: 'urn:com.atlassian.ecosystem.app.webhooks'
info:
  title: Ecosystem App Webhooks Service
  version: 0.1.0
  description: Ecosystem service which publishes events to App webhooks.
  license: 
    name: Apache 2.0
    url: https://www.apache.org/licenses/LICENSE-2.0
  contact:
    url: https://community.developer.atlassian.com/t/rfc-25

channels:
  atlassian/webhooks:
    subscribe:
      bindings:
        http:
          type: request
          method: POST
      summary: Receive notifications of changes to App policy constraints.
      message:
        oneOf:
          - $ref: '#/components/messages/appAccessToWorkspaceUpdatedEventBinary'
          - $ref: '#/components/messages/appAccessToWorkspaceUpdatedEventStructured'

components:
  messages:
    appAccessToWorkspaceUpdatedEventBinary:
      title: Event - App Access to Workspace Changed (binary mode)
      description: |-
       Published when an App's access to a Workspace has changed in some way.

       It could be that the App has been added or removed from a Policy, or the Workspaces covered by a Policy associated with that App have changed.
      traits:
        - $ref: '#/components/messageTraits/eventHeadersBinary'
      payload:
        $ref: '#/components/schemas/appAccessToWorkspaceUpdatedData'
      examples:
        - name: AppAccessToWorkspaceUpdated binary mode
          headers:
            ce_specversion: "1.0"
            ce_type: "avi:ecosystem.app_policy:updated:app_access_to_workspace.v1"
            ce_source: "atlassian.com/ecosystem"
            ce_id: "7a6796c0-746d-4504-92cd-819eca234306"
            ce_time: "{{now(yyyy-MM-dd'T'HH:mm:SS'Z')}}"
            content-type: application/json; charset=utf-8
          payload:
            workspaceAri: "ari:cloud:confluence:46e2390a-101e-4b83-b311-362a16083618:workspace/b5595212-55bb-49e4-8d7f-86d46f4500c1"

    appAccessToWorkspaceUpdatedEventStructured:
      title: Event - App Access to Workspace Changed (structured mode)
      description: |-
       Published when an App's access to a Workspace has changed in some way.

       It could be that the App has been added or removed from a Policy, or the Workspaces covered by a Policy associated with that App have changed.
      headers:
        content-type:
          type: string
          enum:
            - 'application/cloudevents+json; charset=UTF-8'
      payload:
        allOf:
          - $ref: '#/components/schemas/eventHeadersStructured'
        properties:
          data:
            $ref: '#/components/schemas/appAccessToWorkspaceUpdatedData'
      examples:
        - name: AppAccessToWorkspaceUpdated structured mode
          headers:
            content-type: application/cloudevents+json; charset=utf-8
          payload:
            specversion: "1.0"
            type: "avi:ecosystem.app_policy:updated:app_access_to_workspace.v1"
            source: "atlassian.com/ecosystem"
            id: "7a6796c0-746d-4504-92cd-819eca234306"
            time: "{{now(yyyy-MM-dd'T'HH:mm:SS'Z')}}"
            data:
              workspaceAri: "ari:cloud:confluence:46e2390a-101e-4b83-b311-362a16083618:workspace/b5595212-55bb-49e4-8d7f-86d46f4500c1"

  schemas:
    appAccessToWorkspaceUpdatedData:
      type: object
      properties:
        workspaceAri:
          $ref: "#/components/schemas/workspaceAri"

    eventHeadersStructured:
      summary: Message headers in structured content mode.
      description: |- 
       Cloudevent spec mandatory headers, plus a few headers deemed mandatory when publishing cloudevents from Atlassian.  
      type: object
      required:
        - specversion
        - id
        - source
        - type
        - time
      properties:
        specversion:
          type: string
          description: The version of the CloudEvents specification which the event uses.
          enum:
            - "1.0"
        id:
          type: string
          minLength: 1
          description: Identifies the event.
        source:
          type: string
          format: uri-reference
          minLength: 1
          description: Identifies the context in which an event happened.
        type:
          type: string
          minLength: 1
          description: Describes the type of event related to the originating occurrence.
          examples:   
            - "avi:ecosystem.app_policy:updated:app_access_to_workspace.v1"
        time:
          type: string
          #format: date-time
          description: Timestamp of when the occurrence happened. Must adhere to RFC 3339.
        subject:
          type: string
          description: Describes the subject of the event in the context of the event producer (identified by source).
        dataschema:
          type: string
          description: |-
            Identifies the schema for the payload.
            This should dereference to an actual schema document.
          examples:
            -  "https://developer.atlassian.com/platform/ecosystem/events-reference/app_policy/app_access_to_workspace/updated/v1"

    workspaceAri:
      type: string
      description: Workspace ARI
      examples: 
        # https://developer.atlassian.com/platform/atlassian-resource-identifier/resource-owners/registry/#ati%3Acloud%3Aconfluence%3Aworkspace
        - "ari:cloud:confluence:46e2390a-101e-4b83-b311-362a16083618:workspace/b5595212-55bb-49e4-8d7f-86d46f4500c1"
        # https://developer.atlassian.com/platform/atlassian-resource-identifier/resource-owners/registry/#ati%3Acloud%3Ajira%3Aworkspace
        - "ari:cloud:jira:b2eef0a4-755d-4ac6-b117-0c3d9201ff8d:workspace/b2eef0a4-755d-4ac6-b117-0c3d9201ff8d"

  messageTraits:
    eventHeadersBinary:
      headers:
        type: object
        required:
          - ce-specversion
          - ce-id
          - ce-source
          - ce-type
          - ce-time
        properties:
          content-type:
            type: string
            enum:
              - 'application/json; charset=UTF-8'
          ce-specversion:
            type: string
            description: The version of the CloudEvents specification which the event uses.
            enum:
              - "1.0"
          ce-id:
            type: string
            minLength: 1
            description: Identifies the event.
          ce-source:
            type: string
            format: uri-reference
            minLength: 1
            description: Identifies the context in which an event happened.
          ce-type:
            type: string
            minLength: 1
            description: Describes the type of event related to the originating occurrence.
            examples: 
              - "avi:ecosystem.app_policy:updated:app_access_to_workspace.v1"
          ce-time:
            type: string
            format: date-time
            description: Timestamp of when the occurrence happened. Must adhere to RFC 3339.
          ce-subject:
            type: string
            description: Describes the subject of the event in the context of the event producer (identified by source).
          ce-dataschema:
            type: string
            description: |-
             Identifies the schema for the payload.
             This should dereference to an actual schema document.
            examples:
              -  "https://developer.atlassian.com/platform/ecosystem/events-reference/app_policy/app_access_to_workspace/updated/v1"

Or, in short the event payload will be similar to this:

{
  "specversion": "1.0",
  "type": "avi:ecosystem.app_policy:updated:app_access_to_workspace.v1",
  "source": "atlassian.com/ecosystem",
  "id": "{{uuid()}}",
  "time": "{{now(yyyy-MM-dd'T'HH:mm:SS'Z')}}",
  "data": {
    "workspaceAri": "ari:cloud:confluence:73dfba81-fec7-4a75-b59b-e495641293de:workspace/f0dede24-2545-4a2c-899f-8f349c710ba8"
  }
}

(See the appendix for more information about ARIs.)

Event AVI

The exact value for the event AVI is still subject to confirmation but it may be similar to avi:ecosystem.app_policy:updated:app_access_to_workspace, regardless of the App type.

Cloudevents details

The exact values for the type and source attributes are still subject to confirmation.

Events will be published via the HTTP protocol binding, per specification. The content mode will be either binary or structured, determined by the webhook publisher. Subscribers to these events can distinguish between these modes by inspecting the Content-Type header value and parsing accordingly. Or (recommended) use a client SDK which provides a clean abstraction.

Changes to proposal

A summary of changes to what was previously specified:

1. Event semantics

The event will not include an attribute indicating whether access was restricted or unrestricted. The event will only indicate that ā€˜some changeā€™ in restrictions occurred.

Motivation:

  • DEVELOPER EXPERIENCE: As event processing may result in side effects (such as deleting data associated with a now-inaccessible container) and event delivery order is not guaranteed, to avoid race conditions apps are expected to call the API (detailed below) and fetch the current status, instead of relying on data from the event.

2. Event granularity

There will be one event published per app per workspace. Apps are expected to iterate over containers or objects saved in their datastore and check their current status using the new App Policy API.

Motivation:

  • DEVELOPER EXPERIENCE: Org-wide policy changes can lead to a large number of events, and apps may face problems handling a surge of traffic if an event per app per container is published.
  • TRUST EXPERIENCE: The less data about restricted spaces or projects is shared with apps - the better. By relying on apps to iterate over identifiers they already know about we do not disclose information about the total number of blocked containers or objects in a workspace.

3. Use of Cloudevents specification

As Ecosystem Partners become increasingly sophisticated in their handling of events published by Atlassian this is an opportune time to adopt an industry standard for describing event metadata. Cloudevents.io is a mature standard that will allow Partners to use standard SDKs to inspect events and create standardized eventing middleware, amongst other benefits.


New App Policy API

Previously specified.-,New%20REST%20APIs,-accessible%20to%20Forge):

New REST APIs accessible to Forge, Atlassian Connect, and OAuth 2.0 (3LO) apps to retrieve a set of opaque identifiers (e.g. numeric identifiers or UUIDs) for inaccessible spaces.

As previously proposed, a new API will be created to allow apps to query project\space restrictions.

Changes to proposal

Changes here fall into three categories:

1. Require identifiers to be passed as input

Instead of providing an API to enumerate restricted containers or objects, we will be relying on apps to pass batches of identifiers they already know about as input.

Motivation:

  • DEVELOPER EXPERIENCE: In cases when the app knows about only a few identifiers, iterating over them will be faster compared to going over the full list of restricted projects or spaces. The downside is that apps will have to structure their databases to be able to iterate over data keyed by identifiers.
  • TRUST EXPERIENCE: Similarly to ā€œNew event\webhookā€, we want to avoid exposing the total number of spaces or projects the app does not have access to.

2. Provide API to determine if app restrictions are active

There will be a way for apps to determine if the execution context (site) has any active rules that may affect app access to data.

Motivation:

  • DEVELOPER EXPERIENCE: When the app will be using APIs to fetch a list of objects, some of the items in the list may be hidden from the output due to active policies. Users of such apps may not be aware of policies and their effect on apps, since policies are configured by the Org admin. Using this API, apps can determine if there are any rules that can affect app operation, and if so ā€“ display a warning to the end user.

3. Separate API for Connect apps

We are shifting towards GraphQL API. This new set of APIs will be available in GraphQL form. Forge and OAuth 2.0 apps are already able to access those GraphQL APIs since they support auth mechanisms. While we expect Connect apps to gain the same level of access through upcoming compatibility features planned for the Forge and Connect platforms, in the meantime we have settled on providing a REST API proxy for them.

REST API for Connect apps

Request formats are slightly different for Jira and Confluence Connect apps

The proposed API schema is defined using the OpenAPI specification. The nicest way to read this schema is to go to Swagger Editor and replace the sample schema with (copy/paste) the following schema:

API schema:

openapi: 3.0.3

info:
  title: App Policies API
  version: "0.1.0"
  description: Ecosystem service which publishes events to App webhooks.
  contact:
    name: Justin Thirkell
    email: jthirkell@atlassian.com
    url: https://dacrfc.atlassian.com/rfc/1234

servers:
  - url: "https://your-site.atlassian.net/rest/atlassian-connect/latest/app-policies/data-classifications"
    description: Jira
  - url: "https://your-site.atlassian.net/wiki/rest/atlassian-connect/latest/app-policies/data-classifications"
    description: Confluence
    

paths:
  /containers:
    get:
      tags:
        - containers
      summary: Get the status of provided containers
      operationId: getContainerDecisionStatus 
      parameters:
        - name: projects
          in: query
          description: Project ids. Accepted only if app is installed into Jira.
          explode: false
          schema:
            type: array
            maxItems: 20
            minItems: 1
            items:
              type: integer
              minimum: 1
        - name: spaces
          in: query
          description: Spaces ids. Accepted only if app is installed into Confluence.
          explode: false
          schema:
            type: array
            maxItems: 20
            minItems: 1
            items:
              type: integer
              minimum: 1
      responses:
        '200':
          description: Successful operation
          content:
            application/json:
              schema:
                type: object
                properties:
                  containers: 
                    type: array
                    maxItems: 20
                    items:
                      $ref: '#/components/schemas/Result'
                
        '400':
          description: Invalid ID supplied
      security:
        - atlassian_connect_jwt:
            - READ
            
  /objects:
    get:
      tags:
        - objects
      summary: Get the status of provided objects
      operationId: getObjectDecisionStatus 
      parameters:
        - name: issues
          in: query
          description: Issues ids. Accepted only if app is installed into Jira.
          explode: false
          schema:
            type: array
            maxItems: 20
            minItems: 1
            items:
              type: integer
              minimum: 1
        - name: pages
          in: query
          description: Pages ids. Accepted only if app is installed into Confluence.
          explode: false
          schema:
            type: array
            maxItems: 20
            minItems: 1
            items:
              type: integer
              minimum: 1

      responses:
        '200':
          description: Successful operation
          content:
            application/json:
              schema:
                type: object
                properties:
                  objects: 
                    type: array
                    maxItems: 20
                    items:
                      $ref: '#/components/schemas/Result'
                
        '400':
          description: Invalid ID supplied
      security:
        - atlassian_connect_jwt:
            - READ
            
  /constraints:
    get:
      tags:
        - constraints
      summary: Check if the app is subject to policy restrictions in a given installation context
      operationId: checkActiveConstraints 
      responses:
        '200':
          description: Successful operation
          content:
            application/json:
              schema:
                type: object
                properties:
                  constraints:
                    $ref: '#/components/schemas/Constraints'
      security:
        - atlassian_connect_jwt:
            - READ
  
components:
  schemas:
    Result:
      required:
        - id
        - decision
      type: object
      properties:
        id:
          type: integer
          format: int64
          example: 14800
        decision:
          $ref: '#/components/schemas/Decision'
    Decision:
      required:
        - status
      type: object
      properties: 
        status:
          type: string
          description: Policy decision status
          enum:
            - ALLOWED
            - BLOCKED
    Constraints:
      type: object
      properties:
        hasConstraints:
          type: boolean
  securitySchemes:
    atlassian_connect_jwt:
      type: apiKey
      name: authorisation
      in: header

Alternatively, here is a short overview of new API methods:

Get the status of provided containers

Request (Jira Connect App):

GET /rest/atlassian-connect/latest/app-policies/data-classifications/containers ?projects=id1,id2

Request (Confluence Connect App):

GET /wiki/rest/atlassian-connect/latest/app-policies/data-classifications/containers ?spaces=id1,id2

Response:

{
  "containers": [
    {
      "id": "id1",
      "decision": {
        "status": "ALLOWED"
      }
    },
    {
      "id": "id2",
      "decision": {
        "status": "BLOCKED"
      }
    }
  ]
}

Get the status of provided objects

Request (Jira Connect App):

GET /rest/atlassian-connect/latest/app-policies/data-classifications/objects ?issues=id1,id2

Request (Confluence Connect App):

GET /wiki/rest/atlassian-connect/latest/app-policies/data-classifications/objects ?pages=id1,id2

Response:

{
  "objects": [
    {
      "id": "id1",
      "decision": {
        "status": "ALLOWED"
      }
    },
    {
      "id": "id2",
      "decision": {
        "status": "BLOCKED"
      }
    }
  ]
}

Check if the app is subject to policy restrictions in a given installation context

Request (Jira Connect App):

GET /rest/atlassian-connect/latest/app-policies/data-classifications/constraints

Request (Confluence Connect App):

GET /wiki/rest/atlassian-connect/latest/app-policies/data-classifications/constraints

Response:

{
  "constraints": {
    "active": true
  }
}

GraphQL API for Forge and OAuth 2.0 apps

Forge and OAuth 2.0 apps will not be able to access the REST API described above, it is reserved for Connect apps only. GraphQL API will be provided instead. Additionally, we are going to provide a new SDK wrapper as part @forge/api package, similar to Storage API, to simplify integration for Forge apps.

API contract

Type definitions

type Query {
  ecosystem: EcosystemQuery
}
type EcosystemQuery {
  appPolicies: EcosystemAppPolicies
}
type EcosystemAppPolicies {
  dataClassifications(id: ID!): EcosystemDataClassificationsContext
}
type EcosystemDataClassificationsContext {
  id: ID!
  containers(ids: [ID!]!): [EcosystemDataClassificationPolicyResult]
  objects(ids: [ID!]!): [EcosystemDataClassificationPolicyResult]
  hasConstraints: Boolean
}
type EcosystemDataClassificationPolicyResult {
  id: ID!
  decision: EcosystemDataClassificationPolicyDecision!
}
type EcosystemDataClassificationPolicyDecision {
  status: EcosystemDataClassificationPolicyDecisionStatus!
}
enum EcosystemDataClassificationPolicyDecisionStatus {
  ALLOWED
  BLOCKED
}

Get the status of provided containers

Query:

query getContainersDecisions($installationContext: ID!, $containerIds: [ID!]!) {
  ecosystem {
    appPolicies {
      dataClassifications(id: $installationContext) {
        containers(ids: $containerIds) {
          id
          decision {
            status
          }
        }
      }
    }
  }
}

Input:

This example demonstrates usage of confluence space ARI to uniquely identify scopes

{
  "installationContext": "ari:cloud:confluence::site/{siteId}",
  "containerIds": [
    "ari:cloud:confluence:{siteId}:space/{spaceId}"
  ]
}

Output:

{
  "data": {
    "ecosystem": {
      "appPolicies": {
        "dataClassifications": {
          "containers": [
            {
              "id": "ari:cloud:confluence:{siteId}:space/{spaceId}",
              "decision": {
                "status": "ALLOWED"
              }
            }
          ]
        }
      }
    }
  }
}

Get the status of provided objects

Query:

query getObjectsDecisions($installationContext: ID!, $objectIds: [ID!]!) {
  ecosystem {
    appPolicies {
      dataClassifications(id: $installationContext) {
        objects(ids: $objectIds) {
          id
          decision {
            status
          }
        }
      }
    }
  }
}

Input:

This example demonstrates usage of confluence space ARI to uniquely identify scapes

{
  "installationContext": "ari:cloud:confluence::site/{siteId}",
  "objectIds": [
    "ari:cloud:confluence:{siteId}:page/{pageId}"
  ]
}

Output:

{
  "data": {
    "ecosystem": {
      "appPolicies": {
        "dataClassifications": {
          "objects": [
            {
              "id": "ari:cloud:confluence:{siteId}:page/{pageId}",
              "decision": {
                "status": "ALLOWED"
              }
            }
          ]
        }
      }
    }
  }
}

Get if there are any constraints affecting the app in a given context

Query:

query getAppConstraints($installationContext: ID!) {
  ecosystem {
    appPolicies {
      dataClassifications(id: $installationContext) {
        hasConstraints
      }
    }
  }
}

Input:

This example demonstrates usage of workspace ARI to uniquely identify app installation context.

{
  "installationContext": "ari:cloud:confluence::site/{siteId}"
}

Output:

{
  "data": {
    "ecosystem": {
      "appPolicies": {
        "dataClassifications": {
          "hasConstraints": true
        }
      }
    }
  }
}

Scopes

For Connect apps, the App must have the relevant Connect READ scope. We do not expect this to be a change for any affected Apps.

For Forge and OAuth 2.0 apps, read:confluence-space.summary or read:project:jira scopes will be required to access container-level information (depending on the type of container). No additional scopes will be required to check if the app is affected by policy constraints in a workspace.


Appendix

ARIs

This RFC mentions ARIs and these may be unfamiliar to readers. An ARI (Atlassian resource identifier) is a globally unique identifier. ARIs can identify any entity, for example:

  • Jira issues
  • Jira projects
  • Confluence pages
  • Trello boards
  • Atlassian users, (global entities)

Workspace ARI examples:
ari:cloud:jira:ad95fada:workspace/114dfb50
ari:cloud:confluence:1d52f2ee:workspace/cfdef2de

ARIs are particularly important when providing APIs that span multiple products, such as in this RFC which specifies an API that will be used for both Confluence and Jira.

An ARI is an opaque identifier, meaning that although it is human-readable, it should only ever be handled atomically - i.e. not parsed into its constituent parts. The structure of ARIs may change over time and so parsing of ARIs is strongly discouraged.

Containers

Both Confluence and Jira define a model that involves grouping objects into containers. For Jira a container of issues is called a project, for Confluence a container of pages is a space.

When defining APIs that span multiple products such as in this RFC, it is useful to define a term that may be used for any and all products. In this case, it is a container, and so the APIs specify the use of containerAri in various places.

A containerAri is an abstraction that in practice maps to concrete ARIs such as a spaceAri or a projectAri. So whenever you see container in API definitions you can think of this as a space or a project, whatever makes sense in your particular context.

Objects

The corollary of using the term container as an abstraction for projects and spaces is the term objects as an abstraction for Jira issues and Confluence pages.

So whenever you see object in API definitions you can think of this as an issue or a page, whatever makes sense in your particular context.

Asks

  • Is this new API sufficient to allow graceful handling of access rules restrictions for your app?
  • What is your estimated timeframe to implement this flow for your app?
  • Your thoughts on the usage of standards - GraphQL and Cloudevents?
3 Likes

Hi @JustinThirkell,

Thank you for the RFC.

Would it be possible for Atlassian to pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty pretty please not introduce Yet Another Shiny New Technology?

Why does Atlassian think that partners are happy with having to support a different type of format for events compared to Connect install hooks or product events specifically for App permissions?

Does Atlassian really think we have nothing else to do?

Either Atlassian is now expressing its desire to rewrite the entire event code base to support this new format (which can be gradually, does not have to be big bang) or this is just another siloed team doing their own thing and not taking into consideration the bigger picture. If it is the latter, Iā€™ll pass. If that breaks our apps, fine by me.

Same goes for the ARI. There is not other situation in which Atlassian uses ARI. This is not supported by any of the REST APIā€™s. Looking at the format of the ARI there is also no way for us to map it back to an identifier that is accepted by the product APIs exposed to partners.

This RFC is making global changes to the Atlassian Marketplace Partner developer experience, which are not warranted by the scope of this RFC.

THIS IS A SERIOUS FLAW TO THIS RFC

CC: @ibuchanan @tpettersen

18 Likes

1000x this.

Also looking at the greater context:
The whole purpose of this feature is to allow apps to have a great user experience when data access is disabled. When it is hard to implement, very few apps will implement. See Dark Mode as a reference

Hey @JustinThirkell,

Those will be big changes to the ecosystem. That will cause a lot of work for many vendors. Thank you for sharing the according plans with us in advance.

Like @remie, while reading, I was surprised that you want to introduce CloudEvents for those new kinds of events. I am not really opposed to that approach but would expect that this is part of a greater plan in which Atlassian changes the whole Connect & Forge event infrastructure to this new format.

Besides that, I want to raise awareness that we expect Atlassian to thoroughly test whether Forge is ready for those new requirements. Two topics that came to my mind:

  • You are highlighting that CloudEvents has good coverage in terms of SDKs. Is their JavaScript/TypeScript ready and compatible with Forge? I know CloudEvents/sdk-javascript uses packages like ā€œuuidā€ under the hood. Those packages rely on Node.jsā€™s ā€œcryptoā€ package, which, as far as I know, doesnā€™t run in the Node isolate used by the Forge Sandbox. Please align the release of the new Node Runtime with those changes.
  • Atlassian expects apps to iterate over entities when changes occur. At least that will not be possible within the boundaries of the 25-second limited Forge lambdas. I expect that Atlassian tests and maintains examples of how to securely iterate over thousands of pages or issues without the need to maintain a remote server. In case this can not be solved with async events, please make sure to have long-running tasks for Forge ready on time.

Best,
Julian

13 Likes

Agree with @remie - this is not great, especially having logic to handle binary formats, for payloads that a re <2kb? Why not just use normal webhooks.

Also, what are we supposed to do with ARIs? We currently only work with projectIds/projectKeys, since the REST APIs only work with those, how are we supposed to map those ids to the one we store on our side? I understand that internally GraphQL is the way forward, but almost no one, besides a few Forge apps, will be able to work with this.

4 Likes

Hi @JustinThirkell ,
Thanks for sharing the proposal.

I have one quick question: The proposal introduces ARI. This is new for us, and I see it as a big problem. At this moment we vendors are required to move our code to the new Confluence Rest API-v2, and this API uses a new route scheme based on content IDs, not ARIs. Does this imply we will have to handle two naming schemes for the same content?

4 Likes

I agree with @remie here (and everyone else) - please donā€™t add yet another ā€œatlassian standardā€ approach. It will mean that weā€™ll be less likely to adapt to this and in a year after launching Atlassian will be introducing programs/incentives of how to get people to make use of this.

Hi @JustinThirkell

This proposal contains huge divergence from the previous RFC and massive scope creep from a vendor implementation perspective.

How do we map ARIs from an event back to other Atlassian entities?

What factors did Atlassian include in its determination that ā€œthis is an opportune timeā€ for app vendors to adopt CloudEvents?

Iā€™d contest your comment about CloudEvents being a mature standard by quoting their own website:

CloudEvents is a new effort and itā€™s still under active development. However, its working group has received a surprising amount of industry interestā€¦

Please can you provide more information about Atlassianā€™s decision to roll out support for CloudEvents across the entirety of the existing webhook mechanism including timeframes?

How does:

ā€¦event delivery order is not guaranteed, to avoid race conditions apps are expected to call the API (detailed below) and fetch the current status, instead of relying on data from the event.

reduce the possibility of race conditions, given that REST API calls are not guaranteed to take the same amount of time, may be rate limited, are not guaranteed to be processed in the order they arrive at Atlassianā€™s infrastructure, and will not necessarily complete in the order in which they are made?

We are shifting towards GraphQL API

Can you expand on which teams within Atlassian this statement applies to? The Atlassian documentation on GraphQL is distinctly lacking in its Jira documentationā€¦

Thanks,
Jon

12 Likes

Itā€™s funny how this is apparently a company-wide engineering decision that has been communicated within Atlassian, but somehow nobody seemed to care to actually inform the Atlassian Marketplace Partner community.

I guess Atlassian Economy is dead since reorg #528ā€¦

Hi there,

thanks as always for publishing this RFC.

  • I also do not think it necessary to add another (self-proclaimed!) standard to the mix, especially when it requires its own SDK to be used. IMO, the current webhooks implementation is sufficient. Unless youā€™re telling us that you want to migrate all webhooks to this, Iā€™d like to rather avoid it. It adds unnecessary complexity. If you were to avoid the binary format, my acceptance might be higher, because it looks like it may be usable without an extra dependency then, but I still think itā€™s unnecessary.
  • siteIds or cloudIds (they are the same, right?) are not a concept that exists consistently in Connect apps, so receiving Webhooks with it would be hard to process unless the clientKey is also included. Would the webhooks received by Connect apps use the same authorization scheme as the other webhooks? Because those JWTs contain the clientKey, in which case this point is moot
  • Question: I am curious what would happen to, for example, Macros that are in a space that gets restricted at some point and should then be rendered. How would that behave? Found the answer in RFC-14, never mind

Cheers,
Tobi

@JustinThirkell given that the resolution date is set to Oct 2, perhaps it might be a good idea to start responding to feedback if you want to have a meaningful conversation before the deadline.

CC: @ibuchanan

Thank you for your responses, and thank you also for your patience whilst we discuss this internally. As you can see we are taking more time than we anticipated to close out this RFC, which is indicative of the complexities we are dealing with. :sweat_smile: We appreciate that additional integration points can be costly for you to implement, which is why we are taking the time to consider and propose solutions and then gather your feedback to ensure we land on a solution that works for both of us.

While we believe we can address some of the concerns you have raised, we are currently revisiting the user stories we gathered from your initial App Access Rule RFC feedback and reviewing our solution as a whole. Please consider this as an acknowledgement of the importance of balancing the cost of implementation, delivery timeframes, and comprehensiveness of the solution.

We will provide a further update as soon as possible and, as always, we greatly appreciate your feedback. Thank you. :slightly_smiling_face:

Hi @remie, well, snap! I was about to send the reply I just posted to the RFC. As I alluded to, itā€™s fair to say that the requirements we are trying to meet have collectively been rather complex. We have taken the feedback on board, and we are equally motivated to re-engage in discussion and resolve this RFC as soon as possible.
Thanks, :slightly_smiling_face:
Justin

Hi @JustinThirkell ,

Iā€™ve got a question about ARIs: some of your examples contain a / (forward slash). This is also a path delimiter in urls.
This in turn means if we escape (encodeURIComponent) the ARI, we will get a different string. This in turn leads to issues in the qsh calculation in Atlassianā€™s library ACE.
Is it possible to design ARIs such that they are url-safe?
We app developers often need to handle possibly tainted variables, and need to escape them before adding them to a url to call e.g. a REST API.

1 Like

Hi @JustinThirkell,

two points from my side:

Point 1:
I can give a +1 to pretty much all concerns raised by other app vendors. Using additional APIs/standards increases technical complexity on our (app vendorā€™s) side. Please only do this for good reasons.

Point 2:
Iā€™m especially concerned about one feature of our Jira app (Connect) in regards to project/issue restrictions:

  • Our appā€™s feature: Standard customfield(s) can be set with user data from the issueā€™s assignee/reporter. (For use-cases like ā€œManager of Assigneeā€, or ā€œLocation of Reporterā€.) Whenever the assignee/reporter changes, our app updates the customfield(s) accordingly, via REST-API.
  • For this to work, our app must:
    • be able to view the issue (to get issue-events)
    • be able to edit the issue (to set customfields)

That will not work for restricted projects, which is as designed by App Access rules, afaik. In consequence, those customfields will become outdated after assignee/reporter-changes, as they cannot be updated anymore by our app. We see that as a problem, because:

  • Our end-users will surely wonder why those customfields get outdated.
  • We donā€™t see how we can inform the end-user in a meaningful way. Restricted issues and their customfields are completely off limits to our app.

What can we do here, apart from updating our documentation, and handling the additional support requests from customers?

Cheers,
Andreas

2 Likes

Thanks Andreas,
While we have spoken to some Partners already as part of this project, real world use cases being shared like this are super helpful. We appreciate it. :slight_smile:

Update to RFC timeframe

After originally publishing this followup RFC, we received multiple points of feedback which has resulted in our revising the solution (yes, again!). We are going through a few additional checks at our end and intend to resubmit this followup RFC in a week, resolving it a week after that.

  • Publish: 18 Sep 2023
  • Resolve: 02 Oct 2023
  • re-Publish: 12 Oct 2023
  • Resolve: 19 Oct 2023

Thank you for your patience. :slight_smile:

1 Like

For those following along sorry for the delay :sweat_smile: - we are working through a couple of final details and will resubmit this RFC on Monday 16th Oct.

Thank you, Andreas, for bringing up a real use case for discussion. The approach we are looking to take for custom fields is to hide them and leverage the ā€œFind your fieldā€ tool for company managed projects to help product admins understand why certain fields are no longer available. This design is not final, we welcome any feedback you may have:

We are also looking to show a warning in Layout:

Again, this visual is indicative and not final, we welcome any feedback you may have.

Note that in parallel with the customer EAP, we are planning to run a partner EAP with a select group of early access partners soon. Our hope here is to get enough early feedback that we can open up the EAP to all partners early in the new year.

Hope this makes sense.

Cheers,

Angelina

2 Likes

@AngelinaIgnatova, thanks for the insight into locked custom fields. Indeed, internally we thought about this as a possible solution for App Access rules in regards to custom fields. Then at least itā€™s relatively clear to end-users, that a custom field ā€œbelongsā€ to an app, and indeally which app.
So far, so good.

But: I donā€™t see how we as an app can:

  • create locked custom fields
  • update an existing custom field as locked by our app

I did found this docu where it says:

When you install a Jira product or plugin, some fields that are needed by that product or plugin are created and locked.

(I assume that ā€œpluginā€ means ā€œappā€ here. Correct me if Iā€™m wrong.)
But I donā€™t see in any docs and/or the REST API how to actually do this as an app. Can you help here?

Thanks.
Andreas

PS: I donā€™t see locked custom fields mentioned on the subsequent RFC-29. Is that intentional (because out-of-scope), or because you donā€™t want to duplicate the topic? Just curious.