Hey @MattDoar,
Today app management is fully isolated to the site instance and is managed by a site or product admin. Due to this, the risk which you’ve noted is appreciable and can occur. @JuliaDaehne has recently published details about changes happening to app management which elevates app management to admin.atlassian.com. This means that it is now also available and under the purview of Organisational Admins.