If I understand you correctly, you want to build your UI once and ship to both Cloud and Server? If so, you’re not alone! There are a number of vendors that do exactly this.
As an example of a vendor shipping to both Cloud and Server from the same codebase, check out a talk from @edave at AtlasCamp a few years ago: https://www.youtube.com/watch?v=xJlwjLSDVMM
If you choose to build your UI using Atlaskit and React, nothing is stopping you from shipping that same UI to Server. Similarly, if you use AUI to build your UI, nothing prevents you from shipping that in your Cloud product, either.
When it comes to the Atlassian Design Guidelines (ADG) and design differences between Cloud and Server, the differences are disappearing over time. The look and feel of Server and Cloud products is aligning. You can see this in the various announcements from the Server products:
If you decide to match the ADG, your product will not look out of place in either Cloud or Server. However, you do not need to use the ADG when building your product – after all, they are “guidelines”, not “rules”.
There are advantages to using either AUI or Atlaskit in your UI layer. Most notably, as they are in use by the core Atlassian products themselves, they receive updates and improvements from Atlassian staff. They both implement the ADG, so you would also save time interpreting and implementing those.
I hope this helps clarify things for you.