I am wondering how I can change the icon below marked yellow.
Or is this only available for add-ons published via the marketplace?
Yes. As soon as you create a (private) listing for your app the icon should be pulled from the Marketplace. 
1 Like
It’s the same with screenshots, too, right?