@VegardKjelseth we’re running into the exact same issue as well at the moment and noticed the fixed height and min-height attribute at 112px in the div wrapping our iframe like you mentioned.
Have you tried removing the viewPort property from your manifest and see if it worked?
This seems to be suggested in Viewport for JSM portalSubheader won't dynamically resize and in their doc:
Thanks for reporting!
Hope this issue gets picked up soon.