What version started mucking with my config files?

Looking at an upgrade from 4.10.1 to 4.14.4. One thing I just noticed is the system config include and the commenting out of certain key/value pairs. In what version was that started?

The loss of stash.project and stash.repository really messes me up.

Are repository move and rename both covered by RepositoryModifiedEvent?

I know this isn’t an API change per se but shouldn’t it show up somewhere? You know like “We’re no longer going to maintain the project and repository names in the config so if you rely on that data you should make other arrangements.”

Were you using a plugin to maintain the repository names in the config? I’m not sure if the core Stash/Bitbucket product ever did that.

The git config files were changed to make system-wide config changes (like enabling bitmap support) easier. We moved some of the git config properties to a global config file. In the process of moving properties, we commented out unknown properties. We don’t consider the on-disk repository information as API so we didn’t feel the need to announce the changes.

There are two main reasons why we’d prefer people leave the git config files alone:

  1. Several key Bitbucket features, like pull requests and forks, require very specific git config. Editing git config can break those features and/or cause problems with upgrades.

  2. We think we’ve provided all the information you need from our APIs (Java and REST) and don’t think people should need to go to the config for anything.

Is there something that the Bitbucket API isn’t providing you?

Yes, it initially wrote them. Realized i was maintaining them since they didn’t update on moves/renames so i just moved everything into my own section.

We disagree on their usage. I appreciate you need to use them and have no qualms about sharing but I regard a repositories config file as code and therefore they are mine. I use them extensively. I will not rely on anything atlassian writes to them in the future but I would really like to see your policy updated to notify end users of significant changes.

@justin_shapiro

No version of Bitbucket Server ever provided or maintained stash.project or stash.repository. There was an add-on that you could install that did it, but the base product didn’t do it.

In Bitbucket Server 4.12, the system was changed to use includes. As a bonus, I also updated the system to actually maintain the project and repository names (see BSERV-3972). That change had three parts:

  • The key names changed from stash.project and stash.repository to bitbucket.project and bitbucket.repository
  • For performance reasons , the system doesn’t write the properties directly in config anymore; it writes them in repository-config, alongside config
    • config includes repository-config, so the fact that the properties aren’t directly in config should be irrelevant to you if you’re using git config to access them. git config bitbucket.project will return the project, just like git config stash.project might have done previously, without any extra effort
    • Storing them in config, as the now-obsolete add-on did, results in a much more frequent occurrence of lock conflicts. By separating those values out into repository-config, they no longer need to “share” the same config.lock that protects the main configuration.
  • The add-on that could previously be installed is now considered obsolete and the system removes it automatically if it’s detected (and will not allow it to be reinstalled)

Project and repository renames are automatically tracked by the system, now, and repository-config is automatically rewritten when such changes occur to keep it up to date with the repository’s database state.

Again, to reiterate, Bitbucket Server never wrote stash.project or stash.repository. You were depending on a now-obsolete add-on. The new values should be much more reliable.

Going forward, it’s less likely Bitbucket Server will modify config files directly; we’re more likely to change system-config so that we can make changes to all repositories in the system in a single shot (system-config has already changed multiple times since it was originally added in 4.12). However, we reserve the right to change config at any time, without warning. Bitbucket Server manages its repositories, and will take whatever steps are appropriate as part of doing so. The system’s functionality depends on having known, consistent configuration for repositories. When administrators adjust the configuration, it can break or significantly impair system functionality. Realistically, given the breadth of what Git allows to be configured, it is impossible for us to adequately test every configuration that could potentially be applied. As a result, if Bitbucket Server needs specific configuration applied in order to function, it will apply that configuration. The “required” configuration may change at any time, and is an implementation detail of the system.

Best regards,
Bryan Turner
Atlassian Bitbucket

Hmm, do you know what the plugin was? I don’t remember using one but maybe i did.

i still don’t agree about highlighting changes to config. as i said i use it a lot. i did long before i placed my repositories in version 2 dot whatever and continued the practice over the years. i’m troubled by the line you seem to be drawing in the sand as all i asked for was a heads up after the fact so i can plan accordingly.

@justin_shapiro

If you were using the add-on I know about, it would have been the reponame-plugin.

I’m not trying to be difficult. It’s a question of tacit acceptance, and a stance that’s hardened over years of dealing with support issues. Installs where administrators reconfigure their repositories show up in support with unexplained behavior that can take weeks to chase down because the issues can’t be reproduced.

Consider the way libraries like Hibernate or Spring work. They document their interfaces. They do not generally document their implementations, and they do not document when those implementations change. As long as the contract of the interface is maintained, the implementation can be updated according to whatever they need.

By documenting our configuration, we’re tacitly agreeing to it being treated as API. Documenting it is laying the groundwork for administrators to base changes on what we document. It’s encouraging a behavior we have, since Stash 1.0 was released, explicitly advised against: Modifying our repositories on disk. Bitbucket Server manages its repositories, and they belong to the system. How they are curated and configured on disk is an implementation detail, with a contract defined by Git’s protocols (HTTP and SSH hosting) and by Bitbucket Server’s Java and REST APIs. Modifying config files is no different than using Reflection to interact with the internals of a Java library. We can’t prevent you from doing it, but we choose not to help you do it.

What we do help with, what we’d love to help with, is building a system that meets your needs. Why do you edit config files? Is there functionality you need access to? Have you created an issue on jira.atlassian.com to express your needs? Certainly I can’t promise we’ll ship it immediately, but if you need it there’s a good chance someone else does too. We can’t help support use cases we don’t know about.

Best regards,
Bryan Turner
Atlassian Bitbucket

P.S. I’ll note that I explicitly called out the configuration changes on BSERV-3972 when I closed it, and that it was listed in the release notes for Bitbucket Server 4.12. I noted that Bitbucket Server would track the project and repository, and I noted that the property names would change from stash.* to bitbucket.*. I even wrote an overview of how I implemented it.

We’d really like to understand why you modify the git config files on the server. None of the changes you make to git config on the server will affect what clients receive when they clone/pull but it may interfere with how the server works. So, we only see downside with no upside.

We’d love to understand what the upside is for you.

a degree of laziness. flexibility. my favorite: historical reasons. can i store most of the data in the database? sure but if for some reason i move out of this tool i want it. my pre and post hooks are still scripts so the data feeds and gatekeeps them. per repository configuration allows the scripts to remain vanilla. in keeping with that i have aliases defined so even if things are different between repositories such as how we parse versions the hooks still work. i also use the information on the java side for things like merge checks. for instance in some repos we require a jira issue in the commit message. for those both the precommit script and the merge check need to know if they should enforce or not. another thing is mailmap. all of my scripts and plugins respect the mailmap unfortunately the gui does not.

no way to attach an image here huh? i was gonna add a screencap.