My first impression wasn't great. My project is setup to use a git submodule for the wiki, which is just how github works. I created a project file to hold the documentation and the main solution references this. Personally I'd prefer the documentation to exist in the same repo, but that is an issue for another day.
The UI build, as far as I could tell, did not allow for subrepositories, so straight away I had to resort to the appveyor build tool, configured via a yaml file in the root of the repository. This is when the problems with appveyor became immediately apparent.
A while ago I wrote about avoiding spaghetti builds. Appveyor violates every one of these rules. I went into detail on the biggest issues below.
Who's the Boss?
1. Appveyor checks out the code.
2. Appveyor executes the build found in the repository.
3. The build checks out more code.
There is a circular relationship between the source and the build, bound to end in tears. This circular relationship leads to the next issue.
Imagine if you had to commit your code and wait for a cloud service to compile before seeing any errors. This is essentially what appveyor forces upon your build.
Build Configuration == Build Process
I generally want to compile and test on every check in, but integration tests take much longer and I'm happy for them to only run as often as possible. I might want performance tests to be compiled in release mode. I might only want the deployment to be run manually, etc.
These scenarios are impossible with the way configuration is handled by Appveyor.
What is Appveyor
Unfortunately the answer seems to be all three. The only good news is that the CI server is what shows the most promise, which is the only pat of it I'm interested in using.
For a more complex project, with complex configuration scenarios? I think you would drive yourself mad.
I'm hoping they really focus on the CI server part in future and leave the building and deployment to better tools.