See also http://www.intellij.net/forums/thread.jspa?threadID=269614&tstart=0

The whole problem about who to notify when has been a big issue with our use of team city. I would be glad to provide more detailed input about problems or give feedback for possible solutions.

I would like to see notifications to be sent only where there is a new failure, and then only to those users that have made new changes. Don't keep sending notifications if things are still broken. Sometimes the solution may take a significant amount of time to do.

Related: It would also be nice to "silence" notifications for a project, typically because it is a new one and we haven't got the build/run process right and don't want to spam people.

Sounds like you may be interested in TW-2829 as well.

The last line in my previous comments should read: I would like to silence a "build" and not the whole "project".

We hopefully will fix this in Benares.
However, it seems that fixing this will save developers only from one "false" notification: namely when build was failed and my change did not change the way it failed before my commit.

Developer will still be notified if a failing build changes the way it fails and this will continue until the first successful build. (We will address this in the scope of TW-3611, but I am afraid only after Benares)

What we can try to squeeze into Benares is notification only on the first build failure (and no further notifications no matter how they fail). This basically means the same notification we have for specific build configurations: if the build has my change and it failed - notify me.

Do you think this latter feature would be useful/can improve false positives notification problem?

To restate my desire: I want a notification to each individual responsible for a change that causes any new failure in any single build.

It should save developers from many false notifications. Example of what I am asking for:

Build 1 - ok
Build 2 - I make change - 1 new failure - I get notification
Build 3 - You make change - 1 failure, 0 new - no notification sent
Build 4 - You make change - 2 failures, 1 new - you get notification
Build 5 - I make change - 2 failures, 0 new - no notification sent
Build 6 - I make change - 2 failures, 1 new (1 fixed) - I get notification (I boken something else)
Build 7 - You and I make change - 0 failures - everyone is happy


"What we can try to squeeze into Benares is notification only on the first build failure" - does this mean that I would NOT get the notificaiton in "build 6" from my example above?

I really don't fully understand the notification rules and they are not documented anywhere (when I checked a while back). Would like to suggest that the rules be documented better, or better notes on the UI.

Thanks!

What Doyle is asking for is what I would like to see as well. Same goes for the remark about documentation as well!

I'd also like to add my support to Doyle's request. One clarification though - I assume in Build 7 notifications of the fixed build would be sent to both developers.

Mail on first success after failure -> TW-764?

Doyle,
Robert,
Dmitri,

The example described seems to be covered by the rule "notify only on the first build failure, but ignore same-failing builds".
In the example, you will get notification in the Build 6, since the build fails and not the same way as the previous one (another set of tests is failing)

However, the situation becomes more complicated once the build fails with compilation error: I will get a notification about failed compilation, but if somebody else fixes compilation, I will not get notification on the failed tests that I might have broken together with compilation.
Or I commit changes that does not fix anything: no notifications are sent in this case since the build fails the same and I am never notified that my attempted fix did not change anything.

All these can be addressed (see the above mentioned TW-3611 and TW-3614), but the question is what is the minimum set we can implement now to deliver the sufficient value?

Will only adding an option to ignore same-failing build suffice (this issue)?
Or it will be not much without first failed/first successful notifications implemented (TW-764)?

Thank you for noting, we will try to describe them better. See TW-3615

For me, an option to ignore same-failing build has value on its own.

I think the minimum for me is: When a build status gets worse (ie. there is a new failure of any type - either compile or test), then simply notify ONLY those people that made a change that affected that build and thus could be responsible. When things get better, I don't want to be notified. I believe this is where some other people want different behavior than me.

So:

Build 23: 6 failures - 0 new
Build 24: 4 failures (2 fixed by Me) - I do NOT want a notification. But I believe Dmitri would want one.
Build 25: 0 failures (fixed by someone else) - ditto

In essence I want notifications when something UNEXPECTED happens. Making things equal or better is expected IMHO. I personally don't need a "congratulations" notification when I fix a problem.

Hope this helps and points to a simple solution to understand for the user as well as implement for the developer.

This one is mostly fixed in the latest EAP build (5813):
if build fails the same way as previous one "my changes" notifications will not be sent
Also, a new (somewhat vague) setting appeared in the notifications: "Ignore failures not caused by my changes" this will try to send no notifications if the tests failed already passed in some build after my change.

There still can be some issues and if you discover some - please let us know!

OK, thank you. Unfortunately I won't be able to test and give you feedback until TW-3641 is fixed since all of my compiles are failing.

Yegor,
Does this only take into account the junit test results?
How do you propose to integrate a build validation framework not based on junit?
In our case our build failures are dependent on other steps:

• static analysis errors (checkstyle...)
  
• function test errors (in our case fitnesse)
  
• test coverage thresholds failures
  

"build fails the same" means TeamCity-calculated signature of the build failure is the same. AFAIK signature includes compilation error message, if any, and names of the detected failed tests. Currently, TeamCity can detect only JUnit, TestNG and NUnit tests - these are shown on "Tests" tab on the build.

Probably, we can include build status text (supplied via teamcity-info.xml) into the build failure signature so that changing build status text will notify TeamCity build failed differently.

This can make your cases working OK in the "do not notify me if my change does not affect the way the build fails" part.

Considering "Ignore failures not caused by my changes", this will likely require some plugin-based support to make TeamCity support your cases.

Could you please elaborate on what notifications you want/do not want to receive for the build types you mentioned?
I'd assume:
* static analysis errors (checkstyle...)
Send notification only when my changes introduce new items. We can try to do this for Inspections and Duplicates, but third-party tools would need plugin-based support.

• function test errors (in our case fitnesse)

Unless TeamCity supports fitnesse tests as "tests", it will handle the builds just like build with no tests: notify on failed. It seems that TeamCity will regard all builds as failed the same way.

• test coverage thresholds failures

If the build fails notification will be sent (provided user has build failure notification turned on for builds with own changes). Do you expect some specific behaviour here?

I'm trying to test this, but I'm not receiving any mails. (Yes, I have checked the mailserver and my To address!) How can I debug?

Robert,

Please create a separate issue with description of your problem (what notificator do you use, what are notification settings and what notifications do you receive and what not). All the notification sending attempts should be logged into teamcity-server.log by default.

Thanks - since it's probably just user error I haven't opened a bug, but a discussion thread.
http://www.intellij.net/forums/thread.jspa?threadID=271113