MW-1.30.0-wmf.2 deployment blockers
Closed, ResolvedPublic

Related Objects

StatusSubtypeAssignedTask
Resolvedthcipriani
ResolvedPRODUCTION ERRORAnomie
ResolvedMhurd
ResolvedMhurd
Resolved Mholloway
ResolvedABorbaWMF
Resolved ssastry
Resolved ssastry
Resolveddchan
ResolvedJdforrester-WMF
ResolvedJdforrester-WMF
ResolvedCatrope
ResolvedCatrope
ResolvedAnomie
Resolvedmatmarex
Resolved Gilles
ResolvedTheDJ

Event Timeline

Actually, I went too far. Chad will be branching 1.30 this week. I'll rename these after his announce email.

greg renamed this task from MW-1.29.0-wmf.23 deployment blockers to MW-1.30.0-wmf.2 deployment blockers.Apr 25 2017, 9:07 PM
greg updated the task description. (Show Details)

Added some blocking tasks that should be resolved before we branch/roll out wmf.2 (which caused production breakage last week).

Mentioned in SAL (#wikimedia-operations) [2017-05-23T17:09:04Z] <thcipriani> starting branch cut for 1.30.0-wmf.2 T163512

From the newly created blocking sub-task:

setting as UBN! since we're stuck on wmf.1 until we can figure this out.

Just to be clear: as of this moment wmf.2 is on group0 (testwikis) and group1 (non-wikipedias, like commons, wikisource, etc). group2 ('pedias) is the only one on wmf.1. There is no clear indication that reverting group0 and group1 is needed at this moment. Please shout if that is incorrect/changes!

@greg cawiki and hewiki are on wmf.2, if needed they also need to revert.

@greg cawiki and hewiki are on wmf.2, if needed they also need to revert.

Rollback is due to performance issues related to enwiki ( T166345 ), most probably due to load. cawiki / hewiki can stay on wmf.2 for now unless there is a reason to roll them back.

Cross-posting from the blocking task:

Status update from RelEng:

We don't have any plans to roll out wmf.2 any further today (Friday the 26th). We still don't have a root cause and we're going into a 3 day weekend.

We have confirmation from the Performance Team that they (Krinkle) will begin investigating this from their side in a couple of hours time (around 19:30 UTC).

Ping us (probably thcipriani on IRC) if you want any other production app/debug servers to have wmf.2 put on them for testing. We'll/he'll be around and available for that.

Johan, adding it here as Tech News is likely wrong now.

Keeping track of it. Tentatively adding contradictory information to Tech News; will remove whatever's not relevant come Monday.