This morning I ran into a minor issue where someone pushed a totally different project into one of our repositories, turning our FreeRADIUS Configuration repository into a functional clone of the FreeRADIUS source repository. As you might imagine, this was…somewhat disruptive. After casting about for a bit, I found that recovery was not as bad as it seemed. There might be some missing branches after this, but as most of them should have been deleted, I’m not feeling too bad.
This works best (at all?) if you have some repositories floating around with recent, good copies of what you want to recover.
From that machine, I did the following:
$ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
$ git log -n 1
commit f019fb8e8d15a28e721441d21a3d8a754e0d210d
Author: Jacob M. Dawson <dawson@vt.edu>
Date: Wed Oct 8 10:55:17 2014 -0400
Implemented forgotten correct ldap clients config in load-balancing
$ git push -f origin f019fb8e8d15a28e721441d21a3d8a754e0d210d:master
So that reset the master to something decent (the last update we’d made to it, thankfully). From there, I pushed the branches I cared about, as they’d also been removed:
$ git checkout netadmin
$ git push -u origin netadmin
$ git checkout eduroam
$ git push -u origin eduroam
Etc.
Finally, I went about removing all the bad branches and tags (this snapshot is after I’d already nixed most of the branches and moved onto tags, but the point remains).
$ git ls-remote origin
8bde98501d3657dfb183913be8cbf187769fed56 HEAD
8aca08fe308044c0aacd9d7947600b4f866de1c1 refs/heads/eduroam
8bde98501d3657dfb183913be8cbf187769fed56 refs/heads/master
8a9d093e110f84391f512080daea89fffcac39e0 refs/heads/netadmin
a687717f49aca81d45c96d636eaf42cd5c7b5d94 refs/remotes/upstream/HEAD
c7119ee310ab13d4a526fbdfc50a798622595bf9 refs/remotes/upstream/coverity_scan
a687717f49aca81d45c96d636eaf42cd5c7b5d94 refs/remotes/upstream/master
5f715dba4d2dbdb268bf60fcc656352274930941 refs/remotes/upstream/v1.1.x
6040566cfa969da1bce085ee48b4cd3e433e87d8 refs/remotes/upstream/v2.1.x-apple
7df037e8f4f6d8309a896aed53c360a90d380e34 refs/remotes/upstream/v2.x.x
f46e3f1e4b983d905de956144035bbcaced666c2 refs/remotes/upstream/v3.0.x
15a2d332a02b99fbf04ca4dd41310ea1ab109fbe refs/tags/release_0_9_2
8e6b04132fd49c6fa145851e4ba919385cd0b3ba refs/tags/release_0_9_3
05a08578990a46fb0d3e52c85ab276d462f5511f refs/tags/release_1_0_0
815f3cebf703ed5354211299c3be36d0fde6cc1d refs/tags/release_1_0_0_pre1
ee92544adaea26d89148ea5da7f40730629fd2d1 refs/tags/release_1_0_0_pre2
ab5ebe15a89b5e63914a7b3772a76c35b078055f refs/tags/release_1_0_0_pre3
8c6451806c5af7f5a812098dfc1fef4807c6da42 refs/tags/release_1_0_1
90600a278338ec9d680c43a97e0a636d81b52f2e refs/tags/release_1_0_2
334d34a539e869b054fc473f20ab16a1f5e98493 refs/tags/release_1_0_3
198f900c5d6521f92938f8927f893a9879437b8d refs/tags/release_1_0_4
c13c8bdb58aa100edb6cf678b6c8bdaeefe427c4 refs/tags/release_1_0_5
5b757a36fe84b2cf24df6a367afdaf6294d78997 refs/tags/release_1_1_0
8102de9efd8d6b690bad3aa3ee687fccfbd0a55d refs/tags/release_1_1_0_pre0
6efc0c7c54d93c28c79dcd4b7d45d42774fee2ac refs/tags/release_1_1_1
1c8bd5567bb5e73d28e8350102f4e961d37cb735 refs/tags/release_1_1_2
4a44f2aad41216ae9aa0a9dc49b0b1e093f07143 refs/tags/release_1_1_3
2289360fec804c1b560cbf98f4a06baccea45de5 refs/tags/release_1_1_4
6cdb8bec5014efd32327346c989814bedb4c80be refs/tags/release_1_1_5
ecea7d7baa5b24abd3b0aa016f873ff0a9307dd1 refs/tags/release_1_1_6
2ee0fbe67ef0c8c04e88036c98692079f5179165 refs/tags/release_1_1_7
139e62a134fbaceb28eee001ab87f1d1e3092c33 refs/tags/release_1_1_8
74ef9b64f7cd631b13d7a61bd1588a2bfc75ba39 refs/tags/release_2_0_0
64c447ee26d79773230ddf37631058128fe301fe refs/tags/release_2_0_0_pre1
a22023e192fb491903f1e492d74989b0a4324cb9 refs/tags/release_2_0_0_pre2
dfb32dca4abfaf76ed59b81c6b333bcc91be0ee9 refs/tags/release_2_0_1
db35d2eba513c9aa0fdf127b7e66c5c9bb036bc0 refs/tags/release_2_0_2
7cd6369a90399dc8836242dde3ce8a3e9595cb42 refs/tags/release_2_0_3
ae87fd0cf769498d5ae9a809cc2d703a7494561e refs/tags/release_2_0_4
edf2361c554fe2fe2a1b9a50e81369aaa44a74f9 refs/tags/release_2_0_5
bacf36805f38d059f58c5c62141f91611e8a1bbc refs/tags/release_2_1_0
1c8d4d4cad8a07e96c0898fcf1cda8d2f3982495 refs/tags/release_2_1_1
1d1e9b7372abecfc0942b127e5faf90df547e198 refs/tags/release_2_1_10
34c68ba800632ccca564f6bdcb186e32886684c1 refs/tags/release_2_1_11
70c228536c4a9113caf509fbfeab305ccfcd25c8 refs/tags/release_2_1_12
239ba3f92884be49adc90b1383aa8b54cb150fe9 refs/tags/release_2_1_2
4c44a40e68c08b445123100b39855f36b65bf5f9 refs/tags/release_2_1_3
a2c610854bef0f961573f15022d77b1088e8c819 refs/tags/release_2_1_4
29ce823f72ffdfb2051765a6417126ee91e22552 refs/tags/release_2_1_6
af07ace2815910610c0d39de90e4c0cf0735188d refs/tags/release_2_1_7
4d396a436e73b0331167907e2d84931645b72648 refs/tags/release_2_1_8
7c4c5199e6631d1e93c06025a45e8ba967276986 refs/tags/release_2_1_9
393dd3c3a1a9582040c419c9be35fd4ce56d38cd refs/tags/release_2_2_0
b511525f17d3f40ba28d11d7797a827298df1333 refs/tags/release_2_2_1
96c47460fad08af8822769d75636ab6d5253d3bf refs/tags/release_2_2_2
28e67574dc8011e86c462b7e8204e180a3667ac5 refs/tags/release_2_2_3
127137c63569fbb61555df9f8f9c7364a0fa830d refs/tags/release_2_2_4
b75980dc5b8b38f562948492695b9d6151a2cb94 refs/tags/release_2_2_5
580424ea12feeb5933f1aaac33fd5f9e2fa2ee60 refs/tags/release_3_0_0
335a2add3c5da124b4585bd51afed3681c3c5aea refs/tags/release_3_0_0_beta0
4571cd3cc6150ad61e136bf28de8f47e8fca8459 refs/tags/release_3_0_0_beta1
82961d74ffcbf1ec4a4f13b6cc79ddc35d7e656d refs/tags/release_3_0_0_rc0
484451695676f1d38ecd376c37d7e648731d8d77 refs/tags/release_3_0_0_rc1
9dbdad73ca823f5d2fbb0cbc5c34aec714a9e0d3 refs/tags/release_3_0_1
808a9b3a8ff7ebac794519a1e842507c9a99107b refs/tags/release_3_0_2
3366cf0a98513ee15e1b96e3996f929ba5e611a4 refs/tags/release_3_0_3
7c9d5fbe83a67934bff42c1093d50daacbf1c083 refs/tags/release_3_0_4
3d44fdde2fc3b6d4173747b4cf2e2471688f9651 refs/tags/release_3_0_4_rc0
4039aa19fbbea4503a81be519b0c3703783cb38d refs/tags/release_3_0_4_rc1
7053bedffefec30c8d1a70c12ed47dd0325ab8ca refs/tags/release_3_0_4_rc2
There were quite a few, so after digging around some more, I located a post on how to bulk delete remote tags.
This was most helpful, but I found I needed a little bit of tinkering with the regex to get it to fly, ending up with something like this:
$ git ls-remote --tags origin \
| awk '/^([0-9a-f]+)\t(refs\/tags\/release.*$)/ {print ":" $2}' \
| xargs git push origin
The proposed solution is fairly destructive, so I checked my work:
$ git ls-remote origin \
| awk '/^([0-9a-f]+)\t(refs\/tags\/release.*$)/ {print ":" $2}'
:refs/tags/release_0_9_2
:refs/tags/release_0_9_3
:refs/tags/release_1_0_0
:refs/tags/release_1_0_0_pre1
:refs/tags/release_1_0_0_pre2
:refs/tags/release_1_0_0_pre3
:refs/tags/release_1_0_1
:refs/tags/release_1_0_2
:refs/tags/release_1_0_3
:refs/tags/release_1_0_4
:refs/tags/release_1_0_5
:refs/tags/release_1_1_0
:refs/tags/release_1_1_0_pre0
:refs/tags/release_1_1_1
:refs/tags/release_1_1_2
:refs/tags/release_1_1_3
:refs/tags/release_1_1_4
:refs/tags/release_1_1_5
:refs/tags/release_1_1_6
:refs/tags/release_1_1_7
:refs/tags/release_1_1_8
:refs/tags/release_2_0_0
:refs/tags/release_2_0_0_pre1
:refs/tags/release_2_0_0_pre2
:refs/tags/release_2_0_1
:refs/tags/release_2_0_2
:refs/tags/release_2_0_3
:refs/tags/release_2_0_4
:refs/tags/release_2_0_5
:refs/tags/release_2_1_0
:refs/tags/release_2_1_1
:refs/tags/release_2_1_10
:refs/tags/release_2_1_11
:refs/tags/release_2_1_12
:refs/tags/release_2_1_2
:refs/tags/release_2_1_3
:refs/tags/release_2_1_4
:refs/tags/release_2_1_6
:refs/tags/release_2_1_7
:refs/tags/release_2_1_8
:refs/tags/release_2_1_9
:refs/tags/release_2_2_0
:refs/tags/release_2_2_1
:refs/tags/release_2_2_2
:refs/tags/release_2_2_3
:refs/tags/release_2_2_4
:refs/tags/release_2_2_5
:refs/tags/release_3_0_0
:refs/tags/release_3_0_0_beta0
:refs/tags/release_3_0_0_beta0^{}
:refs/tags/release_3_0_0_beta1
:refs/tags/release_3_0_0_rc0
:refs/tags/release_3_0_0_rc1
:refs/tags/release_3_0_1
:refs/tags/release_3_0_2
:refs/tags/release_3_0_3
:refs/tags/release_3_0_4
:refs/tags/release_3_0_4_rc0
:refs/tags/release_3_0_4_rc1
:refs/tags/release_3_0_4_rc2
Work checked, let fly:
$ git ls-remote --tags origin \
| awk '/^([0-9a-f]+)\t(refs\/tags\/release_[01].*$)/ {print ":" $2}' \
| xargs git push origin
To git@git.it.vt.edu:cns/freeradius-config.git
– [deleted] release_0_9_2
– [deleted] release_0_9_3
– [deleted] release_1_0_0
– [deleted] release_1_0_0_pre1
– [deleted] release_1_0_0_pre2
– [deleted] release_1_0_0_pre3
– [deleted] release_1_0_1
– [deleted] release_1_0_2
– [deleted] release_1_0_3
– [deleted] release_1_0_4
– [deleted] release_1_0_5
– [deleted] release_1_1_0
– [deleted] release_1_1_0_pre0
– [deleted] release_1_1_1
– [deleted] release_1_1_2
– [deleted] release_1_1_3
– [deleted] release_1_1_4
– [deleted] release_1_1_5
– [deleted] release_1_1_6
– [deleted] release_1_1_7
– [deleted] release_1_1_8
Remediation
This was clearly a suboptimal event. Fortunately, remediation steps in this case were fairly easy. Most of the other members of the team for this repo are in the ‘Developer’ role, while only I and one other person are in ‘Master’ or ‘Owner.’ As such, I set the master branch to protected, which will prevent these from surprising me in the future.
As an aside, you’ll notice I chopped up the work a bit. The following pair of tags made git fairly unhappy:
:refs/tags/release_3_0_0_beta0
:refs/tags/release_3_0_0_beta0^{}
Removing the plain boring beta0 took the wonky one with it, as I discovered, so in the end, it worked out ok.