
In a wee bit of a spot :( Have a git repo with squashed bugs coming back from the dead - and a conference tomorrow where early adopters are supposed to be trained on using the software. The strange thing is that there doesn't seem to be any indication of corruption or bad merges. There are a few dangling commits, blobs and trees but they don't look to be related. Not having run into this scenario before I'm not quite sure where to start. Any pointers or suggestions? Thanks, -- Scott Elcomb @psema4 http://psema4.com/pubkey.txt http://www.pirateparty.ca/

I don't know if git-bisect would help, but I've found it useful to isolate where a particular change took place. You might find that there's a merge that was done in the wrong direction (it happens). Cheers, Alex On Fri, Nov 7, 2014 at 1:14 PM, Scott Elcomb <psema4@gmail.com> wrote:
In a wee bit of a spot :(
Have a git repo with squashed bugs coming back from the dead - and a conference tomorrow where early adopters are supposed to be trained on using the software.
The strange thing is that there doesn't seem to be any indication of corruption or bad merges. There are a few dangling commits, blobs and trees but they don't look to be related.
Not having run into this scenario before I'm not quite sure where to start. Any pointers or suggestions?
Thanks, -- Scott Elcomb @psema4 http://psema4.com/pubkey.txt http://www.pirateparty.ca/
--- GTALUG Talk Mailing List - talk@gtalug.org http://gtalug.org/mailman/listinfo/talk
-- Alex Beamish Toronto, Ontario

On Fri, Nov 7, 2014 at 1:26 PM, Alex Beamish <talexb@gmail.com> wrote:
I don't know if git-bisect would help, but I've found it useful to isolate where a particular change took place. You might find that there's a merge that was done in the wrong direction (it happens).
Thanks Alex. So far no luck - everything looks about right (but obviously isn't) :( Alas things get even stranger... using one of about a half-dozen similar issues, running a recursive diff on files from both my dev and staging environments show they are the same (same with the dataset the code is working on - they appear to be identical), however the two environments are not acting (sorting, specifically) the same. Code, datasets, db servers, OS's & packages are - for all intents and purposes - the same (dev work takes place inside Vagrant images) In this particular case, on staging we're seeing a bug that was (and according to the file system, *is*) fixed and QA'd to death months ago. On dev it is acting as expected (ie. is fixed). -- Scott Elcomb @psema4 http://psema4.com/pubkey.txt http://www.pirateparty.ca/

On 11/07/2014 02:54 PM, Scott Elcomb wrote:
, however the two environments are not acting (sorting, specifically) the same. Check the character set: sort(1) guarantees it sorts bytes. Gnu sort sorts characters, which are not the same thing, and carries the warning:
*** WARNING *** The locale specified by the environment affects sort order. Set LC_ALL=C to get the traditional sort order that uses native byte values. The Eunuchs wanted to sort lines of bytes separated by ASCII newline characters, 0x0A, and did. Gnu extended this to other character sets, but silently changed the default. Linus would not have been impressed (;-)) --dave -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb@spamcop.net | -- Mark Twain

On Fri, Nov 07, 2014 at 01:14:44PM -0500, Scott Elcomb wrote:
In a wee bit of a spot :(
Have a git repo with squashed bugs coming back from the dead - and a conference tomorrow where early adopters are supposed to be trained on using the software.
The strange thing is that there doesn't seem to be any indication of corruption or bad merges. There are a few dangling commits, blobs and trees but they don't look to be related.
Not having run into this scenario before I'm not quite sure where to start. Any pointers or suggestions?
Some people think the way to check in file changes is to make their changes, copy their file somewhere, then checkout the current version, replace it with their version (without merging any other changes of course), and check it in. It's a great way to loose changes. Of course people simply not understanding the merge resolution in git can also cause things to get lost. -- Len Sorensen

To verify you've got the identical files on dev and staging, you can do some variation on find . -name *.pl | xargs md5sum | sort and compare the output of that command from the two platforms. Assuming there's no configuration information in those files that differ between the environments, that should confirm or deny you have identical installations. Restarting the web server meant something ten years ago, these days, not so much. Apart from that, try to nail down why the bug that's been solved is reappearing -- what fix was made to solve that (it's documented in your git commit comments, right?). Maybe go back to basics and add a few logging lines to confirm that things are working as they should. Good luck -- debugging is indeed a black art. :) Alex On Fri, Nov 7, 2014 at 3:19 PM, Lennart Sorensen < lsorense@csclub.uwaterloo.ca> wrote:
On Fri, Nov 07, 2014 at 01:14:44PM -0500, Scott Elcomb wrote:
In a wee bit of a spot :(
Have a git repo with squashed bugs coming back from the dead - and a conference tomorrow where early adopters are supposed to be trained on using the software.
The strange thing is that there doesn't seem to be any indication of corruption or bad merges. There are a few dangling commits, blobs and trees but they don't look to be related.
Not having run into this scenario before I'm not quite sure where to start. Any pointers or suggestions?
Some people think the way to check in file changes is to make their changes, copy their file somewhere, then checkout the current version, replace it with their version (without merging any other changes of course), and check it in. It's a great way to loose changes.
Of course people simply not understanding the merge resolution in git can also cause things to get lost.
-- Len Sorensen
--- GTALUG Talk Mailing List - talk@gtalug.org http://gtalug.org/mailman/listinfo/talk
-- Alex Beamish Toronto, Ontario

Scott Elcomb wrote:
Have a git repo with squashed bugs coming back from the dead ...
Any chance it's a C project with missing dependencies in the makefiles? That's a classic way of checking out the right source code but having lingering bad binaries. Clock skew can do it too: if the machine hasn't always had stable time then "before" and "after" become wonky. If in doubt, make clean (and verify that doesn't leave anything that isn't source) and rebuild all. -- Anthony de Boer
participants (5)
-
Alex Beamish
-
Anthony de Boer
-
David Collier-Brown
-
Lennart Sorensen
-
Scott Elcomb