A day of waiting
May. 21st, 2002 04:05 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
chrisvenusToday has in a lot of ways been a crap day.
10:30 - we uploaded some files from staging to live. This means that they have been sitting on our staging environment which emulates the live and it has all worked fine on there. Then they get moved to live.
12:30 - I notice that we have had no orders go through in the last two hours. That is they have been palced on the website but they have not gone through to the backend system. This is clearly a bad thing because it means nobody will ever know to deliver these people their stuff.
13:00 - all orders placed since 10:30 are sent through manually. Clients and customers shouldn't notice a thing. *fingers crossed* Code search starts.
13:30 - Why does it work on staging but not on live? Check through all machine specific possibilities.
14:00 - Chris sitting slightly bored as he waits for an order to be placed to see if he can fix it. Live system = no tests. They are all done already don't forget... Much of next few hours is spent waiting for orders to be placed to see if things work.
16:00 - Discover a file that was changed and not on the list of things to promote to live. Thus this critical file didn't get copied up and thus everything broke. File now copied up.
16:05 - Chris waits and writes an LJ entry describing how annoyed he is with the unnamed individual who failed to get this change copied up properly.
I can't believe how much I have wasted this afternoon. Still to be fair it is not my fault. Somebody else failed to copy up the right code and so my code was relying on soemthing that wasn't there. I think I did damn well to a) notice that this problem was occuring and b) to fix it all with no client/customer notice.
Anyway, I should start doing some other stuff now. I'm feeling very drained after 3 hours of solid panicing though. I'll try to work out something not too hard that I can do for the next hour or so.
10:30 - we uploaded some files from staging to live. This means that they have been sitting on our staging environment which emulates the live and it has all worked fine on there. Then they get moved to live.
12:30 - I notice that we have had no orders go through in the last two hours. That is they have been palced on the website but they have not gone through to the backend system. This is clearly a bad thing because it means nobody will ever know to deliver these people their stuff.
13:00 - all orders placed since 10:30 are sent through manually. Clients and customers shouldn't notice a thing. *fingers crossed* Code search starts.
13:30 - Why does it work on staging but not on live? Check through all machine specific possibilities.
14:00 - Chris sitting slightly bored as he waits for an order to be placed to see if he can fix it. Live system = no tests. They are all done already don't forget... Much of next few hours is spent waiting for orders to be placed to see if things work.
16:00 - Discover a file that was changed and not on the list of things to promote to live. Thus this critical file didn't get copied up and thus everything broke. File now copied up.
16:05 - Chris waits and writes an LJ entry describing how annoyed he is with the unnamed individual who failed to get this change copied up properly.
I can't believe how much I have wasted this afternoon. Still to be fair it is not my fault. Somebody else failed to copy up the right code and so my code was relying on soemthing that wasn't there. I think I did damn well to a) notice that this problem was occuring and b) to fix it all with no client/customer notice.
Anyway, I should start doing some other stuff now. I'm feeling very drained after 3 hours of solid panicing though. I'll try to work out something not too hard that I can do for the next hour or so.