We introduced a change that sent requests to a component in our main application server cluster that failed to scale up sufficiently to handle that load. Our main application became unresponsive as a result. We have since reverted that change and are making the necessary changes to ensure this failure does not recur.