How financial trading firms can get the time wrong by 35 seconds
There is a regulation called FINRA rule 7340 which requires US financial traders to keep their clocks synchronized to within one second of the official time from the National Institute of Standards and Technology (NIST). This standard is probably way too weak for modern trading, which often takes place in microseconds, but even so, complying with the standard is a lot harder than it appears. In fact, clocks that are wrong by 35 seconds are not all that unusual because of astronomy and "leap seconds".
NIST atomic clocks are used to synchronize clocks in Global Positioning System (GPS) satellites. Most financial trading companies depend on "network clock" devices that receive time from the GPS satellites and then send it out over computer networks. But earth's orbit and rotation is not as regular as atomic clocks are, so atomic time needs to be adjusted every now and then by a "leap second" just as calendars are adjusted with leap years. Accumulated leap second adjustments now require a 35 second adjustment to atomic clock time. Network clock devices are supposed to keep all of that straight and make the appropriate adjustment, but we keep running into cases where our software sends an alarm about a time source that is 35 seconds off. The error can be caused by something as simple as an incorrectly configured device or by a small chunk of bad flash memory.
Without technology to cross check clocks against each other and to fail over when necessary, simple equipment failures can silently cause time to jump by more than 1/2 minute - a lot more than Rule 7340 permits.