[Dachs-support] question about leap seconds

Markus Demleitner msdemlei at ari.uni-heidelberg.de
Mon Feb 7 16:51:33 CET 2022


Hi Baptiste,

On Mon, Feb 07, 2022 at 03:05:52PM +0100, Baptiste Cecconi wrote:
> While I'm trying to import data into a table, I'm tumbling on a
> leap-second issue (I guess). 

[Skipping an obligatory rant against leap seconds]

> My <rowmaker> element contains: 
> >                                 <map key="time_end">parseISODT(@time_end)</map>
> 
> and the ISO date time I'm sending to the rowmaker is 
> > time_end = '2016-12-31T23:59:60.000'
> 
> I get the error: 
> > Field time_end: While building time_end in make_louis_juno_waves:
> > second must be in 0..59
> 
> As a matter of fact, this ISO date is correct, since there was a
> leap second inserted at this time. This means that the last second
> of year 2016 is indeed '2016-12-31T23:59:60.000'. 

Well, there's always a bit of trouble I've not forseen.  What DaCHS
does here is dissect the iso datestring a bit more robustly than the
normal time parser does and then passes the bits on to
datetime.datetime.

And sure enough:

  >>> import datetime
  >>> datetime.datetime(2016, 12, 31, 23, 59, 60)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  ValueError: second must be in 0..59

That's even documented in the datetime module documentation, which
says:

  Unlike the time module, the datetime module does not support leap seconds.

Ah well.  Live and learn.

I suppose I'll add something to parseISODT that lets people build
time using the time (actually, calendar) module, like this:

  >>> datetime.datetime.fromtimestamp(
  ... calendar.timegm((2016, 12, 31, 23, 59, 60, -1, -1, -1)))
  datetime.datetime(2017, 1, 1, 1, 0)

(where I refuse to rack my brain on whether that is actually
correct).

What you could do until that's out is a procedure application:

  <apply name="parseTimeWithLeapSecond">
    <setup imports="calendar,gavo.utils.texttricks"/>
    <code>
      mat = texttricks._isoDTRE.match(@TIME_INPUT.strip())
      assert mat, "Bad time literal: "+ at TIME_INPUT
      parts = mat.groupdict()
      @parsed_time = datetime.datetime.fromtimestamp(
        calendar.timegm((
          int(parts["year"]), int(parts["month"]), int(parts["day"]), 
          int(parts["hour"]), int(parts["minute"]), int(parts["seconds"], 
          -1, -1, -1))))
    </code>
  </apply>

(where the input key would be TIME_INPUT and the parsed value would
end up in the parsed_time var).  This is untested; expect minor
trouble.  Also, I think time tuples can't have fractional seconds.  I
hope you don't need them.

Let me know how it goes -- I'm always happy to rant against leap
seconds...

          -- Markus


More information about the Dachs-support mailing list