[Dachs-support] ProgrammingError Can't adapt (was: gavo import lock)

[Markus Demleitner]

Hi Carlos,

On Sun, Nov 13, 2016 at 11:21:43AM -0200, carlosadean at linea.gov.br wrote:
> No, I did not receive none of them but after apply the patch the
> upgrade worked. Thanks!

Great.

> However I have bad news...the import is not working =(

Ok, so there's a different problem.  I take the liberty of changing
the Subject.

> -- Trying to import the example I see a weird message:
> 
> [gavo at devel2 inputs [master]]$ gavo imp txt/q.rd 
> Making data import_content
> Starting /opt/dachs/inputs/txt/data/lmc.txt
> Done /opt/dachs/inputs/txt/data/lmc.txt, read 42
> *X*X* Uncaught exception at toplevel
> *** Error: Oops.  Unhandled exception ProgrammingError.
> 
> Exception payload: can't adapt

This message is very weird indeed.  It is coming from psycopg2, the
Postgresql interface used by DaCHS.  It essentially says that there
was an instance of the datatype in the record prepared by DaCHS that
psycopg2 doesn't know how to push into postgres (i.e., something
you've mapped it).

However, after "can't adapt" there should be the repr() of that "odd"
thing.  Since there's nothing there, it's something that either has
and empty repr() or raised an exception while generating a repr().

I'd have to see txt/q to make an educated guess what it might be; if
you can't send it over the public list, feel free to send it (or at
least the relevant rowmaker) to gavo at ari.uni-heidelberg.de.

> -- Also the purge shows an exception
> 
> [gavo at devel2 inputs [master]]$ gavo purge txt/q.rd 
> *X*X* Uncaught exception at toplevel
> *** Error: Oops.  Unhandled exception ValueError.
> 
> Exception payload: 'txt/q.rd' is not a SQL regular identifier

That's a relatively straightforward and unrelated one.  There are two
different commands for dropping things in DaCHS:

(a) gavo drop <rd> [<ddids>] (potentially with a -f) -- this is
essentially the reverse of imp; where gavo imp executes the data
recipes, gavo drop removes the tables built by them; you can pass
data ids to restrict dropping to certain dd-ids.  This is what you'd
have to do here:

gavo drop txt/q

(b) gavo purge <tablename> -- this just removes tables, both from
postgres and DaCHS' metadata collections (e.g., TAP_SCHEMA and
friends).  You want that if the RD is gone or doesn't describe the
table any more.  It's a sledge hammer that completely ignores RDs.
Hence, <tablename> is a SQL table name, like lmc.objects or something
like that.

I admit that it'd be preferable if the two things could somehow be
sensibly merged into one.  But then functionality is so different
that such a merge would incur quite a bit of magic, and I'd rather
not add more of that unless I have to.

We'll have to improve the manpage, though.

          -- Markus