[Dachs-support] UWS and user access control

Ivan Zolotukhin ivan.zolotukhin at gmail.com
Wed May 4 15:50:27 CEST 2016


Hi Markus,

> On Tue, May 03, 2016 at 06:28:58PM +0200, Ivan Zolotukhin wrote:
>> Tested, all works as expected now, except a new bug was discovered
>> when accessing a job xml created with HTTP auth headers:
>>
>> In gavo serve debug:
>>
>> 2016-05-03 18:14:46+0200 [-] *X*X* Can only pass-through bytes on
>> Python 2 (see info for traceback)
>> 2016-05-03 18:14:46+0200 [-] "10.120.0.14" - - [03/May/2016:16:14:46
>> +0000] "GET /stop/lvg/r/uws.xml/7dmR7W HTTP/1.1" 500 530 "-" "-"
>>
>> ... and then nothing happens until HTTP timeout is reached.
>
> Ah -- the background here is that if you pass in a unicode string
> (hence the mention of python 2 -- the problem is probably gone in
> python 3) to any header field in twisted web 1 (which nevow and hence
> DaCHS build upon), the whole header is encoded in (IIRC) UCS-32 --
> which of course (and rightly) confuses HTTP clients.
>
> In my patch yesterday I forgot to guard against unicode leaking in
> the location.  I claim this is fixed in rev. 5028.

Apparently this problem has gone, I don't see this in the logs
anymore. However, I don't have very clean and reproducible testing
setup, so let me continue and if I notice something similar later,
I'll let you know.

I'm also having issues with uws-client as it does not handle HTTP
redirects and therefore simply fails when accessing job through the
incorrect URL (which belongs to another UWS) or even posting to it
(e.g. to delete a job). Probably it's easy to fix by telling urllib2
to follow redirects, but this problem anyway is related to our
separate discussion on exposing jobclass in the list of jobs, see
below.

> If a similar thing happens again (in which case I might not be so
> sure where unicode has crept in), you can help me by running
>
>   sudo ngrep -d lo -x ".*" port 8080
>
> repeating the request and sending me the output of the ngrep.
>
> [unsolicited testimonial: ngrep is a terribly useful tool anyway]

Oh, that's a nice one, thanks.

I also comment on another important part of your message from the end of March.

>> Therefore, the minimal thing from DaCHS for me would be actually
>> exposing jobclass and job owner properties in the list of jobs --
>> ideally in a way which is compatible to access through the python-uws
>> client.

> The list of jobs has a fairly strict format, and there's no way to
> have user, let alone jobclass, without resorting to dirty tricks.

> But perhaps the per-user joblists already do the trick for you?

> As to the job classes, there are a couple of ways out, and lacking a
> use case of my own I cannot really say where I should go.

> (a) I could fix the job URL generation so you can tell from the form
> of the job URL which service generated it
> (http://example.com/res1/svc1/async/jobid vs.
> http://example.com/res2/svc1/async/jobid -- essentially, the two
> segments after the hostname tell you what you're looking at).  Doing
> the right URLs would arguably be the right thing to do anyway, but
> I've shyed away from it as it incurs a code uglification.

> (b) I could, in each UWS, just list the jobs submitted by it.  This
> loses the queue perspective, but perhaps that's a silly argument in the
> first place.  Advantage: clean on the code side.

> (c) If I disentangle the UWS' job tables, the problem goes away
> naturally.  Disadvantage: incurs quite a lot of administrative
> overhead.

> So, that's my take on these things -- essentially, it's now up to you
> to say what you absolutely need, what would be really nice and what
> you can live without...

Now I am sure that I can't live without understanding which job
belongs to which service looking on the list of jobs. I do not mind
which way you prefer, a,b or c. For me b) which you seem to like also
looks fine. I won't repeat my previous arguments, I'd just add that by
implementing this one would avoid hacking uws-client as there will be
no more need to look for job properties using the incorrect URL
belonging to another UWS.

--
With best regards,
 Ivan



More information about the Dachs-support mailing list