[Volute] r4887 - trunk/projects/time-domain/time-series/note

Volute commit messages volutecommits at g-vo.org
Thu Apr 5 17:59:25 CEST 2018


Author: lmichel
Date: Thu Apr  5 17:59:25 2018
New Revision: 4887

Log:
LM contri version 0

Modified:
   trunk/projects/time-domain/time-series/note/laurent.tex

Modified: trunk/projects/time-domain/time-series/note/laurent.tex
==============================================================================
--- trunk/projects/time-domain/time-series/note/laurent.tex	Thu Apr  5 14:07:04 2018	(r4886)
+++ trunk/projects/time-domain/time-series/note/laurent.tex	Thu Apr  5 17:59:25 2018	(r4887)
@@ -26,73 +26,112 @@
 The present section explores an simplified mapping syntax relying on the strengths of the VO-DML workflow while facilitating the job of both data providers and data consumers.
 This is not a complete proposal, but a proof of concept validated on TD data.
 
-\subsubsection{Basics}
-The VODML block is split in 3 sections as 
+\subsubsection{Mapping Syntax}
+\paragraph{Basics}
+The design of this  mapping syntax being not complete, we just describe here the outlines of this simplification and the features required to map the present sample of time domain data. 
+
+The VODML block keeps split in 3 sections as shown below.
+
+\smallskip 
+\scriptsize
+\begin{tabular}{|r|l|}
 
-\begin{tabular}{c|l}
 \hline
-   MODELS & list of the used model  \\
-   GLOBALS & not implemented here \\   
-   TEMPLATES & \makecell[l]{One template per table  \\hosting the mapping of the  \\ data classes contained in that table} \\
+   MODELS & References to all used models including imports \\
+   GLOBALS & Not implemented here \\   
+   TEMPLATES & \makecell[l]{One template per table} \\
 \hline
 \end{tabular}
+\normalsize
+\smallskip 
+
+Each template refers to a particular table and contains de mapping of the classes hosted by that table. The mapping is bases on the 3 concepts necassary to describe a hierarchy of classes:
+
+
+\smallskip 
+\scriptsize
+\begin{tabular}{|r|l|}
+\hline
+   VALUE & \makecell[l]{Model leaf. It either points to a PARAM or a FIELD \\or it has its own value like a literal}  \\ \hline
+   INSTANCE & \makecell[l]{Denotes a class.\\ An INSTANCE is a tuple of elements which be either \\ VALUE, INSTANCE or COLLECTION}  \\ \hline
+   COLLECTION & \makecell[l]{Denotes a list or an array. \\ A collection can contain  be either \\ VALUE, INSTANCE or COLLECTION} \\\hline
+\end{tabular}
+\normalsize
+\smallskip 
+
+ Each one of these elements supports some attributes wich can be mandatory (mand.) or optional (opt). It should be noted that some attributes such as units or formats are not described here as well as the foreign key mechanism. 
+
+\smallskip 
+\scriptsize
+\begin{tabular}{|r|c|c|c|c|c|c|}
+\hline
+    Attribute:     & \textbf{@dmrole} & \textbf{@dmtype} & \textbf{@ref} & \textbf{@tableref} & \textbf{@size} & \textbf{@value} \\ \hline
+   VALUE  & mand. & none & mand. if no @value & none & none & mand. if no @ref \\ \hline   
+   INSTANCE  & mand. & opt. & none & opt. & none & none \\ \hline  
+  COLLECTION  & mand. & none & none & none & opt & none \\ \hline
+
+\end{tabular}
+\normalsize
+\smallskip 
+
+\paragraph{VALUE}
+The @dmrole attribute is the one set in the VO-DML model. Other VO-DML attributes (e.g. dmtype) can be retrieved in the VO-DML file from both VALUE location and dmrole.
+\begin{itemize}
+\item If a VALUE element has a @ref attribute but no @value, its value is taken out from the refered element of the table refered by the current TEMPLATES. The client must first search in the FIELDS, and then in the PARAMS. If the VALUE points onto a FIELD but without specified multiplicity, the client will take the values of the first row.
+\item If a VALUE element has a @value attribute but no @ref, the value of @value is taken.
+\item If a VALUE element has both @value and @ref, the client must first to solve @ref and then @value.
+\end{itemize}
+
+\paragraph{INSTANCE}
+The @dmrole attribute is the one set in the VO-DML model.
+\begin{itemize}
+\item An INSTANCE without @dmtype attribute matches the type set in the VO-DML file. Otherwise @dmtype can specify a subtype of the model one. This feature is used to support abstract classes.
+\item An INSTANCE with a @tableref is mapped in the TEMPLATES designated by @tableref and having the same @dmrole.
+\end{itemize}
 
+\paragraph{COLLECTION}
+The @dmrole attribute is the one set in the VO-DML model. A the COLLECTION elements must have the same type. The number elements in a collection is specified the @size attribute.
+\begin{itemize}
+\item A COLLECTION without @size attribute contains one instance of each element mapped (see example below)
 \tiny 
 \begin{lstlisting}[language=xml]
-<html>
-    <head>
-        <title>Hello</title>
-    </head>
-    <body>Hello</body>
-</html>
+<collection dmrole='Axes'>
+    <instance  dmrole='Axis'>
+        <value dmrole='name' valu='time' />
+        .....
+    </instance>
+    <instance  dmrole='Axis'>
+        <value dmrole='name' value='observable' />
+        .....
+    </instance>
+</collection>
 \end{lstlisting}
 \normalsize
 
-\tiny 
-\begin{lstlisting}[language=python]
-for ac in mapping_generator.mapped_abstract_types :
-        print("Abstract type mapped " + ac)
-        for sc in mapping_generator.get_sub_types(ac):
-            print("   " + sc)
+\item A COLLECTION with a @size attribute contains as many instances of the mapped element as requested by @size. To be consistant, when @size is set, COLLECTION must contain only one mapped entity. If @size equals to * or -1, the collection is populated as long as there is incoming data. This is for instance the case when the COLLECTION maps photometric points read row per row in a table.
+\tiny  
+\begin{lstlisting}[language=xml]
+<collection dmrole='Points' size='-1'>
+    <instance  dmrole='NDPoint'>
+        .....
+    </instance>
+</collection>
 \end{lstlisting}
 \normalsize
+\end{itemize}
 
- \subsubsection{Goal}:
- Showing that a model mapping well designed can allow cient to read and interpret VOTable 
- without consideration for the way data are arranged.
- 
- \subsubsection{Syntax}:
- The Mapping syntax is derived from the VODML mapping proposal. 
- It keeps the same structure <VODML>/<GLOBALS>/<TEMPLATES>, but the tag syntax has been simplified.
- Only the elements useful for the parsing are kept. All other things can be retrieved in the model.
- 
-\subsubsection{Workflow}:
+\subsubsection{Workflow}
+The model used for these serializations, designed with Modelio 3.5, has been presented at (ref ivoa). This serialization has also been tested with SparseCube but the outcomes are less advanced.
+This demonstrator used for the serialization of the data sample relies on 2 specific Python tools especially developped but which can be seen as seeds for further developments.
 
 \begin{itemize}
-\item Model design
-The model has been designed with Modelio 3.5 (see the models directory). 
- A model overview is given by timeseries.diag.png
-\item VODML generation
- The vodml.xml file has been generated by the Tesselation workflow (see on Volute)
-
-\item Mapping template
-A template of the mapping block has been generated by the genAnnotation.py script.
- The purpose of the template is to provide elements which must adapted to be reported 
- by hand in the actual data file.
- Some basics of that mapping can be found on Volute in Tesselation/vo-dml/mappinglite
-\item VOTable annotation
- This is the most tricky part. Mapping elements must be reported into the appropriate templates, 
- @SOURCE pointers must be set and cross-references must be solved.
- There no other way to check the job than running the parser. 
- Unfortunately, understanding errors requires to go deep into the code.
- 
-\item Running the parser
- Install Astropy and run sdss.py
-\end{itemize}
+\item \textit{transform.py} reads a VO-DML model and genererates a single VODML block with unresolved references. The tool supports directives specifying which classes must be used in replacement of the abstract ones. The VODML block must be inserted by hand in the VOtable to be annotated. It must be split in multiple TEMPLATES if data are spread out on muliple tables. The references to actual FIELD and PARAM must also be set by hand.
 
+\item \textit{transform.py} is a basic client which plots the timeseries from the VOTable. The key point is that  \textit{transform.py} is able, to some extent, to read any VOTable mapped with the model.
 
+\end{itemize}
+
+\subsubsection{Conclusions and Prospects}
+This approach confirms the power and versatility of the VO-DML workflow since it doesn't use any feature external to the VODML modeling effort. The main outcome is that a client knowing a model is by construction able to read VOTable annotated with that model. There no need to adapt the client code to specific data collection as long as they are are not too much exotic. The added value of the light syntax sketched here is the same outcomes can be reached with more compact and more readable annotations. This opens the way for light clients such as JavaScript widgets which could have difficulties to process hundreds of XML lines each time they have to parse a VOTable.
  
- KEYPOINTS:
- 1) The code of the parser does not depend on the VOTABLE data
- 2) The time series instance is built from data take out from all the 4 <TABLE>
- ---------------------------------------------
+


More information about the Volutecommits mailing list