# [Volute] r4931 - in trunk/projects/time-domain/time-series/note: . figs

Volute commit messages volutecommits at g-vo.org
Thu Apr 19 21:41:03 CEST 2018

Author: mdittmar
Date: Thu Apr 19 21:41:03 2018
New Revision: 4931

Log:
add initial content for my sections

trunk/projects/time-domain/time-series/note/figs/Mark_SimpleTimeSeries.png   (contents, props changed)
Modified:
trunk/projects/time-domain/time-series/note/TSSerializationNote.bbl
trunk/projects/time-domain/time-series/note/TSSerializationNote.pdf
trunk/projects/time-domain/time-series/note/mark-cube.tex
trunk/projects/time-domain/time-series/note/mark.tex

Modified: trunk/projects/time-domain/time-series/note/TSSerializationNote.bbl
==============================================================================
--- trunk/projects/time-domain/time-series/note/TSSerializationNote.bbl	Thu Apr 19 20:07:32 2018	(r4930)
+++ trunk/projects/time-domain/time-series/note/TSSerializationNote.bbl	Thu Apr 19 21:41:03 2018	(r4931)
@@ -1,3 +1,3 @@
\begin{thebibliography}{xx}
-
+\bibitem{gl:mapping} Lemson, Laurino, et al. \emph{Mapping Data Models to VOTable}, 2017
\end{thebibliography}

Modified: trunk/projects/time-domain/time-series/note/TSSerializationNote.pdf
==============================================================================
Binary file (source and/or target). No diff available.

==============================================================================
Binary file. No diff available.

Modified: trunk/projects/time-domain/time-series/note/mark-cube.tex
==============================================================================
--- trunk/projects/time-domain/time-series/note/mark-cube.tex	Thu Apr 19 20:07:32 2018	(r4930)
+++ trunk/projects/time-domain/time-series/note/mark-cube.tex	Thu Apr 19 21:41:03 2018	(r4931)
@@ -1,2 +1,60 @@
\subsection{Mark}
-MCD's Cube model at a glance
\ No newline at end of file
+\label{subsec:model-mark}
+
+For the purposes of this exercise, we generated a toy timeseries model based off current models ( Dataset, Cube, Coords, Meas ).  The model extends Cube model elements to facilitate the modeling of a wide variety of TimeSeries flavors.
+
+There are multiple ways the TimeSeries model could be derived from Cube.
+The details will depend on a thorough review of the full set of use cases
+and requirements.  This toy model focuses on representing the SimpleTimeSeries
+as a fairly direct extension of SparseCube... with the addition of requiring
+a 'timeAxis'.  The cube component elements are designed to facilitate the
+definition of other TimeSeries described in the CSPTimeseries document, but
+are not heavily considered here at this point.
+
+The Simple Time Series model shown in Fig.~\ref{fig:sts-cube}, is defined as an extension of the Cube data model with the addition of only 3 objects.  This approach maximizes the interoperability of the data and reinforces the concept that these data can be considered a slice through a generalized sparse cube.  By creating a VO-DML compliant model, instances of a Simple Time Series can be annotated using their assigned VODML-IDs.  Applications which understand Cube model instances will \textbf{automatically} understand a very large percentage of the Time Series content.
+
+\begin{figure}[h]
+\begin{center}
+  \includegraphics[width=\textwidth]{figs/Mark_SimpleTimeSeries.png}
+\caption{Simple Time Series as extension of Cube.}\label{fig:sts-cube}
+\end{center}
+\end{figure}
+
+Having separate models for different areas of the 'Universe of Discourse' maximizes interoperability, but can take some practice to interpret.  Below is a descriptive outline of the model structure.
+
+\begin{itemize}
+\item  Dataset - provides generic identification and curation metadata for the dataset (eg: Curation, DataID, etc).
+\item  ObsDataset - adds metadata related to the Observation (eg: Target, Instrument, Configuration, etc)
+\end{itemize}
+
+\noindent\textbf{cube:N-Dimensional Cube} -
+This model defines generic cube data products, either as SparseCube-s (eg: Event lists) or Image-s (eg: nD Pixelated image).  The Simple Time Series model will extend from SparseCube, inheriting the following:
+\begin{itemize}
+\item  A reference to the relevant Dataset Metadata instance which describes it.
+\item  A collection of NDPoint-s, each being a collection of associated Observables. (eg: time, position, mag )
+\item  Observable - identifies the item as 'dependent' or 'independent', and provides the measured data value ( value + error ) which is described in the Measurement model.
+\end{itemize}
+
+\noindent\textbf{meas:Measurement} -
+This model defines measured data and their associated errors.  It is a generic model, so provides the basis for many types of measured data.  In some cases, additional metadata is required to fully describe the measurement and how it was obtained.  These are expected to be defined in models which specialize in that content.
+\begin{itemize}
+\item  CoordMeasure - class of Measure given by a Coordinate 'value' plus one or more Error-s also defined in that model.
+\end{itemize}
+
+\noindent\textbf{spec:Spectral Model} -
+This is a dummy Spectral model for illustrating how a specialized LuminosityMeasure could be defined which provides additional metadata related to how the Luminosity value was determined.. eg: which photometry filter was used.
+\begin{itemize}
+\item  LuminosityMeasure - Extends CoordMeasure, so has a Coordinate value + Errors, and adds a reference to associated Photometry metadata.
+\item  NOTE: being an extension of CoordMeasure, this type is automatically usable in a Cube instance, and therefore, SimpleTimeSeries.
+\end{itemize}
+
+\noindent\textbf{ts:Simple Time Series} -
+The Simple Time Series is modeled as a 'view' through a sparse cube.  Time becomes the independent axis, with other observable axes providing the measured data.
+\begin{itemize}
+\item  SimpleTimeSeries - defined as a SparseCube which MUST contain TSPoint-s.
+\item  TSPoint - adds a single 'independent' timeAxis to the generic NDPoint.
+\item  SampleTime - provides the TimeMeasure for the point.  TimeMeasure is defined in the Measurment model, giving the Coordinate value in JD, MJD, ISO, etc.. plus any associated errors.  The Coordinate is associated with a TimeFrame identifying the timescale and reference position. \textbf{NOTE:} This element includes basically all the information described in Table~\ref{tab:metadata}.
+\end{itemize}
+
+With a validated VO-DML compliant model, we can generate serializations which are annotated using the element VODML-ids.  In Section~\ref{subsec:approach-mark}, we describe the serialized sample files generated for this study based on this model, using the current VO-DML/Mapping syntax proposal.

Modified: trunk/projects/time-domain/time-series/note/mark.tex
==============================================================================
--- trunk/projects/time-domain/time-series/note/mark.tex	Thu Apr 19 20:07:32 2018	(r4930)
+++ trunk/projects/time-domain/time-series/note/mark.tex	Thu Apr 19 21:41:03 2018	(r4931)
@@ -1,2 +1,108 @@
\subsection{Marks's approach}
-Mark's serialization approach at a glance
\ No newline at end of file
+\label{subsec:approach-mark}
+
+For this study, we generated annotation for the 2 sample files according to the current VO-DML Mapping Syntax proposal \cite{gl:mapping}.
+Annotation was generated to both the current Cube model and the toy Time Series model described in Section~\ref{subsec:model-mark}.
+The files themselves may be found at the following URL:
+\begin{small}
+http://volute.g-vo.org/svn/trunk/projects/time-domain/time-series/standardized\_votables/MarkCD/
+\end{small}
+
+\subsubsection{Sample Files}
+
+In the text below we will focus on the annotation to the Time Series model as that is the focus of this study.
+\begin{enumerate}
+\item \textbf{BetaLyr\_Vizier}
+    Example file annotated as multiple SimpleTimeSeries data product instances,
+    one per photometric band.
+
+    The structure is:
+    \begin{itemize}
+    \item 1-instance of ObsDataset
+    \item 5-instances of SimpleTimeSeries data products, one for each band (J,K,L,M,N)
+    \item 5-TSPoint templates, each with 1-Time + 1-Observable (Mag with Error )
+    \end{itemize}
+
+    The use of FIELD elements to store metadata (eg: ObsDataset metadata )
+    presents a challenge for proper annotation.  Being in FIELDs, they must be
+    annotated within a TEMPLATE element, which may have multiple instances.
+    This forces the INSTANCES to identify a particular instance via the
+    ORM PRIMARYKEY/FOREIGNKEY elements.
+
+    For a model which specifies that there should only be 1 instance of that
+    type, this is quite a lot of overhead.  This annotation shows that the
+    syntax can accommodate the structure, but in practical application, it
+    is probably easier for both client and provider to re-cast the VOTable
+    segment to provide these singular elements as PARAMs.  This results in
+    very straight-forward annotation and VOTable.
+
+\item \textbf{GaiaSample3Tables}
+    Example file annotated to the toy TimeSeries model, using the current Mapping syntax proposal, as multiple SimpleTimeSeries data products.
+    \begin{itemize}
+    \item  0-instances of ObsDataset
+    \item  3-instances of SimpleTimeSeries data product
+    \item  3-instances of TSPoint with 3 Observables each (1-Time + 1-Flux\_w\_error + 1-Mag)
+    \end{itemize}
+
+    This appears to be a straight forward representation of 3 SimpleTimeSeries
+    data products, each serialized in their own TABLE.  In this example, the
+    filter metadata is constant for each table, so we've annotated them as
+    individual instances which are referred to by the LuminosityMeasures
+    within the TEMPLATE.
+
+    There is no DatasetMetadata included.
+
+    For annotation purposes, I needed to make the following changes.
+    \begin{itemize}
+    \item Changed VOTABLE tag to version 1.4 to validate against vo-dml schema
+    \item Added IDs to TABLE elements to reference in vo-dml annotation
+    \item Added IDs to PARAM elements for Filter metadata
+    \end{itemize}
+
+\end{enumerate}
+
+\subsubsection{Mapping Syntax}
+
+The syntax used is fully documented in the VO-DML Mapping document~\cite{gl:mapping}.  It is important to note that this syntax was designed to satisfy a broad set of use cases and essential requirements identified by the community.  Any syntax being considered by the community for application, should be evaluated against these requirements to ensure that it will meet the needs of the community.
+
+The document fully describes the syntax, so we will not go into detail here.  The key structural components are:
+\begin{itemize}
+\item <VODML> - primary element, all annotation is isolated from the VOTable serialization.
+\item <MODEL> - identifies which vo-dml compliant models are represented in the serialization
+\item <GLOBALS> - instances which are global in nature. (eg: Frames, Dataset Metadata)
+\item <TEMPLATES> - defines multiple instances by iterating over entries in the template target (rows of a TABLE)
+\item <INSTANCE> - defines a particular instance of a structured object (TimeFrame, SimpleTimeSeries, etc) identified by the vodml-id of the object type (coords:domain.time.TimeFrame, ts:SimpleTimeSeries)
+\end{itemize}
+
+This syntax provides severval benefits over the earlier utype convention, or any utype based annotation syntax, such as:
+\begin{itemize}
+\item Isolating the annotation from the VOTable elements allows data providers to retain their native serialization.
+  \begin{itemize}
+  \item is an additive change to current serialization thread
+  \item allows native code to continue functioning
+  \end{itemize}
+\item Protects against model changes
+  \begin{itemize}
+  \item if a model changes, say elements move from one group to another (which has happened), data providers do not need to change the VOTable GROUP content, just update the annotation section.
+  \end{itemize}
+\item Allows reuse of elements
+  \begin{itemize}
+  \item a single PARAM value=TOPOCENTER can be used for all Frame.refPosition roles.
+  \item a single FIELD name='time', can provide the time measure for multiple SimpleTimeSeries instances.
+  \end{itemize}
+\item Clear association of an instance to a role
+  \begin{itemize}
+  \item The syntax directly associates an instance to the role it plays in the parent object; ds:Target.position is instance of meas:Position
+  \item The same INSTANCE may serve multiple roles. eg: a single Organization instance may serve as both ds:DataID.curator and ds:Curation.publisher
+  \end{itemize}
+\item Allows annotation to multiple models, or versions thereof.
+  \begin{itemize}
+  \item Providers can annotate the same VOTable content as a SparseCube AND SimpleTimeSeries (for example).
+  \item Providers can annotate the same VOTable content to multiple versions of the same model, allowing for smoother transitions by clients.
+  \end{itemize}
+\item Allows clients to 'discover' content it understands.
+  \begin{itemize}
+  \item since any modeled instance ALWAYS has the same vodml-id, client applications can find and work with content that it understands.  The common example is a generic plotting package can find instances containing the roles 'meas:CoordMeasure.coord' and 'meas:CoordMeasure.error' to plot these data without knowing that they reside in a SparseCube, TimeSeries, or other data product.
+  \end{itemize}
+\end{itemize}
+These limitations of the utype annotation approach were the primary drivers for the VO-DML Mapping project.