[Volute] r4025 - in trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0: . VOTables VOTables/Version 1.0 VOTables/Version 1.1 ivoatex ivoatex/tth_C pics pics/XML

Volute commit messages volutecommits at g-vo.org
Mon May 8 13:29:01 CEST 2017


Author: jiri
Date: Mon May  8 13:29:00 2017
New Revision: 4025

Log:
Updated examples and model parts according to ASTERICS DADI Technology Forum 3 in Strasbourg and following discussion. The VOTable 1.1 folder examples are in the state as they were written on the DADI forum, examples in Note already bring them further to a valid VO-DML serialization. After Shanghai IVOA interop we plan to provide valid VO-DML serialized VOTables for all of the examples with changes decided there possibly implemented.

Added:
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_a_2_1.png   (contents, props changed)
Modified:
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/   (props changed)
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/VOTables/   (props changed)
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/VOTables/Version 1.0/   (props changed)
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/VOTables/Version 1.1/   (props changed)
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/cubeDM.tex
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/ivoatex/   (props changed)
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/ivoatex/tth_C/   (props changed)
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/ivoatexmeta.tex
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/   (props changed)
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/   (props changed)
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_a_1.png
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_a_2.png
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_b_1.png
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_c_1.png
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/ligo_1.png
   trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/lsst_1.png

Modified: trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/cubeDM.tex
==============================================================================
--- trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/cubeDM.tex	Sun May  7 15:46:23 2017	(r4024)
+++ trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/cubeDM.tex	Mon May  8 13:29:00 2017	(r4025)
@@ -123,7 +123,7 @@
 \end{figure}
 
 \subsection{Imported Data models}
-We import these data models without modification. We basically copy the model definition into \textit{Time Series Cube DM}, making us completely dependent on what is defined within these - \textit{Time Series Cube DM} woudn't work without them.
+We import these data models without modification. We basically copy the model definition into \textit{Time Series Cube DM}, making us completely dependent on what is defined within these - \textit{Time Series Cube DM} wouldn't work without them.
 
 However, because we use them as black boxes, it does not matter if these data models change - \textit{Time Series Cube DM} has no common parts shared with them.
 
@@ -170,11 +170,13 @@
 
 Note that the purpose of the \textit{Time Series Cube DM} metadata is to enlist all the axes of the cube and provide linkage towards their values and errors, not to completely describe axis domains. 
 
+Also, after intensive discussions, we decided to move statistical metadata about the axis of our data cubes to be moved into a really small external model \textit{Quantity DM} because this will be useful not only for cubes but any numerical data in general.
+
 The axis domain descriptions will be held within other specialized data models, e.g., spatial and time domain in STC (\cite{std:STC}), spectral domain in Photometry DM (\cite{std:PHOTDM}), etc. These external models will only be referenced by the \textit{Time Series Cube DM}. This is where we differ from the original \textit{Cube DM} that already focuses on supporting SIAv2 (\cite{std:SIAv2}) image-based protocol and we do not want to restrict ourselves only to pixelated types of data cubes. 
 
 Withdrawing axis domain definitions as a dependency from the \textit{Time Series Cube DM} helps us in the future when new data models will be created, e.g., for probability distributions or gravitational wave domains. If such a new data model appears, we can just add it as a reference to axis metadata within the data cube and the \textit{Time Series Cube DM} does not need to be changed at all. It also solves the main drawback of previous approaches and that is extensibility - SSAP or SimpleTimeSeries standards would need to be changed if a new domain was to be supported. Moreover, such a domain would not be logically part of spectral or time aspects of those models and therefore it does not belong there.
 
-The only difference between axis coordinate and the value for that coordinate, from the data point of view, is that the axis coordinates are independent while the data values are dependent on these coordinates.
+The only difference between axis coordinate and the value for that coordinate is that the axis coordinates are independent while the data values are fully functionally dependent on these coordinates.
 
 Therefore, we can afford (from the data point of view) to treat them equally, which is one of the biggest advantages of a data cube. We can browse the data cube by filtering the axis coordinates and the data values in the same way. This enables a client for example to slice the time axis of a light curve (coordinates) within an interval and the magnitude axis (values) within another interval using the same algorithm.
 
@@ -204,7 +206,7 @@
 The \textit{SparseCube Dataset} class extends \textit{ObsDataset} class and puts restriction on what can be stored within this collection. An instance of the \textit{SparseCube Dataset} class has an attribute \texttt{sparseCubeData} which holds the collection of \textit{SparseCube} class instances, i.e. \texttt{Time Series Cube} instances.
 
 \subsection {Cube DM}
-In this section we speak about relevant classes from the \textit{Cube DM} package, i.e., \texttt{Sparse Cube} class, \texttt{Time Series Cube} class and \texttt{CubeAxis} class as described on Fig. \ref{fig:model}. 
+In this section we speak about relevant classes from the \textit{Cube DM} package, i.e., \texttt{Sparse Cube} class and \texttt{Time Series Cube} class as described on Fig. \ref{fig:model}. 
 
 
 \subsubsection{Sparse Cube class}
@@ -212,20 +214,31 @@
 
 
 \subsubsection{Time Series Cube class}
-A client can identify the \texttt{Time Series Cube} instance by having the keyword \texttt{timeseries} in the \texttt{dataProductType} attribute of its instance. By defining \textit{CubeAxis} class and referencing the axis domain models from it, we override the \textit{Cube DM} dependency on axis domains, e.g., \textit{STC v2.0 DM}.
+A client can identify the \texttt{Time Series Cube} instance by having the keyword \texttt{timeseries} in the \texttt{dataProductType} attribute of its instance. By using only \textit{ColumnRef} class which is already referenced from \textit{Axis Domain models}, we override the \textit{Cube DM} dependency on axis domains, e.g., \textit{STC v2.0 DM}.
+
+A client reading the data cube might want to know what should it treat as independent (coordinates) and dependent (data) values - they are collected within the \texttt{independent\_axes} and in \texttt{dependent\_axes} attributes of the \texttt{Time Series Cube} instance.  
 
-A client reading the data cube might want to know what should it treat as independent (coordinates) and dependent (data) values - they are collected within the \texttt{independent\_axes} and in \texttt{dependent\_axes} attributes of the \texttt{Time Series Cube} instance. 
+\subsubsection{ColumnRef class}
+ \texttt{independent\_axes} and in \texttt{dependent\_axes} inside \texttt{Time Series Cube class} use the \texttt{ColumnRef} which is a primitive element of the serialization used. In VOTable this would be \texttt{FieldRef} element, in FITS table just the \texttt{name} of the column.
 
-\subsubsection{CubeAxis class}
-The \texttt{CubeAxis} class annotates the most important information about the axis. Where do I find the data (\texttt{field} attribute) and its simple statistical error (\texttt{error} attribute). If a specialized client wants to find more about specific metadata and data of an axis (its frame, domain, all related parameters and fields), it can go to the axis domain model through the \texttt{model} parameter of that axis and find the information in there.
+If a specialized client wants to find more about specific metadata and data of an axis (its frame, domain, all related parameters and fields), it will find this information where he is used to it - in metadata of the dataset and in the parameters of the column itself. In VOTable this would be special \texttt{Param} elements at the beginning of the table and attributes of \texttt{Field} element, e.g., \texttt{name}, \texttt{type}, \texttt{unit}, \texttt{ucd}, etc.
 
 And because this client is specialized towards that domain, it understands metadata in the referenced domain-specific data model (e.g., gravitational wave domain, spectral domain, etc.) and we don't need to make it part of \textit{Time Series Cube DM}.
 
+\subsubsection{Quantity class}
+This class was originally part of the \textit{Cube DM} but we moved it out because statistical information is useful for images and spectra, basically any kind of numerical data. We are still mentioning it here though because we believe that it will have bigger impact on \textit{Cube DM} than other types of data. 
+
+Its main role is to describe statistical distribution of data within the axes of our data cube allowing us to make more informed decisions about what subset of the data we would like to filter. Knowing mean and sigma values or even quantiles of the data and filtering it based on these (e.g. "I want to filter only points that are 3-5 sigma from the mean value.") will provide much better results than trying to guess the ranges based only on minimum and maximum value of that axis.
+
+The Quantity class is referencing the \textit{ColumnRef} class as it must say which column (also error column) it is describing in the dataset. Making it dependent only on this primitive type is again in accordance with the loose coupling principle we are using - making the higher level models independent of each other.
+
 \subsection {Axis Domain DMs}
 An answer to what all can be described with an axis (the axis domain) is not within the scope of this document. If we want to describe our spectral coordinates (e.g., list of filters) by the \textit{Photometry DM} or in \textit{STC DM} or add a completely new model for my axis domain (e.g., Gravitational Wave data), it does not affect the \textit{Time Series Cube DM}. 
 
+These models, however, do always reference a primitive type of their serialization, such as \texttt{ColumnRef}. By using this primitive type dependency (loose coupling for all of our data models), we are essentially making \textit{Cube DM}, \textit{Quantity DM} and \textit{Axis Domain DMs} independent of each other.
+
 \subsection{Time Series Cube DM summary}
-With the growing amount of models and their diversity within the IVOA, keeping models independent of each other is the only viable approach to their evolution and maintenance. By only referencing a simple abstraction of an external model instead of importing every detail of that model, the \textit{Time Series Cube DM} does not need to change every time an \textit{Axis Domain Model} changes or a new one is created - the abstraction of that domain will be still the same.
+With the growing amount of models and their diversity within the IVOA, keeping models independent of each other is the only viable approach to their evolution and maintenance. By only referencing a primitive type instead of an external model or even importing every detail of that model, the \textit{Time Series Cube DM} does not need to change every time an \textit{Axis Domain Model} changes or a new one is created - the abstraction of that domain will be still the same. This information is valid also for the \textit{Quantity DM}.
 
 %More complex data such as Bayesian errors or probability distributions can be seen on Fig.\ref{fig:LSST_probability}.
 
@@ -236,7 +249,7 @@
 At current point of time, the ObsCore suffices our requirements for discovery and anyone can add new parameters to his ObsCore service - except the \textit{DataLink} (\cite{std:DataLink}) metadata discovery. This point is under discussion and, in the end, it may will not demand change to the ObsCore TAP standard.
 
 \section{Serialization}
-We decided to discuss the VOTable format as the first one to support as it is the easiest one to form discussions upon and afterwards close them with a deterministic XSD that can validate whether the serialization is strictly following the standard or not. This section focuses on serialization of the metadata and data defined within the \textit{Time Series Cube DM}, not on serialization of the data model strucutre - that will be handled by VO-DML.
+We decided to discuss the VOTable format as the first one to support as it is the easiest one to form discussions upon and afterwards close them with a deterministic XSD that can validate whether the serialization is strictly following the standard or not. This section focuses on serialization of the metadata and data defined within the \textit{Time Series Cube DM}, not on serialization of the data model structure - that will be handled by VO-DML.
 
 \texttt{GROUP} XML element can be used as an equivalent of the data model classes described in Chap. \ref{chap:model}. These are linked to the IVOA data models by annotations. The \textit{Time Series Cube DM} itself then contains \texttt{independent\_axes} and \texttt{dependent\_axes} \texttt{GROUP} elements which hold the actual information about the axis. 
 
@@ -248,6 +261,8 @@
 
 The basic example of how such a serialization would look like can be seen on Fig. \ref{fig:group_a_1} and more can be found in the Chap. \ref{chap:cases}.
 
+Please note that we enhanced and completed the serialization examples for version 1.1 and updated them in \ref{chap:cases}.
+
 
 \section{Supported science cases}
 \label{chap:cases}
@@ -276,25 +291,25 @@
   \begin{itemize}
   \item{Provenance tracking}:  These use cases require to track original image or its cutouts for every point of the light curve, e.g., link to SIAP cutouts of the relevant region in the original image which can be opened in a graphical client. We are able to transfer the data and describe the metadata for this use case.
   \item {Probability distribution}: Another use case computes and transfers probability distribution functions for the time series, e.g., computed simple Gaussian distribution parameters for the coordinate errors from which the light curve was generated). We are able to transfer the data but for metadata description, there would have to be created a new kind of Probabilistic DM as seen on Fig. \ref{fig:model}. However, when such model comes, we just add the \texttt{model} reference to the \textit{CubeAxis} metadata, without any changes to the \textit{Time Series Cube DM} serialization.
-  \item {\textit{DataLink} for time series}: A basic use case for \textit{Time Series Cube DM} combined with \textit{DataLink} would be storing data linkage for the whole time series instance, e.g., \textit{DataLink} to the periodogram of this time series. Another one would be filtering of light curve points on a supported criteria, e.g., filtering only high quality points of a light curve. Here we are struggling with the metadata transfer for the ObsCore TAP discovery described in Chap. \ref{chap:discovery}. The problem is that with \textit{ObsCore DM} we can get also other data than time series and the \textit{DataLink} does not support metadata restriction based on a \texttt{dataProductType}. We are able to construct such a \textit{DataLink} for the light curves, the only problem is with transfering the metadata about such a domain over ObsCore TAP service.
+  \item {\textit{DataLink} for time series}: A basic use case for \textit{Time Series Cube DM} combined with \textit{DataLink} would be storing data linkage for the whole time series instance, e.g., \textit{DataLink} to the periodogram of this time series. Another one would be filtering of light curve points on a supported criteria, e.g., filtering only high quality points of a light curve. Here we are struggling with the metadata transfer for the ObsCore TAP discovery described in Chap. \ref{chap:discovery}. The problem is that with \textit{ObsCore DM} we can get also other data than time series and the \textit{DataLink} does not support metadata restriction based on a \texttt{dataProductType}. We are able to construct such a \textit{DataLink} for the light curves, the only problem is with transferring the metadata about such a domain over ObsCore TAP service.
  
 \end{itemize}  
  \item {Power spectra, Gravitational waves}: Special use case for generic time series data - we show here how to store not only simple time axis but also its derivatives, e.g., frequency. For Gravitational waves, this would be tracking for example ASD (strain per rtHz) against Frequency (Hz) as seen on the Gravitational wave tutorial (\cite{GravWave}).
 \end{itemize}
 
 \subsection{Simple Light curves}
-This is the basic use case supported up to now with \textit{SimpleTimeSeries} (\cite{std:SimpleTimeSeries}). Now, we can easily support it in the \textit{Time Series Cube DM}. An example of the \textit{DataSet} metadata can be seen on Fig. \ref{fig:group_a_1}, the definition of the cube on Fig. \ref{fig:group_a_2} and the first row on Fig. \ref{fig:group_a_3}. 
+This is the basic use case supported up to now with \textit{SimpleTimeSeries} (\cite{std:SimpleTimeSeries}). Now, we can easily support it in the \textit{Time Series Cube DM}. An example of the \textit{DataSet} metadata can be seen on Fig. \ref{fig:group_a_1}, the definition of the cube on Fig. \ref{fig:group_a_2} and \ref{fig:group_a_2_1} and  the first row on Fig. \ref{fig:group_a_3}. 
 
 The light curve rendered within SPLAT-VO client ((\cite{paper:splat})) can be seen on Fig. \ref{fig:splat_ssa1} and \ref{fig:splat_ssa2}. These images were taken on real data coming within our reference implementation for server and client. More about the reference implementation can be found in Chap. \ref{chap:implementation}.
 
 
 \subsubsection{Examples}
-The examples of simple light curves can be seen on Fig. \ref{fig:group_a_1}, Fig. \ref{fig:group_a_2} and Fig. \ref{fig:group_a_3}. The plot rendered within SPLAT-VO can be seen on Fig. \ref{fig:splat_ssa1}.
+The examples of simple light curves can be seen on Fig. \ref{fig:group_a_1}, Fig. \ref{fig:group_a_2}, Fig. \ref{fig:group_a_2_1} and Fig. \ref{fig:group_a_3}. The plot rendered within SPLAT-VO can be seen on Fig. \ref{fig:splat_ssa1}.
 
 \begin{figure}[h!]
 \includegraphics[width=1\textwidth]{pics/XML/group_a_1.png}
 \centering
-\caption{Dataset metadata serialization. Here we can see dataset \texttt{GROUP} element. There is a dataset (collection) of sparse cubes. The dataset contains the metadata, e.g., the \texttt{DataID}. All of the relevant elements (\texttt{GROUP}, \texttt{PARAM}, \texttt{FIELD}, etc.) are annotated against their respective \textit{Time Series Cube DM} counterparts using the \texttt{vodml} attribute.}
+\caption{Dataset metadata serialization via VO-DML. There is a dataset (collection) of sparse cubes. The dataset contains the metadata, e.g., the \texttt{observationID} or \texttt{creator} metadata. All of the serialized metadata are annotated against their respective counterparts in \textit{Dataset DM} and \textit{Time Series Cube DM} using the \texttt{dmRole} attribute.}
 \label{fig:group_a_1}
 \end{figure}
 
@@ -302,14 +317,21 @@
 \begin{figure}[h!]
 \includegraphics[width=1\textwidth]{pics/XML/group_a_2.png}
 \centering
-\caption{Time series data cube metadata serialization. Here we can see the metadata \texttt{GROUP} element for our time series data cube. The independent axes like temporal and spatial are referencing the \texttt{FIELD} elements holding their data and annotating against their axis models holding their metadata, using the \texttt{GROUPRef} elements, e.g., \texttt{datestc} or \texttt{spatialFrame}. The same goes for dependent axes.}
+\caption{Time series data cube metadata serialization. Here we can see the \texttt{ndcube:Cube} \texttt{dmtype} along with the axis definitions. The independent axes like temporal or spatial are referencing the \texttt{FIELD} elements holding their data. It works the same way for both independent and dependent axes.}
 \label{fig:group_a_2}
 \end{figure}
 
 \begin{figure}[h!]
+\includegraphics[width=1\textwidth]{pics/XML/group_a_2_1.png}
+\centering
+\caption{Statistical information about the data cube. In here we can see the \texttt{q:Quantity} \texttt{dmtype}. The statistical information is held within the \texttt{constant} elements and the reference to the column that is actually holding the data can be recognized by \texttt{dmrole="value"} attribute. Note that for describing statistical information we don't need to have the context held in axis domain metadata.}
+\label{fig:group_a_2_1}
+\end{figure}
+
+\begin{figure}[h!]
 \includegraphics[width=1\textwidth]{pics/XML/group_a_3.png}
 \centering
-\caption{Time series data cube data serialization. Here we can see the \texttt{FIELD} elements referenced from the time series cube metadata. This is a basic light curve holding the heliocentric Julian date for temporal axis, equatorial coordinates and two dependent axes with measurements - flux and magnitude.}
+\caption{Time series data cube data serialization. Here we can see the \texttt{FIELD} elements referenced from the time series cube and quantity metadata. This is a basic light curve holding the heliocentric Julian date for temporal axis, equatorial coordinates and two dependent axes with measurements - flux and magnitude.}
 \label{fig:group_a_3}
 \end{figure}
 
@@ -323,11 +345,11 @@
 
 \subsection{Groups of light curves}
 
-The use case of storing light curves for one object in multiple photometric bands can be supported simply by addition of one independent axis - the spectral axis. Having the light curve points in multiple bands, e.g., UBVRI, is nothing more than having one more discrete spectral axis with coordinates that can have 5 different values. If the spectral axis is described in continuous intervals, the error \texttt{FIELD} would be meaningful, for just a list of filter names we are storing only data value elements, i.e. \texttt{field} attribute of  \texttt{Cube Axis}, with textual information.
+The use case of storing light curves for one object in multiple photometric bands can be supported simply by addition of one independent axis - the spectral axis. Having the light curve points in multiple bands, e.g., UBVRI, is nothing more than having one more discrete spectral axis with coordinates that can have 5 distinct values. If the spectral axis is described in continuous intervals, the error \texttt{FIELD} would be meaningful, for just a list of filter names we are storing only data value elements, i.e. \texttt{value} attribute of  \texttt{Quantity}, with textual information.
 
 It is evident that adding more such "groups" (other independent axes) will produce a lot of duplicated values in the \texttt{DATA} element of our VOTable. However, this is just a question of optimization.
 
-If the need arises, for start, compressing algorithms such as EXI (\cite{std:EXI}) can be used for immediate value of eliminating the duplicities. A cleaner approach would be a table normalization during the transfer - the independent axes (coordinates) would be stored in separate \texttt{TABLE} elements (the same goes for FITS) that would be referenced by the dependent axes records. 
+If the need arises, for start, compressing algorithms such as EXI (\cite{std:EXI}) can be used for immediate value of eliminating the duplicities. A cleaner approach would be a table normalization during the transfer - the independent axes (coordinates) would be stored in separate \texttt{TABLE} elements (the same goes for FITS) that would be referenced by the dependent axes records. This, however, would need an extension to VOTable standard to allow referencing not only between elements but also within data. The requirement is that a value in one data column must be able to reference another value within another table.
 
 The optimization of serialization is not the main aim of this document but can be expanded in the future if deemed necessary.
 
@@ -337,14 +359,14 @@
 \begin{figure}[h!]
 \includegraphics[width=1\textwidth]{pics/XML/group_b_1.png}
 \centering
-\caption{Groups of light curves time series data cube metadata serialization. Here we have two independent axes, the temporal and spectral. This means that the dependent axis (actual measurements, magnitude in this case) can hold values for any combination of the date and spectral axis values. The \textit{Photometry DM} as an axis domain definition is referenced from the \texttt{model} attribute of the \texttt{spectralAxis GROUP} element.}
+\caption{Groups of light curves time series data cube metadata serialization. The \textit{Dataset} and \textit{Quantity} information are the same, the only change we need to do is to add one more axis to the independent collection. This means that the dependent axis (actual measurements, i.e., magnitude, flux) can hold values for any combination of the date and spectral axis values. }
 \label{fig:group_b_1}
 \end{figure}
 
 \begin{figure}[h!]
 \includegraphics[width=1\textwidth]{pics/splat_ssa2.png}
 \centering
-\caption{Group of light curves within SPLAT-VO. As we are still using a valid VOTable serialization, our SPLAT-VO client can render the time series data cube for a light curve taken in 4 filters without modification. The spectral axis is the 3rd rendered axis and is expressed in colors instead of axis coordinates. The axis labels are taken from axis domain metadata referenced by the \textit{Time Series Cube DM}}
+\caption{Group of light curves within SPLAT-VO. As we are still using a valid VOTable serialization, our SPLAT-VO client can render the time series data cube for a light curve taken in 4 filters without modification. The spectral axis is the 3rd rendered axis and is expressed in colors instead of axis coordinates. The axis labels are taken from axis domain metadata already modeled within Photometry DM}
 \label{fig:splat_ssa2}
 \end{figure}
 
@@ -352,18 +374,20 @@
  \label{chap:group_c}
 The Generic time series are the main motivation for using \textit{Time Series Cube DM}, as any kind of data that can be structured into series can be represented by a data cube. In here we also show that we can describe our metadata by using \texttt{PARAM} elements which our client can understand by parsing \texttt{UCDs}. Or, we can describe those by using external data models that the \textit{Time Series Cube DM} only references. 
 
-The fact that the \textit{Time Series Cube DM} will not know about the details of every domain metadata is crucial as it enables us to describe the metadata with our own parameters if a standard model is not yet available, without the need of including them in \textit{Time Series Cube DM}. After the model is standardized, e.g., \textit{STC v2.0, Provenance DM, Probability DM}, etc., we just group the needed metadata under a \texttt{GROUP} element and reference it in our \texttt{Cube axis} description, without the need to change the structure of the data model or even the serialization within the rest of the \textit{VOTable}.
+The fact that the \textit{Time Series Cube DM} will not know about the details of every domain metadata is crucial as it enables us to describe the metadata with our own parameters if a standard model is not yet available, without the need of including them in \textit{Time Series Cube DM}. After the model is standardized, e.g., \textit{STC v2.0, Provenance DM, Probability DM}, etc., we just need to add new stuff defined within that model without the need to change anything within the serialization defined by \textit{Quantity DM} or \textit{Cube DM}.
 
 The axes derived from the basic types can be still tracked as the same type. The frequency for a power spectrum is still a time series in its own sense and we can easily support it within \textit{Time Series Cube DM}. It is true that the the frequency is not technically a sparse axis that we chose as a basis for the time series data cubes, but logically a power spectrum or periodogram can be understood as time series. Moreover, the \textit{Sparse Cube DM} places a restriction on \textit{Time Series Cube DM} to support sparse axes, not to restrict itself only to them.
 
+Another option for supporting these cases would be simply to move our generic definitions from \textit{Time Series Cube DM} to \textit{Cube DM} and not worry about whether a periodogram is a time series when we already can agree that i definitely can be described as a data cube.
+
 
 \subsubsection{Examples}
-A basic example of a generic time series can be seen on Fig. \ref{fig:group_c_1}. Focus on the highlighted model attribute of the hardness ratio dependent axis. Even if the axis domain model does not exist yet, we still want to describe the structure of the data (values and errors of this axis) and store the rest in \texttt{PARAM} elements, which are not part of the data-perspective view provided by \textit{Time Series Cube DM}. If the axis domain model for describing the hardness ratio existed, we would reference it by the \texttt{model} attribute. This reference tells the client which \texttt{PARAM} elements are relevant for the domain of this axis.
+A basic example of a generic time series can be seen on Fig. \ref{fig:group_c_1}. Focus on the highlighted model attribute of the hardness ratio dependent axis. Even if the axis domain model does not exist yet, we still want to describe the structure of the data (values and errors of this axis) and store the rest in \texttt{PARAM} elements, which are not part of the data-perspective view provided by \textit{Time Series Cube DM}. When the axis domain model for hardness ration arrives, we just add new elements from it to the VOTable without the need to change any existing, maintaining full backwards-compatibility.
 
 \begin{figure}[h!]
 \includegraphics[width=1\textwidth]{pics/XML/group_c_1.png}
 \centering
-\caption{Hardness ratio time series example. We omit here the actual \texttt{FIELD} elements for simplicity. These would contain most importantly \texttt{UCDs} for the fields and other metadata.}
+\caption{Hardness ratio time series example. The data cube enlists the axes. The errors for these axes can be found in the \textit{Quantity} metadata (along with the statistics, which we omit here for simplicity) and the column definitions which are storing the actual data. When a model for hardness ratio arrives, it would certainly specify ucd for the hardnessRatio \texttt{FIELD} element.}
 \label{fig:group_c_1}
 \end{figure}
 
@@ -381,7 +405,7 @@
 \begin{figure}[h!]
 \includegraphics[width=1\textwidth]{pics/XML/lsst_1.png}
 \centering
-\caption{Linkage towards original image cutout for one point of the light curve. The data can be seen in the last column of the table with cutout parameters taken from the right ascension and declination of this point of the curve. }
+\caption{Linkage towards original image cutout for one point of the light curve. The data can be seen in the last column of the table with cutout parameters taken from the right ascension and declination of this point of the curve. Note that while the cube could mention the \texttt{SIAPLiink} column as one of its axes, any statistical distribution or plotting of its values would not be meaningful. That is the reason why it is not part of the cube and is described entirely by the \texttt{FIELD} element that might be forged by the \textit{Provenance DM} in the future. }
 \label{fig:lsst_1}
 \end{figure}
 
@@ -417,10 +441,15 @@
 
 
 \subsection {Gravitational waves}
+\label{chap:grav_wave}
 This example is quite similar to the one described in Chap. \ref{chap:group_c}. The only difference is a temporal axis describing a frequency (as time derivative) and then another dependent axis that contains measured data for gravitational wave analysis. This use case is taken from the LIGO tutorial site (\cite{GravWave}).
 
 In our specific uses case, we describe a time series data cube storing gravitational wave analysis, as can be seen on Fig. \ref{fig:ligo_splat_1}. We have a temporal axis of \texttt{frequency} rendered as \texttt{x axis} and the \texttt{strain/rtHz} dependent axis rendered as \texttt{y axis}. The second independent axis holding sampling rate frequency is rendered as 3rd dimension in colors.
 
+Storing the sampling rate frequency as an independent axis would make sense if it would be a configuration of the instrument, theoretically providing different values at the same point of time when measuring on different frequencies. Otherwise its just another dependent axis (and a very primitive one, just a multiple of the values for different sampling frequencies).
+
+We assume that we have the more complicated option here that different sampling frequencies can provide different measurements.
+
 
 \subsubsection{Examples}
 The sample serialization and one row can be seen on Fig. \ref{fig:ligo_1}.
@@ -428,7 +457,7 @@
 \begin{figure}[h!]
 \includegraphics[width=1\textwidth]{pics/XML/ligo_1.png}
 \centering
-\caption{VOTable example for gravitational Wave data. We omit \texttt{FIELD} elements for simplicity, the important thing is that all Gravitational wave specific axes are referencing their domain metadata in external models, so \textit{Time Series Cube DM} does not need to know about them. The sample row is showing a strain for 100 Hz frequency sampled with 4096 Hz.}
+\caption{VOTable example for gravitational Wave data. The independent axes are here the frequency and sampling frequency (explained in Chap. \ref{chap:grav_wave}. The sample row is showing a strain for 100 Hz frequency sampled with 4096 Hz.}
 \label{fig:ligo_1}
 \end{figure}
 
@@ -443,7 +472,7 @@
 \label{chap:implementation}
 We are working on GAVO DaCHS implementation for server-side service providing time series data cubes in the \textit{Time Series Cube DM} format. The service is ready and providing real data, but we have only a development environment ready right now and is under constant change. If you want to have a look on the final version, you can find it on the following web page (\cite{vos2}).
 
-We are also working on the SPLAT-VO modifications so it can understand the \textit{Time Series Cube DM} format. Since we are using standard VOTables for serialization of our data cubes, the modifications are more related to SPLAT-VO understanding the axis domain models like magnitude axis inversion or time axis unit identification. In the current version of the serialization, SPLAT-VO uderstands the data perfectly.
+We are also working on the SPLAT-VO modifications so it can understand the \textit{Time Series Cube DM} format. Since we are using standard VOTables for serialization of our data cubes, the modifications are more related to SPLAT-VO understanding the axis domain models like magnitude axis inversion or time axis unit identification. In the current version of the serialization, SPLAT-VO understands the data perfectly.
 
 
 \section{Summary}

Modified: trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/ivoatexmeta.tex
==============================================================================
--- trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/ivoatexmeta.tex	Sun May  7 15:46:23 2017	(r4024)
+++ trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/ivoatexmeta.tex	Mon May  8 13:29:00 2017	(r4025)
@@ -1,5 +1,5 @@
 % GENERATED FILE -- edit this in the Makefile
-\newcommand{\ivoaDocversion}{1.1}
+\newcommand{\ivoaDocversion}{1.0}
 \newcommand{\ivoaDocdate}{2017-02-05}
 \newcommand{\ivoaDocdatecode}{20170205}
 \newcommand{\ivoaDoctype}{NOTE}

Modified: trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_a_1.png
==============================================================================
Binary file (source and/or target). No diff available.

Modified: trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_a_2.png
==============================================================================
Binary file (source and/or target). No diff available.

Added: trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_a_2_1.png
==============================================================================
Binary file. No diff available.

Modified: trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_b_1.png
==============================================================================
Binary file (source and/or target). No diff available.

Modified: trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/group_c_1.png
==============================================================================
Binary file (source and/or target). No diff available.

Modified: trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/ligo_1.png
==============================================================================
Binary file (source and/or target). No diff available.

Modified: trunk/projects/time-domain/time-series/time-series-cube/ivoa-note-1.0/pics/XML/lsst_1.png
==============================================================================
Binary file (source and/or target). No diff available.


More information about the Volutecommits mailing list