I need your help in designing one table.
I have some groups tables and we need to load data in that group tables from xml files that contain column names and data.
The column name is actually index of some main column like activity_col1,activity_col2 and so on and not fixed every time, there is possibility that
same table file contains 1000 columns sometimes and 10 column values some time also there is maximum limit is also defined so no file will contain more than
2000 column per group.
so I need to design a table that is the best possible solution for this also I need to do the aggregation of column values. The files contain min level data and i need to store this data in min table and after that this min data need to be aggregated in an hour, day, week and month.
if I create max columns in all tables but data will not come every time in all columns so this design seems not good because most of the values will be null.
if I insert column name as rows in column_name column and values against each column value in values column then aggregation will be a tedious task for me
and it will impact performance.