Time Series Overview
A time series is a set or series of values of one data field that are associated with discrete timestamps. Such values could be readings from a sensor, or measurements taken by a human. Or, they could be high or low frequency and associated with a moment in time or a time window (intervals).
In the C3 Agentic AI Platform, you can:
- Model time series data.
- Normalize time series data.
- Evaluate time series data.
Some examples of time series data include the average lumens emitted by a lightbulb over the span of a year, the GDP of a country over the last decade, or the documented occurrence of earthquakes over the last 50 years.
Time series challenges
Raw time series data can occur in a variety of forms since each data source can have a different way of recording data. These include:
- Different units of measurement.
- Different time zones (UTC, local) used by different sensors.
- Different data frequencies.
- Duplicates and overlaps.
- Arrival that is out of order.
- Unreliable readings (outliers, data gaps).
- Non-calendar-normalized data.
Before you can analyze data or use it for machine learning, the data must be preprocessed. You need to remove data or convert it to fit into the categories and time boundaries that you plan to use. Converting your data into a structured format enables the application developers and data scientists to write clean code. It also helps to better utilize a rich set of operators to transform programs.
For specific preprocessing of your time series, see Normalization Modes and other pages in the Time Series Normalization section of this guide.
Data stores for time series
In the C3 Agentic AI Platform, the data services layer is responsible for persisting (or storing) large volumes of data in a scalable manner, while also making data available for analytical calculations.
Data is stored in different persistence technologies based on the use case for specific data. The diagram below explains what use case each data store is suitable for:
Common data stores
The commonly used data stores for time series data are: distributed key-value data stores, relational data stores, and file-based data stores. Each data store has an associated object count that represents the number of objects they work best with. Some examples of storage on the C3 Agentic AI Platform are shown in the next figure.
File-based databases use less computer resources than the other data stores because they only involve placing data into a file. However, the tradeoff is that it takes a longer time to read, compared to high performance stores.
Relational databases are on the opposite spectrum. Due to their structure and framework, they are the easiest to traverse for data. As a consequence, they they use the most computer resources to store data.
This leaves the key-value databases in the middle. They are the fastest to read the data and highly efficient in storing high volume data. Key-value databases are often used to retrieve large blocks of data, like time series.