C3 AI Documentation Home

Use Weather Data in C3 AI Applications

Weather data is retrieved from an external weather data provider (WeatherDataService) and is stored in the Observation Type associated with a WeatherStation that covers a specific geographic region. This data can be loaded and refreshed in the C3 Type System for use in times series data analysis, data modeling, and machine learning.

The following sections detail the typical workflow for loading and configuring weather data using C3 AI Types, as well as ensuring the weather data is complete and refreshed, as needed, for use by a C3 AI application.

See also WeatherDataUpdater and WeatherConfig.

High-level weather data retrieval workflow

The following steps outline the workflow for using weather data in the C3 Agentic AI Platform and C3 Type System.

  1. For all WeatherAware Types, generate WeatherStations for them unless WeatherStation is a calculated field in the Type.

    • Weather stations have unique IDs in the format of: "[2-letter country code]-[Zipcode]" (see Step 2) or "INTL$[lat]$[lng]" (skip Step 2 and 3):
      • CA-H1A_0A1 (old)
      • CA-H1A (new, generalized; for example, zip codes)
    • When WeatherStation is a calculated field derived from other fields, make sure that objects of these fields are also created.

  2. When looking up data for WeatherStations, use the Google Geocoding API. Calling a location through the Google Geocoding API results in the country code and geographic coordinates (latitude and longitude) or zip code, depending on how the WeatherDataService Type is configured.

    • Specific geographic coordinates: The exact location can be requested and results in geographic coordinates (latitude and longitude). For example, if the location requested is "CA-H1A_0A1", the following coordinates result: 45.65243, -73.501205.
    • Generalized location by zip code: The request location data can be generalized to zip codes for the general area. For example, the following results when requesting all "CA-H1A_XXX" zip codes: CA-H1A.

  3. All returned results from the Google Geocoding API are stored in the ZipLatLong Type.

  4. After Step 2 and 3, all the lat/long values are known for all of the requested WeatherStations. Query the external weather data provider with the latitude and longitude for each WeatherStation for weather data (see WeatherDataResult).

Load weather data

Before you start loading data, make sure the WeatherStation timeZone is set to the right value. See TimeZoneUpdater. This value is used if the location is missing a time zone value. It will default to America/Los_Angeles if not specified.

Run TimeZoneUpdater.updateTZ() to update the time zone value to the time zone specific to the weather station requested (for example, New York or Paris). See TimeZoneUpdater for more information.

By default, WeatherDataService retrieves and store 761 days (25 months) of weather Observations. See also ObservationFields. If you want more or less, set the maxHistoryDays to a different integer value to specify the max historical data to be fetched. This is configurable using WeatherConfig.

If data already exists for an WeatherStation, the WeatherDataUpdater only loads data from the last data point in time and does not honor maxHistoryDays.

Ensure "timeZone" is populated on all WeatherStations. It should have been automatically retrieved using Google Geocoding API.

Use the following C3 AI Console command to identify the number of WeatherStation objects without a timeZone:

JavaScript
WeatherStation.fetchCount({filter: Filter.not().exists("timeZone")})

If the output from the above command is not zero, the WeatherStation objects should be corrected before loading the Observations data, if possible.

Most often, this is due to issues in the data load where the postal code that triggers the creation of the WeatherStation cannot be automatically mapped to a time zone. See WeatherStationCreator.

The WeatherDataUpdater MapReduce job has a batch size of 10, so divides the WeatherStations into batches of 10 and queue WeatherStations/10 maps.

See also WeatherConfig.

Start WeatherDataUpdater MapReduce job

The WeatherDataUpdater job is unique per application. Before starting the job on an application, make sure a previous run is not stuck or currently running.

If you see a job that is stuck, cancel it before starting a new one.

JavaScript
WeatherDataUpdater.cancel(WeatherDataUpdater.forId('WeatherDataUpdater').get());

See also Workflow.MapReduce and MapReduceOptions.

Debug missing weather data information

To troubleshoot missing weather data, try the following:

  • Check the last time a cron job to update weather data was invoked. Use the CronJob.forId("trigger-for-weather-data").get() API and JobHistory.fetch({filter: "job.id == 'trigger-for-weather-data'"}) APIs for more details.

  • Check for stuck weather map reduce jobs.

  • Check that all WeatherStations were created. See WeatherAware and WeatherStationCreator.

JavaScript
var filter = Filter.not().exists("weatherStation");

// Query against WeatherAware type: for example, Location
C3.Location.fetch({filter: filter});

  • Check if a valid timeZone exists on the WeatherStation with missing weather data. If not, run the following code snippet:

    JavaScript
    TimeZoneUpdater.updateTZ();

  • Check if WeatherStation "id" looks valid. The ID can assume one of the 3 values in order of precedence: <countryCode>-<postalCode>, <countryName>-<postalCode>, or INTL$<lat>$long. If the id is not one of these, it indicates the corresponding WeatherAware may have incorrect data.

  • Make sure latitude/longitude have more than single digit decimal precision. Single decimal point lat/long precision indicates a large city or district. The external weather data provider rejects lat/longs with such low precision value.

  • Make sure the latitude/longitude are legitimate. Verify the values of lat/long are not interchanged, which might cause it to map an incorrect location.

Set the weather cron job to active

See if the weather Cron job has inactive set to false. See CronJob.

JavaScript
var job = CronJob.forId("trigger-for-weather-data").get().inactive;

If it is inactive, activate the job so it runs when next scheduled.

JavaScript
var weatherCronJob = CronJob.forId('trigger-for-weather-data').get();
weatherCronJob.resume();
c3Grid(CronJob.forId("trigger-for-weather-data").get()); // check that it is now active

Check to see if it has ever run. See JobHistory.

JavaScript
c3Grid(JobHistory.fetch({limit:10, filter: Filter.eq("job", "trigger-for-weather-data"), order:"descending(actualTime)"}));

See Run Logic on a Schedule or Create Long-Running Jobs for more information.

See also

Was this page helpful?