C3 AI Documentation Home

Data Lakehouse Quickstart

This quickstart takes you from an empty Data Lakehouse to a table you can query, entirely in the browser. You upload a CSV in the Create new table wizard, run a SQL query in the SQL Editor, and view the run on the Spark Executions page. If you prefer code, the last section runs the same flow from a notebook.

The scenario uses wind turbine measurements (active power, rotor speed, gear oil temperature) from a fictional utility, Atlas Power. The same pattern works for any tabular dataset.

Before you start

Complete the steps in Data Lakehouse Prerequisites. You also need a CSV file with wind turbine measurements; the header row must include the columns t, activePower, rotorSpeed, gearOilTemp, and asset.

Step 1: Open the Data Lakehouse page

You create and query tables on the Data Lakehouse page of your application:

  1. Open your application in C3 AI Studio.
  2. Select Data Lakehouse in the Data section of the side navigation.

The page opens on the Tables tab.

Tables tab

Step 2: Create a table from your CSV

The Create new table wizard reads your CSV and shows the schema it infers before it creates the table:

  1. On the Tables tab, select Owned by me.
  2. Select Create new table.
  3. Select Browse, and choose your CSV file.
  4. Verify the parsed information:
    • Files: The file you uploaded is listed.
    • Schema: Column names and inferred data types match the CSV header.
    • Target preview: The first rows look correct.
    • File settings: If the preview looks wrong, change the delimiter.
  5. Select Next.
  6. Enter turbineMeasurements as the Table name, and a short Description. Both fields are required.
  7. In Save to Catalog, select your default catalog (datasets in most applications).
  8. Select Upload.

The wizard creates an Iceberg table in the catalog you chose and loads the CSV in one step.

Step 3: Inspect the table

Check the new table's schema and first rows in the SQL Editor before you query it:

  1. Select the SQL Editor tab.
  2. Set Catalog to your default catalog (datasets in most applications) and Namespace to the namespace that holds the new table (dfl by default).
  3. In the View Reference Table panel, select turbineMeasurements from the Select a table list.

The panel shows the table's schema, the current snapshot, the partition schema, and a 20-row data preview.

Table details modal

Step 4: Run a SQL query

In the SQL Editor query area, enter the following query and select Run SQL.

SQL
SELECT asset, AVG(activePower) AS avgPower
FROM turbineMeasurements
GROUP BY asset
ORDER BY avgPower DESC
LIMIT 10

Add a WHERE clause on t to restrict the aggregate to a specific time range (for example, WHERE t >= CURRENT_DATE - INTERVAL '1 YEAR'). Match the date in that clause to the date range in your CSV.

If the cluster is hibernated, a message appears: The Spark cluster is starting up. The query will run once the cluster is available. The query queues until the cluster is Running. Cold start can take several minutes; subsequent queries against a warm cluster return in seconds.

The results pane shows the per-asset averages. Use the results toolbar to download the result as CSV, save it as a new Lakehouse table, or append it to an existing table.

SQL Editor

Step 5: Track the run on the Spark Executions page

The Spark Executions page lists Spark runs, including your query:

  1. In the side navigation, expand Monitoring.
  2. Under Jobs, select Spark Executions.

Your query appears as a SparkExecution row with Origin = SQL Editor. The SparkExecutionOrigin enum defines two values: SQL Editor and Source to DataLake. Select the execution ID to open its details, with the cluster, catalog, namespace, query text, and error message (if any).

Spark Executions detail

What you built

You now have:

  • An Iceberg table in your default catalog with the data from your CSV file.
  • A SQL query that runs against the table and produces an aggregate.
  • A record of the query on the Spark Executions page. The same page shows the Spark cluster summary, where Open Spark UI opens the cluster's Spark UI.

Alternative: run the same flow from a notebook

If you prefer code, the same steps run from a Jupyter notebook or the Studio Console. The notebook path also lets you set primary keys, custom partitions, and sort orders that the Create new table wizard does not expose.

Python
# 1. Start the cluster and get the default catalog
# This example assumes a Spark cluster named dedicated_spark exists. Replace the name with your cluster.
cluster = c3.SparkCluster.forName("dedicated_spark")
cluster.ensureService(waitForReady=True)
ss = cluster.dataSparkSession()
catalog = c3.DataLake.Catalog.inst()

# 2. Create the table with a primary key and partitions
schema = c3.ValueType.fromString(
    "{t: datetime, activePower: double, rotorSpeed: double, "
    "gearOilTemp: double, asset: string}"
)
spec = (
    c3.DataLake.CreateTableSpec.builder()
    .name("turbineMeasurements")
    .schema(schema)
    .partitionExpressions(["asset", "month(t)"])
    .primaryKeys(["asset", "t"])
    .build()
)
table = catalog.createTable(spec)

# 3. Load from CSV
table.importCsv(
    "gcs://atlas-power-sample/turbineMeasurements.csv",
    "append",
    c3.CsvParsingSpec.builder().header(True).build(),
)

importCsv accepts a cluster-accessible URL (gcs://, s3://, or azure://). importCsv does not accept a local file path on your workstation. The Create new table wizard (Step 2) hides this limit by uploading your browser file to the cluster before calling importCsv.

The dataSparkSession() call returns a Data.SparkSession, the C3 Agentic AI Platform's Pandas-on-Spark session with C3 helpers. The notebook write appears on the same Spark Executions page as a SparkExecution row.

Next steps

See also

Was this page helpful?