Optimize Rebuild Performance Using Batch Sizes
Advanced users can configure batch sizes for each dependency type to optimize rebuild performance based on data volume and available system resources. Batch sizes determine how many records are processed in each operation during the rebuild.
Understanding Batch Sizes
Each dependency type supports two parameters:
Batch Size
Controls the number of records processed in a single operation.Sub-Batch Size
Used for nested or dependent operations within certain rebuild phases. This provides more granular control for complex processing steps.
When to Adjust Batch Sizes
In most cases, the default values are sufficient. Consider adjusting batch sizes in the following scenarios:
Large datasets
Reduce batch sizes if rebuild jobs consume excessive memory or experience timeouts.Small datasets
Increase batch sizes to improve throughput when sufficient system resources are available.Resource constraints
Reduce batch sizes if CPU, memory, or database connections are limited.High latency environments
Increase batch sizes to reduce the number of processing cycles.Performance tuning
Adjust batch sizes based on observations from Batch Jobs logs.
Default Batch Sizes
The system provides optimized default values for each dependency type:
| Dependency Type | Batch Size | Sub-Batch Size |
|---|---|---|
| Queue Submission (Invalidations) | 1000 | — |
| Created or Updated (Callbacks) | 1000 | 100 |
| Stored Calculations | 1000 | 1000 |
| Hierarchy Denormalization | 1000 | 1000 |
| Timeseries Normalization | 100 | 100 |
| Metric Dependencies | 100 | 100 |
| Cached Metrics | 100 | 100 |
| Analytics | 100 | 100 |
| ACLs | 1000 | 1000 |
| Partitionable Compaction | 100 | 100 |
Best Practices
Use default values first
The default batch sizes are optimized for typical workloads. Modify them only when necessary.Increase batch size for throughput
Larger batch sizes can improve performance when operations are limited by I/O or network latency.Decrease batch size for stability
Smaller batch sizes reduce memory usage and help avoid timeouts.Monitor and iterate
Review Batch Jobs logs and adjust incrementally based on observed performance.Test before applying broadly
Validate significant changes in a non-production environment before applying them at scale.
Configure Batch Sizes
To modify batch size settings:
- Open the Rebuild Application Dependencies dialog.
- Select a preset or choose Custom.
- Expand the Advanced section.
- Update Batch Size and Sub-Batch Size values as needed.
- Start the rebuild.
If no changes are made, the system uses default values.