Execution Modes¶
Choose the right adapter for your use case.
Overview¶
graph LR
A[Data] --> B{Size?}
B -->|Fits in memory| C{Real-time?}
B -->|Too large| D[Streaming]
C -->|No| E[Batch]
C -->|Yes| F[Online]
| Mode | Use Case | Memory | Features |
|---|---|---|---|
| Batch | Complete datasets | Full | All features |
| Streaming | Large files (>100K) | Chunked | Residuals, robustness |
| Online | Real-time sensors | Fixed window | Incremental updates |
Batch Adapter¶
Standard mode for complete datasets. Supports all features.
When to Use¶
- Dataset fits in memory
- Need intervals, cross-validation, or diagnostics
- Processing complete files
Example¶
Streaming Adapter¶
Process large datasets in chunks with configurable overlap.
When to Use¶
- Dataset >100,000 points
- Memory-constrained environments
- Batch processing pipelines
Parameters¶
| Parameter | Default | Description |
|---|---|---|
chunk_size |
5000 | Points per chunk |
overlap |
500 | Overlap between chunks |
merge_strategy |
Average | How to merge overlaps |
Merge Strategies¶
| Strategy | Behavior |
|---|---|
Average |
Average overlapping values |
Left |
Keep left chunk values |
Right |
Keep right chunk values |
Weighted |
Distance-weighted blend |
Example¶
use fastLowess::prelude::*;
let mut processor = Lowess::new()
.fraction(0.3)
.iterations(2)
.adapter(Streaming)
.chunk_size(5000)
.overlap(500)
.merge_strategy(Average)
.build()?;
// Process chunks (e.g., from a file reader)
for (chunk_x, chunk_y) in data_chunks {
let result = processor.process_chunk(&chunk_x, &chunk_y)?;
write_output(&result.y);
}
// IMPORTANT: Get remaining buffered data
let final_result = processor.finalize()?;
write_output(&final_result.y);
const { StreamingLowess } = require('fastlowess');
const processor = new StreamingLowess(
{ fraction: 0.3, iterations: 2 },
{ chunkSize: 5000, overlap: 500 }
);
// Process chunks
for (const {x, y} of dataChunks) {
const result = processor.processChunk(x, y);
// ...
}
const finalResult = processor.finalize();
import { StreamingLowess } from 'fastlowess-wasm';
const processor = new StreamingLowess(
{ fraction: 0.3, iterations: 2 },
{ chunkSize: 5000, overlap: 500 }
);
// Process chunks
for (const {x, y} of dataChunks) {
const result = processor.processChunk(x, y);
// ...
}
const finalResult = processor.finalize();
Always call finalize()
In Rust, always call processor.finalize() after processing all chunks to retrieve buffered overlap data.
Online Adapter¶
Incremental updates with a sliding window for real-time data.
When to Use¶
- Data arrives incrementally (sensors, streams)
- Need real-time smoothed values
- Fixed memory budget
Parameters¶
| Parameter | Default | Description |
|---|---|---|
window_capacity |
1000 | Max points in window |
min_points |
2 | Points before output starts |
update_mode |
Incremental | Update strategy |
Update Modes¶
| Mode | Behavior | Speed |
|---|---|---|
Incremental |
Update only affected fits | Faster |
Full |
Recompute entire window | More accurate |
Example¶
use fastLowess::prelude::*;
let mut processor = Lowess::new()
.fraction(0.2)
.iterations(1)
.adapter(Online)
.window_capacity(100)
.min_points(5)
.update_mode(Incremental)
.build()?;
// Process points as they arrive
for (x, y) in sensor_stream {
if let Some(output) = processor.add_point(x, y)? {
println!("Smoothed: {:.2}", output.smoothed);
}
}
const { OnlineLowess } = require('fastlowess');
const processor = new OnlineLowess(
{ fraction: 0.2, iterations: 1 },
{ windowCapacity: 100, minPoints: 5, updateMode: "incremental" }
);
// Add points
for (const [x, y] of sensorStream) {
const smoothed = processor.update(x, y);
if (smoothed !== null) {
console.log(`Smoothed: ${smoothed.toFixed(2)}`);
}
}
import { OnlineLowess } from 'fastlowess-wasm';
const processor = new OnlineLowess(
{ fraction: 0.2, iterations: 1 },
{ windowCapacity: 100, minPoints: 5, updateMode: "incremental" }
);
// Add points
for (const [x, y] of sensorStream) {
const smoothed = processor.update(x, y);
if (smoothed !== null) {
console.log(`Smoothed: ${smoothed.toFixed(2)}`);
}
}
Feature Comparison¶
| Feature | Batch | Streaming | Online |
|---|---|---|---|
| Confidence intervals | ✓ | ✗ | ✗ |
| Prediction intervals | ✓ | ✗ | ✗ |
| Cross-validation | ✓ | ✗ | ✗ |
| Diagnostics | ✓ | ✓ | ✗ |
| Residuals | ✓ | ✓ | ✓ |
| Robustness weights | ✓ | ✓ | ✓ |
| Parallel execution | ✓ | ✓ | ✗ |
| GPU acceleration | ✓ | ✗ | ✗ |
Next Steps¶
- Parameters — All configuration options
- Tutorials — Real-time processing guide