Machine Learning & Deep Learning
We start with the statistics: which parameters actually drive your outcomes, and by how much. Only then do we reach for non-linear models and deep learning, where the data shows they earn their complexity.
1 · Foundation
- Data audit & lineage
- Feature engineering
2 · Regression
- Parameter sets
- Influence & significance
3 · Non-linear
- Tree ensembles
- Kernel & additive models
4 · Deep learning
- Sequences, images, graphs
- Validate & deploy
1. Statistical Foundations: Regression Done Properly
Regression is still the fastest way to understand a system. A well-built regression model tells you which inputs matter, in which direction, by how much and with what confidence. That is exactly what engineers, analysts and executives need before trusting a more opaque model.
- Linear and multivariate regression as a transparent, defensible baseline.
- Regularized regression (ridge, lasso, elastic net) to handle many correlated inputs and keep models stable.
- Generalized linear models for counts, rates and pass/fail outcomes such as yield or defect occurrence.
2. Establishing the Parameter Set
Most of the value is in choosing the right inputs. We work with your domain experts to define a parameter set that is complete enough to explain the outcome and lean enough to stay stable:
- Input and output classification: separating controllable process inputs from measured outputs (for example KPIV/KPOV in manufacturing) so the model answers the question you can act on.
- Collinearity checks (correlation analysis, variance inflation factors) so overlapping signals do not distort each other.
- Feature selection by regularization, stepwise testing and cross-validation, with every dropped variable documented.
- Designed experiments where historical data cannot separate the effects you care about.
3. Measuring Influence
Once the parameter set is fixed, we quantify how much each input moves the result, and which individual records are pulling the model:
Parameter Effects
Standardized coefficients, confidence intervals and significance tests that rank inputs by real-world impact.
Influential Records
Leverage, residual and Cook's distance diagnostics that flag the runs, lots or customers distorting the fit.
Sensitivity
Partial-dependence and what-if analysis showing how the outcome responds as each input changes.
4. Non-Linear Approaches
When residuals show curvature, thresholds or interactions that a linear model cannot capture, we move to non-linear methods and keep the regression baseline as the benchmark they must beat:
- Tree ensembles (random forest, gradient boosting) for interactions and thresholds, with SHAP values to keep each prediction explainable.
- Generalized additive models that capture smooth non-linear effects while staying interpretable input by input.
- Kernel methods such as support vector regression for smaller, high-dimensional datasets.
- Graph features from knowledge graphs (embeddings, centrality, community membership) that add relationship context no flat table contains.
5. Deep Learning
Deep learning earns its place when the signal lives in raw, high-dimensional data:
- Sequence models (1-D convolutional networks, recurrent networks, transformers) for sensor time series and process traces.
- Convolutional networks for images, inspection maps and engineering drawings.
- Language models and embeddings for unstructured text, from clinical notes to news events.
- Graph neural networks for problems where the relationships themselves are the signal.
6. Predicting Outcomes Mid-Process
In multi-step manufacturing such as semiconductor fabrication, the goal is a yield or parametric prediction after every step, not only when the wafer is finished. The challenge is that the model sees a different amount of data at each point: one step’s variables after step 1, five after step 5. We use three deep-learning architectures that handle this growing input in different ways:
Padded MLP + Masking
The baseline. Steps that have not run yet are filled with placeholder values, and a mask tells the network which steps are real. It is fast to build, easy to debug and works directly with SHAP, which makes it the floor every other model has to beat.
Causal Transformer
The production target. Each process step is a token in a sequence. Attention reads each step in light of the steps before it, and causal masking means the model never sees steps that have not happened yet. One forward pass gives a prediction after every step, and the model scales to new targets and product lines.
Hierarchical Encoder
The modular option. A small encoder for each step produces an embedding, and an aggregator (attention pooling or an RNN) combines whichever steps have completed. You can see which step drove a prediction, and adding or changing a step means retraining one encoder, not the whole model.
We build the padded MLP first to validate the data and set the baseline, then deploy the causal transformer as the long-lived production model. The hierarchical design stays available when the process changes often or when per-step explanations matter most, so the three form one engineering plan rather than three competing options.
7. Validation & Deployment
Every model ships with honest validation (hold-out and time-based splits, error analysis against the baseline) and a path to production: versioned, point-in-time training datasets, batch or real-time scoring, drift monitoring and scheduled retraining.
In Practice
- Demand forecasting with random forest and linear regression pipelines that use graph-derived features from a Neo4j knowledge graph.
- Quality intelligence for a multi-line manufacturer: FastRP embeddings, KNN similarity and statistical outlier detection across hundreds of millions of sampling records.
- Event modeling with BERT sentence embeddings over a multi-billion-record global news graph, plus TensorFlow multi-label text classification.
- Signal processing and clustering of EEG time series for a medical-device platform.