Advanced Hybrid Transformer LSTM Technique with Attention and TS Mixer for Drilling Rate of Penetration Prediction
Saddam Hussain Khan
公開日: 2025/8/7
Abstract
Accurate prediction of the Rate of Penetration (ROP) is pivotal for drilling optimization, yet it remains a persistent challenge due to the nonlinear, dynamic, and heterogeneous nature of drilling data. This study introduces a novel hybrid deep learning architecture in which input data are first processed through a customized Long Short-Term Memory (LSTM) network to capture multi-scale temporal dependencies aligned with drilling operational cycles, and the resulting features are subsequently refined by an Enhanced Transformer encoder with drilling-specific positional encodings and real-time optimization. Concurrently, the same input is directed to a Time-Series Mixer (TS-Mixer) block that enables efficient cross-feature modeling of static and categorical attributes such as lithology indices and mud properties. The outputs from the enhanced Transformer and TS-Mixer are concatenated, after which an adaptive attention selectively emphasizes the most informative feature representations for accurate ROP prediction. The proposed framework fuses sequential memory, static feature interactions, global contextual learning, and dynamic feature weighting, providing a comprehensive solution to the heterogeneous and event-driven nature of drilling dynamics. Evaluation on a real-world drilling dataset demonstrates benchmark-leading performance, achieving an Rsqaure of 0.9988 and a MAPE of 1.447%, significantly surpassing standalone and hybrid baselines. Model interpretability is achieved through SHAP and LIME, and comparisons between actual and predicted curves, along with bias checks, confirm the accuracy and fairness of the model across various scenarios. This advanced hybrid approach enables dependable real-time ROP prediction, supporting the development of intelligent, cost-effective drilling optimization systems with significant operational benefits.