Get our free extension to see links to code for papers anywhere online!Free add-on: code for papers everywhere!Free add-on: See code for papers anywhere!

Ali Kashefi, Tapan Mukerji

Fourier neural operators (FNOs) are invariant with respect to the size of input images, and thus images with any size can be fed into FNO-based frameworks without any modification of network architectures, in contrast to traditional convolutional neural networks (CNNs). Leveraging the advantage of FNOs, we propose a novel deep-learning framework for classifying images with varying sizes. Particularly, we simultaneously train the proposed network on multi-sized images. As a practical application, we consider the problem of predicting the label (e.g., permeability) of three-dimensional digital porous media. To construct the framework, an intuitive approach is to connect FNO layers to a classifier using adaptive max pooling. First, we show that this approach is only effective for porous media with fixed sizes, whereas it fails for porous media of varying sizes. To overcome this limitation, we introduce our approach: instead of using adaptive max pooling, we use static max pooling with the size of channel width of FNO layers. Since the channel width of the FNO layers is independent of input image size, the introduced framework can handle multi-sized images during training. We show the effectiveness of the introduced framework and compare its performance with the intuitive approach through the example of the classification of three-dimensional digital porous media of varying sizes.

Via

Jaehong Chung, Rasool Ahmad, WaiChing Sun, Wei Cai, Tapan Mukerji

This study presents a Graph Neural Networks (GNNs)-based approach for predicting the effective elastic moduli of rocks from their digital CT-scan images. We use the Mapper algorithm to transform 3D digital rock images into graph datasets, encapsulating essential geometrical information. These graphs, after training, prove effective in predicting elastic moduli. Our GNN model shows robust predictive capabilities across various graph sizes derived from various subcube dimensions. Not only does it perform well on the test dataset, but it also maintains high prediction accuracy for unseen rocks and unexplored subcube sizes. Comparative analysis with Convolutional Neural Networks (CNNs) reveals the superior performance of GNNs in predicting unseen rock properties. Moreover, the graph representation of microstructures significantly reduces GPU memory requirements (compared to the grid representation for CNNs), enabling greater flexibility in the batch size selection. This work demonstrates the potential of GNN models in enhancing the prediction accuracy of rock properties and boosting the efficiency of digital rock analysis.

Via

Rasool Ahmad, Mingliang Liu, Michael Ortiz, Tapan Mukerji, Wei Cai

Determining effective elastic properties of rocks from their pore-scale digital images is a key goal of digital rock physics (DRP). Direct numerical simulation (DNS) of elastic behavior, however, incurs high computational cost; and surrogate machine learning (ML) model, particularly convolutional neural network (CNN), show promises to accelerate homogenization process. 3D CNN models, however, are unable to handle large images due to memory issues. To address this challenge, we propose a novel method that combines 3D CNN with hierarchical homogenization method (HHM). The surrogate 3D CNN model homogenizes only small subimages, and a DNS is used to homogenize the intermediate image obtained by assembling small subimages. The 3D CNN model is designed to output the homogenized elastic constants within the Hashin-Shtrikman (HS) bounds of the input images. The 3D CNN model is first trained on data comprising equal proportions of five sandstone (quartz mineralogy) images, and, subsequently, fine-tuned for specific rocks using transfer learning. The proposed method is applied to homogenize the rock images of size 300x300x300 and 600x600x600 voxels, and the predicted homogenized elastic moduli are shown to agree with that obtained from the brute-force DNS. The transferability of the trained 3D CNN model (using transfer learning) is further demonstrated by predicting the homogenized elastic moduli of a limestone rock with calcite mineralogy. The surrogate 3D CNN model in combination with the HHM is thus shown to be a promising tool for the homogenization of large 3D digital rock images and other random media

Via

Ali Kashefi, Leonidas J. Guibas, Tapan Mukerji

Regular physics-informed neural networks (PINNs) predict the solution of partial differential equations using sparse labeled data but only over a single domain. On the other hand, fully supervised learning models are first trained usually over a few thousand domains with known solutions (i.e., labeled data) and then predict the solution over a few hundred unseen domains. Physics-informed PointNet (PIPN) is primarily designed to fill this gap between PINNs (as weakly supervised learning models) and fully supervised learning models. In this article, we demonstrate that PIPN predicts the solution of desired partial differential equations over a few hundred domains simultaneously, while it only uses sparse labeled data. This framework benefits fast geometric designs in the industry when only sparse labeled data are available. Particularly, we show that PIPN predicts the solution of a plane stress problem over more than 500 domains with different geometries, simultaneously. Moreover, we pioneer implementing the concept of remarkable batch size (i.e., the number of geometries fed into PIPN at each sub-epoch) into PIPN. Specifically, we try batch sizes of 7, 14, 19, 38, 76, and 133. Additionally, the effect of the PIPN size, symmetric function in the PIPN architecture, and static and dynamic weights for the component of the sparse labeled data in the loss function are investigated.

Via

Ali Kashefi, Tapan Mukerji

ChatGPT is a large language model recently released by the OpenAI company. In this technical report, we explore for the first time the capability of ChatGPT for programming numerical algorithms. Specifically, we examine the capability of GhatGPT for generating codes for numerical algorithms in different programming languages, for debugging and improving written codes by users, for completing missed parts of numerical codes, rewriting available codes in other programming languages, and for parallelizing serial codes. Additionally, we assess if ChatGPT can recognize if given codes are written by humans or machines. To reach this goal, we consider a variety of mathematical problems such as the Poisson equation, the diffusion equation, the incompressible Navier-Stokes equations, compressible inviscid flow, eigenvalue problems, solving linear systems of equations, storing sparse matrices, etc. Furthermore, we exemplify scientific machine learning such as physics-informed neural networks and convolutional neural networks with applications to computational physics. Through these examples, we investigate the successes, failures, and challenges of ChatGPT. Examples of failures are producing singular matrices, operations on arrays with incompatible sizes, programming interruption for relatively long codes, etc. Our outcomes suggest that ChatGPT can successfully program numerical algorithms in different programming languages, but certain limitations and challenges exist that require further improvement of this machine learning model.

Via

Divakar Vashisth, Tapan Mukerji

Petrophysical inversion is an important aspect of reservoir modeling. However due to the lack of a unique and straightforward relationship between seismic traces and rock properties, predicting petrophysical properties directly from seismic data is a complex task. Many studies have attempted to identify the direct end-to-end link using supervised machine learning techniques, but face different challenges such as a lack of large petrophysical training dataset or estimates that may not conform with physics or depositional history of the rocks. We present a rock and wave physics informed neural network (RW-PINN) model that can estimate porosity directly from seismic image traces with no or limited number of wells, with predictions that are consistent with rock physics and geologic knowledge of deposition. As an example, we use the uncemented sand rock physics model and normal-incidence wave physics to guide the learning of RW-PINN to eventually get good estimates of porosities from normal-incidence seismic traces and limited well data. Training RW-PINN with few wells (weakly supervised) helps in tackling the problem of non-uniqueness as different porosity logs can give similar seismic traces. We use weighted normalized root mean square error loss function to train the weakly supervised network and demonstrate the impact of different weights on porosity predictions. The RW-PINN estimated porosities and seismic traces are compared to predictions from a completely supervised model, which gives slightly better porosity estimates but poorly matches the seismic traces, in addition to requiring a large amount of labeled training data. In this paper, we demonstrate the complete workflow for executing petrophysical inversion of seismic data using self-supervised or weakly supervised rock physics informed neural networks.

Via

Ali Kashefi, Tapan Mukerji

We propose a novel deep learning framework for predicting permeability of porous media from their digital images. Unlike convolutional neural networks, instead of feeding the whole image volume as inputs to the network, we model the boundary between solid matrix and pore spaces as point clouds and feed them as inputs to a neural network based on the PointNet architecture. This approach overcomes the challenge of memory restriction of graphics processing units and its consequences on the choice of batch size, and convergence. Compared to convolutional neural networks, the proposed deep learning methodology provides freedom to select larger batch sizes, due to reducing significantly the size of network inputs. Specifically, we use the classification branch of PointNet and adjust it for a regression task. As a test case, two and three dimensional synthetic digital rock images are considered. We investigate the effect of different components of our neural network on its performance. We compare our deep learning strategy with a convolutional neural network from various perspectives, specifically for maximum possible batch size. We inspect the generalizability of our network by predicting the permeability of real-world rock samples as well as synthetic digital rocks that are statistically different from the samples used during training. The network predicts the permeability of digital rocks a few thousand times faster than a Lattice Boltzmann solver with a high level of prediction accuracy.

Via

John Mern, Anil Yildiz, Larry Bush, Tapan Mukerji, Mykel J. Kochenderfer

Online solvers for partially observable Markov decision processes have difficulty scaling to problems with large action spaces. This paper proposes a method called PA-POMCPOW to sample a subset of the action space that provides varying mixtures of exploitation and exploration for inclusion in a search tree. The proposed method first evaluates the action space according to a score function that is a linear combination of expected reward and expected information gain. The actions with the highest score are then added to the search tree during tree expansion. Experiments show that PA-POMCPOW is able to outperform existing state-of-the-art solvers on problems with large discrete action spaces.

Via

John Mern, Anil Yildiz, Zachary Sunberg, Tapan Mukerji, Mykel J. Kochenderfer

Online solvers for partially observable Markov decision processes have difficulty scaling to problems with large action spaces. Monte Carlo tree search with progressive widening attempts to improve scaling by sampling from the action space to construct a policy search tree. The performance of progressive widening search is dependent upon the action sampling policy, often requiring problem-specific samplers. In this work, we present a general method for efficient action sampling based on Bayesian optimization. The proposed method uses a Gaussian process to model a belief over the action-value function and selects the action that will maximize the expected improvement in the optimal action value. We implement the proposed approach in a new online tree search algorithm called Bayesian Optimized Monte Carlo Planning (BOMCP). Several experiments show that BOMCP is better able to scale to large action space POMDPs than existing state-of-the-art tree search solvers.

Via