Research

Abstract

Numerical simulations consist of many components that affect the simulation accuracy and the required computational resources. However, finding an optimal combination of components and their parameters under constraints can be a difficult, time-consuming and often manual process. Classical adaptivity does not fully solve the problem, as it comes with significant implementation cost and is difficult to expand to multi-dimensional parameter spaces. Also, many existing data-based optimization approaches treat the optimization problem as a black-box, thus requiring a large amount of data. We present a constrained, model-based Bayesian optimization approach that avoids black-box models by leveraging existing knowledge about the simulation components and properties of the simulation behavior. The main focus of this paper is on the stochastic modeling ansatz for simulation error and run time as optimization objective and constraint, respectively. To account for data covering multiple orders of magnitude, our approach operates on a logarithmic scale. The models use a priori knowledge of the simulation components such as convergence orders and run time estimates. Together with suitable priors for the model parameters, the model is able to make accurate predictions of the simulation behavior. Reliably modeling the simulation behavior yields a fast optimization procedure because it enables the optimizer to quickly indicate promising parameter values. We test our approach experimentally using the multi-scale muscle simulation framework OpenDiHu and show that we successfully optimize the time step widths in a time splitting approach in terms of minimizing the overall error under run time constraints.

Abstract

SYCL provides programmers with four, and in the case of AdaptiveCpp even five, ways for calling and writing a device kernel. This paper analyzes the performance of these diverse kernel invocation types for DPC++ and AdaptiveCpp as SYCL implementations on an NVIDIA A100 GPU, an AMD Instinct MI210 GPU, and a dual-socket AMD EPYC 9274F CPU. Using the example of a kernel matrix assembly, we show why the performance can differ by a factor of 100 in the worst case on the same hardware for the same problem using different SYCL implementations and kernel invocation types.

Abstract

In machine learning and scientific computing, some of the biggest challenges are efficient and performant portable computing. With our Parallel Least Squares Support Vector Machine (PLSSVM) library, we have not only developed an unrivaled Support Vector Machine (SVM) implementation for huge dense data sets, but we have also created a representative benchmark for a frequently encountered task in scientific computing, a (implicit) matrix-vector multiplication. PLSSVM supports multiple backends—OpenMP, CUDA, HIP, OpenCL, and SYCL—to be able to target the most widely used hardware platforms in machine learning and scientific computing. In this paper, we use PLSSVM to compare different DPC++ and Open SYCL (formerly known as hipSYCL) versions over the period of one year. Furthermore, we compared two versions (one from February and the other from November 2022) with each other and report their respective performance evolution in depth. We also put these results in relation to our other implemented backends and report their performance portability on three different hardware platforms, an NVIDIA and AMD GPU and an Intel CPU. Our results show that installing new DPC++ and Open SYCL versions can have surprisingly vast impacts in both directions. In our case, the nd_range kernel runtimes were up to faster on an NVIDIA GPU when using a newer DPC++ compiler. Also for Open SYCL, using the new omp.accelerated compilation flow improves the nd_range performance on CPUs by over . When compared to OpenCL, in our results, SYCL also offers a better performance portability while being easier to use, indicated by drastically fewer lines of code needed in our PLSSVM library. While OpenCL only has a performance portability of , DPC++ achieved the highest value with within the performance metric provided by Pennycook et al. 23. The code, utility scripts, and documentation are all publicly available on GitHub: https://github.com/SC-SGS/PLSSVM.

Abstract

In machine learning and scientific computing, some of the biggest challenges are efficient and performant portable computing. With our Parallel Least Squares Support Vector Machine (PLSSVM) library, we have not only developed an unrivaled Support Vector Machine (SVM) implementation for huge dense data sets, but we have also created a representative benchmark for a frequently encountered task in scientific computing, a (implicit) matrix-vector multiplication. PLSSVM supports multiple backends—OpenMP, CUDA, HIP, OpenCL, and SYCL—to be able to target the most widely used hardware platforms in machine learning and scientific computing. In this paper, we use PLSSVM to compare different DPC++ and Open SYCL (formerly known as hipSYCL) versions over the period of one year. Furthermore, we compared two versions (one from February and the other from November 2022) with each other and report their respective performance evolution in depth. We also put these results in relation to our other implemented backends and report their performance portability on three different hardware platforms, an NVIDIA and AMD GPU and an Intel CPU. Our results show that installing new DPC++ and Open SYCL versions can have surprisingly vast impacts in both directions. In our case, the nd_range kernel runtimes were up to faster on an NVIDIA GPU when using a newer DPC++ compiler. Also for Open SYCL, using the new omp.accelerated compilation flow improves the nd_range performance on CPUs by over . When compared to OpenCL, in our results, SYCL also offers a better performance portability while being easier to use, indicated by drastically fewer lines of code needed in our PLSSVM library. While OpenCL only has a performance portability of , DPC++ achieved the highest value with within the performance metric provided by Pennycook et al. 23. The code, utility scripts, and documentation are all publicly available on GitHub: https://github.com/SC-SGS/PLSSVM.

Abstract

We introduce a partitioned coupling approach for iterative coupling of flow processes in deformable fractures embedded in a poro-elastic medium that is enhanced by interface quasi-Newton (IQN) methods. In this scope, a unique computational decomposition into a fracture flow and a poro-elastic domain is developed, where communication and numerical coupling of the individual solvers are realized by consulting the open-source library preCICE. The underlying physical problem is introduced by a brief derivation of the governing equations and interface conditions of fracture flow and poro-elastic domain followed by a detailed discussion of the partitioned coupling scheme. We evaluate the proposed implementation and undertake a convergence study to compare a classical interface quasi-Newton inverse least-squares (IQN-ILS) with the more advanced interface quasi-Newton inverse multi-vector Jacobian (IQN-IMVJ) method. These coupling approaches are verified for an academic test case before the generality of the proposed strategy is demonstrated by simulations of two complex fracture networks. In contrast to the development of specific solvers, we promote the simplicity and computational efficiency of the proposed partitioned coupling approach using preCICE and FEniCS for parallel computations of hydro-mechanical processes in complex, three-dimensional fracture networks.

Abstract

Machine learning algorithms must be able to efficiently cope with massive data sets. Therefore, they have to scale well on any modern system and be able to exploit the computing power of accelerators independent of their vendor. In the field of supervised learning, Support Vector Machines (SVMs) are widely used. However, even modern and optimized implementations such as LIBSVM or ThunderSVM do not scale well for large non-trivial dense data sets on cutting-edge hardware: Most SVM implementations are based on Sequential Minimal Optimization, an optimized though inherent sequential algorithm. Hence, they are not well-suited for highly parallel GPUs. Furthermore, we are not aware of a performance portable implementation that supports CPUs and GPUs from different vendors. We have developed the PLSSVM library to solve both issues. First, we resort to the formulation of the SVM as a least squares problem. Training an SVM then boils down to solving a system of linear equations for which highly parallel algorithms are known. Second, we provide a hardware independent yet efficient implementation: PLSSVM uses different interchangeable backends--OpenMP, CUDA, OpenCL, SYCL--supporting modern hardware from various vendors like NVIDIA, AMD, or Intel on multiple GPUs. PLSSVM can be used as a drop-in replacement for LIBSVM. We observe a speedup on CPUs of up to 10 compared to LIBSVM and on GPUs of up to 14 compared to ThunderSVM. Our implementation scales on many-core CPUs with a parallel speedup of 74.7 on up to 256 CPU threads and on multiple GPUs with a parallel speedup of 3.71 on four GPUs. The code, utility scripts, and documentation are all available on GitHub: https://github.com/SC-SGS/PLSSVM.

Abstract

The new software FEniCS–preCICE is a middle software layer, sitting in between the existing finite-element library FEniCS and the coupling library preCICE. The middle layer simplifies coupling (existing) FEniCS application codes to other simulation software via preCICE. To this end, FEniCS–preCICE converts between FEniCS and preCICE mesh and data structures, provides easy-to-use coupling conditions, and manages data checkpointing for implicit coupling. The new software is a library itself and follows a FEniCS-native style. Only a few lines of additional code are necessary to prepare a FEniCS application code for coupling. We illustrate the functionality of FEniCS–preCICE by two examples: a FEniCS heat conduction code coupled to OpenFOAM and a FEniCS linear elasticity code coupled to SU2. The results of both scenarios are compared with other simulation software showing good agreement.

Abstract

We present a numerical scheme for solving an inverse problem for parameter estimation in tumor growth models for glioblastomas, a form of aggressive primary brain tumor. The growth model is a reaction–diffusion partial differential equation (PDE) for the tumor concentration. We use a PDE-constrained optimization formulation for the inverse problem. The unknown parameters are the reaction coefficient (proliferation), the diffusion coefficient (infiltration), and the initial condition field for the tumor PDE. Segmentation of magnetic resonance imaging (MRI) scans drive the inverse problem where segmented tumor regions serve as partial observations of the tumor concentration. Like most cases in clinical practice, we use data from a single time snapshot. Moreover, the precise time relative to the initiation of the tumor is unknown, which poses an additional difficulty for inversion. We perform a frozen-coefficient spectral analysis and show that the inverse problem is severely ill-posed. We introduce a biophysically motivated regularization on the structure and magnitude of the tumor initial condition. In particular, we assume that the tumor starts at a few locations (enforced with a sparsity constraint on the initial condition of the tumor) and that the initial condition magnitude in the maximum norm is equal to one. We solve the resulting optimization problem using an inexact quasi-Newton method combined with a compressive sampling algorithm for the sparsity constraint. Our implementation uses PETSc and AccFFT libraries. We conduct numerical experiments on synthetic and clinical images to highlight the improved performance of our solver over a previously existing solver that uses standard two-norm regularization for the calibration parameters. The existing solver is unable to localize the initial condition. Our new solver can localize the initial condition and recover infiltration and proliferation. In clinical datasets (for which the ground truth is unknown), our solver results in qualitatively different solutions compared to the two-norm regularized solver.

Abstract

In this work, we present a scalable and efficient parallel solver for the partitioned solution of fluid–structure interaction problems through multi-code coupling. Two instances of an in-house parallel software, TermoFluids, are used to solve the fluid and the structural sub-problems, coupled together on the interface via the preCICE coupling library. For fluid flow, the Arbitrary Lagrangian–Eulerian form of the Navier–Stokes equations is solved on an unstructured conforming grid using a second-order finite-volume discretization. A parallel dynamic mesh method for unstructured meshes is used to track the moving boundary. For the structural problem, the nonlinear elastodynamics equations are solved on an unstructured grid using a second-order finite-volume method. A semi-implicit FSI coupling method is used which segregates the fluid pressure term and couples it strongly to the structure, while the remaining fluid terms and the geometrical nonlinearities are only loosely coupled. A robust and advanced multi-vector quasi-Newton method is used for the coupling iterations between the solvers. Both the fluid and the structural solver use distributed-memory parallelism. The intra-solver communication required for data update in the solution process is carried out using non-blocking point-to-point communicators. The inter-code communication is fully parallel and point-to-point, avoiding any central communication unit. Inside each single-physics solver, the load is balanced by dividing the computational domain into fairly equal blocks for each process. Additionally, a load balancing model is used at the inter-code level to minimize the overall idle time of the processes. Two practical test cases in the context of hemodynamics are studied, demonstrating the accuracy and computational efficiency of the coupled solver. Strong scalability test results show a parallel efficiency of 83\% on 10,080 CPU cores.

Abstract

We investigate a partitioned coupling scheme applied to a system of free flow over a porous medium. The coupling scheme follows a partitioned approach which means that the flow fields in the two domains are solved separately and information is exchanged over the sharp interface that separates the free-flow and the porous-medium domain. Technically, the coupling is realized via the open-source library preCICE, employing a pure black-box approach such that different solver frameworks can be used with highly specialized solvers in each of the flow domains. We investigate the partitioned coupling approach numerically by comparing it to a monolithic coupling scheme with respect to convergence and accuracy. This is the first time a partitioned black-box coupling is used for coupling free flow and porous-media flow. The coupling approach is numerically validated and different partitioned coupling approaches are compared with each other.

Abstract

In this tutorial review paper, we present our minimally invasive approach for integrating dynamically adaptive tree-structured grids into existing simulation software that has been developed for regular Cartesian grids. We introduce different physical models that we target and that span a wide range of typical simulation characteristics -- from grid-based Lattice-Boltzmann, finite volume and finite difference discretized models to particle-based molecular dynamics models. We derive the respective typical data access requirements and extensions of the algorithms to adaptively refined grids along with possible grid adaptivity criteria. In addition, after introducing basics of tree-structured adaptively refined grids, we present the adaptive grid framework p4est and our enhancement of p4est in order to provide a grid and partitioning infrastructure that can easily be used in existing simulation codes. Finally, we explain how such a grid infrastructure can be integrated into regular grid codes in general in three major steps and how we integrated p4est in the soft matter simulation package ESPResSo in particular. A summary of results fro m previously published performance and scalability studies together with new results for more realistic coupled simulation scenarios shows the efficiency and validity of the resulting new version of ESPResSo.

Abstract

We present SIBIA (Scalable Integrated Biophysics-based Image Analysis), a framework for joint image registration and biophysical inversion and we apply it to analyse MR images of glioblastomas (primary brain tumors). We have two applications in mind. The first one is normal-to-abnormal image registration in the presence of tumor-induced topology differences. The second one is biophysical inversion based on single-time patient data. The underlying optimization problem is highly non-linear and non-convex and has not been solved before with a gradient-based approach. Given the segmentation of a normal brain MRI and the segmentation of a cancer patient MRI, we determine tumor growth parameters and a registration map so that if we ``grow a tumor'' (using our tumor model) in the normal brain and then register it to the patient image, then the registration mismatch is as small as possible. This ``$\backslash$emphcoupled problem'' two-way couples the biophysical inversion and the registration problem. In the image registration step we solve a large-deformation diffeomorphic registration problem parameterized by an Eulerian velocity field. In the biophysical inversion step we estimate parameters in a reaction-diffusion tumor growth model that is formulated as a partial differential equation (PDE). In SIBIA, we couple these two sub-components in an iterative manner. We first presented the components of SIBIA in ``Gholami et al, Framework for Scalable Biophysics-based Image Analysis, IEEE/ACM Proceedings of the SC2017'', in which we derived parallel distributed memory algorithms and software modules for the decoupled registration and biophysical inverse problems. In this paper, our contributions are the introduction of a PDE-constrained optimization formulation of the coupled problem, and the derivation of a Picard iterative solution scheme. We perform extensive tests to experimentally assess the performance of our method on synthetic and clinical datasets. We demonstrate the convergence of the SIBIA optimization solver in different usage scenarios. We demonstrate that using SIBIA, we can accurately solve the coupled problem in three dimensions (256^3 resolution) in a few minutes using 11 dual-x86 nodes.

Abstract

We present a novel formulation for the calibration of a biophysical tumor growth model from a single-time snapshot, MRI scan of a glioblastoma patient. Tumor growth models are typically nonlinear parabolic partial differential equations (PDEs). Thus, we have to generate a second snapshot to be able to extract significant information from a single patient snapshot. We create this two-snapshot scenario as follows. We use an atlas (an average of several scans of healthy individuals) as a substitute for an earlier, pretumor, MRI scan of the patient. Then, using the patient scan and the atlas, we combine image-registration algorithms and parameter estimation algorithms to achieve a better estimate of the healthy patient scan and the tumor growth parameters that are consistent with the data. Our scheme is based on our recent work (Scheufele et al, ``Biophysically constrained diffeomorphic image registration, Tumor growth, Atlas registration, Adjoint-based methods, Parallel algorithms'', CMAME, 2018), but apply a different and novel scheme where the tumor growth simulation in contrast to the previous work is executed in the patient brain domain and not in the atlas domain yielding more meaningful patient-specific results. As a basis, we use a PDE-constrained optimization framework. We derive a modified Picard-iteration-type solution strategy in which we alternate between registration and tumor parameter estimation in a new way. In addition, we consider an â„“1 sparsity constraint on the initial condition for the tumor and integrate it with the new joint inversion scheme. We solve the subproblems with a reduced-space, inexact Gauss-Newton-Krylov/quasi-Newton methods. We present results using real brain data with synthetic tumor data that show that the new scheme reconstructs the tumor parameters in a more accurate and reliable way compared to our earlier scheme.

Abstract

We apply POD-DEIM model order reduction to a 0D/1D model used to simulate the propagation of action potentials through the myocardium or along skeletal muscle fibers. This corresponding system of ODEs (reaction) and PDEs (diffusion) is called the monodomain equation. 0D sets of ODEs describing the ionic currents flowing across the cell membrane are coupled along muscle fibers through a \$1\$D diffusion process for the transmembrane potential. Due to the strong coupling of the transmembrane potential and other state variables describing the behavior of the membrane, a total reduction strategy including all degrees of freedom turns out to be more efficient than a reduction of only the transmembrane potential. The total reduction approach is four orders of magnitude more accurate than partial reduction and shows a faster convergence in the number of POD modes with respect to the mesh refinement. A speedup of \$2.7\$ is achieved for a 1D mesh with \$320\$ nodes. Considering the DEIM approximation in combination with the total reduction, the nonlinear functions corresponding to the ionic state variables are also approximated in addition to the nonlinear ionic current in the monodomain equation. We observe that the same number of DEIM interpolation points as the number of POD modes is the optimal choice regarding stability, accuracy and runtime for the current POD-DEIM approach.

Abstract

We apply the POD-DEIM model order reduction to the propagation of the transmembrane potential along \$1\$D muscle fibers. This propagation is represented using the monodomain partial differential equation. The monodomain equation, which is a reaction-diffusion equation, is coupled through its reaction term with a set of ordinary differential equations, which provide the ionic current across the cell membrane. Due to the strong coupling of the transmembrane potential and ionic state variables, we reduce them all together proposing a total reduction strategy. We compare the current strategy with the conventional strategy of reducing the transmembrane potential. Considering the current approach, the discrete system matrix is slightly modified to adjust for the size. However, size of the precomputed reduced system matrix remains the same, which means the same computational cost. The current approach appears to be four orders of magnitude more accurate considering the equivalent number of modes on the same grid in comparison to the conventional approach. Moreover, it shows a faster convergence in the number of POD modes with respect to the grid refinement. Using the DEIM approximation of nonlinear functions in combination with the total reduction, the nonlinear functions corresponding to the ionic state variables are also approximated besides the nonlinear ionic current in the monodomain equation. For the current POD-DEIM approach, it appears that the same number of DEIM interpolation points as the number of POD modes is the optimal choice regarding stability, accuracy and runtime.

Abstract

We present our approach for a scalable implementation of coupled soft matter simulations for inhomogeneous applications based on the simulation package ESPResSo and an extended version of the adaptive grid framework p4est. Our main contribution in this paper is the development and implementation of a joint partitioning of two or more distinct octree-based grids based on the concept of a finest common tree. This concept guarantees that, on all grids, the same process is responsible for each point in space and, thus, avoids communication of data in overlapping volumes handled in different partitions. We achieve up to 85 \% parallel efficiency in a weak scaling setting. Our proposed algorithms take only a small fraction of the overall runtime of grid adaption.

Abstract

We present our approach for a scalable implementation of coupled soft matter simulations for inhomogeneous applications based on the simulation package ESPResSo and an extended version of the adaptive grid framework p4est. Our main contribution in this paper is the development and implementation of a joint partitioning of two or more distinct octree-based grids based on the concept of a finest common tree. This concept guarantees that, on all grids, the same process is responsible for each point in space and, thus, avoids communication of data in overlapping volumes handled in different partitions. We achieve up to 85 \% parallel efficiency in a weak scaling setting. Our proposed algorithms take only a small fraction of the overall runtime of grid adaption.

Abstract

Cabin air filters are applied to prevent small particles such as pollen, fine dust and soot amongst others from being transferred into the interior (cabin) of a vehicle. The filter media often make use of the so called electret effect as means for achieving high filtration efficiency at low pressure drop. Thereby, electrostatic filtration effects are supplemented to the well-known mechanical collection mechanisms (such as inertia, diffusion,...). Besides the interference of several fiber-particle interactions (Coulombic attraction/repulsion, induced dipolar forces, image charge effects) particle-particle interactions potentially play an important role. However, this effect is completely neglected in previous research studies due to the high degree of complexity 1. In this work, we present a detailed investigation of the particle behaviour in the inflow area and transition area to the filter media. For a precise description of the underlying physical procedures the simulation is based on a four-way coupling. This approach takes into account the reciprocal influence between the fluid flow and the particle motion as well as the interactions between single electrostatically charged particles. The software package ESPResSo 2 used in this work is based on a molecular dynamic approach and provides the advantage of efficient algorithms for the modelling of electrostatic interactions. In order to emulate the air flow, the molecular dynamic simulation is coupled with a Lattice-Boltzmann fluid. The presented talk focuses on the influence of the particle-particle interactions on the filtration performance. It is elaborated whether the fully coupled system is necessary in order to reflect reality more closely or the simulation can be simplified to reduce the degree of complexity and thus the computational costs. REFERENCES 1 S. Rief, A. Latz, A. Wiegmann, “Computer simulation of Air Filtration including electric surface charges in three-dimensional fibrous micro structures”, Filtration 6.2, (2006). 2 A. Arnold, O. Lenz, S. Kesselheim, R. Weeber, F. Fahrenberger, D. Roehm, P. Ko\AA¡ovan and C. Holm, “ESPResSo 3.1: Molecular Dynamic Software for Coarse-Grained Models”, Lecture Notes in Computational Science and Engineering, (2013).

Abstract

Within the last decade, very sophisticated numerical methods for the iterative and partitioned solution of fluid-structure interaction problems have been developed that allow for high accuracy and very complex scenarios. The combination of these two aspects – accuracy and com- plexity – demands very high computational grid resolutions and, thus, high performance computing methods designed for massively parallel hardware architectures. For those architectures, currently used coupling method that mainly work with a staggered execution of the fluid and structure solver, i.e., the execution of one solver after the other in every outer iteration, lead to severe load imbal- ances: if the flow solver, e.g., scales on a very large number of processors but the structural solver does not due to its limited amount of data and required operations, almost all processors assigned to the coupled simulations are idle while the structure solver executes. We propose two new iterative coupling methods that allow for simultaneous execution of flow and structure solvers. In both cases, we show that pure fixed-point iterations based on the parallel execution of the solvers do not lead to good results, but the combination of parallel solver execution and so-called quasi-Newton methods yields very efficient and robust methods. Those methods are known to be very efficient also for the stabilization of critical scenarios solved with the standard staggered solver execution. We demon- strate the competitive convergence of our methods for various established benchmark scenarios. Both methods are perfectly suited for use with black-box solvers because the quasi-Newton approach uses solely in- and output information of the solvers to approximate the effect of the unknown Jacobians that would be required in a standard Newton solver.

Abstract

Abstract In the emerging field of multi-physics simulations, we often face the challenge to establish new connections between physical fields, to add additional aspects to existing models, or to exchange a solver for one of the involved physical fields. If in such cases a fast prototyping of a coupled simulation environment is required, a partitioned setup using existing codes for each physical field is the optimal choice. As accurate models require also accurate numerics, multi-physics simulations typically use very high grid resolutions and, accordingly, are run on massively parallel computers. Here, we face the challenge to combine flexibility with parallel scalability and hardware efficiency. In this paper, we present the coupling tool preCICE which offers the complete coupling functionality required for a fast development of a multi-physics environment using existing, possibly black-box solvers. We hereby restrict ourselves to bidirectional surface coupling which is too expensive to be done via file communication, but in contrast to volume coupling still a candidate for distributed memory parallelism between the involved solvers. The paper gives an overview of the numerical functionalities implemented in preCICE as well as the user interfaces, i.e., the application programming interface and configuration options. Our numerical examples and the list of different open-source and commercial codes that have already been used with preCICE in coupled simulations show the high flexibility, the correctness, and the high performance and parallel scalability of coupled simulations with preCICE as the coupling unit.

Abstract

We present the minimally-invasive exchange of the regular Cartesian grid in the lattice-Boltzmann solver of ESPResSo by a dynamically-adaptive octree grid. Octree grids are favoured by computer scientists over other grid types as they are very memory-efficient. In addition, they represent a natural generalisation of regular Cartesian grids, such that most discretisation details of a regular grid solver can be maintained. Optimised codes, however, require a special tree-oriented grid traversal, which typically conflicts with existing simulation codes using various iterators, some for only parts of the grid, e.g., boundaries. ESPResSo is a large software package developed for soft-matter simulations involving fluid flow, electrostatic, and electrokinetic effects, and molecular dynamics. The currently used regular Cartesian grid hinders the simulation of realistic domain sizes and significant time periods, a problem that can be solved using grid adaptivity. In a first step, we focus on the lattice-Boltzmann flow solver in ESPResSo. p4est is a grid framework, that already provides dynamically adaptive quadtree and octree grids together with high-level interfaces for flexible grid traversals with direct neighbour access in all grid components. In this paper, we first describe extensions of p4est that were necessary to fulfill certain application requirements. The second part of our work consists of the minimally-invasive changes in ESPResSo preserving the expertise accumulated in the software's implementation over years. Our numerical results demonstrate physical correctness of the implementation, good parallel scalability and low overhead of the dynamical grid adaptivity. These are prerequisites to actually profit from grid adaptivity in terms of being able to simulate larger domains over longer time periods with limited computational resources. Thus, the current status forms a solid basis for further steps such as the development of refinement criteria, the setup of more realistic application scenarios, and a GPU implementation.

Abstract

We present a numerical method for solving the scalar advection-diffusion equation using adaptive mesh re- finement. Our solver has three unique characteristics: (1) it supports arbitrary-order accuracy in space; (2) it allows different discretizations for the velocity and scalar advected quantity; and (3) it combines the method of characteristics with an integral equation formulation. In particular, our solver is based on a second-order accurate, unconditionally stable, semi-Lagrangian scheme combined with a spatially-adaptive Chebyshev octree for discretization. We study the convergence, single-node perfor- mance, strong scaling, and weak scaling of our scheme for several challenging flows that cannot be resolved efficiently without using high-order accurate discretizations. For example, we consider problems for which switching from 4th order to 14th order approximation results in two orders of magnitude speedups for a fixed accuracy. For our largest run, we solve a problem with one billion unknowns on a tree with maximum depth equal to 10 and 14th-order elements on 16,384 cores on the ”STAMPEDE” system at the Texas Advanced Computing Center.

Abstract

Mankind produces enormous amounts of data through simulations, observations, and experiments. With improving technology such as more precise measurements and faster supercomputers this data grows rapidly and has to be stored, computed, and evaluated. Thus, efficient algorithms are needed to process all this data in order to extract useful information. Most of this data is high-dimensional and it depends on a variety of parameters. Methods based on regular grids fail to handle it reasonably since the complexity of the problem exponentially depends on the number of dimensions. However, methods for sparse grids reduce this complexity while maintaining the accuracy. In this work a parallel approach for regression and classification tasks on adaptively refined sparse grids with higher order basis functions is presented. The developed algorithms are tested and analyzed on graphic accelerator cards. The results of a previous work about a parallel approach for the hierarchisation on regular sparse grids are reused to develop highly parallel algorithms for the hierarchisation, multi-evaluation, and the transposed evaluation on adaptively refined sparse grids. Furthermore, several optimizations for these sparse grid operations are explained. The hierarchisation as well as the multi-evaluation make use of the hierarchical structure of the grid to reduce the problem’s complexity, and the effects of sorting the evaluation points along a space-filling curve on the runtime of the operations are analyzed. The space-filling curve methods not only improve the cache usage and thread coherence of the parallel algorithms, but they also reduce the computational effort of algorithms based on the streaming approach. The transposed evaluation achieves speed-ups by factors from 2 to up to 11 compared to a parallel implementation without space-filling curve optimizations. The effects on the thread coherence and the cache usage improve the runtime of the hierarchical evaluation by a factor of up to 10 for adaptively refined grids. Additionally, a benchmark analysis of the developed methods with respect to the maximal polynomial degree shows that the degree for most of the basis functions is strongly limited by the grid level. Thus, the runtime and the complexity do not behave in a linear fashion with respect to the maximal polynomial degree as expected.

Abstract

For multi-field simulations involving a larger number of different physical fields and in cases where the involved fields or simulation codes change due to new modelling insigts, e.g., flexible and robust partitioned coupling schemes are an important prerequisite to keep time-to-solution within reasonable limits. They allow for a fast, almost plug-and-play combination of existing established codes to the respective multi-field simulation environment. In this paper, we study a class of coupling approaches that we originally introduced in order to improve the parallel scalability of partitioned simulations. Due to the symmetric structure of these coupling methods and the use of 'long' vectors of coupling data comprising the input and output of all involved codes at a time, they turn out to be particularly suited also for simulations involving more than two coupled fields. As standard two-field coupling schemes are not suited for such cases as shown in our numerical results, this allows the simulation of a new range of applications in a partitioned way.

Abstract

This book presents selected papers from the 3rd International Workshop on Computational Engineering held in Stuttgart from October 6 to 10, 2014, bringing together innovative contributions from related fields with computer science and mathematics as an important technical basis among others. The workshop discussed the state of the art and the further evolution of numerical techniques for simulation in engineering and science. We focus on current trends in numerical simulation in science and engineering, new requirements arising from rapidly increasing parallelism in computer architectures, and novel mathematical approaches. Accordingly, the chapters of the book particularly focus on parallel algorithms and performance optimization, coupled systems, and complex applications and optimization.

Abstract

To simulate fluid-acoustic interaction, we couple inviscid Euler equations in the near-field, which is relevant for noise generation, to linearized Euler equations in the far-field. This allows us to separate the critical scales and treat each domain with an individual discretization. Both fields are computed by the high-order discontinuous Galerkin solver Ateles, while we couple the solvers at the interface by the library preCICE. We discuss a detailed performance analysis of the coupled simulation on massively parallel systems. Furthermore, to show the full potential of our approach, we simulate a flow around a sphere.

Abstract

The partitioned simulation of fluid–structure interactions offers great flexibility in terms of exchanging flow and structure solver and using existing established codes. However, it often suffers from slow convergence and limited parallel scalability. Quasi-Newton or accelerated fixed-point iterations are a very efficient way to solve the convergence issue. At the same time, they stabilize and speed up not only the standard staggered fluid–structure coupling iterations, but also the variant with simultaneous execution of flow and structure solver that is fairly inefficient if no acceleration methods for the underlying fixed-point iteration are used. In this chapter, we present a review on combinations of iteration patterns (parallel and staggered) and of quasi-Newton methods and compare their suitability in terms of convergence speed, robustness, and parallel scalability. Some of these variants use the so-called manifold mapping that yields an additional speedup by using an approach that can be interpreted as a generalization of the multi-level idea.

Abstract

Strongly coupled fluid-structure interaction simulations often suffer from slow convergence, limited parallel scalability or difficulties in using black-box solvers. As partitioned simulations still play an important role in cases where new combinations of models, discretizations and codes have to be tested in an easy and fast way, we propose a combination of a parallel black-box coupling with a manifold mapping algorithm as an acceleration method. In this approach, we combine a com- putationally inexpensive low-fidelity FSI model with a high-fidelity FSI model to reduce the number of coupling iterations of the high fidelity FSI model. Information from previous time steps is taken into account with a secant update step similar to the Broyden update. The used black-box approach is applied for an incompressible laminar flow over a fixed cylinder with an attached flexible flap and a wave prop- agation in a three-dimensional elastic tube problem. A reduction of approximately 55 \% in terms of high fidelity iterations is achieved compared to the Anderson mix- ing method if the fluid and the structure solvers are executed in parallel.

Abstract

leer

Abstract

Using software for large-scale simulations has become an important research method in many disciplines. With increasingly complex simulations, simulation software becomes a valuable assest. Yet, the quality of many simulation codes is worrying. In this paper, we want to collect and structure the challenges for a systematic simulation software engineering as a reference and the basis for further research. We describe our own experiences with developing simulation software and collaborating with non-computer-scientists. We complement our experienced challenges with a brief literature review. We structured the challenges for simulation software engineering into six areas: motivation and recognition; education and training; developer turnover; software length of life; verification, validation and debugging; and efficiency vs. maintainability. Overcoming these challenges needs efforts from research agencies, scientific computing researchers as well as software engineering researchers.

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

leer

Abstract

The central theme of the book is the flow of information from experimental approaches in biofilm research to simulation and modeling of complex wastewater systems. Probably the greatest challenge in wastewater research lies in using the methods and the results obtained in one scientific discipline to design intelligent experiments in other disciplines, and eventually to improve the knowledge base the practitioner needs to run wastewater treatment plants. The purpose of Biofilms in Wastewater Treatment is to provide engineers with the knowledge needed to apply the new insights gained by researchers. The authors provide an authoritative insight into the function of biofilms on a technical and on a lab-scale, cover some of the exciting new basic microbiological and wastewater engineering research involving molecular biology techniques and microscopy, and discuss recent attempts to predict the development of biofilms. This book is divided into 3 sections: Modeling and Simulation; Architecture, Population Structure and Function; and From Fundamentals to Practical Application, which all start with a scientific question. Individual chapters attempt to answer the question and present different angles of looking at problems. In addition there is an extensive glossary to familiarize the non-expert with unfamiliar terminology used by microbiologists and computational scientists.

Abstract

leer

Abstract

leer