Turbulence Introduction¶
The Apex Turbulence module creates effects using APEX Basic IOS particles. The combination of APEX Turbulence and APEX Particles creates a particle system that can be used to create a wide variety of effects using a light weight particle system allowing a large number of particles to be active simultaneously. The effects that Turbulence is designed to create are local around the user with a high fidelity simulation. Good targets for APEX Turbulence are things like jet packs or jet engine exhaust, steam exhaust from a steam engine, and dry ice effects.
The area where a Turbulence Actor is active is called the Turbulence Grid. APEX Particles within the grid are subjected to forces that vary depending upon where in the grid the particles are located and simulation time. The forces and how they vary are within the grid are authored and are known as a Basic Field Sampler which are authored using the APEX BasicFS module.
To perturb the movements of particles within the Turbulence Grid more than the Basic Field Samplers alone can do the Turbulence module supports adding APEX Shapes that block the movement of particles within the shapes. There can be an unlimited number of APEX Shapes created by the game engine and can be moved both within and outside of the Turbulence Grid to block the particles. When an APEX Shape is added to a Turbulence Actor the particles within the grid the APEX Particles within the Grid can collide with the shape and will not occupy the same sapce. These shapes can be visible or invisible as the game engine renderer implements. Currently the APEX Shapes are Sphere, Box, Plane and Capsule.
Note¶
The Turbulence module is only supported on Windows.
Turbulence Assets¶
This link describles all of the Turbulence asset parameters.
Turbulence Actors¶
This link describles all of the Turbulence actor parameters.
Interaction with PhysX Shapes¶
Turbulence actors can interact with PhysX shapes if the collision filtering settings are set correctly (set in the Turbulence asset or actor).
Controlling the effect of PhysX Shapes¶
Colliding PhysX Shapes impart angular and linear velocity onto the grid, and this velocity is determined by the current and previous pose of the shape. There are additionally four parameters that can be set per Turbulence Actor which will affect how colliding shapes affect the grid’s velocity field. These are AngularVelocityMultiplier, AngularVelocityClamp, LinearVelocityMultiplier and LinearVelocityClamp. The Multiplier will be multiplied with the computed velocity, and the resultant velocity will be clamped to the Clamp value. Artists can use these variables to control how much or how little the collision objects affect the grid. To set these values you can use these two API calls on the TurbulenceFS actor:
setAngularVelocityMultiplierAndClamp
setLinearVelocityMultiplierAndClamp
Definition of BasicFS¶
Turbulence Jets have now been replaced by a new Module called BasicFS. BasicFS acts like a jet force inside the turbulence grid and as a torroidal force directly affecting the particles outside the grid. Both the force direction and amount can be specified and these variables can be animated/oscillated over time. Besides Jets BasicFS provides Attractors which attract particles to specified point with specified force.
BasicFS Asset¶
BasicFS assets can be authored by artists. Here are some of the important attributes of the BasicFS asset:
Variables for affecting the shape of basic field samplers in a grid¶
gridShapeRadius(float) Radius of sphere or capsule shape inside of the grid
gridShapeHeight(float) Height of capsule shape inside of the grid
gridBoundaryFadePercentage(float) Percentage of distance from boundary to center where fade out starts. FadePercentage of 0 removes
the fade region, and FadePercentage of 1 makes the fade region take up the entire region of the shape (fade region will start at center of shape)
Variables for affecting the shape of the Turbulence shape outside the grid¶
nearRadius(float)radius of field core, velocity up to this radius is constant
pivotRadius(float) the center of the toroid
farRadius(float)the outside radius of the toroid
directionalStretch(float) the ratio of the height of the shape to its base
Variables for direction and strength¶
fieldDirection(VEC3)
fieldDirectionDeviationAngle(float)
fieldDirectionOscillationPeriod(float)
fieldStrength(float)
fieldStrengthDeviationPercentage(float)
fieldStrengthOscillationPeriod(float)
averageStartDistance(float)
averageEndDistance(float)
Field Variation Details¶
The direction of the field varies with time. This effect fades as a function of the distrance from the center of the field. Inside the grid the oscillating direction is used, and in fake field this oscillation is faded out starting from average start distance. Average start distance and average end distance control the blending of the two fields - the field from fixed direction and the field from variable direction (oscillation). Before start distance there’s only oscillation field & after end distance only fixed field, and there is a smooth-step interpolating between the two (starting from average start distance up to average end distance).
LOD Support¶
Level Of Detail is not supported anymore.
Turbulence Thermodynamics¶
By having heat in the system you can create effects such as hot air rising past an obstacle, cold air wafting down off of objects, and mushroom clouds. Users have the ability to add heat sources to a simulation; these heat sources increase the local temperature in the simulation (which is an additional variable that is tracked in addition to velocity in the grid), and this temperature is advected through the grid (as the fluid moves it carries its local temperature with it). At each time step, the temperature in the grid is used to add velocity impulses to the grid in a user specified direction.
Note that using heat is more powerful than using just jet forces for creating certain types of effects. For example, to have smoke rise past a sphere (rising smoke hits a sphere, then flows around it and continues going upwards) jet forces are insufficient, since once the rising fluid hits the sphere obstacle its velocity is set to zero and it has no more force moving it upwards to flow around the sphere.
How to Use¶
In order to use heat in the simulation you have to:
1. Enable it using: TurbulenceFSActor::setUseHeat(bool). If heat is not enabled the simulation will ignore all heat sources, and the temperature field. Heat is disabled by default since its use does increase workload and decrease performance. 2. Add heat sources to the simulation, covered in “Heat Sources” below. 3. Configure how the simulation should react to the temperature, covered in “Extra Parameters” below.
Heat Sources¶
Heat is added to the system using two types of heat sources, under the NvParameterized included reference, geometryType:
- Box - PxVec3 extents
- Sphere - PxF32 radius
Creating the Heat Source¶
The HeatSourceActorParams NvParameterized class contains the parameters necessary to create a HeatSource Actor:
- position - The heat source may be placed with a particular position.
- averageTemperature - The arithmetic mean of a sample. Normal distribution parameter.
- stdTemperature - Standard deviation of temperature. Normal distribution parameter.
The heat source actor is created, added and removed to an apex scene like all other actors.
Heat sources can be manipulated by¶
- Modifying their underlying APEXShape properties; you can access the APEXShape by calling the interface HeatSourceActor::getSphereShape()or HeatSourceActor::getBoxShape()
- Setting their temperature, defined by an average and a standard deviation. All fluid cells falling inside the heat source will get a random temperature defined by this average and standard deviation.
void HeatSourceActor::setTemperature(float averageTemperature, float stdTemperature);
Extra Parameters¶
The user has the ability to configure how the simulation should react to the temperature, and these options are set through the interface:
TurbulenceFSActor::setHeatBasedParameters(float forceMultiplier, float ambientTemperature, PxVec3 heatForceDirection);
The ambient temperature provides a cutoff for the minimum temperature that adds velocity in the system - if the temperature of the fluid is less than the ambient temperature no velocity is added to the simulation.
The forceMultiplier controls how much the difference in temperature affects the velocity, based on the simple formula:
Impulse_strength = forceMultiplier*(temp - ambientTemp);
The force direction controls what direction the heat induced velocity is applied in - if we want to create hot air rising up then the axis would be set to the up vector, and if we want to create a cold fluid sinking under gravity we would set this axis to -up. An example of using Turbulence heat is shown in the SimpleTurbulence sample in the scene TurbulenceHeatScene (scene 5).
Debug Visualization¶
All APEX modules come with debug visualization rendering, to help understand what is happening in an APEX scene. When you hit the V key in SimpleTurbulence, default APEX visualization rendering is turned on, as well as default PhysX SDK visualization. In addition to the default settings, APEX Turbulence LOD benefits and support visualization are shown. All visualization options will be described below.
Click here for a list of the Turbulence debug visualization parameters
For general information on how to use debug visualization within APEX, please see Debug Visualization.
Starting SimpleTurbulence and hitting ‘V’ (for visualization) displays the following:
Visualizing Velocity Fields¶
Velocity Fields can be visualized in two ways. The first is with 3d veleocity vectors and the second allows visualization of how particles flow through the grid which are termed stream lines. Turbulence debug visualization provides methods of viewing the spatially varying values of velocity fields.
Velocity Vectors¶
The velocity field can be visualized as shaded lines originating at the simulation grid cell centers and extending in the direction of the local velocity (these lines are shaded white at their beginning and red at their end). These parameters control the visualization of the velocity vectors:
VISUALIZE_TURBULENCE_FS_FIELD
TURBULENCE_FS_FIELD_SCALE
TURBULENCE_FS_FIELD_SPACING
Stream Lines¶
The velocity field can also be visualized as streamlines. Streamlines render the paths of particles initialized at regular intervals on the grid and traced through the velocity field.
| Visualization of a streamline (white curved line is the streamline, and the red arrows are the underlying velocity vector field): | |
|---|---|
These parameters control the visualization of the streamlines:
VISUALIZE_TURBULENCE_FS_STREAMLINES
TURBULENCE_FS_STREAMLINES.grid
TURBULENCE_FS_STREAMLINES.tstep
TURBULENCE_FS_STREAMLINES.tmax
Temperature¶
The temperature field is visualized as 3D crosses centered at grid cell centers. The size of the cross denotes the temperature amount at that cell.
These parameters control the visualization of the temperature:
VISUALIZE_TURBULENCE_FS_TEMPERATURE
TURBULENCE_FS_TEMPERATURE_SCALE
TURBULENCE_FS_TEMPERATURE_SPACING
Plane¶
Hit ‘Z’ on running sample to display the plane which is intersecting the bounding box. Lines where faces of bounding box are intersected by the plane will be highlighted.
After you turn plane visualization on both velocity vectors and streamlines start originate on the plane (but still in the box).
These parameters control the visualization of the temperature:
VISUALIZE_TURBULENCE_FS_PLANE
TURBULENCE_FS_PLANE.normal
TURBULENCE_FS_PLANE.offset
Given plane also intersects Euler’s grid, each box (parallelepiped in common case) at some angle. To visualize this, hit ‘X’ while running sample with debug visualization turned on or set the parameter (but note that enabling this option will significantly affect performance and reduce FPS).
VISUALIZE_TURBULENCE_FS_GRID
Miscellaneous Features¶
Implicit Collision Transforms¶
The collision implicit in the Turbulence asset can be arbitrarily transformed using two variables:
implicitCollisionRadius; // 3D radius of collision implicit
//implicitXForm a 4x4 column major matrix denoting the orientation and translation of the implicit
For example, to get an object with a 45 degree rotation around the z axis and a translation of (1,0,0) the implicitXForm is set to
<value name="implicitXForm" type="Mat44">(0.707,0.707,0,0, -0.707,0.707,0,0, 0,0,1,0, -1.0,0,0.0,1)</value>
Add a Single Direction Velocity to a Grid¶
External Actors can add a single directional velocity to the grid using the interface
TurbulenceFSActor::setExternalVelocity(PxVec3 vel);
Note that if multiple calls to this function are made only the last call is honored (i.e. the velocities are not accumulated). Also note that that the value set by this function is persistent across simulation frames (you can call the function once to set the external wind value and until you call this function again with a new value, potentially many frames later, the previous value will continue to get used).
Accessing Field Data as Render Surfaces¶
Field data access is implemented through renderable object defined in TurbulenceRenderable. It has getRenderData() method, which returns TurbulenceRenderData structure, containing render surfaces and their descriptions. Each TurbulenceFS actor has its own renderable object, which stores all needed information and can live after the original actor has been released. User can get TurbulenceRenderable object from TurbulenceFS actor by calling acquireRenderableReference() method, and after using its render data user should release the renderable object:
TurbulenceRenderable* turbulenceRenderable = turbulenceActor->acquireRenderableReference();
if (turbulenceRenderable)
{
const TurbulenceRenderData* renderData = turbulenceRenderable->getRenderData();
//use renderData to access render surfaces
turbulenceRenderable->release();
}
Note
Before accessing render data in renderables, user should call ModuleTurbulenceFS::prepareRenderables(const Scene&) once for each rendering frame.
There are 3 different types of field data, which user can access, defined in TurbulenceFieldType:
- VELOCITY (float4: xyz - 3D velocity, w - velocity magnitude)
- DENSITY (float: density value)
- FLAME (float4: x - temperature, y - remaining fuel, z - burned fuel, w - density)
Note
User have to call enableOutputVelocityField()/enableOutputDensityField()/enableOutputFlameField() methods in ModuleTurbulenceFS to enable needed field types output!
By default Turbulence uses its own format for each field type, but user has an option to override this by implementing getTurbulenceRenderLayout method in TurbulenceRenderCallback. All supported formats for each field type are defined in TurbulenceVelocityFormat/TurbulenceDensityFormat/TurbulenceFlameFormat enums.
To get field data from render surface on CPU user have to use map/unmap methods of UserRenderSurface. By default APEX creates simple render surface internally based on given field data format, it just stores field data in CPU memory, and user need to map render surface, then copy its content to user graphics resource, and finally unmap it. This way is simple but not efficient, so APEX has another way for user to provide custom implementation for render surface, by overriding createRenderSurface() method in TurbulenceRenderCallback. In this custom implementation for CPU access user need to implement map/unmap methods, and can directly map/unmap graphics surface, eliminating data copy.
Turbulence simulation is done on GPU, and there is overhead to copy data from GPU (compute) to CPU and if needed back from CPU to GPU (graphics), and to eliminate this overhead (in case when data is needed only on GPU) Turbulence supports compute-graphics interop, when Turbulence writes render data directly into user graphics surface on GPU. In order for interop to work, user have to implement getCUDAgraphicsResource() method in custom implementation of UserRenderSurface.
Errors and Warnings¶
APEX Turbulence outputs the following error and warning messages using the standard APEX error stream.
| ERROR CODE | MESSAGE | Explanation |
|---|---|---|
| APEX_INVALID_OPERATION | Not yet implemented! | The Turbulence Module does not support dynamically editing the NvParameterized::Interface |
| APEX_INVALID_OPERATION | Not implemented! | The Turbulence module does not support preparing an asset for another platform |
| APEX_DEBUG_WARNING | Driver error = %04d from file <%s>, line %i. | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - invalid value | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - out of memory | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - not initialized | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - launch failure | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - out of resources | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - lauch timeout | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - invalid context | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - error unknown | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - error deitialized | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - error invalid handle | |
| APEX_DEBUG_WARNING | TURBULENCE FATAL ERROR - unknown error | |
| APEX_DEBUG_WARNING | [ERROR] Advection3DDevice::initialize_storage - u dimensions mismatch | |
| APEX_DEBUG_WARNING | [ERROR] Advection3DDevice::initialize_storage - v dimensions mismatch | |
| APEX_DEBUG_WARNING | [ERROR] Advection3DDevice::initialize_storage - v dimensions mismatch | |
| APEX_DEBUG_WARNING | [ERROR] Advection3DDevice::initialize_storage - u,v,w layout mismatch | |
| APEX_DEBUG_WARNING | [ERROR] Advection3DDevice::initialize_storage - u has no ghost cells | |
| APEX_DEBUG_WARNING | [ERROR] Advection3DDevice::initialize_storage - v has no ghost cells | |
| APEX_DEBUG_WARNING | [ERROR] Advection3DDevice::initialize_storage - w has no ghost cells | |
| APEX_DEBUG_WARNING | [ERROR] Advection3DDevice::initialize_storage - garbage hx,hy,hz value | |
| APEX_DEBUG_WARNING | [ERROR] AdvectionMacCormack3DDeviceF::initialize_storage - error allocating extra grids | |
| APEX_DEBUG_WARNING | [ERROR] AdvectionUpwind3DDeviceF::invoke_kernel_downsample - bad level = %d | |
| APEX_DEBUG_WARNING | [ERROR] AdvectionUpwind3DDevice::invoke_kernel_downsample - CUDA error “%s” | A CUDA error occurred. A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] AdvectionUpwind3DDeviceF::invoke_kernel_upsample - bad level = %d | |
| APEX_DEBUG_WARNING | [ERROR] AdvectionUpwind3DDevice::invoke_kernel_upsample - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] AdvectionUpwind3DDevice::invoke_kernel_apply_upwind - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | Before BC: Max u at level %d = %f | |
| APEX_DEBUG_WARNING | After: Max u at level %d = %f | |
| APEX_DEBUG_WARNING | [ERROR] AdvectionUpwind3DDeviceF::solve - garbage req_dt %f! | |
| APEX_DEBUG_WARNING | level %d | |
| APEX_DEBUG_WARNING | [ERROR] AdvectionUpwind3DDeviceF::solve - too fast level = %d, _num_levels = %d! | |
| APEX_DEBUG_WARNING | Max u at level %d = %f | |
| APEX_DEBUG_WARNING | Max deriv_u at level %d = %f | |
| APEX_DEBUG_WARNING | [ERROR] Advection3D::initialize_storage - error allocating derivative grids | |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_boundary_conditions_level1_nocorners - invalid boundary condition types %d, %d, %d, %d, %d,%d | |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_boundary_conditions_level1_nocorners - must have at least 1 ghost point | |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_boundary_conditions_level1_nocorners - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_mac_boundary_conditions_level1 - invalid boundary condition types %d, %d, %d, %d, %d,%d | |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_mac_boundary_conditions_level1 - layout mismatch | |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_mac_boundary_conditions_level1 - must have at least 1 ghost point | |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_mac_boundary_conditions_level1 - v_grid dimension mismatch (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_mac_boundary_conditions_level1 - w_grid dimension mismatch (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_mac_boundary_conditions_level1 - CUDA error in x “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_mac_boundary_conditions_level1 - CUDA error in y “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_mac_boundary_conditions_level1 - CUDA error in z “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] apply_3d_boundary_conditions_level1_nocorners - invalid boundary condition types %d, %d, %d, %d, %d,%d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDeviceT::clear_zero - cudaMemset failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::copy_interior_data - nx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::copy_interior_data - cudaMemcpy failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::copy_all_data - pnx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::copy_all_data - cudaMemcpy failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::copy_interior_data - nx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::copy_interior_data - cudaMemcpy failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::copy_all_data - pnx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::copy_all_data - cudaMemcpy failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::linear_combination - pnx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDeviceF::linear_combination - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::linear_combination - pnx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::linear_combination - pnx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDeviceF::linear_combination - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] Grid1DDevice::init - gpu buffer alloc failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::copy_interior_data - nx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::copy_interior_data - pnx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::linear_combination - nx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::linear_combination - nx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::linear_combination - nx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::init - host buffer alloc failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::copy_interior_data - nx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::copy_interior_data - cudaMemcpy failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::copy_all_data - pnx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::copy_all_data - cudaMemcpy failed with %s | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::copy_all_data - pnx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost copy_partial_data_async - pnx mismatch: %d != %d | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDeviceT::clear_zero - cudaMemset failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDeviceF::clear - CUDA error | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDeviceF::clear - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - cudaMemcpy failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - non specialized function called | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - cudaMemcpy failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - cudaMemcpy failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::linear_combination - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDeviceF::linear_combination - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::linear_combination - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::linear_combination - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDeviceF::linear_combination - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] Grid3DDevice::init - gpu buffer alloc failed | |
| APEX_DEBUG_WARNING | ERROR: calling base implementation of initCUDAFunction | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DHost::copy_interior_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DHost::copy_interior_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DHost::copy_interior_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::linear_combination - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid1DHost::linear_combination - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DHost::init - host buffer alloc failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DHost::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DHost::copy_all_data - cudaMemcpy failed | |
| APEX_DEBUG_WARNING | [ERROR] Grid3DHost::copy_all_data - mismatch: (%d, %d, %d) != (%d, %d, %d) | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3DDeviceF::invoke_kernel_calculate_relax - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3DDeviceF::invoke_kernel_calculate_residual - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3DDeviceF::invoke_kernel_restrict - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3DDeviceF::invoke_kernel_prolong - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::initialize_storage - garbage hx,hy,hz value %f %f %f | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::relax - failed at level %d | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::restrict_residuals - failed at level %d -> %d | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::prolong - failed at level %d -> %d | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::apply_boundary_conditions - failed at level %d | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::apply_boundary_conditions - failed at level %d | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3::initialize_storage - error in grid calculation | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::initialize_storage - rhs has invalid ghost cells (%d,%d,%d), must be >= 1 | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::initialize_storage - rhs dimension mismatch (%d,%d,%d) != (%d,%d,%d) | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::initialize_storage - could not initialize _r_grid[0] | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::initialize_storage - could not initialize _u_grid[0] | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::initialize_storage - could not initialize _u_grid[%d] | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::initialize_storage - could not initialize _b_grid[%d] | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::initialize_storage - could not initialize _r_grid[%d] | |
| APEX_DEBUG_WARNING | [ERROR] MultigridPressure3D::initialize_storage - grid dimension mismatch | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DBase::convert_bc_to_poisson_eqn - invalid boundary condition type %d | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DBase::initialize_base_storage - u dimensions mismatch | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DBase::initialize_base_storage - v dimensions mismatch | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DBase::initialize_base_storage - v dimensions mismatch | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DBase::initialize_base_storage - u,v,w layout mismatch | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::initialize_base_storage - must have at least one ghost cell on all sides | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DBase::initialize_base_storage - garbage hx,hy,hz value | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::calculate_divergence - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::subtract_grad_p - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::solve - could not enforce boundary conditions | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::solve - could not calculate divergence | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::solve - could not solve for pressure | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::solve - could not subtract gradient of pressure | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::solve - could not enforce boundary conditions | |
| APEX_DEBUG_WARNING | Max divergence: before = %.010f, after = %.010f (reduction %fx) | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::initialize_storage - failed to initialize base storage | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::initialize_storage - failed to initialize divergence | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::initialize_storage - divergence layout mismatch | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::initialize_storage - failed to initialize pressure_solver | |
| APEX_DEBUG_WARNING | [ERROR] ProjectDivergence3DDevice::initialize_storage - pressure layout mismatch | |
| APEX_DEBUG_WARNING | ERROR: calling base implementation of initCUDAFunction | |
| APEX_DEBUG_WARNING | [ERROR] sample_points_3d - position arrays do not have matching sizes | |
| APEX_DEBUG_WARNING | [ERROR] sample_points_3d - CUDA error | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] sample_points_3d - position arrays do not have matching sizes | |
| APEX_DEBUG_WARNING | [ERROR] sample_weighted_points_3d - CUDA error | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] calculate_velocity_weights - CUDA error “%s” | A CUDA error occurred. |
| APEX_DEBUG_WARNING | [ERROR] sample_points_mac_grid_3d - failed on u sampling | |
| APEX_DEBUG_WARNING | [ERROR] sample_points_mac_grid_3d - failed on v sampling | |
| APEX_DEBUG_WARNING | [ERROR] sample_points_mac_grid_3d - failed on w sampling | |
| APEX_DEBUG_WARNING | [ERROR] sample_points_minmax_3d - position arrays do not have matching sizes | |
| APEX_DEBUG_WARNING | [ERROR] sample_points_minmax_3d - CUDA error “%s” | A CUDA error occurred. |