Raycasting

  1. Raycasting in general
  2. Raycasting in Voreen
  3. Example: rc_simple.frag

Raycasting in general

Raycasting is a technique to visualize volume data at interactive frame rates. For every pixel of the image plane a ray is traced through the scene starting from the viewer. The ray equation is given by a starting point and a direction. If the ray hits the volume the color of the pixel is calculated by sampling the data values of the ray at a finite number of positions in the volume. On each sample the transfer function is applied and composited with accumulated values of the ray. There are several acceleration techniques to speed up raycasting e.g. early ray termination and empty space skipping.1


Raycasting in general

Fig. 1: Raycasting in general


In pseudocode raycasting can be described as following:

Lookup volume entry position
Lookup volume exit position
Compute ray of sight direction
While in volume
    Lookup data value at ray position
    Apply transfer function to data value
    Accumulate color and opacity
    Advance along ray

With the advent of ShaderModel 3.0 it is possible to use dynamic branches and loops
in fragment and vertex programs. Therefore raycasting can now be performed on the GPU. For that we need to pass Entry- and Exitparams to the relevant fragment program. We use two float-textures, where the color value encodes the entry and respectively the exit points. To generate these textures we render a cube with an edge length of 1 (from (0,0,0) to (1,1,1)) and set the vertex color to the coordinates. OpenGL will interpolate the color values between the vertices automatically. For the entry points texture we enable back face culling and respectively front face culling for the exit points texture.

Figure 2 shows an example of 2 textures.


Entry-Exit-Textures

Fig. 2: Entry-Exit-Textures

Raycasting in Voreen


First some general information about the pipeline architecture in Voreen. All classes that render something are derived from the class Renderer, especially every raycaster is subclass of Renderer. To manage all the different render-targets we have a class named TextureContainer. In this class all render-targets have fixed identifiers, e.g. targetTypeImage or targetTypeExitParams. If you want to render into the framebuffer you have to render to the finalTarget in the TextureContainer or use the CopyToScreenRenderer. You can also combine several renderer with the two classes PipelineRenderer and MultiRenderer. Messages to a raycaster can be send by calling processMessage(Message* msg). For further details see the documentation of Message class.

The general process of raycasting in Voreen is as follows:
  1. Create a raycasting-object (usually with the RendererFactory)

  2. Call initializeGL()-method

  3. Call the render method of the raycasting-object
    1. Entry- and Exit-Textures were created (create method in EntryExitParams)

    2. Textures were bound through glActiveTexture and glBindTexture

    3. pass Uniforms to the shader

    4. Render quad from (-1, -1) to (1, 1) (Viewport)
Further details to the points 3.2. – 3.4. can be found in the following code of the SimpleRaycaster::render()-method.

int entryParams = portMapping->getTarget("image.entryparams");
int exitParams = portMapping->getTarget("image.exitparams");

if (!volumeContainer_) {
   tc_->releaseTarget(entryParams);
   tc_->releaseTarget(exitParams);
   return;
}
int dest = portMapping->getTarget("image.output");

tc_->setActiveTarget(dest,"SimpleRaycaster::image.output");
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT);
// compile program
if (firstPass_) {
  compileShader();
  LGL_ERROR;
}

// bind entry params
glActiveTexture(tm_.getGLTexUnit(entryParamsTexUnit_));
glBindTexture(tc_->getGLTexTarget(entryParams), tc_->getGLTexID(entryParams));
glActiveTexture(tm_.getGLTexUnit(entryParamsDepthTexUnit_));
glBindTexture(tc_->getGLDepthTexTarget(entryParams), tc_->getGLDepthTexID(entryParams));
LGL_ERROR;

// bind exit params
glActiveTexture(tm_.getGLTexUnit(exitParamsTexUnit_));
glBindTexture(tc_->getGLTexTarget(exitParams), tc_->getGLTexID(exitParams));
glActiveTexture(tm_.getGLTexUnit(exitParamsDepthTexUnit_));
glBindTexture(tc_->getGLDepthTexTarget(exitParams), tc_->getGLDepthTexID(exitParams));
LGL_ERROR;

// vector containing the volume textures to bind; is passed to bindVolumeTextures();
std::vector<VolumeTextureStruct> volumeTextures;

// add main volume
volumeTextures.push_back(VolumeTextureStruct(
     getCurrentDataset()->getTexture(),
     volTexUnit_,
     "volume_",
     "volumeParameters_")
);

// bind transfer function
glActiveTexture(tm_.getGLTexUnit(transferTexUnit_));
if (transferFunc_.get())
    transferFunc_.get()->bind();

// initialize shader
raycastPrg_->activate();

if (firstPass_) {
   // set common uniforms used by all shaders
   setGlobalShaderParameters(raycastPrg_);
   // bind the volumes and pass the necessary information to the shader
   bindVolumeTextures(raycastPrg_, volumeTextures);
   // pass the remaining uniforms to the shader
   raycastPrg_->setUniform("entryParams_", (GLint) tm_.getTexUnit(entryParamsTexUnit_));
   raycastPrg_->setUniform("entryParamsDepth_",
                                      (GLint) tm_.getTexUnit(entryParamsDepthTexUnit_));
   raycastPrg_->setUniform("exitParams_", (GLint) tm_.getTexUnit(exitParamsTexUnit_));
   raycastPrg_->setUniform("exitParamsDepth_",
                                      (GLint) tm_.getTexUnit(exitParamsDepthTexUnit_));
   raycastPrg_->setUniform("lowerThreshold_", lowerTH_.get());
   raycastPrg_->setUniform("upperThreshold_", upperTH_.get());
   raycastPrg_->setUniform("transferFunc_", (GLint) tm_.getTexUnit(transferTexUnit_));

}

renderQuad();

raycastPrg_->deactivate();
glActiveTexture(TexUnitMapper::getGLTexUnitFromInt(0));
LGL_ERROR;
  • Line 1 and 2: get number of render target for Entry- and ExitTextures from TextureContainer
  • Line 9 to 11: the output render target is allocated and activated
  • Line 12: color and depth buffer are cleared
  • Line 12 to 17: the attached shader is compiled (if needed)
  • Line 16, 24, 31 and 74: for error handling purposes only
  • Line 20 and 22: texture unit for entry parameters is activated
  • Line 21 and 23: texture for entry parameters is bound
  • Line 27 and 29: texture unit for exit parameters is activated
  • Line 28 and 30: texture for exit parameters is bound
  • Line 34: texture unit for volume dataset is activated and bound
  • Line 44-48: if a transfer function is defined for this raycaster, the appropriate texture unit is activated and the texture is bound
  • Line 50: shader program is activated
  • Line 53-66: the needed uniforms are passed to the shader
  • Line 70: a quad from (-1, -1) to (1, 1) is rendered (viewport) — thereby the fragment program is executed which performs the raycasting
  • Line 72: deactivation of shader program
  • Line 73: active texture unit is set to 0

Example code – rc_simple.frag


rc_simple.frag
uniform sampler2D backTex_;
uniform sampler2D frontTex_;
uniform sampler3D volume_;
uniform sampler1D transferFunc_;
uniform float lowerThreshold_;
uniform float upperThreshold_;
uniform vec2 size_;
const float Samplings = 250.0;
 	
vec4 directRendering(in vec3 first, in vec3 last, in float threshold, in float upperThreshold) {
    vec3 direction = last - first;
    int steps = int(floor(Samplings * length(direction)));
    vec3 diff1 = direction / float(steps);
    vec4 value;
    vec4 result = vec4(0.0);
    float scale = 1.0/(upperThreshold - threshold);
    for (int i=0; i<steps; i++) {
        value = texture3D(volume_, first);
        first += diff1;
        if (value.a >= threshold && value.a <=upperThreshold) {
            value.a = (value.a-threshold)*scale;
            vec4 texVal = texture1D(transferFunc_, value.a);
            result.rgb += (1.0-result.a)*texVal.a*texVal.rgb;
            result.a += (1.0-result.a)*texVal.a;
            if (result.a >= 0.9) {
                i = steps;
                result.a = 1.0;
            }
        }
    }
    return result;
}
	
void main() {
    vec4 fragCoord = vec4(gl_FragCoord.xy/size_, 0.0, 0.0);
    vec3 frontPos = texture2D(frontTex_, fragCoord.xy).rgb;
    vec3 backPos = texture2D(backTex_, fragCoord.xy).rgb;
    gl_FragColor = directRendering(frontPos, backPos, lowerThreshold_, upperThreshold_);
}
  • from Line 1 to 8 constants and variables are declared
  • Line 35: texture coordinates are calculated relative to window coordinate position
  • Line 36 and 37: texture coordinates are used to lookup entry- and exit-params
  • Line 38: call of raycast-method directRendering
  • Line 11: direction for the ray is computed
  • Line 12: number of steps inside the volume calculated
  • Line 13: calculation of step width
  • Line 18: lookup of data value at actual ray position
  • Line 19: increase ray position by 1 step
  • Line 20: test if data value lies between threshold-constants
  • Line 21: scaling of data value
  • from Line 22 to 24 the transfer function is read, applied and value added to calculated values before
  • Line 25: test if accumulated values exceed the threshold (early ray termination)

References

1 J. Krüger, R. Westermann
Acceleration Techniques for GPU-based Volume Rendering – IEEE Visualization 2003


2 S. Stegmaier, M. Stengert, T. Klein, T. Ertl
A Simple and Flexible Volume Rendering Framework for Graphics-Hardware-based Raycasting in Proceedings of the International Workshop on Volume Graphics ’05, pages 187-195, 2005