Tips/tricks related to Computer Graphics, GPUs and other programming

NOTE: This tutorial builds up on my previous 2 tutorials: how to setup OpenGL ES 2.0 on Android and how to render to texture.

———————————————————————————————————-
I’ll first link to the apk, source code and the repository:
apk here.
Source code here.
Google Code Repository  here. NOTE: Branch is “shadowMapping”
———————————————————————————————————–

UPDATE:

Thanks to Henry’s comment below I was able to fix most of the artifacts showing up. Turns out I had dithering enabled, so I had to disable it to remove the dithering effect [ GLES20.glDisable(GLES20.GL_DITHER) ]. Updated images below – it works but has very slight artifacts when it comes to the mesh’s self-shadows.

Introduction:

Shadow Mapping is a popular technique for rendering shadows in games and other real-time applications. Given how GPUs on mobile devices (iPhone/Android/whatever) are getting more and more powerful every day I thought I might try implementing it on my Samsung Captivate (Galaxy S).

While my implementation “works”, it does not good look at all. Shadow-mapping is prone to many artifacts , which I wasn’t able to properly fix (UPDATE: Main artifacts fixed).I am posting this without the fix in the hopes that somebody can help me out too as to what I am doing wrong.

Here are the updated images (I changed the color of the plane to get a better contrast):

Textured Cube (with proper shadows)

Dragon Head

Octahedron


Here are the old images:

Octahedron Mesh (.OFF) with shadows + artifacts

Dragon-head Mesh (.OFF)

Textured Cube (.OBJ)

I would like to credit Fabien Sanglard for his shadow mapping tutorials which I used as a basis for my code. I looked at his DEngine code as an example. In any case let’s begin.

What is Shadow Mapping?

If you are reading this chances are that you already know what shadow mapping is and how it works, but let me give a quick overview of the steps involved:

  1. Generating the Shadow Map:The “shadow map” (aka a depth map) is just a texture which stores the depth of the scene from the viewpoint of the (directional) light. Scene is rendered from the light’s viewpoint into a texture.
  2. Rendering the Scene:The scene is rendered from the usual camera’s viewpoint. But along with the usual information the lightviewmatrix and the depth texture are passed in to the shader, and a texture projection is used to compare whether the current pixel being shaded is covered by a shadow or not.And that’s basically it! Shadow Mapping isn’t too complex, but it has a habit of producing a lot of nasty artifacts. So we will have to be careful when generating proper shadow maps, the matrices and doing the shading.

Implementation:

  1. The Scene:The scene is just composed of one mesh object (octahedron[.OFF], dragon’s head[.OFF] or textured cube[.OBJ]) above a plane (.OFF) with a rotating light. You can switch between the meshes and pause/resume the rotation of the light through the menus. I modified the phong shader to produce the shadows – the gouraud and normal mapping shaders function as before. To look at how all this is done please refer to my previous tutorials.
    NOTE: Shadow Mapping requires a “spotlight” to function – my shaders use “point lights”. To get shadows from point lights you have to render from multiple views around the light (cube map generation), so I just treat my point light as a spotlight for generating shadows, and render the rest of the non-shadowed scene as if lit by a point light.
  2. Generating the Shadow Map:To render the shadow map you have to render from the light’s viewpoint. The size of the depth map texture in my code is 512 X 512. You can view the depth map through the “View depth map” option in the menu. The rendering is done in the renderDepthToTexture()function:
    // Cull front faces for shadow generation
    // Culling front faces is suggested for removing artifacts, but it doesn't seem to
    // present a noticeable improvement
    GLES20.glCullFace(GLES20.GL_FRONT);// Bind the shadow map texture now...
    GLES20.glViewport(0, 0, this.texW, this.texH);// bind the generated framebuffer
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fb[0])
    
    // specify texture as color attachment
    GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, renderTex[0], 0);
    
    // attach render buffer as depth buffer
    GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_DEPTH_ATTACHMENT, GLES20.GL_RENDERBUFFER, depthRb[0]);
    
    // check status
    int status = GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER);
    if (status != GLES20.GL_FRAMEBUFFER_COMPLETE)
    return false;
    
    /*** DRAW ***/
    // Clear color and buffers
    GLES20.glClearColor(1.0f, 1.0f, 1.0f, 1.0f);
    GLES20.glClear( GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);
    
    // depth map shaders
    // I used a new shader to generate the depth map.
    Shader shader = _shaders[this.DEPTHMAP_SHADER];
    int _program = shader.get_program();
    
    // Start using the shader
    GLES20.glUseProgram(_program);
    checkGlError("glUseProgram");
    
    // View from the light's perspective
    Matrix.setLookAtM(lMVMatrix, 0, lightPos[0], lightPos[1], lightPos[2],
    lightPos[4], lightPos[5], lightPos[6],
    lightPos[7], lightPos[8], lightPos[9]);
    float ratio2 = (float)texW /texH;
    Matrix.frustumM(lProjMatrix, 0, -ratio2, ratio2, -1, 1, 1f, 5000f);
    
    // modelviewprojection matrix
    Matrix.multiplyMM(lMVPMatrix,0, lProjMatrix, 0, lMVMatrix, 0);
    
    // send to the shader
    GLES20.glUniformMatrix4fv(GLES20.glGetUniformLocation(_program, "uMVPMatrix"), 1, false, lMVPMatrix, 0);
    
    /// DRAW ALL THE OBJECTS
    drawAllObjects(_program, false, false);
    
    // render the depth buffer?
    if (viewDepthTex) {
    renderToQuad();
    return true;
    }
    
    /**** Else, render with shadow now --
    */
    renderWithShadow(lMVPMatrix);
    

    Hopefully the code above is self-explanatory. After we render the depth to the texture we can either view the depth texture or render the shadows. Next I am going to look at the depth map pixel shader which generates the depth map and then the “renderWithShadow” function.

    The depth map pixel shader is written below. We basically just render the depth into a RGB texture. There might be better ways to do this, but this gave me good results.

    // Pixel shader to generate the Depth Map
    // Used for shadow mapping - generates depth map from the light's viewpoint precision highp float;
    
    varying vec4 position; 
    
    // taken from Fabien Sangalard's DEngine - this is used to pack the depth into a 32-bit texture (GL_RGBA)
    vec4 pack (float depth)
    {
    	const vec4 bitSh = vec4(256.0 * 256.0 * 256.0,
    					   256.0 * 256.0,
    					   256.0,
    					  1.0);
    	const vec4 bitMsk = vec4(0,
    					     1.0 / 256.0,
    					     1.0 / 256.0,
    				             1.0 / 256.0);
    	vec4 comp = fract(depth * bitSh);
    	comp -= comp.xxyz * bitMsk;
    	return comp;
    }
    
    void main() {
    	// the depth
    	float normalizedDistance  = position.z / position.w;
            // scale it from 0-1
    	normalizedDistance = (normalizedDistance + 1.0) / 2.0;
    	
    	// bias (to remove artifacts)
    	normalizedDistance += 0.0005;
    
    	// pack value into 32-bit RGBA texture
    	gl_FragColor = pack(normalizedDistance);
    	
    }
    
  3. Rendering the Scene with Shadows:Now we have our depth texture. All we need to do is render the scene from the camera’s viewpoint. The only new variable we are passing in this time into the phong shader is the modelview matrix from the light’s viewpoint. The rendering is done in the “renderWithShadow(lMVPMatrix)” function. It works pretty much the same way as the rendering functions shown in the previous tutorials, so look at those first if you want to get an idea of how this works. The shadow map texture is attached as “shadowTexture”.We will modify our phong vertex shader slightly to accommodate the shadow projection matrix:
    // Vertex shader Phong Shading
    ...
    // the shadow projection matrix
    uniform mat4 shadowProjMatrix;
    varying vec4 shadowCoord;
    ...
    attribute vec4 aPosition;
    ...void main() {
    ...
    
    shadowCoord = shadowProjMatrix * aPosition;
    ...
    }
    

    We are going to use these shadow coordinates to perform a texture lookup in our pixel shader and use them for testing whether the pixel is in shadow or not:

    ...
    
    // This function looks up the shadow texture
    // and returns whether the pixel is in shadow or not
    float getShadowFactor(vec4 lightZ)
    {
    	vec4 packedZValue = texture2D(shadowTexture, lightZ.st);
    
    	// unpack the value stored to get the depth. 
    	const vec4 bitShifts = vec4(1.0 / (256.0 * 256.0 * 256.0),
    						1.0 / (256.0 * 256.0),
    						1.0 / 256.0,
    						1);
    	float shadow = dot(packedZValue , bitShifts);
    
    	return float(shadow > lightZ.z);
    }
    
    void main() {
    ...
    ...
    // ambient, diffuse and specular terms have been calculated
    
    // Shadow
    float sValue = 1.0; // for ambient + diffuse
    float sValue2 = 1.0; // for specular
    if (shadowCoord.w > 0.0) {
    // get the shadow coordinates for the texture
    vec4 lightZ = shadowCoord / 45.0;
    
    // scale + bias
    lightZ = (lightZ + 1.0) /2.0;
    lightZ.z += 0.0005;
    
    // get the shadow value
    sValue = getShadowFactor(lightZ);
    
    // scale the value from 0.5-1.0 to get a softer shadow
    float newMin = 0.5;
    float v1 = (1.0)/(1.0 - newMin);
    float v2 = sValue/v1;
    sValue2 = sValue + newMin;
    }
    
    gl_FragColor = ( (ambientTerm + diffuseTerm) * sValue2 + specularTerm * sValue) ;
    }
    

    The scene has been rendered. That’s pretty much all there is to it – the next step would be implementing percentage-closer filtering and other techniques to produce better shadows.

    Any comments and questions are welcome.

    ———————————————————————————————————-
    Links to the apk, source code and the repository:
    apk here.
    Source code here.
    Google Code Repository  here. NOTE: Branch is “shadowMapping”
    ———————————————————————————————————–

About these ads

Comments on: "Android OpenGL ES 2.0 – Shadow Mapping" (29)

  1. Henrik R said:

    Tried turning off dithering? Those artifacts are curiously similar to an ordered dither pattern.

    • Thank you for that comment! Turning off Dithering certainly helped a lot and eliminated most of the artifacts. There’s still some dark bands showing up..l’ll try to fix them.

      • I suggest you implement some sort of percentage closer filtering, then you won’t have the problem anymore:

        if (shadowCoord.w > 1.0) {

        float x = 0.0;
        float y = 0.0;
        for (y = -35.0 ; y <=35.0 ; y+=10.0)
        for (x = -35.0 ; x <=35.0 ; x+=10.0)
        shadow += lookup(vec2(x,y));

        shadow /= 64.0 ;
        }

        float lookup( vec2 offSet)
        {
        // Values are multiplied by ShadowCoord.w because shadow2DProj does a W division for us.
        return shadow2DProj(shadowMap, shadowCoord.w + vec4(offSet.x * 0.0001 * shadowCoord.w, offSet.y * 0.0001 * shadowCoord.w, 0.05, 0.0) ).w;
        }

      • Thanks for the comment. The dithering issue was fixed – as I said my next step would be to implement some kind of pcf. I’ll give your code a try and see what happens

      • I forgot to mention that for shadow2DProj to work you need to have a sampler2DShadow instead of a sampler2D. I used that code for an example a few years ago. You can find the results here: http://www.youtube.com/watch?v=2u4TPK4PYPs

    • The even better approach would be to use variance shadow mapping because PCF is simply too slow to be applicable for games for example.

      http://fabiensanglard.net/shadowmappingVSM/

    • Can you give me the kind of pcf code?

  2. Yes seems like PCF might be too slow. I wonder how mobile engines like Unity and the Unreal Engine implement shadows. I’ll look into VSM

    • I tried some things with your example and was able to provide quite good results with a few tricks. For example, I added a polygon offset (glPolygonOffset(1.1, 4.0); glEnable(GL_POLYGON_OFFSET_FILL);) to reduce the amount of artifacts. You already use front face culling to avoid self shadowing but what will probably enhance your shadow quality best is to simply increase the size of the depth map. For example use sizes that would be suitable for PC versions (1200 * 1080) for example. This helped a lot.

      If you like, I can also provide the modified shader code for the PCF. Unfortunately, PCF is too slow. I would like to see an implementation of the VSM, however, I do not have enough time of focusing on shader development at the moment.

      • Thanks for the polygon offset hint. I think that would definitely help. And yes increasing the depth map size is an obvious one – but 1200 X 1080 would be way too big for any kind of mobile application.

        If you can post a link to the PCF version I would appreciate it (and anyone else reading this!). I would like to give VSM a try but unfortunately a little busy with other stuff nowadays.

  3. Is packing the depth into a RGBA texture really necessary? I thought modern ogl es 2.0 allowed you to render to a depth-only FBO.

  4. matti777ti said:

    Though the funny thing here is that GL_DEPTH_COMPONENT is not listed among the choices for a OpenGL ES 2.0 texture types (http://developer.meego.com/api/1.2/opengles-2.0/glTexImage2D.html) but such code does compile for MeeGo Harmattan 1.2 at least… :o Also checked the PowerVR specs and they state the same. Going to try to get your stuff working first, then gonna try the actual depth texture. :)

    Good job btw, havent seen many pieces of shadow mapping code for modern OpenGL around!

    • Hmm that is odd indeed.

      And thanks for the support! Have to give most of the credit to Fabien Sanglard for his shadow mapping tutorial and iOS ES 2.0 code.

  5. Well, yes, I’m aware of his site, but your code is way more readable even if it is Java. ;)

    One question: you’re not multiplying with the “bias” matrix as far as I can see. Is this because of the (normalizedDepth +1.0) / 2.0 in the shadow renderer?

    • That takes care of the conversion from -1 to 1 to 0 to 1. But I am adding a certain bias to the normalized distance (0.0005) which is the same thing as multiplying with the bias matrix I think. I played around with the value a bit to see what looks sufficient.

  6. matti777ti said:

    As far as I understand the concept, the job of the bias matrix is to transform the coordinates from the unit cube [-1,1] space to [0,1] so they can be used as texture u,v indices to the shadow map?

    • Yes the bias matrix helps in scaling from -1,1 to 0,1. But it’s not so that it can be used as u,v texture lookups, rather so that we can store the actual depth from 0-1 in a texture.

  7. matti777ti said:

    Ok got it to work. Some artifacts though, using the glCullFace(GL_FRONT) somehow fooks up the shadows a bit and objects crossing the edge of the shadow map cause an ugly glitch but looking into it.. the shadowing shader destroys the performance on the device though, went from 60+ FPS to 17-19. Disabling rendering of the shadow map causes very minor effect, so it’s the actual shadowing shader to blame. One reason more to look into using a actual shadow texture & shadow2DProj(), think they might be more optimized.

    Again, thanks for publishing all this.

    • yeah the actual shadowing is quite slow – but what device are you using? Mine is definitely not running at 17-19fps (though I haven’t looked at actual numbers).

      I used texture2D instead of shadow2Dproj – probably should give shadow2dproj a try.

  8. The device is Nokia N9 which has roughly the same hardware than 1-year old iPhone(4) / Android – PowerVR SGX530 GPU. I’ll look into this performance issue after work today, there is probably some stupid thing causing this, I’d assume I could get away with a performance drop of like 15% with the shadows, instead of this 75% I’m experiencing now :)

    If I get the depth texture/shadow2DProj() thing working, I’ll post the results!

  9. matti777ti said:

    Finally got the depth texture version working. Some essential code included here: http://developer.qt.nokia.com/forums/viewthread/11734/#66487

  10. […] shadowmapshttp://blog.shayanja…shadow-mapping/ […]

  11. Thanks for this. I know this is old, and I wonder now in 2014 if this is quite fast enough. 2 small bugs I noticed (caused crash)

    // light variables
    float[] lightC = {0.5f, 0.5f, 0.5f,0.0f}; // 4th float needed as uniform is vec4

    float[] ma2 = {1.0f, 215f/255f, 0.0f,0.0f}; // 4th float needed as uniform is vec4

    Dies on this:
    GLES20.glUniform4fv(GLES20.glGetUniformLocation(_program, “matAmbient”), 1, matAmbient2, 0);

  12. how can i get the source? i could not open the link。

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: