http://www.altdevblogaday.com/2012/04/14/software-rasterizer-part-1/

Anyone interested in graphics should have some kind understanding as to how the graphics pipeline works and how the rasterizer works. Knowledge of math (especially linear algebra) helps greatly.

Now instead of downloading BezierView to your own computer and running a .exe file, you can just run it online through a webpage. You can find the still-being-updated version here. For now you can view tensor-product, triangular and rational patches along with polyhedra. It is being currently updated to include a better interface and to have support for all types.

There are some shortcomings to writing BezierView in WebGL – the most obvious one is that Javascript is much slower compared to C/C++. And the online version cannot handle huge files which the offline version can. And since WebGL is based on OpenGL ES 2.0, we cannot take advantage of the tessellator engine in modern graphics cards to evaluate parametric patches on the GPU (which results in a significant speedup). So all the evaluation is done on the CPU and the resultant vertices sent to the GPU. I guess we’ll have to wait for OpenGL ES 4.0 (or WebCL to be finalized – a Javascript implementation of OpenCL).

BezierView was developed using the excellent Three.js library. I highly recommend using it if you are developing a WebGL application.

The source code is available on GitHub.

You can find BezierView here.

1. Using ARToolkit to demonstrate how touch-screens work:  

   We had to come up with a way to use ARToolkit (Augmented Reality). I thought of attaching some AR markers to a Nintendo DS and demonstrating how 3 different kinds of touch-screens work (resistive, capacitive and infrared). Written in C

2. Virtual Reality experience: Rescuing someone from a fire:

   Now we started getting into the heavy stuff. We had to create a Virtual Reality 3D experience using a Head-Mounted Display, object tracking and small space. At the suggestion of my teammate, colleague and friend we ended up using the Ogre 3D Engine for the project. We also used Google SketchUp models and wrote the program in C++.

3. VR experience: Target practice on a military base:

   This was my favorite project. We (me and my colleague Aritra Nath) created “virtual” shooting, where we attached a marker to a Nintendo Wiimote (which was in a gun shell, and you pressed the trigger to shoot). We had an actual barrier in the real-world which was nearly pixel-accurate in the virtual world (well, not quite). Used Ogreand Sketchup again.

4. New VR Interface for doing 3D landscaping around a house:

   The 4th project involved creating a VR experience where the user could “create” new content. My team ended up creating a way to do 3D landscaping only using fingers and a wiimote. Used the same tools as before.

I consider myself lucky that I was able to take such a course. Though it was kind of grueling at times (especially since all the teams had to share the virtual reality hardware), it was also fun and rewarding at the end.

You can access the file here. I have only tested this for integers, so might need a little bit of work for other data types. The following methods have been implemented:

• add(data)
• remove(data)

Traversals:

• preorder()
• inorder()
• postorder()
• bfTraversal() (Breadth-first traversal)

Helper methods:

• isLeaf(node) – checks if a node is a leaf
• lcAncestor(value1, value2) – finds the least common ancestor of two nodes with values value1 and value2
• printAllPaths() – prints all the possible paths from the root to the leaves

I won’t go over how a BST works, but hopefully the code is self-explanatory.

If there are any questions let me know – hope people find it useful.

I’ve been meaning to get into OpenCL for a while, so I thought I would write a simple program which would evaluate Bicubic Bezier patches (surfaces) in parallel.
I also wanted to program in Python more, so I decided to use PyOpenCL. You can find more info about the OpenCL implementation in Python from the link.

I am not going to go over what a Bezier patch is and how it can be evaluated. There is a straightforward equation for evaluating one which I will be showing later on, but for more info about Bezier patches you can find other info online.

Let’s begin.

Introduction:

If you have a surface made of multiple Bezier patches they can be evaluated in parallel. The new shaders in DirectX 11 (Hull and Domain shaders) make this easier for graphics applications and rendering. There is a straightforward method of evaluating these patches using a simple equation. I am going to demonstrate the simple, straightforward way first (method2-bb.py) and then another way which I thought would work faster (simple-bb.py).

What you’ll need:

• Python (Preferably 2.7)
• NumPy
• PyOpenCL
• A Graphics card which supports OpenCL. Any recent card should work fine (I used an old Nvidia 8800 GT).

OpenCL:

This is not intended to be an OpenCL tutorial, but I will be going over what I did in PyOpenCL (getting the context, setting up the input buffers, writing the kernels, getting the output, etc.) For a more detailed look into OpenCL I would suggest OpenCL tutorials by Nvidia.
I will not be going over in detail what a work-item is, what a work-group is, etc.

Quick note for CUDA users: For some reason CUDA and OpenCL use different terminology. I will be using OpenCL terminology throughout this post. So here’s a quick translation for CUDA users:

• CUDA thread = OpenCL work-item
• CUDA block = OpenCL work-group
• CUDA shared memory = OpenCL local memory
• CUDA local memory = OpenCL private memory

Input:

The input to both programs is a BezierView file with the control point coordinates of all the patches. BezierView is a program developed by my graduate research lab (SurfLab) for the purposes of rendering different kinds of surfaces and viewing their properties (like Gaussian curvature, highlight lines, etc.)

Here is a sample file: cube2.bv.

The input method is called readBezierFile and returns a 1-D list with all the vertices in it (16 vertices per patch).

NumPy:

PyOpenCL requires the use of NumPy arrays as input buffers to the OpenCL kernels.

NumPy is a python module used for scientific computing, and it allows better creation of multi-dimensional arrays which are widely used for OpenCL. I won’t go over in much detail about what NumPy is, but I will show you how I use it.

First, we need to convert the regular Python list with all the vertices in it to a NumPy array:

import numpy

...

# try to read from a file here - returns the array
vertices = readBezierFile("cube2.bv")

# create numpy array
npVertices = array( vertices, dtype = float32)

The function “array” converts a Python list to a NumPy array. The “dtype” represents the type of values we will be storing in the array (32-bit floating numbers in this case).

Initial Setup:

I read the file with all the OpenCL kernels in it first (bezier.cl)

After that I setup the UV-buffer. Evaluating a Bicubic patch requires U and V values. The more UV-value pairs there are the more detailed surface is output (from 0.0 to 1.0). The UV-value generation code is:

# Array of UV values - 36 in total (detail by 0.2)
uvValues = empty( (36, 2)).astype(numpy.float32)

index = 0
for u in range(0, 12, 2): # step = 2
    for v in range(0, 12, 2):
        # conver the ints to floats
        fU = float(u)
        fV = float(v)
        uvValues[index] = [ fU/10.0, fV/10.0]
        index = index + 1

The “empty” function is used for creating a NumPy array with empty values. We are going to be generating an array of size 36 * 2. (UV values spaced by 0.2)

OpenCL Setup:

Before we do computation we have to do some OpenCL setup. This can be a little tedious when writing in C/C++, but PyOpenCL makes it much easier.

First, we have to create an OpenCL Context. This can be done in two ways (I used the second one for my clarification):

1. This simple way will automatically choose a device for you (usually the GPU if it’s available). Very convenient.
   

   ctx = cl.create_some_context()
   

2. This way goes through all the platforms/devices and creates a context specifically with that device.
   

   # Platform test
   for found_platform in cl.get_platforms():
       if found_platform.name == 'NVIDIA CUDA':
           my_platform = found_platform
           print "Selected platform:", my_platform.name
   
   for device in my_platform.get_devices():
     dev_type = cl.device_type.to_string(device.type)
     if dev_type == 'GPU':
           dev = device
           print "Selected device: ", dev_type
   
   # context
   ctx = cl.Context([dev])
   

Next we need to create the Command Queue:

cq = cl.CommandQueue(ctx, properties=cl.command_queue_properties.PROFILING_ENABLE)

You can just pass in the Context to the function, but I want to point out that I passed in the PROFILING_ENABLE flag, which allows us to determine how much time the operation took. This will allow us to compare the different methods.

Now finally we setup the input/output buffers to be sent to the GPU:

# memory flags
mf = cl.mem_flags

# input buffers
# the control point vertices
vertex_buffer = cl.Buffer(ctx, mf.READ_ONLY | mf.COPY_HOST_PTR, hostbuf=npVertices)

# the uv buffer
uv_buffer = cl.Buffer(ctx, mf.READ_ONLY | mf.COPY_HOST_PTR, hostbuf=uvValues)

# final output
output_buffer = cl.Buffer(ctx, mf.WRITE_ONLY, uvValues.nbytes * 2 * numPatches)

The output buffer is to be written to (WRITE_ONLY), and we specify the size of the output buffer in terms of bytes. the “nbytes” property in a NumPy array lets us know the size in terms of bytes of that array.
For each patch we will have 36 * 4 * 4 bytes.
36 = number of UV pairs
4 = number of floating-point values output for each UV-value
4 = number of bytes for a float32 value
Hopefully that is clear.

Now we can start evaluating!

Method #1: The simple, straightforward way:

The first method directly evaluates the patch for 1 UV-pair. The kernel takes in the UV-pair, reads in the control points for the corresponding patch and then outputs it to the output buffer. The Python evaluation code is as follows:

# the global size (number of uv-values * number of patches): 36 * numPatches
numUVs = uvValues.size/2 # 72/2 = 36
globalSize = numUVs * numPatches

localSize = numUVs # 36 work-items per work-group

# evaluate
exec_evt = prg.bezierEval2Multiple(cq, (globalSize,), (localSize,), vertex_buffer, uv_buffer, output_buffer)
exec_evt.wait()

First we calculate the global memory size (total number of kernels needed), and then the local memory size (# of work-items per work-group). The global memory size is basically the number of UV-value pairs.

We will need this information to send to the evaluation kernel. The kernel name is “bezierEval2Multiple” (apologies for the incoherent names).
The kernel needs to take in the Command Queue, the global memory size and the local memory size first.

After that we pass in the input and output buffers as parameters. Once the kernel is called we wait for it to finish (exec_evt.wait()).

The kernel is as follows (in bezier.cl):

__kernel void bezierEval2Multiple(__global const float4 *controlPoints, __global float2 *uvValues,
                        __global float4 *output) {

   // get the global id and the corresponding patch number
   int gid = get_global_id(0);

   // get the patch number - the patch info we will have to read
   int numPatch = gid/get_local_size(0);

   // get the uv values - local id goes from 0-35
   int lid = get_local_id(0);
   float2 uv = uvValues[lid];

   // evaluate row1
   float4 b00 = evalCubicCurve(controlPoints[numPatch * 16 + 0], controlPoints[numPatch * 16 + 1], controlPoints[numPatch * 16 + 2], controlPoints[numPatch * 16 + 3], uv.x);

   float4 b01 = evalCubicCurve(controlPoints[numPatch * 16 + 4], controlPoints[numPatch * 16 + 5], controlPoints[numPatch * 16 + 6], controlPoints[numPatch * 16 + 7], uv.x);

   float4 b02 = evalCubicCurve(controlPoints[numPatch * 16 + 8], controlPoints[numPatch * 16 + 9], controlPoints[numPatch * 16 + 10], controlPoints[numPatch * 16 + 11], uv.x);

   float4 b03 = evalCubicCurve(controlPoints[numPatch * 16 + 12], controlPoints[numPatch * 16 + 13], controlPoints[numPatch * 16 + 14], controlPoints[numPatch * 16 + 15], uv.x);

  // evaluated point
  output[gid] = evalCubicCurve(b00, b01, b02, b03, uv.y);
}

For each kernel we need to figure out which Bezier patch we are going to evaluate, what UV-pair we are going to use and where we need to place our output in the output buffer.

The global id let’s us know where we need to place our output.
To get the patch we just divide our global id by the local memory size.
The UV-pair we want to evaluate is our local id. The local memory size is 36, which is the # of UV-value pairs, so we can easily find out which one we need to use.

Finally we evaluate the Bezier patch and place it in the output buffer. A simple kernel.

Method #2: The complex, trying-to-be-more-parallel way:

The “problem” I noticed with the first kernel is how all the work-items in each work-group have to read the same control points from global memory again and again. At this point I was still iffy on the concept of “local memory” so I read up on it and found out that reading from local memory is faster than global memory (obviously). So will using local memory allow me to make the whole process faster? I decided to come up with a different way which will use a lot more work-items but where each work-item does “less” work (only simple bilinear interpolation across selective control points).

This method did not turn out to be faster. My advisor already told me before I fully implemented this that the first method will be much faster, and he was right. Turns out it was 3 times slower. In any case here is the concept:

Bezier Patches can be evaluated using what is known as De Casteljau’s Algorithm (The link is for curves but can be easily adapted to surfaces). It basically involves linearly interpolating (bilinearly interpolating for surfaces) the control points and their results until you get the evaluated point. I try to parallelize the interpolation part so it can be done faster.

Here are a few images showing the steps for a UV-pair of (0.5, 0.5):

First, here are the 16 control points of a patch:

16 Bicubic Control Points

The next step would be to do a bilinear interpolation across each “square” (4) of control points per kernel. The results are the blue points shown below: (since U and V are both 0.5, the point lies in the middle)

First bilinear interpolation pass (results in biquadratic patch)

We repeat the process for this biquadratic patch: (results are points in red)

Second interpolation pass (results in linear patch)

Finally we do one last bilinear interpolation to get the evaluated point (in green):

The point at (U,V) = (0.5, 0.5)

You have probably figured it out by now, but the way this method works in parallel is that it generates 9 work-items per UV-pair. Each work-item does a bilinear interpolation and stores it’s result in local memory. This completes the first pass.

After the first pass we need to repeat the process, but this time we only need 4 work-items. This is where I encounter a problem: the other 5 work-items don’t do anything. Then for the final evaluation I only use 1 work-item, with others not doing work.

The kernel (bezierEvalMutiple) shown below:

__kernel void bezierEvalMultiple(int numPatches, __global const float4 *controlPoints, __global float2 *uvValues,
                     __global float4 *output, __local float4 *local_points, __local float4 *local_points2) {

	// get the global id
	int gid = get_global_id(0)/get_local_size(0); // divide by 9
	
	// get the patch number
	int numPatch = gid/36; //get_local_size(0);
	
	// get the uv values 
	float2 uv = uvValues[gid];//get_global_id(0)];
	
	//output[gid] = (float4)(numPatch, numPatch, 0, 0);
	
	// get the row
	int lid = get_local_id(0);
	int patchNum = numPatch * 16; 	// the patch to deal with
	int row = lid/3;
	int index = row + numPatch + lid;
	
	// get the 4 control points you'll need
	/****************
	  b10--b11
	   |   |
	  b00--b01
	****************/
	
	float4 b00 = controlPoints[index];
	float4 b01 = controlPoints[index+1];
	float4 b10 = controlPoints[index+4];
	float4 b11 = controlPoints[index+5];
	
	// do linear interpolation across u first
	float4 val1 = lerp(b00, b01, uv.x);
	float4 val2 = lerp(b10, b11, uv.x);
	
	// then across v
	float4 newPoint = lerp(val1, val2, uv.y);
	
	// store it in the local memory
	local_points[lid] = newPoint;
	
	// synchronize in work-group
	barrier(CLK_LOCAL_MEM_FENCE);
	
	// only do it for certain work-items (now evaluating quadratic)
	if (lid < 4) {
		row = lid/2; 		// 2 = degree
		float4 b002 = local_points[row + lid];
		float4 b012 = local_points[row + lid+1];
		float4 b102 = local_points[row + lid+3];
		float4 b112 = local_points[row + lid+4];
		
		// do linear interpolation across u first
		float4 val12 = lerp(b002, b012, uv.x);
		float4 val22 = lerp(b102, b112, uv.x);
	
		// then across v
		float4 newPoint2 = lerp(val12, val22, uv.y);
		
		local_points2[lid] = newPoint2;
		barrier(CLK_LOCAL_MEM_FENCE);
		
		// output - only do it for the first work-item
		if (lid == 1) {
			float4 val13 = lerp(local_points2[0], local_points2[1], uv.x);
			float4 val23 = lerp(local_points2[2], local_points2[3], uv.x);
			
			float4 newPoint3 = lerp(val13, val23, uv.y);
			output[gid] = newPoint3;
		}
	}
}

I won’t go over much of this (you can probably figure it out), but I just wanted to mention how I synchronize between work-items (in a work-group) at each pass. You have to call barrier(CLK_LOCAL_MEM_FENCE); to ensure synchronization at each point.

This method runs 3 times slower – I would guess mainly because it requires 9 times more work-items and just straightforward multiplication is fast enough. At the very least I learned how to use local memory and synchronize between work-items.

Output:

In the end we want to find out the time elapsed and read back the output. This is how it’s done in Python:

elapsed = exec_evt.profile.end - exec_evt.profile.start
print("Execution time of test: %g " % elapsed)

# read back the result
eval = empty( numUVs * 4 * numPatches).astype(numpy.float32)
cl.enqueue_read_buffer(cq, output_buffer, eval).wait()

We create an empty NumPy buffer and then call enqueue_read_buffer to read from the output buffer into the array.

Conclusion:

I showed a simple example of using (Py)OpenCL for evaluating Bezier patches. The next step would be to render them using PyOpenGL, and perhaps allow user manipulation of the control points. This project helped me get a start on OpenCL and learn some of the basics. I would recommend using PyOpenCL since it allows you to write applications quickly.

If you have any questions, comments or suggestions please feel free to ask.

———————————————————–
GitHub Repository here
———————————————————–

Late last year a friend of mine brought to my attention a problem he encountered: he wrote a bunch of text in a textbox and when he clicked submit the server responded with an error. He had lost all he wrote which was frustrating. I thought it would be useful to have an extension which records everything you type in one of those boxes so you can recover it if you lose it.

Another friend suggested it would be nice if the extension kept a history of all your posts (if you are one of those people who comment across many sites a lot). And so I came up with “Comment Save” (which has roughly 500 users as I type this).I’ll let the description describe it:

As featured on Lifehacker!

http://lifehacker.com/5715657/comment-save-keeps-track-of-your-comments-on-the-web

Ever written a large post or comment on a website, click submit, only to see your comment get lost in the internet wilderness? Or your tab/window closed somehow before you could submit?

Or maybe you just like to keep track of all the posts you made on different websites.

Comment Save (CS) helps you keep track of what you have been writing as you type. Icon beside the address bar brings up a window showing you the last post you wrote alongwith a link to the page. Link in the window allows you to view your post history.

You can also filter websites where you don’t want any of your posts to be tracked.

Works on sites like:
- Youtube
- Facebook/Google+
- Gawker blogs (Gawker/Gizmodo/Jezebel/Lifehacker/etc.)
- Engadget/Techcrunch/etc. (Sites which use Disqus)
- WordPress/Tumblr
- Forums of all kinds
- Digg/Reddit
- CNN/New York Times/The Guardian/other news websites
- And lots more!

FEATURES:
——————
* Records your post as you type so you can recover it if you lose it.
* Keeps a history of all your posts along with a link to the page and the time.
* Allows you to add filters so you can disable tracking on certain websites.
* Simple checkbox in the popup window enables/disable tracking globally

The latest version (0.4) added tracking for Google+ posts.

The extension helped me learn Javascript more (and also helped me deal with HTML Web Databases). The source code for the extension is open-source and has been released under the GPL v2 license. The source code can be accessed on Google Code here. Feel free to look through it.

If you have any feature requests for the extension please let me know. Some features I have thought about adding include:

• Delete all comments after a set time. (User-specified time, for ex. 10 minutes, 1 day, 1 month, etc.) (UPDATE: Added in Version 0.5 on 9th August 2011)
• Search Comments: Within that include searching by website, within a specific timeframe, etc.
• Delete Comments by website

Any other suggestions would be welcome!

———————————————————————————————————-
GitHub Repository and Visual Studio 2010 files:
GitHub Repository here
———————————————————————————————————–

Images for all the render targets/textures (Diffuse, Normals, Depth, Ambient Occlusion, Composite with Ambient Occlusion, Composite without AO):

Diffuse Texture

View-space Normals

Depth Texture

Ambient Occlusion

Composite with Ambient Occlusion

Composite without AO

What is Deferred Shading?

If you are reading this chances are you already know what deferred shading is and how it works, so I won’t go into too much detail. Wikipedia has a good overview and there are other places which can describe it better.

Basically deferred shading involves rendering your view-space normals, your depth-buffer, specular/diffuse, ambient occlusion, etc. values into different textures and then using them at a later pass to create a final composite image (by rendering a full-screen quad). There are several advantages and some disadvantages (increased memory bandwidth due to multiple render targets, inability to handle transparency, anti-aliasing, etc.)

Implementation:

I started off by modifying an existing DirectX10 project. “Tutorial10″ from Microsoft’s DirectX SDK Samples was used as a starting point, which uses the DXUT Library to specify setup and input (think GLUT for OpenGL). So my scene itself is simple (just one character mesh) as I mainly wanted to focus on the rendering part (I might add a more complex scene later). The steps which I’m going to cover are:

1. Setup the Multiple Render Targets and full-screen quad.
2. Call simple pass (which uses the geometry shader) to render view-space normals and depth into a texture.
3. Call pass to generate the AO texture from the previously generated normals and depth.
4. Gaussian Blur passes to smooth out the noise from the AO texture.
5. View Texture (either the composite lit one or one of the others)

1. Setting up Render Targets:

I’ll just show how to setup the multiple render targets for the first pass, and then you can figure out the setup for the AO textures. First we declare all the variables we need:

// The Multiple Render Targets
ID3D10Texture2D*                    _mrtTex;// Environment map
ID3D10RenderTargetView*             _mrtRTV;// Render target view for the mrts
ID3D10ShaderResourceView*           _mrtSRV;// Shader resource view for the mrts
ID3D10EffectShaderResourceVariable* _mrtTextureVariable = NULL; // for sending in the mrts
ID3D10Texture2D*                    _mrtMapDepth;// Depth
ID3D10DepthStencilView*             _mrtDSV;// Depth stencil view

These are the minimum variables you need. You need the textures (ID3D10Texture2D), the RenderTargetViews, the ShaderResourceViews and the EffectShaderResourceVariable (to send in the texture to the shader). You also need a DepthStencilView for the depth.

The setup is done in the SetupMRTs(..) function:

//----------------------------------------------
// Sets up the Multiple Render Targets
//----------------------------------------------
HRESULT SetupMRTs(ID3D10Device* pd3dDevice) {
    HRESULT hr;

    // Create depth stencil texture.
    D3D10_TEXTURE2D_DESC dstex;
    ZeroMemory( &dstex, sizeof(dstex) );
    dstex.Width = _width * TEXSCALE;
    dstex.Height = _height * TEXSCALE;
    dstex.MipLevels = 1;
    dstex.ArraySize = NUMRTS;
    dstex.SampleDesc.Count = 1;
    dstex.SampleDesc.Quality = 0;
    dstex.Format = DXGI_FORMAT_D32_FLOAT;
    dstex.Usage = D3D10_USAGE_DEFAULT;
    dstex.BindFlags =  D3D10_BIND_DEPTH_STENCIL;
    dstex.CPUAccessFlags = 0;

    _mrtMapDepth = NULL;

    V_RETURN(  pd3dDevice->CreateTexture2D( &dstex, NULL, &_mrtMapDepth ));

    // Create the depth stencil view for the mrts
    D3D10_DEPTH_STENCIL_VIEW_DESC DescDS;
    DescDS.Format = dstex.Format;
    DescDS.ViewDimension = D3D10_DSV_DIMENSION_TEXTURE2DARRAY;
    DescDS.Texture2DArray.FirstArraySlice = 0;
    DescDS.Texture2DArray.ArraySize = NUMRTS;
    DescDS.Texture2DArray.MipSlice = 0;

    _mrtDSV = NULL;
    V_RETURN (  pd3dDevice->CreateDepthStencilView( _mrtMapDepth, &DescDS, &_mrtDSV ) );

    // Create all the multiple render target textures
    dstex.Format = DXGI_FORMAT_R16G16B16A16_UNORM;
    dstex.BindFlags = D3D10_BIND_RENDER_TARGET | D3D10_BIND_SHADER_RESOURCE;
    _mrtTex = NULL;
    V_RETURN ( pd3dDevice->CreateTexture2D( &dstex, NULL, &_mrtTex ) );

    // Create the 3 render target view
    D3D10_RENDER_TARGET_VIEW_DESC DescRT;
    DescRT.Format = dstex.Format;
    DescRT.ViewDimension = D3D10_RTV_DIMENSION_TEXTURE2DARRAY;
    DescRT.Texture2DArray.FirstArraySlice = 0;
    DescRT.Texture2DArray.ArraySize = NUMRTS;
    DescRT.Texture2DArray.MipSlice = 0;
    V_RETURN ( pd3dDevice->CreateRenderTargetView( _mrtTex, &DescRT, &_mrtRTV ) );

    // Create the shader resource view for the cubic env map
    D3D10_SHADER_RESOURCE_VIEW_DESC SRVDesc;
    ZeroMemory( &SRVDesc, sizeof( SRVDesc ) );
    SRVDesc.Format = dstex.Format;
    SRVDesc.ViewDimension = D3D10_SRV_DIMENSION_TEXTURE2DARRAY;
    SRVDesc.Texture2DArray.ArraySize = NUMRTS;
    SRVDesc.Texture2DArray.FirstArraySlice = 0;
    SRVDesc.Texture2DArray.MipLevels = 1;
    SRVDesc.Texture2DArray.MostDetailedMip = 0;
    _mrtSRV = NULL;
    V_RETURN ( pd3dDevice->CreateShaderResourceView( _mrtTex, &SRVDesc, &_mrtSRV ) );

    return S_OK;
}

I know the above might look overwhelming at first, but it’s the standard way of generating a render target in DX10. The first thing we do is generate the Depth texture and the Depth Stencil View, making sure we specify the ArraySize (the # of render targets). Also make sure that the ViewDimension is specified as D3D10_RTV_DIMENSION_TEXTURE2DARRAY, since we have an array of Textures (in HLSL the variable will be declared as Texture2DArray). If we are using just one render target we would use D3D10_RTV…_TEXTURE2D instead (for the Ambient Occlusion Textures).

Next we generate the Textures themselves, the RenderTargetView and the ShaderResourceView. Make sure the BindFlags are D3D10_BIND_RENDER_TARGET | D3D10_BIND_SHADER_RESOURCE, indicating that the textures will be used as both Render Targets and Shader Resources.

2. Rendering View-Space Normals and Depth:

In the main draw function (OND3D10FrameRender), before we render to texture we first make sure that we save the old viewport and render target:

// Save the old RT and DS buffer views
ID3D10RenderTargetView* apOldRTVs[1] = { NULL };
ID3D10DepthStencilView* pOldDS = NULL;
pd3dDevice->OMGetRenderTargets( 1, apOldRTVs, &pOldDS );

// Save the old viewport
D3D10_VIEWPORT OldVP;
UINT cRT = 1;
pd3dDevice->RSGetViewports( &cRT, &OldVP );

/** Start rendering to all the textures **/
RenderTextures(pd3dDevice);

Next we are going to look at the RenderTextures(..) function:

//--------------------------------------------------------------------
// Renders all the textures:
// -Diffuse
// -Normals
// -Position
// -Depth
//--------------------------------------------------------------------
void RenderTextures( ID3D10Device* pd3dDevice) {
  // Set a new viewport for rendering to texture(s)
  D3D10_VIEWPORT SMVP;
  SMVP.Height = _height * TEXSCALE;
  SMVP.Width = _width * TEXSCALE;
  SMVP.MinDepth = 0;
  SMVP.MaxDepth = 1;
  SMVP.TopLeftX = 0;
  SMVP.TopLeftY = 0;
  pd3dDevice->RSSetViewports( 1, &SMVP );

  float ClearColor[4] = { 0.0f, 0.125f, 0.3f, 1.0f };//{ 0.0f, 0.0f, 0.0f, 1.0f };

  // Clear Textures
  pd3dDevice->ClearRenderTargetView( _mrtRTV, ClearColor );
  pd3dDevice->ClearDepthStencilView( _mrtDSV, D3D10_CLEAR_DEPTH, 1.0, 0 );

  // set input layout
  pd3dDevice->IASetInputLayout( g_pVertexLayout );

  // Set all the render targets
  ID3D10RenderTargetView* aRTViews[ 1 ] = { _mrtRTV };
  UINT numRenderTargets = sizeof( aRTViews ) / sizeof( aRTViews[0] );
  pd3dDevice->OMSetRenderTargets( numRenderTargets, aRTViews, _mrtDSV );

  // Render the objects

  //_aoTextureVariable->SetResource(  _aoSRV );

  // Get the technique
  D3D10_TECHNIQUE_DESC techDesc;
  g_pTechnique->GetDesc( &techDesc );

  // send the camera variables
  g_pProjectionVariable->SetMatrix( ( float* )g_Camera.GetProjMatrix() );
  g_pViewVariable->SetMatrix( ( float* )g_Camera.GetViewMatrix() );
  g_pWorldVariable->SetMatrix( ( float* )&g_World );

  /** Render the Mesh ***/
  UINT Strides[1];
  UINT Offsets[1];
  ID3D10Buffer* pVB[1];
  pVB[0] = g_Mesh.GetVB10( 0, 0 );
  Strides[0] = ( UINT )g_Mesh.GetVertexStride( 0, 0 );
  Offsets[0] = 0;
  pd3dDevice->IASetVertexBuffers( 0, 1, pVB, Strides, Offsets );
  pd3dDevice->IASetIndexBuffer( g_Mesh.GetIB10( 0 ), g_Mesh.GetIBFormat10( 0 ), 0 );

  //D3D10_TECHNIQUE_DESC techDesc;
  g_pTechnique->GetDesc( &techDesc );
  SDKMESH_SUBSET* pSubset = NULL;
  ID3D10ShaderResourceView* pDiffuseRV = NULL;
  D3D10_PRIMITIVE_TOPOLOGY PrimType;

  for( UINT subset = 0; subset < g_Mesh.GetNumSubsets( 0 ); ++subset )   {     pSubset = g_Mesh.GetSubset( 0, subset );     PrimType = g_Mesh.GetPrimitiveType10( ( SDKMESH_PRIMITIVE_TYPE )pSubset->PrimitiveType );
    pd3dDevice->IASetPrimitiveTopology( PrimType );

    pDiffuseRV = g_Mesh.GetMaterial( pSubset->MaterialID )->pDiffuseRV10;
    g_ptxDiffuseVariable->SetResource( pDiffuseRV );

    g_pTechnique->GetPassByIndex( 2 )->Apply( 0 );
    pd3dDevice->DrawIndexed( ( UINT )pSubset->IndexCount, 0, ( UINT )pSubset->VertexStart );
  }
} // End Render Textures

Next we are going to look at the HLSL shader code for rendering to the multiple targets:

NOTE: Before you start coding in HLSL (in Visual Studio) make sure you setup the debugger for it. It saved me a lot of time and headache. Thanks to Bobby Anguelov for the instructions.

/******* Multiple Render Target Functions***************/

// Geometry Shader input - vertex shader output
struct GS_IN
{
    float4 Pos	: POSITION;  // WorldViewProj position
    float4 PosWV: TEXCOORD1; // World View Position
    float3 Norm : NORMAL;    // Normal
    float2 Tex	: TEXCOORD0; // Texture coord
};

// Pixel Shader in - Geometry Shader out
struct PS_MRT_INPUT
{
    float4 Pos  : SV_POSITION; 
    float4 PosWV: TEXCOORD1;    // World View Position
    float3 Norm: NORMAL;        //normal
    float2 Tex : TEXCOORD0;
    uint RTIndex : SV_RenderTargetArrayIndex; // which render target to write to (0-2)
};

//---------------------------------------------------------------------
// Simple Vertex Shader for MRT
//---------------------------------------------------------------------
GS_IN VSMRT( VS_INPUT input )
{
    GS_IN output = (GS_IN)0;
    
    input.Pos += input.Norm*Puffiness;
    
    output.PosWV = mul( float4(input.Pos,1), World );
    output.PosWV = mul( output.PosWV, View );
    output.Pos = mul( output.PosWV, Projection );
    output.Norm = mul( input.Norm, World );
    output.Norm = mul( output.Norm, View );
    output.Norm = mul( output.Norm, Projection );
    output.Tex = input.Tex;
    
    return output;
}

//----------------------------------------------------------------------
// Geometry Shader For MRT
//----------------------------------------------------------------------
[maxvertexcount(12)]
void GSMRT( triangle GS_IN input[3], inout TriangleStream<PS_MRT_INPUT> CubeMapStream )
{

    // 0 = diffuse color
    // 1 = normals
    // 3 = depth

    for( int f = 0; f < 4; ++f )
    {
        // Compute screen coordinates
        PS_MRT_INPUT output;
        output.RTIndex = f;
        for( int v = 0; v < 3; v++ )
        {
            output.Pos =  input[v].Pos;	// position
            output.PosWV =  input[v].PosWV;	// position
	    output.Norm = input[v].Norm;	// normal
			
            output.Tex = input[v].Tex;
            CubeMapStream.Append( output );
        }
        CubeMapStream.RestartStrip();
    }

} // end of geometry shader

float4 PSMRT( PS_MRT_INPUT input ) : SV_Target
{
	if (input.RTIndex == 0)	{	 // diffuse
	   return g_txDiffuse.Sample( samLinear, input.Tex  );
	}
	else if (input.RTIndex == 1) { // normal
	   // convert normal to texture space [-1;+1] -> [0;1]
	   float4 normal;
	   normal.xyz = input.Norm * 0.5 + 0.5;
	   normal.w = 1.0;
	   return normal;
	}
	else if (input.RTIndex == 2) // position - not actually used
	   return input.PosWV;
	else {			     // depth
          float normalizedDistance = input.Pos.z / input.Pos.w;
	  
          normalizedDistance = 1.0f - normalizedDistance;
	  return float4(normalizedDistance, normalizedDistance, normalizedDistance, normalizedDistance);
	}
    //return input.Pos;
}

I am just going to focus on the geometry shader (GSMRT()) and pixel shader (PSMRT()). The code above renders 4 values: The Diffuse colour (from texture), the view-space normals, the view-space position and the depth. I actually don’t use the view-space position since I reconstruct the position from depth later on, but I thought I’d just leave it in there.

The geometry shader is essential for rendering to multiple targets. The “RTIndex” variable is used for telling the pixel shader to render to which target. Otherwise we just pass through the information (normals, UV-values, position, etc.)

In the pixel shader we read the RTIndex value and then write out the values we want to. Hopefully the code isn’t too complex and is comprehensible. Now you can go ahead and create the final render (composite texture), or apply other effects. I thought I might give Screen-Space Ambient Occlusion a try.

3. Ambient Occlusion & Gaussian Blur:

Ambient Occlusion is a popular technique for adding realism to scenes. It takes into account “attenuation of light due to occlusion”. Screen-Space Ambient Occlusion has been the most-popular and cost-effective way of implementing AO. The videogame Crysis was the first to implement it, and there have been multiple variants since. I implemented the SSAO algorithm
featured on GameDev.net. I won’t go over the shader code here since GameDev has a pretty good explanation.

To render Ambient Occlusion the function “RenderAmbientOcclusion()” is called. The first step is to change the render target, and then we render a full-screen quad. The first pass will generate the AO texture, but it has a lot of noise. To “smooth out” the noise we apply gaussian blur. The blur shaders were taken from here (it’s basically sampling from around the current pixel and averaging to smooth out – do it once for horizontal and then once for vertical).

SSAO requires a lot of computation – my framerate goes from 60 to ~11 when I turn on AO (though it’s roughly ~23 without gaussian blur). There shouldn’t be such a huge drop in the framerate; the texture lookup for the random vector is the bottleneck, and I can probably optimize it further.

Finally we can go on to create the final render.

4. Viewing Composite Texture:

To view the composite rendering we once again render a full-screen quad and pass in all the textures (diffuse, depth, normals, ambient occlusion). NOTE: The light variables are hard-coded into the shader (ideally you’d want to send them in).

The final shader is simple enough:

// Pixel Shader for rendering full-screen quad
// 0 = diffuse 
// 1 = normal (put specular in .w?)
// 2 = position
// 3 = depth
// 4 = ambient occlusion
// 5 = composite
float4 PSQuad( PS_INPUT input) : SV_Target 
{
   // get all the values

   // Diffuse
   float4 diffuse = _mrtTextures.Sample( samPoint, float3(input.Tex, 0) );
   if (TexToRender == 0)
   	return diffuse;

    // normals 
    float4 normals = _mrtTextures.Sample( samPoint, float3(input.Tex, 1) );
    normals =  (normals - 0.5) * 2.0;
    if (TexToRender == 1)
 	return normals;

    // depth
    float4 depth	= _mrtTextures.Sample( samPoint, float3(input.Tex, 3) );
    // discard
    if (depth.x == 0.0)
	return float4( 0.0f, 0.125f, 0.3f, 1.0f );
    if (TexToRender == 3)
	return depth;

    // position
    // reconstructed from depth value
    depth = 1.0 - depth;
    float4 H = float4(input.Tex.x * 2.0 - 1.0, 
                     (1.0 - input.Tex.y) * 2.0 - 1.0,  
		     depth.x,
		     1);
    float4 D = mul(H, ProjectionInverse);
    float4 position = D / D.w;
    if (TexToRender == 2)
        return position;
   
    // ambient occlusion
    float4 ao;
    if (UseAO == true) {
    	ao  = _aoTexture.Sample( samLinear, input.Tex );
    	if (TexToRender == 5)
	    return ao;

    // else calculate the light value - phong shading
    
    float3 lightDir = vLightPos - position.xyz;
    float3 eyeVec   = -position.xyz;
    float3 N        = normalize(normals);
    float3 E        = normalize(eyeVec);
    float3 L        = normalize(lightDir);
    float3 reflectV = reflect(-L, normals.xyz);

    // diffuse
    float4 dTerm = diffuse * max(dot(N, L), 0.0);

    // specular
    float4 specular = matSpecular * pow(max(dot(reflectV, E), 0.0), matShininess);

    float4 outputColor = dTerm;// + specular;
    if (UseAO == true) {
    	outputColor = outputColor * ao;
    }

    return outputColor;
}

First I read in all the different textures. And depending on which texture the user wants to view the value is returned. If the composite texture is to be viewed phong shading is performed. Though I calculate the specular value I don’t use it in my final rendering since it gives an overly plastic look. And depending on whether ambient occlusion is turned or not further the color is appropriately darkened.

Conclusion:

I went over how to setup multiple render targets and write to them, rendering a full-screen quad and ambient occlusion, and then creating a composite texture. If there is something in my code you don’t understand or have suggestions to improve please feel free to ask/let me know.

I will probably try to create a better scene and come up with better images. I might also try implementing shadow mapping since it shouldn’t be that difficult to include it in a deferred renderer.

NOTE: When you exit my DirectX program it gives errors – I realize I don’t release all the resources but it should be ok.

———————————————————————————————————-
GitHub Repository and Visual Studio 2010 files:
GitHub Repository here
———————————————————————————————————–

Credits:

1. Debugging HLSL
2. A Simple and Practical Approach to SSAO
3. Gaussian Blur Filter Shader

Someone requested an Anaglyph WebGL Renderer for it, so I thought I might give it a try. After reading up on the code I came up with an example you can find here (NOTE: Works best on latest version of Chrome and Firefox).

The author of Three.js has already fixed my version and incorporated an actual anaglyph renderer into his code. Here is an example.

If you are interested in WebGL development I would recommend checking out Three.js.

———————————————————————————————————-
I’ll first link to the apk, source code and the repository:
apk here.
Source code here.
Google Code Repository  here. NOTE: Branch is “shadowMapping”
———————————————————————————————————–

UPDATE:

Thanks to Henry’s comment below I was able to fix most of the artifacts showing up. Turns out I had dithering enabled, so I had to disable it to remove the dithering effect [ GLES20.glDisable(GLES20.GL_DITHER) ]. Updated images below – it works but has very slight artifacts when it comes to the mesh’s self-shadows.

Introduction:

Shadow Mapping is a popular technique for rendering shadows in games and other real-time applications. Given how GPUs on mobile devices (iPhone/Android/whatever) are getting more and more powerful every day I thought I might try implementing it on my Samsung Captivate (Galaxy S).

While my implementation “works”, it does not good look at all. Shadow-mapping is prone to many artifacts , which I wasn’t able to properly fix (UPDATE: Main artifacts fixed).I am posting this without the fix in the hopes that somebody can help me out too as to what I am doing wrong.

Here are the updated images (I changed the color of the plane to get a better contrast):

Textured Cube (with proper shadows)

Dragon Head

Octahedron

Here are the old images:

Octahedron Mesh (.OFF) with shadows + artifacts

Dragon-head Mesh (.OFF)

Textured Cube (.OBJ)

I would like to credit Fabien Sanglard for his shadow mapping tutorials which I used as a basis for my code. I looked at his DEngine code as an example. In any case let’s begin.

What is Shadow Mapping?

If you are reading this chances are that you already know what shadow mapping is and how it works, but let me give a quick overview of the steps involved:

1. Generating the Shadow Map:The “shadow map” (aka a depth map) is just a texture which stores the depth of the scene from the viewpoint of the (directional) light. Scene is rendered from the light’s viewpoint into a texture.
2. Rendering the Scene:The scene is rendered from the usual camera’s viewpoint. But along with the usual information the lightviewmatrix and the depth texture are passed in to the shader, and a texture projection is used to compare whether the current pixel being shaded is covered by a shadow or not.And that’s basically it! Shadow Mapping isn’t too complex, but it has a habit of producing a lot of nasty artifacts. So we will have to be careful when generating proper shadow maps, the matrices and doing the shading.

Implementation:

1. The Scene:The scene is just composed of one mesh object (octahedron[.OFF], dragon’s head[.OFF] or textured cube[.OBJ]) above a plane (.OFF) with a rotating light. You can switch between the meshes and pause/resume the rotation of the light through the menus. I modified the phong shader to produce the shadows – the gouraud and normal mapping shaders function as before. To look at how all this is done please refer to my previous tutorials.
   NOTE: Shadow Mapping requires a “spotlight” to function – my shaders use “point lights”. To get shadows from point lights you have to render from multiple views around the light (cube map generation), so I just treat my point light as a spotlight for generating shadows, and render the rest of the non-shadowed scene as if lit by a point light.
2. Generating the Shadow Map:To render the shadow map you have to render from the light’s viewpoint. The size of the depth map texture in my code is 512 X 512. You can view the depth map through the “View depth map” option in the menu. The rendering is done in the renderDepthToTexture()function:

   // Cull front faces for shadow generation
   // Culling front faces is suggested for removing artifacts, but it doesn't seem to
   // present a noticeable improvement
   GLES20.glCullFace(GLES20.GL_FRONT);// Bind the shadow map texture now...
   GLES20.glViewport(0, 0, this.texW, this.texH);// bind the generated framebuffer
   GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fb[0])
   
   // specify texture as color attachment
   GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, renderTex[0], 0);
   
   // attach render buffer as depth buffer
   GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_DEPTH_ATTACHMENT, GLES20.GL_RENDERBUFFER, depthRb[0]);
   
   // check status
   int status = GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER);
   if (status != GLES20.GL_FRAMEBUFFER_COMPLETE)
   return false;
   
   /*** DRAW ***/
   // Clear color and buffers
   GLES20.glClearColor(1.0f, 1.0f, 1.0f, 1.0f);
   GLES20.glClear( GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);
   
   // depth map shaders
   // I used a new shader to generate the depth map.
   Shader shader = _shaders[this.DEPTHMAP_SHADER];
   int _program = shader.get_program();
   
   // Start using the shader
   GLES20.glUseProgram(_program);
   checkGlError("glUseProgram");
   
   // View from the light's perspective
   Matrix.setLookAtM(lMVMatrix, 0, lightPos[0], lightPos[1], lightPos[2],
   lightPos[4], lightPos[5], lightPos[6],
   lightPos[7], lightPos[8], lightPos[9]);
   float ratio2 = (float)texW /texH;
   Matrix.frustumM(lProjMatrix, 0, -ratio2, ratio2, -1, 1, 1f, 5000f);
   
   // modelviewprojection matrix
   Matrix.multiplyMM(lMVPMatrix,0, lProjMatrix, 0, lMVMatrix, 0);
   
   // send to the shader
   GLES20.glUniformMatrix4fv(GLES20.glGetUniformLocation(_program, "uMVPMatrix"), 1, false, lMVPMatrix, 0);
   
   /// DRAW ALL THE OBJECTS
   drawAllObjects(_program, false, false);
   
   // render the depth buffer?
   if (viewDepthTex) {
   renderToQuad();
   return true;
   }
   
   /**** Else, render with shadow now --
   */
   renderWithShadow(lMVPMatrix);
   

   Hopefully the code above is self-explanatory. After we render the depth to the texture we can either view the depth texture or render the shadows. Next I am going to look at the depth map pixel shader which generates the depth map and then the “renderWithShadow” function.

   The depth map pixel shader is written below. We basically just render the depth into a RGB texture. There might be better ways to do this, but this gave me good results.

   // Pixel shader to generate the Depth Map
   // Used for shadow mapping - generates depth map from the light's viewpoint precision highp float;
   
   varying vec4 position; 
   
   // taken from Fabien Sangalard's DEngine - this is used to pack the depth into a 32-bit texture (GL_RGBA)
   vec4 pack (float depth)
   {
   	const vec4 bitSh = vec4(256.0 * 256.0 * 256.0,
   					   256.0 * 256.0,
   					   256.0,
   					  1.0);
   	const vec4 bitMsk = vec4(0,
   					     1.0 / 256.0,
   					     1.0 / 256.0,
   				             1.0 / 256.0);
   	vec4 comp = fract(depth * bitSh);
   	comp -= comp.xxyz * bitMsk;
   	return comp;
   }
   
   void main() {
   	// the depth
   	float normalizedDistance  = position.z / position.w;
           // scale it from 0-1
   	normalizedDistance = (normalizedDistance + 1.0) / 2.0;
   	
   	// bias (to remove artifacts)
   	normalizedDistance += 0.0005;
   
   	// pack value into 32-bit RGBA texture
   	gl_FragColor = pack(normalizedDistance);
   	
   }
   

3. Rendering the Scene with Shadows:Now we have our depth texture. All we need to do is render the scene from the camera’s viewpoint. The only new variable we are passing in this time into the phong shader is the modelview matrix from the light’s viewpoint. The rendering is done in the “renderWithShadow(lMVPMatrix)” function. It works pretty much the same way as the rendering functions shown in the previous tutorials, so look at those first if you want to get an idea of how this works. The shadow map texture is attached as “shadowTexture”.We will modify our phong vertex shader slightly to accommodate the shadow projection matrix:

   // Vertex shader Phong Shading
   ...
   // the shadow projection matrix
   uniform mat4 shadowProjMatrix;
   varying vec4 shadowCoord;
   ...
   attribute vec4 aPosition;
   ...void main() {
   ...
   
   shadowCoord = shadowProjMatrix * aPosition;
   ...
   }
   

   We are going to use these shadow coordinates to perform a texture lookup in our pixel shader and use them for testing whether the pixel is in shadow or not:

   ...
   
   // This function looks up the shadow texture
   // and returns whether the pixel is in shadow or not
   float getShadowFactor(vec4 lightZ)
   {
   	vec4 packedZValue = texture2D(shadowTexture, lightZ.st);
   
   	// unpack the value stored to get the depth. 
   	const vec4 bitShifts = vec4(1.0 / (256.0 * 256.0 * 256.0),
   						1.0 / (256.0 * 256.0),
   						1.0 / 256.0,
   						1);
   	float shadow = dot(packedZValue , bitShifts);
   
   	return float(shadow > lightZ.z);
   }
   
   void main() {
   ...
   ...
   // ambient, diffuse and specular terms have been calculated
   
   // Shadow
   float sValue = 1.0; // for ambient + diffuse
   float sValue2 = 1.0; // for specular
   if (shadowCoord.w > 0.0) {
   // get the shadow coordinates for the texture
   vec4 lightZ = shadowCoord / 45.0;
   
   // scale + bias
   lightZ = (lightZ + 1.0) /2.0;
   lightZ.z += 0.0005;
   
   // get the shadow value
   sValue = getShadowFactor(lightZ);
   
   // scale the value from 0.5-1.0 to get a softer shadow
   float newMin = 0.5;
   float v1 = (1.0)/(1.0 - newMin);
   float v2 = sValue/v1;
   sValue2 = sValue + newMin;
   }
   
   gl_FragColor = ( (ambientTerm + diffuseTerm) * sValue2 + specularTerm * sValue) ;
   }
   

   The scene has been rendered. That’s pretty much all there is to it – the next step would be implementing percentage-closer filtering and other techniques to produce better shadows.

   Any comments and questions are welcome.

   ———————————————————————————————————-
   Links to the apk, source code and the repository:
   apk here.
   Source code here.
   Google Code Repository  here. NOTE: Branch is “shadowMapping”
   ———————————————————————————————————–

———————————————————————————————————-
I’ll first link to the apk, source code and the repository:
apk here.
Source code here.
Google Code Repository  here. NOTE: Branch is “renderToTex”
———————————————————————————————————–

Rendering to a texture is important when it comes to various graphics techniques and algorithms (Shadow Mapping, cube map generation, Deferred Shading, etc.) So this quick tutorial will demonstrate how to setup a texture for rendering (and then how to display that texture on screen).

What is rendered to the texture?
We are going to render the same objects as in the previous tutorial (where you can choose between gouraud/phong/normal mapping shaders). The texture is going to be a simple quad which will then be displayed on the full screen. I have a Samsung Captivate(Samsung Galaxy S) which has a screen resolution of 800 X 480, so I use those values as the height and width of my texture. Adjust accordingly to your phone (I was a tad lazy to read in the values from the system itself). Let’s begin:

1. New Variable declarations:

The variables below are for creating the quad to render the texture onto:

// the full-screen quad buffers
final float x = 10.0f;
final float y = 15.0f;
final float z = 2.0f;
// vertex information - clockwise
                       // x,  y, z, nx, ny, nz, u, v
final float _quadv[] = { -x, -y, z, 0, 0, -1, 0, 0,
			-x,  y, z, 0, 0, -1, 0, 1,
			 x,  y, z, 0, 0, -1, 1, 1,
			 x, -y, z, 0, 0, -1, 1, 0
			};

private FloatBuffer _qvb;
// index
final int _quadi[] = { 0, 1, 2,
		      2, 3, 0 };
private IntBuffer _qib;

Variables used for rendering to texture are declared below:

// RENDER TO TEXTURE VARIABLES
int[] fb, depthRb, renderTex; // the framebuffer, the renderbuffer and the texture to render
int texW = 480 * 2;           // the texture's width
int texH = 800 * 2;           // the texture's height
IntBuffer texBuffer;          //  Buffer to store the texture

1. Setup Render To Texture:

The method setupRenderToTexture() (Hopefully the comments give enough detail):

// create the ints for the framebuffer, depth render buffer and texture
fb = new int[1];
depthRb = new int[1];
renderTex = new int[1];

// generate
GLES20.glGenFramebuffers(1, fb, 0);
GLES20.glGenRenderbuffers(1, depthRb, 0); // the depth buffer
GLES20.glGenTextures(1, renderTex, 0);

// generate texture
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, renderTex[0]);

// parameters - we have to make sure we clamp the textures to the edges
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S,
		GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T,
		GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER,
		GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER,
		GLES20.GL_LINEAR);

// create it
// create an empty intbuffer first
int[] buf = new int[texW * texH];
texBuffer = ByteBuffer.allocateDirect(buf.length
		* FLOAT_SIZE_BYTES).order(ByteOrder.nativeOrder()).asIntBuffer();;

// generate the textures
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_RGB, texW, texH, 0, GLES20.GL_RGB, GLES20.GL_UNSIGNED_SHORT_5_6_5, texBuffer);

// create render buffer and bind 16-bit depth buffer
GLES20.glBindRenderbuffer(GLES20.GL_RENDERBUFFER, depthRb[0]);
GLES20.glRenderbufferStorage(GLES20.GL_RENDERBUFFER, GLES20.GL_DEPTH_COMPONENT16, texW, texH);

1. Rendering:

The rendering begins in the onDrawFrame() method. First we render to texture:

// viewport should match texture size
Matrix.frustumM(mProjMatrix, 0, -ratio, ratio, -1, 1, 0.5f, 10);
GLES20.glViewport(0, 0, this.texW, this.texH);

// Bind the framebuffer
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fb[0]);

// specify texture as color attachment
GLES20.glFramebufferTexture2D(GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, renderTex[0], 0);

// attach render buffer as depth buffer
GLES20.glFramebufferRenderbuffer(GLES20.GL_FRAMEBUFFER, GLES20.GL_DEPTH_ATTACHMENT, GLES20.GL_RENDERBUFFER, depthRb[0]);

// check status
int status = GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER);
if (status != GLES20.GL_FRAMEBUFFER_COMPLETE)
	return false;

// Clear the texture (buffer) and then render as usual...
GLES20.glClearColor(.0f, .0f, .0f, 1.0f);
GLES20.glClear( GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);

....usual rendering continues....

The final step is to render the quad itself to cover the screen fully (this is done using the gouraud shader):

// Bind the default framebuffer (to render to the screen) - indicated by '0'
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);

GLES20.glClearColor(.0f, .0f, .0f, 1.0f);
GLES20.glClear( GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);

...setup all the matrices as usual....

// send the vertex coordinates
_qvb.position(TRIANGLE_VERTICES_DATA_POS_OFFSET);
GLES20.glVertexAttribPointer(GLES20.glGetAttribLocation(_program, "aPosition"), 3, GLES20.GL_FLOAT, false,
		TRIANGLE_VERTICES_DATA_STRIDE_BYTES, _qvb);
GLES20.glEnableVertexAttribArray(GLES20.glGetAttribLocation(_program, "aPosition"));

// the normal info (though shouldn't be needed)
_qvb.position(TRIANGLE_VERTICES_DATA_NOR_OFFSET);
GLES20.glVertexAttribPointer(GLES20.glGetAttribLocation(_program, "aNormal"), 3, GLES20.GL_FLOAT, false,
		TRIANGLE_VERTICES_DATA_STRIDE_BYTES, _qvb);
GLES20.glEnableVertexAttribArray(GLES20.glGetAttribLocation(_program, "aNormal"));

// bind the framebuffer texture - this is what will be attached to the quad
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, renderTex[0]);
GLES20.glUniform1i(GLES20.glGetUniformLocation(_program, "texture1"), 0);

// enable texturing? [fix - sending float is waste]
GLES20.glUniform1f(GLES20.glGetUniformLocation(_program, "hasTexture"), 1.0f);

// texture coordinates
_qvb.position(TRIANGLE_VERTICES_DATA_TEX_OFFSET);
GLES20.glVertexAttribPointer(GLES20.glGetAttribLocation(_program, "textureCoord"), 2, GLES20.GL_FLOAT, false,
		TRIANGLE_VERTICES_DATA_STRIDE_BYTES, _qvb);
GLES20.glEnableVertexAttribArray(GLES20.glGetAttribLocation(_program, "textureCoord"));//GLES20.glEnableVertexAttribArray(shader.maTextureHandle);

// Draw with indices
GLES20.glDrawElements(GLES20.GL_TRIANGLES, _quadi.length, GLES20.GL_UNSIGNED_INT, _qib);

And that’s it! Hopefully all the comments in the code snippets above explain everything. The quad and texture sizes are something I just fiddled around with, so adjust accordingly to the device you are testing on.

If you have any questions, suggestions or comments please let me know.

———————————————————————————————————-
Links to the apk, source code and the repository:
apk here.
Source code here.
Google Code Repository  here. NOTE: Branch is “renderToTex”
———————————————————————————————————–