XNA – Alpha Mapping with Pixel Shaders

I’ve recently been playing with an idea to make a simple space shooter for the xbox with Micorsofts XNA framework.

As always I don’t want to take the simplest path when doing things, and instead try to do something the hard way without shortcuts and hopefully learn something I dont know while doing it.

So far in this project I’ve been able to implement Farseer (a very good physics engine for C#) and have added a few boxes, a destructible planet with gravity and a simple sprite effect (bullet trails and engine smoke).

Of those things the destructible terrain was a bit difficult but the most difficult was making the terrain of the texture disappear as it is destroyed.

There are several places on the net where you can find people doing this using the SetData and GetData methods (Riemers XNA Tutorials explains this in detail). However, when there is alot going on you will notice slowdowns with this method. The simple reason behind the slow downs is that the Set/Get -Data method brings the texture back from the GPU to your RAM, and the calculations are then done on the CPU.

So thinking there should to be a better way to do this I googled alot and found that what I want to do is create a  alpha map pixel shader…

So first of all, what is a Pixel shader?
Well simply put it is a function that runs on the GPU instead of on the CPU. The GPU runs this function on every pixel in a specified texture. Pixel shaders can pretty much create any effect you can create in photoshop, and can do this very fast, much faster than the CPU can. In the video above I’m applying the Alpha Mapping shader without the CPU being impacted at all, and the GPU is just barely working.

The second question then, what is Alpha Mapping?
This is when you take a picture, such as the planet texture below and then use another picture to decide what parts of the original picture should be drawn.


In the video above I’m generating the Alpha map texture using a RenderTarget2D, but for this tutorial/example I’m going to keep it alot simpler than that and just render out the planet texture with our mask applied.

So fire up Visual Studio, start a new XNA 4 project and add the following class variables to your Game1 Class (or whatever you renamed Game1 to).

Texture2D _Planet;
Texture2D _AlphaMap;
Effect _AlphaShader;

Now download these two textures and import them into your content pipeline. I placed mine in a /Textures folder.

redplanet512 whitedotsblackbg

Next were going to populate the variables with some data. Ignore that the “Effects/AlphaMap” dosent exist, we’ll be adding that next. Copy the code below into the LoadContent() method.

        protected override void LoadContent()
            spriteBatch = new SpriteBatch(GraphicsDevice);

            _Planet = Content.Load("Textures/RedPlanet512");
            _AlphaMapp = Content.Load("Textures/Dots");
            _AlphaShader = Content.Load("Effects/AlphaMap");

To add the Effect (the shader) to your Xna 4 project side click on the content pipeline and click Add  > New Item… > Effect File. Name it “AlphaMap.fx” and press Add.
Paste the following code into it. Take time to read the comments to understand the different parts.

// this is the texture we are trying to render
uniform extern texture ScreenTexture;
sampler screen = sampler_state
  // get the texture we are trying to render from the gpu.
  Texture = ;

// this is the alpha map texture, we set this from the C# code.
uniform extern texture MaskTexture;
sampler mask = sampler_state
  Texture = ;

// here we do the real work.
float4 PixelShaderFunction(float2 inCoord: TEXCOORD0) : COLOR
  // we retrieve the color in the original texture at
  // the current coordinate remember that this function
  // is run on every pixel in our texture.
  float4 color = tex2D(screen, inCoord);

  // Since we are using a black and white mask the black
  // area will have a value of 0 and the white areas will
  // have a value of 255. Hence the black areas will subtract
  // nothing from our original color, and the white areas of
  // our mask will subtract all color from the color.
  color.rgba = color.rgba - tex2D(mask, inCoord).r;

  // return the new color of the pixel.
  return color;

  pass P0
    // The xbox can only run pixel shader 2.0
    // and for our purpose that is plenty too..
    PixelShader = compile ps_2_0 PixelShaderFunction();

The final bit is to use all these different parts in the Draw() method.

        protected override void Draw(GameTime gameTime)

            // set the Mask to use for our shader.
            // note that "MaskTexture" corresponds to the public variable in AlphaMap.fx

            // start a spritebatch for our effect
            spriteBatch.Begin(SpriteSortMode.Immediate, BlendState.AlphaBlend,
                null, null, null, _AlphaShader);

                new Vector2(400, 300),
                null, Color.White, 0f,
                new Vector2(_Planet.Width / 2, _Planet.Height / 2),
                1f, SpriteEffects.None, 1f);


If you did everything right you should end up with something like this:

If you want to check out a working example, here’s a link to the solution.
AlphaMappingExample.zip (you will need Visual Studio 2010 and XNA 4 to run this)

Received a few questions about how to modify the Texture2D alpha mask during run time in the comments, which led to this article Alpha Mapping with RenderTarget2D.

  • Pingback: cafelearn72

  • Pingback: Cut holes in a Texture2D | PHP Developer Resource

  • weedman

    thanks :)

    • syntaxwarrior

      Your welcome ;)

  • Cani

    I wonder how you managed the collision detection between the planet and the various boxes.
    I tried something like this

    private bool CheckCollision(Vector2 position)
    Color[] textureData = new Color[1];

    int x = (int)MathHelper.Clamp(position.X, 0, mask.Width-1);
    int y = (int)MathHelper.Clamp(position.Y, 0, mask.Height-1);

    mask.GetData(0, new Rectangle(x, y, 1, 1), textureData, 0, 1);

    if (textureData[0].A == 0.0f)
    return true;
    return false;

    for checking a single position (or some small area given by a rectangle). This works fine if you do it once per update. But doing this several times of course slows everything down. Getting the whole alpha mask seems to be no good option, too.

    • Jens Berfenfeldt

      I used Farseer, a 2D physics framework: http://farseerphysics.codeplex
      It is the best one Ive found for C#.

      • Cani

        Thanks for your answer. I’m aware you used Farseer. My question was targeting on the interface between the alpha mask and farseer. Do you use the PolygonTools.CreatePolygon()-method to create a polygon from the Texture2D every frame? As you have written getting the data from GPU to RAM is pretty slow. Or do you get create the polygon data once and update it parallel for every destruction?

  • Michael Aquilina

    Very nice :) Just FYI you have an extra SpriteBatch.End(); in your last code example.

    • Jens Berfenfeldt

      Fixed, thanks. :)

  • gogaz

    Hi, how do you delete a simple circle in a given position with this effect ?
    thanks !

  • Duncan Garrett

    This shader seems to ignore any colors applied via the Draw() function. Is there a way to include colors added in this manner?

    Also, the sample code on the site seems to be missing and in the effect code. (And Disqus feels that these should be closed)

    • Jens Berfenfeldt

      It should be possible. I have not done anything like that but I would guess that you could send a color as a variable (similar to how I send a texture in my example) to the shader and then modify this line:
      color.rgba = color.rgba – tex2D(mask, inCoord).r;
      to also incorporate the color you want to add to the pixel after you have subtracted the mask.

  • Pablo Henrique Bertaco

    Error 1 Errors compiling C:…AlphaMap.fx:

    C:…AlphaMap.fx(6,13): error X3000: syntax error: unexpected token ‘;’ C:…AlphaMap.fx 6 13

    • JensB

      Did you copy & paste the code or did you download and try the complete solution?
      If it was copy paste it may be that you got some extra characters from the web page. if not I’m guessing there has been some change to XNA since I wrote this example.
      Since this was written XNA has been discontinued and there is no support for it in VS2012.

      If you are looking for a new game framework I can strongly suggest Monogame which is built to be very similar to XNA.

      • Pablo Henrique Bertaco

        I copy & paste from this page.
        the error is pointing at
        // get the texture we are trying to render from the gpu.
        Texture = ;”
        in the effect file AlphaMap.fx, line 6
        At the downloaded file it is
        // get the texture we are trying to render.
        Texture = ; =}

        • JensB

          At the bottom of the post there is a link to the complete solution, try downloading that and building it. Using visual studio 2010 I can compile it, but XNA won’t run under windows 8.1 it seems.