Vertex shader spriteframe origin and size

cocos2d-x C++
1

I have a fragment shader that basically converts texture coordinates (UV) to sprite frame coordinates (a subset of the total UV), then blurs the current pixel based on that spriteframe-relative Y pos. Pixels at y == 0 (again, in sprite frame coordinates) are not blurred at all. Pixels at y == 1 are blurred to the max.

The way I’m currently accomplishing this is to pass in two uniforms:

  1. vec2 containing the sprite frame’s origin
  2. vec2 containing the sprite frame’s size

Now this performs badly with lots of sprites because custom uniforms cannot currently be batched by cocos2d-x. An ideal solution would be to use a vertex shader and no custom uniforms.

I’ve tried messing with a_position and even multiplying it by the MV matrix and the P matrix, but no luck. It would help if somebody could explain what exactly is a_position within the cocos2d-x paradigm.

I see that cocos2d-x already has a_texCoord, a_texCoord1, a_texCoord2, a_texCoord3. It appears that 1-3 are not being used.

Would it be possible to go about passing in the current sprite frame’s origin as a_texCoord1 and size as a_texCoord2? If so, where in cocos2d-x’ rendering code would I start doing that?

I’m looking at Texture2D::drawAtPoint and Texture2D::drawInRect maybe but not sure if these are the right places or if this crazy strategy is even on the right track.

2

Yeah for per quad data you’ll want to use attributes to support batching.

If you can pack your data into 3 floats (pos), 4 floats (color), and 2 floats (texcoord) by, for example, using the alpha channel as an argument into a shader function that outputs the correct frame size based on the alpha value then you’re good to go with just Sprite and modifying its 3 attributes correctly.

I’m not sure how easily you can modify or subclass Sprite to do what you want without writing some custom render code since TriangleCommand defaults to V3F_C4B_T2F for its vertex buffer attributes. It probably would be easiest to write a simple Sprite subclass (e.g. BlurSprite) along with a simple TriangleCommand subclass (e.g. BlurTriangleCommand) and then modify the struct Triangles appropriately for your additional attributes and any supporting code.

We do something similar with our custom tile map so that we pass in a normal (a_normal) and 2nd texture coord (a_texCoord1) for lighting, fog of war, and a couple tile-based effects (as opposed to screen position). However, we based it off FastTMXTileLayer class and thus we’re batching all quads internally into a single render command. This is another avenue you could take, custom batch all your sprites together as a single draw command. Then you can also provide uniforms to the entire batch as well.

Here’s our vertex buffer code for that

struct ST_V3F_C4B_T2Fx3_N3F
{
    cocos2d::Vec3     vertices;        // 12 bytes (pos)
    cocos2d::Color4B  colors;          // 4 bytes (color + opacity)
    cocos2d::Tex2F    texCoord0        // 8 bytes (uv s.frame coord)
    cocos2d::Tex2F    texCoord1;       // 8 bytes (tile coord)
    cocos2d::Tex2F    texCoord2;       // 8 bytes (extra remove if not need)
    cocos2d::Vec3     normals;         // 12 bytes (normal, could pack smaller)
};

struct ST_V3F_C4B_T2Fx3_N3F_Quad
{
    ST_V3F_C4B_T2Fx3_N3F    tl;
    ST_V3F_C4B_T2Fx3_N3F    bl;
    ST_V3F_C4B_T2Fx3_N3F    tr;
    ST_V3F_C4B_T2Fx3_N3F    br;
};

// in ::updateTotalQuads() 
quad.bl.texCoord1 = Tex2F(u, v); // tile coordinate
quad.bl.normals.x = //default tile normal map

// in ::updateVertexBuffer()
_vData = VertexData::create();
_vertexBuffer = VertexBuffer::create(sizeof(ST_V3F_C4B_T2Fx3_N3F), (int)_totalQuads.size() * 4);
_vData->setStream(_vertexBuffer, VertexStreamAttribute(0, GLProgram::VERTEX_ATTRIB_POSITION, GL_FLOAT, 3));
_vData->setStream(_vertexBuffer, VertexStreamAttribute(offsetof(ST_V3F_C4B_T2Fx3_N3F, colors), GLProgram::VERTEX_ATTRIB_COLOR, GL_UNSIGNED_BYTE, 4, true));
_vData->setStream(_vertexBuffer, VertexStreamAttribute(offsetof(ST_V3F_C4B_T2Fx3_N3F, texCoord0), GLProgram::VERTEX_ATTRIB_TEX_COORD, GL_FLOAT, 2));
_vData->setStream(_vertexBuffer, VertexStreamAttribute(offsetof(ST_V3F_C4B_T2Fx3_N3F, texCoord1), GLProgram::VERTEX_ATTRIB_TEX_COORD1, GL_FLOAT, 2));
_vData->setStream(_vertexBuffer, VertexStreamAttribute(offsetof(ST_V3F_C4B_T2Fx3_N3F, normals), GLProgram::VERTEX_ATTRIB_NORMAL, GL_FLOAT, 3));

// shader
attribute vec2 a_texCoord1;
attribute vec3 a_normal;

For implementing your own drawing for You can take a look at the Mesh class to see how it handles usage of adding in the normal attribute.

Hopefully I’ve given enough info and ideas that you can figure out how to move forward.

Others can feel free to give different or better advice.

1 Like
3

Note you can also look at Mesh and Sprite3D since they support normal for sure and I think possibly 1-4 tex coordinates, but probably at least texcoord 0 and 1.

Lastly, you can of course do all of this with custom attribute names, and pure OpenGL code, but might as well use the extra attributes lying around in the engine for a little less work, heh.

1 Like
4

Your idea of packing the data into the sprite’s color is wicked awesome. Trying that now. Tricky because setColor only accepts bytes via Color3B.

5

Well setOpacity should set the alpha float if that was your issue.
Edit: ah, you probably meant you’d only get 1/255 precision instead of the standard float precision
Also keep in mind GL ES often prefers medium precision in fragment shaders, so unpack in vert shader if possible.

1 Like
6

Actually now that you mention it, I think a PR request should be created to either switch the stored value to be Color4F and calc the set/get Color3B. This will possibly break something, so might need some thought behind it, but yeah you should be able to set the floating point values directly :frowning:

1 Like
7

@natweiss here’s an interesting article that discusses “abusing” both attributes and textures as well as the vertex shader to create some interesting animations and particle-esque renderings.

Also there’s Sean Berret’s ( https://github.com/nothings/obbg ) voxel renderer as one of I’m sure many many examples** one could find somewhere on the intertubes where he’s utilizing various attribute packing and using textures for uniform & vertex data, with a couple DEFINE macros to change which way data is input into the GPU pipeline.

** You probably won’t want to actually read through Sean’s code until you really needed to, but it’s one of the few I’ve actually seen and figured I’d post here for posterity.

1 Like
8

Nice! Great links here. I’m amazed. That face / mask talking is mind blowing how awesome and smooth it looks. Animations via texture data. Who knew??

Well, as soon as I land on a particular solution I will update this thread with the details.

9

Okay, got it working. :slight_smile: Used a method along the lines of the Gamasutra article you shared. Basically the data is baked into a texture.

Here’s some of the spritesheet:

953dbe8a4ebadd17.png860×575

And here’s some of the spritesheet with the baked texture overlaid at 50% opacity:

772139efdf220007.png860×575

So the technique is basically like this:

  • Loop over all sprite frames in the texture
  • Create a LayerGradient for each and render it into a RenderTexture
  • Use the RenderTexture to sample the correct alpha value for any given position in the texture

The only tricky part was looking up all the sprite frames for a given texture. This can be done without modifying cocos2d-x by subclassing SpriteFrameCache. Here’s my code for that:

class Spriteframes : public SpriteFrameCache
{
	public:
		typedef map<string, SpriteFrame*> framesType;

		static void getAll(framesType& frames, Texture2D* tex)
		{
			auto cacher = (Spriteframes*)SpriteFrameCache::getInstance();
			for (auto& p : cacher->_spriteFrames)
				if (p.second->getTexture() == tex)
					frames[p.first] = p.second;
		}
};
1 Like