Multipass rendering / Using RenderTexture for effects

cocos2d-x C++
1

I’ve got a blur shader working which uses RenderTexture.
It works by doing two passes, first a horizontal blur (which is saved in a RenderTexture) and then drawing the RenderTexture’s sprite with a vertical blur.
Since I wanted to get the blur working first I didn’t worry on how to implement it at a “high level” so to speak. This is how to have it work so the usage is smooth and easy and can be used in different nodes without much trouble.

I’m using the blur on an interface screen which is a sprite (lets say it’s the interface’s background) containing, as childs, other sprites and buttons. To blur this I set the shader of the outermost sprite, the container, do both passes and then add the RenderTexture’s sprite as a child of the interface. Therefore the blurred sprite is being drawn on top the regular interface (if you offset the blurred sprite’s position you can see the normal interface beneath). Obviously the correct thing would be to either draw the normal interface or draw it blurred, but how could this be implemented?

My question is how to the rendering of multipass effects like this. I thought about overriding the draw() function but this might mess up the rendering onto the RenderTexture?

Thanks for any advice.

2

Hello @bolin, I would run my shader on every child of interface(background Sprite, button textures etc) and would change my shader setup a little to display blur sprite texture on original sprite itself, something from top of my head would look like this,

void Test::makeBlurTexture(Sprite * texture){
RenderTexture *rt = RenderTexture::create(width,height); // dun remember the API exactly, it might change according to your needs
rt->begin();
// ...... run your code

rt->end();

texture->setTexture(rt->getSprite()->getTexure());

}

to use it lets say i have a Sprite background with one button as its child,

Sprite* backgroundSprite = Sprite::create("Background.png");
Button* clickMe = Button::create("Normal.png", "Normal.png", "Normal.png");
backgroundSprite->addChild(clickMe);

makeBlurTexture(backgroundSprite);
makeBlurTexture(clickMe->getNormalImage());

NOTE:- This is over top of my head and needs testing to confirm, Api will differ and OP should use the correct one. Excuse typo errors

Regards

3

With such approach you will blur each element separately, not the whole scene as single image. And it will give absolutely different result. Also scene may contain labels/spine skeletons/layers/etc.

4

I would recommend something like this:

  • make custom Node - BlurNode, which content should be blurred
  • init both render textures
  • override visit method:
    • don’t visit children
    • perform draw
    • visit final result (blurred sprite)

Override draw method:

  • perform blur

So, my approach:

  • override visit method to skip visiting children, and to visit result sprite.
  • override draw to perform actual blur;

When you want to blur something, you create that BlurNode, add it to the scene, and move all content you want to blur to that node. When you want to unblur - move all content back and remove BlurNode. That functionality also can be implemented inside BlurNode.

That looks slightly tricky but without support of some techniques in cocos2d-x all approaches will be tricky.

Hope you’ll understand what i have tried to say :slight_smile:

5

@MikhailSh May be i understood OP question a little wrong, and considered child rendering in usage example, Your approach is good but user might not need to do all that, the code approach shared above will be suffice if he wants to blur only Sprite or a complex scene, along with above code i will manage hierarchy manually while creating my scene keeping in mind what needs to be blurred and what not basically use concepts like HUD etc to differentiate my UI and game Scene, a simple example would be something like this:-

Scene/Layer  {
             |    SP1->CH1
             |           -> CH2->GCH1
             |    SP2->CH1              
             }

SP1 = Node starting heirarchy
CH1 = Child Sprite/Node
GCH1 =  grand Child Node/Sprite
SP2 = HUD(other elements which doesn't need blur)

so i can simply pass SP1 intoo above makeBlurTexture()to get what i wanted

6

Hey, thanks for the answers. I was already doing the “whole node” blurring so I would only have to set the blur shader in the upper most node.

@MikhailSh I get the general idea but could you share some code for the overrides? I think I’m missing something here.
If I don’t visit the children in my overridden visit method how would the blurring work (since I have to call visit on the BlurNode)?

I’ve created a BlurNode and I’m adding my main interface class which inherits from Sprite and has buttons and other stuff added as childs.

So far I’ve done the following:

void BlurNode::visit(Renderer *renderer, const Mat4 &parentTransform, uint32_t parentFlags)
{
    // quick return if not visible. children won't be drawn.
    if (!_visible)
    {
       return;
    }
        
    uint32_t flags = processParentFlags(parentTransform, parentFlags);
        
    _director->pushMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_MODELVIEW);
    _director->loadMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_MODELVIEW, _modelViewTransform);
        
    bool visibleByCamera = isVisitableByVisitingCamera();
        
    if(!_children.empty())
    {
        sortAllChildren();
           
        if (visibleByCamera) {
           for(auto it=_children.cbegin(); it != _children.cend(); ++it)
               (*it)->draw(renderer, _modelViewTransform, flags);
        }
    }
        
    _director->popMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_MODELVIEW);
}

void BlurNode::draw(Renderer *renderer, const Mat4 &transform, uint32_t flags)
{ 
    this->_firstPass->begin();
    this->visit(Director::getInstance()->getRenderer(), Director::getInstance()->getMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_MODELVIEW), true);
    this->_firstPass->end();
    
    {
        // DO HORIZONTAL BLUR SHADER SET UP
        this->_firstPass->getSprite()->setGLProgramState(blur);
    }
    
    this->_secondPass->setPosition(this->getPosition());
    
    this->_secondPass->begin();
    this->_firstPass->getSprite()->visit();
    this->_secondPass->end();
    
    {
        // DO VERTICAL BLUR SHADER SET UP
        this->_secondPass->getSprite()->setGLProgramState(blur);
    }
}

firstPass and secondPass are both RenderTextures.
This way, since I’m not visiting children, only the interface’s background gets drawn and not any of the buttons and stuff I’ve added to it.
If I decide to visit all of BlurNode children in visit() then I’ll get a crash since visit() is also called in draw() and there’s a infinite loop :smile:

Maybe I have to visit all of the BlurNode’s children from its draw method? I’ve tried this by replacing

this->visit(Director::getInstance()->getRenderer(), Director::getInstance()->getMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_MODELVIEW), true);

with:

for(auto it=_children.cbegin(); it != _children.cend(); ++it)
    (*it)->visit(renderer, _modelViewTransform, flags);

But I’m getting the same result, only the interface background is getting drawn.

7

I haven’t compiled this code, just to show general idea:
Create final blur sprite from render texture:

this->blurredSprite = Sprite::createWithTexture(_secondPass->getSprite()->getTexture());
this->blurredSprite->setAnchorPoint(Point::ZERO);

Visit method, general idea is to draw on screen only self and blurred sprite:

void BlurNode::visit(Renderer *renderer, const Mat4 &parentTransform, uint32_t parentFlags)
{
    // quick return if not visible. children won't be drawn.
    if (!_visible)  {
        return;
    }
    
    uint32_t flags = processParentFlags(parentTransform, parentFlags);
    
    _director->pushMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_MODELVIEW);
    _director->loadMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_MODELVIEW, _modelViewTransform);
    
    bool visibleByCamera = isVisitableByVisitingCamera();
    if (visibleByCamera) {
        this->draw(renderer, _modelViewTransform, flags);
        this->blurredSprite->visit(renderer, _modelViewTransform, flags);
    }
    _director->popMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_MODELVIEW);
}

Now in draw method we update blurred sprite:

void BlurNode::draw(Renderer *renderer, const Mat4 &transform, uint32_t flags)
{ 
    this->_firstPass->begin();

    if(!_children.empty())
    {
        sortAllChildren();
        for( int i = 0; i < _children.size(); i++ )
        {
            auto node = _children.at(i);
            node->visit(renderer, transform, flags);
        }
    }

    this->_firstPass->end();

    {
        // DO HORIZONTAL BLUR SHADER SET UP
        this->_firstPass->getSprite()->setGLProgramState(blur);
    }

    this->_secondPass->begin();
    this->_firstPass->getSprite()->visit();
    this->_secondPass->end();

    {
        // DO VERTICAL BLUR SHADER SET UP
        this->blurredSprite->setGLProgramState(blur);
    }
}

Something like this. So i’m not sure what matrix should be passed to children while performing first pass. May be should be passed IDENTITY?..

8

This makes sense, thanks.
I haven’t tried it yet since I’ve been working on other stuff but when I do I’ll update the thread with the resolved issue.

9

Was inspired by this theme and have made slightly more general purpose class - PostEffectNode. Though i haven’t tested it much. Sample.zip (489.9 KB)

It allows to:

  • add one or more passes;
  • enable/disable post processing;
  • if picture is static and post processing is expensive - result can be drawn once into final render texture, until you request to redraw it. For example, to blur scene while some alert is shown in front of it.

There is possible optimizations:

  • use PingPong technique. But it requires RenderTexture modification;
  • use render textures with smaller size for effects that don’t require high quality (like blur);
1 Like