Strange behavior using RenderTexture to capture then remove nodes

I’m working on a piece of code that can capture an arbitrary number of nodes using a RenderTexture, apply the captured image to a background Sprite then remove all the nodes except for the background Sprite to save resources. Unfortunately, after I capture the image the first time and then remove all the temporary nodes any subsequent captures with the RenderTexture appear as if I’m not visiting my background sprite. Just to be clear I am visiting the background sprite first then all the temporary nodes.

How I expect this code to work is every time I capture the background sprite along with any additional nodes the captured image should continually be compounded with each capture.

Code from HelloWorld::init()

// Create a render texture object used to capture the screen
mRenderTexture = RenderTexture::create(visibleSize.width, visibleSize.height);
mRenderTexture->retain();

// Create a Sprite used to display the image captured by the render texture
// Note: mTarget_sp is a child of the main layer so it too will be captured by the render texture
Size size = mRenderTexture->getSprite()->getTexture()->getContentSize();
mTarget_sp = Sprite::createWithTexture(mRenderTexture->getSprite()->getTexture(), Rect(0, 0, size.width, size.height));
mTarget_sp->setPosition(Vec2::ZERO);
mTarget_sp->setAnchorPoint(Vec2::ZERO);
mTarget_sp->setFlippedY(true);
this->addChild(mTarget_sp);

// Add the Menu button, the "Hello World" label, and the cocos sprite

I used the menuCloseCallback function to trigger the capture with the RenderTexture.

// Called when you press the one menu button
void HelloWorld::menuCloseCallback(Ref* pSender)
{
    // Save the texture of mTarget_sp before capturing the screen
    saveSpriteToFile(mTarget_sp, "before_capture");
    
    // Use the render texture to capture every drawable object on the HelloWorld layer
    mRenderTexture->beginWithClear(0.0f, 0.0f, 0.0f, 0.0f);
    this->visit();
    mRenderTexture->end();
    
    // Apply the captured textured to mTarget_sp
    mTarget_sp->setTexture(mRenderTexture->getSprite()->getTexture());
    
    // Save the texture of mTarget_sp after capturing the screen
    saveSpriteToFile(mTarget_sp, "after_capture");
    
    // Delay the call to HelloWorld::clearNodes() by 1 second
    string key = "key";
    Director::getInstance()->getScheduler()->schedule(CC_CALLBACK_1(HelloWorld::clearNodes, this), this, 1, 0, 0, false, key);
}

The program would crash if I tried to remove the “extra” nodes from menuCloseCallback so I added that functionality to a separate function which I call after a delay.

// Removes all but a few nodes from the main layer
void HelloWorld::clearNodes(float dt)
{
    // Iterate through every child on the HelloWorld layer and remove them
    // unless it is mTarget_sp or the menu object
    for (Node* child : this->getChildren())
    {
        if (child != mTarget_sp && child != mMenu)
        {
            child->removeFromParent();
        }
    }
}

Lastly, I added a debug function which saves the contents of a sprite to a file using a RenderTexture.

// Debug function used to save the texture for a Sprite
void HelloWorld::saveSpriteToFile(Sprite* sp, string filename)
{
    Size winSize = Director::getInstance()->getWinSize();
    
    // Use a render texture to capture the texture of the passed in sprite
    RenderTexture* rt = RenderTexture::create(winSize.width, winSize.height);
    rt->beginWithClear(0.0f, 0.0f, 0.0f, 0.0f);
    sp->visit();
    rt->end();
    
    // Use a static counter to differentiate the saved images
    static int ctr = 0;
    filename += "_" + std::to_string(ctr) + ".png";
    
    // Save the sprite's captured texture to a file
    if (rt->saveToFile(filename, true) == false)
    {
        log("Error: failed to save sprite to filename: %s", filename.c_str());
    }
    
    // Increment the static counter
    ++ctr;
}

The first time I trigger menuCloseCallback I expect mTarget_sp to be fully transparent and after saving its texture to “before_capture_0.png” I find this is true.

After the texture from mRenderTexture is applied to mTarget_sp this is the captured image and it is as I expected after the first pass => “after_capture_1.png”

By the time I’ve pressed the menu button the second time both the “Hello World” label and the cocos2dx sprite have been removed from the main layer but mTarget_sp is displaying their content. When the menu button is pressed a second time I would think that visiting the mTarget_sp would cause it’s texture to be captured then just applied to itself but it appears as if it is not being captured.

mTarget_sp before the second capture => “before_capture_2.png”

And here is mTarget_sp after the second capture => “after_capture_3.png”

Something is happening that I don’t understand. If anyone could help me understand why the captured content is disappearing and how I can fix it I would be much appreciative. Also, any subsequent captures all look the same as “after_capture_3.png”.

What I believe is probably happening is that because mTarget_sp and mRenderTexture share the same texture, when you use mRenderTexture->beginWithClear(0.0f, 0.0f, 0.0f, 0.0f); in menuCloseCallback(), it clears the texture, which means mTarget_sp is blank when it is drawn for the mRenderTexture.

Changing the line to mRenderTexture->begin(); solves the issue for me.

Also, it seems the line mTarget_sp->setTexture(mRenderTexture->getSprite()->getTexture()); is unnecessary, as it’s likely that rendering to RenderTexture simply alters the existing texture rather than creating a new one.

And also, regarding it crashing if you remove the temporary sprites immediately, I imagine this is because the sprites still need to be in memory when the Renderer renders the RenderTexture. So you probably only need to wait until the Renderer renders next before removing the sprites. I tried setting the delay time for the sceduled callback to clearNodes to 0 (which effectively means “next frame”) and it worked without issue for me.

Lastly, just a heads up, I seem to get strange behaviour when drawing to the RenderTexture multiple times. The edges of the texture seems to smear and are getting pushed out towards the edges. You’ll see what I mean if it happens to you. Not sure why it is happening, though.

Thank you, this solves the issue I was seeing, but as you pointed out now the texture is being smeared at the edges. I’ll spend some time to try to fix this new issue and post my findings here.

The code you are using right now establishes a “Rendering Feedback Loop” (you can read more about it here in section 4.4.4).
Basically a “Rendering Feedback Loop” describes the situation in which you have the same texture attached to a currently bound framebuffer, and bound to an arbitrary texture unit. This will result in undefined behaviour, which I believe is what you are seeing as smearing